2023-11-28T10:55:03.682 INFO:root:teuthology version: 0.0.1.dev209+g5ae09a5 2023-11-28T10:55:03.683 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --name neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub} --owner scheduled_neesingh@teuthology --archive /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469955 -- /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469955/orig.config.yaml 2023-11-28T10:55:03.725 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-28T10:55:03.802 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469955 branch: wip-neesingh-testing-231123 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub} email: neesingh@redhat.com first_in_suite: false job_id: '7469955' kernel: client: branch: testing kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 8.stream overrides: admin_socket: branch: wip-neesingh-testing-231123 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - inode wrongly marked free - bad backtrace on inode - inode table repaired for inode - Scrub error on inode - Scrub error on dir - Metadata damage detected sha1: 2f5f6028b893ad41357cd06e827373f0fe2a1232 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 2f5f6028b893ad41357cd06e827373f0fe2a1232 kclient: syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: issue_54557 sha1: e67b67438c1e5e13734a7203ff9e57e30b8e4d32 owner: scheduled_neesingh@teuthology priority: 10 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 9437 sha1: 2f5f6028b893ad41357cd06e827373f0fe2a1232 sleep_before_teardown: 0 suite: fs suite_branch: issue_54557 suite_path: /home/teuthworker/src/github.com_neesingh-rh_ceph_e67b67438c1e5e13734a7203ff9e57e30b8e4d32/qa suite_relpath: qa suite_repo: https://github.com/neesingh-rh/ceph suite_sha1: e67b67438c1e5e13734a7203ff9e57e30b8e4d32 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi092.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLBWKDssAcujaTJGYoh0m+6KRf79YlBnyP6+4+KzFgJNnb+zlPv8t0bboGt8tbKRVck/k776TBljTkSk6kIOe1M= smithi169.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBspMRSneSLchVXtu1lFxgIfmFRbU9QHxtOh73ymQiDEyhUjA07U4CTxv98VSN9qWSUXOJ7F0WXBIc5AsA4w4ss= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: null - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_forward_scrub teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 5ae09a5cc7064ca2bf994df19f3006b79794789a timestamp: 2023-11-28_10:48:03 tube: smithi user: neesingh verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.232157 2023-11-28T10:55:03.802 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_neesingh-rh_ceph_e67b67438c1e5e13734a7203ff9e57e30b8e4d32/qa; will attempt to use it 2023-11-28T10:55:03.803 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_neesingh-rh_ceph_e67b67438c1e5e13734a7203ff9e57e30b8e4d32/qa/tasks 2023-11-28T10:55:03.803 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-11-28T10:55:03.805 INFO:teuthology.task.internal:Checking packages... 2023-11-28T10:55:03.827 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '2f5f6028b893ad41357cd06e827373f0fe2a1232' 2023-11-28T10:55:03.828 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-11-28T10:55:03.828 INFO:teuthology.packaging:ref: None 2023-11-28T10:55:03.828 INFO:teuthology.packaging:tag: None 2023-11-28T10:55:03.828 INFO:teuthology.packaging:branch: wip-neesingh-testing-231123 2023-11-28T10:55:03.828 INFO:teuthology.packaging:sha1: 2f5f6028b893ad41357cd06e827373f0fe2a1232 2023-11-28T10:55:03.828 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-neesingh-testing-231123 2023-11-28T10:55:04.005 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-2697.g2f5f6028 2023-11-28T10:55:04.005 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-11-28T10:55:04.011 INFO:teuthology.task.internal:no buildpackages task found 2023-11-28T10:55:04.011 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-11-28T10:55:04.017 INFO:teuthology.task.internal:Saving configuration 2023-11-28T10:55:04.031 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-11-28T10:55:04.037 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-28T10:55:04.061 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi092.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469955', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-28 10:49:56.154011', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLBWKDssAcujaTJGYoh0m+6KRf79YlBnyP6+4+KzFgJNnb+zlPv8t0bboGt8tbKRVck/k776TBljTkSk6kIOe1M='} 2023-11-28T10:55:04.087 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi169.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469955', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-28 10:49:56.152407', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBspMRSneSLchVXtu1lFxgIfmFRbU9QHxtOh73ymQiDEyhUjA07U4CTxv98VSN9qWSUXOJ7F0WXBIc5AsA4w4ss='} 2023-11-28T10:55:04.087 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-11-28T10:55:04.093 INFO:teuthology.task.internal:roles: ubuntu@smithi092.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-11-28T10:55:04.094 INFO:teuthology.task.internal:roles: ubuntu@smithi169.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-11-28T10:55:04.094 INFO:teuthology.run_tasks:Running task console_log... 2023-11-28T10:55:04.158 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f0f8fb68a60>, signals=[15]) 2023-11-28T10:55:04.158 INFO:teuthology.run_tasks:Running task internal.connect... 2023-11-28T10:55:04.164 INFO:teuthology.task.internal:Opening connections... 2023-11-28T10:55:04.165 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi092.front.sepia.ceph.com 2023-11-28T10:55:04.167 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T10:55:04.237 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi169.front.sepia.ceph.com 2023-11-28T10:55:04.237 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T10:55:04.306 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-11-28T10:55:04.313 DEBUG:teuthology.orchestra.run.smithi092:> uname -m 2023-11-28T10:55:04.335 INFO:teuthology.orchestra.run.smithi092.stdout:x86_64 2023-11-28T10:55:04.335 DEBUG:teuthology.orchestra.run.smithi092:> cat /etc/os-release 2023-11-28T10:55:04.397 INFO:teuthology.orchestra.run.smithi092.stdout:NAME="CentOS Stream" 2023-11-28T10:55:04.397 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION="8" 2023-11-28T10:55:04.397 INFO:teuthology.orchestra.run.smithi092.stdout:ID="centos" 2023-11-28T10:55:04.397 INFO:teuthology.orchestra.run.smithi092.stdout:ID_LIKE="rhel fedora" 2023-11-28T10:55:04.397 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION_ID="8" 2023-11-28T10:55:04.397 INFO:teuthology.orchestra.run.smithi092.stdout:PLATFORM_ID="platform:el8" 2023-11-28T10:55:04.397 INFO:teuthology.orchestra.run.smithi092.stdout:PRETTY_NAME="CentOS Stream 8" 2023-11-28T10:55:04.397 INFO:teuthology.orchestra.run.smithi092.stdout:ANSI_COLOR="0;31" 2023-11-28T10:55:04.397 INFO:teuthology.orchestra.run.smithi092.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-11-28T10:55:04.398 INFO:teuthology.orchestra.run.smithi092.stdout:HOME_URL="https://centos.org/" 2023-11-28T10:55:04.398 INFO:teuthology.orchestra.run.smithi092.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-28T10:55:04.398 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-28T10:55:04.398 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-28T10:55:04.398 INFO:teuthology.lock.ops:Updating smithi092.front.sepia.ceph.com on lock server 2023-11-28T10:55:04.428 DEBUG:teuthology.orchestra.run.smithi169:> uname -m 2023-11-28T10:55:04.451 INFO:teuthology.orchestra.run.smithi169.stdout:x86_64 2023-11-28T10:55:04.451 DEBUG:teuthology.orchestra.run.smithi169:> cat /etc/os-release 2023-11-28T10:55:04.512 INFO:teuthology.orchestra.run.smithi169.stdout:NAME="CentOS Stream" 2023-11-28T10:55:04.512 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION="8" 2023-11-28T10:55:04.513 INFO:teuthology.orchestra.run.smithi169.stdout:ID="centos" 2023-11-28T10:55:04.513 INFO:teuthology.orchestra.run.smithi169.stdout:ID_LIKE="rhel fedora" 2023-11-28T10:55:04.513 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION_ID="8" 2023-11-28T10:55:04.513 INFO:teuthology.orchestra.run.smithi169.stdout:PLATFORM_ID="platform:el8" 2023-11-28T10:55:04.513 INFO:teuthology.orchestra.run.smithi169.stdout:PRETTY_NAME="CentOS Stream 8" 2023-11-28T10:55:04.513 INFO:teuthology.orchestra.run.smithi169.stdout:ANSI_COLOR="0;31" 2023-11-28T10:55:04.513 INFO:teuthology.orchestra.run.smithi169.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-11-28T10:55:04.513 INFO:teuthology.orchestra.run.smithi169.stdout:HOME_URL="https://centos.org/" 2023-11-28T10:55:04.513 INFO:teuthology.orchestra.run.smithi169.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-28T10:55:04.513 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-28T10:55:04.513 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-28T10:55:04.514 INFO:teuthology.lock.ops:Updating smithi169.front.sepia.ceph.com on lock server 2023-11-28T10:55:04.545 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-11-28T10:55:04.554 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-11-28T10:55:04.561 INFO:teuthology.task.internal:Checking for old test directory... 2023-11-28T10:55:04.562 DEBUG:teuthology.orchestra.run.smithi092:> test '!' -e /home/ubuntu/cephtest 2023-11-28T10:55:04.564 DEBUG:teuthology.orchestra.run.smithi169:> test '!' -e /home/ubuntu/cephtest 2023-11-28T10:55:04.585 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-11-28T10:55:04.591 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-11-28T10:55:04.591 DEBUG:teuthology.orchestra.run.smithi092:> test -z $(ls -A /var/lib/ceph) 2023-11-28T10:55:04.624 DEBUG:teuthology.orchestra.run.smithi169:> test -z $(ls -A /var/lib/ceph) 2023-11-28T10:55:04.657 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-11-28T10:55:04.712 INFO:teuthology.run_tasks:Running task kernel... 2023-11-28T10:55:04.727 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2023-11-28T10:55:04.727 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2023-11-28T10:55:04.727 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2023-11-28T10:55:04.727 DEBUG:teuthology.orchestra.run.smithi092:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-28T10:55:04.728 DEBUG:teuthology.orchestra.run.smithi169:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-28T10:55:04.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T10:55:04.749 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2023-11-28T10:55:04.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T10:55:04.752 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2023-11-28T10:55:04.918 DEBUG:teuthology.task.kernel:sha1 for client.1 is 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-28T10:55:04.919 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "8247bbb5b470f7e2a1f3eefcc16e24cf1338f606"... 2023-11-28T10:55:04.919 DEBUG:teuthology.orchestra.run.smithi169:> uname -r 2023-11-28T10:55:04.940 INFO:teuthology.orchestra.run.smithi169.stdout:4.18.0-489.el8.x86_64 2023-11-28T10:55:04.941 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-489.el8.x86_64 vs 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-28T10:55:04.941 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-11-28T10:55:04.941 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum upgrade -y linux-firmware 2023-11-28T10:55:04.983 DEBUG:teuthology.task.kernel:sha1 for client.0 is 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-28T10:55:04.983 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "8247bbb5b470f7e2a1f3eefcc16e24cf1338f606"... 2023-11-28T10:55:04.984 DEBUG:teuthology.orchestra.run.smithi092:> uname -r 2023-11-28T10:55:05.005 INFO:teuthology.orchestra.run.smithi092.stdout:4.18.0-489.el8.x86_64 2023-11-28T10:55:05.006 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-489.el8.x86_64 vs 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-28T10:55:05.006 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-11-28T10:55:05.006 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum upgrade -y linux-firmware 2023-11-28T10:55:57.680 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 8 - AppStream 699 kB/s | 34 MB 00:50 2023-11-28T10:56:06.338 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 8 - BaseOS 23 MB/s | 55 MB 00:02 2023-11-28T10:56:14.721 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 8 - Extras 38 kB/s | 18 kB 00:00 2023-11-28T10:56:15.055 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 8 - Extras common packages 45 kB/s | 6.9 kB 00:00 2023-11-28T10:56:15.373 INFO:teuthology.orchestra.run.smithi169.stdout:Copr repo for python3-asyncssh owned by ceph 21 kB/s | 3.5 kB 00:00 2023-11-28T10:56:16.107 INFO:teuthology.orchestra.run.smithi169.stdout:Extra Packages for Enterprise Linux 27 MB/s | 16 MB 00:00 2023-11-28T10:56:20.015 INFO:teuthology.orchestra.run.smithi169.stdout:lab-extras 198 kB/s | 24 kB 00:00 2023-11-28T10:56:23.997 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2023-11-28T10:56:23.998 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2023-11-28T10:56:23.998 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repo Size 2023-11-28T10:56:23.998 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2023-11-28T10:56:23.998 INFO:teuthology.orchestra.run.smithi169.stdout:Upgrading: 2023-11-28T10:56:23.998 INFO:teuthology.orchestra.run.smithi169.stdout: linux-firmware noarch 20230824-118.git0e048b06.el8 baseos 286 M 2023-11-28T10:56:23.998 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-28T10:56:23.999 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2023-11-28T10:56:23.999 INFO:teuthology.orchestra.run.smithi169.stdout:================================================================================ 2023-11-28T10:56:23.999 INFO:teuthology.orchestra.run.smithi169.stdout:Upgrade 1 Package 2023-11-28T10:56:23.999 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-28T10:56:23.999 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 286 M 2023-11-28T10:56:23.999 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2023-11-28T10:56:35.833 INFO:teuthology.orchestra.run.smithi169.stdout:linux-firmware-20230824-118.git0e048b06.el8.noa 25 MB/s | 286 MB 00:11 2023-11-28T10:56:35.833 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2023-11-28T10:56:35.833 INFO:teuthology.orchestra.run.smithi169.stdout:Total 24 MB/s | 286 MB 00:11 2023-11-28T10:56:37.533 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2023-11-28T10:56:37.676 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2023-11-28T10:56:37.677 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2023-11-28T10:56:39.457 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2023-11-28T10:56:39.480 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2023-11-28T10:56:45.807 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 8 - AppStream 357 kB/s | 34 MB 01:38 2023-11-28T10:56:54.357 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 8 - BaseOS 23 MB/s | 55 MB 00:02 2023-11-28T10:56:58.583 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2023-11-28T10:56:58.932 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : linux-firmware-20230824-118.git0e048b06.el8.noarch 1/2 2023-11-28T10:56:59.157 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : linux-firmware-20230217-113.git83f1d778.el8.noarch 2/2 2023-11-28T10:56:59.383 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: linux-firmware-20230217-113.git83f1d778.el8.noarch 2/2 2023-11-28T10:56:59.383 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : linux-firmware-20230824-118.git0e048b06.el8.noarch 1/2 2023-11-28T10:57:00.208 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : linux-firmware-20230217-113.git83f1d778.el8.noarch 2/2 2023-11-28T10:57:00.208 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-28T10:57:00.209 INFO:teuthology.orchestra.run.smithi169.stdout:Upgraded: 2023-11-28T10:57:00.209 INFO:teuthology.orchestra.run.smithi169.stdout: linux-firmware-20230824-118.git0e048b06.el8.noarch 2023-11-28T10:57:00.209 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-28T10:57:00.209 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2023-11-28T10:57:00.392 INFO:teuthology.task.kernel:Downloading kernel 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 on client.1... 2023-11-28T10:57:00.393 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-28T10:57:00.628 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/8/flavors/default/x86_64/ 2023-11-28T10:57:00.629 DEBUG:teuthology.orchestra.run.smithi169:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_g8247bbb5b470-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/8/flavors/default/x86_64/ --input-file=- 2023-11-28T10:57:00.632 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi169.front.sepia.ceph.com to complete... 2023-11-28T10:57:03.322 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 8 - Extras 17 kB/s | 18 kB 00:01 2023-11-28T10:57:04.181 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 8 - Extras common packages 10 kB/s | 6.9 kB 00:00 2023-11-28T10:57:04.474 INFO:teuthology.orchestra.run.smithi092.stdout:Copr repo for python3-asyncssh owned by ceph 25 kB/s | 3.5 kB 00:00 2023-11-28T10:57:06.385 INFO:teuthology.orchestra.run.smithi092.stdout:Extra Packages for Enterprise Linux 9.1 MB/s | 16 MB 00:01 2023-11-28T10:57:10.174 INFO:teuthology.orchestra.run.smithi092.stdout:lab-extras 419 kB/s | 24 kB 00:00 2023-11-28T10:57:14.104 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2023-11-28T10:57:14.104 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2023-11-28T10:57:14.105 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repo Size 2023-11-28T10:57:14.105 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2023-11-28T10:57:14.105 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrading: 2023-11-28T10:57:14.105 INFO:teuthology.orchestra.run.smithi092.stdout: linux-firmware noarch 20230824-118.git0e048b06.el8 baseos 286 M 2023-11-28T10:57:14.105 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T10:57:14.105 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2023-11-28T10:57:14.105 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2023-11-28T10:57:14.105 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrade 1 Package 2023-11-28T10:57:14.105 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T10:57:14.106 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 286 M 2023-11-28T10:57:14.106 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2023-11-28T10:57:25.707 INFO:teuthology.orchestra.run.smithi092.stdout:linux-firmware-20230824-118.git0e048b06.el8.noa 25 MB/s | 286 MB 00:11 2023-11-28T10:57:25.707 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2023-11-28T10:57:25.707 INFO:teuthology.orchestra.run.smithi092.stdout:Total 25 MB/s | 286 MB 00:11 2023-11-28T10:57:27.412 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2023-11-28T10:57:27.478 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2023-11-28T10:57:27.479 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2023-11-28T10:57:29.250 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2023-11-28T10:57:29.282 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2023-11-28T10:57:36.245 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28 10:57:36 URL:https://4.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/8/flavors/default/x86_64/kernel-6.6.0_g8247bbb5b470-1.x86_64.rpm [383178732/383178732] -> "/tmp/kernel.x86_64.rpm" [1] 2023-11-28T10:57:36.245 INFO:teuthology.orchestra.run.smithi169.stderr:FINISHED --2023-11-28 10:57:36-- 2023-11-28T10:57:36.245 INFO:teuthology.orchestra.run.smithi169.stderr:Total wall clock time: 36s 2023-11-28T10:57:36.245 INFO:teuthology.orchestra.run.smithi169.stderr:Downloaded: 1 files, 365M in 35s (10.4 MB/s) 2023-11-28T10:57:36.247 INFO:teuthology.task.kernel:Installing kernel 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 on client.1... 2023-11-28T10:57:36.247 DEBUG:teuthology.orchestra.run.smithi169:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-11-28T10:57:37.916 INFO:teuthology.orchestra.run.smithi169.stdout:Verifying... ######################################## 2023-11-28T10:57:37.958 INFO:teuthology.orchestra.run.smithi169.stdout:Preparing... ######################################## 2023-11-28T10:57:38.068 INFO:teuthology.orchestra.run.smithi169.stdout:Updating / installing... 2023-11-28T10:57:48.412 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2023-11-28T10:57:48.763 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : linux-firmware-20230824-118.git0e048b06.el8.noarch 1/2 2023-11-28T10:57:48.993 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : linux-firmware-20230217-113.git83f1d778.el8.noarch 2/2 2023-11-28T10:57:49.219 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: linux-firmware-20230217-113.git83f1d778.el8.noarch 2/2 2023-11-28T10:57:49.219 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : linux-firmware-20230824-118.git0e048b06.el8.noarch 1/2 2023-11-28T10:57:50.043 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : linux-firmware-20230217-113.git83f1d778.el8.noarch 2/2 2023-11-28T10:57:50.044 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T10:57:50.044 INFO:teuthology.orchestra.run.smithi092.stdout:Upgraded: 2023-11-28T10:57:50.044 INFO:teuthology.orchestra.run.smithi092.stdout: linux-firmware-20230824-118.git0e048b06.el8.noarch 2023-11-28T10:57:50.044 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T10:57:50.044 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2023-11-28T10:57:50.217 INFO:teuthology.task.kernel:Downloading kernel 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 on client.0... 2023-11-28T10:57:50.217 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-28T10:57:50.379 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/8/flavors/default/x86_64/ 2023-11-28T10:57:50.380 DEBUG:teuthology.orchestra.run.smithi092:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_g8247bbb5b470-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/8/flavors/default/x86_64/ --input-file=- 2023-11-28T10:57:50.382 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi092.front.sepia.ceph.com to complete... 2023-11-28T10:58:03.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28 10:58:03 URL:https://4.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/8/flavors/default/x86_64/kernel-6.6.0_g8247bbb5b470-1.x86_64.rpm [383178732/383178732] -> "/tmp/kernel.x86_64.rpm" [1] 2023-11-28T10:58:03.565 INFO:teuthology.orchestra.run.smithi092.stderr:FINISHED --2023-11-28 10:58:03-- 2023-11-28T10:58:03.565 INFO:teuthology.orchestra.run.smithi092.stderr:Total wall clock time: 13s 2023-11-28T10:58:03.565 INFO:teuthology.orchestra.run.smithi092.stderr:Downloaded: 1 files, 365M in 13s (28.4 MB/s) 2023-11-28T10:58:03.566 INFO:teuthology.task.kernel:Installing kernel 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 on client.0... 2023-11-28T10:58:03.566 DEBUG:teuthology.orchestra.run.smithi092:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-11-28T10:58:05.243 INFO:teuthology.orchestra.run.smithi092.stdout:Verifying... ######################################## 2023-11-28T10:58:05.285 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing... ######################################## 2023-11-28T10:58:05.394 INFO:teuthology.orchestra.run.smithi092.stdout:Updating / installing... 2023-11-28T10:58:16.175 INFO:teuthology.orchestra.run.smithi169.stdout:kernel-6.6.0_g8247bbb5b470-1 ######################################## 2023-11-28T10:58:17.517 INFO:teuthology.orchestra.run.smithi169.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-11-28T10:58:41.264 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi169.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-11-28T10:58:41.264 DEBUG:teuthology.orchestra.run.smithi169:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-11-28T10:58:41.324 INFO:teuthology.orchestra.run.smithi169.stdout:/boot/System.map-6.6.0-g8247bbb5b470 2023-11-28T10:58:41.324 INFO:teuthology.orchestra.run.smithi169.stdout:/boot/config-6.6.0-g8247bbb5b470 2023-11-28T10:58:41.325 INFO:teuthology.orchestra.run.smithi169.stdout:/boot/vmlinuz-6.6.0-g8247bbb5b470 2023-11-28T10:58:41.325 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470 2023-11-28T10:58:41.325 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel 2023-11-28T10:58:41.325 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch 2023-11-28T10:58:41.325 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86 2023-11-28T10:58:41.325 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto 2023-11-28T10:58:41.325 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-11-28T10:58:41.325 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-11-28T10:58:41.325 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-11-28T10:58:41.325 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-11-28T10:58:41.325 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-11-28T10:58:41.326 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-11-28T10:58:41.326 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-11-28T10:58:41.326 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crc32c-intel.ko 2023-11-28T10:58:41.326 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-11-28T10:58:41.326 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-11-28T10:58:41.326 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-11-28T10:58:41.326 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-11-28T10:58:41.326 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-avx2.ko 2023-11-28T10:58:41.326 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-11-28T10:58:41.326 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-11-28T10:58:41.326 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-11-28T10:58:41.327 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-11-28T10:58:41.327 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-11-28T10:58:41.327 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-11-28T10:58:41.327 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-11-28T10:58:41.327 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel 2023-11-28T10:58:41.327 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu 2023-11-28T10:58:41.327 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/mce 2023-11-28T10:58:41.327 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-11-28T10:58:41.327 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm 2023-11-28T10:58:41.327 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm-amd.ko 2023-11-28T10:58:41.327 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm-intel.ko 2023-11-28T10:58:41.327 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm.ko 2023-11-28T10:58:41.328 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto 2023-11-28T10:58:41.328 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ansi_cprng.ko 2023-11-28T10:58:41.328 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/anubis.ko 2023-11-28T10:58:41.328 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/arc4.ko 2023-11-28T10:58:41.328 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx 2023-11-28T10:58:41.328 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_memcpy.ko 2023-11-28T10:58:41.328 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_pq.ko 2023-11-28T10:58:41.328 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_raid6_recov.ko 2023-11-28T10:58:41.328 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_tx.ko 2023-11-28T10:58:41.328 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_xor.ko 2023-11-28T10:58:41.328 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/raid6test.ko 2023-11-28T10:58:41.329 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/authenc.ko 2023-11-28T10:58:41.329 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/authencesn.ko 2023-11-28T10:58:41.329 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blake2b_generic.ko 2023-11-28T10:58:41.329 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blowfish_common.ko 2023-11-28T10:58:41.329 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blowfish_generic.ko 2023-11-28T10:58:41.329 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/camellia_generic.ko 2023-11-28T10:58:41.329 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast5_generic.ko 2023-11-28T10:58:41.329 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast6_generic.ko 2023-11-28T10:58:41.329 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast_common.ko 2023-11-28T10:58:41.329 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ccm.ko 2023-11-28T10:58:41.329 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cmac.ko 2023-11-28T10:58:41.329 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crc32_generic.ko 2023-11-28T10:58:41.330 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_engine.ko 2023-11-28T10:58:41.330 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_user.ko 2023-11-28T10:58:41.330 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cts.ko 2023-11-28T10:58:41.330 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/des_generic.ko 2023-11-28T10:58:41.330 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/dh_generic.ko 2023-11-28T10:58:41.330 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/echainiv.ko 2023-11-28T10:58:41.330 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/essiv.ko 2023-11-28T10:58:41.330 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/fcrypt.ko 2023-11-28T10:58:41.330 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/khazad.ko 2023-11-28T10:58:41.330 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lz4.ko 2023-11-28T10:58:41.330 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lz4hc.ko 2023-11-28T10:58:41.331 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/md4.ko 2023-11-28T10:58:41.331 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/michael_mic.ko 2023-11-28T10:58:41.331 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/pcbc.ko 2023-11-28T10:58:41.331 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/pcrypt.ko 2023-11-28T10:58:41.331 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/rmd160.ko 2023-11-28T10:58:41.331 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/seed.ko 2023-11-28T10:58:41.331 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/serpent_generic.ko 2023-11-28T10:58:41.331 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/tcrypt.ko 2023-11-28T10:58:41.331 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/tea.ko 2023-11-28T10:58:41.331 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/twofish_common.ko 2023-11-28T10:58:41.331 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/twofish_generic.ko 2023-11-28T10:58:41.331 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/vmac.ko 2023-11-28T10:58:41.332 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/wp512.ko 2023-11-28T10:58:41.332 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xcbc.ko 2023-11-28T10:58:41.332 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xor.ko 2023-11-28T10:58:41.332 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xxhash_generic.ko 2023-11-28T10:58:41.332 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers 2023-11-28T10:58:41.332 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi 2023-11-28T10:58:41.332 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_ipmi.ko 2023-11-28T10:58:41.332 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_pad.ko 2023-11-28T10:58:41.332 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/custom_method.ko 2023-11-28T10:58:41.332 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/ec_sys.ko 2023-11-28T10:58:41.332 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/platform_profile.ko 2023-11-28T10:58:41.333 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/sbs.ko 2023-11-28T10:58:41.333 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/sbshc.ko 2023-11-28T10:58:41.333 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/video.ko 2023-11-28T10:58:41.333 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata 2023-11-28T10:58:41.333 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/acard-ahci.ko 2023-11-28T10:58:41.333 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ahci_platform.ko 2023-11-28T10:58:41.333 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ata_generic.ko 2023-11-28T10:58:41.333 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/libahci_platform.ko 2023-11-28T10:58:41.333 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_acpi.ko 2023-11-28T10:58:41.333 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ali.ko 2023-11-28T10:58:41.334 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_amd.ko 2023-11-28T10:58:41.334 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_artop.ko 2023-11-28T10:58:41.334 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_atiixp.ko 2023-11-28T10:58:41.334 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_atp867x.ko 2023-11-28T10:58:41.334 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cmd640.ko 2023-11-28T10:58:41.334 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cmd64x.ko 2023-11-28T10:58:41.334 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cypress.ko 2023-11-28T10:58:41.334 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_efar.ko 2023-11-28T10:58:41.334 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt366.ko 2023-11-28T10:58:41.334 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt37x.ko 2023-11-28T10:58:41.334 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt3x2n.ko 2023-11-28T10:58:41.335 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt3x3.ko 2023-11-28T10:58:41.335 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_it8213.ko 2023-11-28T10:58:41.335 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_it821x.ko 2023-11-28T10:58:41.335 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_jmicron.ko 2023-11-28T10:58:41.335 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_marvell.ko 2023-11-28T10:58:41.335 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_mpiix.ko 2023-11-28T10:58:41.335 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_netcell.ko 2023-11-28T10:58:41.335 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ninja32.ko 2023-11-28T10:58:41.335 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ns87410.ko 2023-11-28T10:58:41.335 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ns87415.ko 2023-11-28T10:58:41.335 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_oldpiix.ko 2023-11-28T10:58:41.335 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_opti.ko 2023-11-28T10:58:41.336 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_optidma.ko 2023-11-28T10:58:41.336 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pcmcia.ko 2023-11-28T10:58:41.336 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pdc2027x.ko 2023-11-28T10:58:41.336 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-11-28T10:58:41.336 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_piccolo.ko 2023-11-28T10:58:41.336 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_rdc.ko 2023-11-28T10:58:41.336 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sch.ko 2023-11-28T10:58:41.336 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_serverworks.ko 2023-11-28T10:58:41.336 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sil680.ko 2023-11-28T10:58:41.336 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sis.ko 2023-11-28T10:58:41.336 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sl82c105.ko 2023-11-28T10:58:41.337 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_triflex.ko 2023-11-28T10:58:41.337 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_via.ko 2023-11-28T10:58:41.337 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pdc_adma.ko 2023-11-28T10:58:41.337 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_inic162x.ko 2023-11-28T10:58:41.337 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_mv.ko 2023-11-28T10:58:41.337 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_nv.ko 2023-11-28T10:58:41.337 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_promise.ko 2023-11-28T10:58:41.337 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_qstor.ko 2023-11-28T10:58:41.337 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sil.ko 2023-11-28T10:58:41.337 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sil24.ko 2023-11-28T10:58:41.337 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sis.ko 2023-11-28T10:58:41.337 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_svw.ko 2023-11-28T10:58:41.338 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sx4.ko 2023-11-28T10:58:41.338 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_uli.ko 2023-11-28T10:58:41.338 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_via.ko 2023-11-28T10:58:41.338 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_vsc.ko 2023-11-28T10:58:41.338 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/auxdisplay 2023-11-28T10:58:41.338 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/auxdisplay/ks0108.ko 2023-11-28T10:58:41.338 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base 2023-11-28T10:58:41.338 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap 2023-11-28T10:58:41.338 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap/regmap-i2c.ko 2023-11-28T10:58:41.338 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap/regmap-mmio.ko 2023-11-28T10:58:41.338 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/bcma 2023-11-28T10:58:41.339 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/bcma/bcma.ko 2023-11-28T10:58:41.339 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block 2023-11-28T10:58:41.339 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/aoe 2023-11-28T10:58:41.339 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/aoe/aoe.ko 2023-11-28T10:58:41.339 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/brd.ko 2023-11-28T10:58:41.339 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/drbd 2023-11-28T10:58:41.339 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/drbd/drbd.ko 2023-11-28T10:58:41.339 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/floppy.ko 2023-11-28T10:58:41.339 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/loop.ko 2023-11-28T10:58:41.339 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/mtip32xx 2023-11-28T10:58:41.339 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-11-28T10:58:41.339 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/nbd.ko 2023-11-28T10:58:41.340 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/null_blk 2023-11-28T10:58:41.340 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/null_blk/null_blk.ko 2023-11-28T10:58:41.340 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/pktcdvd.ko 2023-11-28T10:58:41.340 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/rbd.ko 2023-11-28T10:58:41.340 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/virtio_blk.ko 2023-11-28T10:58:41.340 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkback 2023-11-28T10:58:41.340 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-11-28T10:58:41.340 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkfront.ko 2023-11-28T10:58:41.340 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/zram 2023-11-28T10:58:41.340 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/zram/zram.ko 2023-11-28T10:58:41.340 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char 2023-11-28T10:58:41.340 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hangcheck-timer.ko 2023-11-28T10:58:41.341 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random 2023-11-28T10:58:41.341 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/amd-rng.ko 2023-11-28T10:58:41.341 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/intel-rng.ko 2023-11-28T10:58:41.341 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-11-28T10:58:41.341 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/via-rng.ko 2023-11-28T10:58:41.341 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/virtio-rng.ko 2023-11-28T10:58:41.341 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi 2023-11-28T10:58:41.341 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-11-28T10:58:41.341 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-11-28T10:58:41.341 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-11-28T10:58:41.341 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_si.ko 2023-11-28T10:58:41.341 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-11-28T10:58:41.342 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-11-28T10:58:41.342 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/lp.ko 2023-11-28T10:58:41.342 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/mwave 2023-11-28T10:58:41.342 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/mwave/mwave.ko 2023-11-28T10:58:41.342 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ppdev.ko 2023-11-28T10:58:41.342 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tlclk.ko 2023-11-28T10:58:41.342 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm 2023-11-28T10:58:41.342 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm.ko 2023-11-28T10:58:41.342 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_atmel.ko 2023-11-28T10:58:41.342 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_infineon.ko 2023-11-28T10:58:41.342 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_nsc.ko 2023-11-28T10:58:41.343 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_tis.ko 2023-11-28T10:58:41.343 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-11-28T10:58:41.343 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/uv_mmtimer.ko 2023-11-28T10:58:41.343 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/virtio_console.ko 2023-11-28T10:58:41.343 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq 2023-11-28T10:58:41.343 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-11-28T10:58:41.343 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-11-28T10:58:41.343 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/p4-clockmod.ko 2023-11-28T10:58:41.343 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-11-28T10:58:41.343 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/powernow-k8.ko 2023-11-28T10:58:41.343 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/speedstep-lib.ko 2023-11-28T10:58:41.343 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto 2023-11-28T10:58:41.344 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp 2023-11-28T10:58:41.344 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-11-28T10:58:41.344 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ccp.ko 2023-11-28T10:58:41.344 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel 2023-11-28T10:58:41.344 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat 2023-11-28T10:58:41.344 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_common 2023-11-28T10:58:41.344 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-11-28T10:58:41.344 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-11-28T10:58:41.344 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-11-28T10:58:41.344 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/padlock-aes.ko 2023-11-28T10:58:41.344 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/padlock-sha.ko 2023-11-28T10:58:41.344 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/virtio 2023-11-28T10:58:41.345 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-11-28T10:58:41.345 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dca 2023-11-28T10:58:41.345 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dca/dca.ko 2023-11-28T10:58:41.345 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/devfreq 2023-11-28T10:58:41.345 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-11-28T10:58:41.345 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma 2023-11-28T10:58:41.345 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/dw 2023-11-28T10:58:41.345 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/dw/dw_dmac.ko 2023-11-28T10:58:41.345 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ioat 2023-11-28T10:58:41.345 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ioat/ioatdma.ko 2023-11-28T10:58:41.345 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac 2023-11-28T10:58:41.345 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/amd64_edac.ko 2023-11-28T10:58:41.346 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/e752x_edac.ko 2023-11-28T10:58:41.346 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/edac_mce_amd.ko 2023-11-28T10:58:41.346 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i3000_edac.ko 2023-11-28T10:58:41.346 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i3200_edac.ko 2023-11-28T10:58:41.346 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i5100_edac.ko 2023-11-28T10:58:41.346 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i5400_edac.ko 2023-11-28T10:58:41.346 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i7300_edac.ko 2023-11-28T10:58:41.346 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i7core_edac.ko 2023-11-28T10:58:41.346 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i82975x_edac.ko 2023-11-28T10:58:41.346 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/ie31200_edac.ko 2023-11-28T10:58:41.346 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/sb_edac.ko 2023-11-28T10:58:41.347 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/x38_edac.ko 2023-11-28T10:58:41.347 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware 2023-11-28T10:58:41.347 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/edd.ko 2023-11-28T10:58:41.347 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/iscsi_ibft.ko 2023-11-28T10:58:41.347 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio 2023-11-28T10:58:41.347 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-ich.ko 2023-11-28T10:58:41.347 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-viperboard.ko 2023-11-28T10:58:41.347 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid 2023-11-28T10:58:41.347 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-appleir.ko 2023-11-28T10:58:41.347 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-aureal.ko 2023-11-28T10:58:41.347 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-axff.ko 2023-11-28T10:58:41.347 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-dr.ko 2023-11-28T10:58:41.348 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-elecom.ko 2023-11-28T10:58:41.348 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-elo.ko 2023-11-28T10:58:41.348 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-emsff.ko 2023-11-28T10:58:41.348 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gaff.ko 2023-11-28T10:58:41.348 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gt683r.ko 2023-11-28T10:58:41.348 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gyration.ko 2023-11-28T10:58:41.348 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtek-kbd.ko 2023-11-28T10:58:41.348 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtek-mouse.ko 2023-11-28T10:58:41.348 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtekff.ko 2023-11-28T10:58:41.348 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-hyperv.ko 2023-11-28T10:58:41.348 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-icade.ko 2023-11-28T10:58:41.348 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-keytouch.ko 2023-11-28T10:58:41.348 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-kye.ko 2023-11-28T10:58:41.349 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lcpower.ko 2023-11-28T10:58:41.349 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-led.ko 2023-11-28T10:58:41.349 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lenovo.ko 2023-11-28T10:58:41.349 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech-dj.ko 2023-11-28T10:58:41.349 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-11-28T10:58:41.349 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-multitouch.ko 2023-11-28T10:58:41.349 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-ortek.ko 2023-11-28T10:58:41.349 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-penmount.ko 2023-11-28T10:58:41.349 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-petalynx.ko 2023-11-28T10:58:41.349 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-picolcd.ko 2023-11-28T10:58:41.349 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-pl.ko 2023-11-28T10:58:41.349 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-plantronics.ko 2023-11-28T10:58:41.349 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-primax.ko 2023-11-28T10:58:41.350 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-rmi.ko 2023-11-28T10:58:41.350 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-arvo.ko 2023-11-28T10:58:41.350 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-common.ko 2023-11-28T10:58:41.350 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-isku.ko 2023-11-28T10:58:41.350 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-kone.ko 2023-11-28T10:58:41.350 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-11-28T10:58:41.350 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-konepure.ko 2023-11-28T10:58:41.350 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-11-28T10:58:41.350 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-lua.ko 2023-11-28T10:58:41.350 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-pyra.ko 2023-11-28T10:58:41.350 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-ryos.ko 2023-11-28T10:58:41.350 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-savu.ko 2023-11-28T10:58:41.350 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat.ko 2023-11-28T10:58:41.350 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-saitek.ko 2023-11-28T10:58:41.351 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-samsung.ko 2023-11-28T10:58:41.351 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sensor-hub.ko 2023-11-28T10:58:41.351 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sjoy.ko 2023-11-28T10:58:41.351 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sony.ko 2023-11-28T10:58:41.351 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-speedlink.ko 2023-11-28T10:58:41.351 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-steelseries.ko 2023-11-28T10:58:41.351 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sunplus.ko 2023-11-28T10:58:41.351 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-thrustmaster.ko 2023-11-28T10:58:41.351 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-tivo.ko 2023-11-28T10:58:41.351 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-tmff.ko 2023-11-28T10:58:41.351 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-topseed.ko 2023-11-28T10:58:41.351 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-twinhan.ko 2023-11-28T10:58:41.351 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-uclogic.ko 2023-11-28T10:58:41.351 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-waltop.ko 2023-11-28T10:58:41.352 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-wiimote.ko 2023-11-28T10:58:41.352 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-xinmo.ko 2023-11-28T10:58:41.352 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-zpff.ko 2023-11-28T10:58:41.352 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-zydacron.ko 2023-11-28T10:58:41.352 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/uhid.ko 2023-11-28T10:58:41.352 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/wacom.ko 2023-11-28T10:58:41.352 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv 2023-11-28T10:58:41.352 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_balloon.ko 2023-11-28T10:58:41.352 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_utils.ko 2023-11-28T10:58:41.352 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_vmbus.ko 2023-11-28T10:58:41.352 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon 2023-11-28T10:58:41.352 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/abituguru.ko 2023-11-28T10:58:41.352 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/abituguru3.ko 2023-11-28T10:58:41.352 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/acpi_power_meter.ko 2023-11-28T10:58:41.352 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ad7414.ko 2023-11-28T10:58:41.353 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ad7418.ko 2023-11-28T10:58:41.353 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adc128d818.ko 2023-11-28T10:58:41.353 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1025.ko 2023-11-28T10:58:41.353 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1026.ko 2023-11-28T10:58:41.353 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1029.ko 2023-11-28T10:58:41.353 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1031.ko 2023-11-28T10:58:41.353 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm9240.ko 2023-11-28T10:58:41.353 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ads7828.ko 2023-11-28T10:58:41.353 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7410.ko 2023-11-28T10:58:41.353 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7411.ko 2023-11-28T10:58:41.353 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7462.ko 2023-11-28T10:58:41.353 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7470.ko 2023-11-28T10:58:41.353 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7475.ko 2023-11-28T10:58:41.353 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7x10.ko 2023-11-28T10:58:41.354 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/amc6821.ko 2023-11-28T10:58:41.354 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/applesmc.ko 2023-11-28T10:58:41.354 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asb100.ko 2023-11-28T10:58:41.354 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asc7621.ko 2023-11-28T10:58:41.354 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asus_atk0110.ko 2023-11-28T10:58:41.354 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/atxp1.ko 2023-11-28T10:58:41.354 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/coretemp.ko 2023-11-28T10:58:41.354 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-11-28T10:58:41.354 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/dme1737.ko 2023-11-28T10:58:41.354 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ds1621.ko 2023-11-28T10:58:41.354 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ds620.ko 2023-11-28T10:58:41.354 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc1403.ko 2023-11-28T10:58:41.354 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc6w201.ko 2023-11-28T10:58:41.355 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f71805f.ko 2023-11-28T10:58:41.355 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f71882fg.ko 2023-11-28T10:58:41.355 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f75375s.ko 2023-11-28T10:58:41.355 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/fam15h_power.ko 2023-11-28T10:58:41.355 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/fschmd.ko 2023-11-28T10:58:41.355 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/g760a.ko 2023-11-28T10:58:41.355 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/g762.ko 2023-11-28T10:58:41.355 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/gl518sm.ko 2023-11-28T10:58:41.355 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/gl520sm.ko 2023-11-28T10:58:41.355 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/hwmon-vid.ko 2023-11-28T10:58:41.355 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/i5500_temp.ko 2023-11-28T10:58:41.355 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/i5k_amb.ko 2023-11-28T10:58:41.355 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ibmaem.ko 2023-11-28T10:58:41.356 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ibmpex.ko 2023-11-28T10:58:41.356 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ina209.ko 2023-11-28T10:58:41.356 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ina2xx.ko 2023-11-28T10:58:41.356 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/it87.ko 2023-11-28T10:58:41.356 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/k10temp.ko 2023-11-28T10:58:41.356 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/k8temp.ko 2023-11-28T10:58:41.356 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lineage-pem.ko 2023-11-28T10:58:41.356 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm63.ko 2023-11-28T10:58:41.356 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm73.ko 2023-11-28T10:58:41.356 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm75.ko 2023-11-28T10:58:41.356 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm77.ko 2023-11-28T10:58:41.357 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm78.ko 2023-11-28T10:58:41.357 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm80.ko 2023-11-28T10:58:41.357 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm83.ko 2023-11-28T10:58:41.357 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm85.ko 2023-11-28T10:58:41.357 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm87.ko 2023-11-28T10:58:41.357 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm90.ko 2023-11-28T10:58:41.357 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm92.ko 2023-11-28T10:58:41.357 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm93.ko 2023-11-28T10:58:41.357 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95234.ko 2023-11-28T10:58:41.357 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95241.ko 2023-11-28T10:58:41.357 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95245.ko 2023-11-28T10:58:41.357 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc2945.ko 2023-11-28T10:58:41.358 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4151.ko 2023-11-28T10:58:41.358 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4215.ko 2023-11-28T10:58:41.358 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4222.ko 2023-11-28T10:58:41.358 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4245.ko 2023-11-28T10:58:41.358 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4260.ko 2023-11-28T10:58:41.358 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4261.ko 2023-11-28T10:58:41.358 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max16065.ko 2023-11-28T10:58:41.358 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max1619.ko 2023-11-28T10:58:41.358 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max1668.ko 2023-11-28T10:58:41.358 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max197.ko 2023-11-28T10:58:41.358 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6639.ko 2023-11-28T10:58:41.358 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6650.ko 2023-11-28T10:58:41.359 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6697.ko 2023-11-28T10:58:41.359 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/mcp3021.ko 2023-11-28T10:58:41.359 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6683.ko 2023-11-28T10:58:41.359 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6775-core.ko 2023-11-28T10:58:41.359 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6775.ko 2023-11-28T10:58:41.359 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct7802.ko 2023-11-28T10:58:41.359 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pc87360.ko 2023-11-28T10:58:41.359 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pc87427.ko 2023-11-28T10:58:41.359 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pcf8591.ko 2023-11-28T10:58:41.359 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus 2023-11-28T10:58:41.359 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-11-28T10:58:41.360 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-11-28T10:58:41.360 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-11-28T10:58:41.360 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max16064.ko 2023-11-28T10:58:41.360 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max34440.ko 2023-11-28T10:58:41.360 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max8688.ko 2023-11-28T10:58:41.360 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-11-28T10:58:41.360 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-11-28T10:58:41.360 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-11-28T10:58:41.360 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-11-28T10:58:41.360 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-11-28T10:58:41.360 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-11-28T10:58:41.360 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/powr1220.ko 2023-11-28T10:58:41.361 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch5627.ko 2023-11-28T10:58:41.361 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch5636.ko 2023-11-28T10:58:41.361 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch56xx-common.ko 2023-11-28T10:58:41.361 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sht15.ko 2023-11-28T10:58:41.361 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sht21.ko 2023-11-28T10:58:41.361 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/shtc1.ko 2023-11-28T10:58:41.361 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sis5595.ko 2023-11-28T10:58:41.361 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47b397.ko 2023-11-28T10:58:41.361 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47m1.ko 2023-11-28T10:58:41.361 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47m192.ko 2023-11-28T10:58:41.361 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/thmc50.ko 2023-11-28T10:58:41.362 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp102.ko 2023-11-28T10:58:41.362 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp103.ko 2023-11-28T10:58:41.362 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp401.ko 2023-11-28T10:58:41.362 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp421.ko 2023-11-28T10:58:41.362 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/via-cputemp.ko 2023-11-28T10:58:41.362 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/via686a.ko 2023-11-28T10:58:41.362 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/vt1211.ko 2023-11-28T10:58:41.362 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/vt8231.ko 2023-11-28T10:58:41.362 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83627ehf.ko 2023-11-28T10:58:41.362 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83627hf.ko 2023-11-28T10:58:41.362 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83781d.ko 2023-11-28T10:58:41.362 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83791d.ko 2023-11-28T10:58:41.363 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83792d.ko 2023-11-28T10:58:41.363 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83793.ko 2023-11-28T10:58:41.363 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83795.ko 2023-11-28T10:58:41.363 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83l785ts.ko 2023-11-28T10:58:41.363 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83l786ng.ko 2023-11-28T10:58:41.363 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c 2023-11-28T10:58:41.363 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos 2023-11-28T10:58:41.363 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-11-28T10:58:41.363 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-11-28T10:58:41.363 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses 2023-11-28T10:58:41.363 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-11-28T10:58:41.364 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-11-28T10:58:41.364 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-11-28T10:58:41.364 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-11-28T10:58:41.364 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-11-28T10:58:41.364 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-11-28T10:58:41.364 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-11-28T10:58:41.364 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-11-28T10:58:41.364 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-i801.ko 2023-11-28T10:58:41.364 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-isch.ko 2023-11-28T10:58:41.364 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-11-28T10:58:41.364 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-11-28T10:58:41.364 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-11-28T10:58:41.364 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-parport.ko 2023-11-28T10:58:41.365 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-11-28T10:58:41.365 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-11-28T10:58:41.365 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-11-28T10:58:41.365 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-11-28T10:58:41.365 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-11-28T10:58:41.365 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-11-28T10:58:41.365 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-via.ko 2023-11-28T10:58:41.365 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-11-28T10:58:41.365 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-11-28T10:58:41.365 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-dev.ko 2023-11-28T10:58:41.365 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-mux.ko 2023-11-28T10:58:41.365 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-11-28T10:58:41.366 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-smbus.ko 2023-11-28T10:58:41.366 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-stub.ko 2023-11-28T10:58:41.366 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband 2023-11-28T10:58:41.366 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core 2023-11-28T10:58:41.366 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_cm.ko 2023-11-28T10:58:41.366 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_core.ko 2023-11-28T10:58:41.366 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_umad.ko 2023-11-28T10:58:41.366 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-11-28T10:58:41.366 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/iw_cm.ko 2023-11-28T10:58:41.366 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/rdma_cm.ko 2023-11-28T10:58:41.366 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-11-28T10:58:41.367 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw 2023-11-28T10:58:41.367 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/cxgb4 2023-11-28T10:58:41.367 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-11-28T10:58:41.367 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx4 2023-11-28T10:58:41.367 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-11-28T10:58:41.367 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx5 2023-11-28T10:58:41.367 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-11-28T10:58:41.367 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mthca 2023-11-28T10:58:41.367 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-11-28T10:58:41.367 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp 2023-11-28T10:58:41.367 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ipoib 2023-11-28T10:58:41.367 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-11-28T10:58:41.368 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/iser 2023-11-28T10:58:41.368 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-11-28T10:58:41.368 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/isert 2023-11-28T10:58:41.368 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-11-28T10:58:41.368 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srp 2023-11-28T10:58:41.368 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-11-28T10:58:41.368 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srpt 2023-11-28T10:58:41.368 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-11-28T10:58:41.368 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input 2023-11-28T10:58:41.368 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport 2023-11-28T10:58:41.368 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-11-28T10:58:41.368 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/fm801-gp.ko 2023-11-28T10:58:41.369 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/gameport.ko 2023-11-28T10:58:41.369 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/joydev.ko 2023-11-28T10:58:41.369 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard 2023-11-28T10:58:41.369 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/gpio_keys.ko 2023-11-28T10:58:41.369 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse 2023-11-28T10:58:41.369 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/appletouch.ko 2023-11-28T10:58:41.369 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/bcm5974.ko 2023-11-28T10:58:41.369 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/cyapatp.ko 2023-11-28T10:58:41.369 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/elan_i2c.ko 2023-11-28T10:58:41.369 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/sermouse.ko 2023-11-28T10:58:41.369 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-11-28T10:58:41.370 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/synaptics_usb.ko 2023-11-28T10:58:41.370 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/vsxxxaa.ko 2023-11-28T10:58:41.370 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/rmi4 2023-11-28T10:58:41.370 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/rmi4/rmi_core.ko 2023-11-28T10:58:41.370 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio 2023-11-28T10:58:41.370 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/altera_ps2.ko 2023-11-28T10:58:41.370 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/arc_ps2.ko 2023-11-28T10:58:41.370 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-11-28T10:58:41.370 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/serio_raw.ko 2023-11-28T10:58:41.370 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/sparse-keymap.ko 2023-11-28T10:58:41.370 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu 2023-11-28T10:58:41.370 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/amd 2023-11-28T10:58:41.371 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/amd/iommu_v2.ko 2023-11-28T10:58:41.371 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds 2023-11-28T10:58:41.371 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-blinkm.ko 2023-11-28T10:58:41.371 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lm3530.ko 2023-11-28T10:58:41.371 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lp3944.ko 2023-11-28T10:58:41.371 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-ss4200.ko 2023-11-28T10:58:41.371 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger 2023-11-28T10:58:41.371 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-11-28T10:58:41.371 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-11-28T10:58:41.371 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-11-28T10:58:41.371 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-11-28T10:58:41.371 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-11-28T10:58:41.372 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-11-28T10:58:41.372 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-11-28T10:58:41.372 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-11-28T10:58:41.372 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md 2023-11-28T10:58:41.372 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/bcache 2023-11-28T10:58:41.372 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/bcache/bcache.ko 2023-11-28T10:58:41.372 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-bio-prison.ko 2023-11-28T10:58:41.372 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-cache-smq.ko 2023-11-28T10:58:41.372 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-cache.ko 2023-11-28T10:58:41.372 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-crypt.ko 2023-11-28T10:58:41.372 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-delay.ko 2023-11-28T10:58:41.372 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-flakey.ko 2023-11-28T10:58:41.373 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-log-userspace.ko 2023-11-28T10:58:41.373 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-multipath.ko 2023-11-28T10:58:41.373 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-queue-length.ko 2023-11-28T10:58:41.373 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-raid.ko 2023-11-28T10:58:41.373 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-round-robin.ko 2023-11-28T10:58:41.373 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-service-time.ko 2023-11-28T10:58:41.373 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-switch.ko 2023-11-28T10:58:41.373 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-thin-pool.ko 2023-11-28T10:58:41.373 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-verity.ko 2023-11-28T10:58:41.373 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/faulty.ko 2023-11-28T10:58:41.373 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/linear.ko 2023-11-28T10:58:41.373 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/multipath.ko 2023-11-28T10:58:41.374 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/persistent-data 2023-11-28T10:58:41.374 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-11-28T10:58:41.374 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid0.ko 2023-11-28T10:58:41.374 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid1.ko 2023-11-28T10:58:41.374 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid10.ko 2023-11-28T10:58:41.374 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid456.ko 2023-11-28T10:58:41.374 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message 2023-11-28T10:58:41.374 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion 2023-11-28T10:58:41.374 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptbase.ko 2023-11-28T10:58:41.374 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptctl.ko 2023-11-28T10:58:41.374 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptfc.ko 2023-11-28T10:58:41.374 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptlan.ko 2023-11-28T10:58:41.375 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptsas.ko 2023-11-28T10:58:41.375 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptscsih.ko 2023-11-28T10:58:41.375 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptspi.ko 2023-11-28T10:58:41.375 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd 2023-11-28T10:58:41.375 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lpc_ich.ko 2023-11-28T10:58:41.375 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lpc_sch.ko 2023-11-28T10:58:41.375 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/mfd-core.ko 2023-11-28T10:58:41.375 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/sm501.ko 2023-11-28T10:58:41.375 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/viperboard.ko 2023-11-28T10:58:41.375 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/vx855.ko 2023-11-28T10:58:41.375 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/wl1273-core.ko 2023-11-28T10:58:41.375 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc 2023-11-28T10:58:41.376 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/altera-stapl 2023-11-28T10:58:41.376 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-11-28T10:58:41.376 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/apds9802als.ko 2023-11-28T10:58:41.376 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/apds990x.ko 2023-11-28T10:58:41.376 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/bh1770glc.ko 2023-11-28T10:58:41.376 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/cb710 2023-11-28T10:58:41.376 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/cb710/cb710.ko 2023-11-28T10:58:41.376 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom 2023-11-28T10:58:41.376 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/at24.ko 2023-11-28T10:58:41.376 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/eeprom.ko 2023-11-28T10:58:41.376 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-11-28T10:58:41.377 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/max6875.ko 2023-11-28T10:58:41.377 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/enclosure.ko 2023-11-28T10:58:41.377 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/hpilo.ko 2023-11-28T10:58:41.377 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/isl29003.ko 2023-11-28T10:58:41.377 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/isl29020.ko 2023-11-28T10:58:41.377 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d 2023-11-28T10:58:41.377 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-11-28T10:58:41.377 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-11-28T10:58:41.377 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei 2023-11-28T10:58:41.377 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei-me.ko 2023-11-28T10:58:41.377 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei-txe.ko 2023-11-28T10:58:41.377 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei.ko 2023-11-28T10:58:41.378 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-gru 2023-11-28T10:58:41.378 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-gru/gru.ko 2023-11-28T10:58:41.378 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp 2023-11-28T10:58:41.378 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xp.ko 2023-11-28T10:58:41.378 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xpc.ko 2023-11-28T10:58:41.378 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-11-28T10:58:41.378 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tifm_7xx1.ko 2023-11-28T10:58:41.378 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tifm_core.ko 2023-11-28T10:58:41.378 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tsl2550.ko 2023-11-28T10:58:41.378 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_balloon.ko 2023-11-28T10:58:41.378 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_vmci 2023-11-28T10:58:41.378 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-11-28T10:58:41.379 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net 2023-11-28T10:58:41.379 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/bonding 2023-11-28T10:58:41.379 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/bonding/bonding.ko 2023-11-28T10:58:41.379 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa 2023-11-28T10:58:41.379 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/b53 2023-11-28T10:58:41.379 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/b53/b53_common.ko 2023-11-28T10:58:41.379 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/bcm-sf2.ko 2023-11-28T10:58:41.379 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6060.ko 2023-11-28T10:58:41.379 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6xxx 2023-11-28T10:58:41.379 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-11-28T10:58:41.379 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dummy.ko 2023-11-28T10:58:41.379 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/eql.ko 2023-11-28T10:58:41.380 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet 2023-11-28T10:58:41.380 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com 2023-11-28T10:58:41.380 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-11-28T10:58:41.380 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-11-28T10:58:41.380 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-11-28T10:58:41.380 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-11-28T10:58:41.380 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390 2023-11-28T10:58:41.380 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/8390.ko 2023-11-28T10:58:41.380 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-11-28T10:58:41.380 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-11-28T10:58:41.380 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-11-28T10:58:41.380 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/adaptec 2023-11-28T10:58:41.381 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-11-28T10:58:41.381 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/agere 2023-11-28T10:58:41.381 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/agere/et131x.ko 2023-11-28T10:58:41.381 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/alteon 2023-11-28T10:58:41.381 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-11-28T10:58:41.381 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/altera 2023-11-28T10:58:41.381 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-11-28T10:58:41.381 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd 2023-11-28T10:58:41.381 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-11-28T10:58:41.381 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-11-28T10:58:41.381 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-11-28T10:58:41.382 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros 2023-11-28T10:58:41.382 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/alx 2023-11-28T10:58:41.382 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-11-28T10:58:41.382 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1c 2023-11-28T10:58:41.382 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-11-28T10:58:41.382 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1e 2023-11-28T10:58:41.382 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-11-28T10:58:41.382 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx 2023-11-28T10:58:41.382 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-11-28T10:58:41.382 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-11-28T10:58:41.382 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom 2023-11-28T10:58:41.382 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-11-28T10:58:41.383 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-11-28T10:58:41.383 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-11-28T10:58:41.383 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-11-28T10:58:41.383 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-11-28T10:58:41.383 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/genet 2023-11-28T10:58:41.383 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-11-28T10:58:41.383 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-11-28T10:58:41.383 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade 2023-11-28T10:58:41.383 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/bna 2023-11-28T10:58:41.383 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-11-28T10:58:41.383 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio 2023-11-28T10:58:41.384 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb 2023-11-28T10:58:41.384 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-11-28T10:58:41.384 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-11-28T10:58:41.384 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-11-28T10:58:41.384 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-11-28T10:58:41.384 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-11-28T10:58:41.384 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-11-28T10:58:41.384 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-11-28T10:58:41.384 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-11-28T10:58:41.384 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-11-28T10:58:41.384 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco 2023-11-28T10:58:41.384 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/enic 2023-11-28T10:58:41.385 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-11-28T10:58:41.385 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec 2023-11-28T10:58:41.385 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip 2023-11-28T10:58:41.385 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-11-28T10:58:41.385 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-11-28T10:58:41.385 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-11-28T10:58:41.385 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-11-28T10:58:41.385 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-11-28T10:58:41.385 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-11-28T10:58:41.385 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink 2023-11-28T10:58:41.385 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-11-28T10:58:41.385 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-11-28T10:58:41.385 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dnet.ko 2023-11-28T10:58:41.386 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex 2023-11-28T10:58:41.386 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/benet 2023-11-28T10:58:41.386 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-11-28T10:58:41.386 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ethoc.ko 2023-11-28T10:58:41.386 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/fealnx.ko 2023-11-28T10:58:41.386 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel 2023-11-28T10:58:41.386 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e100.ko 2023-11-28T10:58:41.386 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000 2023-11-28T10:58:41.386 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-11-28T10:58:41.386 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000e 2023-11-28T10:58:41.386 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-11-28T10:58:41.387 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/fm10k 2023-11-28T10:58:41.387 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-11-28T10:58:41.387 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/i40e 2023-11-28T10:58:41.387 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-11-28T10:58:41.387 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/iavf 2023-11-28T10:58:41.387 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-11-28T10:58:41.387 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igb 2023-11-28T10:58:41.387 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-11-28T10:58:41.387 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igbvf 2023-11-28T10:58:41.387 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-11-28T10:58:41.387 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbe 2023-11-28T10:58:41.387 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-11-28T10:58:41.388 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbevf 2023-11-28T10:58:41.388 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-11-28T10:58:41.388 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/jme.ko 2023-11-28T10:58:41.388 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell 2023-11-28T10:58:41.388 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-11-28T10:58:41.388 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/skge.ko 2023-11-28T10:58:41.388 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-11-28T10:58:41.388 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox 2023-11-28T10:58:41.388 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4 2023-11-28T10:58:41.388 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-11-28T10:58:41.388 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-11-28T10:58:41.388 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5 2023-11-28T10:58:41.389 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-11-28T10:58:41.389 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-11-28T10:58:41.389 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel 2023-11-28T10:58:41.389 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-11-28T10:58:41.389 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom 2023-11-28T10:58:41.389 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/myri10ge 2023-11-28T10:58:41.389 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-11-28T10:58:41.389 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi 2023-11-28T10:58:41.389 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-11-28T10:58:41.389 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-11-28T10:58:41.389 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/neterion 2023-11-28T10:58:41.389 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-11-28T10:58:41.390 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/nvidia 2023-11-28T10:58:41.390 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-11-28T10:58:41.390 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines 2023-11-28T10:58:41.390 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-11-28T10:58:41.390 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-11-28T10:58:41.390 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic 2023-11-28T10:58:41.390 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/netxen 2023-11-28T10:58:41.390 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-11-28T10:58:41.390 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-11-28T10:58:41.390 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-11-28T10:58:41.390 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-11-28T10:58:41.391 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rdc 2023-11-28T10:58:41.391 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-11-28T10:58:41.391 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek 2023-11-28T10:58:41.391 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-11-28T10:58:41.391 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-11-28T10:58:41.391 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/atp.ko 2023-11-28T10:58:41.391 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-11-28T10:58:41.391 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rocker 2023-11-28T10:58:41.391 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-11-28T10:58:41.391 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sfc 2023-11-28T10:58:41.391 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-11-28T10:58:41.391 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/silan 2023-11-28T10:58:41.392 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-11-28T10:58:41.392 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis 2023-11-28T10:58:41.392 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/sis190.ko 2023-11-28T10:58:41.392 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/sis900.ko 2023-11-28T10:58:41.392 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc 2023-11-28T10:58:41.392 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-11-28T10:58:41.392 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-11-28T10:58:41.392 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-11-28T10:58:41.392 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-11-28T10:58:41.392 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro 2023-11-28T10:58:41.393 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac 2023-11-28T10:58:41.393 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-11-28T10:58:41.393 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-11-28T10:58:41.393 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun 2023-11-28T10:58:41.393 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/cassini.ko 2023-11-28T10:58:41.393 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/niu.ko 2023-11-28T10:58:41.393 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/sungem.ko 2023-11-28T10:58:41.393 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-11-28T10:58:41.393 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/tehuti 2023-11-28T10:58:41.393 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-11-28T10:58:41.393 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ti 2023-11-28T10:58:41.394 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ti/tlan.ko 2023-11-28T10:58:41.394 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via 2023-11-28T10:58:41.394 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-11-28T10:58:41.394 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-11-28T10:58:41.394 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet 2023-11-28T10:58:41.394 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-11-28T10:58:41.394 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-11-28T10:58:41.394 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/xircom 2023-11-28T10:58:41.394 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-11-28T10:58:41.394 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/geneve.ko 2023-11-28T10:58:41.394 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/hyperv 2023-11-28T10:58:41.394 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-11-28T10:58:41.395 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ieee802154 2023-11-28T10:58:41.395 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ieee802154/fakelb.ko 2023-11-28T10:58:41.395 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ifb.ko 2023-11-28T10:58:41.395 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ipvlan 2023-11-28T10:58:41.395 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ipvlan/ipvlan.ko 2023-11-28T10:58:41.395 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/macvlan.ko 2023-11-28T10:58:41.395 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/macvtap.ko 2023-11-28T10:58:41.396 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio 2023-11-28T10:58:41.396 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio.ko 2023-11-28T10:58:41.396 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-11-28T10:58:41.396 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-11-28T10:58:41.396 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-regmap.ko 2023-11-28T10:58:41.396 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mii.ko 2023-11-28T10:58:41.396 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/net_failover.ko 2023-11-28T10:58:41.397 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/netconsole.ko 2023-11-28T10:58:41.397 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/nlmon.ko 2023-11-28T10:58:41.397 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs 2023-11-28T10:58:41.397 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/pcs-lynx.ko 2023-11-28T10:58:41.397 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-11-28T10:58:41.397 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy 2023-11-28T10:58:41.397 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/amd.ko 2023-11-28T10:58:41.397 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-11-28T10:58:41.397 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-11-28T10:58:41.397 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm7xxx.ko 2023-11-28T10:58:41.397 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm87xx.ko 2023-11-28T10:58:41.397 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/broadcom.ko 2023-11-28T10:58:41.398 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/cicada.ko 2023-11-28T10:58:41.398 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/davicom.ko 2023-11-28T10:58:41.398 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/dp83640.ko 2023-11-28T10:58:41.398 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/et1011c.ko 2023-11-28T10:58:41.398 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/icplus.ko 2023-11-28T10:58:41.398 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/lxt.ko 2023-11-28T10:58:41.398 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/marvell.ko 2023-11-28T10:58:41.398 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/micrel.ko 2023-11-28T10:58:41.398 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/national.ko 2023-11-28T10:58:41.398 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/phylink.ko 2023-11-28T10:58:41.398 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/qsemi.ko 2023-11-28T10:58:41.399 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/realtek.ko 2023-11-28T10:58:41.399 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/smsc.ko 2023-11-28T10:58:41.399 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/ste10Xp.ko 2023-11-28T10:58:41.399 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/vitesse.ko 2023-11-28T10:58:41.399 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp 2023-11-28T10:58:41.399 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/bsd_comp.ko 2023-11-28T10:58:41.399 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_async.ko 2023-11-28T10:58:41.399 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_deflate.ko 2023-11-28T10:58:41.399 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_generic.ko 2023-11-28T10:58:41.399 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_mppe.ko 2023-11-28T10:58:41.399 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_synctty.ko 2023-11-28T10:58:41.399 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pppoe.ko 2023-11-28T10:58:41.399 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pppox.ko 2023-11-28T10:58:41.400 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pptp.ko 2023-11-28T10:58:41.400 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip 2023-11-28T10:58:41.400 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip/slhc.ko 2023-11-28T10:58:41.400 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip/slip.ko 2023-11-28T10:58:41.400 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/sungem_phy.ko 2023-11-28T10:58:41.400 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/tap.ko 2023-11-28T10:58:41.400 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team 2023-11-28T10:58:41.400 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team.ko 2023-11-28T10:58:41.400 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_activebackup.ko 2023-11-28T10:58:41.400 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_broadcast.ko 2023-11-28T10:58:41.400 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-11-28T10:58:41.400 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_random.ko 2023-11-28T10:58:41.401 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-11-28T10:58:41.401 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/tun.ko 2023-11-28T10:58:41.401 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/veth.ko 2023-11-28T10:58:41.401 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/virtio_net.ko 2023-11-28T10:58:41.401 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vmxnet3 2023-11-28T10:58:41.401 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-11-28T10:58:41.401 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vxlan 2023-11-28T10:58:41.401 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vxlan/vxlan.ko 2023-11-28T10:58:41.401 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netback 2023-11-28T10:58:41.401 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netback/xen-netback.ko 2023-11-28T10:58:41.401 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netfront.ko 2023-11-28T10:58:41.401 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ntb 2023-11-28T10:58:41.402 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ntb/ntb.ko 2023-11-28T10:58:41.402 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme 2023-11-28T10:58:41.402 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host 2023-11-28T10:58:41.402 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/nvme-core.ko 2023-11-28T10:58:41.402 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/nvme.ko 2023-11-28T10:58:41.402 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport 2023-11-28T10:58:41.402 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport.ko 2023-11-28T10:58:41.402 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_cs.ko 2023-11-28T10:58:41.402 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_pc.ko 2023-11-28T10:58:41.402 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_serial.ko 2023-11-28T10:58:41.402 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci 2023-11-28T10:58:41.403 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug 2023-11-28T10:58:41.403 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-11-28T10:58:41.403 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/pcie 2023-11-28T10:58:41.403 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/pcie/aer_inject.ko 2023-11-28T10:58:41.403 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/xen-pcifront.ko 2023-11-28T10:58:41.403 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia 2023-11-28T10:58:41.403 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/i82092.ko 2023-11-28T10:58:41.403 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/pd6729.ko 2023-11-28T10:58:41.403 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/yenta_socket.ko 2023-11-28T10:58:41.403 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl 2023-11-28T10:58:41.403 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/intel 2023-11-28T10:58:41.403 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-11-28T10:58:41.404 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform 2023-11-28T10:58:41.404 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86 2023-11-28T10:58:41.404 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/acer-wmi.ko 2023-11-28T10:58:41.404 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/acerhdf.ko 2023-11-28T10:58:41.404 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/apple-gmux.ko 2023-11-28T10:58:41.404 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-laptop.ko 2023-11-28T10:58:41.404 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-11-28T10:58:41.404 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-wmi.ko 2023-11-28T10:58:41.404 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/classmate-laptop.ko 2023-11-28T10:58:41.405 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-11-28T10:58:41.405 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-11-28T10:58:41.405 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-11-28T10:58:41.405 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-11-28T10:58:41.405 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/hdaps.ko 2023-11-28T10:58:41.405 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel 2023-11-28T10:58:41.405 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-11-28T10:58:41.405 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel_ips.ko 2023-11-28T10:58:41.405 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/msi-wmi.ko 2023-11-28T10:58:41.405 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/mxm-wmi.ko 2023-11-28T10:58:41.405 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-11-28T10:58:41.405 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/samsung-laptop.ko 2023-11-28T10:58:41.406 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/samsung-q10.ko 2023-11-28T10:58:41.406 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/topstar-laptop.ko 2023-11-28T10:58:41.406 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-11-28T10:58:41.406 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/toshiba_haps.ko 2023-11-28T10:58:41.406 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/wmi-bmof.ko 2023-11-28T10:58:41.406 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/wmi.ko 2023-11-28T10:58:41.406 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap 2023-11-28T10:58:41.406 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/intel_rapl_common.ko 2023-11-28T10:58:41.406 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/intel_rapl_msr.ko 2023-11-28T10:58:41.406 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps 2023-11-28T10:58:41.406 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients 2023-11-28T10:58:41.407 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps-gpio.ko 2023-11-28T10:58:41.407 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps-ldisc.ko 2023-11-28T10:58:41.407 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps_parport.ko 2023-11-28T10:58:41.407 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/pps_core.ko 2023-11-28T10:58:41.407 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp 2023-11-28T10:58:41.407 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ptp.ko 2023-11-28T10:58:41.407 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ptp_kvm.ko 2023-11-28T10:58:41.407 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc 2023-11-28T10:58:41.407 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-bq32k.ko 2023-11-28T10:58:41.407 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1286.ko 2023-11-28T10:58:41.407 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1307.ko 2023-11-28T10:58:41.408 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1374.ko 2023-11-28T10:58:41.408 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1511.ko 2023-11-28T10:58:41.408 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1553.ko 2023-11-28T10:58:41.408 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1672.ko 2023-11-28T10:58:41.408 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1742.ko 2023-11-28T10:58:41.408 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds2404.ko 2023-11-28T10:58:41.408 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds3232.ko 2023-11-28T10:58:41.408 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-em3027.ko 2023-11-28T10:58:41.408 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-fm3130.ko 2023-11-28T10:58:41.408 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-isl12022.ko 2023-11-28T10:58:41.408 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-isl1208.ko 2023-11-28T10:58:41.408 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m41t80.ko 2023-11-28T10:58:41.409 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t35.ko 2023-11-28T10:58:41.409 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t59.ko 2023-11-28T10:58:41.409 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-max6900.ko 2023-11-28T10:58:41.409 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-msm6242.ko 2023-11-28T10:58:41.409 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf2127.ko 2023-11-28T10:58:41.409 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf85063.ko 2023-11-28T10:58:41.409 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8523.ko 2023-11-28T10:58:41.409 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8563.ko 2023-11-28T10:58:41.409 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8583.ko 2023-11-28T10:58:41.409 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rp5c01.ko 2023-11-28T10:58:41.409 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rs5c372.ko 2023-11-28T10:58:41.409 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-11-28T10:58:41.410 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rx8025.ko 2023-11-28T10:58:41.410 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rx8581.ko 2023-11-28T10:58:41.410 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-11-28T10:58:41.410 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-x1205.ko 2023-11-28T10:58:41.410 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi 2023-11-28T10:58:41.410 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-9xxx.ko 2023-11-28T10:58:41.410 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-sas.ko 2023-11-28T10:58:41.410 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-xxxx.ko 2023-11-28T10:58:41.410 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/BusLogic.ko 2023-11-28T10:58:41.410 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/a100u2w.ko 2023-11-28T10:58:41.410 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aacraid 2023-11-28T10:58:41.411 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aacraid/aacraid.ko 2023-11-28T10:58:41.411 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/advansys.ko 2023-11-28T10:58:41.411 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx 2023-11-28T10:58:41.411 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-11-28T10:58:41.411 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-11-28T10:58:41.411 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/am53c974.ko 2023-11-28T10:58:41.411 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/arcmsr 2023-11-28T10:58:41.411 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-11-28T10:58:41.411 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/atp870u.ko 2023-11-28T10:58:41.411 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/be2iscsi 2023-11-28T10:58:41.411 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-11-28T10:58:41.411 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bfa 2023-11-28T10:58:41.411 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bfa/bfa.ko 2023-11-28T10:58:41.412 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2fc 2023-11-28T10:58:41.412 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-11-28T10:58:41.412 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2i 2023-11-28T10:58:41.412 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-11-28T10:58:41.412 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ch.ko 2023-11-28T10:58:41.412 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/csiostor 2023-11-28T10:58:41.412 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/csiostor/csiostor.ko 2023-11-28T10:58:41.412 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi 2023-11-28T10:58:41.412 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb3i 2023-11-28T10:58:41.412 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-11-28T10:58:41.413 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb4i 2023-11-28T10:58:41.413 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-11-28T10:58:41.413 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-11-28T10:58:41.413 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/dc395x.ko 2023-11-28T10:58:41.413 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler 2023-11-28T10:58:41.413 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-11-28T10:58:41.413 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-11-28T10:58:41.413 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-11-28T10:58:41.413 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-11-28T10:58:41.413 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/dmx3191d.ko 2023-11-28T10:58:41.413 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esas2r 2023-11-28T10:58:41.413 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esas2r/esas2r.ko 2023-11-28T10:58:41.413 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esp_scsi.ko 2023-11-28T10:58:41.414 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe 2023-11-28T10:58:41.414 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/fcoe.ko 2023-11-28T10:58:41.414 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-11-28T10:58:41.414 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fnic 2023-11-28T10:58:41.414 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fnic/fnic.ko 2023-11-28T10:58:41.414 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hpsa.ko 2023-11-28T10:58:41.414 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hptiop.ko 2023-11-28T10:58:41.414 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hv_storvsc.ko 2023-11-28T10:58:41.414 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/initio.ko 2023-11-28T10:58:41.414 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ipr.ko 2023-11-28T10:58:41.414 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ips.ko 2023-11-28T10:58:41.415 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/isci 2023-11-28T10:58:41.415 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/isci/isci.ko 2023-11-28T10:58:41.415 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-11-28T10:58:41.415 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/iscsi_tcp.ko 2023-11-28T10:58:41.415 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libfc 2023-11-28T10:58:41.415 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libfc/libfc.ko 2023-11-28T10:58:41.415 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libiscsi.ko 2023-11-28T10:58:41.415 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libiscsi_tcp.ko 2023-11-28T10:58:41.415 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libsas 2023-11-28T10:58:41.415 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libsas/libsas.ko 2023-11-28T10:58:41.415 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/lpfc 2023-11-28T10:58:41.415 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/lpfc/lpfc.ko 2023-11-28T10:58:41.416 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid 2023-11-28T10:58:41.416 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid.ko 2023-11-28T10:58:41.416 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-11-28T10:58:41.416 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-11-28T10:58:41.416 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-11-28T10:58:41.416 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mpt3sas 2023-11-28T10:58:41.416 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-11-28T10:58:41.416 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvsas 2023-11-28T10:58:41.416 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvsas/mvsas.ko 2023-11-28T10:58:41.416 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvumi.ko 2023-11-28T10:58:41.416 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pm8001 2023-11-28T10:58:41.416 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-11-28T10:58:41.417 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pmcraid.ko 2023-11-28T10:58:41.417 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla1280.ko 2023-11-28T10:58:41.417 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx 2023-11-28T10:58:41.417 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-11-28T10:58:41.417 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-11-28T10:58:41.417 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla4xxx 2023-11-28T10:58:41.417 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-11-28T10:58:41.417 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/raid_class.ko 2023-11-28T10:58:41.417 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_debug.ko 2023-11-28T10:58:41.417 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_fc.ko 2023-11-28T10:58:41.417 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-11-28T10:58:41.417 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_sas.ko 2023-11-28T10:58:41.418 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_spi.ko 2023-11-28T10:58:41.418 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_srp.ko 2023-11-28T10:58:41.418 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ses.ko 2023-11-28T10:58:41.418 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/st.ko 2023-11-28T10:58:41.418 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/stex.ko 2023-11-28T10:58:41.418 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/sym53c8xx_2 2023-11-28T10:58:41.418 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-11-28T10:58:41.418 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/virtio_scsi.ko 2023-11-28T10:58:41.418 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/vmw_pvscsi.ko 2023-11-28T10:58:41.418 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/wd719x.ko 2023-11-28T10:58:41.418 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ssb 2023-11-28T10:58:41.419 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ssb/ssb.ko 2023-11-28T10:58:41.419 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target 2023-11-28T10:58:41.419 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/iscsi 2023-11-28T10:58:41.419 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-11-28T10:58:41.419 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/loopback 2023-11-28T10:58:41.419 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/loopback/tcm_loop.ko 2023-11-28T10:58:41.419 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_file.ko 2023-11-28T10:58:41.419 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_iblock.ko 2023-11-28T10:58:41.419 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_mod.ko 2023-11-28T10:58:41.419 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_pscsi.ko 2023-11-28T10:58:41.419 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_user.ko 2023-11-28T10:58:41.421 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/tcm_fc 2023-11-28T10:58:41.421 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-11-28T10:58:41.421 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal 2023-11-28T10:58:41.421 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel 2023-11-28T10:58:41.421 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal 2023-11-28T10:58:41.421 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-11-28T10:58:41.421 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-11-28T10:58:41.421 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-11-28T10:58:41.421 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-11-28T10:58:41.422 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-11-28T10:58:41.422 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-11-28T10:58:41.422 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-11-28T10:58:41.422 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-11-28T10:58:41.422 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-11-28T10:58:41.422 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-11-28T10:58:41.422 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-11-28T10:58:41.422 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-11-28T10:58:41.422 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-11-28T10:58:41.422 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-11-28T10:58:41.422 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-11-28T10:58:41.422 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty 2023-11-28T10:58:41.423 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ipwireless 2023-11-28T10:58:41.423 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-11-28T10:58:41.423 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/n_gsm.ko 2023-11-28T10:58:41.423 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/n_hdlc.ko 2023-11-28T10:58:41.423 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/nozomi.ko 2023-11-28T10:58:41.423 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial 2023-11-28T10:58:41.423 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250 2023-11-28T10:58:41.423 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-11-28T10:58:41.423 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/arc_uart.ko 2023-11-28T10:58:41.423 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/jsm 2023-11-28T10:58:41.423 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/jsm/jsm.ko 2023-11-28T10:58:41.423 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs 2023-11-28T10:58:41.424 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/core 2023-11-28T10:58:41.424 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/core/ufshcd-core.ko 2023-11-28T10:58:41.424 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host 2023-11-28T10:58:41.424 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-11-28T10:58:41.424 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio 2023-11-28T10:58:41.424 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio.ko 2023-11-28T10:58:41.424 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_aec.ko 2023-11-28T10:58:41.424 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_cif.ko 2023-11-28T10:58:41.424 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_pci_generic.ko 2023-11-28T10:58:41.424 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_sercos3.ko 2023-11-28T10:58:41.424 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb 2023-11-28T10:58:41.424 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class 2023-11-28T10:58:41.424 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/cdc-acm.ko 2023-11-28T10:58:41.425 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/cdc-wdm.ko 2023-11-28T10:58:41.425 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/usblp.ko 2023-11-28T10:58:41.425 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/usbtmc.ko 2023-11-28T10:58:41.425 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host 2023-11-28T10:58:41.425 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/sl811-hcd.ko 2023-11-28T10:58:41.425 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image 2023-11-28T10:58:41.425 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/mdc800.ko 2023-11-28T10:58:41.425 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/microtek.ko 2023-11-28T10:58:41.425 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc 2023-11-28T10:58:41.425 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/adutux.ko 2023-11-28T10:58:41.425 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/appledisplay.ko 2023-11-28T10:58:41.425 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/emi26.ko 2023-11-28T10:58:41.425 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/emi62.ko 2023-11-28T10:58:41.426 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ezusb.ko 2023-11-28T10:58:41.426 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/idmouse.ko 2023-11-28T10:58:41.426 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/iowarrior.ko 2023-11-28T10:58:41.426 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/isight_firmware.ko 2023-11-28T10:58:41.426 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ldusb.ko 2023-11-28T10:58:41.426 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/legousbtower.ko 2023-11-28T10:58:41.426 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/sisusbvga 2023-11-28T10:58:41.426 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-11-28T10:58:41.426 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/trancevibrator.ko 2023-11-28T10:58:41.426 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usb3503.ko 2023-11-28T10:58:41.426 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usblcd.ko 2023-11-28T10:58:41.426 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usbsevseg.ko 2023-11-28T10:58:41.427 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/uss720.ko 2023-11-28T10:58:41.427 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/yurex.ko 2023-11-28T10:58:41.427 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy 2023-11-28T10:58:41.427 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy/phy-generic.ko 2023-11-28T10:58:41.427 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial 2023-11-28T10:58:41.427 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/aircable.ko 2023-11-28T10:58:41.427 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ark3116.ko 2023-11-28T10:58:41.427 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/belkin_sa.ko 2023-11-28T10:58:41.427 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ch341.ko 2023-11-28T10:58:41.427 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cp210x.ko 2023-11-28T10:58:41.427 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cyberjack.ko 2023-11-28T10:58:41.427 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cypress_m8.ko 2023-11-28T10:58:41.427 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/digi_acceleport.ko 2023-11-28T10:58:41.428 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/empeg.ko 2023-11-28T10:58:41.428 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ftdi_sio.ko 2023-11-28T10:58:41.428 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/garmin_gps.ko 2023-11-28T10:58:41.428 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/io_edgeport.ko 2023-11-28T10:58:41.428 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/io_ti.ko 2023-11-28T10:58:41.428 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ipaq.ko 2023-11-28T10:58:41.428 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ipw.ko 2023-11-28T10:58:41.428 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ir-usb.ko 2023-11-28T10:58:41.428 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-11-28T10:58:41.428 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/keyspan.ko 2023-11-28T10:58:41.428 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/keyspan_pda.ko 2023-11-28T10:58:41.428 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/kl5kusb105.ko 2023-11-28T10:58:41.429 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/kobil_sct.ko 2023-11-28T10:58:41.429 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mct_u232.ko 2023-11-28T10:58:41.429 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mos7720.ko 2023-11-28T10:58:41.429 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mos7840.ko 2023-11-28T10:58:41.429 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/navman.ko 2023-11-28T10:58:41.429 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/omninet.ko 2023-11-28T10:58:41.429 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/opticon.ko 2023-11-28T10:58:41.429 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/option.ko 2023-11-28T10:58:41.429 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/oti6858.ko 2023-11-28T10:58:41.429 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/pl2303.ko 2023-11-28T10:58:41.429 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/qcaux.ko 2023-11-28T10:58:41.429 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/qcserial.ko 2023-11-28T10:58:41.430 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/quatech2.ko 2023-11-28T10:58:41.430 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/safe_serial.ko 2023-11-28T10:58:41.430 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/sierra.ko 2023-11-28T10:58:41.430 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/spcp8x5.ko 2023-11-28T10:58:41.430 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ssu100.ko 2023-11-28T10:58:41.430 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/symbolserial.ko 2023-11-28T10:58:41.430 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-11-28T10:58:41.430 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-11-28T10:58:41.430 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb_debug.ko 2023-11-28T10:58:41.430 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb_wwan.ko 2023-11-28T10:58:41.430 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/visor.ko 2023-11-28T10:58:41.430 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/whiteheat.ko 2023-11-28T10:58:41.431 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/xsens_mt.ko 2023-11-28T10:58:41.431 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage 2023-11-28T10:58:41.431 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/uas.ko 2023-11-28T10:58:41.431 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-alauda.ko 2023-11-28T10:58:41.431 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-cypress.ko 2023-11-28T10:58:41.431 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-datafab.ko 2023-11-28T10:58:41.431 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-11-28T10:58:41.431 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-freecom.ko 2023-11-28T10:58:41.431 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-isd200.ko 2023-11-28T10:58:41.431 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-11-28T10:58:41.431 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-karma.ko 2023-11-28T10:58:41.431 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-onetouch.ko 2023-11-28T10:58:41.432 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-realtek.ko 2023-11-28T10:58:41.432 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-sddr09.ko 2023-11-28T10:58:41.432 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-sddr55.ko 2023-11-28T10:58:41.432 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-usbat.ko 2023-11-28T10:58:41.432 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/usb-storage.ko 2023-11-28T10:58:41.432 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip 2023-11-28T10:58:41.432 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/usbip-core.ko 2023-11-28T10:58:41.432 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/usbip-host.ko 2023-11-28T10:58:41.432 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-11-28T10:58:41.432 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio 2023-11-28T10:58:41.432 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci 2023-11-28T10:58:41.432 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-11-28T10:58:41.432 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/vfio-pci.ko 2023-11-28T10:58:41.433 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/vfio.ko 2023-11-28T10:58:41.433 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-11-28T10:58:41.433 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost 2023-11-28T10:58:41.433 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost.ko 2023-11-28T10:58:41.433 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_iotlb.ko 2023-11-28T10:58:41.433 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_net.ko 2023-11-28T10:58:41.433 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_scsi.ko 2023-11-28T10:58:41.433 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video 2023-11-28T10:58:41.433 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight 2023-11-28T10:58:41.433 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/apple_bl.ko 2023-11-28T10:58:41.433 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/lcd.ko 2023-11-28T10:58:41.433 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/platform_lcd.ko 2023-11-28T10:58:41.434 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio 2023-11-28T10:58:41.434 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio.ko 2023-11-28T10:58:41.434 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_balloon.ko 2023-11-28T10:58:41.434 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_mmio.ko 2023-11-28T10:58:41.434 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci.ko 2023-11-28T10:58:41.434 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-11-28T10:58:41.434 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-11-28T10:58:41.434 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_ring.ko 2023-11-28T10:58:41.434 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1 2023-11-28T10:58:41.434 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters 2023-11-28T10:58:41.434 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ds2482.ko 2023-11-28T10:58:41.434 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ds2490.ko 2023-11-28T10:58:41.435 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves 2023-11-28T10:58:41.435 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-11-28T10:58:41.435 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-11-28T10:58:41.435 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-11-28T10:58:41.435 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-11-28T10:58:41.435 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-11-28T10:58:41.435 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-11-28T10:58:41.435 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-11-28T10:58:41.435 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-11-28T10:58:41.435 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-11-28T10:58:41.435 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_smem.ko 2023-11-28T10:58:41.435 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_therm.ko 2023-11-28T10:58:41.435 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/wire.ko 2023-11-28T10:58:41.436 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog 2023-11-28T10:58:41.436 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/alim1535_wdt.ko 2023-11-28T10:58:41.436 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/alim7101_wdt.ko 2023-11-28T10:58:41.436 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/f71808e_wdt.ko 2023-11-28T10:58:41.436 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/hpwdt.ko 2023-11-28T10:58:41.436 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/i6300esb.ko 2023-11-28T10:58:41.436 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-11-28T10:58:41.436 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/iTCO_wdt.ko 2023-11-28T10:58:41.436 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ib700wdt.ko 2023-11-28T10:58:41.436 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ibmasr.ko 2023-11-28T10:58:41.436 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-11-28T10:58:41.436 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/it8712f_wdt.ko 2023-11-28T10:58:41.436 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/it87_wdt.ko 2023-11-28T10:58:41.437 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/machzwd.ko 2023-11-28T10:58:41.437 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/nv_tco.ko 2023-11-28T10:58:41.437 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pcwd_pci.ko 2023-11-28T10:58:41.437 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pcwd_usb.ko 2023-11-28T10:58:41.437 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-11-28T10:58:41.437 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sch311x_wdt.ko 2023-11-28T10:58:41.437 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/softdog.ko 2023-11-28T10:58:41.437 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sp5100_tco.ko 2023-11-28T10:58:41.437 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/via_wdt.ko 2023-11-28T10:58:41.437 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-11-28T10:58:41.437 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83877f_wdt.ko 2023-11-28T10:58:41.438 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83977f_wdt.ko 2023-11-28T10:58:41.438 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/wdt_pci.ko 2023-11-28T10:58:41.438 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/xen_wdt.ko 2023-11-28T10:58:41.438 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen 2023-11-28T10:58:41.438 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-acpi-processor.ko 2023-11-28T10:58:41.438 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-evtchn.ko 2023-11-28T10:58:41.438 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-gntalloc.ko 2023-11-28T10:58:41.438 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-gntdev.ko 2023-11-28T10:58:41.438 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-pciback 2023-11-28T10:58:41.438 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-11-28T10:58:41.438 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-privcmd.ko 2023-11-28T10:58:41.438 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xenfs 2023-11-28T10:58:41.439 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xenfs/xenfs.ko 2023-11-28T10:58:41.439 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs 2023-11-28T10:58:41.439 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/9p 2023-11-28T10:58:41.439 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/9p/9p.ko 2023-11-28T10:58:41.439 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/affs 2023-11-28T10:58:41.439 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/affs/affs.ko 2023-11-28T10:58:41.439 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/befs 2023-11-28T10:58:41.439 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/befs/befs.ko 2023-11-28T10:58:41.439 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/binfmt_misc.ko 2023-11-28T10:58:41.439 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/btrfs 2023-11-28T10:58:41.439 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/btrfs/btrfs.ko 2023-11-28T10:58:41.439 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cachefiles 2023-11-28T10:58:41.439 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cachefiles/cachefiles.ko 2023-11-28T10:58:41.440 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ceph 2023-11-28T10:58:41.440 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ceph/ceph.ko 2023-11-28T10:58:41.440 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/coda 2023-11-28T10:58:41.440 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/coda/coda.ko 2023-11-28T10:58:41.440 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cramfs 2023-11-28T10:58:41.440 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cramfs/cramfs.ko 2023-11-28T10:58:41.440 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/dlm 2023-11-28T10:58:41.440 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/dlm/dlm.ko 2023-11-28T10:58:41.440 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ecryptfs 2023-11-28T10:58:41.440 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ecryptfs/ecryptfs.ko 2023-11-28T10:58:41.440 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/f2fs 2023-11-28T10:58:41.440 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/f2fs/f2fs.ko 2023-11-28T10:58:41.441 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat 2023-11-28T10:58:41.441 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/fat.ko 2023-11-28T10:58:41.441 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/msdos.ko 2023-11-28T10:58:41.441 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/vfat.ko 2023-11-28T10:58:41.441 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fscache 2023-11-28T10:58:41.441 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fscache/fscache.ko 2023-11-28T10:58:41.441 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse 2023-11-28T10:58:41.441 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse/cuse.ko 2023-11-28T10:58:41.441 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse/fuse.ko 2023-11-28T10:58:41.441 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/gfs2 2023-11-28T10:58:41.441 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/gfs2/gfs2.ko 2023-11-28T10:58:41.441 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfs 2023-11-28T10:58:41.442 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfs/hfs.ko 2023-11-28T10:58:41.442 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfsplus 2023-11-28T10:58:41.442 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfsplus/hfsplus.ko 2023-11-28T10:58:41.442 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/isofs 2023-11-28T10:58:41.442 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/isofs/isofs.ko 2023-11-28T10:58:41.442 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/jfs 2023-11-28T10:58:41.442 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/jfs/jfs.ko 2023-11-28T10:58:41.442 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/lockd 2023-11-28T10:58:41.442 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/lockd/lockd.ko 2023-11-28T10:58:41.442 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/minix 2023-11-28T10:58:41.442 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/minix/minix.ko 2023-11-28T10:58:41.442 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/netfs 2023-11-28T10:58:41.443 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/netfs/netfs.ko 2023-11-28T10:58:41.443 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs 2023-11-28T10:58:41.443 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/blocklayout 2023-11-28T10:58:41.443 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-11-28T10:58:41.443 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/filelayout 2023-11-28T10:58:41.443 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-11-28T10:58:41.443 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/flexfilelayout 2023-11-28T10:58:41.443 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-11-28T10:58:41.443 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfs.ko 2023-11-28T10:58:41.443 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv3.ko 2023-11-28T10:58:41.443 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv4.ko 2023-11-28T10:58:41.443 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common 2023-11-28T10:58:41.444 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/grace.ko 2023-11-28T10:58:41.444 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/nfs_acl.ko 2023-11-28T10:58:41.444 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfsd 2023-11-28T10:58:41.444 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfsd/nfsd.ko 2023-11-28T10:58:41.444 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nilfs2 2023-11-28T10:58:41.444 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nilfs2/nilfs2.ko 2023-11-28T10:58:41.444 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls 2023-11-28T10:58:41.444 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-celtic.ko 2023-11-28T10:58:41.444 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-centeuro.ko 2023-11-28T10:58:41.444 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-croatian.ko 2023-11-28T10:58:41.444 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-cyrillic.ko 2023-11-28T10:58:41.444 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-gaelic.ko 2023-11-28T10:58:41.445 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-greek.ko 2023-11-28T10:58:41.445 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-iceland.ko 2023-11-28T10:58:41.445 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-inuit.ko 2023-11-28T10:58:41.445 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-roman.ko 2023-11-28T10:58:41.445 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-romanian.ko 2023-11-28T10:58:41.445 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-turkish.ko 2023-11-28T10:58:41.445 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1250.ko 2023-11-28T10:58:41.445 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1251.ko 2023-11-28T10:58:41.445 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1255.ko 2023-11-28T10:58:41.445 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp737.ko 2023-11-28T10:58:41.445 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp775.ko 2023-11-28T10:58:41.445 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp850.ko 2023-11-28T10:58:41.445 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp852.ko 2023-11-28T10:58:41.446 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp855.ko 2023-11-28T10:58:41.446 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp857.ko 2023-11-28T10:58:41.446 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp860.ko 2023-11-28T10:58:41.446 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp861.ko 2023-11-28T10:58:41.446 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp862.ko 2023-11-28T10:58:41.446 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp863.ko 2023-11-28T10:58:41.446 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp864.ko 2023-11-28T10:58:41.446 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp865.ko 2023-11-28T10:58:41.446 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp866.ko 2023-11-28T10:58:41.446 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp869.ko 2023-11-28T10:58:41.446 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp874.ko 2023-11-28T10:58:41.446 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp932.ko 2023-11-28T10:58:41.447 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp936.ko 2023-11-28T10:58:41.447 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp949.ko 2023-11-28T10:58:41.447 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp950.ko 2023-11-28T10:58:41.447 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_euc-jp.ko 2023-11-28T10:58:41.447 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-1.ko 2023-11-28T10:58:41.447 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-13.ko 2023-11-28T10:58:41.447 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-14.ko 2023-11-28T10:58:41.447 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-15.ko 2023-11-28T10:58:41.447 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-2.ko 2023-11-28T10:58:41.447 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-3.ko 2023-11-28T10:58:41.447 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-4.ko 2023-11-28T10:58:41.447 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-5.ko 2023-11-28T10:58:41.447 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-6.ko 2023-11-28T10:58:41.448 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-7.ko 2023-11-28T10:58:41.448 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-9.ko 2023-11-28T10:58:41.448 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-r.ko 2023-11-28T10:58:41.448 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-ru.ko 2023-11-28T10:58:41.448 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-u.ko 2023-11-28T10:58:41.448 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_ucs2_utils.ko 2023-11-28T10:58:41.448 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_utf8.ko 2023-11-28T10:58:41.448 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2 2023-11-28T10:58:41.448 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/cluster 2023-11-28T10:58:41.448 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-11-28T10:58:41.448 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlm 2023-11-28T10:58:41.448 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-11-28T10:58:41.449 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlmfs 2023-11-28T10:58:41.449 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-11-28T10:58:41.449 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2.ko 2023-11-28T10:58:41.449 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-11-28T10:58:41.449 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-11-28T10:58:41.449 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-11-28T10:58:41.449 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/overlayfs 2023-11-28T10:58:41.449 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/overlayfs/overlay.ko 2023-11-28T10:58:41.449 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/pstore 2023-11-28T10:58:41.449 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/pstore/ramoops.ko 2023-11-28T10:58:41.449 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/reiserfs 2023-11-28T10:58:41.449 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/reiserfs/reiserfs.ko 2023-11-28T10:58:41.450 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/romfs 2023-11-28T10:58:41.450 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/romfs/romfs.ko 2023-11-28T10:58:41.450 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb 2023-11-28T10:58:41.450 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/client 2023-11-28T10:58:41.450 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/client/cifs.ko 2023-11-28T10:58:41.450 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common 2023-11-28T10:58:41.450 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/cifs_arc4.ko 2023-11-28T10:58:41.450 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/cifs_md4.ko 2023-11-28T10:58:41.450 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/squashfs 2023-11-28T10:58:41.450 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/squashfs/squashfs.ko 2023-11-28T10:58:41.450 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/sysv 2023-11-28T10:58:41.450 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/sysv/sysv.ko 2023-11-28T10:58:41.451 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/udf 2023-11-28T10:58:41.451 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/udf/udf.ko 2023-11-28T10:58:41.451 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ufs 2023-11-28T10:58:41.451 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ufs/ufs.ko 2023-11-28T10:58:41.451 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/xfs 2023-11-28T10:58:41.451 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/xfs/xfs.ko 2023-11-28T10:58:41.451 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel 2023-11-28T10:58:41.451 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/rcu 2023-11-28T10:58:41.451 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/rcu/rcutorture.ko 2023-11-28T10:58:41.451 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/torture.ko 2023-11-28T10:58:41.451 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/trace 2023-11-28T10:58:41.451 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-11-28T10:58:41.451 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib 2023-11-28T10:58:41.452 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/asn1_encoder.ko 2023-11-28T10:58:41.452 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc-itu-t.ko 2023-11-28T10:58:41.452 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc8.ko 2023-11-28T10:58:41.452 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto 2023-11-28T10:58:41.452 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/libarc4.ko 2023-11-28T10:58:41.452 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/libdes.ko 2023-11-28T10:58:41.452 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/libcrc32c.ko 2023-11-28T10:58:41.452 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lru_cache.ko 2023-11-28T10:58:41.452 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4 2023-11-28T10:58:41.452 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/lz4_compress.ko 2023-11-28T10:58:41.452 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/lz4hc_compress.ko 2023-11-28T10:58:41.452 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/math 2023-11-28T10:58:41.453 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/math/cordic.ko 2023-11-28T10:58:41.453 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/raid6 2023-11-28T10:58:41.453 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/raid6/raid6_pq.ko 2023-11-28T10:58:41.453 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/reed_solomon 2023-11-28T10:58:41.453 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/reed_solomon/reed_solomon.ko 2023-11-28T10:58:41.453 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_bm.ko 2023-11-28T10:58:41.453 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_fsm.ko 2023-11-28T10:58:41.453 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_kmp.ko 2023-11-28T10:58:41.453 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/zstd 2023-11-28T10:58:41.453 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/zstd/zstd_compress.ko 2023-11-28T10:58:41.453 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/mm 2023-11-28T10:58:41.453 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/mm/hwpoison-inject.ko 2023-11-28T10:58:41.454 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net 2023-11-28T10:58:41.454 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan 2023-11-28T10:58:41.454 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/6lowpan.ko 2023-11-28T10:58:41.454 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_dest.ko 2023-11-28T10:58:41.454 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_fragment.ko 2023-11-28T10:58:41.454 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_hop.ko 2023-11-28T10:58:41.454 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_ipv6.ko 2023-11-28T10:58:41.454 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_mobility.ko 2023-11-28T10:58:41.454 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_routing.ko 2023-11-28T10:58:41.454 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_udp.ko 2023-11-28T10:58:41.454 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802 2023-11-28T10:58:41.454 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/garp.ko 2023-11-28T10:58:41.455 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/mrp.ko 2023-11-28T10:58:41.455 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/p8022.ko 2023-11-28T10:58:41.455 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/psnap.ko 2023-11-28T10:58:41.455 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/stp.ko 2023-11-28T10:58:41.455 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/8021q 2023-11-28T10:58:41.455 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/8021q/8021q.ko 2023-11-28T10:58:41.455 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p 2023-11-28T10:58:41.455 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet.ko 2023-11-28T10:58:41.455 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_fd.ko 2023-11-28T10:58:41.455 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_rdma.ko 2023-11-28T10:58:41.455 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_virtio.ko 2023-11-28T10:58:41.455 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/batman-adv 2023-11-28T10:58:41.456 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/batman-adv/batman-adv.ko 2023-11-28T10:58:41.456 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge 2023-11-28T10:58:41.456 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/br_netfilter.ko 2023-11-28T10:58:41.456 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/bridge.ko 2023-11-28T10:58:41.456 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter 2023-11-28T10:58:41.456 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-11-28T10:58:41.456 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_among.ko 2023-11-28T10:58:41.456 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_arp.ko 2023-11-28T10:58:41.456 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-11-28T10:58:41.456 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-11-28T10:58:41.456 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_ip.ko 2023-11-28T10:58:41.456 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-11-28T10:58:41.456 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_limit.ko 2023-11-28T10:58:41.457 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_log.ko 2023-11-28T10:58:41.457 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_mark.ko 2023-11-28T10:58:41.457 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-11-28T10:58:41.457 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-11-28T10:58:41.457 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-11-28T10:58:41.457 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-11-28T10:58:41.457 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_snat.ko 2023-11-28T10:58:41.457 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_stp.ko 2023-11-28T10:58:41.457 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-11-28T10:58:41.457 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-11-28T10:58:41.457 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-11-28T10:58:41.457 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-11-28T10:58:41.458 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtables.ko 2023-11-28T10:58:41.458 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-11-28T10:58:41.458 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-11-28T10:58:41.458 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-11-28T10:58:41.458 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ceph 2023-11-28T10:58:41.458 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ceph/libceph.ko 2023-11-28T10:58:41.458 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core 2023-11-28T10:58:41.458 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core/failover.ko 2023-11-28T10:58:41.458 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core/pktgen.ko 2023-11-28T10:58:41.458 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp 2023-11-28T10:58:41.458 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp.ko 2023-11-28T10:58:41.458 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_diag.ko 2023-11-28T10:58:41.458 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_ipv4.ko 2023-11-28T10:58:41.459 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_ipv6.ko 2023-11-28T10:58:41.459 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dns_resolver 2023-11-28T10:58:41.459 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dns_resolver/dns_resolver.ko 2023-11-28T10:58:41.459 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa 2023-11-28T10:58:41.459 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/dsa_core.ko 2023-11-28T10:58:41.459 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_brcm.ko 2023-11-28T10:58:41.459 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_dsa.ko 2023-11-28T10:58:41.459 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_none.ko 2023-11-28T10:58:41.459 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_trailer.ko 2023-11-28T10:58:41.459 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154 2023-11-28T10:58:41.459 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/6lowpan 2023-11-28T10:58:41.459 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-11-28T10:58:41.460 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ieee802154.ko 2023-11-28T10:58:41.460 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ieee802154_socket.ko 2023-11-28T10:58:41.460 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4 2023-11-28T10:58:41.460 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ah4.ko 2023-11-28T10:58:41.460 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/esp4.ko 2023-11-28T10:58:41.460 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/fou.ko 2023-11-28T10:58:41.460 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/gre.ko 2023-11-28T10:58:41.460 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/inet_diag.ko 2023-11-28T10:58:41.460 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_gre.ko 2023-11-28T10:58:41.460 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_tunnel.ko 2023-11-28T10:58:41.460 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_vti.ko 2023-11-28T10:58:41.460 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ipcomp.ko 2023-11-28T10:58:41.461 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ipip.ko 2023-11-28T10:58:41.461 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter 2023-11-28T10:58:41.461 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arp_tables.ko 2023-11-28T10:58:41.461 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-11-28T10:58:41.461 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-11-28T10:58:41.461 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-11-28T10:58:41.461 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-11-28T10:58:41.461 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-11-28T10:58:41.461 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-11-28T10:58:41.461 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-11-28T10:58:41.461 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-11-28T10:58:41.461 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-11-28T10:58:41.462 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_security.ko 2023-11-28T10:58:41.462 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-11-28T10:58:41.462 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-11-28T10:58:41.462 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-11-28T10:58:41.462 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-11-28T10:58:41.462 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-11-28T10:58:41.462 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-11-28T10:58:41.462 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-11-28T10:58:41.462 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-11-28T10:58:41.462 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-11-28T10:58:41.462 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-11-28T10:58:41.462 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_bic.ko 2023-11-28T10:58:41.463 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_dctcp.ko 2023-11-28T10:58:41.463 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_diag.ko 2023-11-28T10:58:41.463 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_highspeed.ko 2023-11-28T10:58:41.463 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_htcp.ko 2023-11-28T10:58:41.463 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_hybla.ko 2023-11-28T10:58:41.463 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_illinois.ko 2023-11-28T10:58:41.463 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_lp.ko 2023-11-28T10:58:41.463 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_scalable.ko 2023-11-28T10:58:41.463 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_vegas.ko 2023-11-28T10:58:41.463 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_veno.ko 2023-11-28T10:58:41.463 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_westwood.ko 2023-11-28T10:58:41.463 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_yeah.ko 2023-11-28T10:58:41.463 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tunnel4.ko 2023-11-28T10:58:41.464 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/udp_diag.ko 2023-11-28T10:58:41.464 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/udp_tunnel.ko 2023-11-28T10:58:41.464 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/xfrm4_tunnel.ko 2023-11-28T10:58:41.464 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6 2023-11-28T10:58:41.464 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ah6.ko 2023-11-28T10:58:41.464 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/esp6.ko 2023-11-28T10:58:41.464 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/fou6.ko 2023-11-28T10:58:41.464 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_tunnel.ko 2023-11-28T10:58:41.464 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-11-28T10:58:41.464 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_vti.ko 2023-11-28T10:58:41.464 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ipcomp6.ko 2023-11-28T10:58:41.464 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter 2023-11-28T10:58:41.465 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-11-28T10:58:41.465 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-11-28T10:58:41.465 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-11-28T10:58:41.465 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-11-28T10:58:41.465 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-11-28T10:58:41.465 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-11-28T10:58:41.465 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-11-28T10:58:41.465 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-11-28T10:58:41.465 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-11-28T10:58:41.465 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-11-28T10:58:41.465 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-11-28T10:58:41.465 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-11-28T10:58:41.466 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-11-28T10:58:41.466 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-11-28T10:58:41.466 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-11-28T10:58:41.466 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-11-28T10:58:41.466 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-11-28T10:58:41.466 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-11-28T10:58:41.466 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-11-28T10:58:41.466 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-11-28T10:58:41.466 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-11-28T10:58:41.466 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-11-28T10:58:41.466 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-11-28T10:58:41.466 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-11-28T10:58:41.467 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/sit.ko 2023-11-28T10:58:41.467 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/tunnel6.ko 2023-11-28T10:58:41.467 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/xfrm6_tunnel.ko 2023-11-28T10:58:41.467 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/key 2023-11-28T10:58:41.467 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/key/af_key.ko 2023-11-28T10:58:41.467 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp 2023-11-28T10:58:41.467 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_core.ko 2023-11-28T10:58:41.467 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_debugfs.ko 2023-11-28T10:58:41.467 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_eth.ko 2023-11-28T10:58:41.467 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ip.ko 2023-11-28T10:58:41.467 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ip6.ko 2023-11-28T10:58:41.467 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_netlink.ko 2023-11-28T10:58:41.468 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ppp.ko 2023-11-28T10:58:41.468 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc 2023-11-28T10:58:41.468 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc/llc.ko 2023-11-28T10:58:41.468 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mac802154 2023-11-28T10:58:41.468 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mac802154/mac802154.ko 2023-11-28T10:58:41.468 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mpls 2023-11-28T10:58:41.468 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mpls/mpls_gso.ko 2023-11-28T10:58:41.468 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter 2023-11-28T10:58:41.468 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset 2023-11-28T10:58:41.468 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set.ko 2023-11-28T10:58:41.468 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-11-28T10:58:41.468 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-11-28T10:58:41.468 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-11-28T10:58:41.469 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-11-28T10:58:41.469 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-11-28T10:58:41.469 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-11-28T10:58:41.469 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-11-28T10:58:41.469 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-11-28T10:58:41.469 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-11-28T10:58:41.469 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-11-28T10:58:41.469 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-11-28T10:58:41.469 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-11-28T10:58:41.469 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-11-28T10:58:41.469 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-11-28T10:58:41.469 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-11-28T10:58:41.470 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs 2023-11-28T10:58:41.470 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs.ko 2023-11-28T10:58:41.470 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-11-28T10:58:41.470 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-11-28T10:58:41.470 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-11-28T10:58:41.470 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-11-28T10:58:41.470 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-11-28T10:58:41.470 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-11-28T10:58:41.470 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-11-28T10:58:41.470 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-11-28T10:58:41.470 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-11-28T10:58:41.470 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-11-28T10:58:41.470 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-11-28T10:58:41.471 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-11-28T10:58:41.471 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-11-28T10:58:41.471 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conncount.ko 2023-11-28T10:58:41.471 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack.ko 2023-11-28T10:58:41.471 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-11-28T10:58:41.471 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-11-28T10:58:41.471 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-11-28T10:58:41.471 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_h323.ko 2023-11-28T10:58:41.471 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_irc.ko 2023-11-28T10:58:41.471 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-11-28T10:58:41.471 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-11-28T10:58:41.471 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-11-28T10:58:41.471 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_sane.ko 2023-11-28T10:58:41.472 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_sip.ko 2023-11-28T10:58:41.472 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-11-28T10:58:41.472 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-11-28T10:58:41.472 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_dup_netdev.ko 2023-11-28T10:58:41.472 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_flow_table.ko 2023-11-28T10:58:41.472 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_flow_table_inet.ko 2023-11-28T10:58:41.472 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_log_syslog.ko 2023-11-28T10:58:41.472 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat.ko 2023-11-28T10:58:41.472 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_amanda.ko 2023-11-28T10:58:41.472 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_ftp.ko 2023-11-28T10:58:41.472 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_irc.ko 2023-11-28T10:58:41.472 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_sip.ko 2023-11-28T10:58:41.473 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_tftp.ko 2023-11-28T10:58:41.473 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_synproxy_core.ko 2023-11-28T10:58:41.473 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_tables.ko 2023-11-28T10:58:41.473 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink.ko 2023-11-28T10:58:41.473 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_acct.ko 2023-11-28T10:58:41.473 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_log.ko 2023-11-28T10:58:41.473 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_osf.ko 2023-11-28T10:58:41.473 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_queue.ko 2023-11-28T10:58:41.473 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_chain_nat.ko 2023-11-28T10:58:41.473 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_compat.ko 2023-11-28T10:58:41.473 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_ct.ko 2023-11-28T10:58:41.473 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_dup_netdev.ko 2023-11-28T10:58:41.473 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_fib.ko 2023-11-28T10:58:41.474 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_fwd_netdev.ko 2023-11-28T10:58:41.474 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_hash.ko 2023-11-28T10:58:41.474 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_limit.ko 2023-11-28T10:58:41.474 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_log.ko 2023-11-28T10:58:41.474 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_masq.ko 2023-11-28T10:58:41.474 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_nat.ko 2023-11-28T10:58:41.474 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_numgen.ko 2023-11-28T10:58:41.474 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_queue.ko 2023-11-28T10:58:41.474 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_quota.ko 2023-11-28T10:58:41.474 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_redir.ko 2023-11-28T10:58:41.474 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_reject.ko 2023-11-28T10:58:41.474 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_reject_inet.ko 2023-11-28T10:58:41.475 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_socket.ko 2023-11-28T10:58:41.475 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_tunnel.ko 2023-11-28T10:58:41.475 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_AUDIT.ko 2023-11-28T10:58:41.475 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CHECKSUM.ko 2023-11-28T10:58:41.475 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CLASSIFY.ko 2023-11-28T10:58:41.475 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-11-28T10:58:41.475 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CT.ko 2023-11-28T10:58:41.475 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_DSCP.ko 2023-11-28T10:58:41.475 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_HL.ko 2023-11-28T10:58:41.475 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_HMARK.ko 2023-11-28T10:58:41.475 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_IDLETIMER.ko 2023-11-28T10:58:41.475 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_LED.ko 2023-11-28T10:58:41.476 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_LOG.ko 2023-11-28T10:58:41.476 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_MASQUERADE.ko 2023-11-28T10:58:41.476 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NETMAP.ko 2023-11-28T10:58:41.476 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NFLOG.ko 2023-11-28T10:58:41.476 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NFQUEUE.ko 2023-11-28T10:58:41.476 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_RATEEST.ko 2023-11-28T10:58:41.476 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_REDIRECT.ko 2023-11-28T10:58:41.476 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_SECMARK.ko 2023-11-28T10:58:41.476 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TCPMSS.ko 2023-11-28T10:58:41.476 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-11-28T10:58:41.476 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TEE.ko 2023-11-28T10:58:41.476 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TPROXY.ko 2023-11-28T10:58:41.477 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TRACE.ko 2023-11-28T10:58:41.477 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_addrtype.ko 2023-11-28T10:58:41.477 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_bpf.ko 2023-11-28T10:58:41.477 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cgroup.ko 2023-11-28T10:58:41.477 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cluster.ko 2023-11-28T10:58:41.477 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_comment.ko 2023-11-28T10:58:41.477 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connbytes.ko 2023-11-28T10:58:41.477 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connlabel.ko 2023-11-28T10:58:41.477 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connlimit.ko 2023-11-28T10:58:41.477 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connmark.ko 2023-11-28T10:58:41.477 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_conntrack.ko 2023-11-28T10:58:41.477 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cpu.ko 2023-11-28T10:58:41.477 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_dccp.ko 2023-11-28T10:58:41.478 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_devgroup.ko 2023-11-28T10:58:41.478 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_dscp.ko 2023-11-28T10:58:41.478 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ecn.ko 2023-11-28T10:58:41.478 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_esp.ko 2023-11-28T10:58:41.478 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_hashlimit.ko 2023-11-28T10:58:41.478 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_helper.ko 2023-11-28T10:58:41.478 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_hl.ko 2023-11-28T10:58:41.478 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ipcomp.ko 2023-11-28T10:58:41.478 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_iprange.ko 2023-11-28T10:58:41.478 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ipvs.ko 2023-11-28T10:58:41.478 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_l2tp.ko 2023-11-28T10:58:41.478 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_length.ko 2023-11-28T10:58:41.479 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_limit.ko 2023-11-28T10:58:41.479 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_mac.ko 2023-11-28T10:58:41.479 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_mark.ko 2023-11-28T10:58:41.479 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_multiport.ko 2023-11-28T10:58:41.479 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_nat.ko 2023-11-28T10:58:41.479 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_nfacct.ko 2023-11-28T10:58:41.479 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_osf.ko 2023-11-28T10:58:41.479 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_owner.ko 2023-11-28T10:58:41.479 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_physdev.ko 2023-11-28T10:58:41.479 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_pkttype.ko 2023-11-28T10:58:41.479 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_policy.ko 2023-11-28T10:58:41.479 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_quota.ko 2023-11-28T10:58:41.480 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_rateest.ko 2023-11-28T10:58:41.480 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_realm.ko 2023-11-28T10:58:41.480 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_recent.ko 2023-11-28T10:58:41.480 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_sctp.ko 2023-11-28T10:58:41.480 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_set.ko 2023-11-28T10:58:41.480 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_socket.ko 2023-11-28T10:58:41.480 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_state.ko 2023-11-28T10:58:41.480 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_statistic.ko 2023-11-28T10:58:41.480 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_string.ko 2023-11-28T10:58:41.480 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_tcpmss.ko 2023-11-28T10:58:41.480 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_time.ko 2023-11-28T10:58:41.480 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_u32.ko 2023-11-28T10:58:41.481 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netlink 2023-11-28T10:58:41.481 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netlink/netlink_diag.ko 2023-11-28T10:58:41.481 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/nsh 2023-11-28T10:58:41.481 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/nsh/nsh.ko 2023-11-28T10:58:41.481 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch 2023-11-28T10:58:41.481 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/openvswitch.ko 2023-11-28T10:58:41.481 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-geneve.ko 2023-11-28T10:58:41.481 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-gre.ko 2023-11-28T10:58:41.481 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-vxlan.ko 2023-11-28T10:58:41.481 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/packet 2023-11-28T10:58:41.481 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/packet/af_packet_diag.ko 2023-11-28T10:58:41.481 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds 2023-11-28T10:58:41.481 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds.ko 2023-11-28T10:58:41.482 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds_rdma.ko 2023-11-28T10:58:41.482 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds_tcp.ko 2023-11-28T10:58:41.482 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched 2023-11-28T10:58:41.482 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_csum.ko 2023-11-28T10:58:41.482 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_gact.ko 2023-11-28T10:58:41.482 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_ipt.ko 2023-11-28T10:58:41.482 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_mirred.ko 2023-11-28T10:58:41.482 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_nat.ko 2023-11-28T10:58:41.482 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_pedit.ko 2023-11-28T10:58:41.482 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_police.ko 2023-11-28T10:58:41.482 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_simple.ko 2023-11-28T10:58:41.482 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_skbedit.ko 2023-11-28T10:58:41.483 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_vlan.ko 2023-11-28T10:58:41.483 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_basic.ko 2023-11-28T10:58:41.483 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_bpf.ko 2023-11-28T10:58:41.483 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_flow.ko 2023-11-28T10:58:41.483 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_fw.ko 2023-11-28T10:58:41.483 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_route.ko 2023-11-28T10:58:41.483 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_u32.ko 2023-11-28T10:58:41.483 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_cmp.ko 2023-11-28T10:58:41.483 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_ipset.ko 2023-11-28T10:58:41.483 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_meta.ko 2023-11-28T10:58:41.483 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_nbyte.ko 2023-11-28T10:58:41.483 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_text.ko 2023-11-28T10:58:41.484 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_u32.ko 2023-11-28T10:58:41.484 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_choke.ko 2023-11-28T10:58:41.484 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_codel.ko 2023-11-28T10:58:41.484 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_drr.ko 2023-11-28T10:58:41.484 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_fq.ko 2023-11-28T10:58:41.484 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_gred.ko 2023-11-28T10:58:41.484 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_hfsc.ko 2023-11-28T10:58:41.484 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_hhf.ko 2023-11-28T10:58:41.484 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_htb.ko 2023-11-28T10:58:41.484 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_ingress.ko 2023-11-28T10:58:41.484 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_mqprio.ko 2023-11-28T10:58:41.484 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_mqprio_lib.ko 2023-11-28T10:58:41.484 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_multiq.ko 2023-11-28T10:58:41.485 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_netem.ko 2023-11-28T10:58:41.485 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_pie.ko 2023-11-28T10:58:41.485 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_plug.ko 2023-11-28T10:58:41.485 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_prio.ko 2023-11-28T10:58:41.485 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_qfq.ko 2023-11-28T10:58:41.485 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_red.ko 2023-11-28T10:58:41.485 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_sfb.ko 2023-11-28T10:58:41.485 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_sfq.ko 2023-11-28T10:58:41.485 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_tbf.ko 2023-11-28T10:58:41.485 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_teql.ko 2023-11-28T10:58:41.485 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp 2023-11-28T10:58:41.485 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/sctp.ko 2023-11-28T10:58:41.486 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/sctp_diag.ko 2023-11-28T10:58:41.486 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc 2023-11-28T10:58:41.486 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss 2023-11-28T10:58:41.486 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-11-28T10:58:41.486 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-11-28T10:58:41.486 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/sunrpc.ko 2023-11-28T10:58:41.486 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/xprtrdma 2023-11-28T10:58:41.486 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-11-28T10:58:41.486 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc 2023-11-28T10:58:41.486 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/diag.ko 2023-11-28T10:58:41.486 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/tipc.ko 2023-11-28T10:58:41.486 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/unix 2023-11-28T10:58:41.487 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/unix/unix_diag.ko 2023-11-28T10:58:41.487 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock 2023-11-28T10:58:41.487 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-11-28T10:58:41.487 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-11-28T10:58:41.487 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock.ko 2023-11-28T10:58:41.487 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock_diag.ko 2023-11-28T10:58:41.487 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock_loopback.ko 2023-11-28T10:58:41.487 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm 2023-11-28T10:58:41.487 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm/xfrm_ipcomp.ko 2023-11-28T10:58:41.487 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security 2023-11-28T10:58:41.487 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys 2023-11-28T10:58:41.487 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/encrypted-keys 2023-11-28T10:58:41.487 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-11-28T10:58:41.488 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/trusted-keys 2023-11-28T10:58:41.488 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/trusted-keys/trusted.ko 2023-11-28T10:58:41.488 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt 2023-11-28T10:58:41.488 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt/lib 2023-11-28T10:58:41.488 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt/lib/irqbypass.ko 2023-11-28T10:58:41.488 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.alias 2023-11-28T10:58:41.488 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.alias.bin 2023-11-28T10:58:41.488 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin 2023-11-28T10:58:41.488 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin.bin 2023-11-28T10:58:41.488 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin.modinfo 2023-11-28T10:58:41.488 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.dep 2023-11-28T10:58:41.488 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.dep.bin 2023-11-28T10:58:41.488 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.devname 2023-11-28T10:58:41.489 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.order 2023-11-28T10:58:41.489 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.softdep 2023-11-28T10:58:41.489 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.symbols 2023-11-28T10:58:41.489 INFO:teuthology.orchestra.run.smithi169.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.symbols.bin 2023-11-28T10:58:41.489 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-g8247bbb5b470 2023-11-28T10:58:41.490 DEBUG:teuthology.orchestra.run.smithi169:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-11-28T10:58:41.518 INFO:teuthology.orchestra.run.smithi169.stdout:postinstall scriptlet (using /bin/sh): 2023-11-28T10:58:41.518 INFO:teuthology.orchestra.run.smithi169.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-g8247bbb5b470 -a -r /boot/System.map-6.6.0-g8247bbb5b470 ]; then 2023-11-28T10:58:41.518 INFO:teuthology.orchestra.run.smithi169.stdout:cp /boot/vmlinuz-6.6.0-g8247bbb5b470 /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm 2023-11-28T10:58:41.519 INFO:teuthology.orchestra.run.smithi169.stdout:cp /boot/System.map-6.6.0-g8247bbb5b470 /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-28T10:58:41.519 INFO:teuthology.orchestra.run.smithi169.stdout:rm -f /boot/vmlinuz-6.6.0-g8247bbb5b470 /boot/System.map-6.6.0-g8247bbb5b470 2023-11-28T10:58:41.519 INFO:teuthology.orchestra.run.smithi169.stdout:/sbin/installkernel 6.6.0-g8247bbb5b470 /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-28T10:58:41.519 INFO:teuthology.orchestra.run.smithi169.stdout:rm -f /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-28T10:58:41.519 INFO:teuthology.orchestra.run.smithi169.stdout:fi 2023-11-28T10:58:41.519 INFO:teuthology.orchestra.run.smithi169.stdout:preuninstall scriptlet (using /bin/sh): 2023-11-28T10:58:41.519 INFO:teuthology.orchestra.run.smithi169.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-11-28T10:58:41.519 INFO:teuthology.orchestra.run.smithi169.stdout:new-kernel-pkg --remove 6.6.0-g8247bbb5b470 --rminitrd --initrdfile=/boot/initramfs-6.6.0-g8247bbb5b470.img 2023-11-28T10:58:41.519 INFO:teuthology.orchestra.run.smithi169.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-11-28T10:58:41.519 INFO:teuthology.orchestra.run.smithi169.stdout:kernel-install remove 6.6.0-g8247bbb5b470 2023-11-28T10:58:41.519 INFO:teuthology.orchestra.run.smithi169.stdout:fi 2023-11-28T10:58:41.520 INFO:teuthology.orchestra.run.smithi169.stdout:postuninstall scriptlet (using /bin/sh): 2023-11-28T10:58:41.520 INFO:teuthology.orchestra.run.smithi169.stdout:if [ -x /sbin/update-bootloader ]; then 2023-11-28T10:58:41.520 INFO:teuthology.orchestra.run.smithi169.stdout:/sbin/update-bootloader --remove 6.6.0-g8247bbb5b470 2023-11-28T10:58:41.520 INFO:teuthology.orchestra.run.smithi169.stdout:fi 2023-11-28T10:58:41.521 DEBUG:teuthology.orchestra.run.smithi169:> sudo rpm -qi grub2-tools 2023-11-28T10:58:41.599 INFO:teuthology.orchestra.run.smithi169.stdout:Name : grub2-tools 2023-11-28T10:58:41.600 INFO:teuthology.orchestra.run.smithi169.stdout:Epoch : 1 2023-11-28T10:58:41.600 INFO:teuthology.orchestra.run.smithi169.stdout:Version : 2.02 2023-11-28T10:58:41.600 INFO:teuthology.orchestra.run.smithi169.stdout:Release : 129.el8 2023-11-28T10:58:41.600 INFO:teuthology.orchestra.run.smithi169.stdout:Architecture: x86_64 2023-11-28T10:58:41.600 INFO:teuthology.orchestra.run.smithi169.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2023-11-28T10:58:41.600 INFO:teuthology.orchestra.run.smithi169.stdout:Group : System Environment/Base 2023-11-28T10:58:41.600 INFO:teuthology.orchestra.run.smithi169.stdout:Size : 9265586 2023-11-28T10:58:41.600 INFO:teuthology.orchestra.run.smithi169.stdout:License : GPLv3+ 2023-11-28T10:58:41.600 INFO:teuthology.orchestra.run.smithi169.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-11-28T10:58:41.600 INFO:teuthology.orchestra.run.smithi169.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-11-28T10:58:41.601 INFO:teuthology.orchestra.run.smithi169.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-11-28T10:58:41.601 INFO:teuthology.orchestra.run.smithi169.stdout:Build Host : kbuilder.bsys.centos.org 2023-11-28T10:58:41.601 INFO:teuthology.orchestra.run.smithi169.stdout:Relocations : (not relocatable) 2023-11-28T10:58:41.601 INFO:teuthology.orchestra.run.smithi169.stdout:Packager : CentOS BuildSystem 2023-11-28T10:58:41.601 INFO:teuthology.orchestra.run.smithi169.stdout:Vendor : CentOS 2023-11-28T10:58:41.601 INFO:teuthology.orchestra.run.smithi169.stdout:URL : http://www.gnu.org/software/grub/ 2023-11-28T10:58:41.601 INFO:teuthology.orchestra.run.smithi169.stdout:Summary : Support tools for GRUB. 2023-11-28T10:58:41.601 INFO:teuthology.orchestra.run.smithi169.stdout:Description : 2023-11-28T10:58:41.601 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-28T10:58:41.601 INFO:teuthology.orchestra.run.smithi169.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-11-28T10:58:41.601 INFO:teuthology.orchestra.run.smithi169.stdout:customizable bootloader with modular architecture. It supports a rich 2023-11-28T10:58:41.602 INFO:teuthology.orchestra.run.smithi169.stdout:variety of kernel formats, file systems, computer architectures and 2023-11-28T10:58:41.602 INFO:teuthology.orchestra.run.smithi169.stdout:hardware devices. 2023-11-28T10:58:41.602 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-28T10:58:41.602 INFO:teuthology.orchestra.run.smithi169.stdout:This subpackage provides tools for support of all platforms. 2023-11-28T10:58:41.603 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-11-28T10:58:41.603 INFO:teuthology.task.kernel:Updating grub on smithi169 to boot 6.6.0-g8247bbb5b470 2023-11-28T10:58:41.603 DEBUG:teuthology.orchestra.run.smithi169:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-11-28T10:58:43.672 INFO:teuthology.orchestra.run.smithi092.stdout:kernel-6.6.0_g8247bbb5b470-1 ######################################## 2023-11-28T10:58:44.597 INFO:teuthology.orchestra.run.smithi169.stderr:Generating grub configuration file ... 2023-11-28T10:58:44.966 INFO:teuthology.orchestra.run.smithi092.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-11-28T10:58:47.058 INFO:teuthology.orchestra.run.smithi169.stderr:done 2023-11-28T10:58:47.060 DEBUG:teuthology.orchestra.run.smithi169:> mktemp 2023-11-28T10:58:47.101 INFO:teuthology.orchestra.run.smithi169.stdout:/tmp/tmp.VJVwO6GEAn 2023-11-28T10:58:47.101 DEBUG:teuthology.orchestra.run.smithi169:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.VJVwO6GEAn 2023-11-28T10:58:47.179 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 0666 /tmp/tmp.VJVwO6GEAn 2023-11-28T10:58:47.346 DEBUG:teuthology.orchestra.remote:smithi169:/tmp/tmp.VJVwO6GEAn is 6KB 2023-11-28T10:58:47.396 DEBUG:teuthology.orchestra.run.smithi169:> rm -fr /tmp/tmp.VJVwO6GEAn 2023-11-28T10:58:47.414 DEBUG:teuthology.orchestra.run.smithi169:> sudo /bin/ls /boot/loader/entries || true 2023-11-28T10:58:47.490 INFO:teuthology.orchestra.run.smithi169.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2023-11-28T10:58:47.491 INFO:teuthology.orchestra.run.smithi169.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2023-11-28T10:58:47.491 INFO:teuthology.orchestra.run.smithi169.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2023-11-28T10:58:47.491 INFO:teuthology.orchestra.run.smithi169.stdout:a1b29d6c289747bdb22930598d39874a-6.6.0-g8247bbb5b470.conf 2023-11-28T10:58:47.492 DEBUG:teuthology.orchestra.run.smithi169:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-6.6.0-g8247bbb5b470 2023-11-28T10:58:47.661 DEBUG:teuthology.orchestra.run.smithi169:> sudo shutdown -r now 2023-11-28T10:59:09.807 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi092.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-11-28T10:59:09.807 DEBUG:teuthology.orchestra.run.smithi092:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-11-28T10:59:09.868 INFO:teuthology.orchestra.run.smithi092.stdout:/boot/System.map-6.6.0-g8247bbb5b470 2023-11-28T10:59:09.868 INFO:teuthology.orchestra.run.smithi092.stdout:/boot/config-6.6.0-g8247bbb5b470 2023-11-28T10:59:09.868 INFO:teuthology.orchestra.run.smithi092.stdout:/boot/vmlinuz-6.6.0-g8247bbb5b470 2023-11-28T10:59:09.868 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470 2023-11-28T10:59:09.868 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel 2023-11-28T10:59:09.868 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch 2023-11-28T10:59:09.869 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86 2023-11-28T10:59:09.869 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto 2023-11-28T10:59:09.869 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-11-28T10:59:09.869 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-11-28T10:59:09.869 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-11-28T10:59:09.869 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-11-28T10:59:09.869 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-11-28T10:59:09.869 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-11-28T10:59:09.869 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-11-28T10:59:09.869 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crc32c-intel.ko 2023-11-28T10:59:09.869 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-11-28T10:59:09.869 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-11-28T10:59:09.870 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-11-28T10:59:09.870 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-11-28T10:59:09.870 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-avx2.ko 2023-11-28T10:59:09.870 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-11-28T10:59:09.870 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-11-28T10:59:09.870 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-11-28T10:59:09.870 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-11-28T10:59:09.870 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-11-28T10:59:09.870 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-11-28T10:59:09.870 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-11-28T10:59:09.870 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel 2023-11-28T10:59:09.870 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu 2023-11-28T10:59:09.870 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/mce 2023-11-28T10:59:09.871 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-11-28T10:59:09.871 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm 2023-11-28T10:59:09.871 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm-amd.ko 2023-11-28T10:59:09.871 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm-intel.ko 2023-11-28T10:59:09.871 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm.ko 2023-11-28T10:59:09.871 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto 2023-11-28T10:59:09.871 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ansi_cprng.ko 2023-11-28T10:59:09.871 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/anubis.ko 2023-11-28T10:59:09.871 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/arc4.ko 2023-11-28T10:59:09.871 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx 2023-11-28T10:59:09.871 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_memcpy.ko 2023-11-28T10:59:09.871 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_pq.ko 2023-11-28T10:59:09.872 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_raid6_recov.ko 2023-11-28T10:59:09.872 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_tx.ko 2023-11-28T10:59:09.872 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_xor.ko 2023-11-28T10:59:09.872 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/raid6test.ko 2023-11-28T10:59:09.872 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/authenc.ko 2023-11-28T10:59:09.872 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/authencesn.ko 2023-11-28T10:59:09.872 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blake2b_generic.ko 2023-11-28T10:59:09.872 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blowfish_common.ko 2023-11-28T10:59:09.872 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blowfish_generic.ko 2023-11-28T10:59:09.872 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/camellia_generic.ko 2023-11-28T10:59:09.872 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast5_generic.ko 2023-11-28T10:59:09.872 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast6_generic.ko 2023-11-28T10:59:09.872 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast_common.ko 2023-11-28T10:59:09.873 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ccm.ko 2023-11-28T10:59:09.873 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cmac.ko 2023-11-28T10:59:09.873 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crc32_generic.ko 2023-11-28T10:59:09.873 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_engine.ko 2023-11-28T10:59:09.873 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_user.ko 2023-11-28T10:59:09.873 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cts.ko 2023-11-28T10:59:09.873 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/des_generic.ko 2023-11-28T10:59:09.873 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/dh_generic.ko 2023-11-28T10:59:09.873 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/echainiv.ko 2023-11-28T10:59:09.873 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/essiv.ko 2023-11-28T10:59:09.873 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/fcrypt.ko 2023-11-28T10:59:09.873 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/khazad.ko 2023-11-28T10:59:09.873 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lz4.ko 2023-11-28T10:59:09.874 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lz4hc.ko 2023-11-28T10:59:09.874 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/md4.ko 2023-11-28T10:59:09.874 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/michael_mic.ko 2023-11-28T10:59:09.874 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/pcbc.ko 2023-11-28T10:59:09.874 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/pcrypt.ko 2023-11-28T10:59:09.874 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/rmd160.ko 2023-11-28T10:59:09.874 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/seed.ko 2023-11-28T10:59:09.874 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/serpent_generic.ko 2023-11-28T10:59:09.874 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/tcrypt.ko 2023-11-28T10:59:09.874 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/tea.ko 2023-11-28T10:59:09.874 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/twofish_common.ko 2023-11-28T10:59:09.874 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/twofish_generic.ko 2023-11-28T10:59:09.874 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/vmac.ko 2023-11-28T10:59:09.874 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/wp512.ko 2023-11-28T10:59:09.875 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xcbc.ko 2023-11-28T10:59:09.875 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xor.ko 2023-11-28T10:59:09.875 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xxhash_generic.ko 2023-11-28T10:59:09.875 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers 2023-11-28T10:59:09.875 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi 2023-11-28T10:59:09.875 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_ipmi.ko 2023-11-28T10:59:09.875 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_pad.ko 2023-11-28T10:59:09.875 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/custom_method.ko 2023-11-28T10:59:09.875 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/ec_sys.ko 2023-11-28T10:59:09.875 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/platform_profile.ko 2023-11-28T10:59:09.875 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/sbs.ko 2023-11-28T10:59:09.875 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/sbshc.ko 2023-11-28T10:59:09.876 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/video.ko 2023-11-28T10:59:09.876 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata 2023-11-28T10:59:09.876 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/acard-ahci.ko 2023-11-28T10:59:09.876 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ahci_platform.ko 2023-11-28T10:59:09.876 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ata_generic.ko 2023-11-28T10:59:09.876 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/libahci_platform.ko 2023-11-28T10:59:09.876 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_acpi.ko 2023-11-28T10:59:09.876 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ali.ko 2023-11-28T10:59:09.876 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_amd.ko 2023-11-28T10:59:09.876 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_artop.ko 2023-11-28T10:59:09.876 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_atiixp.ko 2023-11-28T10:59:09.876 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_atp867x.ko 2023-11-28T10:59:09.876 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cmd640.ko 2023-11-28T10:59:09.877 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cmd64x.ko 2023-11-28T10:59:09.877 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cypress.ko 2023-11-28T10:59:09.877 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_efar.ko 2023-11-28T10:59:09.877 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt366.ko 2023-11-28T10:59:09.877 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt37x.ko 2023-11-28T10:59:09.877 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt3x2n.ko 2023-11-28T10:59:09.877 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt3x3.ko 2023-11-28T10:59:09.877 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_it8213.ko 2023-11-28T10:59:09.877 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_it821x.ko 2023-11-28T10:59:09.877 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_jmicron.ko 2023-11-28T10:59:09.877 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_marvell.ko 2023-11-28T10:59:09.877 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_mpiix.ko 2023-11-28T10:59:09.877 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_netcell.ko 2023-11-28T10:59:09.878 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ninja32.ko 2023-11-28T10:59:09.878 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ns87410.ko 2023-11-28T10:59:09.878 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ns87415.ko 2023-11-28T10:59:09.878 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_oldpiix.ko 2023-11-28T10:59:09.878 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_opti.ko 2023-11-28T10:59:09.878 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_optidma.ko 2023-11-28T10:59:09.878 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pcmcia.ko 2023-11-28T10:59:09.878 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pdc2027x.ko 2023-11-28T10:59:09.878 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-11-28T10:59:09.878 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_piccolo.ko 2023-11-28T10:59:09.878 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_rdc.ko 2023-11-28T10:59:09.878 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sch.ko 2023-11-28T10:59:09.879 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_serverworks.ko 2023-11-28T10:59:09.879 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sil680.ko 2023-11-28T10:59:09.879 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sis.ko 2023-11-28T10:59:09.879 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sl82c105.ko 2023-11-28T10:59:09.879 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_triflex.ko 2023-11-28T10:59:09.879 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_via.ko 2023-11-28T10:59:09.879 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pdc_adma.ko 2023-11-28T10:59:09.879 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_inic162x.ko 2023-11-28T10:59:09.879 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_mv.ko 2023-11-28T10:59:09.879 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_nv.ko 2023-11-28T10:59:09.879 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_promise.ko 2023-11-28T10:59:09.880 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_qstor.ko 2023-11-28T10:59:09.880 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sil.ko 2023-11-28T10:59:09.880 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sil24.ko 2023-11-28T10:59:09.880 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sis.ko 2023-11-28T10:59:09.880 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_svw.ko 2023-11-28T10:59:09.880 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sx4.ko 2023-11-28T10:59:09.880 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_uli.ko 2023-11-28T10:59:09.880 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_via.ko 2023-11-28T10:59:09.880 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_vsc.ko 2023-11-28T10:59:09.880 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/auxdisplay 2023-11-28T10:59:09.880 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/auxdisplay/ks0108.ko 2023-11-28T10:59:09.880 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base 2023-11-28T10:59:09.880 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap 2023-11-28T10:59:09.881 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap/regmap-i2c.ko 2023-11-28T10:59:09.881 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap/regmap-mmio.ko 2023-11-28T10:59:09.881 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/bcma 2023-11-28T10:59:09.881 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/bcma/bcma.ko 2023-11-28T10:59:09.881 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block 2023-11-28T10:59:09.881 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/aoe 2023-11-28T10:59:09.881 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/aoe/aoe.ko 2023-11-28T10:59:09.881 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/brd.ko 2023-11-28T10:59:09.881 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/drbd 2023-11-28T10:59:09.881 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/drbd/drbd.ko 2023-11-28T10:59:09.881 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/floppy.ko 2023-11-28T10:59:09.881 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/loop.ko 2023-11-28T10:59:09.882 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/mtip32xx 2023-11-28T10:59:09.882 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-11-28T10:59:09.882 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/nbd.ko 2023-11-28T10:59:09.882 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/null_blk 2023-11-28T10:59:09.882 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/null_blk/null_blk.ko 2023-11-28T10:59:09.882 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/pktcdvd.ko 2023-11-28T10:59:09.882 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/rbd.ko 2023-11-28T10:59:09.882 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/virtio_blk.ko 2023-11-28T10:59:09.882 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkback 2023-11-28T10:59:09.882 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-11-28T10:59:09.882 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkfront.ko 2023-11-28T10:59:09.882 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/zram 2023-11-28T10:59:09.883 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/zram/zram.ko 2023-11-28T10:59:09.883 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char 2023-11-28T10:59:09.883 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hangcheck-timer.ko 2023-11-28T10:59:09.883 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random 2023-11-28T10:59:09.883 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/amd-rng.ko 2023-11-28T10:59:09.883 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/intel-rng.ko 2023-11-28T10:59:09.883 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-11-28T10:59:09.883 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/via-rng.ko 2023-11-28T10:59:09.883 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/virtio-rng.ko 2023-11-28T10:59:09.883 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi 2023-11-28T10:59:09.883 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-11-28T10:59:09.884 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-11-28T10:59:09.884 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-11-28T10:59:09.884 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_si.ko 2023-11-28T10:59:09.884 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-11-28T10:59:09.884 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-11-28T10:59:09.884 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/lp.ko 2023-11-28T10:59:09.884 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/mwave 2023-11-28T10:59:09.884 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/mwave/mwave.ko 2023-11-28T10:59:09.884 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ppdev.ko 2023-11-28T10:59:09.884 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tlclk.ko 2023-11-28T10:59:09.884 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm 2023-11-28T10:59:09.885 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm.ko 2023-11-28T10:59:09.885 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_atmel.ko 2023-11-28T10:59:09.885 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_infineon.ko 2023-11-28T10:59:09.885 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_nsc.ko 2023-11-28T10:59:09.885 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_tis.ko 2023-11-28T10:59:09.885 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-11-28T10:59:09.885 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/uv_mmtimer.ko 2023-11-28T10:59:09.885 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/virtio_console.ko 2023-11-28T10:59:09.885 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq 2023-11-28T10:59:09.886 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-11-28T10:59:09.886 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-11-28T10:59:09.886 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/p4-clockmod.ko 2023-11-28T10:59:09.886 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-11-28T10:59:09.886 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/powernow-k8.ko 2023-11-28T10:59:09.886 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/speedstep-lib.ko 2023-11-28T10:59:09.886 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto 2023-11-28T10:59:09.886 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp 2023-11-28T10:59:09.886 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-11-28T10:59:09.886 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ccp.ko 2023-11-28T10:59:09.887 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel 2023-11-28T10:59:09.887 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat 2023-11-28T10:59:09.887 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_common 2023-11-28T10:59:09.887 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-11-28T10:59:09.887 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-11-28T10:59:09.887 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-11-28T10:59:09.887 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/padlock-aes.ko 2023-11-28T10:59:09.887 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/padlock-sha.ko 2023-11-28T10:59:09.887 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/virtio 2023-11-28T10:59:09.887 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-11-28T10:59:09.888 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dca 2023-11-28T10:59:09.888 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dca/dca.ko 2023-11-28T10:59:09.888 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/devfreq 2023-11-28T10:59:09.888 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-11-28T10:59:09.888 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma 2023-11-28T10:59:09.888 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/dw 2023-11-28T10:59:09.888 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/dw/dw_dmac.ko 2023-11-28T10:59:09.888 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ioat 2023-11-28T10:59:09.888 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ioat/ioatdma.ko 2023-11-28T10:59:09.888 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac 2023-11-28T10:59:09.888 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/amd64_edac.ko 2023-11-28T10:59:09.888 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/e752x_edac.ko 2023-11-28T10:59:09.889 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/edac_mce_amd.ko 2023-11-28T10:59:09.889 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i3000_edac.ko 2023-11-28T10:59:09.889 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i3200_edac.ko 2023-11-28T10:59:09.889 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i5100_edac.ko 2023-11-28T10:59:09.889 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i5400_edac.ko 2023-11-28T10:59:09.889 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i7300_edac.ko 2023-11-28T10:59:09.889 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i7core_edac.ko 2023-11-28T10:59:09.889 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i82975x_edac.ko 2023-11-28T10:59:09.889 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/ie31200_edac.ko 2023-11-28T10:59:09.889 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/sb_edac.ko 2023-11-28T10:59:09.889 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/x38_edac.ko 2023-11-28T10:59:09.890 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware 2023-11-28T10:59:09.890 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/edd.ko 2023-11-28T10:59:09.890 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/iscsi_ibft.ko 2023-11-28T10:59:09.890 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio 2023-11-28T10:59:09.890 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-ich.ko 2023-11-28T10:59:09.890 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-viperboard.ko 2023-11-28T10:59:09.890 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid 2023-11-28T10:59:09.890 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-appleir.ko 2023-11-28T10:59:09.890 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-aureal.ko 2023-11-28T10:59:09.890 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-axff.ko 2023-11-28T10:59:09.890 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-dr.ko 2023-11-28T10:59:09.890 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-elecom.ko 2023-11-28T10:59:09.891 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-elo.ko 2023-11-28T10:59:09.891 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-emsff.ko 2023-11-28T10:59:09.891 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gaff.ko 2023-11-28T10:59:09.891 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gt683r.ko 2023-11-28T10:59:09.891 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gyration.ko 2023-11-28T10:59:09.891 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtek-kbd.ko 2023-11-28T10:59:09.891 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtek-mouse.ko 2023-11-28T10:59:09.891 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtekff.ko 2023-11-28T10:59:09.891 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-hyperv.ko 2023-11-28T10:59:09.891 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-icade.ko 2023-11-28T10:59:09.891 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-keytouch.ko 2023-11-28T10:59:09.891 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-kye.ko 2023-11-28T10:59:09.892 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lcpower.ko 2023-11-28T10:59:09.892 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-led.ko 2023-11-28T10:59:09.892 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lenovo.ko 2023-11-28T10:59:09.892 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech-dj.ko 2023-11-28T10:59:09.892 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-11-28T10:59:09.892 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-multitouch.ko 2023-11-28T10:59:09.892 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-ortek.ko 2023-11-28T10:59:09.892 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-penmount.ko 2023-11-28T10:59:09.892 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-petalynx.ko 2023-11-28T10:59:09.892 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-picolcd.ko 2023-11-28T10:59:09.892 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-pl.ko 2023-11-28T10:59:09.892 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-plantronics.ko 2023-11-28T10:59:09.893 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-primax.ko 2023-11-28T10:59:09.893 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-rmi.ko 2023-11-28T10:59:09.893 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-arvo.ko 2023-11-28T10:59:09.893 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-common.ko 2023-11-28T10:59:09.893 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-isku.ko 2023-11-28T10:59:09.893 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-kone.ko 2023-11-28T10:59:09.893 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-11-28T10:59:09.893 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-konepure.ko 2023-11-28T10:59:09.893 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-11-28T10:59:09.893 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-lua.ko 2023-11-28T10:59:09.893 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-pyra.ko 2023-11-28T10:59:09.894 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-ryos.ko 2023-11-28T10:59:09.894 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-savu.ko 2023-11-28T10:59:09.894 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat.ko 2023-11-28T10:59:09.894 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-saitek.ko 2023-11-28T10:59:09.894 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-samsung.ko 2023-11-28T10:59:09.894 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sensor-hub.ko 2023-11-28T10:59:09.894 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sjoy.ko 2023-11-28T10:59:09.894 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sony.ko 2023-11-28T10:59:09.894 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-speedlink.ko 2023-11-28T10:59:09.894 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-steelseries.ko 2023-11-28T10:59:09.894 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sunplus.ko 2023-11-28T10:59:09.895 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-thrustmaster.ko 2023-11-28T10:59:09.895 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-tivo.ko 2023-11-28T10:59:09.895 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-tmff.ko 2023-11-28T10:59:09.895 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-topseed.ko 2023-11-28T10:59:09.895 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-twinhan.ko 2023-11-28T10:59:09.895 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-uclogic.ko 2023-11-28T10:59:09.895 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-waltop.ko 2023-11-28T10:59:09.895 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-wiimote.ko 2023-11-28T10:59:09.895 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-xinmo.ko 2023-11-28T10:59:09.895 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-zpff.ko 2023-11-28T10:59:09.895 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-zydacron.ko 2023-11-28T10:59:09.895 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/uhid.ko 2023-11-28T10:59:09.896 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/wacom.ko 2023-11-28T10:59:09.896 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv 2023-11-28T10:59:09.896 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_balloon.ko 2023-11-28T10:59:09.896 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_utils.ko 2023-11-28T10:59:09.896 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_vmbus.ko 2023-11-28T10:59:09.896 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon 2023-11-28T10:59:09.896 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/abituguru.ko 2023-11-28T10:59:09.896 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/abituguru3.ko 2023-11-28T10:59:09.896 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/acpi_power_meter.ko 2023-11-28T10:59:09.896 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ad7414.ko 2023-11-28T10:59:09.896 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ad7418.ko 2023-11-28T10:59:09.897 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adc128d818.ko 2023-11-28T10:59:09.897 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1025.ko 2023-11-28T10:59:09.897 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1026.ko 2023-11-28T10:59:09.897 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1029.ko 2023-11-28T10:59:09.897 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1031.ko 2023-11-28T10:59:09.897 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm9240.ko 2023-11-28T10:59:09.897 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ads7828.ko 2023-11-28T10:59:09.897 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7410.ko 2023-11-28T10:59:09.897 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7411.ko 2023-11-28T10:59:09.897 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7462.ko 2023-11-28T10:59:09.897 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7470.ko 2023-11-28T10:59:09.897 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7475.ko 2023-11-28T10:59:09.898 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7x10.ko 2023-11-28T10:59:09.898 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/amc6821.ko 2023-11-28T10:59:09.898 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/applesmc.ko 2023-11-28T10:59:09.898 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asb100.ko 2023-11-28T10:59:09.898 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asc7621.ko 2023-11-28T10:59:09.898 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asus_atk0110.ko 2023-11-28T10:59:09.898 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/atxp1.ko 2023-11-28T10:59:09.898 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/coretemp.ko 2023-11-28T10:59:09.898 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-11-28T10:59:09.898 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/dme1737.ko 2023-11-28T10:59:09.898 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ds1621.ko 2023-11-28T10:59:09.898 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ds620.ko 2023-11-28T10:59:09.899 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc1403.ko 2023-11-28T10:59:09.899 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc6w201.ko 2023-11-28T10:59:09.899 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f71805f.ko 2023-11-28T10:59:09.899 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f71882fg.ko 2023-11-28T10:59:09.899 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f75375s.ko 2023-11-28T10:59:09.899 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/fam15h_power.ko 2023-11-28T10:59:09.899 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/fschmd.ko 2023-11-28T10:59:09.899 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/g760a.ko 2023-11-28T10:59:09.899 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/g762.ko 2023-11-28T10:59:09.899 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/gl518sm.ko 2023-11-28T10:59:09.899 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/gl520sm.ko 2023-11-28T10:59:09.900 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/hwmon-vid.ko 2023-11-28T10:59:09.900 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/i5500_temp.ko 2023-11-28T10:59:09.900 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/i5k_amb.ko 2023-11-28T10:59:09.900 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ibmaem.ko 2023-11-28T10:59:09.900 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ibmpex.ko 2023-11-28T10:59:09.900 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ina209.ko 2023-11-28T10:59:09.900 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ina2xx.ko 2023-11-28T10:59:09.900 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/it87.ko 2023-11-28T10:59:09.900 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/k10temp.ko 2023-11-28T10:59:09.900 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/k8temp.ko 2023-11-28T10:59:09.900 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lineage-pem.ko 2023-11-28T10:59:09.900 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm63.ko 2023-11-28T10:59:09.901 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm73.ko 2023-11-28T10:59:09.901 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm75.ko 2023-11-28T10:59:09.901 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm77.ko 2023-11-28T10:59:09.901 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm78.ko 2023-11-28T10:59:09.901 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm80.ko 2023-11-28T10:59:09.901 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm83.ko 2023-11-28T10:59:09.901 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm85.ko 2023-11-28T10:59:09.901 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm87.ko 2023-11-28T10:59:09.901 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm90.ko 2023-11-28T10:59:09.901 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm92.ko 2023-11-28T10:59:09.902 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm93.ko 2023-11-28T10:59:09.902 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95234.ko 2023-11-28T10:59:09.902 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95241.ko 2023-11-28T10:59:09.902 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95245.ko 2023-11-28T10:59:09.902 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc2945.ko 2023-11-28T10:59:09.902 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4151.ko 2023-11-28T10:59:09.902 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4215.ko 2023-11-28T10:59:09.902 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4222.ko 2023-11-28T10:59:09.902 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4245.ko 2023-11-28T10:59:09.902 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4260.ko 2023-11-28T10:59:09.902 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4261.ko 2023-11-28T10:59:09.903 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max16065.ko 2023-11-28T10:59:09.903 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max1619.ko 2023-11-28T10:59:09.903 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max1668.ko 2023-11-28T10:59:09.903 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max197.ko 2023-11-28T10:59:09.903 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6639.ko 2023-11-28T10:59:09.903 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6650.ko 2023-11-28T10:59:09.903 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6697.ko 2023-11-28T10:59:09.903 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/mcp3021.ko 2023-11-28T10:59:09.903 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6683.ko 2023-11-28T10:59:09.903 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6775-core.ko 2023-11-28T10:59:09.903 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6775.ko 2023-11-28T10:59:09.903 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct7802.ko 2023-11-28T10:59:09.904 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pc87360.ko 2023-11-28T10:59:09.905 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pc87427.ko 2023-11-28T10:59:09.905 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pcf8591.ko 2023-11-28T10:59:09.905 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus 2023-11-28T10:59:09.905 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-11-28T10:59:09.905 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-11-28T10:59:09.905 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-11-28T10:59:09.905 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max16064.ko 2023-11-28T10:59:09.906 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max34440.ko 2023-11-28T10:59:09.906 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max8688.ko 2023-11-28T10:59:09.906 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-11-28T10:59:09.906 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-11-28T10:59:09.906 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-11-28T10:59:09.906 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-11-28T10:59:09.906 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-11-28T10:59:09.906 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-11-28T10:59:09.906 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/powr1220.ko 2023-11-28T10:59:09.906 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch5627.ko 2023-11-28T10:59:09.906 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch5636.ko 2023-11-28T10:59:09.907 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch56xx-common.ko 2023-11-28T10:59:09.907 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sht15.ko 2023-11-28T10:59:09.907 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sht21.ko 2023-11-28T10:59:09.907 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/shtc1.ko 2023-11-28T10:59:09.907 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sis5595.ko 2023-11-28T10:59:09.907 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47b397.ko 2023-11-28T10:59:09.907 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47m1.ko 2023-11-28T10:59:09.907 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47m192.ko 2023-11-28T10:59:09.907 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/thmc50.ko 2023-11-28T10:59:09.907 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp102.ko 2023-11-28T10:59:09.907 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp103.ko 2023-11-28T10:59:09.907 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp401.ko 2023-11-28T10:59:09.908 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp421.ko 2023-11-28T10:59:09.908 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/via-cputemp.ko 2023-11-28T10:59:09.908 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/via686a.ko 2023-11-28T10:59:09.908 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/vt1211.ko 2023-11-28T10:59:09.908 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/vt8231.ko 2023-11-28T10:59:09.908 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83627ehf.ko 2023-11-28T10:59:09.908 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83627hf.ko 2023-11-28T10:59:09.908 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83781d.ko 2023-11-28T10:59:09.908 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83791d.ko 2023-11-28T10:59:09.908 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83792d.ko 2023-11-28T10:59:09.909 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83793.ko 2023-11-28T10:59:09.909 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83795.ko 2023-11-28T10:59:09.909 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83l785ts.ko 2023-11-28T10:59:09.909 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83l786ng.ko 2023-11-28T10:59:09.909 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c 2023-11-28T10:59:09.909 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos 2023-11-28T10:59:09.909 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-11-28T10:59:09.909 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-11-28T10:59:09.909 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses 2023-11-28T10:59:09.909 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-11-28T10:59:09.909 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-11-28T10:59:09.909 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-11-28T10:59:09.910 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-11-28T10:59:09.910 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-11-28T10:59:09.910 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-11-28T10:59:09.910 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-11-28T10:59:09.910 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-11-28T10:59:09.910 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-i801.ko 2023-11-28T10:59:09.910 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-isch.ko 2023-11-28T10:59:09.910 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-11-28T10:59:09.910 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-11-28T10:59:09.910 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-11-28T10:59:09.910 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-parport.ko 2023-11-28T10:59:09.910 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-11-28T10:59:09.911 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-11-28T10:59:09.911 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-11-28T10:59:09.911 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-11-28T10:59:09.911 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-11-28T10:59:09.911 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-11-28T10:59:09.911 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-via.ko 2023-11-28T10:59:09.911 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-11-28T10:59:09.911 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-11-28T10:59:09.911 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-dev.ko 2023-11-28T10:59:09.911 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-mux.ko 2023-11-28T10:59:09.911 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-11-28T10:59:09.912 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-smbus.ko 2023-11-28T10:59:09.912 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-stub.ko 2023-11-28T10:59:09.912 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband 2023-11-28T10:59:09.912 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core 2023-11-28T10:59:09.912 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_cm.ko 2023-11-28T10:59:09.912 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_core.ko 2023-11-28T10:59:09.912 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_umad.ko 2023-11-28T10:59:09.912 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-11-28T10:59:09.912 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/iw_cm.ko 2023-11-28T10:59:09.912 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/rdma_cm.ko 2023-11-28T10:59:09.912 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-11-28T10:59:09.913 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw 2023-11-28T10:59:09.913 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/cxgb4 2023-11-28T10:59:09.913 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-11-28T10:59:09.913 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx4 2023-11-28T10:59:09.913 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-11-28T10:59:09.913 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx5 2023-11-28T10:59:09.913 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-11-28T10:59:09.913 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mthca 2023-11-28T10:59:09.913 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-11-28T10:59:09.913 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp 2023-11-28T10:59:09.913 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ipoib 2023-11-28T10:59:09.913 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-11-28T10:59:09.914 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/iser 2023-11-28T10:59:09.914 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-11-28T10:59:09.914 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/isert 2023-11-28T10:59:09.914 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-11-28T10:59:09.914 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srp 2023-11-28T10:59:09.914 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-11-28T10:59:09.914 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srpt 2023-11-28T10:59:09.914 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-11-28T10:59:09.914 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input 2023-11-28T10:59:09.914 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport 2023-11-28T10:59:09.914 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-11-28T10:59:09.915 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/fm801-gp.ko 2023-11-28T10:59:09.915 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/gameport.ko 2023-11-28T10:59:09.915 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/joydev.ko 2023-11-28T10:59:09.915 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard 2023-11-28T10:59:09.915 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/gpio_keys.ko 2023-11-28T10:59:09.915 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse 2023-11-28T10:59:09.915 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/appletouch.ko 2023-11-28T10:59:09.915 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/bcm5974.ko 2023-11-28T10:59:09.915 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/cyapatp.ko 2023-11-28T10:59:09.915 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/elan_i2c.ko 2023-11-28T10:59:09.915 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/sermouse.ko 2023-11-28T10:59:09.915 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-11-28T10:59:09.916 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/synaptics_usb.ko 2023-11-28T10:59:09.916 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/vsxxxaa.ko 2023-11-28T10:59:09.916 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/rmi4 2023-11-28T10:59:09.916 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/rmi4/rmi_core.ko 2023-11-28T10:59:09.916 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio 2023-11-28T10:59:09.916 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/altera_ps2.ko 2023-11-28T10:59:09.916 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/arc_ps2.ko 2023-11-28T10:59:09.916 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-11-28T10:59:09.916 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/serio_raw.ko 2023-11-28T10:59:09.916 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/sparse-keymap.ko 2023-11-28T10:59:09.916 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu 2023-11-28T10:59:09.916 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/amd 2023-11-28T10:59:09.917 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/amd/iommu_v2.ko 2023-11-28T10:59:09.917 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds 2023-11-28T10:59:09.917 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-blinkm.ko 2023-11-28T10:59:09.917 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lm3530.ko 2023-11-28T10:59:09.917 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lp3944.ko 2023-11-28T10:59:09.917 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-ss4200.ko 2023-11-28T10:59:09.917 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger 2023-11-28T10:59:09.917 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-11-28T10:59:09.917 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-11-28T10:59:09.918 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-11-28T10:59:09.918 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-11-28T10:59:09.918 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-11-28T10:59:09.918 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-11-28T10:59:09.918 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-11-28T10:59:09.918 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-11-28T10:59:09.918 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md 2023-11-28T10:59:09.918 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/bcache 2023-11-28T10:59:09.918 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/bcache/bcache.ko 2023-11-28T10:59:09.918 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-bio-prison.ko 2023-11-28T10:59:09.918 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-cache-smq.ko 2023-11-28T10:59:09.919 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-cache.ko 2023-11-28T10:59:09.919 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-crypt.ko 2023-11-28T10:59:09.919 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-delay.ko 2023-11-28T10:59:09.919 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-flakey.ko 2023-11-28T10:59:09.919 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-log-userspace.ko 2023-11-28T10:59:09.919 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-multipath.ko 2023-11-28T10:59:09.919 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-queue-length.ko 2023-11-28T10:59:09.919 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-raid.ko 2023-11-28T10:59:09.919 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-round-robin.ko 2023-11-28T10:59:09.919 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-service-time.ko 2023-11-28T10:59:09.919 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-switch.ko 2023-11-28T10:59:09.920 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-thin-pool.ko 2023-11-28T10:59:09.920 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-verity.ko 2023-11-28T10:59:09.920 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/faulty.ko 2023-11-28T10:59:09.920 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/linear.ko 2023-11-28T10:59:09.920 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/multipath.ko 2023-11-28T10:59:09.920 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/persistent-data 2023-11-28T10:59:09.920 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-11-28T10:59:09.920 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid0.ko 2023-11-28T10:59:09.920 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid1.ko 2023-11-28T10:59:09.920 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid10.ko 2023-11-28T10:59:09.920 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid456.ko 2023-11-28T10:59:09.920 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message 2023-11-28T10:59:09.921 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion 2023-11-28T10:59:09.921 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptbase.ko 2023-11-28T10:59:09.921 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptctl.ko 2023-11-28T10:59:09.921 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptfc.ko 2023-11-28T10:59:09.921 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptlan.ko 2023-11-28T10:59:09.921 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptsas.ko 2023-11-28T10:59:09.921 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptscsih.ko 2023-11-28T10:59:09.921 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptspi.ko 2023-11-28T10:59:09.921 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd 2023-11-28T10:59:09.921 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lpc_ich.ko 2023-11-28T10:59:09.921 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lpc_sch.ko 2023-11-28T10:59:09.921 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/mfd-core.ko 2023-11-28T10:59:09.922 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/sm501.ko 2023-11-28T10:59:09.922 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/viperboard.ko 2023-11-28T10:59:09.922 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/vx855.ko 2023-11-28T10:59:09.922 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/wl1273-core.ko 2023-11-28T10:59:09.922 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc 2023-11-28T10:59:09.922 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/altera-stapl 2023-11-28T10:59:09.922 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-11-28T10:59:09.922 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/apds9802als.ko 2023-11-28T10:59:09.922 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/apds990x.ko 2023-11-28T10:59:09.922 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/bh1770glc.ko 2023-11-28T10:59:09.922 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/cb710 2023-11-28T10:59:09.922 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/cb710/cb710.ko 2023-11-28T10:59:09.923 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom 2023-11-28T10:59:09.923 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/at24.ko 2023-11-28T10:59:09.923 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/eeprom.ko 2023-11-28T10:59:09.923 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-11-28T10:59:09.923 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/max6875.ko 2023-11-28T10:59:09.923 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/enclosure.ko 2023-11-28T10:59:09.923 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/hpilo.ko 2023-11-28T10:59:09.923 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/isl29003.ko 2023-11-28T10:59:09.923 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/isl29020.ko 2023-11-28T10:59:09.923 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d 2023-11-28T10:59:09.923 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-11-28T10:59:09.923 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-11-28T10:59:09.924 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei 2023-11-28T10:59:09.924 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei-me.ko 2023-11-28T10:59:09.924 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei-txe.ko 2023-11-28T10:59:09.924 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei.ko 2023-11-28T10:59:09.924 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-gru 2023-11-28T10:59:09.924 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-gru/gru.ko 2023-11-28T10:59:09.924 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp 2023-11-28T10:59:09.924 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xp.ko 2023-11-28T10:59:09.924 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xpc.ko 2023-11-28T10:59:09.924 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-11-28T10:59:09.924 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tifm_7xx1.ko 2023-11-28T10:59:09.924 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tifm_core.ko 2023-11-28T10:59:09.925 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tsl2550.ko 2023-11-28T10:59:09.925 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_balloon.ko 2023-11-28T10:59:09.925 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_vmci 2023-11-28T10:59:09.925 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-11-28T10:59:09.925 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net 2023-11-28T10:59:09.926 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/bonding 2023-11-28T10:59:09.926 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/bonding/bonding.ko 2023-11-28T10:59:09.926 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa 2023-11-28T10:59:09.926 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/b53 2023-11-28T10:59:09.926 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/b53/b53_common.ko 2023-11-28T10:59:09.927 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/bcm-sf2.ko 2023-11-28T10:59:09.927 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6060.ko 2023-11-28T10:59:09.927 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6xxx 2023-11-28T10:59:09.927 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-11-28T10:59:09.928 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dummy.ko 2023-11-28T10:59:09.928 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/eql.ko 2023-11-28T10:59:09.928 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet 2023-11-28T10:59:09.928 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com 2023-11-28T10:59:09.928 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-11-28T10:59:09.929 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-11-28T10:59:09.929 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-11-28T10:59:09.929 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-11-28T10:59:09.929 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390 2023-11-28T10:59:09.929 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/8390.ko 2023-11-28T10:59:09.929 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-11-28T10:59:09.930 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-11-28T10:59:09.930 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-11-28T10:59:09.930 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/adaptec 2023-11-28T10:59:09.930 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-11-28T10:59:09.930 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/agere 2023-11-28T10:59:09.930 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/agere/et131x.ko 2023-11-28T10:59:09.930 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/alteon 2023-11-28T10:59:09.931 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-11-28T10:59:09.931 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/altera 2023-11-28T10:59:09.931 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-11-28T10:59:09.931 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd 2023-11-28T10:59:09.931 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-11-28T10:59:09.931 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-11-28T10:59:09.931 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-11-28T10:59:09.931 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros 2023-11-28T10:59:09.931 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/alx 2023-11-28T10:59:09.931 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-11-28T10:59:09.932 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1c 2023-11-28T10:59:09.932 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-11-28T10:59:09.932 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1e 2023-11-28T10:59:09.932 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-11-28T10:59:09.932 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx 2023-11-28T10:59:09.932 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-11-28T10:59:09.932 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-11-28T10:59:09.932 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom 2023-11-28T10:59:09.932 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-11-28T10:59:09.933 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-11-28T10:59:09.933 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-11-28T10:59:09.933 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-11-28T10:59:09.933 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-11-28T10:59:09.933 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/genet 2023-11-28T10:59:09.933 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-11-28T10:59:09.934 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-11-28T10:59:09.934 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade 2023-11-28T10:59:09.934 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/bna 2023-11-28T10:59:09.934 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-11-28T10:59:09.934 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio 2023-11-28T10:59:09.934 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb 2023-11-28T10:59:09.934 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-11-28T10:59:09.934 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-11-28T10:59:09.934 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-11-28T10:59:09.934 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-11-28T10:59:09.935 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-11-28T10:59:09.935 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-11-28T10:59:09.935 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-11-28T10:59:09.935 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-11-28T10:59:09.935 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-11-28T10:59:09.935 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco 2023-11-28T10:59:09.935 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/enic 2023-11-28T10:59:09.935 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-11-28T10:59:09.935 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec 2023-11-28T10:59:09.935 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip 2023-11-28T10:59:09.935 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-11-28T10:59:09.936 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-11-28T10:59:09.936 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-11-28T10:59:09.936 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-11-28T10:59:09.936 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-11-28T10:59:09.936 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-11-28T10:59:09.936 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink 2023-11-28T10:59:09.936 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-11-28T10:59:09.936 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-11-28T10:59:09.936 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dnet.ko 2023-11-28T10:59:09.936 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex 2023-11-28T10:59:09.936 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/benet 2023-11-28T10:59:09.937 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-11-28T10:59:09.937 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ethoc.ko 2023-11-28T10:59:09.937 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/fealnx.ko 2023-11-28T10:59:09.937 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel 2023-11-28T10:59:09.937 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e100.ko 2023-11-28T10:59:09.937 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000 2023-11-28T10:59:09.937 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-11-28T10:59:09.937 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000e 2023-11-28T10:59:09.937 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-11-28T10:59:09.937 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/fm10k 2023-11-28T10:59:09.937 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-11-28T10:59:09.938 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/i40e 2023-11-28T10:59:09.938 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-11-28T10:59:09.938 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/iavf 2023-11-28T10:59:09.938 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-11-28T10:59:09.938 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igb 2023-11-28T10:59:09.938 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-11-28T10:59:09.938 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igbvf 2023-11-28T10:59:09.938 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-11-28T10:59:09.938 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbe 2023-11-28T10:59:09.939 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-11-28T10:59:09.939 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbevf 2023-11-28T10:59:09.939 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-11-28T10:59:09.939 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/jme.ko 2023-11-28T10:59:09.939 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell 2023-11-28T10:59:09.939 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-11-28T10:59:09.939 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/skge.ko 2023-11-28T10:59:09.939 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-11-28T10:59:09.939 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox 2023-11-28T10:59:09.939 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4 2023-11-28T10:59:09.940 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-11-28T10:59:09.940 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-11-28T10:59:09.940 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5 2023-11-28T10:59:09.940 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-11-28T10:59:09.940 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-11-28T10:59:09.940 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel 2023-11-28T10:59:09.940 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-11-28T10:59:09.940 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom 2023-11-28T10:59:09.940 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/myri10ge 2023-11-28T10:59:09.940 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-11-28T10:59:09.940 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi 2023-11-28T10:59:09.941 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-11-28T10:59:09.941 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-11-28T10:59:09.941 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/neterion 2023-11-28T10:59:09.941 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-11-28T10:59:09.941 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/nvidia 2023-11-28T10:59:09.941 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-11-28T10:59:09.941 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines 2023-11-28T10:59:09.941 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-11-28T10:59:09.941 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-11-28T10:59:09.941 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic 2023-11-28T10:59:09.941 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/netxen 2023-11-28T10:59:09.942 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-11-28T10:59:09.942 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-11-28T10:59:09.942 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-11-28T10:59:09.942 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-11-28T10:59:09.942 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rdc 2023-11-28T10:59:09.942 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-11-28T10:59:09.942 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek 2023-11-28T10:59:09.942 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-11-28T10:59:09.942 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-11-28T10:59:09.942 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/atp.ko 2023-11-28T10:59:09.942 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-11-28T10:59:09.943 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rocker 2023-11-28T10:59:09.943 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-11-28T10:59:09.943 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sfc 2023-11-28T10:59:09.943 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-11-28T10:59:09.943 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/silan 2023-11-28T10:59:09.943 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-11-28T10:59:09.943 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis 2023-11-28T10:59:09.943 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/sis190.ko 2023-11-28T10:59:09.943 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/sis900.ko 2023-11-28T10:59:09.943 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc 2023-11-28T10:59:09.943 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-11-28T10:59:09.943 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-11-28T10:59:09.944 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-11-28T10:59:09.944 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-11-28T10:59:09.944 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro 2023-11-28T10:59:09.944 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac 2023-11-28T10:59:09.944 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-11-28T10:59:09.944 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-11-28T10:59:09.944 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun 2023-11-28T10:59:09.944 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/cassini.ko 2023-11-28T10:59:09.945 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/niu.ko 2023-11-28T10:59:09.945 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/sungem.ko 2023-11-28T10:59:09.945 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-11-28T10:59:09.945 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/tehuti 2023-11-28T10:59:09.945 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-11-28T10:59:09.945 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ti 2023-11-28T10:59:09.945 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ti/tlan.ko 2023-11-28T10:59:09.945 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via 2023-11-28T10:59:09.945 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-11-28T10:59:09.945 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-11-28T10:59:09.945 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet 2023-11-28T10:59:09.946 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-11-28T10:59:09.946 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-11-28T10:59:09.946 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/xircom 2023-11-28T10:59:09.946 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-11-28T10:59:09.946 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/geneve.ko 2023-11-28T10:59:09.946 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/hyperv 2023-11-28T10:59:09.946 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-11-28T10:59:09.946 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ieee802154 2023-11-28T10:59:09.946 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ieee802154/fakelb.ko 2023-11-28T10:59:09.946 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ifb.ko 2023-11-28T10:59:09.946 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ipvlan 2023-11-28T10:59:09.947 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ipvlan/ipvlan.ko 2023-11-28T10:59:09.947 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/macvlan.ko 2023-11-28T10:59:09.947 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/macvtap.ko 2023-11-28T10:59:09.947 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio 2023-11-28T10:59:09.947 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio.ko 2023-11-28T10:59:09.947 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-11-28T10:59:09.947 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-11-28T10:59:09.947 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-regmap.ko 2023-11-28T10:59:09.947 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mii.ko 2023-11-28T10:59:09.947 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/net_failover.ko 2023-11-28T10:59:09.947 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/netconsole.ko 2023-11-28T10:59:09.948 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/nlmon.ko 2023-11-28T10:59:09.948 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs 2023-11-28T10:59:09.948 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/pcs-lynx.ko 2023-11-28T10:59:09.948 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-11-28T10:59:09.948 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy 2023-11-28T10:59:09.948 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/amd.ko 2023-11-28T10:59:09.948 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-11-28T10:59:09.948 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-11-28T10:59:09.948 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm7xxx.ko 2023-11-28T10:59:09.948 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm87xx.ko 2023-11-28T10:59:09.948 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/broadcom.ko 2023-11-28T10:59:09.949 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/cicada.ko 2023-11-28T10:59:09.949 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/davicom.ko 2023-11-28T10:59:09.949 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/dp83640.ko 2023-11-28T10:59:09.949 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/et1011c.ko 2023-11-28T10:59:09.949 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/icplus.ko 2023-11-28T10:59:09.949 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/lxt.ko 2023-11-28T10:59:09.949 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/marvell.ko 2023-11-28T10:59:09.949 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/micrel.ko 2023-11-28T10:59:09.949 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/national.ko 2023-11-28T10:59:09.949 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/phylink.ko 2023-11-28T10:59:09.949 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/qsemi.ko 2023-11-28T10:59:09.949 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/realtek.ko 2023-11-28T10:59:09.950 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/smsc.ko 2023-11-28T10:59:09.950 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/ste10Xp.ko 2023-11-28T10:59:09.950 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/vitesse.ko 2023-11-28T10:59:09.950 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp 2023-11-28T10:59:09.950 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/bsd_comp.ko 2023-11-28T10:59:09.950 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_async.ko 2023-11-28T10:59:09.950 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_deflate.ko 2023-11-28T10:59:09.950 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_generic.ko 2023-11-28T10:59:09.950 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_mppe.ko 2023-11-28T10:59:09.950 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_synctty.ko 2023-11-28T10:59:09.950 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pppoe.ko 2023-11-28T10:59:09.951 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pppox.ko 2023-11-28T10:59:09.951 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pptp.ko 2023-11-28T10:59:09.951 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip 2023-11-28T10:59:09.951 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip/slhc.ko 2023-11-28T10:59:09.951 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip/slip.ko 2023-11-28T10:59:09.951 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/sungem_phy.ko 2023-11-28T10:59:09.951 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/tap.ko 2023-11-28T10:59:09.951 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team 2023-11-28T10:59:09.951 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team.ko 2023-11-28T10:59:09.951 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_activebackup.ko 2023-11-28T10:59:09.951 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_broadcast.ko 2023-11-28T10:59:09.952 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-11-28T10:59:09.952 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_random.ko 2023-11-28T10:59:09.952 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-11-28T10:59:09.952 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/tun.ko 2023-11-28T10:59:09.952 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/veth.ko 2023-11-28T10:59:09.952 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/virtio_net.ko 2023-11-28T10:59:09.952 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vmxnet3 2023-11-28T10:59:09.952 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-11-28T10:59:09.952 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vxlan 2023-11-28T10:59:09.952 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vxlan/vxlan.ko 2023-11-28T10:59:09.953 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netback 2023-11-28T10:59:09.953 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netback/xen-netback.ko 2023-11-28T10:59:09.953 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netfront.ko 2023-11-28T10:59:09.953 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ntb 2023-11-28T10:59:09.953 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ntb/ntb.ko 2023-11-28T10:59:09.953 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme 2023-11-28T10:59:09.953 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host 2023-11-28T10:59:09.953 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/nvme-core.ko 2023-11-28T10:59:09.953 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/nvme.ko 2023-11-28T10:59:09.953 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport 2023-11-28T10:59:09.953 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport.ko 2023-11-28T10:59:09.954 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_cs.ko 2023-11-28T10:59:09.954 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_pc.ko 2023-11-28T10:59:09.954 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_serial.ko 2023-11-28T10:59:09.954 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci 2023-11-28T10:59:09.954 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug 2023-11-28T10:59:09.954 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-11-28T10:59:09.954 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/pcie 2023-11-28T10:59:09.954 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/pcie/aer_inject.ko 2023-11-28T10:59:09.954 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/xen-pcifront.ko 2023-11-28T10:59:09.954 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia 2023-11-28T10:59:09.955 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/i82092.ko 2023-11-28T10:59:09.955 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/pd6729.ko 2023-11-28T10:59:09.955 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/yenta_socket.ko 2023-11-28T10:59:09.955 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl 2023-11-28T10:59:09.955 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/intel 2023-11-28T10:59:09.955 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-11-28T10:59:09.955 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform 2023-11-28T10:59:09.955 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86 2023-11-28T10:59:09.955 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/acer-wmi.ko 2023-11-28T10:59:09.955 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/acerhdf.ko 2023-11-28T10:59:09.955 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/apple-gmux.ko 2023-11-28T10:59:09.956 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-laptop.ko 2023-11-28T10:59:09.956 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-11-28T10:59:09.956 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-wmi.ko 2023-11-28T10:59:09.956 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/classmate-laptop.ko 2023-11-28T10:59:09.956 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-11-28T10:59:09.956 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-11-28T10:59:09.956 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-11-28T10:59:09.956 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-11-28T10:59:09.956 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/hdaps.ko 2023-11-28T10:59:09.956 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel 2023-11-28T10:59:09.956 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-11-28T10:59:09.957 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel_ips.ko 2023-11-28T10:59:09.957 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/msi-wmi.ko 2023-11-28T10:59:09.957 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/mxm-wmi.ko 2023-11-28T10:59:09.957 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-11-28T10:59:09.957 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/samsung-laptop.ko 2023-11-28T10:59:09.957 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/samsung-q10.ko 2023-11-28T10:59:09.957 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/topstar-laptop.ko 2023-11-28T10:59:09.957 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-11-28T10:59:09.957 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/toshiba_haps.ko 2023-11-28T10:59:09.957 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/wmi-bmof.ko 2023-11-28T10:59:09.957 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/wmi.ko 2023-11-28T10:59:09.958 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap 2023-11-28T10:59:09.958 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/intel_rapl_common.ko 2023-11-28T10:59:09.958 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/intel_rapl_msr.ko 2023-11-28T10:59:09.958 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps 2023-11-28T10:59:09.958 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients 2023-11-28T10:59:09.958 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps-gpio.ko 2023-11-28T10:59:09.958 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps-ldisc.ko 2023-11-28T10:59:09.958 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps_parport.ko 2023-11-28T10:59:09.958 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/pps_core.ko 2023-11-28T10:59:09.958 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp 2023-11-28T10:59:09.958 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ptp.ko 2023-11-28T10:59:09.959 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ptp_kvm.ko 2023-11-28T10:59:09.959 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc 2023-11-28T10:59:09.959 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-bq32k.ko 2023-11-28T10:59:09.959 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1286.ko 2023-11-28T10:59:09.959 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1307.ko 2023-11-28T10:59:09.959 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1374.ko 2023-11-28T10:59:09.959 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1511.ko 2023-11-28T10:59:09.959 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1553.ko 2023-11-28T10:59:09.959 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1672.ko 2023-11-28T10:59:09.959 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1742.ko 2023-11-28T10:59:09.959 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds2404.ko 2023-11-28T10:59:09.959 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds3232.ko 2023-11-28T10:59:09.960 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-em3027.ko 2023-11-28T10:59:09.960 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-fm3130.ko 2023-11-28T10:59:09.960 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-isl12022.ko 2023-11-28T10:59:09.960 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-isl1208.ko 2023-11-28T10:59:09.960 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m41t80.ko 2023-11-28T10:59:09.960 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t35.ko 2023-11-28T10:59:09.960 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t59.ko 2023-11-28T10:59:09.960 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-max6900.ko 2023-11-28T10:59:09.960 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-msm6242.ko 2023-11-28T10:59:09.960 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf2127.ko 2023-11-28T10:59:09.960 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf85063.ko 2023-11-28T10:59:09.961 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8523.ko 2023-11-28T10:59:09.961 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8563.ko 2023-11-28T10:59:09.961 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8583.ko 2023-11-28T10:59:09.961 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rp5c01.ko 2023-11-28T10:59:09.961 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rs5c372.ko 2023-11-28T10:59:09.961 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-11-28T10:59:09.961 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rx8025.ko 2023-11-28T10:59:09.961 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rx8581.ko 2023-11-28T10:59:09.961 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-11-28T10:59:09.961 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-x1205.ko 2023-11-28T10:59:09.961 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi 2023-11-28T10:59:09.962 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-9xxx.ko 2023-11-28T10:59:09.962 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-sas.ko 2023-11-28T10:59:09.962 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-xxxx.ko 2023-11-28T10:59:09.962 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/BusLogic.ko 2023-11-28T10:59:09.962 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/a100u2w.ko 2023-11-28T10:59:09.962 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aacraid 2023-11-28T10:59:09.962 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aacraid/aacraid.ko 2023-11-28T10:59:09.962 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/advansys.ko 2023-11-28T10:59:09.962 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx 2023-11-28T10:59:09.962 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-11-28T10:59:09.963 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-11-28T10:59:09.963 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/am53c974.ko 2023-11-28T10:59:09.963 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/arcmsr 2023-11-28T10:59:09.963 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-11-28T10:59:09.963 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/atp870u.ko 2023-11-28T10:59:09.963 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/be2iscsi 2023-11-28T10:59:09.964 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-11-28T10:59:09.964 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bfa 2023-11-28T10:59:09.964 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bfa/bfa.ko 2023-11-28T10:59:09.964 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2fc 2023-11-28T10:59:09.964 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-11-28T10:59:09.964 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2i 2023-11-28T10:59:09.964 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-11-28T10:59:09.964 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ch.ko 2023-11-28T10:59:09.964 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/csiostor 2023-11-28T10:59:09.964 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/csiostor/csiostor.ko 2023-11-28T10:59:09.964 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi 2023-11-28T10:59:09.965 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb3i 2023-11-28T10:59:09.965 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-11-28T10:59:09.966 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb4i 2023-11-28T10:59:09.966 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-11-28T10:59:09.966 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-11-28T10:59:09.966 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/dc395x.ko 2023-11-28T10:59:09.966 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler 2023-11-28T10:59:09.966 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-11-28T10:59:09.967 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-11-28T10:59:09.967 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-11-28T10:59:09.967 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-11-28T10:59:09.967 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/dmx3191d.ko 2023-11-28T10:59:09.967 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esas2r 2023-11-28T10:59:09.967 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esas2r/esas2r.ko 2023-11-28T10:59:09.967 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esp_scsi.ko 2023-11-28T10:59:09.967 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe 2023-11-28T10:59:09.967 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/fcoe.ko 2023-11-28T10:59:09.967 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-11-28T10:59:09.967 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fnic 2023-11-28T10:59:09.967 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fnic/fnic.ko 2023-11-28T10:59:09.968 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hpsa.ko 2023-11-28T10:59:09.968 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hptiop.ko 2023-11-28T10:59:09.968 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hv_storvsc.ko 2023-11-28T10:59:09.968 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/initio.ko 2023-11-28T10:59:09.968 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ipr.ko 2023-11-28T10:59:09.968 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ips.ko 2023-11-28T10:59:09.968 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/isci 2023-11-28T10:59:09.969 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/isci/isci.ko 2023-11-28T10:59:09.969 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-11-28T10:59:09.969 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/iscsi_tcp.ko 2023-11-28T10:59:09.969 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libfc 2023-11-28T10:59:09.969 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libfc/libfc.ko 2023-11-28T10:59:09.969 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libiscsi.ko 2023-11-28T10:59:09.969 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libiscsi_tcp.ko 2023-11-28T10:59:09.969 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libsas 2023-11-28T10:59:09.969 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libsas/libsas.ko 2023-11-28T10:59:09.970 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/lpfc 2023-11-28T10:59:09.970 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/lpfc/lpfc.ko 2023-11-28T10:59:09.970 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid 2023-11-28T10:59:09.970 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid.ko 2023-11-28T10:59:09.970 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-11-28T10:59:09.970 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-11-28T10:59:09.970 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-11-28T10:59:09.970 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mpt3sas 2023-11-28T10:59:09.970 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-11-28T10:59:09.970 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvsas 2023-11-28T10:59:09.970 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvsas/mvsas.ko 2023-11-28T10:59:09.970 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvumi.ko 2023-11-28T10:59:09.971 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pm8001 2023-11-28T10:59:09.971 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-11-28T10:59:09.971 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pmcraid.ko 2023-11-28T10:59:09.971 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla1280.ko 2023-11-28T10:59:09.971 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx 2023-11-28T10:59:09.971 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-11-28T10:59:09.971 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-11-28T10:59:09.971 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla4xxx 2023-11-28T10:59:09.971 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-11-28T10:59:09.971 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/raid_class.ko 2023-11-28T10:59:09.971 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_debug.ko 2023-11-28T10:59:09.972 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_fc.ko 2023-11-28T10:59:09.972 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-11-28T10:59:09.972 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_sas.ko 2023-11-28T10:59:09.972 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_spi.ko 2023-11-28T10:59:09.972 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_srp.ko 2023-11-28T10:59:09.972 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ses.ko 2023-11-28T10:59:09.972 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/st.ko 2023-11-28T10:59:09.972 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/stex.ko 2023-11-28T10:59:09.972 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/sym53c8xx_2 2023-11-28T10:59:09.972 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-11-28T10:59:09.972 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/virtio_scsi.ko 2023-11-28T10:59:09.972 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/vmw_pvscsi.ko 2023-11-28T10:59:09.973 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/wd719x.ko 2023-11-28T10:59:09.973 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ssb 2023-11-28T10:59:09.973 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ssb/ssb.ko 2023-11-28T10:59:09.973 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target 2023-11-28T10:59:09.973 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/iscsi 2023-11-28T10:59:09.973 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-11-28T10:59:09.973 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/loopback 2023-11-28T10:59:09.973 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/loopback/tcm_loop.ko 2023-11-28T10:59:09.973 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_file.ko 2023-11-28T10:59:09.973 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_iblock.ko 2023-11-28T10:59:09.973 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_mod.ko 2023-11-28T10:59:09.973 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_pscsi.ko 2023-11-28T10:59:09.974 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_user.ko 2023-11-28T10:59:09.974 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/tcm_fc 2023-11-28T10:59:09.974 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-11-28T10:59:09.974 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal 2023-11-28T10:59:09.974 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel 2023-11-28T10:59:09.974 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal 2023-11-28T10:59:09.974 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-11-28T10:59:09.974 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-11-28T10:59:09.974 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-11-28T10:59:09.974 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-11-28T10:59:09.974 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-11-28T10:59:09.975 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-11-28T10:59:09.975 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-11-28T10:59:09.975 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-11-28T10:59:09.975 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-11-28T10:59:09.975 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-11-28T10:59:09.975 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-11-28T10:59:09.975 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-11-28T10:59:09.975 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-11-28T10:59:09.975 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-11-28T10:59:09.975 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-11-28T10:59:09.975 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty 2023-11-28T10:59:09.975 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ipwireless 2023-11-28T10:59:09.976 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-11-28T10:59:09.976 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/n_gsm.ko 2023-11-28T10:59:09.976 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/n_hdlc.ko 2023-11-28T10:59:09.976 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/nozomi.ko 2023-11-28T10:59:09.976 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial 2023-11-28T10:59:09.976 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250 2023-11-28T10:59:09.976 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-11-28T10:59:09.976 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/arc_uart.ko 2023-11-28T10:59:09.976 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/jsm 2023-11-28T10:59:09.976 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/jsm/jsm.ko 2023-11-28T10:59:09.976 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs 2023-11-28T10:59:09.976 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/core 2023-11-28T10:59:09.977 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/core/ufshcd-core.ko 2023-11-28T10:59:09.977 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host 2023-11-28T10:59:09.977 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-11-28T10:59:09.977 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio 2023-11-28T10:59:09.977 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio.ko 2023-11-28T10:59:09.977 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_aec.ko 2023-11-28T10:59:09.977 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_cif.ko 2023-11-28T10:59:09.977 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_pci_generic.ko 2023-11-28T10:59:09.977 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_sercos3.ko 2023-11-28T10:59:09.977 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb 2023-11-28T10:59:09.977 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class 2023-11-28T10:59:09.977 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/cdc-acm.ko 2023-11-28T10:59:09.978 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/cdc-wdm.ko 2023-11-28T10:59:09.978 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/usblp.ko 2023-11-28T10:59:09.978 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/usbtmc.ko 2023-11-28T10:59:09.978 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host 2023-11-28T10:59:09.978 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/sl811-hcd.ko 2023-11-28T10:59:09.978 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image 2023-11-28T10:59:09.978 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/mdc800.ko 2023-11-28T10:59:09.979 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/microtek.ko 2023-11-28T10:59:09.979 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc 2023-11-28T10:59:09.979 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/adutux.ko 2023-11-28T10:59:09.979 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/appledisplay.ko 2023-11-28T10:59:09.979 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/emi26.ko 2023-11-28T10:59:09.979 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/emi62.ko 2023-11-28T10:59:09.979 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ezusb.ko 2023-11-28T10:59:09.979 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/idmouse.ko 2023-11-28T10:59:09.979 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/iowarrior.ko 2023-11-28T10:59:09.979 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/isight_firmware.ko 2023-11-28T10:59:09.980 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ldusb.ko 2023-11-28T10:59:09.980 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/legousbtower.ko 2023-11-28T10:59:09.980 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/sisusbvga 2023-11-28T10:59:09.980 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-11-28T10:59:09.980 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/trancevibrator.ko 2023-11-28T10:59:09.980 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usb3503.ko 2023-11-28T10:59:09.980 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usblcd.ko 2023-11-28T10:59:09.980 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usbsevseg.ko 2023-11-28T10:59:09.980 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/uss720.ko 2023-11-28T10:59:09.980 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/yurex.ko 2023-11-28T10:59:09.980 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy 2023-11-28T10:59:09.980 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy/phy-generic.ko 2023-11-28T10:59:09.981 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial 2023-11-28T10:59:09.981 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/aircable.ko 2023-11-28T10:59:09.981 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ark3116.ko 2023-11-28T10:59:09.981 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/belkin_sa.ko 2023-11-28T10:59:09.981 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ch341.ko 2023-11-28T10:59:09.981 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cp210x.ko 2023-11-28T10:59:09.981 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cyberjack.ko 2023-11-28T10:59:09.981 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cypress_m8.ko 2023-11-28T10:59:09.981 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/digi_acceleport.ko 2023-11-28T10:59:09.981 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/empeg.ko 2023-11-28T10:59:09.982 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ftdi_sio.ko 2023-11-28T10:59:09.982 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/garmin_gps.ko 2023-11-28T10:59:09.982 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/io_edgeport.ko 2023-11-28T10:59:09.982 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/io_ti.ko 2023-11-28T10:59:09.982 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ipaq.ko 2023-11-28T10:59:09.982 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ipw.ko 2023-11-28T10:59:09.982 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ir-usb.ko 2023-11-28T10:59:09.982 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-11-28T10:59:09.982 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/keyspan.ko 2023-11-28T10:59:09.982 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/keyspan_pda.ko 2023-11-28T10:59:09.982 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/kl5kusb105.ko 2023-11-28T10:59:09.983 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/kobil_sct.ko 2023-11-28T10:59:09.983 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mct_u232.ko 2023-11-28T10:59:09.983 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mos7720.ko 2023-11-28T10:59:09.983 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mos7840.ko 2023-11-28T10:59:09.983 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/navman.ko 2023-11-28T10:59:09.983 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/omninet.ko 2023-11-28T10:59:09.983 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/opticon.ko 2023-11-28T10:59:09.983 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/option.ko 2023-11-28T10:59:09.983 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/oti6858.ko 2023-11-28T10:59:09.983 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/pl2303.ko 2023-11-28T10:59:09.983 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/qcaux.ko 2023-11-28T10:59:09.983 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/qcserial.ko 2023-11-28T10:59:09.984 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/quatech2.ko 2023-11-28T10:59:09.984 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/safe_serial.ko 2023-11-28T10:59:09.984 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/sierra.ko 2023-11-28T10:59:09.984 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/spcp8x5.ko 2023-11-28T10:59:09.984 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ssu100.ko 2023-11-28T10:59:09.984 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/symbolserial.ko 2023-11-28T10:59:09.984 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-11-28T10:59:09.984 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-11-28T10:59:09.984 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb_debug.ko 2023-11-28T10:59:09.984 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb_wwan.ko 2023-11-28T10:59:09.984 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/visor.ko 2023-11-28T10:59:09.984 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/whiteheat.ko 2023-11-28T10:59:09.985 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/xsens_mt.ko 2023-11-28T10:59:09.985 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage 2023-11-28T10:59:09.985 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/uas.ko 2023-11-28T10:59:09.985 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-alauda.ko 2023-11-28T10:59:09.985 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-cypress.ko 2023-11-28T10:59:09.985 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-datafab.ko 2023-11-28T10:59:09.985 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-11-28T10:59:09.985 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-freecom.ko 2023-11-28T10:59:09.985 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-isd200.ko 2023-11-28T10:59:09.985 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-11-28T10:59:09.985 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-karma.ko 2023-11-28T10:59:09.985 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-onetouch.ko 2023-11-28T10:59:09.986 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-realtek.ko 2023-11-28T10:59:09.986 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-sddr09.ko 2023-11-28T10:59:09.986 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-sddr55.ko 2023-11-28T10:59:09.986 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-usbat.ko 2023-11-28T10:59:09.986 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/usb-storage.ko 2023-11-28T10:59:09.986 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip 2023-11-28T10:59:09.986 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/usbip-core.ko 2023-11-28T10:59:09.986 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/usbip-host.ko 2023-11-28T10:59:09.986 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-11-28T10:59:09.986 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio 2023-11-28T10:59:09.986 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci 2023-11-28T10:59:09.987 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-11-28T10:59:09.987 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/vfio-pci.ko 2023-11-28T10:59:09.987 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/vfio.ko 2023-11-28T10:59:09.987 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-11-28T10:59:09.987 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost 2023-11-28T10:59:09.987 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost.ko 2023-11-28T10:59:09.987 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_iotlb.ko 2023-11-28T10:59:09.987 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_net.ko 2023-11-28T10:59:09.987 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_scsi.ko 2023-11-28T10:59:09.987 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video 2023-11-28T10:59:09.987 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight 2023-11-28T10:59:09.987 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/apple_bl.ko 2023-11-28T10:59:09.988 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/lcd.ko 2023-11-28T10:59:09.988 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/platform_lcd.ko 2023-11-28T10:59:09.988 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio 2023-11-28T10:59:09.988 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio.ko 2023-11-28T10:59:09.988 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_balloon.ko 2023-11-28T10:59:09.988 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_mmio.ko 2023-11-28T10:59:09.988 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci.ko 2023-11-28T10:59:09.988 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-11-28T10:59:09.988 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-11-28T10:59:09.988 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_ring.ko 2023-11-28T10:59:09.988 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1 2023-11-28T10:59:09.988 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters 2023-11-28T10:59:09.989 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ds2482.ko 2023-11-28T10:59:09.989 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ds2490.ko 2023-11-28T10:59:09.989 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves 2023-11-28T10:59:09.989 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-11-28T10:59:09.989 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-11-28T10:59:09.989 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-11-28T10:59:09.989 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-11-28T10:59:09.989 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-11-28T10:59:09.989 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-11-28T10:59:09.989 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-11-28T10:59:09.989 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-11-28T10:59:09.990 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-11-28T10:59:09.990 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_smem.ko 2023-11-28T10:59:09.990 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_therm.ko 2023-11-28T10:59:09.990 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/wire.ko 2023-11-28T10:59:09.990 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog 2023-11-28T10:59:09.990 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/alim1535_wdt.ko 2023-11-28T10:59:09.990 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/alim7101_wdt.ko 2023-11-28T10:59:09.990 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/f71808e_wdt.ko 2023-11-28T10:59:09.990 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/hpwdt.ko 2023-11-28T10:59:09.990 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/i6300esb.ko 2023-11-28T10:59:09.990 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-11-28T10:59:09.990 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/iTCO_wdt.ko 2023-11-28T10:59:09.991 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ib700wdt.ko 2023-11-28T10:59:09.991 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ibmasr.ko 2023-11-28T10:59:09.991 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-11-28T10:59:09.991 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/it8712f_wdt.ko 2023-11-28T10:59:09.991 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/it87_wdt.ko 2023-11-28T10:59:09.991 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/machzwd.ko 2023-11-28T10:59:09.991 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/nv_tco.ko 2023-11-28T10:59:09.991 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pcwd_pci.ko 2023-11-28T10:59:09.991 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pcwd_usb.ko 2023-11-28T10:59:09.991 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-11-28T10:59:09.991 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sch311x_wdt.ko 2023-11-28T10:59:09.991 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/softdog.ko 2023-11-28T10:59:09.992 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sp5100_tco.ko 2023-11-28T10:59:09.992 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/via_wdt.ko 2023-11-28T10:59:09.992 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-11-28T10:59:09.992 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83877f_wdt.ko 2023-11-28T10:59:09.992 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83977f_wdt.ko 2023-11-28T10:59:09.992 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/wdt_pci.ko 2023-11-28T10:59:09.992 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/xen_wdt.ko 2023-11-28T10:59:09.992 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen 2023-11-28T10:59:09.992 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-acpi-processor.ko 2023-11-28T10:59:09.992 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-evtchn.ko 2023-11-28T10:59:09.992 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-gntalloc.ko 2023-11-28T10:59:09.993 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-gntdev.ko 2023-11-28T10:59:09.993 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-pciback 2023-11-28T10:59:09.993 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-11-28T10:59:09.993 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-privcmd.ko 2023-11-28T10:59:09.993 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xenfs 2023-11-28T10:59:09.993 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xenfs/xenfs.ko 2023-11-28T10:59:09.993 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs 2023-11-28T10:59:09.993 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/9p 2023-11-28T10:59:09.993 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/9p/9p.ko 2023-11-28T10:59:09.993 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/affs 2023-11-28T10:59:09.993 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/affs/affs.ko 2023-11-28T10:59:09.993 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/befs 2023-11-28T10:59:09.993 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/befs/befs.ko 2023-11-28T10:59:09.994 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/binfmt_misc.ko 2023-11-28T10:59:09.994 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/btrfs 2023-11-28T10:59:09.994 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/btrfs/btrfs.ko 2023-11-28T10:59:09.994 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cachefiles 2023-11-28T10:59:09.994 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cachefiles/cachefiles.ko 2023-11-28T10:59:09.994 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ceph 2023-11-28T10:59:09.994 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ceph/ceph.ko 2023-11-28T10:59:09.994 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/coda 2023-11-28T10:59:09.994 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/coda/coda.ko 2023-11-28T10:59:09.994 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cramfs 2023-11-28T10:59:09.994 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cramfs/cramfs.ko 2023-11-28T10:59:09.995 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/dlm 2023-11-28T10:59:09.995 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/dlm/dlm.ko 2023-11-28T10:59:09.995 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ecryptfs 2023-11-28T10:59:09.995 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ecryptfs/ecryptfs.ko 2023-11-28T10:59:09.995 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/f2fs 2023-11-28T10:59:09.995 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/f2fs/f2fs.ko 2023-11-28T10:59:09.995 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat 2023-11-28T10:59:09.995 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/fat.ko 2023-11-28T10:59:09.995 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/msdos.ko 2023-11-28T10:59:09.995 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/vfat.ko 2023-11-28T10:59:09.995 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fscache 2023-11-28T10:59:09.995 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fscache/fscache.ko 2023-11-28T10:59:09.996 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse 2023-11-28T10:59:09.996 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse/cuse.ko 2023-11-28T10:59:09.996 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse/fuse.ko 2023-11-28T10:59:09.996 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/gfs2 2023-11-28T10:59:09.996 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/gfs2/gfs2.ko 2023-11-28T10:59:09.996 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfs 2023-11-28T10:59:09.996 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfs/hfs.ko 2023-11-28T10:59:09.996 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfsplus 2023-11-28T10:59:09.996 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfsplus/hfsplus.ko 2023-11-28T10:59:09.996 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/isofs 2023-11-28T10:59:09.996 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/isofs/isofs.ko 2023-11-28T10:59:09.996 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/jfs 2023-11-28T10:59:09.997 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/jfs/jfs.ko 2023-11-28T10:59:09.997 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/lockd 2023-11-28T10:59:09.997 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/lockd/lockd.ko 2023-11-28T10:59:09.997 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/minix 2023-11-28T10:59:09.997 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/minix/minix.ko 2023-11-28T10:59:09.997 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/netfs 2023-11-28T10:59:09.997 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/netfs/netfs.ko 2023-11-28T10:59:09.997 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs 2023-11-28T10:59:09.997 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/blocklayout 2023-11-28T10:59:09.997 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-11-28T10:59:09.997 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/filelayout 2023-11-28T10:59:09.997 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-11-28T10:59:09.998 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/flexfilelayout 2023-11-28T10:59:09.998 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-11-28T10:59:09.998 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfs.ko 2023-11-28T10:59:09.998 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv3.ko 2023-11-28T10:59:09.998 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv4.ko 2023-11-28T10:59:09.998 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common 2023-11-28T10:59:09.998 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/grace.ko 2023-11-28T10:59:09.998 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/nfs_acl.ko 2023-11-28T10:59:09.999 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfsd 2023-11-28T10:59:09.999 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfsd/nfsd.ko 2023-11-28T10:59:09.999 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nilfs2 2023-11-28T10:59:09.999 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nilfs2/nilfs2.ko 2023-11-28T10:59:09.999 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls 2023-11-28T10:59:09.999 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-celtic.ko 2023-11-28T10:59:09.999 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-centeuro.ko 2023-11-28T10:59:09.999 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-croatian.ko 2023-11-28T10:59:09.999 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-cyrillic.ko 2023-11-28T10:59:09.999 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-gaelic.ko 2023-11-28T10:59:09.999 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-greek.ko 2023-11-28T10:59:09.999 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-iceland.ko 2023-11-28T10:59:10.000 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-inuit.ko 2023-11-28T10:59:10.000 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-roman.ko 2023-11-28T10:59:10.000 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-romanian.ko 2023-11-28T10:59:10.000 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-turkish.ko 2023-11-28T10:59:10.000 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1250.ko 2023-11-28T10:59:10.000 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1251.ko 2023-11-28T10:59:10.000 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1255.ko 2023-11-28T10:59:10.000 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp737.ko 2023-11-28T10:59:10.000 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp775.ko 2023-11-28T10:59:10.001 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp850.ko 2023-11-28T10:59:10.001 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp852.ko 2023-11-28T10:59:10.001 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp855.ko 2023-11-28T10:59:10.001 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp857.ko 2023-11-28T10:59:10.001 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp860.ko 2023-11-28T10:59:10.001 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp861.ko 2023-11-28T10:59:10.001 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp862.ko 2023-11-28T10:59:10.001 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp863.ko 2023-11-28T10:59:10.001 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp864.ko 2023-11-28T10:59:10.002 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp865.ko 2023-11-28T10:59:10.002 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp866.ko 2023-11-28T10:59:10.002 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp869.ko 2023-11-28T10:59:10.002 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp874.ko 2023-11-28T10:59:10.002 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp932.ko 2023-11-28T10:59:10.002 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp936.ko 2023-11-28T10:59:10.002 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp949.ko 2023-11-28T10:59:10.002 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp950.ko 2023-11-28T10:59:10.002 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_euc-jp.ko 2023-11-28T10:59:10.002 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-1.ko 2023-11-28T10:59:10.003 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-13.ko 2023-11-28T10:59:10.003 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-14.ko 2023-11-28T10:59:10.003 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-15.ko 2023-11-28T10:59:10.003 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-2.ko 2023-11-28T10:59:10.003 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-3.ko 2023-11-28T10:59:10.003 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-4.ko 2023-11-28T10:59:10.003 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-5.ko 2023-11-28T10:59:10.003 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-6.ko 2023-11-28T10:59:10.003 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-7.ko 2023-11-28T10:59:10.003 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-9.ko 2023-11-28T10:59:10.003 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-r.ko 2023-11-28T10:59:10.004 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-ru.ko 2023-11-28T10:59:10.004 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-u.ko 2023-11-28T10:59:10.004 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_ucs2_utils.ko 2023-11-28T10:59:10.004 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_utf8.ko 2023-11-28T10:59:10.004 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2 2023-11-28T10:59:10.004 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/cluster 2023-11-28T10:59:10.004 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-11-28T10:59:10.004 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlm 2023-11-28T10:59:10.004 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-11-28T10:59:10.004 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlmfs 2023-11-28T10:59:10.004 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-11-28T10:59:10.004 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2.ko 2023-11-28T10:59:10.005 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-11-28T10:59:10.005 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-11-28T10:59:10.005 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-11-28T10:59:10.005 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/overlayfs 2023-11-28T10:59:10.005 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/overlayfs/overlay.ko 2023-11-28T10:59:10.005 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/pstore 2023-11-28T10:59:10.005 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/pstore/ramoops.ko 2023-11-28T10:59:10.005 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/reiserfs 2023-11-28T10:59:10.005 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/reiserfs/reiserfs.ko 2023-11-28T10:59:10.005 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/romfs 2023-11-28T10:59:10.005 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/romfs/romfs.ko 2023-11-28T10:59:10.006 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb 2023-11-28T10:59:10.006 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/client 2023-11-28T10:59:10.006 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/client/cifs.ko 2023-11-28T10:59:10.006 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common 2023-11-28T10:59:10.006 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/cifs_arc4.ko 2023-11-28T10:59:10.006 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/cifs_md4.ko 2023-11-28T10:59:10.006 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/squashfs 2023-11-28T10:59:10.006 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/squashfs/squashfs.ko 2023-11-28T10:59:10.006 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/sysv 2023-11-28T10:59:10.006 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/sysv/sysv.ko 2023-11-28T10:59:10.007 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/udf 2023-11-28T10:59:10.007 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/udf/udf.ko 2023-11-28T10:59:10.007 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ufs 2023-11-28T10:59:10.007 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ufs/ufs.ko 2023-11-28T10:59:10.007 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/xfs 2023-11-28T10:59:10.007 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/xfs/xfs.ko 2023-11-28T10:59:10.007 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel 2023-11-28T10:59:10.007 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/rcu 2023-11-28T10:59:10.007 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/rcu/rcutorture.ko 2023-11-28T10:59:10.007 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/torture.ko 2023-11-28T10:59:10.007 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/trace 2023-11-28T10:59:10.008 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-11-28T10:59:10.008 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib 2023-11-28T10:59:10.008 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/asn1_encoder.ko 2023-11-28T10:59:10.008 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc-itu-t.ko 2023-11-28T10:59:10.008 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc8.ko 2023-11-28T10:59:10.008 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto 2023-11-28T10:59:10.008 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/libarc4.ko 2023-11-28T10:59:10.008 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/libdes.ko 2023-11-28T10:59:10.008 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/libcrc32c.ko 2023-11-28T10:59:10.008 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lru_cache.ko 2023-11-28T10:59:10.008 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4 2023-11-28T10:59:10.009 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/lz4_compress.ko 2023-11-28T10:59:10.009 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/lz4hc_compress.ko 2023-11-28T10:59:10.009 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/math 2023-11-28T10:59:10.009 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/math/cordic.ko 2023-11-28T10:59:10.009 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/raid6 2023-11-28T10:59:10.009 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/raid6/raid6_pq.ko 2023-11-28T10:59:10.009 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/reed_solomon 2023-11-28T10:59:10.009 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/reed_solomon/reed_solomon.ko 2023-11-28T10:59:10.009 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_bm.ko 2023-11-28T10:59:10.009 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_fsm.ko 2023-11-28T10:59:10.009 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_kmp.ko 2023-11-28T10:59:10.009 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/zstd 2023-11-28T10:59:10.010 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/zstd/zstd_compress.ko 2023-11-28T10:59:10.010 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/mm 2023-11-28T10:59:10.010 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/mm/hwpoison-inject.ko 2023-11-28T10:59:10.010 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net 2023-11-28T10:59:10.010 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan 2023-11-28T10:59:10.010 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/6lowpan.ko 2023-11-28T10:59:10.010 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_dest.ko 2023-11-28T10:59:10.010 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_fragment.ko 2023-11-28T10:59:10.010 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_hop.ko 2023-11-28T10:59:10.010 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_ipv6.ko 2023-11-28T10:59:10.010 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_mobility.ko 2023-11-28T10:59:10.011 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_routing.ko 2023-11-28T10:59:10.011 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_udp.ko 2023-11-28T10:59:10.011 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802 2023-11-28T10:59:10.011 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/garp.ko 2023-11-28T10:59:10.011 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/mrp.ko 2023-11-28T10:59:10.011 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/p8022.ko 2023-11-28T10:59:10.011 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/psnap.ko 2023-11-28T10:59:10.011 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/stp.ko 2023-11-28T10:59:10.011 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/8021q 2023-11-28T10:59:10.011 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/8021q/8021q.ko 2023-11-28T10:59:10.011 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p 2023-11-28T10:59:10.011 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet.ko 2023-11-28T10:59:10.012 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_fd.ko 2023-11-28T10:59:10.012 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_rdma.ko 2023-11-28T10:59:10.012 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_virtio.ko 2023-11-28T10:59:10.012 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/batman-adv 2023-11-28T10:59:10.012 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/batman-adv/batman-adv.ko 2023-11-28T10:59:10.012 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge 2023-11-28T10:59:10.012 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/br_netfilter.ko 2023-11-28T10:59:10.012 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/bridge.ko 2023-11-28T10:59:10.012 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter 2023-11-28T10:59:10.012 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-11-28T10:59:10.012 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_among.ko 2023-11-28T10:59:10.012 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_arp.ko 2023-11-28T10:59:10.013 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-11-28T10:59:10.013 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-11-28T10:59:10.013 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_ip.ko 2023-11-28T10:59:10.013 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-11-28T10:59:10.013 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_limit.ko 2023-11-28T10:59:10.013 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_log.ko 2023-11-28T10:59:10.013 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_mark.ko 2023-11-28T10:59:10.013 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-11-28T10:59:10.013 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-11-28T10:59:10.013 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-11-28T10:59:10.013 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-11-28T10:59:10.013 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_snat.ko 2023-11-28T10:59:10.014 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_stp.ko 2023-11-28T10:59:10.014 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-11-28T10:59:10.014 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-11-28T10:59:10.014 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-11-28T10:59:10.014 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-11-28T10:59:10.014 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtables.ko 2023-11-28T10:59:10.014 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-11-28T10:59:10.014 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-11-28T10:59:10.014 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-11-28T10:59:10.014 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ceph 2023-11-28T10:59:10.014 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ceph/libceph.ko 2023-11-28T10:59:10.015 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core 2023-11-28T10:59:10.015 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core/failover.ko 2023-11-28T10:59:10.015 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core/pktgen.ko 2023-11-28T10:59:10.015 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp 2023-11-28T10:59:10.015 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp.ko 2023-11-28T10:59:10.015 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_diag.ko 2023-11-28T10:59:10.015 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_ipv4.ko 2023-11-28T10:59:10.015 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_ipv6.ko 2023-11-28T10:59:10.015 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dns_resolver 2023-11-28T10:59:10.015 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dns_resolver/dns_resolver.ko 2023-11-28T10:59:10.016 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa 2023-11-28T10:59:10.016 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/dsa_core.ko 2023-11-28T10:59:10.016 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_brcm.ko 2023-11-28T10:59:10.016 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_dsa.ko 2023-11-28T10:59:10.016 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_none.ko 2023-11-28T10:59:10.016 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_trailer.ko 2023-11-28T10:59:10.016 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154 2023-11-28T10:59:10.017 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/6lowpan 2023-11-28T10:59:10.017 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-11-28T10:59:10.017 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ieee802154.ko 2023-11-28T10:59:10.017 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ieee802154_socket.ko 2023-11-28T10:59:10.017 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4 2023-11-28T10:59:10.017 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ah4.ko 2023-11-28T10:59:10.017 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/esp4.ko 2023-11-28T10:59:10.018 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/fou.ko 2023-11-28T10:59:10.018 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/gre.ko 2023-11-28T10:59:10.018 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/inet_diag.ko 2023-11-28T10:59:10.018 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_gre.ko 2023-11-28T10:59:10.018 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_tunnel.ko 2023-11-28T10:59:10.018 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_vti.ko 2023-11-28T10:59:10.018 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ipcomp.ko 2023-11-28T10:59:10.019 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ipip.ko 2023-11-28T10:59:10.019 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter 2023-11-28T10:59:10.019 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arp_tables.ko 2023-11-28T10:59:10.019 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-11-28T10:59:10.019 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-11-28T10:59:10.019 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-11-28T10:59:10.019 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-11-28T10:59:10.019 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-11-28T10:59:10.020 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-11-28T10:59:10.020 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-11-28T10:59:10.020 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-11-28T10:59:10.020 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-11-28T10:59:10.020 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_security.ko 2023-11-28T10:59:10.020 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-11-28T10:59:10.020 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-11-28T10:59:10.020 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-11-28T10:59:10.021 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-11-28T10:59:10.021 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-11-28T10:59:10.021 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-11-28T10:59:10.021 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-11-28T10:59:10.021 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-11-28T10:59:10.021 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-11-28T10:59:10.022 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-11-28T10:59:10.022 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_bic.ko 2023-11-28T10:59:10.022 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_dctcp.ko 2023-11-28T10:59:10.022 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_diag.ko 2023-11-28T10:59:10.023 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_highspeed.ko 2023-11-28T10:59:10.023 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_htcp.ko 2023-11-28T10:59:10.023 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_hybla.ko 2023-11-28T10:59:10.023 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_illinois.ko 2023-11-28T10:59:10.024 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_lp.ko 2023-11-28T10:59:10.024 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_scalable.ko 2023-11-28T10:59:10.024 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_vegas.ko 2023-11-28T10:59:10.024 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_veno.ko 2023-11-28T10:59:10.024 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_westwood.ko 2023-11-28T10:59:10.024 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_yeah.ko 2023-11-28T10:59:10.024 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tunnel4.ko 2023-11-28T10:59:10.024 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/udp_diag.ko 2023-11-28T10:59:10.025 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/udp_tunnel.ko 2023-11-28T10:59:10.025 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/xfrm4_tunnel.ko 2023-11-28T10:59:10.025 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6 2023-11-28T10:59:10.025 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ah6.ko 2023-11-28T10:59:10.025 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/esp6.ko 2023-11-28T10:59:10.025 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/fou6.ko 2023-11-28T10:59:10.025 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_tunnel.ko 2023-11-28T10:59:10.026 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-11-28T10:59:10.026 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_vti.ko 2023-11-28T10:59:10.026 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ipcomp6.ko 2023-11-28T10:59:10.026 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter 2023-11-28T10:59:10.026 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-11-28T10:59:10.026 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-11-28T10:59:10.026 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-11-28T10:59:10.027 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-11-28T10:59:10.027 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-11-28T10:59:10.027 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-11-28T10:59:10.027 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-11-28T10:59:10.027 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-11-28T10:59:10.027 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-11-28T10:59:10.027 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-11-28T10:59:10.028 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-11-28T10:59:10.028 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-11-28T10:59:10.028 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-11-28T10:59:10.028 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-11-28T10:59:10.028 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-11-28T10:59:10.028 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-11-28T10:59:10.028 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-11-28T10:59:10.028 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-11-28T10:59:10.028 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-11-28T10:59:10.028 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-11-28T10:59:10.029 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-11-28T10:59:10.029 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-11-28T10:59:10.029 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-11-28T10:59:10.029 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-11-28T10:59:10.029 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/sit.ko 2023-11-28T10:59:10.029 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/tunnel6.ko 2023-11-28T10:59:10.029 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/xfrm6_tunnel.ko 2023-11-28T10:59:10.029 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/key 2023-11-28T10:59:10.029 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/key/af_key.ko 2023-11-28T10:59:10.029 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp 2023-11-28T10:59:10.029 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_core.ko 2023-11-28T10:59:10.030 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_debugfs.ko 2023-11-28T10:59:10.030 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_eth.ko 2023-11-28T10:59:10.030 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ip.ko 2023-11-28T10:59:10.030 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ip6.ko 2023-11-28T10:59:10.030 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_netlink.ko 2023-11-28T10:59:10.030 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ppp.ko 2023-11-28T10:59:10.030 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc 2023-11-28T10:59:10.030 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc/llc.ko 2023-11-28T10:59:10.030 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mac802154 2023-11-28T10:59:10.030 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mac802154/mac802154.ko 2023-11-28T10:59:10.030 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mpls 2023-11-28T10:59:10.031 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mpls/mpls_gso.ko 2023-11-28T10:59:10.031 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter 2023-11-28T10:59:10.031 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset 2023-11-28T10:59:10.031 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set.ko 2023-11-28T10:59:10.031 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-11-28T10:59:10.031 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-11-28T10:59:10.031 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-11-28T10:59:10.031 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-11-28T10:59:10.031 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-11-28T10:59:10.031 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-11-28T10:59:10.032 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-11-28T10:59:10.032 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-11-28T10:59:10.032 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-11-28T10:59:10.032 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-11-28T10:59:10.032 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-11-28T10:59:10.032 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-11-28T10:59:10.032 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-11-28T10:59:10.032 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-11-28T10:59:10.032 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-11-28T10:59:10.032 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs 2023-11-28T10:59:10.032 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs.ko 2023-11-28T10:59:10.033 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-11-28T10:59:10.033 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-11-28T10:59:10.033 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-11-28T10:59:10.033 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-11-28T10:59:10.033 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-11-28T10:59:10.033 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-11-28T10:59:10.033 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-11-28T10:59:10.033 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-11-28T10:59:10.033 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-11-28T10:59:10.033 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-11-28T10:59:10.034 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-11-28T10:59:10.034 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-11-28T10:59:10.034 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-11-28T10:59:10.034 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conncount.ko 2023-11-28T10:59:10.034 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack.ko 2023-11-28T10:59:10.034 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-11-28T10:59:10.034 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-11-28T10:59:10.035 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-11-28T10:59:10.035 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_h323.ko 2023-11-28T10:59:10.035 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_irc.ko 2023-11-28T10:59:10.035 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-11-28T10:59:10.035 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-11-28T10:59:10.035 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-11-28T10:59:10.035 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_sane.ko 2023-11-28T10:59:10.035 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_sip.ko 2023-11-28T10:59:10.035 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-11-28T10:59:10.035 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-11-28T10:59:10.035 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_dup_netdev.ko 2023-11-28T10:59:10.035 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_flow_table.ko 2023-11-28T10:59:10.036 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_flow_table_inet.ko 2023-11-28T10:59:10.036 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_log_syslog.ko 2023-11-28T10:59:10.036 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat.ko 2023-11-28T10:59:10.036 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_amanda.ko 2023-11-28T10:59:10.036 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_ftp.ko 2023-11-28T10:59:10.036 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_irc.ko 2023-11-28T10:59:10.036 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_sip.ko 2023-11-28T10:59:10.036 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_tftp.ko 2023-11-28T10:59:10.036 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_synproxy_core.ko 2023-11-28T10:59:10.036 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_tables.ko 2023-11-28T10:59:10.036 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink.ko 2023-11-28T10:59:10.036 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_acct.ko 2023-11-28T10:59:10.037 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_log.ko 2023-11-28T10:59:10.037 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_osf.ko 2023-11-28T10:59:10.037 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_queue.ko 2023-11-28T10:59:10.037 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_chain_nat.ko 2023-11-28T10:59:10.037 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_compat.ko 2023-11-28T10:59:10.037 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_ct.ko 2023-11-28T10:59:10.037 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_dup_netdev.ko 2023-11-28T10:59:10.037 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_fib.ko 2023-11-28T10:59:10.037 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_fwd_netdev.ko 2023-11-28T10:59:10.037 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_hash.ko 2023-11-28T10:59:10.037 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_limit.ko 2023-11-28T10:59:10.037 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_log.ko 2023-11-28T10:59:10.038 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_masq.ko 2023-11-28T10:59:10.038 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_nat.ko 2023-11-28T10:59:10.038 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_numgen.ko 2023-11-28T10:59:10.038 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_queue.ko 2023-11-28T10:59:10.038 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_quota.ko 2023-11-28T10:59:10.038 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_redir.ko 2023-11-28T10:59:10.038 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_reject.ko 2023-11-28T10:59:10.038 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_reject_inet.ko 2023-11-28T10:59:10.038 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_socket.ko 2023-11-28T10:59:10.038 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_tunnel.ko 2023-11-28T10:59:10.038 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_AUDIT.ko 2023-11-28T10:59:10.039 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CHECKSUM.ko 2023-11-28T10:59:10.039 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CLASSIFY.ko 2023-11-28T10:59:10.039 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-11-28T10:59:10.039 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CT.ko 2023-11-28T10:59:10.039 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_DSCP.ko 2023-11-28T10:59:10.039 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_HL.ko 2023-11-28T10:59:10.039 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_HMARK.ko 2023-11-28T10:59:10.039 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_IDLETIMER.ko 2023-11-28T10:59:10.039 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_LED.ko 2023-11-28T10:59:10.039 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_LOG.ko 2023-11-28T10:59:10.039 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_MASQUERADE.ko 2023-11-28T10:59:10.040 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NETMAP.ko 2023-11-28T10:59:10.040 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NFLOG.ko 2023-11-28T10:59:10.040 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NFQUEUE.ko 2023-11-28T10:59:10.040 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_RATEEST.ko 2023-11-28T10:59:10.040 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_REDIRECT.ko 2023-11-28T10:59:10.040 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_SECMARK.ko 2023-11-28T10:59:10.040 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TCPMSS.ko 2023-11-28T10:59:10.040 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-11-28T10:59:10.040 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TEE.ko 2023-11-28T10:59:10.040 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TPROXY.ko 2023-11-28T10:59:10.040 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TRACE.ko 2023-11-28T10:59:10.040 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_addrtype.ko 2023-11-28T10:59:10.040 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_bpf.ko 2023-11-28T10:59:10.041 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cgroup.ko 2023-11-28T10:59:10.041 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cluster.ko 2023-11-28T10:59:10.041 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_comment.ko 2023-11-28T10:59:10.041 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connbytes.ko 2023-11-28T10:59:10.041 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connlabel.ko 2023-11-28T10:59:10.041 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connlimit.ko 2023-11-28T10:59:10.041 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connmark.ko 2023-11-28T10:59:10.041 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_conntrack.ko 2023-11-28T10:59:10.041 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cpu.ko 2023-11-28T10:59:10.041 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_dccp.ko 2023-11-28T10:59:10.041 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_devgroup.ko 2023-11-28T10:59:10.042 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_dscp.ko 2023-11-28T10:59:10.042 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ecn.ko 2023-11-28T10:59:10.042 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_esp.ko 2023-11-28T10:59:10.042 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_hashlimit.ko 2023-11-28T10:59:10.042 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_helper.ko 2023-11-28T10:59:10.042 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_hl.ko 2023-11-28T10:59:10.042 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ipcomp.ko 2023-11-28T10:59:10.042 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_iprange.ko 2023-11-28T10:59:10.042 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ipvs.ko 2023-11-28T10:59:10.042 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_l2tp.ko 2023-11-28T10:59:10.042 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_length.ko 2023-11-28T10:59:10.042 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_limit.ko 2023-11-28T10:59:10.043 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_mac.ko 2023-11-28T10:59:10.043 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_mark.ko 2023-11-28T10:59:10.043 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_multiport.ko 2023-11-28T10:59:10.043 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_nat.ko 2023-11-28T10:59:10.043 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_nfacct.ko 2023-11-28T10:59:10.043 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_osf.ko 2023-11-28T10:59:10.043 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_owner.ko 2023-11-28T10:59:10.043 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_physdev.ko 2023-11-28T10:59:10.043 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_pkttype.ko 2023-11-28T10:59:10.043 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_policy.ko 2023-11-28T10:59:10.043 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_quota.ko 2023-11-28T10:59:10.043 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_rateest.ko 2023-11-28T10:59:10.044 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_realm.ko 2023-11-28T10:59:10.044 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_recent.ko 2023-11-28T10:59:10.044 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_sctp.ko 2023-11-28T10:59:10.044 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_set.ko 2023-11-28T10:59:10.044 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_socket.ko 2023-11-28T10:59:10.044 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_state.ko 2023-11-28T10:59:10.044 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_statistic.ko 2023-11-28T10:59:10.044 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_string.ko 2023-11-28T10:59:10.044 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_tcpmss.ko 2023-11-28T10:59:10.044 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_time.ko 2023-11-28T10:59:10.044 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_u32.ko 2023-11-28T10:59:10.044 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netlink 2023-11-28T10:59:10.045 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netlink/netlink_diag.ko 2023-11-28T10:59:10.045 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/nsh 2023-11-28T10:59:10.045 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/nsh/nsh.ko 2023-11-28T10:59:10.045 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch 2023-11-28T10:59:10.045 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/openvswitch.ko 2023-11-28T10:59:10.045 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-geneve.ko 2023-11-28T10:59:10.045 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-gre.ko 2023-11-28T10:59:10.045 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-vxlan.ko 2023-11-28T10:59:10.045 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/packet 2023-11-28T10:59:10.045 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/packet/af_packet_diag.ko 2023-11-28T10:59:10.045 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds 2023-11-28T10:59:10.045 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds.ko 2023-11-28T10:59:10.046 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds_rdma.ko 2023-11-28T10:59:10.046 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds_tcp.ko 2023-11-28T10:59:10.046 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched 2023-11-28T10:59:10.046 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_csum.ko 2023-11-28T10:59:10.046 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_gact.ko 2023-11-28T10:59:10.046 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_ipt.ko 2023-11-28T10:59:10.046 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_mirred.ko 2023-11-28T10:59:10.046 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_nat.ko 2023-11-28T10:59:10.046 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_pedit.ko 2023-11-28T10:59:10.046 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_police.ko 2023-11-28T10:59:10.046 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_simple.ko 2023-11-28T10:59:10.046 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_skbedit.ko 2023-11-28T10:59:10.047 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_vlan.ko 2023-11-28T10:59:10.047 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_basic.ko 2023-11-28T10:59:10.047 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_bpf.ko 2023-11-28T10:59:10.047 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_flow.ko 2023-11-28T10:59:10.047 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_fw.ko 2023-11-28T10:59:10.047 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_route.ko 2023-11-28T10:59:10.047 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_u32.ko 2023-11-28T10:59:10.047 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_cmp.ko 2023-11-28T10:59:10.048 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_ipset.ko 2023-11-28T10:59:10.048 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_meta.ko 2023-11-28T10:59:10.048 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_nbyte.ko 2023-11-28T10:59:10.048 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_text.ko 2023-11-28T10:59:10.048 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_u32.ko 2023-11-28T10:59:10.048 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_choke.ko 2023-11-28T10:59:10.048 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_codel.ko 2023-11-28T10:59:10.048 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_drr.ko 2023-11-28T10:59:10.048 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_fq.ko 2023-11-28T10:59:10.049 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_gred.ko 2023-11-28T10:59:10.049 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_hfsc.ko 2023-11-28T10:59:10.049 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_hhf.ko 2023-11-28T10:59:10.049 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_htb.ko 2023-11-28T10:59:10.049 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_ingress.ko 2023-11-28T10:59:10.049 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_mqprio.ko 2023-11-28T10:59:10.049 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_mqprio_lib.ko 2023-11-28T10:59:10.049 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_multiq.ko 2023-11-28T10:59:10.049 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_netem.ko 2023-11-28T10:59:10.049 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_pie.ko 2023-11-28T10:59:10.049 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_plug.ko 2023-11-28T10:59:10.050 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_prio.ko 2023-11-28T10:59:10.050 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_qfq.ko 2023-11-28T10:59:10.050 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_red.ko 2023-11-28T10:59:10.050 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_sfb.ko 2023-11-28T10:59:10.050 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_sfq.ko 2023-11-28T10:59:10.050 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_tbf.ko 2023-11-28T10:59:10.050 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_teql.ko 2023-11-28T10:59:10.050 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp 2023-11-28T10:59:10.050 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/sctp.ko 2023-11-28T10:59:10.050 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/sctp_diag.ko 2023-11-28T10:59:10.050 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc 2023-11-28T10:59:10.050 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss 2023-11-28T10:59:10.051 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-11-28T10:59:10.051 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-11-28T10:59:10.051 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/sunrpc.ko 2023-11-28T10:59:10.051 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/xprtrdma 2023-11-28T10:59:10.051 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-11-28T10:59:10.051 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc 2023-11-28T10:59:10.051 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/diag.ko 2023-11-28T10:59:10.051 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/tipc.ko 2023-11-28T10:59:10.051 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/unix 2023-11-28T10:59:10.051 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/unix/unix_diag.ko 2023-11-28T10:59:10.051 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock 2023-11-28T10:59:10.051 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-11-28T10:59:10.052 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-11-28T10:59:10.052 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock.ko 2023-11-28T10:59:10.052 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock_diag.ko 2023-11-28T10:59:10.052 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock_loopback.ko 2023-11-28T10:59:10.052 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm 2023-11-28T10:59:10.052 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm/xfrm_ipcomp.ko 2023-11-28T10:59:10.052 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security 2023-11-28T10:59:10.052 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys 2023-11-28T10:59:10.052 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/encrypted-keys 2023-11-28T10:59:10.052 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-11-28T10:59:10.052 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/trusted-keys 2023-11-28T10:59:10.052 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/trusted-keys/trusted.ko 2023-11-28T10:59:10.053 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt 2023-11-28T10:59:10.053 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt/lib 2023-11-28T10:59:10.053 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt/lib/irqbypass.ko 2023-11-28T10:59:10.053 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.alias 2023-11-28T10:59:10.053 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.alias.bin 2023-11-28T10:59:10.053 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin 2023-11-28T10:59:10.053 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin.bin 2023-11-28T10:59:10.053 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin.modinfo 2023-11-28T10:59:10.053 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.dep 2023-11-28T10:59:10.053 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.dep.bin 2023-11-28T10:59:10.053 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.devname 2023-11-28T10:59:10.053 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.order 2023-11-28T10:59:10.054 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.softdep 2023-11-28T10:59:10.054 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.symbols 2023-11-28T10:59:10.054 INFO:teuthology.orchestra.run.smithi092.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.symbols.bin 2023-11-28T10:59:10.054 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-g8247bbb5b470 2023-11-28T10:59:10.054 DEBUG:teuthology.orchestra.run.smithi092:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-11-28T10:59:10.083 INFO:teuthology.orchestra.run.smithi092.stdout:postinstall scriptlet (using /bin/sh): 2023-11-28T10:59:10.084 INFO:teuthology.orchestra.run.smithi092.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-g8247bbb5b470 -a -r /boot/System.map-6.6.0-g8247bbb5b470 ]; then 2023-11-28T10:59:10.084 INFO:teuthology.orchestra.run.smithi092.stdout:cp /boot/vmlinuz-6.6.0-g8247bbb5b470 /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm 2023-11-28T10:59:10.084 INFO:teuthology.orchestra.run.smithi092.stdout:cp /boot/System.map-6.6.0-g8247bbb5b470 /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-28T10:59:10.084 INFO:teuthology.orchestra.run.smithi092.stdout:rm -f /boot/vmlinuz-6.6.0-g8247bbb5b470 /boot/System.map-6.6.0-g8247bbb5b470 2023-11-28T10:59:10.084 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/installkernel 6.6.0-g8247bbb5b470 /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-28T10:59:10.084 INFO:teuthology.orchestra.run.smithi092.stdout:rm -f /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-28T10:59:10.084 INFO:teuthology.orchestra.run.smithi092.stdout:fi 2023-11-28T10:59:10.084 INFO:teuthology.orchestra.run.smithi092.stdout:preuninstall scriptlet (using /bin/sh): 2023-11-28T10:59:10.084 INFO:teuthology.orchestra.run.smithi092.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-11-28T10:59:10.084 INFO:teuthology.orchestra.run.smithi092.stdout:new-kernel-pkg --remove 6.6.0-g8247bbb5b470 --rminitrd --initrdfile=/boot/initramfs-6.6.0-g8247bbb5b470.img 2023-11-28T10:59:10.085 INFO:teuthology.orchestra.run.smithi092.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-11-28T10:59:10.085 INFO:teuthology.orchestra.run.smithi092.stdout:kernel-install remove 6.6.0-g8247bbb5b470 2023-11-28T10:59:10.085 INFO:teuthology.orchestra.run.smithi092.stdout:fi 2023-11-28T10:59:10.085 INFO:teuthology.orchestra.run.smithi092.stdout:postuninstall scriptlet (using /bin/sh): 2023-11-28T10:59:10.085 INFO:teuthology.orchestra.run.smithi092.stdout:if [ -x /sbin/update-bootloader ]; then 2023-11-28T10:59:10.085 INFO:teuthology.orchestra.run.smithi092.stdout:/sbin/update-bootloader --remove 6.6.0-g8247bbb5b470 2023-11-28T10:59:10.085 INFO:teuthology.orchestra.run.smithi092.stdout:fi 2023-11-28T10:59:10.085 DEBUG:teuthology.orchestra.run.smithi092:> sudo rpm -qi grub2-tools 2023-11-28T10:59:10.163 INFO:teuthology.orchestra.run.smithi092.stdout:Name : grub2-tools 2023-11-28T10:59:10.163 INFO:teuthology.orchestra.run.smithi092.stdout:Epoch : 1 2023-11-28T10:59:10.163 INFO:teuthology.orchestra.run.smithi092.stdout:Version : 2.02 2023-11-28T10:59:10.163 INFO:teuthology.orchestra.run.smithi092.stdout:Release : 129.el8 2023-11-28T10:59:10.163 INFO:teuthology.orchestra.run.smithi092.stdout:Architecture: x86_64 2023-11-28T10:59:10.163 INFO:teuthology.orchestra.run.smithi092.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2023-11-28T10:59:10.164 INFO:teuthology.orchestra.run.smithi092.stdout:Group : System Environment/Base 2023-11-28T10:59:10.164 INFO:teuthology.orchestra.run.smithi092.stdout:Size : 9265586 2023-11-28T10:59:10.164 INFO:teuthology.orchestra.run.smithi092.stdout:License : GPLv3+ 2023-11-28T10:59:10.164 INFO:teuthology.orchestra.run.smithi092.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-11-28T10:59:10.164 INFO:teuthology.orchestra.run.smithi092.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-11-28T10:59:10.164 INFO:teuthology.orchestra.run.smithi092.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-11-28T10:59:10.164 INFO:teuthology.orchestra.run.smithi092.stdout:Build Host : kbuilder.bsys.centos.org 2023-11-28T10:59:10.164 INFO:teuthology.orchestra.run.smithi092.stdout:Relocations : (not relocatable) 2023-11-28T10:59:10.164 INFO:teuthology.orchestra.run.smithi092.stdout:Packager : CentOS BuildSystem 2023-11-28T10:59:10.164 INFO:teuthology.orchestra.run.smithi092.stdout:Vendor : CentOS 2023-11-28T10:59:10.164 INFO:teuthology.orchestra.run.smithi092.stdout:URL : http://www.gnu.org/software/grub/ 2023-11-28T10:59:10.165 INFO:teuthology.orchestra.run.smithi092.stdout:Summary : Support tools for GRUB. 2023-11-28T10:59:10.165 INFO:teuthology.orchestra.run.smithi092.stdout:Description : 2023-11-28T10:59:10.165 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T10:59:10.165 INFO:teuthology.orchestra.run.smithi092.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-11-28T10:59:10.165 INFO:teuthology.orchestra.run.smithi092.stdout:customizable bootloader with modular architecture. It supports a rich 2023-11-28T10:59:10.165 INFO:teuthology.orchestra.run.smithi092.stdout:variety of kernel formats, file systems, computer architectures and 2023-11-28T10:59:10.165 INFO:teuthology.orchestra.run.smithi092.stdout:hardware devices. 2023-11-28T10:59:10.165 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T10:59:10.165 INFO:teuthology.orchestra.run.smithi092.stdout:This subpackage provides tools for support of all platforms. 2023-11-28T10:59:10.166 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-11-28T10:59:10.166 INFO:teuthology.task.kernel:Updating grub on smithi092 to boot 6.6.0-g8247bbb5b470 2023-11-28T10:59:10.166 DEBUG:teuthology.orchestra.run.smithi092:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-11-28T10:59:12.743 INFO:teuthology.orchestra.run.smithi092.stderr:Generating grub configuration file ... 2023-11-28T10:59:14.960 INFO:teuthology.orchestra.run.smithi092.stderr:done 2023-11-28T10:59:14.962 DEBUG:teuthology.orchestra.run.smithi092:> mktemp 2023-11-28T10:59:14.993 INFO:teuthology.orchestra.run.smithi092.stdout:/tmp/tmp.hmIvtgkAUp 2023-11-28T10:59:14.994 DEBUG:teuthology.orchestra.run.smithi092:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.hmIvtgkAUp 2023-11-28T10:59:15.071 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0666 /tmp/tmp.hmIvtgkAUp 2023-11-28T10:59:15.225 DEBUG:teuthology.orchestra.remote:smithi092:/tmp/tmp.hmIvtgkAUp is 6KB 2023-11-28T10:59:15.272 DEBUG:teuthology.orchestra.run.smithi092:> rm -fr /tmp/tmp.hmIvtgkAUp 2023-11-28T10:59:15.291 DEBUG:teuthology.orchestra.run.smithi092:> sudo /bin/ls /boot/loader/entries || true 2023-11-28T10:59:15.364 INFO:teuthology.orchestra.run.smithi092.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2023-11-28T10:59:15.365 INFO:teuthology.orchestra.run.smithi092.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2023-11-28T10:59:15.365 INFO:teuthology.orchestra.run.smithi092.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2023-11-28T10:59:15.365 INFO:teuthology.orchestra.run.smithi092.stdout:a1b29d6c289747bdb22930598d39874a-6.6.0-g8247bbb5b470.conf 2023-11-28T10:59:15.366 DEBUG:teuthology.orchestra.run.smithi092:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-6.6.0-g8247bbb5b470 2023-11-28T10:59:15.521 DEBUG:teuthology.orchestra.run.smithi092:> sudo shutdown -r now 2023-11-28T10:59:17.666 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2023-11-28T10:59:17.666 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2023-11-28T10:59:17.667 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T10:59:36.166 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.169 2023-11-28T10:59:45.174 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2023-11-28T10:59:45.175 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T10:59:45.525 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-11-28T10:59:45.525 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2023-11-28T10:59:45.526 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T10:59:48.230 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.169 2023-11-28T11:00:00.242 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2023-11-28T11:00:00.243 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T11:00:04.066 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.92 2023-11-28T11:00:13.074 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2023-11-28T11:00:13.075 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T11:00:16.130 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.92 2023-11-28T11:00:18.658 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.169 2023-11-28T11:00:28.140 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2023-11-28T11:00:28.141 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T11:00:33.662 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2023-11-28T11:00:33.663 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T11:00:33.944 DEBUG:teuthology.orchestra.run.smithi169:> true 2023-11-28T11:00:34.395 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi169.front.sepia.ceph.com' 2023-11-28T11:00:34.395 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.6.0-g8247bbb5b470"... 2023-11-28T11:00:34.395 DEBUG:teuthology.orchestra.run.smithi169:> uname -r 2023-11-28T11:00:34.441 INFO:teuthology.orchestra.run.smithi169.stdout:6.6.0-g8247bbb5b470 2023-11-28T11:00:34.442 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-g8247bbb5b470 vs 6.6.0-g8247bbb5b470 2023-11-28T11:00:34.442 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-28T11:00:34.442 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-11-28T11:00:35.443 DEBUG:teuthology.parallel:result is None 2023-11-28T11:00:59.932 DEBUG:teuthology.orchestra.run.smithi092:> true 2023-11-28T11:01:00.336 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi092.front.sepia.ceph.com' 2023-11-28T11:01:00.336 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.6.0-g8247bbb5b470"... 2023-11-28T11:01:00.336 DEBUG:teuthology.orchestra.run.smithi092:> uname -r 2023-11-28T11:01:00.381 INFO:teuthology.orchestra.run.smithi092.stdout:6.6.0-g8247bbb5b470 2023-11-28T11:01:00.381 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-g8247bbb5b470 vs 6.6.0-g8247bbb5b470 2023-11-28T11:01:00.382 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-28T11:01:00.382 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-11-28T11:01:01.382 DEBUG:teuthology.parallel:result is None 2023-11-28T11:01:01.383 INFO:teuthology.run_tasks:Running task internal.base... 2023-11-28T11:01:01.391 INFO:teuthology.task.internal:Creating test directory... 2023-11-28T11:01:01.391 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-28T11:01:01.396 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-28T11:01:01.448 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-11-28T11:01:01.455 INFO:teuthology.run_tasks:Running task internal.archive... 2023-11-28T11:01:01.463 INFO:teuthology.task.internal:Creating archive directory... 2023-11-28T11:01:01.463 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-28T11:01:01.468 DEBUG:teuthology.orchestra.run.smithi169:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-28T11:01:01.535 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-11-28T11:01:01.541 INFO:teuthology.task.internal:Enabling coredump saving... 2023-11-28T11:01:01.541 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-28T11:01:01.547 DEBUG:teuthology.orchestra.run.smithi169:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-28T11:01:01.671 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-28T11:01:01.703 INFO:teuthology.orchestra.run.smithi169.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-28T11:01:01.729 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-28T11:01:01.762 INFO:teuthology.orchestra.run.smithi169.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-28T11:01:01.765 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-11-28T11:01:01.772 INFO:teuthology.task.internal:Configuring sudo... 2023-11-28T11:01:01.773 DEBUG:teuthology.orchestra.run.smithi092:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-28T11:01:01.777 DEBUG:teuthology.orchestra.run.smithi169:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-28T11:01:01.847 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-11-28T11:01:01.857 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-11-28T11:01:01.857 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-28T11:01:01.861 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-28T11:01:01.907 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-28T11:01:02.020 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-28T11:01:02.090 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-28T11:01:02.164 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-28T11:01:02.234 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-28T11:01:02.234 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-28T11:01:02.303 DEBUG:teuthology.orchestra.run.smithi169:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-28T11:01:02.377 DEBUG:teuthology.orchestra.run.smithi169:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-28T11:01:02.452 DEBUG:teuthology.orchestra.run.smithi169:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-28T11:01:02.527 DEBUG:teuthology.orchestra.run.smithi169:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-28T11:01:02.597 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-28T11:01:02.597 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-28T11:01:02.667 DEBUG:teuthology.orchestra.run.smithi092:> sudo service rsyslog restart 2023-11-28T11:01:02.671 DEBUG:teuthology.orchestra.run.smithi169:> sudo service rsyslog restart 2023-11-28T11:01:02.793 INFO:teuthology.orchestra.run.smithi092.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-28T11:01:02.834 INFO:teuthology.orchestra.run.smithi169.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-28T11:01:03.217 INFO:teuthology.run_tasks:Running task internal.timer... 2023-11-28T11:01:06.352 INFO:teuthology.task.internal:Starting timer... 2023-11-28T11:01:06.352 INFO:teuthology.run_tasks:Running task pcp... 2023-11-28T11:01:06.396 INFO:teuthology.run_tasks:Running task selinux... 2023-11-28T11:01:06.406 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-11-28T11:01:06.406 DEBUG:teuthology.orchestra.run.smithi092:> sudo service auditd rotate 2023-11-28T11:01:06.532 INFO:teuthology.orchestra.run.smithi092.stdout:Rotating logs: 2023-11-28T11:01:06.536 DEBUG:teuthology.orchestra.run.smithi169:> sudo service auditd rotate 2023-11-28T11:01:06.669 INFO:teuthology.orchestra.run.smithi169.stdout:Rotating logs: 2023-11-28T11:01:06.672 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-11-28T11:01:06.673 DEBUG:teuthology.orchestra.run.smithi092:> /usr/sbin/getenforce 2023-11-28T11:01:06.720 INFO:teuthology.orchestra.run.smithi092.stdout:Permissive 2023-11-28T11:01:06.720 DEBUG:teuthology.orchestra.run.smithi169:> /usr/sbin/getenforce 2023-11-28T11:01:06.774 INFO:teuthology.orchestra.run.smithi169.stdout:Permissive 2023-11-28T11:01:06.775 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi092.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi169.front.sepia.ceph.com': 'permissive'} 2023-11-28T11:01:06.775 DEBUG:teuthology.orchestra.run.smithi092:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-28T11:01:06.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:01:06.846 DEBUG:teuthology.orchestra.run.smithi169:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-28T11:01:06.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:01:06.918 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-11-28T11:01:06.919 DEBUG:teuthology.orchestra.run.smithi092:> sudo /usr/sbin/setenforce permissive 2023-11-28T11:01:06.991 DEBUG:teuthology.orchestra.run.smithi169:> sudo /usr/sbin/setenforce permissive 2023-11-28T11:01:07.062 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-11-28T11:01:07.072 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-11-28T11:01:07.072 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-11-28T11:01:07.090 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-11-28T11:01:07.091 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi092.front.sepia.ceph.com,smithi169.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-11-28T11:13:30.617 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi092.front.sepia.ceph.com'), Remote(name='ubuntu@smithi169.front.sepia.ceph.com')] 2023-11-28T11:13:30.618 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2023-11-28T11:13:30.620 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T11:13:30.706 DEBUG:teuthology.orchestra.run.smithi092:> true 2023-11-28T11:13:30.824 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi092.front.sepia.ceph.com' 2023-11-28T11:13:30.825 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2023-11-28T11:13:30.825 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T11:13:30.907 DEBUG:teuthology.orchestra.run.smithi169:> true 2023-11-28T11:13:31.024 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi169.front.sepia.ceph.com' 2023-11-28T11:13:31.024 INFO:teuthology.run_tasks:Running task clock... 2023-11-28T11:13:31.034 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-11-28T11:13:31.035 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-28T11:13:31.035 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-28T11:13:31.039 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-28T11:13:31.040 DEBUG:teuthology.orchestra.run.smithi169:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-28T11:13:31.112 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-28T11:13:31.140 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-28T11:13:31.144 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-28T11:13:31.173 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-28T11:13:31.233 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: ntpd: command not found 2023-11-28T11:13:31.260 INFO:teuthology.orchestra.run.smithi169.stderr:sudo: ntpd: command not found 2023-11-28T11:13:31.261 INFO:teuthology.orchestra.run.smithi092.stdout:506 Cannot talk to daemon 2023-11-28T11:13:31.288 INFO:teuthology.orchestra.run.smithi169.stdout:506 Cannot talk to daemon 2023-11-28T11:13:31.292 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-28T11:13:31.321 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-28T11:13:31.325 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-28T11:13:31.354 INFO:teuthology.orchestra.run.smithi169.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-28T11:13:31.427 INFO:teuthology.orchestra.run.smithi092.stderr:bash: ntpq: command not found 2023-11-28T11:13:31.435 INFO:teuthology.orchestra.run.smithi092.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-28T11:13:31.435 INFO:teuthology.orchestra.run.smithi092.stdout:=============================================================================== 2023-11-28T11:13:31.435 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:13:31.435 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:13:31.435 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:13:31.435 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:13:31.443 INFO:teuthology.orchestra.run.smithi169.stderr:bash: ntpq: command not found 2023-11-28T11:13:31.450 INFO:teuthology.orchestra.run.smithi169.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-28T11:13:31.450 INFO:teuthology.orchestra.run.smithi169.stdout:=============================================================================== 2023-11-28T11:13:31.450 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:13:31.451 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:13:31.451 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:13:31.451 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:13:31.452 INFO:teuthology.run_tasks:Running task install... 2023-11-28T11:13:31.461 DEBUG:teuthology.task.install:project ceph 2023-11-28T11:13:31.461 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232'}} 2023-11-28T11:13:31.461 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232'} 2023-11-28T11:13:31.461 INFO:teuthology.task.install:Using flavor: default 2023-11-28T11:13:31.467 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-11-28T11:13:31.467 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-11-28T11:13:31.468 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232', 'tag': None, 'wait_for_package': False} 2023-11-28T11:13:31.468 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2f5f6028b893ad41357cd06e827373f0fe2a1232 2023-11-28T11:13:31.470 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232', 'tag': None, 'wait_for_package': False} 2023-11-28T11:13:31.470 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2f5f6028b893ad41357cd06e827373f0fe2a1232 2023-11-28T11:13:31.618 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/ 2023-11-28T11:13:31.618 INFO:teuthology.task.install.rpm:Package version is 18.0.0-2697.g2f5f6028 2023-11-28T11:13:31.684 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/ 2023-11-28T11:13:31.684 INFO:teuthology.task.install.rpm:Package version is 18.0.0-2697.g2f5f6028 2023-11-28T11:13:31.791 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-28T11:13:31.791 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-28T11:13:31.791 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-28T11:13:31.854 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-28T11:13:31.854 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-28T11:13:31.854 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-28T11:13:31.860 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-28T11:13:31.861 DEBUG:teuthology.orchestra.run.smithi092:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f5f6028b893ad41357cd06e827373f0fe2a1232/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-28T11:13:31.926 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-28T11:13:31.927 DEBUG:teuthology.orchestra.run.smithi169:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f5f6028b893ad41357cd06e827373f0fe2a1232/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-28T11:13:31.973 DEBUG:teuthology.orchestra.run.smithi092:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-28T11:13:32.042 DEBUG:teuthology.orchestra.run.smithi169:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-28T11:13:32.068 DEBUG:teuthology.orchestra.run.smithi092:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-28T11:13:32.138 INFO:teuthology.orchestra.run.smithi092.stdout:check_obsoletes = 1 2023-11-28T11:13:32.139 DEBUG:teuthology.orchestra.run.smithi169:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-28T11:13:32.140 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum clean all 2023-11-28T11:13:32.209 INFO:teuthology.orchestra.run.smithi169.stdout:check_obsoletes = 1 2023-11-28T11:13:32.212 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum clean all 2023-11-28T11:13:32.491 INFO:teuthology.orchestra.run.smithi092.stdout:76 files removed 2023-11-28T11:13:32.527 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-28T11:13:32.573 INFO:teuthology.orchestra.run.smithi169.stdout:76 files removed 2023-11-28T11:13:32.612 DEBUG:teuthology.orchestra.run.smithi169:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-28T11:13:33.395 INFO:teuthology.orchestra.run.smithi092.stdout:ceph packages for x86_64 247 kB/s | 80 kB 00:00 2023-11-28T11:13:33.707 INFO:teuthology.orchestra.run.smithi169.stdout:ceph packages for x86_64 150 kB/s | 80 kB 00:00 2023-11-28T11:13:33.770 INFO:teuthology.orchestra.run.smithi092.stdout:ceph noarch packages 28 kB/s | 9.4 kB 00:00 2023-11-28T11:13:34.129 INFO:teuthology.orchestra.run.smithi092.stdout:ceph source packages 4.5 kB/s | 1.5 kB 00:00 2023-11-28T11:13:34.141 INFO:teuthology.orchestra.run.smithi169.stdout:ceph noarch packages 24 kB/s | 9.4 kB 00:00 2023-11-28T11:13:34.492 INFO:teuthology.orchestra.run.smithi169.stdout:ceph source packages 4.6 kB/s | 1.5 kB 00:00 2023-11-28T11:15:30.590 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 8 - AppStream 301 kB/s | 34 MB 01:56 2023-11-28T11:15:31.211 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 8 - AppStream 300 kB/s | 34 MB 01:56 2023-11-28T11:15:40.039 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 8 - BaseOS 21 MB/s | 55 MB 00:02 2023-11-28T11:15:40.928 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 8 - BaseOS 19 MB/s | 55 MB 00:02 2023-11-28T11:15:48.830 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 8 - Extras 43 kB/s | 18 kB 00:00 2023-11-28T11:15:49.331 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 8 - Extras common packages 22 kB/s | 6.9 kB 00:00 2023-11-28T11:15:49.519 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 8 - Extras 43 kB/s | 18 kB 00:00 2023-11-28T11:15:50.020 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 8 - Extras common packages 22 kB/s | 6.9 kB 00:00 2023-11-28T11:15:51.607 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 8 - PowerTools 3.1 MB/s | 6.6 MB 00:02 2023-11-28T11:15:52.421 INFO:teuthology.orchestra.run.smithi169.stdout:CentOS Stream 8 - PowerTools 2.9 MB/s | 6.6 MB 00:02 2023-11-28T11:15:53.149 INFO:teuthology.orchestra.run.smithi092.stdout:Copr repo for python3-asyncssh owned by ceph 30 kB/s | 3.5 kB 00:00 2023-11-28T11:15:53.931 INFO:teuthology.orchestra.run.smithi169.stdout:Copr repo for python3-asyncssh owned by ceph 26 kB/s | 3.5 kB 00:00 2023-11-28T11:15:54.350 INFO:teuthology.orchestra.run.smithi092.stdout:Extra Packages for Enterprise Linux 15 MB/s | 16 MB 00:01 2023-11-28T11:15:55.625 INFO:teuthology.orchestra.run.smithi169.stdout:Extra Packages for Enterprise Linux 10 MB/s | 16 MB 00:01 2023-11-28T11:15:58.850 INFO:teuthology.orchestra.run.smithi092.stdout:lab-extras 267 kB/s | 24 kB 00:00 2023-11-28T11:16:00.164 INFO:teuthology.orchestra.run.smithi169.stdout:lab-extras 406 kB/s | 24 kB 00:00 2023-11-28T11:16:03.318 INFO:teuthology.orchestra.run.smithi092.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-28T11:16:03.319 INFO:teuthology.orchestra.run.smithi092.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-28T11:16:03.333 INFO:teuthology.orchestra.run.smithi092.stdout:Package NetworkManager-1:1.40.16-4.el8.x86_64 is already installed. 2023-11-28T11:16:03.333 INFO:teuthology.orchestra.run.smithi092.stdout:Package NetworkManager-initscripts-updown-1:1.40.16-4.el8.noarch is already installed. 2023-11-28T11:16:03.334 INFO:teuthology.orchestra.run.smithi092.stdout:Package iproute-6.2.0-1.el8.x86_64 is already installed. 2023-11-28T11:16:03.335 INFO:teuthology.orchestra.run.smithi092.stdout:Package util-linux-2.32.1-41.el8.x86_64 is already installed. 2023-11-28T11:16:03.336 INFO:teuthology.orchestra.run.smithi092.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-28T11:16:03.338 INFO:teuthology.orchestra.run.smithi092.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-28T11:16:03.341 INFO:teuthology.orchestra.run.smithi092.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-11-28T11:16:03.343 INFO:teuthology.orchestra.run.smithi092.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-28T11:16:03.606 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2023-11-28T11:16:03.620 INFO:teuthology.orchestra.run.smithi092.stdout:==================================================================================================== 2023-11-28T11:16:03.620 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2023-11-28T11:16:03.620 INFO:teuthology.orchestra.run.smithi092.stdout:==================================================================================================== 2023-11-28T11:16:03.621 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2023-11-28T11:16:03.621 INFO:teuthology.orchestra.run.smithi092.stdout: bison x86_64 3.0.4-10.el8 appstream 688 k 2023-11-28T11:16:03.621 INFO:teuthology.orchestra.run.smithi092.stdout: ceph x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 6.4 k 2023-11-28T11:16:03.621 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-base x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 5.4 M 2023-11-28T11:16:03.621 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-fuse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 845 k 2023-11-28T11:16:03.621 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 165 k 2023-11-28T11:16:03.621 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 1.5 M 2023-11-28T11:16:03.621 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-cephadm noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 123 k 2023-11-28T11:16:03.621 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-dashboard noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 1.7 M 2023-11-28T11:16:03.621 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 7.4 M 2023-11-28T11:16:03.621 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-rook noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 47 k 2023-11-28T11:16:03.621 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-radosgw x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 13 M 2023-11-28T11:16:03.621 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-test x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 47 M 2023-11-28T11:16:03.622 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-volume noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 263 k 2023-11-28T11:16:03.622 INFO:teuthology.orchestra.run.smithi092.stdout: cephadm noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 83 k 2023-11-28T11:16:03.622 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-mirror x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 257 k 2023-11-28T11:16:03.622 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-top noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 18 k 2023-11-28T11:16:03.622 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libelf-devel x86_64 0.190-1.el8 baseos 62 k 2023-11-28T11:16:03.622 INFO:teuthology.orchestra.run.smithi092.stdout: flex x86_64 2.6.1-9.el8 appstream 320 k 2023-11-28T11:16:03.622 INFO:teuthology.orchestra.run.smithi092.stdout: libaio-devel x86_64 0.3.112-1.el8 baseos 19 k 2023-11-28T11:16:03.622 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs-devel x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 30 k 2023-11-28T11:16:03.622 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 767 k 2023-11-28T11:16:03.622 INFO:teuthology.orchestra.run.smithi092.stdout: librados-devel x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 139 k 2023-11-28T11:16:03.622 INFO:teuthology.orchestra.run.smithi092.stdout: libtool x86_64 2.4.6-25.el8 appstream 709 k 2023-11-28T11:16:03.622 INFO:teuthology.orchestra.run.smithi092.stdout: libuuid-devel x86_64 2.32.1-43.el8 baseos 100 k 2023-11-28T11:16:03.623 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-devel x86_64 1:1.1.1k-11.el8 baseos 3.2 M 2023-11-28T11:16:03.623 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql x86_64 10.23-2.module_el8+572+929c87ac appstream 1.6 M 2023-11-28T11:16:03.623 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-contrib x86_64 10.23-2.module_el8+572+929c87ac appstream 901 k 2023-11-28T11:16:03.623 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-server x86_64 10.23-2.module_el8+572+929c87ac appstream 5.1 M 2023-11-28T11:16:03.623 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cephfs x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 190 k 2023-11-28T11:16:03.623 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rados x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 371 k 2023-11-28T11:16:03.623 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rbd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 364 k 2023-11-28T11:16:03.623 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rgw x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 107 k 2023-11-28T11:16:03.623 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-fuse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 94 k 2023-11-28T11:16:03.623 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-mirror x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.4 M 2023-11-28T11:16:03.623 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-nbd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 180 k 2023-11-28T11:16:03.623 INFO:teuthology.orchestra.run.smithi092.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 baseos 74 k 2023-11-28T11:16:03.624 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrading: 2023-11-28T11:16:03.624 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-11.el8 baseos 144 k 2023-11-28T11:16:03.624 INFO:teuthology.orchestra.run.smithi092.stdout: acl x86_64 2.2.53-3.el8 baseos 83 k 2023-11-28T11:16:03.624 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-debuginfod-client x86_64 0.190-1.el8 baseos 77 k 2023-11-28T11:16:03.624 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libelf x86_64 0.190-1.el8 baseos 232 k 2023-11-28T11:16:03.624 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libs x86_64 0.190-1.el8 baseos 305 k 2023-11-28T11:16:03.624 INFO:teuthology.orchestra.run.smithi092.stdout: iproute x86_64 6.2.0-5.el8 baseos 854 k 2023-11-28T11:16:03.624 INFO:teuthology.orchestra.run.smithi092.stdout: libacl x86_64 2.2.53-3.el8 baseos 35 k 2023-11-28T11:16:03.624 INFO:teuthology.orchestra.run.smithi092.stdout: libacl-devel x86_64 2.2.53-3.el8 baseos 85 k 2023-11-28T11:16:03.624 INFO:teuthology.orchestra.run.smithi092.stdout: libblkid x86_64 2.32.1-43.el8 baseos 220 k 2023-11-28T11:16:03.624 INFO:teuthology.orchestra.run.smithi092.stdout: libfdisk x86_64 2.32.1-43.el8 baseos 254 k 2023-11-28T11:16:03.624 INFO:teuthology.orchestra.run.smithi092.stdout: libmount x86_64 2.32.1-43.el8 baseos 236 k 2023-11-28T11:16:03.625 INFO:teuthology.orchestra.run.smithi092.stdout: librados2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.7 M 2023-11-28T11:16:03.625 INFO:teuthology.orchestra.run.smithi092.stdout: librbd1 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.6 M 2023-11-28T11:16:03.625 INFO:teuthology.orchestra.run.smithi092.stdout: libsmartcols x86_64 2.32.1-43.el8 baseos 179 k 2023-11-28T11:16:03.625 INFO:teuthology.orchestra.run.smithi092.stdout: libuuid x86_64 2.32.1-43.el8 baseos 99 k 2023-11-28T11:16:03.625 INFO:teuthology.orchestra.run.smithi092.stdout: openssl x86_64 1:1.1.1k-11.el8 baseos 737 k 2023-11-28T11:16:03.625 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-libs x86_64 1:1.1.1k-11.el8 baseos 1.5 M 2023-11-28T11:16:03.625 INFO:teuthology.orchestra.run.smithi092.stdout: util-linux x86_64 2.32.1-43.el8 baseos 2.5 M 2023-11-28T11:16:03.625 INFO:teuthology.orchestra.run.smithi092.stdout: util-linux-user x86_64 2.32.1-43.el8 baseos 103 k 2023-11-28T11:16:03.625 INFO:teuthology.orchestra.run.smithi092.stdout: xfsdump x86_64 3.1.8-7.el8 baseos 334 k 2023-11-28T11:16:03.625 INFO:teuthology.orchestra.run.smithi092.stdout: xfsprogs x86_64 5.0.0-12.el8 baseos 1.1 M 2023-11-28T11:16:03.625 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2023-11-28T11:16:03.625 INFO:teuthology.orchestra.run.smithi092.stdout: automake noarch 1.16.1-8.el8 appstream 740 k 2023-11-28T11:16:03.626 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-common x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 22 M 2023-11-28T11:16:03.626 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-grafana-dashboards noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 23 k 2023-11-28T11:16:03.626 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mds x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 2.3 M 2023-11-28T11:16:03.626 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-modules-core noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 236 k 2023-11-28T11:16:03.626 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mon x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 4.6 M 2023-11-28T11:16:03.626 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-osd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 18 M 2023-11-28T11:16:03.626 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-prometheus-alerts noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 15 k 2023-11-28T11:16:03.626 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-selinux x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 24 k 2023-11-28T11:16:03.626 INFO:teuthology.orchestra.run.smithi092.stdout: fuse x86_64 2.9.7-17.el8 baseos 83 k 2023-11-28T11:16:03.626 INFO:teuthology.orchestra.run.smithi092.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-28T11:16:03.626 INFO:teuthology.orchestra.run.smithi092.stdout: jq x86_64 1.6-8.el8 appstream 203 k 2023-11-28T11:16:03.626 INFO:teuthology.orchestra.run.smithi092.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 baseos 48 k 2023-11-28T11:16:03.627 INFO:teuthology.orchestra.run.smithi092.stdout: krb5-devel x86_64 1.18.2-22.el8 baseos 561 k 2023-11-28T11:16:03.627 INFO:teuthology.orchestra.run.smithi092.stdout: libcephsqlite x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 137 k 2023-11-28T11:16:03.627 INFO:teuthology.orchestra.run.smithi092.stdout: libcom_err-devel x86_64 1.45.6-5.el8 baseos 39 k 2023-11-28T11:16:03.627 INFO:teuthology.orchestra.run.smithi092.stdout: libgfortran x86_64 8.5.0-21.el8 baseos 645 k 2023-11-28T11:16:03.627 INFO:teuthology.orchestra.run.smithi092.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-28T11:16:03.627 INFO:teuthology.orchestra.run.smithi092.stdout: libpq x86_64 13.5-1.el8 appstream 198 k 2023-11-28T11:16:03.627 INFO:teuthology.orchestra.run.smithi092.stdout: libquadmath x86_64 8.5.0-21.el8 baseos 172 k 2023-11-28T11:16:03.627 INFO:teuthology.orchestra.run.smithi092.stdout: librabbitmq x86_64 0.9.0-4.el8 baseos 47 k 2023-11-28T11:16:03.627 INFO:teuthology.orchestra.run.smithi092.stdout: libradosstriper1 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 474 k 2023-11-28T11:16:03.627 INFO:teuthology.orchestra.run.smithi092.stdout: librdkafka x86_64 0.11.4-3.el8 appstream 354 k 2023-11-28T11:16:03.627 INFO:teuthology.orchestra.run.smithi092.stdout: librgw2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 6.5 M 2023-11-28T11:16:03.627 INFO:teuthology.orchestra.run.smithi092.stdout: libselinux-devel x86_64 2.9-8.el8 baseos 200 k 2023-11-28T11:16:03.628 INFO:teuthology.orchestra.run.smithi092.stdout: libsepol-devel x86_64 2.9-3.el8 baseos 87 k 2023-11-28T11:16:03.628 INFO:teuthology.orchestra.run.smithi092.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-28T11:16:03.628 INFO:teuthology.orchestra.run.smithi092.stdout: libverto-devel x86_64 0.3.2-2.el8 baseos 18 k 2023-11-28T11:16:03.628 INFO:teuthology.orchestra.run.smithi092.stdout: libzstd-devel x86_64 1.4.4-1.el8 baseos 44 k 2023-11-28T11:16:03.628 INFO:teuthology.orchestra.run.smithi092.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2023-11-28T11:16:03.628 INFO:teuthology.orchestra.run.smithi092.stdout: oniguruma x86_64 6.8.2-2.el8 appstream 187 k 2023-11-28T11:16:03.628 INFO:teuthology.orchestra.run.smithi092.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2023-11-28T11:16:03.628 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2023-11-28T11:16:03.628 INFO:teuthology.orchestra.run.smithi092.stdout: pcre2-devel x86_64 10.32-3.el8 baseos 605 k 2023-11-28T11:16:03.628 INFO:teuthology.orchestra.run.smithi092.stdout: pcre2-utf16 x86_64 10.32-3.el8 baseos 229 k 2023-11-28T11:16:03.628 INFO:teuthology.orchestra.run.smithi092.stdout: pcre2-utf32 x86_64 10.32-3.el8 baseos 220 k 2023-11-28T11:16:03.628 INFO:teuthology.orchestra.run.smithi092.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-11-28T11:16:03.629 INFO:teuthology.orchestra.run.smithi092.stdout: python3-babel noarch 2.5.1-7.el8 appstream 4.8 M 2023-11-28T11:16:03.629 INFO:teuthology.orchestra.run.smithi092.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-28T11:16:03.629 INFO:teuthology.orchestra.run.smithi092.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-28T11:16:03.629 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-28T11:16:03.629 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-argparse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 45 k 2023-11-28T11:16:03.629 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-common x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 117 k 2023-11-28T11:16:03.629 INFO:teuthology.orchestra.run.smithi092.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-28T11:16:03.629 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2023-11-28T11:16:03.629 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-28T11:16:03.629 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-28T11:16:03.629 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cryptography x86_64 3.2.1-6.el8 baseos 641 k 2023-11-28T11:16:03.630 INFO:teuthology.orchestra.run.smithi092.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-28T11:16:03.630 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-28T11:16:03.630 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-28T11:16:03.630 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jinja2 noarch 2.10.1-3.el8 appstream 538 k 2023-11-28T11:16:03.630 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2023-11-28T11:16:03.630 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2023-11-28T11:16:03.630 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2023-11-28T11:16:03.630 INFO:teuthology.orchestra.run.smithi092.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-28T11:16:03.630 INFO:teuthology.orchestra.run.smithi092.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-28T11:16:03.630 INFO:teuthology.orchestra.run.smithi092.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2023-11-28T11:16:03.630 INFO:teuthology.orchestra.run.smithi092.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2023-11-28T11:16:03.630 INFO:teuthology.orchestra.run.smithi092.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-28T11:16:03.631 INFO:teuthology.orchestra.run.smithi092.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-11-28T11:16:03.631 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2023-11-28T11:16:03.631 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2023-11-28T11:16:03.631 INFO:teuthology.orchestra.run.smithi092.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2023-11-28T11:16:03.631 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-28T11:16:03.631 INFO:teuthology.orchestra.run.smithi092.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-28T11:16:03.631 INFO:teuthology.orchestra.run.smithi092.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2023-11-28T11:16:03.631 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2023-11-28T11:16:03.631 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2023-11-28T11:16:03.631 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2023-11-28T11:16:03.631 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2023-11-28T11:16:03.631 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2023-11-28T11:16:03.632 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-28T11:16:03.632 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2023-11-28T11:16:03.632 INFO:teuthology.orchestra.run.smithi092.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-28T11:16:03.632 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-28T11:16:03.632 INFO:teuthology.orchestra.run.smithi092.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 appstream 14 M 2023-11-28T11:16:03.632 INFO:teuthology.orchestra.run.smithi092.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-28T11:16:03.632 INFO:teuthology.orchestra.run.smithi092.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-28T11:16:03.632 INFO:teuthology.orchestra.run.smithi092.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-28T11:16:03.632 INFO:teuthology.orchestra.run.smithi092.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-28T11:16:03.632 INFO:teuthology.orchestra.run.smithi092.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-28T11:16:03.632 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-28T11:16:03.633 INFO:teuthology.orchestra.run.smithi092.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-28T11:16:03.633 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-28T11:16:03.633 INFO:teuthology.orchestra.run.smithi092.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2023-11-28T11:16:03.633 INFO:teuthology.orchestra.run.smithi092.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-28T11:16:03.633 INFO:teuthology.orchestra.run.smithi092.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2023-11-28T11:16:03.633 INFO:teuthology.orchestra.run.smithi092.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-11-28T11:16:03.633 INFO:teuthology.orchestra.run.smithi092.stdout: uuid x86_64 1.6.2-43.el8 appstream 64 k 2023-11-28T11:16:03.633 INFO:teuthology.orchestra.run.smithi092.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2023-11-28T11:16:03.633 INFO:teuthology.orchestra.run.smithi092.stdout: zlib-devel x86_64 1.2.11-21.el8 baseos 58 k 2023-11-28T11:16:03.633 INFO:teuthology.orchestra.run.smithi092.stdout:Enabling module streams: 2023-11-28T11:16:03.633 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql 10 2023-11-28T11:16:03.633 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:16:03.633 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2023-11-28T11:16:03.634 INFO:teuthology.orchestra.run.smithi092.stdout:==================================================================================================== 2023-11-28T11:16:03.634 INFO:teuthology.orchestra.run.smithi092.stdout:Install 128 Packages 2023-11-28T11:16:03.634 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrade 21 Packages 2023-11-28T11:16:03.634 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:16:03.634 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 212 M 2023-11-28T11:16:03.634 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2023-11-28T11:16:04.509 INFO:teuthology.orchestra.run.smithi092.stdout:(1/149): ceph-18.0.0-2697.g2f5f6028.el8.x86_64. 33 kB/s | 6.4 kB 00:00 2023-11-28T11:16:04.667 INFO:teuthology.orchestra.run.smithi169.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-28T11:16:04.668 INFO:teuthology.orchestra.run.smithi169.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-28T11:16:04.681 INFO:teuthology.orchestra.run.smithi169.stdout:Package NetworkManager-1:1.40.16-4.el8.x86_64 is already installed. 2023-11-28T11:16:04.681 INFO:teuthology.orchestra.run.smithi169.stdout:Package NetworkManager-initscripts-updown-1:1.40.16-4.el8.noarch is already installed. 2023-11-28T11:16:04.682 INFO:teuthology.orchestra.run.smithi169.stdout:Package iproute-6.2.0-1.el8.x86_64 is already installed. 2023-11-28T11:16:04.684 INFO:teuthology.orchestra.run.smithi169.stdout:Package util-linux-2.32.1-41.el8.x86_64 is already installed. 2023-11-28T11:16:04.685 INFO:teuthology.orchestra.run.smithi169.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-28T11:16:04.687 INFO:teuthology.orchestra.run.smithi169.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-28T11:16:04.690 INFO:teuthology.orchestra.run.smithi169.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-11-28T11:16:04.691 INFO:teuthology.orchestra.run.smithi169.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-28T11:16:04.809 INFO:teuthology.orchestra.run.smithi092.stdout:(2/149): ceph-base-18.0.0-2697.g2f5f6028.el8.x8 11 MB/s | 5.4 MB 00:00 2023-11-28T11:16:04.860 INFO:teuthology.orchestra.run.smithi092.stdout:(3/149): ceph-immutable-object-cache-18.0.0-269 3.3 MB/s | 165 kB 00:00 2023-11-28T11:16:04.955 INFO:teuthology.orchestra.run.smithi169.stdout:Dependencies resolved. 2023-11-28T11:16:04.970 INFO:teuthology.orchestra.run.smithi169.stdout:==================================================================================================== 2023-11-28T11:16:04.970 INFO:teuthology.orchestra.run.smithi169.stdout: Package Arch Version Repository Size 2023-11-28T11:16:04.970 INFO:teuthology.orchestra.run.smithi169.stdout:==================================================================================================== 2023-11-28T11:16:04.970 INFO:teuthology.orchestra.run.smithi169.stdout:Installing: 2023-11-28T11:16:04.970 INFO:teuthology.orchestra.run.smithi169.stdout: bison x86_64 3.0.4-10.el8 appstream 688 k 2023-11-28T11:16:04.970 INFO:teuthology.orchestra.run.smithi169.stdout: ceph x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 6.4 k 2023-11-28T11:16:04.970 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-base x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 5.4 M 2023-11-28T11:16:04.970 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-fuse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 845 k 2023-11-28T11:16:04.970 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 165 k 2023-11-28T11:16:04.971 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 1.5 M 2023-11-28T11:16:04.971 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-cephadm noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 123 k 2023-11-28T11:16:04.971 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-dashboard noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 1.7 M 2023-11-28T11:16:04.971 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 7.4 M 2023-11-28T11:16:04.971 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-rook noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 47 k 2023-11-28T11:16:04.971 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-radosgw x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 13 M 2023-11-28T11:16:04.971 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-test x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 47 M 2023-11-28T11:16:04.971 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-volume noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 263 k 2023-11-28T11:16:04.971 INFO:teuthology.orchestra.run.smithi169.stdout: cephadm noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 83 k 2023-11-28T11:16:04.971 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-mirror x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 257 k 2023-11-28T11:16:04.971 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-top noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 18 k 2023-11-28T11:16:04.972 INFO:teuthology.orchestra.run.smithi169.stdout: elfutils-libelf-devel x86_64 0.190-1.el8 baseos 62 k 2023-11-28T11:16:04.972 INFO:teuthology.orchestra.run.smithi169.stdout: flex x86_64 2.6.1-9.el8 appstream 320 k 2023-11-28T11:16:04.972 INFO:teuthology.orchestra.run.smithi169.stdout: libaio-devel x86_64 0.3.112-1.el8 baseos 19 k 2023-11-28T11:16:04.972 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs-devel x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 30 k 2023-11-28T11:16:04.972 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 767 k 2023-11-28T11:16:04.972 INFO:teuthology.orchestra.run.smithi169.stdout: librados-devel x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 139 k 2023-11-28T11:16:04.972 INFO:teuthology.orchestra.run.smithi169.stdout: libtool x86_64 2.4.6-25.el8 appstream 709 k 2023-11-28T11:16:04.972 INFO:teuthology.orchestra.run.smithi169.stdout: libuuid-devel x86_64 2.32.1-43.el8 baseos 100 k 2023-11-28T11:16:04.972 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-devel x86_64 1:1.1.1k-11.el8 baseos 3.2 M 2023-11-28T11:16:04.972 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql x86_64 10.23-2.module_el8+572+929c87ac appstream 1.6 M 2023-11-28T11:16:04.972 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-contrib x86_64 10.23-2.module_el8+572+929c87ac appstream 901 k 2023-11-28T11:16:04.972 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-server x86_64 10.23-2.module_el8+572+929c87ac appstream 5.1 M 2023-11-28T11:16:04.973 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cephfs x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 190 k 2023-11-28T11:16:04.973 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rados x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 371 k 2023-11-28T11:16:04.973 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rbd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 364 k 2023-11-28T11:16:04.973 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rgw x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 107 k 2023-11-28T11:16:04.973 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-fuse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 94 k 2023-11-28T11:16:04.973 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-mirror x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.4 M 2023-11-28T11:16:04.973 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-nbd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 180 k 2023-11-28T11:16:04.973 INFO:teuthology.orchestra.run.smithi169.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 baseos 74 k 2023-11-28T11:16:04.973 INFO:teuthology.orchestra.run.smithi169.stdout:Upgrading: 2023-11-28T11:16:04.973 INFO:teuthology.orchestra.run.smithi169.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-11.el8 baseos 144 k 2023-11-28T11:16:04.973 INFO:teuthology.orchestra.run.smithi169.stdout: acl x86_64 2.2.53-3.el8 baseos 83 k 2023-11-28T11:16:04.974 INFO:teuthology.orchestra.run.smithi169.stdout: elfutils-debuginfod-client x86_64 0.190-1.el8 baseos 77 k 2023-11-28T11:16:04.974 INFO:teuthology.orchestra.run.smithi169.stdout: elfutils-libelf x86_64 0.190-1.el8 baseos 232 k 2023-11-28T11:16:04.974 INFO:teuthology.orchestra.run.smithi169.stdout: elfutils-libs x86_64 0.190-1.el8 baseos 305 k 2023-11-28T11:16:04.974 INFO:teuthology.orchestra.run.smithi169.stdout: iproute x86_64 6.2.0-5.el8 baseos 854 k 2023-11-28T11:16:04.974 INFO:teuthology.orchestra.run.smithi169.stdout: libacl x86_64 2.2.53-3.el8 baseos 35 k 2023-11-28T11:16:04.974 INFO:teuthology.orchestra.run.smithi169.stdout: libacl-devel x86_64 2.2.53-3.el8 baseos 85 k 2023-11-28T11:16:04.974 INFO:teuthology.orchestra.run.smithi169.stdout: libblkid x86_64 2.32.1-43.el8 baseos 220 k 2023-11-28T11:16:04.974 INFO:teuthology.orchestra.run.smithi169.stdout: libfdisk x86_64 2.32.1-43.el8 baseos 254 k 2023-11-28T11:16:04.974 INFO:teuthology.orchestra.run.smithi169.stdout: libmount x86_64 2.32.1-43.el8 baseos 236 k 2023-11-28T11:16:04.974 INFO:teuthology.orchestra.run.smithi169.stdout: librados2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.7 M 2023-11-28T11:16:04.974 INFO:teuthology.orchestra.run.smithi169.stdout: librbd1 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.6 M 2023-11-28T11:16:04.975 INFO:teuthology.orchestra.run.smithi169.stdout: libsmartcols x86_64 2.32.1-43.el8 baseos 179 k 2023-11-28T11:16:04.975 INFO:teuthology.orchestra.run.smithi169.stdout: libuuid x86_64 2.32.1-43.el8 baseos 99 k 2023-11-28T11:16:04.975 INFO:teuthology.orchestra.run.smithi169.stdout: openssl x86_64 1:1.1.1k-11.el8 baseos 737 k 2023-11-28T11:16:04.975 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-libs x86_64 1:1.1.1k-11.el8 baseos 1.5 M 2023-11-28T11:16:04.975 INFO:teuthology.orchestra.run.smithi169.stdout: util-linux x86_64 2.32.1-43.el8 baseos 2.5 M 2023-11-28T11:16:04.975 INFO:teuthology.orchestra.run.smithi169.stdout: util-linux-user x86_64 2.32.1-43.el8 baseos 103 k 2023-11-28T11:16:04.975 INFO:teuthology.orchestra.run.smithi169.stdout: xfsdump x86_64 3.1.8-7.el8 baseos 334 k 2023-11-28T11:16:04.975 INFO:teuthology.orchestra.run.smithi169.stdout: xfsprogs x86_64 5.0.0-12.el8 baseos 1.1 M 2023-11-28T11:16:04.975 INFO:teuthology.orchestra.run.smithi169.stdout:Installing dependencies: 2023-11-28T11:16:04.975 INFO:teuthology.orchestra.run.smithi169.stdout: automake noarch 1.16.1-8.el8 appstream 740 k 2023-11-28T11:16:04.975 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-common x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 22 M 2023-11-28T11:16:04.975 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-grafana-dashboards noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 23 k 2023-11-28T11:16:04.976 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mds x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 2.3 M 2023-11-28T11:16:04.976 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-modules-core noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 236 k 2023-11-28T11:16:04.976 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mon x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 4.6 M 2023-11-28T11:16:04.976 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-osd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 18 M 2023-11-28T11:16:04.976 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-prometheus-alerts noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 15 k 2023-11-28T11:16:04.976 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-selinux x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 24 k 2023-11-28T11:16:04.976 INFO:teuthology.orchestra.run.smithi169.stdout: fuse x86_64 2.9.7-17.el8 baseos 83 k 2023-11-28T11:16:04.976 INFO:teuthology.orchestra.run.smithi169.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-28T11:16:04.976 INFO:teuthology.orchestra.run.smithi169.stdout: jq x86_64 1.6-8.el8 appstream 203 k 2023-11-28T11:16:04.976 INFO:teuthology.orchestra.run.smithi169.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 baseos 48 k 2023-11-28T11:16:04.976 INFO:teuthology.orchestra.run.smithi169.stdout: krb5-devel x86_64 1.18.2-22.el8 baseos 561 k 2023-11-28T11:16:04.976 INFO:teuthology.orchestra.run.smithi169.stdout: libcephsqlite x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 137 k 2023-11-28T11:16:04.977 INFO:teuthology.orchestra.run.smithi169.stdout: libcom_err-devel x86_64 1.45.6-5.el8 baseos 39 k 2023-11-28T11:16:04.977 INFO:teuthology.orchestra.run.smithi169.stdout: libgfortran x86_64 8.5.0-21.el8 baseos 645 k 2023-11-28T11:16:04.977 INFO:teuthology.orchestra.run.smithi169.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-28T11:16:04.977 INFO:teuthology.orchestra.run.smithi169.stdout: libpq x86_64 13.5-1.el8 appstream 198 k 2023-11-28T11:16:04.977 INFO:teuthology.orchestra.run.smithi169.stdout: libquadmath x86_64 8.5.0-21.el8 baseos 172 k 2023-11-28T11:16:04.977 INFO:teuthology.orchestra.run.smithi169.stdout: librabbitmq x86_64 0.9.0-4.el8 baseos 47 k 2023-11-28T11:16:04.977 INFO:teuthology.orchestra.run.smithi169.stdout: libradosstriper1 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 474 k 2023-11-28T11:16:04.977 INFO:teuthology.orchestra.run.smithi169.stdout: librdkafka x86_64 0.11.4-3.el8 appstream 354 k 2023-11-28T11:16:04.977 INFO:teuthology.orchestra.run.smithi169.stdout: librgw2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 6.5 M 2023-11-28T11:16:04.977 INFO:teuthology.orchestra.run.smithi169.stdout: libselinux-devel x86_64 2.9-8.el8 baseos 200 k 2023-11-28T11:16:04.977 INFO:teuthology.orchestra.run.smithi169.stdout: libsepol-devel x86_64 2.9-3.el8 baseos 87 k 2023-11-28T11:16:04.978 INFO:teuthology.orchestra.run.smithi169.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-28T11:16:04.978 INFO:teuthology.orchestra.run.smithi169.stdout: libverto-devel x86_64 0.3.2-2.el8 baseos 18 k 2023-11-28T11:16:04.978 INFO:teuthology.orchestra.run.smithi169.stdout: libzstd-devel x86_64 1.4.4-1.el8 baseos 44 k 2023-11-28T11:16:04.978 INFO:teuthology.orchestra.run.smithi169.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2023-11-28T11:16:04.978 INFO:teuthology.orchestra.run.smithi169.stdout: oniguruma x86_64 6.8.2-2.el8 appstream 187 k 2023-11-28T11:16:04.978 INFO:teuthology.orchestra.run.smithi169.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2023-11-28T11:16:04.978 INFO:teuthology.orchestra.run.smithi169.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2023-11-28T11:16:04.978 INFO:teuthology.orchestra.run.smithi169.stdout: pcre2-devel x86_64 10.32-3.el8 baseos 605 k 2023-11-28T11:16:04.978 INFO:teuthology.orchestra.run.smithi169.stdout: pcre2-utf16 x86_64 10.32-3.el8 baseos 229 k 2023-11-28T11:16:04.978 INFO:teuthology.orchestra.run.smithi169.stdout: pcre2-utf32 x86_64 10.32-3.el8 baseos 220 k 2023-11-28T11:16:04.978 INFO:teuthology.orchestra.run.smithi169.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-11-28T11:16:04.978 INFO:teuthology.orchestra.run.smithi169.stdout: python3-babel noarch 2.5.1-7.el8 appstream 4.8 M 2023-11-28T11:16:04.979 INFO:teuthology.orchestra.run.smithi169.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-28T11:16:04.979 INFO:teuthology.orchestra.run.smithi169.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-28T11:16:04.979 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-28T11:16:04.979 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-argparse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 45 k 2023-11-28T11:16:04.979 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-common x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 117 k 2023-11-28T11:16:04.979 INFO:teuthology.orchestra.run.smithi169.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-28T11:16:04.979 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2023-11-28T11:16:04.979 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-28T11:16:04.979 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-28T11:16:04.979 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cryptography x86_64 3.2.1-6.el8 baseos 641 k 2023-11-28T11:16:04.979 INFO:teuthology.orchestra.run.smithi169.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-28T11:16:04.979 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-28T11:16:04.979 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-28T11:16:04.980 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jinja2 noarch 2.10.1-3.el8 appstream 538 k 2023-11-28T11:16:04.980 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2023-11-28T11:16:04.980 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2023-11-28T11:16:04.980 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2023-11-28T11:16:04.980 INFO:teuthology.orchestra.run.smithi169.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-28T11:16:04.980 INFO:teuthology.orchestra.run.smithi169.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-28T11:16:04.980 INFO:teuthology.orchestra.run.smithi169.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2023-11-28T11:16:04.980 INFO:teuthology.orchestra.run.smithi169.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2023-11-28T11:16:04.980 INFO:teuthology.orchestra.run.smithi169.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-28T11:16:04.980 INFO:teuthology.orchestra.run.smithi169.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-11-28T11:16:04.980 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2023-11-28T11:16:04.980 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2023-11-28T11:16:04.980 INFO:teuthology.orchestra.run.smithi169.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2023-11-28T11:16:04.981 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-28T11:16:04.981 INFO:teuthology.orchestra.run.smithi169.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-28T11:16:04.981 INFO:teuthology.orchestra.run.smithi169.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2023-11-28T11:16:04.981 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2023-11-28T11:16:04.981 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2023-11-28T11:16:04.981 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2023-11-28T11:16:04.981 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2023-11-28T11:16:04.981 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2023-11-28T11:16:04.981 INFO:teuthology.orchestra.run.smithi169.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-28T11:16:04.981 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2023-11-28T11:16:04.981 INFO:teuthology.orchestra.run.smithi169.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-28T11:16:04.981 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-28T11:16:04.981 INFO:teuthology.orchestra.run.smithi169.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 appstream 14 M 2023-11-28T11:16:04.981 INFO:teuthology.orchestra.run.smithi169.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-28T11:16:04.982 INFO:teuthology.orchestra.run.smithi169.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-28T11:16:04.982 INFO:teuthology.orchestra.run.smithi169.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-28T11:16:04.982 INFO:teuthology.orchestra.run.smithi169.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-28T11:16:04.982 INFO:teuthology.orchestra.run.smithi169.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-28T11:16:04.982 INFO:teuthology.orchestra.run.smithi169.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-28T11:16:04.982 INFO:teuthology.orchestra.run.smithi169.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-28T11:16:04.982 INFO:teuthology.orchestra.run.smithi169.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-28T11:16:04.982 INFO:teuthology.orchestra.run.smithi169.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2023-11-28T11:16:04.982 INFO:teuthology.orchestra.run.smithi169.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-28T11:16:04.982 INFO:teuthology.orchestra.run.smithi169.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2023-11-28T11:16:04.982 INFO:teuthology.orchestra.run.smithi169.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-11-28T11:16:04.982 INFO:teuthology.orchestra.run.smithi169.stdout: uuid x86_64 1.6.2-43.el8 appstream 64 k 2023-11-28T11:16:04.983 INFO:teuthology.orchestra.run.smithi169.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2023-11-28T11:16:04.983 INFO:teuthology.orchestra.run.smithi169.stdout: zlib-devel x86_64 1.2.11-21.el8 baseos 58 k 2023-11-28T11:16:04.983 INFO:teuthology.orchestra.run.smithi169.stdout:Enabling module streams: 2023-11-28T11:16:04.983 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql 10 2023-11-28T11:16:04.983 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-28T11:16:04.983 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction Summary 2023-11-28T11:16:04.983 INFO:teuthology.orchestra.run.smithi169.stdout:==================================================================================================== 2023-11-28T11:16:04.983 INFO:teuthology.orchestra.run.smithi169.stdout:Install 128 Packages 2023-11-28T11:16:04.983 INFO:teuthology.orchestra.run.smithi169.stdout:Upgrade 21 Packages 2023-11-28T11:16:04.983 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-28T11:16:04.984 INFO:teuthology.orchestra.run.smithi169.stdout:Total download size: 212 M 2023-11-28T11:16:04.984 INFO:teuthology.orchestra.run.smithi169.stdout:Downloading Packages: 2023-11-28T11:16:05.018 INFO:teuthology.orchestra.run.smithi092.stdout:(4/149): ceph-mds-18.0.0-2697.g2f5f6028.el8.x86 14 MB/s | 2.3 MB 00:00 2023-11-28T11:16:05.060 INFO:teuthology.orchestra.run.smithi092.stdout:(5/149): ceph-fuse-18.0.0-2697.g2f5f6028.el8.x8 1.5 MB/s | 845 kB 00:00 2023-11-28T11:16:05.127 INFO:teuthology.orchestra.run.smithi092.stdout:(6/149): ceph-mgr-18.0.0-2697.g2f5f6028.el8.x86 14 MB/s | 1.5 MB 00:00 2023-11-28T11:16:05.766 INFO:teuthology.orchestra.run.smithi169.stdout:(1/149): ceph-18.0.0-2697.g2f5f6028.el8.x86_64. 16 kB/s | 6.4 kB 00:00 2023-11-28T11:16:06.177 INFO:teuthology.orchestra.run.smithi092.stdout:(7/149): ceph-osd-18.0.0-2697.g2f5f6028.el8.x86 17 MB/s | 18 MB 00:01 2023-11-28T11:16:06.291 INFO:teuthology.orchestra.run.smithi169.stdout:(2/149): ceph-fuse-18.0.0-2697.g2f5f6028.el8.x8 1.6 MB/s | 845 kB 00:00 2023-11-28T11:16:06.441 INFO:teuthology.orchestra.run.smithi169.stdout:(3/149): ceph-immutable-object-cache-18.0.0-269 1.1 MB/s | 165 kB 00:00 2023-11-28T11:16:06.819 INFO:teuthology.orchestra.run.smithi092.stdout:(8/149): ceph-mon-18.0.0-2697.g2f5f6028.el8.x86 2.6 MB/s | 4.6 MB 00:01 2023-11-28T11:16:06.952 INFO:teuthology.orchestra.run.smithi092.stdout:(9/149): ceph-selinux-18.0.0-2697.g2f5f6028.el8 183 kB/s | 24 kB 00:00 2023-11-28T11:16:07.128 INFO:teuthology.orchestra.run.smithi092.stdout:(10/149): ceph-radosgw-18.0.0-2697.g2f5f6028.el 14 MB/s | 13 MB 00:00 2023-11-28T11:16:07.178 INFO:teuthology.orchestra.run.smithi092.stdout:(11/149): cephfs-mirror-18.0.0-2697.g2f5f6028.e 5.1 MB/s | 257 kB 00:00 2023-11-28T11:16:07.228 INFO:teuthology.orchestra.run.smithi092.stdout:(12/149): libcephfs-devel-18.0.0-2697.g2f5f6028 614 kB/s | 30 kB 00:00 2023-11-28T11:16:07.295 INFO:teuthology.orchestra.run.smithi092.stdout:(13/149): libcephfs2-18.0.0-2697.g2f5f6028.el8. 11 MB/s | 767 kB 00:00 2023-11-28T11:16:07.304 INFO:teuthology.orchestra.run.smithi169.stdout:(4/149): ceph-mds-18.0.0-2697.g2f5f6028.el8.x86 2.6 MB/s | 2.3 MB 00:00 2023-11-28T11:16:07.345 INFO:teuthology.orchestra.run.smithi092.stdout:(14/149): libcephsqlite-18.0.0-2697.g2f5f6028.e 2.7 MB/s | 137 kB 00:00 2023-11-28T11:16:07.396 INFO:teuthology.orchestra.run.smithi092.stdout:(15/149): librados-devel-18.0.0-2697.g2f5f6028. 2.7 MB/s | 139 kB 00:00 2023-11-28T11:16:07.446 INFO:teuthology.orchestra.run.smithi092.stdout:(16/149): libradosstriper1-18.0.0-2697.g2f5f602 9.3 MB/s | 474 kB 00:00 2023-11-28T11:16:07.896 INFO:teuthology.orchestra.run.smithi092.stdout:(17/149): librgw2-18.0.0-2697.g2f5f6028.el8.x86 14 MB/s | 6.5 MB 00:00 2023-11-28T11:16:07.942 INFO:teuthology.orchestra.run.smithi169.stdout:(5/149): ceph-mgr-18.0.0-2697.g2f5f6028.el8.x86 2.4 MB/s | 1.5 MB 00:00 2023-11-28T11:16:07.947 INFO:teuthology.orchestra.run.smithi092.stdout:(18/149): python3-ceph-argparse-18.0.0-2697.g2f 887 kB/s | 45 kB 00:00 2023-11-28T11:16:07.989 INFO:teuthology.orchestra.run.smithi092.stdout:(19/149): python3-ceph-common-18.0.0-2697.g2f5f 2.8 MB/s | 117 kB 00:00 2023-11-28T11:16:08.039 INFO:teuthology.orchestra.run.smithi092.stdout:(20/149): python3-cephfs-18.0.0-2697.g2f5f6028. 3.7 MB/s | 190 kB 00:00 2023-11-28T11:16:08.176 INFO:teuthology.orchestra.run.smithi169.stdout:(6/149): ceph-base-18.0.0-2697.g2f5f6028.el8.x8 1.9 MB/s | 5.4 MB 00:02 2023-11-28T11:16:08.265 INFO:teuthology.orchestra.run.smithi092.stdout:(21/149): python3-rados-18.0.0-2697.g2f5f6028.e 1.6 MB/s | 371 kB 00:00 2023-11-28T11:16:08.340 INFO:teuthology.orchestra.run.smithi092.stdout:(22/149): python3-rbd-18.0.0-2697.g2f5f6028.el8 4.8 MB/s | 364 kB 00:00 2023-11-28T11:16:08.399 INFO:teuthology.orchestra.run.smithi092.stdout:(23/149): python3-rgw-18.0.0-2697.g2f5f6028.el8 1.8 MB/s | 107 kB 00:00 2023-11-28T11:16:08.449 INFO:teuthology.orchestra.run.smithi092.stdout:(24/149): rbd-fuse-18.0.0-2697.g2f5f6028.el8.x8 1.9 MB/s | 94 kB 00:00 2023-11-28T11:16:08.816 INFO:teuthology.orchestra.run.smithi092.stdout:(25/149): ceph-common-18.0.0-2697.g2f5f6028.el8 4.8 MB/s | 22 MB 00:04 2023-11-28T11:16:08.883 INFO:teuthology.orchestra.run.smithi092.stdout:(26/149): rbd-mirror-18.0.0-2697.g2f5f6028.el8. 8.0 MB/s | 3.4 MB 00:00 2023-11-28T11:16:08.917 INFO:teuthology.orchestra.run.smithi092.stdout:(27/149): rbd-nbd-18.0.0-2697.g2f5f6028.el8.x86 1.8 MB/s | 180 kB 00:00 2023-11-28T11:16:08.942 INFO:teuthology.orchestra.run.smithi092.stdout:(28/149): ceph-grafana-dashboards-18.0.0-2697.g 393 kB/s | 23 kB 00:00 2023-11-28T11:16:09.009 INFO:teuthology.orchestra.run.smithi092.stdout:(29/149): ceph-mgr-cephadm-18.0.0-2697.g2f5f602 1.3 MB/s | 123 kB 00:00 2023-11-28T11:16:09.084 INFO:teuthology.orchestra.run.smithi092.stdout:(30/149): ceph-mgr-dashboard-18.0.0-2697.g2f5f6 12 MB/s | 1.7 MB 00:00 2023-11-28T11:16:09.134 INFO:teuthology.orchestra.run.smithi092.stdout:(31/149): ceph-mgr-modules-core-18.0.0-2697.g2f 4.6 MB/s | 236 kB 00:00 2023-11-28T11:16:09.176 INFO:teuthology.orchestra.run.smithi092.stdout:(32/149): ceph-mgr-rook-18.0.0-2697.g2f5f6028.e 1.1 MB/s | 47 kB 00:00 2023-11-28T11:16:09.218 INFO:teuthology.orchestra.run.smithi092.stdout:(33/149): ceph-prometheus-alerts-18.0.0-2697.g2 348 kB/s | 15 kB 00:00 2023-11-28T11:16:09.268 INFO:teuthology.orchestra.run.smithi169.stdout:(7/149): ceph-mon-18.0.0-2697.g2f5f6028.el8.x86 3.5 MB/s | 4.6 MB 00:01 2023-11-28T11:16:09.268 INFO:teuthology.orchestra.run.smithi092.stdout:(34/149): ceph-volume-18.0.0-2697.g2f5f6028.el8 5.2 MB/s | 263 kB 00:00 2023-11-28T11:16:09.319 INFO:teuthology.orchestra.run.smithi092.stdout:(35/149): cephadm-18.0.0-2697.g2f5f6028.el8.noa 1.6 MB/s | 83 kB 00:00 2023-11-28T11:16:09.352 INFO:teuthology.orchestra.run.smithi092.stdout:(36/149): cephfs-top-18.0.0-2697.g2f5f6028.el8. 548 kB/s | 18 kB 00:00 2023-11-28T11:16:11.619 INFO:teuthology.orchestra.run.smithi092.stdout:(37/149): automake-1.16.1-8.el8.noarch.rpm 326 kB/s | 740 kB 00:02 2023-11-28T11:16:11.786 INFO:teuthology.orchestra.run.smithi092.stdout:(38/149): ceph-mgr-diskprediction-local-18.0.0- 2.7 MB/s | 7.4 MB 00:02 2023-11-28T11:16:12.293 INFO:teuthology.orchestra.run.smithi169.stdout:(8/149): ceph-common-18.0.0-2697.g2f5f6028.el8. 3.1 MB/s | 22 MB 00:06 2023-11-28T11:16:12.343 INFO:teuthology.orchestra.run.smithi169.stdout:(9/149): ceph-selinux-18.0.0-2697.g2f5f6028.el8 491 kB/s | 24 kB 00:00 2023-11-28T11:16:12.819 INFO:teuthology.orchestra.run.smithi092.stdout:(39/149): flex-2.6.1-9.el8.x86_64.rpm 309 kB/s | 320 kB 00:01 2023-11-28T11:16:12.843 INFO:teuthology.orchestra.run.smithi169.stdout:(10/149): ceph-osd-18.0.0-2697.g2f5f6028.el8.x8 3.8 MB/s | 18 MB 00:04 2023-11-28T11:16:12.952 INFO:teuthology.orchestra.run.smithi169.stdout:(11/149): cephfs-mirror-18.0.0-2697.g2f5f6028.e 2.3 MB/s | 257 kB 00:00 2023-11-28T11:16:13.010 INFO:teuthology.orchestra.run.smithi169.stdout:(12/149): libcephfs-devel-18.0.0-2697.g2f5f6028 525 kB/s | 30 kB 00:00 2023-11-28T11:16:13.244 INFO:teuthology.orchestra.run.smithi169.stdout:(13/149): libcephfs2-18.0.0-2697.g2f5f6028.el8. 3.2 MB/s | 767 kB 00:00 2023-11-28T11:16:13.311 INFO:teuthology.orchestra.run.smithi169.stdout:(14/149): libcephsqlite-18.0.0-2697.g2f5f6028.e 2.0 MB/s | 137 kB 00:00 2023-11-28T11:16:13.382 INFO:teuthology.orchestra.run.smithi169.stdout:(15/149): librados-devel-18.0.0-2697.g2f5f6028. 2.0 MB/s | 139 kB 00:00 2023-11-28T11:16:13.411 INFO:teuthology.orchestra.run.smithi092.stdout:(40/149): jq-1.6-8.el8.x86_64.rpm 343 kB/s | 203 kB 00:00 2023-11-28T11:16:13.538 INFO:teuthology.orchestra.run.smithi169.stdout:(16/149): libradosstriper1-18.0.0-2697.g2f5f602 2.9 MB/s | 474 kB 00:00 2023-11-28T11:16:13.753 INFO:teuthology.orchestra.run.smithi092.stdout:(41/149): bison-3.0.4-10.el8.x86_64.rpm 323 kB/s | 688 kB 00:02 2023-11-28T11:16:13.995 INFO:teuthology.orchestra.run.smithi092.stdout:(42/149): libpq-13.5-1.el8.x86_64.rpm 339 kB/s | 198 kB 00:00 2023-11-28T11:16:14.887 INFO:teuthology.orchestra.run.smithi169.stdout:(17/149): librgw2-18.0.0-2697.g2f5f6028.el8.x86 4.8 MB/s | 6.5 MB 00:01 2023-11-28T11:16:14.911 INFO:teuthology.orchestra.run.smithi092.stdout:(43/149): librdkafka-0.11.4-3.el8.x86_64.rpm 305 kB/s | 354 kB 00:01 2023-11-28T11:16:14.953 INFO:teuthology.orchestra.run.smithi169.stdout:(18/149): python3-ceph-argparse-18.0.0-2697.g2f 674 kB/s | 45 kB 00:00 2023-11-28T11:16:15.020 INFO:teuthology.orchestra.run.smithi169.stdout:(19/149): python3-ceph-common-18.0.0-2697.g2f5f 1.7 MB/s | 117 kB 00:00 2023-11-28T11:16:15.104 INFO:teuthology.orchestra.run.smithi169.stdout:(20/149): python3-cephfs-18.0.0-2697.g2f5f6028. 2.2 MB/s | 190 kB 00:00 2023-11-28T11:16:15.221 INFO:teuthology.orchestra.run.smithi169.stdout:(21/149): python3-rados-18.0.0-2697.g2f5f6028.e 3.1 MB/s | 371 kB 00:00 2023-11-28T11:16:15.355 INFO:teuthology.orchestra.run.smithi169.stdout:(22/149): ceph-radosgw-18.0.0-2697.g2f5f6028.el 2.1 MB/s | 13 MB 00:06 2023-11-28T11:16:15.388 INFO:teuthology.orchestra.run.smithi169.stdout:(23/149): python3-rbd-18.0.0-2697.g2f5f6028.el8 2.1 MB/s | 364 kB 00:00 2023-11-28T11:16:15.413 INFO:teuthology.orchestra.run.smithi169.stdout:(24/149): python3-rgw-18.0.0-2697.g2f5f6028.el8 1.8 MB/s | 107 kB 00:00 2023-11-28T11:16:15.447 INFO:teuthology.orchestra.run.smithi169.stdout:(25/149): rbd-fuse-18.0.0-2697.g2f5f6028.el8.x8 1.6 MB/s | 94 kB 00:00 2023-11-28T11:16:15.539 INFO:teuthology.orchestra.run.smithi169.stdout:(26/149): rbd-nbd-18.0.0-2697.g2f5f6028.el8.x86 1.9 MB/s | 180 kB 00:00 2023-11-28T11:16:15.589 INFO:teuthology.orchestra.run.smithi169.stdout:(27/149): ceph-grafana-dashboards-18.0.0-2697.g 463 kB/s | 23 kB 00:00 2023-11-28T11:16:15.648 INFO:teuthology.orchestra.run.smithi169.stdout:(28/149): ceph-mgr-cephadm-18.0.0-2697.g2f5f602 2.1 MB/s | 123 kB 00:00 2023-11-28T11:16:15.728 INFO:teuthology.orchestra.run.smithi092.stdout:(44/149): lttng-ust-2.8.1-11.el8.x86_64.rpm 318 kB/s | 259 kB 00:00 2023-11-28T11:16:15.862 INFO:teuthology.orchestra.run.smithi092.stdout:(45/149): libtool-2.4.6-25.el8.x86_64.rpm 380 kB/s | 709 kB 00:01 2023-11-28T11:16:16.015 INFO:teuthology.orchestra.run.smithi169.stdout:(29/149): ceph-mgr-dashboard-18.0.0-2697.g2f5f6 4.7 MB/s | 1.7 MB 00:00 2023-11-28T11:16:16.329 INFO:teuthology.orchestra.run.smithi092.stdout:(46/149): oniguruma-6.8.2-2.el8.x86_64.rpm 312 kB/s | 187 kB 00:00 2023-11-28T11:16:16.407 INFO:teuthology.orchestra.run.smithi169.stdout:(30/149): rbd-mirror-18.0.0-2697.g2f5f6028.el8. 3.5 MB/s | 3.4 MB 00:00 2023-11-28T11:16:16.524 INFO:teuthology.orchestra.run.smithi169.stdout:(31/149): ceph-mgr-modules-core-18.0.0-2697.g2f 2.0 MB/s | 236 kB 00:00 2023-11-28T11:16:16.585 INFO:teuthology.orchestra.run.smithi169.stdout:(32/149): ceph-mgr-rook-18.0.0-2697.g2f5f6028.e 815 kB/s | 47 kB 00:00 2023-11-28T11:16:16.641 INFO:teuthology.orchestra.run.smithi169.stdout:(33/149): ceph-prometheus-alerts-18.0.0-2697.g2 248 kB/s | 15 kB 00:00 2023-11-28T11:16:16.750 INFO:teuthology.orchestra.run.smithi169.stdout:(34/149): ceph-volume-18.0.0-2697.g2f5f6028.el8 2.4 MB/s | 263 kB 00:00 2023-11-28T11:16:16.839 INFO:teuthology.orchestra.run.smithi169.stdout:(35/149): cephadm-18.0.0-2697.g2f5f6028.el8.noa 931 kB/s | 83 kB 00:00 2023-11-28T11:16:16.909 INFO:teuthology.orchestra.run.smithi169.stdout:(36/149): cephfs-top-18.0.0-2697.g2f5f6028.el8. 261 kB/s | 18 kB 00:00 2023-11-28T11:16:17.626 INFO:teuthology.orchestra.run.smithi169.stdout:(37/149): ceph-test-18.0.0-2697.g2f5f6028.el8.x 8.8 MB/s | 47 MB 00:05 2023-11-28T11:16:17.727 INFO:teuthology.orchestra.run.smithi169.stdout:(38/149): ceph-mgr-diskprediction-local-18.0.0- 4.3 MB/s | 7.4 MB 00:01 2023-11-28T11:16:18.812 INFO:teuthology.orchestra.run.smithi092.stdout:(47/149): ceph-test-18.0.0-2697.g2f5f6028.el8.x 3.9 MB/s | 47 MB 00:11 2023-11-28T11:16:20.085 INFO:teuthology.orchestra.run.smithi169.stdout:(39/149): automake-1.16.1-8.el8.noarch.rpm 233 kB/s | 740 kB 00:03 2023-11-28T11:16:20.110 INFO:teuthology.orchestra.run.smithi169.stdout:(40/149): bison-3.0.4-10.el8.x86_64.rpm 277 kB/s | 688 kB 00:02 2023-11-28T11:16:20.839 INFO:teuthology.orchestra.run.smithi169.stdout:(41/149): libpq-13.5-1.el8.x86_64.rpm 271 kB/s | 198 kB 00:00 2023-11-28T11:16:20.952 INFO:teuthology.orchestra.run.smithi169.stdout:(42/149): jq-1.6-8.el8.x86_64.rpm 234 kB/s | 203 kB 00:00 2023-11-28T11:16:21.202 INFO:teuthology.orchestra.run.smithi169.stdout:(43/149): flex-2.6.1-9.el8.x86_64.rpm 92 kB/s | 320 kB 00:03 2023-11-28T11:16:22.179 INFO:teuthology.orchestra.run.smithi169.stdout:(44/149): librdkafka-0.11.4-3.el8.x86_64.rpm 264 kB/s | 354 kB 00:01 2023-11-28T11:16:22.861 INFO:teuthology.orchestra.run.smithi169.stdout:(45/149): oniguruma-6.8.2-2.el8.x86_64.rpm 275 kB/s | 187 kB 00:00 2023-11-28T11:16:23.520 INFO:teuthology.orchestra.run.smithi169.stdout:(46/149): lttng-ust-2.8.1-11.el8.x86_64.rpm 112 kB/s | 259 kB 00:02 2023-11-28T11:16:23.853 INFO:teuthology.orchestra.run.smithi169.stdout:(47/149): libtool-2.4.6-25.el8.x86_64.rpm 244 kB/s | 709 kB 00:02 2023-11-28T11:16:27.802 INFO:teuthology.orchestra.run.smithi092.stdout:(48/149): openblas-0.3.15-4.el8.x86_64.rpm 405 kB/s | 4.7 MB 00:11 2023-11-28T11:16:29.711 INFO:teuthology.orchestra.run.smithi092.stdout:(49/149): postgresql-contrib-10.23-2.module_el8 473 kB/s | 901 kB 00:01 2023-11-28T11:16:29.944 INFO:teuthology.orchestra.run.smithi092.stdout:(50/149): openblas-threads-0.3.15-4.el8.x86_64. 365 kB/s | 4.9 MB 00:13 2023-11-28T11:16:30.203 INFO:teuthology.orchestra.run.smithi169.stdout:(48/149): postgresql-10.23-2.module_el8+572+929 252 kB/s | 1.6 MB 00:06 2023-11-28T11:16:33.738 INFO:teuthology.orchestra.run.smithi169.stdout:(49/149): postgresql-contrib-10.23-2.module_el8 255 kB/s | 901 kB 00:03 2023-11-28T11:16:40.010 INFO:teuthology.orchestra.run.smithi092.stdout:(51/149): postgresql-10.23-2.module_el8+572+929 75 kB/s | 1.6 MB 00:21 2023-11-28T11:16:40.102 INFO:teuthology.orchestra.run.smithi169.stdout:(50/149): openblas-0.3.15-4.el8.x86_64.rpm 281 kB/s | 4.7 MB 00:17 2023-11-28T11:16:43.026 INFO:teuthology.orchestra.run.smithi092.stdout:(52/149): postgresql-server-10.23-2.module_el8+ 393 kB/s | 5.1 MB 00:13 2023-11-28T11:16:43.109 INFO:teuthology.orchestra.run.smithi092.stdout:(53/149): python3-babel-2.5.1-7.el8.noarch.rpm 372 kB/s | 4.8 MB 00:13 2023-11-28T11:16:43.243 INFO:teuthology.orchestra.run.smithi092.stdout:(54/149): python3-jsonpatch-1.21-2.el8.noarch.r 123 kB/s | 27 kB 00:00 2023-11-28T11:16:43.268 INFO:teuthology.orchestra.run.smithi092.stdout:(55/149): python3-jsonpointer-1.10-11.el8.noarc 126 kB/s | 20 kB 00:00 2023-11-28T11:16:43.485 INFO:teuthology.orchestra.run.smithi092.stdout:(56/149): python3-markupsafe-0.23-19.el8.x86_64 179 kB/s | 39 kB 00:00 2023-11-28T11:16:43.752 INFO:teuthology.orchestra.run.smithi092.stdout:(57/149): python3-mako-1.0.6-14.el8.noarch.rpm 309 kB/s | 157 kB 00:00 2023-11-28T11:16:44.468 INFO:teuthology.orchestra.run.smithi092.stdout:(58/149): python3-numpy-f2py-1.14.3-10.el8.x86_ 314 kB/s | 225 kB 00:00 2023-11-28T11:16:44.685 INFO:teuthology.orchestra.run.smithi092.stdout:(59/149): python3-prettytable-0.7.2-14.el8.noar 203 kB/s | 44 kB 00:00 2023-11-28T11:16:45.052 INFO:teuthology.orchestra.run.smithi092.stdout:(60/149): python3-pyOpenSSL-19.0.0-1.el8.noarch 282 kB/s | 103 kB 00:00 2023-11-28T11:16:45.544 INFO:teuthology.orchestra.run.smithi092.stdout:(61/149): python3-pyasn1-0.3.7-6.el8.noarch.rpm 257 kB/s | 126 kB 00:00 2023-11-28T11:16:46.019 INFO:teuthology.orchestra.run.smithi092.stdout:(62/149): python3-pyasn1-modules-0.3.7-6.el8.no 232 kB/s | 110 kB 00:00 2023-11-28T11:16:46.286 INFO:teuthology.orchestra.run.smithi092.stdout:(63/149): python3-pytz-2017.2-11.el8.noarch.rpm 214 kB/s | 57 kB 00:00 2023-11-28T11:16:51.336 INFO:teuthology.orchestra.run.smithi092.stdout:(64/149): python3-jinja2-2.10.1-3.el8.noarch.rp 47 kB/s | 538 kB 00:11 2023-11-28T11:16:53.944 INFO:teuthology.orchestra.run.smithi092.stdout:(65/149): python3-numpy-1.14.3-10.el8.x86_64.rp 365 kB/s | 3.7 MB 00:10 2023-11-28T11:16:54.827 INFO:teuthology.orchestra.run.smithi092.stdout:(66/149): socat-1.7.4.1-1.el8.x86_64.rpm 366 kB/s | 323 kB 00:00 2023-11-28T11:16:55.169 INFO:teuthology.orchestra.run.smithi092.stdout:(67/149): uuid-1.6.2-43.el8.x86_64.rpm 187 kB/s | 64 kB 00:00 2023-11-28T11:16:55.436 INFO:teuthology.orchestra.run.smithi092.stdout:(68/149): xmlstarlet-1.6.1-20.el8.x86_64.rpm 265 kB/s | 71 kB 00:00 2023-11-28T11:16:55.811 INFO:teuthology.orchestra.run.smithi092.stdout:(69/149): elfutils-libelf-devel-0.190-1.el8.x86 166 kB/s | 62 kB 00:00 2023-11-28T11:16:56.003 INFO:teuthology.orchestra.run.smithi092.stdout:(70/149): fuse-2.9.7-17.el8.x86_64.rpm 433 kB/s | 83 kB 00:00 2023-11-28T11:16:56.129 INFO:teuthology.orchestra.run.smithi092.stdout:(71/149): keyutils-libs-devel-1.5.10-9.el8.x86_ 385 kB/s | 48 kB 00:00 2023-11-28T11:16:56.421 INFO:teuthology.orchestra.run.smithi092.stdout:(72/149): krb5-devel-1.18.2-22.el8.x86_64.rpm 1.9 MB/s | 561 kB 00:00 2023-11-28T11:16:56.529 INFO:teuthology.orchestra.run.smithi092.stdout:(73/149): libaio-devel-0.3.112-1.el8.x86_64.rpm 172 kB/s | 19 kB 00:00 2023-11-28T11:16:56.638 INFO:teuthology.orchestra.run.smithi092.stdout:(74/149): libcom_err-devel-1.45.6-5.el8.x86_64. 358 kB/s | 39 kB 00:00 2023-11-28T11:16:56.813 INFO:teuthology.orchestra.run.smithi092.stdout:(75/149): libgfortran-8.5.0-21.el8.x86_64.rpm 3.6 MB/s | 645 kB 00:00 2023-11-28T11:16:56.930 INFO:teuthology.orchestra.run.smithi092.stdout:(76/149): libquadmath-8.5.0-21.el8.x86_64.rpm 1.4 MB/s | 172 kB 00:00 2023-11-28T11:16:57.038 INFO:teuthology.orchestra.run.smithi092.stdout:(77/149): librabbitmq-0.9.0-4.el8.x86_64.rpm 433 kB/s | 47 kB 00:00 2023-11-28T11:16:57.155 INFO:teuthology.orchestra.run.smithi092.stdout:(78/149): libselinux-devel-2.9-8.el8.x86_64.rpm 1.7 MB/s | 200 kB 00:00 2023-11-28T11:16:57.264 INFO:teuthology.orchestra.run.smithi092.stdout:(79/149): libsepol-devel-2.9-3.el8.x86_64.rpm 804 kB/s | 87 kB 00:00 2023-11-28T11:16:57.381 INFO:teuthology.orchestra.run.smithi092.stdout:(80/149): libuuid-devel-2.32.1-43.el8.x86_64.rp 855 kB/s | 100 kB 00:00 2023-11-28T11:16:57.489 INFO:teuthology.orchestra.run.smithi092.stdout:(81/149): libverto-devel-0.3.2-2.el8.x86_64.rpm 167 kB/s | 18 kB 00:00 2023-11-28T11:16:57.598 INFO:teuthology.orchestra.run.smithi092.stdout:(82/149): libzstd-devel-1.4.4-1.el8.x86_64.rpm 405 kB/s | 44 kB 00:00 2023-11-28T11:16:57.923 INFO:teuthology.orchestra.run.smithi092.stdout:(83/149): openssl-devel-1.1.1k-11.el8.x86_64.rp 9.8 MB/s | 3.2 MB 00:00 2023-11-28T11:16:58.049 INFO:teuthology.orchestra.run.smithi092.stdout:(84/149): pcre2-devel-10.32-3.el8.x86_64.rpm 4.7 MB/s | 605 kB 00:00 2023-11-28T11:16:58.099 INFO:teuthology.orchestra.run.smithi092.stdout:(85/149): python3-werkzeug-0.12.2-4.el8.noarch. 68 kB/s | 457 kB 00:06 2023-11-28T11:16:58.158 INFO:teuthology.orchestra.run.smithi092.stdout:(86/149): pcre2-utf16-10.32-3.el8.x86_64.rpm 2.1 MB/s | 229 kB 00:00 2023-11-28T11:16:58.266 INFO:teuthology.orchestra.run.smithi092.stdout:(87/149): python3-cffi-1.11.5-6.el8.x86_64.rpm 2.2 MB/s | 249 kB 00:00 2023-11-28T11:16:58.392 INFO:teuthology.orchestra.run.smithi092.stdout:(88/149): python3-cryptography-3.2.1-6.el8.x86_ 5.0 MB/s | 641 kB 00:00 2023-11-28T11:16:58.500 INFO:teuthology.orchestra.run.smithi092.stdout:(89/149): python3-jwt-1.6.1-2.el8.noarch.rpm 394 kB/s | 43 kB 00:00 2023-11-28T11:16:58.600 INFO:teuthology.orchestra.run.smithi092.stdout:(90/149): python3-oauthlib-2.1.0-1.el8.noarch.r 1.5 MB/s | 155 kB 00:00 2023-11-28T11:16:58.651 INFO:teuthology.orchestra.run.smithi092.stdout:(91/149): pcre2-utf32-10.32-3.el8.x86_64.rpm 400 kB/s | 220 kB 00:00 2023-11-28T11:16:58.709 INFO:teuthology.orchestra.run.smithi092.stdout:(92/149): python3-pycparser-2.14-14.el8.noarch. 1.0 MB/s | 109 kB 00:00 2023-11-28T11:16:58.760 INFO:teuthology.orchestra.run.smithi092.stdout:(93/149): python3-requests-oauthlib-1.0.0-1.el8 393 kB/s | 43 kB 00:00 2023-11-28T11:16:58.818 INFO:teuthology.orchestra.run.smithi092.stdout:(94/149): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 686 kB/s | 74 kB 00:00 2023-11-28T11:16:58.869 INFO:teuthology.orchestra.run.smithi092.stdout:(95/149): zlib-devel-1.2.11-21.el8.x86_64.rpm 535 kB/s | 58 kB 00:00 2023-11-28T11:16:58.900 INFO:teuthology.orchestra.run.smithi169.stdout:(51/149): python3-babel-2.5.1-7.el8.noarch.rpm 261 kB/s | 4.8 MB 00:18 2023-11-28T11:17:01.002 INFO:teuthology.orchestra.run.smithi092.stdout:(96/149): gperftools-libs-2.7-9.el8.x86_64.rpm 140 kB/s | 306 kB 00:02 2023-11-28T11:17:01.019 INFO:teuthology.orchestra.run.smithi092.stdout:(97/149): liboath-2.6.2-3.el8.x86_64.rpm 27 kB/s | 59 kB 00:02 2023-11-28T11:17:01.231 INFO:teuthology.orchestra.run.smithi092.stdout:(98/149): libunwind-1.3.1-3.el8.x86_64.rpm 328 kB/s | 75 kB 00:00 2023-11-28T11:17:01.258 INFO:teuthology.orchestra.run.smithi169.stdout:(52/149): python3-jinja2-2.10.1-3.el8.noarch.rp 228 kB/s | 538 kB 00:02 2023-11-28T11:17:01.458 INFO:teuthology.orchestra.run.smithi169.stdout:(53/149): python3-jsonpatch-1.21-2.el8.noarch.r 133 kB/s | 27 kB 00:00 2023-11-28T11:17:01.586 INFO:teuthology.orchestra.run.smithi092.stdout:(99/149): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 123 kB/s | 44 kB 00:00 2023-11-28T11:17:01.625 INFO:teuthology.orchestra.run.smithi169.stdout:(54/149): python3-jsonpointer-1.10-11.el8.noarc 120 kB/s | 20 kB 00:00 2023-11-28T11:17:01.803 INFO:teuthology.orchestra.run.smithi092.stdout:(100/149): python3-beautifulsoup4-4.6.3-2.el8.1 855 kB/s | 185 kB 00:00 2023-11-28T11:17:01.878 INFO:teuthology.orchestra.run.smithi092.stdout:(101/149): python3-asyncssh-2.7.0-2.el8.noarch. 554 kB/s | 476 kB 00:00 2023-11-28T11:17:02.020 INFO:teuthology.orchestra.run.smithi092.stdout:(102/149): python3-certifi-2018.10.15-7.el8.noa 115 kB/s | 16 kB 00:00 2023-11-28T11:17:02.129 INFO:teuthology.orchestra.run.smithi092.stdout:(103/149): python3-cachetools-3.1.1-4.el8.noarc 101 kB/s | 33 kB 00:00 2023-11-28T11:17:02.154 INFO:teuthology.orchestra.run.smithi092.stdout:(104/149): python3-cheroot-8.5.2-1.el8.noarch.r 1.3 MB/s | 173 kB 00:00 2023-11-28T11:17:02.338 INFO:teuthology.orchestra.run.smithi092.stdout:(105/149): python3-cherrypy-18.4.0-1.el8.noarch 1.8 MB/s | 384 kB 00:00 2023-11-28T11:17:02.424 INFO:teuthology.orchestra.run.smithi169.stdout:(55/149): python3-mako-1.0.6-14.el8.noarch.rpm 197 kB/s | 157 kB 00:00 2023-11-28T11:17:02.613 INFO:teuthology.orchestra.run.smithi092.stdout:(106/149): python3-google-auth-1.1.1-10.el8.noa 177 kB/s | 81 kB 00:00 2023-11-28T11:17:02.726 INFO:teuthology.orchestra.run.smithi169.stdout:(56/149): python3-markupsafe-0.23-19.el8.x86_64 128 kB/s | 39 kB 00:00 2023-11-28T11:17:03.385 INFO:teuthology.orchestra.run.smithi169.stdout:(57/149): postgresql-server-10.23-2.module_el8+ 177 kB/s | 5.1 MB 00:29 2023-11-28T11:17:04.435 INFO:teuthology.orchestra.run.smithi169.stdout:(58/149): python3-numpy-f2py-1.14.3-10.el8.x86_ 214 kB/s | 225 kB 00:01 2023-11-28T11:17:04.769 INFO:teuthology.orchestra.run.smithi169.stdout:(59/149): python3-prettytable-0.7.2-14.el8.noar 132 kB/s | 44 kB 00:00 2023-11-28T11:17:05.302 INFO:teuthology.orchestra.run.smithi169.stdout:(60/149): python3-pyOpenSSL-19.0.0-1.el8.noarch 194 kB/s | 103 kB 00:00 2023-11-28T11:17:05.919 INFO:teuthology.orchestra.run.smithi169.stdout:(61/149): python3-pyasn1-0.3.7-6.el8.noarch.rpm 205 kB/s | 126 kB 00:00 2023-11-28T11:17:06.727 INFO:teuthology.orchestra.run.smithi169.stdout:(62/149): python3-pyasn1-modules-0.3.7-6.el8.no 136 kB/s | 110 kB 00:00 2023-11-28T11:17:07.111 INFO:teuthology.orchestra.run.smithi169.stdout:(63/149): python3-pytz-2017.2-11.el8.noarch.rpm 149 kB/s | 57 kB 00:00 2023-11-28T11:17:07.346 INFO:teuthology.orchestra.run.smithi092.stdout:(107/149): python3-jaraco-6.2-6.el8.noarch.rpm 2.2 kB/s | 11 kB 00:05 2023-11-28T11:17:07.804 INFO:teuthology.orchestra.run.smithi092.stdout:(108/149): python3-jaraco-functools-2.0-4.el8.n 3.4 kB/s | 18 kB 00:05 2023-11-28T11:17:07.896 INFO:teuthology.orchestra.run.smithi092.stdout:(109/149): python3-logutils-0.3.5-11.el8.noarch 532 kB/s | 49 kB 00:00 2023-11-28T11:17:08.704 INFO:teuthology.orchestra.run.smithi092.stdout:(110/149): python3-more-itertools-7.2.0-3.el8.n 73 kB/s | 59 kB 00:00 2023-11-28T11:17:11.829 INFO:teuthology.orchestra.run.smithi092.stdout:(111/149): python3-natsort-7.1.1-2.el8.noarch.r 19 kB/s | 60 kB 00:03 2023-11-28T11:17:18.944 INFO:teuthology.orchestra.run.smithi169.stdout:(64/149): openblas-threads-0.3.15-4.el8.x86_64. 90 kB/s | 4.9 MB 00:55 2023-11-28T11:17:19.360 INFO:teuthology.orchestra.run.smithi169.stdout:(65/149): python3-numpy-1.14.3-10.el8.x86_64.rp 230 kB/s | 3.7 MB 00:16 2023-11-28T11:17:20.537 INFO:teuthology.orchestra.run.smithi092.stdout:(112/149): python3-pecan-1.3.2-9.el8.noarch.rpm 32 kB/s | 283 kB 00:08 2023-11-28T11:17:20.611 INFO:teuthology.orchestra.run.smithi092.stdout:(113/149): python3-kubernetes-11.0.0-6.el8.noar 90 kB/s | 1.2 MB 00:13 2023-11-28T11:17:20.777 INFO:teuthology.orchestra.run.smithi169.stdout:(66/149): socat-1.7.4.1-1.el8.x86_64.rpm 228 kB/s | 323 kB 00:01 2023-11-28T11:17:21.144 INFO:teuthology.orchestra.run.smithi169.stdout:(67/149): uuid-1.6.2-43.el8.x86_64.rpm 174 kB/s | 64 kB 00:00 2023-11-28T11:17:21.578 INFO:teuthology.orchestra.run.smithi169.stdout:(68/149): xmlstarlet-1.6.1-20.el8.x86_64.rpm 163 kB/s | 71 kB 00:00 2023-11-28T11:17:21.920 INFO:teuthology.orchestra.run.smithi092.stdout:(114/149): python3-repoze-lru-0.7-6.el8.noarch. 26 kB/s | 34 kB 00:01 2023-11-28T11:17:21.970 INFO:teuthology.orchestra.run.smithi092.stdout:(115/149): python3-portend-2.6-1.el8.noarch.rpm 11 kB/s | 16 kB 00:01 2023-11-28T11:17:21.986 INFO:teuthology.orchestra.run.smithi169.stdout:(69/149): elfutils-libelf-devel-0.190-1.el8.x86 152 kB/s | 62 kB 00:00 2023-11-28T11:17:22.218 INFO:teuthology.orchestra.run.smithi169.stdout:(70/149): fuse-2.9.7-17.el8.x86_64.rpm 360 kB/s | 83 kB 00:00 2023-11-28T11:17:22.236 INFO:teuthology.orchestra.run.smithi092.stdout:(116/149): python3-rsa-4.9-2.el8.noarch.rpm 241 kB/s | 64 kB 00:00 2023-11-28T11:17:22.345 INFO:teuthology.orchestra.run.smithi169.stdout:(71/149): keyutils-libs-devel-1.5.10-9.el8.x86_ 380 kB/s | 48 kB 00:00 2023-11-28T11:17:22.354 INFO:teuthology.orchestra.run.smithi092.stdout:(117/149): python3-simplegeneric-0.8.1-17.el8.n 160 kB/s | 19 kB 00:00 2023-11-28T11:17:22.479 INFO:teuthology.orchestra.run.smithi092.stdout:(118/149): python3-singledispatch-3.4.0.3-18.el 191 kB/s | 24 kB 00:00 2023-11-28T11:17:22.604 INFO:teuthology.orchestra.run.smithi092.stdout:(119/149): python3-tempora-1.14.1-5.el8.noarch. 234 kB/s | 29 kB 00:00 2023-11-28T11:17:22.653 INFO:teuthology.orchestra.run.smithi169.stdout:(72/149): krb5-devel-1.18.2-22.el8.x86_64.rpm 1.8 MB/s | 561 kB 00:00 2023-11-28T11:17:22.721 INFO:teuthology.orchestra.run.smithi092.stdout:(120/149): python3-trustme-0.6.0-4.el8.noarch.r 236 kB/s | 27 kB 00:00 2023-11-28T11:17:22.754 INFO:teuthology.orchestra.run.smithi169.stdout:(73/149): python3-werkzeug-0.12.2-4.el8.noarch. 120 kB/s | 457 kB 00:03 2023-11-28T11:17:22.771 INFO:teuthology.orchestra.run.smithi169.stdout:(74/149): libaio-devel-0.3.112-1.el8.x86_64.rpm 159 kB/s | 19 kB 00:00 2023-11-28T11:17:22.963 INFO:teuthology.orchestra.run.smithi169.stdout:(75/149): libgfortran-8.5.0-21.el8.x86_64.rpm 3.3 MB/s | 645 kB 00:00 2023-11-28T11:17:23.005 INFO:teuthology.orchestra.run.smithi092.stdout:(121/149): python3-waitress-1.4.3-1.el8.noarch. 902 kB/s | 255 kB 00:00 2023-11-28T11:17:23.088 INFO:teuthology.orchestra.run.smithi169.stdout:(76/149): libcom_err-devel-1.45.6-5.el8.x86_64. 116 kB/s | 39 kB 00:00 2023-11-28T11:17:23.113 INFO:teuthology.orchestra.run.smithi169.stdout:(77/149): libquadmath-8.5.0-21.el8.x86_64.rpm 1.1 MB/s | 172 kB 00:00 2023-11-28T11:17:23.163 INFO:teuthology.orchestra.run.smithi092.stdout:(122/149): python3-webob-1.8.5-1.el8.1.noarch.r 1.5 MB/s | 251 kB 00:00 2023-11-28T11:17:23.255 INFO:teuthology.orchestra.run.smithi169.stdout:(78/149): libselinux-devel-2.9-8.el8.x86_64.rpm 1.4 MB/s | 200 kB 00:00 2023-11-28T11:17:23.289 INFO:teuthology.orchestra.run.smithi092.stdout:(123/149): python3-websocket-client-0.56.0-5.el 490 kB/s | 61 kB 00:00 2023-11-28T11:17:23.314 INFO:teuthology.orchestra.run.smithi169.stdout:(79/149): librabbitmq-0.9.0-4.el8.x86_64.rpm 210 kB/s | 47 kB 00:00 2023-11-28T11:17:23.372 INFO:teuthology.orchestra.run.smithi169.stdout:(80/149): libsepol-devel-2.9-3.el8.x86_64.rpm 744 kB/s | 87 kB 00:00 2023-11-28T11:17:23.489 INFO:teuthology.orchestra.run.smithi169.stdout:(81/149): libverto-devel-0.3.2-2.el8.x86_64.rpm 155 kB/s | 18 kB 00:00 2023-11-28T11:17:23.523 INFO:teuthology.orchestra.run.smithi169.stdout:(82/149): libuuid-devel-2.32.1-43.el8.x86_64.rp 477 kB/s | 100 kB 00:00 2023-11-28T11:17:23.607 INFO:teuthology.orchestra.run.smithi169.stdout:(83/149): libzstd-devel-1.4.4-1.el8.x86_64.rpm 374 kB/s | 44 kB 00:00 2023-11-28T11:17:23.774 INFO:teuthology.orchestra.run.smithi169.stdout:(84/149): pcre2-devel-10.32-3.el8.x86_64.rpm 3.5 MB/s | 605 kB 00:00 2023-11-28T11:17:23.916 INFO:teuthology.orchestra.run.smithi169.stdout:(85/149): pcre2-utf16-10.32-3.el8.x86_64.rpm 1.6 MB/s | 229 kB 00:00 2023-11-28T11:17:24.050 INFO:teuthology.orchestra.run.smithi169.stdout:(86/149): pcre2-utf32-10.32-3.el8.x86_64.rpm 1.6 MB/s | 220 kB 00:00 2023-11-28T11:17:24.150 INFO:teuthology.orchestra.run.smithi169.stdout:(87/149): openssl-devel-1.1.1k-11.el8.x86_64.rp 5.1 MB/s | 3.2 MB 00:00 2023-11-28T11:17:24.175 INFO:teuthology.orchestra.run.smithi169.stdout:(88/149): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.9 MB/s | 249 kB 00:00 2023-11-28T11:17:24.283 INFO:teuthology.orchestra.run.smithi169.stdout:(89/149): python3-cryptography-3.2.1-6.el8.x86_ 4.7 MB/s | 641 kB 00:00 2023-11-28T11:17:24.309 INFO:teuthology.orchestra.run.smithi169.stdout:(90/149): python3-jwt-1.6.1-2.el8.noarch.rpm 319 kB/s | 43 kB 00:00 2023-11-28T11:17:24.392 INFO:teuthology.orchestra.run.smithi169.stdout:(91/149): python3-oauthlib-2.1.0-1.el8.noarch.r 1.4 MB/s | 155 kB 00:00 2023-11-28T11:17:24.426 INFO:teuthology.orchestra.run.smithi169.stdout:(92/149): python3-pycparser-2.14-14.el8.noarch. 935 kB/s | 109 kB 00:00 2023-11-28T11:17:24.493 INFO:teuthology.orchestra.run.smithi169.stdout:(93/149): python3-requests-oauthlib-1.0.0-1.el8 426 kB/s | 43 kB 00:00 2023-11-28T11:17:24.527 INFO:teuthology.orchestra.run.smithi169.stdout:(94/149): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 744 kB/s | 74 kB 00:00 2023-11-28T11:17:24.602 INFO:teuthology.orchestra.run.smithi169.stdout:(95/149): zlib-devel-1.2.11-21.el8.x86_64.rpm 535 kB/s | 58 kB 00:00 2023-11-28T11:17:24.952 INFO:teuthology.orchestra.run.smithi169.stdout:(96/149): liboath-2.6.2-3.el8.x86_64.rpm 168 kB/s | 59 kB 00:00 2023-11-28T11:17:25.128 INFO:teuthology.orchestra.run.smithi169.stdout:(97/149): gperftools-libs-2.7-9.el8.x86_64.rpm 510 kB/s | 306 kB 00:00 2023-11-28T11:17:25.153 INFO:teuthology.orchestra.run.smithi169.stdout:(98/149): libunwind-1.3.1-3.el8.x86_64.rpm 375 kB/s | 75 kB 00:00 2023-11-28T11:17:25.264 INFO:teuthology.orchestra.run.smithi092.stdout:(124/149): python3-webtest-2.0.33-1.el8.noarch. 43 kB/s | 85 kB 00:01 2023-11-28T11:17:25.269 INFO:teuthology.orchestra.run.smithi169.stdout:(99/149): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 375 kB/s | 44 kB 00:00 2023-11-28T11:17:25.337 INFO:teuthology.orchestra.run.smithi169.stdout:(100/149): python3-asyncssh-2.7.0-2.el8.noarch. 2.2 MB/s | 476 kB 00:00 2023-11-28T11:17:25.364 INFO:teuthology.orchestra.run.smithi092.stdout:(125/149): python3-zc-lockfile-2.0-2.el8.noarch 231 kB/s | 23 kB 00:00 2023-11-28T11:17:25.437 INFO:teuthology.orchestra.run.smithi169.stdout:(101/149): python3-beautifulsoup4-4.6.3-2.el8.1 1.1 MB/s | 185 kB 00:00 2023-11-28T11:17:25.454 INFO:teuthology.orchestra.run.smithi169.stdout:(102/149): python3-cachetools-3.1.1-4.el8.noarc 282 kB/s | 33 kB 00:00 2023-11-28T11:17:25.571 INFO:teuthology.orchestra.run.smithi169.stdout:(103/149): python3-certifi-2018.10.15-7.el8.noa 122 kB/s | 16 kB 00:00 2023-11-28T11:17:25.588 INFO:teuthology.orchestra.run.smithi169.stdout:(104/149): python3-cheroot-8.5.2-1.el8.noarch.r 1.3 MB/s | 173 kB 00:00 2023-11-28T11:17:25.680 INFO:teuthology.orchestra.run.smithi169.stdout:(105/149): python3-google-auth-1.1.1-10.el8.noa 885 kB/s | 81 kB 00:00 2023-11-28T11:17:25.755 INFO:teuthology.orchestra.run.smithi169.stdout:(106/149): python3-cherrypy-18.4.0-1.el8.noarch 2.0 MB/s | 384 kB 00:00 2023-11-28T11:17:25.814 INFO:teuthology.orchestra.run.smithi169.stdout:(107/149): python3-jaraco-6.2-6.el8.noarch.rpm 83 kB/s | 11 kB 00:00 2023-11-28T11:17:25.906 INFO:teuthology.orchestra.run.smithi169.stdout:(108/149): python3-jaraco-functools-2.0-4.el8.n 117 kB/s | 18 kB 00:00 2023-11-28T11:17:26.023 INFO:teuthology.orchestra.run.smithi169.stdout:(109/149): python3-logutils-0.3.5-11.el8.noarch 417 kB/s | 49 kB 00:00 2023-11-28T11:17:26.148 INFO:teuthology.orchestra.run.smithi169.stdout:(110/149): python3-kubernetes-11.0.0-6.el8.noar 3.5 MB/s | 1.2 MB 00:00 2023-11-28T11:17:26.204 INFO:teuthology.orchestra.run.smithi169.stdout:(111/149): python3-more-itertools-7.2.0-3.el8.n 325 kB/s | 59 kB 00:00 2023-11-28T11:17:26.266 INFO:teuthology.orchestra.run.smithi169.stdout:(112/149): python3-natsort-7.1.1-2.el8.noarch.r 515 kB/s | 60 kB 00:00 2023-11-28T11:17:26.357 INFO:teuthology.orchestra.run.smithi169.stdout:(113/149): python3-pecan-1.3.2-9.el8.noarch.rpm 1.8 MB/s | 283 kB 00:00 2023-11-28T11:17:26.383 INFO:teuthology.orchestra.run.smithi169.stdout:(114/149): python3-portend-2.6-1.el8.noarch.rpm 133 kB/s | 16 kB 00:00 2023-11-28T11:17:26.475 INFO:teuthology.orchestra.run.smithi169.stdout:(115/149): python3-repoze-lru-0.7-6.el8.noarch. 287 kB/s | 34 kB 00:00 2023-11-28T11:17:26.550 INFO:teuthology.orchestra.run.smithi169.stdout:(116/149): python3-routes-2.4.1-12.el8.noarch.r 1.2 MB/s | 196 kB 00:00 2023-11-28T11:17:26.584 INFO:teuthology.orchestra.run.smithi169.stdout:(117/149): python3-rsa-4.9-2.el8.noarch.rpm 592 kB/s | 64 kB 00:00 2023-11-28T11:17:26.676 INFO:teuthology.orchestra.run.smithi169.stdout:(118/149): python3-simplegeneric-0.8.1-17.el8.n 149 kB/s | 19 kB 00:00 2023-11-28T11:17:26.709 INFO:teuthology.orchestra.run.smithi169.stdout:(119/149): python3-singledispatch-3.4.0.3-18.el 191 kB/s | 24 kB 00:00 2023-11-28T11:17:26.793 INFO:teuthology.orchestra.run.smithi169.stdout:(120/149): python3-tempora-1.14.1-5.el8.noarch. 250 kB/s | 29 kB 00:00 2023-11-28T11:17:26.818 INFO:teuthology.orchestra.run.smithi169.stdout:(121/149): python3-trustme-0.6.0-4.el8.noarch.r 253 kB/s | 27 kB 00:00 2023-11-28T11:17:26.985 INFO:teuthology.orchestra.run.smithi169.stdout:(122/149): python3-waitress-1.4.3-1.el8.noarch. 1.3 MB/s | 255 kB 00:00 2023-11-28T11:17:27.011 INFO:teuthology.orchestra.run.smithi169.stdout:(123/149): python3-webob-1.8.5-1.el8.1.noarch.r 1.3 MB/s | 251 kB 00:00 2023-11-28T11:17:27.103 INFO:teuthology.orchestra.run.smithi169.stdout:(124/149): python3-websocket-client-0.56.0-5.el 524 kB/s | 61 kB 00:00 2023-11-28T11:17:27.128 INFO:teuthology.orchestra.run.smithi169.stdout:(125/149): python3-webtest-2.0.33-1.el8.noarch. 731 kB/s | 85 kB 00:00 2023-11-28T11:17:27.220 INFO:teuthology.orchestra.run.smithi169.stdout:(126/149): python3-zc-lockfile-2.0-2.el8.noarch 196 kB/s | 23 kB 00:00 2023-11-28T11:17:27.587 INFO:teuthology.orchestra.run.smithi169.stdout:(127/149): thrift-0.13.0-2.el8.x86_64.rpm 3.8 MB/s | 1.7 MB 00:00 2023-11-28T11:17:27.654 INFO:teuthology.orchestra.run.smithi169.stdout:(128/149): librados2-18.0.0-2697.g2f5f6028.el8. 8.4 MB/s | 3.7 MB 00:00 2023-11-28T11:17:27.771 INFO:teuthology.orchestra.run.smithi169.stdout:(129/149): NetworkManager-initscripts-updown-1. 1.3 MB/s | 144 kB 00:00 2023-11-28T11:17:27.888 INFO:teuthology.orchestra.run.smithi169.stdout:(130/149): acl-2.2.53-3.el8.x86_64.rpm 707 kB/s | 83 kB 00:00 2023-11-28T11:17:27.996 INFO:teuthology.orchestra.run.smithi169.stdout:(131/149): elfutils-debuginfod-client-0.190-1.e 709 kB/s | 77 kB 00:00 2023-11-28T11:17:28.114 INFO:teuthology.orchestra.run.smithi169.stdout:(132/149): elfutils-libelf-0.190-1.el8.x86_64.r 1.9 MB/s | 232 kB 00:00 2023-11-28T11:17:28.206 INFO:teuthology.orchestra.run.smithi169.stdout:(133/149): librbd1-18.0.0-2697.g2f5f6028.el8.x8 5.8 MB/s | 3.6 MB 00:00 2023-11-28T11:17:28.231 INFO:teuthology.orchestra.run.smithi169.stdout:(134/149): elfutils-libs-0.190-1.el8.x86_64.rpm 2.5 MB/s | 305 kB 00:00 2023-11-28T11:17:28.339 INFO:teuthology.orchestra.run.smithi169.stdout:(135/149): iproute-6.2.0-5.el8.x86_64.rpm 6.3 MB/s | 854 kB 00:00 2023-11-28T11:17:28.356 INFO:teuthology.orchestra.run.smithi169.stdout:(136/149): libacl-2.2.53-3.el8.x86_64.rpm 278 kB/s | 35 kB 00:00 2023-11-28T11:17:28.440 INFO:teuthology.orchestra.run.smithi169.stdout:(137/149): libacl-devel-2.2.53-3.el8.x86_64.rpm 849 kB/s | 85 kB 00:00 2023-11-28T11:17:28.474 INFO:teuthology.orchestra.run.smithi169.stdout:(138/149): libblkid-2.32.1-43.el8.x86_64.rpm 1.8 MB/s | 220 kB 00:00 2023-11-28T11:17:28.557 INFO:teuthology.orchestra.run.smithi169.stdout:(139/149): libfdisk-2.32.1-43.el8.x86_64.rpm 2.1 MB/s | 254 kB 00:00 2023-11-28T11:17:28.591 INFO:teuthology.orchestra.run.smithi169.stdout:(140/149): libmount-2.32.1-43.el8.x86_64.rpm 2.0 MB/s | 236 kB 00:00 2023-11-28T11:17:28.666 INFO:teuthology.orchestra.run.smithi169.stdout:(141/149): libsmartcols-2.32.1-43.el8.x86_64.rp 1.6 MB/s | 179 kB 00:00 2023-11-28T11:17:28.691 INFO:teuthology.orchestra.run.smithi169.stdout:(142/149): libuuid-2.32.1-43.el8.x86_64.rpm 988 kB/s | 99 kB 00:00 2023-11-28T11:17:28.792 INFO:teuthology.orchestra.run.smithi169.stdout:(143/149): openssl-1.1.1k-11.el8.x86_64.rpm 5.8 MB/s | 737 kB 00:00 2023-11-28T11:17:28.942 INFO:teuthology.orchestra.run.smithi169.stdout:(144/149): openssl-libs-1.1.1k-11.el8.x86_64.rp 5.9 MB/s | 1.5 MB 00:00 2023-11-28T11:17:29.001 INFO:teuthology.orchestra.run.smithi169.stdout:(145/149): util-linux-2.32.1-43.el8.x86_64.rpm 12 MB/s | 2.5 MB 00:00 2023-11-28T11:17:29.051 INFO:teuthology.orchestra.run.smithi169.stdout:(146/149): util-linux-user-2.32.1-43.el8.x86_64 948 kB/s | 103 kB 00:00 2023-11-28T11:17:29.110 INFO:teuthology.orchestra.run.smithi169.stdout:(147/149): xfsdump-3.1.8-7.el8.x86_64.rpm 3.0 MB/s | 334 kB 00:00 2023-11-28T11:17:29.209 INFO:teuthology.orchestra.run.smithi169.stdout:(148/149): xfsprogs-5.0.0-12.el8.x86_64.rpm 6.8 MB/s | 1.1 MB 00:00 2023-11-28T11:17:30.163 INFO:teuthology.orchestra.run.smithi092.stdout:(126/149): python3-routes-2.4.1-12.el8.noarch.r 24 kB/s | 196 kB 00:08 2023-11-28T11:17:31.097 INFO:teuthology.orchestra.run.smithi092.stdout:(127/149): librados2-18.0.0-2697.g2f5f6028.el8. 3.9 MB/s | 3.7 MB 00:00 2023-11-28T11:17:31.572 INFO:teuthology.orchestra.run.smithi092.stdout:(128/149): librbd1-18.0.0-2697.g2f5f6028.el8.x8 7.5 MB/s | 3.6 MB 00:00 2023-11-28T11:17:31.697 INFO:teuthology.orchestra.run.smithi092.stdout:(129/149): NetworkManager-initscripts-updown-1. 1.1 MB/s | 144 kB 00:00 2023-11-28T11:17:31.822 INFO:teuthology.orchestra.run.smithi092.stdout:(130/149): acl-2.2.53-3.el8.x86_64.rpm 661 kB/s | 83 kB 00:00 2023-11-28T11:17:31.964 INFO:teuthology.orchestra.run.smithi092.stdout:(131/149): elfutils-debuginfod-client-0.190-1.e 541 kB/s | 77 kB 00:00 2023-11-28T11:17:32.073 INFO:teuthology.orchestra.run.smithi092.stdout:(132/149): elfutils-libelf-0.190-1.el8.x86_64.r 2.1 MB/s | 232 kB 00:00 2023-11-28T11:17:32.190 INFO:teuthology.orchestra.run.smithi092.stdout:(133/149): elfutils-libs-0.190-1.el8.x86_64.rpm 2.6 MB/s | 305 kB 00:00 2023-11-28T11:17:32.307 INFO:teuthology.orchestra.run.smithi092.stdout:(134/149): python3-scipy-1.0.0-21.module_el8.5. 305 kB/s | 14 MB 00:46 2023-11-28T11:17:32.340 INFO:teuthology.orchestra.run.smithi092.stdout:(135/149): iproute-6.2.0-5.el8.x86_64.rpm 5.6 MB/s | 854 kB 00:00 2023-11-28T11:17:32.441 INFO:teuthology.orchestra.run.smithi092.stdout:(136/149): libacl-2.2.53-3.el8.x86_64.rpm 261 kB/s | 35 kB 00:00 2023-11-28T11:17:32.466 INFO:teuthology.orchestra.run.smithi092.stdout:(137/149): libacl-devel-2.2.53-3.el8.x86_64.rpm 677 kB/s | 85 kB 00:00 2023-11-28T11:17:32.585 INFO:teuthology.orchestra.run.smithi092.stdout:(138/149): libfdisk-2.32.1-43.el8.x86_64.rpm 2.1 MB/s | 254 kB 00:00 2023-11-28T11:17:32.700 INFO:teuthology.orchestra.run.smithi092.stdout:(139/149): libmount-2.32.1-43.el8.x86_64.rpm 2.0 MB/s | 236 kB 00:00 2023-11-28T11:17:32.800 INFO:teuthology.orchestra.run.smithi092.stdout:(140/149): libblkid-2.32.1-43.el8.x86_64.rpm 614 kB/s | 220 kB 00:00 2023-11-28T11:17:32.825 INFO:teuthology.orchestra.run.smithi092.stdout:(141/149): libsmartcols-2.32.1-43.el8.x86_64.rp 1.4 MB/s | 179 kB 00:00 2023-11-28T11:17:32.926 INFO:teuthology.orchestra.run.smithi092.stdout:(142/149): libuuid-2.32.1-43.el8.x86_64.rpm 791 kB/s | 99 kB 00:00 2023-11-28T11:17:32.968 INFO:teuthology.orchestra.run.smithi092.stdout:(143/149): openssl-1.1.1k-11.el8.x86_64.rpm 5.1 MB/s | 737 kB 00:00 2023-11-28T11:17:33.143 INFO:teuthology.orchestra.run.smithi092.stdout:(144/149): util-linux-2.32.1-43.el8.x86_64.rpm 14 MB/s | 2.5 MB 00:00 2023-11-28T11:17:33.235 INFO:teuthology.orchestra.run.smithi092.stdout:(145/149): openssl-libs-1.1.1k-11.el8.x86_64.rp 4.8 MB/s | 1.5 MB 00:00 2023-11-28T11:17:33.260 INFO:teuthology.orchestra.run.smithi092.stdout:(146/149): util-linux-user-2.32.1-43.el8.x86_64 881 kB/s | 103 kB 00:00 2023-11-28T11:17:33.352 INFO:teuthology.orchestra.run.smithi092.stdout:(147/149): xfsdump-3.1.8-7.el8.x86_64.rpm 2.8 MB/s | 334 kB 00:00 2023-11-28T11:17:33.394 INFO:teuthology.orchestra.run.smithi092.stdout:(148/149): xfsprogs-5.0.0-12.el8.x86_64.rpm 8.1 MB/s | 1.1 MB 00:00 2023-11-28T11:17:35.444 INFO:teuthology.orchestra.run.smithi092.stdout:(149/149): thrift-0.13.0-2.el8.x86_64.rpm 176 kB/s | 1.7 MB 00:10 2023-11-28T11:17:35.446 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2023-11-28T11:17:35.446 INFO:teuthology.orchestra.run.smithi092.stdout:Total 2.3 MB/s | 212 MB 01:31 2023-11-28T11:17:37.426 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2023-11-28T11:17:37.877 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2023-11-28T11:17:37.877 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2023-11-28T11:17:39.387 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2023-11-28T11:17:39.391 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2023-11-28T11:17:44.965 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2023-11-28T11:17:45.164 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: openssl-libs-1:1.1.1k-11.el8.x86_64 1/1 2023-11-28T11:17:45.351 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : openssl-libs-1:1.1.1k-11.el8.x86_64 1/170 2023-11-28T11:17:45.591 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: openssl-libs-1:1.1.1k-11.el8.x86_64 1/170 2023-11-28T11:17:45.810 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/170 2023-11-28T11:17:45.963 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 3/170 2023-11-28T11:17:46.039 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 3/170 2023-11-28T11:17:46.271 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 4/170 2023-11-28T11:17:46.361 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 4/170 2023-11-28T11:17:46.573 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/170 2023-11-28T11:17:46.771 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 6/170 2023-11-28T11:17:46.857 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 6/170 2023-11-28T11:17:47.076 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 7/170 2023-11-28T11:17:47.366 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : elfutils-libelf-0.190-1.el8.x86_64 8/170 2023-11-28T11:17:47.562 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 9/170 2023-11-28T11:17:48.013 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 9/170 2023-11-28T11:17:48.187 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 10/170 2023-11-28T11:17:48.650 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 10/170 2023-11-28T11:17:48.837 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 11/170 2023-11-28T11:17:48.969 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 11/170 2023-11-28T11:17:49.154 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 12/170 2023-11-28T11:17:49.273 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 12/170 2023-11-28T11:17:49.638 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 13/170 2023-11-28T11:17:49.908 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 14/170 2023-11-28T11:17:50.013 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 14/170 2023-11-28T11:17:50.192 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libpq-13.5-1.el8.x86_64 15/170 2023-11-28T11:17:50.453 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 16/170 2023-11-28T11:17:50.909 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 17/170 2023-11-28T11:17:51.375 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : postgresql-10.23-2.module_el8+572+929c87ac.x86_6 18/170 2023-11-28T11:17:51.553 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 19/170 2023-11-28T11:17:51.672 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 19/170 2023-11-28T11:17:51.820 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 20/170 2023-11-28T11:17:51.888 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 20/170 2023-11-28T11:17:52.115 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : libacl-2.2.53-3.el8.x86_64 21/170 2023-11-28T11:17:52.312 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 22/170 2023-11-28T11:17:52.501 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : zlib-devel-1.2.11-21.el8.x86_64 23/170 2023-11-28T11:17:52.686 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libquadmath-8.5.0-21.el8.x86_64 24/170 2023-11-28T11:17:52.848 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libquadmath-8.5.0-21.el8.x86_64 24/170 2023-11-28T11:17:53.028 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libgfortran-8.5.0-21.el8.x86_64 25/170 2023-11-28T11:17:53.693 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libgfortran-8.5.0-21.el8.x86_64 25/170 2023-11-28T11:17:53.944 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openblas-0.3.15-4.el8.x86_64 26/170 2023-11-28T11:17:54.627 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 26/170 2023-11-28T11:17:54.836 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 27/170 2023-11-28T11:17:55.659 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 27/170 2023-11-28T11:17:56.028 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 28/170 2023-11-28T11:17:56.228 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 29/170 2023-11-28T11:17:56.384 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 30/170 2023-11-28T11:17:56.604 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 31/170 2023-11-28T11:17:56.793 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8. 32/170 2023-11-28T11:17:56.947 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el 33/170 2023-11-28T11:17:57.217 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_6 34/170 2023-11-28T11:17:58.296 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 35/170 2023-11-28T11:17:58.681 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 36/170 2023-11-28T11:17:58.995 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 37/170 2023-11-28T11:17:59.210 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 38/170 2023-11-28T11:17:59.289 INFO:teuthology.orchestra.run.smithi169.stdout:(149/149): python3-scipy-1.0.0-21.module_el8.5. 269 kB/s | 14 MB 00:52 2023-11-28T11:17:59.292 INFO:teuthology.orchestra.run.smithi169.stdout:-------------------------------------------------------------------------------- 2023-11-28T11:17:59.292 INFO:teuthology.orchestra.run.smithi169.stdout:Total 1.9 MB/s | 212 MB 01:54 2023-11-28T11:18:01.300 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 38/170 2023-11-28T11:18:01.313 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction check 2023-11-28T11:18:01.630 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 39/170 2023-11-28T11:18:01.770 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction check succeeded. 2023-11-28T11:18:01.770 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction test 2023-11-28T11:18:01.816 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/170 2023-11-28T11:18:02.039 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rsa-4.9-2.el8.noarch 41/170 2023-11-28T11:18:02.225 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 42/170 2023-11-28T11:18:02.410 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_6 43/170 2023-11-28T11:18:02.613 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : elfutils-libs-0.190-1.el8.x86_64 44/170 2023-11-28T11:18:02.835 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : elfutils-debuginfod-client-0.190-1.el8.x86_64 45/170 2023-11-28T11:18:03.042 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 46/170 2023-11-28T11:18:03.136 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 46/170 2023-11-28T11:18:03.308 INFO:teuthology.orchestra.run.smithi169.stdout:Transaction test succeeded. 2023-11-28T11:18:03.312 INFO:teuthology.orchestra.run.smithi169.stdout:Running transaction 2023-11-28T11:18:03.326 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 47/170 2023-11-28T11:18:03.412 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 47/170 2023-11-28T11:18:03.627 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 48/170 2023-11-28T11:18:03.813 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 49/170 2023-11-28T11:18:03.992 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 50/170 2023-11-28T11:18:04.546 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 50/170 2023-11-28T11:18:04.851 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 51/170 2023-11-28T11:18:04.880 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 51/170 2023-11-28T11:18:05.090 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 52/170 2023-11-28T11:18:05.356 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 53/170 2023-11-28T11:18:05.583 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 54/170 2023-11-28T11:18:05.784 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 55/170 2023-11-28T11:18:05.962 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 56/170 2023-11-28T11:18:06.181 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 57/170 2023-11-28T11:18:06.391 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 58/170 2023-11-28T11:18:06.590 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 59/170 2023-11-28T11:18:06.804 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 60/170 2023-11-28T11:18:06.999 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 61/170 2023-11-28T11:18:07.181 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 62/170 2023-11-28T11:18:07.357 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 63/170 2023-11-28T11:18:07.526 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-portend-2.6-1.el8.noarch 64/170 2023-11-28T11:18:07.720 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 65/170 2023-11-28T11:18:07.967 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 66/170 2023-11-28T11:18:08.205 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 67/170 2023-11-28T11:18:08.415 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 68/170 2023-11-28T11:18:08.750 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 69/170 2023-11-28T11:18:09.018 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 70/170 2023-11-28T11:18:09.212 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 71/170 2023-11-28T11:18:09.252 INFO:teuthology.orchestra.run.smithi169.stdout: Preparing : 1/1 2023-11-28T11:18:09.447 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: openssl-libs-1:1.1.1k-11.el8.x86_64 1/1 2023-11-28T11:18:09.654 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : openssl-libs-1:1.1.1k-11.el8.x86_64 1/170 2023-11-28T11:18:09.924 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: openssl-libs-1:1.1.1k-11.el8.x86_64 1/170 2023-11-28T11:18:10.036 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 72/170 2023-11-28T11:18:10.120 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/170 2023-11-28T11:18:10.257 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 73/170 2023-11-28T11:18:10.316 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 3/170 2023-11-28T11:18:10.332 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 73/170 2023-11-28T11:18:10.551 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 74/170 2023-11-28T11:18:10.553 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 3/170 2023-11-28T11:18:10.805 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 4/170 2023-11-28T11:18:10.819 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 75/170 2023-11-28T11:18:10.900 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 4/170 2023-11-28T11:18:11.124 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/170 2023-11-28T11:18:11.235 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 76/170 2023-11-28T11:18:11.338 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 6/170 2023-11-28T11:18:11.421 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 6/170 2023-11-28T11:18:11.498 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 77/170 2023-11-28T11:18:11.620 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 7/170 2023-11-28T11:18:11.690 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 78/170 2023-11-28T11:18:11.901 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : elfutils-libelf-0.190-1.el8.x86_64 8/170 2023-11-28T11:18:11.944 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 79/170 2023-11-28T11:18:12.138 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 9/170 2023-11-28T11:18:12.166 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 80/170 2023-11-28T11:18:12.595 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 9/170 2023-11-28T11:18:12.801 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 10/170 2023-11-28T11:18:13.236 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 10/170 2023-11-28T11:18:13.430 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 11/170 2023-11-28T11:18:13.439 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 81/170 2023-11-28T11:18:13.574 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 11/170 2023-11-28T11:18:13.763 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 12/170 2023-11-28T11:18:13.801 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 82/170 2023-11-28T11:18:13.879 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 12/170 2023-11-28T11:18:14.008 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 83/170 2023-11-28T11:18:14.234 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 84/170 2023-11-28T11:18:14.239 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 13/170 2023-11-28T11:18:14.514 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 14/170 2023-11-28T11:18:14.582 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 85/170 2023-11-28T11:18:14.619 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 14/170 2023-11-28T11:18:14.818 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libpq-13.5-1.el8.x86_64 15/170 2023-11-28T11:18:14.834 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 86/170 2023-11-28T11:18:15.077 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 87/170 2023-11-28T11:18:15.095 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 16/170 2023-11-28T11:18:15.303 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : pcre2-utf32-10.32-3.el8.x86_64 88/170 2023-11-28T11:18:15.555 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 17/170 2023-11-28T11:18:15.670 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : pcre2-utf16-10.32-3.el8.x86_64 89/170 2023-11-28T11:18:15.785 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : postgresql-10.23-2.module_el8+572+929c87ac.x86_6 18/170 2023-11-28T11:18:15.870 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : pcre2-devel-10.32-3.el8.x86_64 90/170 2023-11-28T11:18:15.980 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 19/170 2023-11-28T11:18:16.025 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 91/170 2023-11-28T11:18:16.103 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 19/170 2023-11-28T11:18:16.207 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libverto-devel-0.3.2-2.el8.x86_64 92/170 2023-11-28T11:18:16.263 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 20/170 2023-11-28T11:18:16.361 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 20/170 2023-11-28T11:18:16.552 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 93/170 2023-11-28T11:18:16.566 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : libacl-2.2.53-3.el8.x86_64 21/170 2023-11-28T11:18:16.801 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libselinux-devel-2.9-8.el8.x86_64 94/170 2023-11-28T11:18:16.806 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 22/170 2023-11-28T11:18:16.955 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcom_err-devel-1.45.6-5.el8.x86_64 95/170 2023-11-28T11:18:17.024 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : zlib-devel-1.2.11-21.el8.x86_64 23/170 2023-11-28T11:18:17.154 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 96/170 2023-11-28T11:18:17.171 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libquadmath-8.5.0-21.el8.x86_64 24/170 2023-11-28T11:18:17.333 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libquadmath-8.5.0-21.el8.x86_64 24/170 2023-11-28T11:18:17.425 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : krb5-devel-1.18.2-22.el8.x86_64 97/170 2023-11-28T11:18:17.504 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libgfortran-8.5.0-21.el8.x86_64 25/170 2023-11-28T11:18:17.633 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : fuse-2.9.7-17.el8.x86_64 98/170 2023-11-28T11:18:17.868 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 99/170 2023-11-28T11:18:18.097 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : uuid-1.6.2-43.el8.x86_64 100/170 2023-11-28T11:18:18.167 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libgfortran-8.5.0-21.el8.x86_64 25/170 2023-11-28T11:18:18.190 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 100/170 2023-11-28T11:18:18.340 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 101/170 2023-11-28T11:18:18.412 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : openblas-0.3.15-4.el8.x86_64 26/170 2023-11-28T11:18:19.098 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 26/170 2023-11-28T11:18:19.262 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 27/170 2023-11-28T11:18:20.131 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 27/170 2023-11-28T11:18:20.481 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 28/170 2023-11-28T11:18:20.721 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 29/170 2023-11-28T11:18:20.886 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 30/170 2023-11-28T11:18:21.034 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 102/170 2023-11-28T11:18:21.124 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 31/170 2023-11-28T11:18:21.289 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 102/170 2023-11-28T11:18:21.361 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8. 32/170 2023-11-28T11:18:21.482 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el 33/170 2023-11-28T11:18:21.714 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_6 34/170 2023-11-28T11:18:21.956 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 102/170 2023-11-28T11:18:22.798 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 35/170 2023-11-28T11:18:23.019 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 103/170 2023-11-28T11:18:23.210 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 36/170 2023-11-28T11:18:23.242 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 103/170 2023-11-28T11:18:23.421 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 104/170 2023-11-28T11:18:23.564 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 37/170 2023-11-28T11:18:23.795 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 38/170 2023-11-28T11:18:25.893 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 38/170 2023-11-28T11:18:26.233 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 39/170 2023-11-28T11:18:26.452 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/170 2023-11-28T11:18:26.649 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rsa-4.9-2.el8.noarch 41/170 2023-11-28T11:18:26.868 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 42/170 2023-11-28T11:18:27.086 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_6 43/170 2023-11-28T11:18:27.293 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : elfutils-libs-0.190-1.el8.x86_64 44/170 2023-11-28T11:18:27.498 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : elfutils-debuginfod-client-0.190-1.el8.x86_64 45/170 2023-11-28T11:18:27.694 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 46/170 2023-11-28T11:18:27.804 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 46/170 2023-11-28T11:18:27.994 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 47/170 2023-11-28T11:18:28.105 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 47/170 2023-11-28T11:18:28.274 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 48/170 2023-11-28T11:18:28.492 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 49/170 2023-11-28T11:18:28.660 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 50/170 2023-11-28T11:18:29.298 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 50/170 2023-11-28T11:18:29.603 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 51/170 2023-11-28T11:18:29.632 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 51/170 2023-11-28T11:18:29.836 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 52/170 2023-11-28T11:18:30.096 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 53/170 2023-11-28T11:18:30.302 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 54/170 2023-11-28T11:18:30.492 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 55/170 2023-11-28T11:18:30.640 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 56/170 2023-11-28T11:18:30.844 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 57/170 2023-11-28T11:18:31.053 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 58/170 2023-11-28T11:18:31.294 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 59/170 2023-11-28T11:18:31.523 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 60/170 2023-11-28T11:18:31.676 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 61/170 2023-11-28T11:18:31.867 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 62/170 2023-11-28T11:18:32.043 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 63/170 2023-11-28T11:18:32.221 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-portend-2.6-1.el8.noarch 64/170 2023-11-28T11:18:32.381 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 65/170 2023-11-28T11:18:32.612 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 66/170 2023-11-28T11:18:32.918 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 67/170 2023-11-28T11:18:33.110 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 68/170 2023-11-28T11:18:33.460 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 69/170 2023-11-28T11:18:33.720 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 70/170 2023-11-28T11:18:33.949 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 71/170 2023-11-28T11:18:34.784 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 72/170 2023-11-28T11:18:35.001 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 73/170 2023-11-28T11:18:35.075 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 73/170 2023-11-28T11:18:35.305 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 74/170 2023-11-28T11:18:35.591 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 75/170 2023-11-28T11:18:36.048 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 76/170 2023-11-28T11:18:36.317 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 77/170 2023-11-28T11:18:36.492 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 78/170 2023-11-28T11:18:36.774 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 79/170 2023-11-28T11:18:37.011 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 80/170 2023-11-28T11:18:38.336 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 81/170 2023-11-28T11:18:38.729 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 82/170 2023-11-28T11:18:38.969 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 83/170 2023-11-28T11:18:39.180 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 84/170 2023-11-28T11:18:39.502 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 85/170 2023-11-28T11:18:39.796 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 86/170 2023-11-28T11:18:40.097 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 87/170 2023-11-28T11:18:40.339 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : pcre2-utf32-10.32-3.el8.x86_64 88/170 2023-11-28T11:18:40.727 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : pcre2-utf16-10.32-3.el8.x86_64 89/170 2023-11-28T11:18:40.998 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : pcre2-devel-10.32-3.el8.x86_64 90/170 2023-11-28T11:18:41.587 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 91/170 2023-11-28T11:18:41.807 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libverto-devel-0.3.2-2.el8.x86_64 92/170 2023-11-28T11:18:42.151 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 93/170 2023-11-28T11:18:42.430 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libselinux-devel-2.9-8.el8.x86_64 94/170 2023-11-28T11:18:42.575 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libcom_err-devel-1.45.6-5.el8.x86_64 95/170 2023-11-28T11:18:42.812 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 96/170 2023-11-28T11:18:43.079 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : krb5-devel-1.18.2-22.el8.x86_64 97/170 2023-11-28T11:18:43.262 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : fuse-2.9.7-17.el8.x86_64 98/170 2023-11-28T11:18:43.504 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 99/170 2023-11-28T11:18:43.617 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 104/170 2023-11-28T11:18:43.617 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /sys 2023-11-28T11:18:43.617 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /proc 2023-11-28T11:18:43.617 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /dev 2023-11-28T11:18:43.618 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /run 2023-11-28T11:18:43.618 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /mnt 2023-11-28T11:18:43.618 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /var/tmp 2023-11-28T11:18:43.618 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /home 2023-11-28T11:18:43.618 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /tmp 2023-11-28T11:18:43.618 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /dev 2023-11-28T11:18:43.618 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:18:43.725 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : uuid-1.6.2-43.el8.x86_64 100/170 2023-11-28T11:18:43.803 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 100/170 2023-11-28T11:18:43.934 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 101/170 2023-11-28T11:18:44.009 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 105/170 2023-11-28T11:18:45.722 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 105/170 2023-11-28T11:18:45.955 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 106/170 2023-11-28T11:18:46.222 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 106/170 2023-11-28T11:18:46.425 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 107/170 2023-11-28T11:18:46.775 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 102/170 2023-11-28T11:18:46.940 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 107/170 2023-11-28T11:18:47.142 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 108/170 2023-11-28T11:18:47.192 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 108/170 2023-11-28T11:18:47.303 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 102/170 2023-11-28T11:18:47.400 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 109/170 2023-11-28T11:18:47.612 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 110/170 2023-11-28T11:18:47.791 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 111/170 2023-11-28T11:18:47.868 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 111/170 2023-11-28T11:18:47.979 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 102/170 2023-11-28T11:18:48.183 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : jq-1.6-8.el8.x86_64 112/170 2023-11-28T11:18:48.299 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 103/170 2023-11-28T11:18:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : automake-1.16.1-8.el8.noarch 113/170 2023-11-28T11:18:48.496 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 103/170 2023-11-28T11:18:48.666 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 104/170 2023-11-28T11:18:49.027 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 114/170 2023-11-28T11:18:49.208 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 114/170 2023-11-28T11:18:49.315 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.e 115/170 2023-11-28T11:18:49.494 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028. 116/170 2023-11-28T11:18:49.723 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 117/170 2023-11-28T11:18:49.981 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 117/170 2023-11-28T11:18:50.204 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 118/170 2023-11-28T11:18:51.289 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 118/170 2023-11-28T11:18:51.481 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 119/170 2023-11-28T11:18:51.611 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 119/170 2023-11-28T11:18:51.991 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el 120/170 2023-11-28T11:18:52.237 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 121/170 2023-11-28T11:18:52.303 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 121/170 2023-11-28T11:18:52.473 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 122/170 2023-11-28T11:18:52.501 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 122/170 2023-11-28T11:18:52.716 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 123/170 2023-11-28T11:18:52.906 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libtool-2.4.6-25.el8.x86_64 124/170 2023-11-28T11:18:57.668 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 124/170 2023-11-28T11:18:59.161 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 125/170 2023-11-28T11:18:59.444 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 126/170 2023-11-28T11:18:59.506 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 126/170 2023-11-28T11:18:59.756 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 127/170 2023-11-28T11:18:59.826 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 127/170 2023-11-28T11:19:00.030 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 128/170 2023-11-28T11:19:00.432 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 128/170 2023-11-28T11:19:00.655 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 129/170 2023-11-28T11:19:00.930 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 129/170 2023-11-28T11:19:01.258 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : postgresql-contrib-10.23-2.module_el8+572+929c87 130/170 2023-11-28T11:19:02.535 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 131/170 2023-11-28T11:19:02.868 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openssl-devel-1:1.1.1k-11.el8.x86_64 132/170 2023-11-28T11:19:03.112 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : elfutils-libelf-devel-0.190-1.el8.x86_64 133/170 2023-11-28T11:19:03.386 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 134/170 2023-11-28T11:19:03.584 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 135/170 2023-11-28T11:19:03.780 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_ 136/170 2023-11-28T11:19:04.107 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : acl-2.2.53-3.el8.x86_64 137/170 2023-11-28T11:19:04.263 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : libacl-devel-2.2.53-3.el8.x86_64 138/170 2023-11-28T11:19:05.431 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: postgresql-server-10.23-2.module_el8+572+929c87a 139/170 2023-11-28T11:19:05.704 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : postgresql-server-10.23-2.module_el8+572+929c87a 139/170 2023-11-28T11:19:05.791 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: postgresql-server-10.23-2.module_el8+572+929c87a 139/170 2023-11-28T11:19:06.050 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : xfsdump-3.1.8-7.el8.x86_64 140/170 2023-11-28T11:19:06.288 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 141/170 2023-11-28T11:19:06.546 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 142/170 2023-11-28T11:19:06.932 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 143/170 2023-11-28T11:19:07.338 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : iproute-6.2.0-5.el8.x86_64 144/170 2023-11-28T11:19:07.597 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : openssl-1:1.1.1k-11.el8.x86_64 145/170 2023-11-28T11:19:07.753 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : NetworkManager-initscripts-updown-1:1.40.16-11.e 146/170 2023-11-28T11:19:07.781 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-11.e 146/170 2023-11-28T11:19:07.977 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 147/170 2023-11-28T11:19:08.228 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flex-2.6.1-9.el8.x86_64 148/170 2023-11-28T11:19:08.344 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 148/170 2023-11-28T11:19:08.628 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : bison-3.0.4-10.el8.x86_64 149/170 2023-11-28T11:19:08.655 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 149/170 2023-11-28T11:19:08.846 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : elfutils-debuginfod-client-0.189-1.el8.x86_64 150/170 2023-11-28T11:19:09.071 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : elfutils-libs-0.189-1.el8.x86_64 151/170 2023-11-28T11:19:09.081 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 104/170 2023-11-28T11:19:09.081 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /sys 2023-11-28T11:19:09.081 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /proc 2023-11-28T11:19:09.081 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /dev 2023-11-28T11:19:09.081 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /run 2023-11-28T11:19:09.081 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /mnt 2023-11-28T11:19:09.081 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /var/tmp 2023-11-28T11:19:09.081 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /home 2023-11-28T11:19:09.081 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /tmp 2023-11-28T11:19:09.081 INFO:teuthology.orchestra.run.smithi169.stdout:skipping the directory /dev 2023-11-28T11:19:09.082 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-28T11:19:09.280 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : openssl-1:1.1.1k-9.el8.x86_64 152/170 2023-11-28T11:19:09.513 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 105/170 2023-11-28T11:19:09.525 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 153/170 2023-11-28T11:19:09.525 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 154/170 2023-11-28T11:19:09.832 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 154/170 2023-11-28T11:19:09.832 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 155/170 2023-11-28T11:19:10.123 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 155/170 2023-11-28T11:19:10.237 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : libacl-devel-2.2.53-1.el8.x86_64 156/170 2023-11-28T11:19:10.255 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-4.el 157/170 2023-11-28T11:19:10.375 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : NetworkManager-initscripts-updown-1:1.40.16-4.el 157/170 2023-11-28T11:19:10.376 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 158/170 2023-11-28T11:19:10.706 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 158/170 2023-11-28T11:19:10.926 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : acl-2.2.53-1.el8.x86_64 159/170 2023-11-28T11:19:11.154 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : iproute-6.2.0-1.el8.x86_64 160/170 2023-11-28T11:19:11.213 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 105/170 2023-11-28T11:19:11.376 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : util-linux-user-2.32.1-41.el8.x86_64 161/170 2023-11-28T11:19:11.453 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 106/170 2023-11-28T11:19:11.637 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : util-linux-2.32.1-41.el8.x86_64 162/170 2023-11-28T11:19:11.638 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : libfdisk-2.32.1-41.el8.x86_64 163/170 2023-11-28T11:19:11.719 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 106/170 2023-11-28T11:19:11.880 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libfdisk-2.32.1-41.el8.x86_64 163/170 2023-11-28T11:19:11.881 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : libmount-2.32.1-41.el8.x86_64 164/170 2023-11-28T11:19:11.913 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 107/170 2023-11-28T11:19:12.114 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libmount-2.32.1-41.el8.x86_64 164/170 2023-11-28T11:19:12.114 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : libblkid-2.32.1-41.el8.x86_64 165/170 2023-11-28T11:19:12.347 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libblkid-2.32.1-41.el8.x86_64 165/170 2023-11-28T11:19:12.347 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : libuuid-2.32.1-41.el8.x86_64 166/170 2023-11-28T11:19:12.429 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 107/170 2023-11-28T11:19:12.589 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libuuid-2.32.1-41.el8.x86_64 166/170 2023-11-28T11:19:12.589 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : libsmartcols-2.32.1-41.el8.x86_64 167/170 2023-11-28T11:19:12.612 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 108/170 2023-11-28T11:19:12.664 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 108/170 2023-11-28T11:19:12.854 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 109/170 2023-11-28T11:19:12.855 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libsmartcols-2.32.1-41.el8.x86_64 167/170 2023-11-28T11:19:13.095 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : elfutils-libelf-0.189-1.el8.x86_64 168/170 2023-11-28T11:19:13.100 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 110/170 2023-11-28T11:19:13.253 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 111/170 2023-11-28T11:19:13.254 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : libacl-2.2.53-1.el8.x86_64 169/170 2023-11-28T11:19:13.254 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : openssl-libs-1:1.1.1k-9.el8.x86_64 170/170 2023-11-28T11:19:13.383 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 111/170 2023-11-28T11:19:13.692 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : jq-1.6-8.el8.x86_64 112/170 2023-11-28T11:19:13.802 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: openssl-libs-1:1.1.1k-9.el8.x86_64 170/170 2023-11-28T11:19:13.802 INFO:teuthology.orchestra.run.smithi092.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-11-28T11:19:13.802 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:19:13.912 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : automake-1.16.1-8.el8.noarch 113/170 2023-11-28T11:19:14.544 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 1/170 2023-11-28T11:19:14.544 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 2/170 2023-11-28T11:19:14.545 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 3/170 2023-11-28T11:19:14.545 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 4/170 2023-11-28T11:19:14.545 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 5/170 2023-11-28T11:19:14.545 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 6/170 2023-11-28T11:19:14.545 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 7/170 2023-11-28T11:19:14.545 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 8/170 2023-11-28T11:19:14.545 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 9/170 2023-11-28T11:19:14.545 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 10/170 2023-11-28T11:19:14.545 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 11/170 2023-11-28T11:19:14.545 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 12/170 2023-11-28T11:19:14.545 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 13/170 2023-11-28T11:19:14.545 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_ 14/170 2023-11-28T11:19:14.546 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 15/170 2023-11-28T11:19:14.546 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 16/170 2023-11-28T11:19:14.546 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_6 17/170 2023-11-28T11:19:14.546 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 18/170 2023-11-28T11:19:14.546 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 19/170 2023-11-28T11:19:14.546 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el 20/170 2023-11-28T11:19:14.546 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8. 21/170 2023-11-28T11:19:14.546 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_6 22/170 2023-11-28T11:19:14.546 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 23/170 2023-11-28T11:19:14.546 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 24/170 2023-11-28T11:19:14.546 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 25/170 2023-11-28T11:19:14.547 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 26/170 2023-11-28T11:19:14.547 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 27/170 2023-11-28T11:19:14.547 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 28/170 2023-11-28T11:19:14.547 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028. 29/170 2023-11-28T11:19:14.547 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 30/170 2023-11-28T11:19:14.547 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 31/170 2023-11-28T11:19:14.547 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 32/170 2023-11-28T11:19:14.547 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el 33/170 2023-11-28T11:19:14.547 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 34/170 2023-11-28T11:19:14.547 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.e 35/170 2023-11-28T11:19:14.547 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 36/170 2023-11-28T11:19:14.547 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 37/170 2023-11-28T11:19:14.548 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 38/170 2023-11-28T11:19:14.548 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : automake-1.16.1-8.el8.noarch 39/170 2023-11-28T11:19:14.548 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : bison-3.0.4-10.el8.x86_64 40/170 2023-11-28T11:19:14.548 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flex-2.6.1-9.el8.x86_64 41/170 2023-11-28T11:19:14.548 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : jq-1.6-8.el8.x86_64 42/170 2023-11-28T11:19:14.548 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libpq-13.5-1.el8.x86_64 43/170 2023-11-28T11:19:14.548 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 44/170 2023-11-28T11:19:14.548 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 45/170 2023-11-28T11:19:14.548 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 46/170 2023-11-28T11:19:14.548 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 47/170 2023-11-28T11:19:14.548 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 48/170 2023-11-28T11:19:14.548 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 49/170 2023-11-28T11:19:14.549 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : postgresql-10.23-2.module_el8+572+929c87ac.x86_6 50/170 2023-11-28T11:19:14.549 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : postgresql-contrib-10.23-2.module_el8+572+929c87 51/170 2023-11-28T11:19:14.549 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : postgresql-server-10.23-2.module_el8+572+929c87a 52/170 2023-11-28T11:19:14.549 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 53/170 2023-11-28T11:19:14.549 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 54/170 2023-11-28T11:19:14.549 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 55/170 2023-11-28T11:19:14.549 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 56/170 2023-11-28T11:19:14.549 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 57/170 2023-11-28T11:19:14.549 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 58/170 2023-11-28T11:19:14.549 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 59/170 2023-11-28T11:19:14.549 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 60/170 2023-11-28T11:19:14.549 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 61/170 2023-11-28T11:19:14.550 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 62/170 2023-11-28T11:19:14.550 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 63/170 2023-11-28T11:19:14.550 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 64/170 2023-11-28T11:19:14.550 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 65/170 2023-11-28T11:19:14.550 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 66/170 2023-11-28T11:19:14.550 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 67/170 2023-11-28T11:19:14.550 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 68/170 2023-11-28T11:19:14.550 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 69/170 2023-11-28T11:19:14.550 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 70/170 2023-11-28T11:19:14.550 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : elfutils-libelf-devel-0.190-1.el8.x86_64 71/170 2023-11-28T11:19:14.551 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 72/170 2023-11-28T11:19:14.551 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 73/170 2023-11-28T11:19:14.551 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : krb5-devel-1.18.2-22.el8.x86_64 74/170 2023-11-28T11:19:14.551 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 75/170 2023-11-28T11:19:14.551 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcom_err-devel-1.45.6-5.el8.x86_64 76/170 2023-11-28T11:19:14.551 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libgfortran-8.5.0-21.el8.x86_64 77/170 2023-11-28T11:19:14.551 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libquadmath-8.5.0-21.el8.x86_64 78/170 2023-11-28T11:19:14.551 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 79/170 2023-11-28T11:19:14.551 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libselinux-devel-2.9-8.el8.x86_64 80/170 2023-11-28T11:19:14.551 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 81/170 2023-11-28T11:19:14.551 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 82/170 2023-11-28T11:19:14.551 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libverto-devel-0.3.2-2.el8.x86_64 83/170 2023-11-28T11:19:14.552 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 84/170 2023-11-28T11:19:14.552 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-devel-1:1.1.1k-11.el8.x86_64 85/170 2023-11-28T11:19:14.552 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : pcre2-devel-10.32-3.el8.x86_64 86/170 2023-11-28T11:19:14.552 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : pcre2-utf16-10.32-3.el8.x86_64 87/170 2023-11-28T11:19:14.552 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : pcre2-utf32-10.32-3.el8.x86_64 88/170 2023-11-28T11:19:14.552 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 89/170 2023-11-28T11:19:14.552 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 90/170 2023-11-28T11:19:14.552 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 91/170 2023-11-28T11:19:14.552 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 92/170 2023-11-28T11:19:14.552 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 93/170 2023-11-28T11:19:14.552 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 94/170 2023-11-28T11:19:14.552 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 95/170 2023-11-28T11:19:14.552 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : zlib-devel-1.2.11-21.el8.x86_64 96/170 2023-11-28T11:19:14.553 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 97/170 2023-11-28T11:19:14.553 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 98/170 2023-11-28T11:19:14.553 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 99/170 2023-11-28T11:19:14.553 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 100/170 2023-11-28T11:19:14.553 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 101/170 2023-11-28T11:19:14.553 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 102/170 2023-11-28T11:19:14.553 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 103/170 2023-11-28T11:19:14.553 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 104/170 2023-11-28T11:19:14.553 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 105/170 2023-11-28T11:19:14.553 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 106/170 2023-11-28T11:19:14.553 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 107/170 2023-11-28T11:19:14.553 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 108/170 2023-11-28T11:19:14.554 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 109/170 2023-11-28T11:19:14.554 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 110/170 2023-11-28T11:19:14.554 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 111/170 2023-11-28T11:19:14.554 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 112/170 2023-11-28T11:19:14.554 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 113/170 2023-11-28T11:19:14.554 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 114/170 2023-11-28T11:19:14.554 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-portend-2.6-1.el8.noarch 115/170 2023-11-28T11:19:14.554 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 116/170 2023-11-28T11:19:14.554 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 117/170 2023-11-28T11:19:14.554 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 118/170 2023-11-28T11:19:14.554 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 119/170 2023-11-28T11:19:14.555 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 120/170 2023-11-28T11:19:14.555 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 121/170 2023-11-28T11:19:14.555 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 122/170 2023-11-28T11:19:14.555 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 123/170 2023-11-28T11:19:14.555 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 124/170 2023-11-28T11:19:14.555 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 125/170 2023-11-28T11:19:14.555 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 126/170 2023-11-28T11:19:14.555 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 127/170 2023-11-28T11:19:14.555 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 128/170 2023-11-28T11:19:14.555 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 129/170 2023-11-28T11:19:14.555 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/170 2023-11-28T11:19:14.555 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 131/170 2023-11-28T11:19:14.556 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/170 2023-11-28T11:19:14.556 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-11.e 133/170 2023-11-28T11:19:14.556 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-4.el 134/170 2023-11-28T11:19:14.556 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : acl-2.2.53-3.el8.x86_64 135/170 2023-11-28T11:19:14.556 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : acl-2.2.53-1.el8.x86_64 136/170 2023-11-28T11:19:14.556 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : elfutils-debuginfod-client-0.190-1.el8.x86_64 137/170 2023-11-28T11:19:14.556 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : elfutils-debuginfod-client-0.189-1.el8.x86_64 138/170 2023-11-28T11:19:14.556 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : elfutils-libelf-0.190-1.el8.x86_64 139/170 2023-11-28T11:19:14.556 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : elfutils-libelf-0.189-1.el8.x86_64 140/170 2023-11-28T11:19:14.556 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : elfutils-libs-0.190-1.el8.x86_64 141/170 2023-11-28T11:19:14.556 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : elfutils-libs-0.189-1.el8.x86_64 142/170 2023-11-28T11:19:14.556 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : iproute-6.2.0-5.el8.x86_64 143/170 2023-11-28T11:19:14.557 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : iproute-6.2.0-1.el8.x86_64 144/170 2023-11-28T11:19:14.557 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libacl-2.2.53-3.el8.x86_64 145/170 2023-11-28T11:19:14.557 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libacl-2.2.53-1.el8.x86_64 146/170 2023-11-28T11:19:14.557 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libacl-devel-2.2.53-3.el8.x86_64 147/170 2023-11-28T11:19:14.557 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libacl-devel-2.2.53-1.el8.x86_64 148/170 2023-11-28T11:19:14.557 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 149/170 2023-11-28T11:19:14.557 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libblkid-2.32.1-41.el8.x86_64 150/170 2023-11-28T11:19:14.557 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 151/170 2023-11-28T11:19:14.557 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libfdisk-2.32.1-41.el8.x86_64 152/170 2023-11-28T11:19:14.557 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 153/170 2023-11-28T11:19:14.557 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libmount-2.32.1-41.el8.x86_64 154/170 2023-11-28T11:19:14.557 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 155/170 2023-11-28T11:19:14.558 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libsmartcols-2.32.1-41.el8.x86_64 156/170 2023-11-28T11:19:14.558 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 157/170 2023-11-28T11:19:14.558 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libuuid-2.32.1-41.el8.x86_64 158/170 2023-11-28T11:19:14.558 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-1:1.1.1k-11.el8.x86_64 159/170 2023-11-28T11:19:14.558 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-1:1.1.1k-9.el8.x86_64 160/170 2023-11-28T11:19:14.558 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-libs-1:1.1.1k-11.el8.x86_64 161/170 2023-11-28T11:19:14.558 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-libs-1:1.1.1k-9.el8.x86_64 162/170 2023-11-28T11:19:14.558 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 163/170 2023-11-28T11:19:14.558 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : util-linux-2.32.1-41.el8.x86_64 164/170 2023-11-28T11:19:14.558 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 165/170 2023-11-28T11:19:14.558 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : util-linux-user-2.32.1-41.el8.x86_64 166/170 2023-11-28T11:19:14.558 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xfsdump-3.1.8-7.el8.x86_64 167/170 2023-11-28T11:19:14.559 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 168/170 2023-11-28T11:19:14.559 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 169/170 2023-11-28T11:19:14.566 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 114/170 2023-11-28T11:19:14.756 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 114/170 2023-11-28T11:19:14.895 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.e 115/170 2023-11-28T11:19:15.117 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028. 116/170 2023-11-28T11:19:15.320 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 117/170 2023-11-28T11:19:15.581 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 117/170 2023-11-28T11:19:15.877 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 118/170 2023-11-28T11:19:16.966 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 118/170 2023-11-28T11:19:17.178 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 119/170 2023-11-28T11:19:17.268 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 170/170 2023-11-28T11:19:17.268 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:19:17.268 INFO:teuthology.orchestra.run.smithi092.stdout:Upgraded: 2023-11-28T11:19:17.268 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-initscripts-updown-1:1.40.16-11.el8.noarch 2023-11-28T11:19:17.268 INFO:teuthology.orchestra.run.smithi092.stdout: acl-2.2.53-3.el8.x86_64 2023-11-28T11:19:17.268 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-debuginfod-client-0.190-1.el8.x86_64 2023-11-28T11:19:17.268 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libelf-0.190-1.el8.x86_64 2023-11-28T11:19:17.268 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libs-0.190-1.el8.x86_64 2023-11-28T11:19:17.268 INFO:teuthology.orchestra.run.smithi092.stdout: iproute-6.2.0-5.el8.x86_64 2023-11-28T11:19:17.269 INFO:teuthology.orchestra.run.smithi092.stdout: libacl-2.2.53-3.el8.x86_64 2023-11-28T11:19:17.269 INFO:teuthology.orchestra.run.smithi092.stdout: libacl-devel-2.2.53-3.el8.x86_64 2023-11-28T11:19:17.269 INFO:teuthology.orchestra.run.smithi092.stdout: libblkid-2.32.1-43.el8.x86_64 2023-11-28T11:19:17.269 INFO:teuthology.orchestra.run.smithi092.stdout: libfdisk-2.32.1-43.el8.x86_64 2023-11-28T11:19:17.269 INFO:teuthology.orchestra.run.smithi092.stdout: libmount-2.32.1-43.el8.x86_64 2023-11-28T11:19:17.269 INFO:teuthology.orchestra.run.smithi092.stdout: librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:17.269 INFO:teuthology.orchestra.run.smithi092.stdout: librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:17.269 INFO:teuthology.orchestra.run.smithi092.stdout: libsmartcols-2.32.1-43.el8.x86_64 2023-11-28T11:19:17.269 INFO:teuthology.orchestra.run.smithi092.stdout: libuuid-2.32.1-43.el8.x86_64 2023-11-28T11:19:17.269 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-1:1.1.1k-11.el8.x86_64 2023-11-28T11:19:17.269 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-libs-1:1.1.1k-11.el8.x86_64 2023-11-28T11:19:17.270 INFO:teuthology.orchestra.run.smithi092.stdout: util-linux-2.32.1-43.el8.x86_64 2023-11-28T11:19:17.270 INFO:teuthology.orchestra.run.smithi092.stdout: util-linux-user-2.32.1-43.el8.x86_64 2023-11-28T11:19:17.270 INFO:teuthology.orchestra.run.smithi092.stdout: xfsdump-3.1.8-7.el8.x86_64 2023-11-28T11:19:17.270 INFO:teuthology.orchestra.run.smithi092.stdout: xfsprogs-5.0.0-12.el8.x86_64 2023-11-28T11:19:17.270 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2023-11-28T11:19:17.270 INFO:teuthology.orchestra.run.smithi092.stdout: automake-1.16.1-8.el8.noarch 2023-11-28T11:19:17.270 INFO:teuthology.orchestra.run.smithi092.stdout: bison-3.0.4-10.el8.x86_64 2023-11-28T11:19:17.270 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:17.270 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:17.270 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:17.270 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:17.270 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:19:17.271 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:17.271 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:17.271 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:17.271 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:19:17.271 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:19:17.271 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:19:17.271 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:19:17.271 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:19:17.271 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:17.271 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:17.271 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:19:17.272 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:17.272 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:17.272 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:17.272 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:19:17.272 INFO:teuthology.orchestra.run.smithi092.stdout: cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:19:17.272 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:17.272 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:19:17.272 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libelf-devel-0.190-1.el8.x86_64 2023-11-28T11:19:17.272 INFO:teuthology.orchestra.run.smithi092.stdout: flex-2.6.1-9.el8.x86_64 2023-11-28T11:19:17.272 INFO:teuthology.orchestra.run.smithi092.stdout: fuse-2.9.7-17.el8.x86_64 2023-11-28T11:19:17.272 INFO:teuthology.orchestra.run.smithi092.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-28T11:19:17.273 INFO:teuthology.orchestra.run.smithi092.stdout: jq-1.6-8.el8.x86_64 2023-11-28T11:19:17.273 INFO:teuthology.orchestra.run.smithi092.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-11-28T11:19:17.273 INFO:teuthology.orchestra.run.smithi092.stdout: krb5-devel-1.18.2-22.el8.x86_64 2023-11-28T11:19:17.273 INFO:teuthology.orchestra.run.smithi092.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-28T11:19:17.273 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:17.273 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:17.273 INFO:teuthology.orchestra.run.smithi092.stdout: libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:17.273 INFO:teuthology.orchestra.run.smithi092.stdout: libcom_err-devel-1.45.6-5.el8.x86_64 2023-11-28T11:19:17.273 INFO:teuthology.orchestra.run.smithi092.stdout: libgfortran-8.5.0-21.el8.x86_64 2023-11-28T11:19:17.273 INFO:teuthology.orchestra.run.smithi092.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-28T11:19:17.274 INFO:teuthology.orchestra.run.smithi092.stdout: libpq-13.5-1.el8.x86_64 2023-11-28T11:19:17.274 INFO:teuthology.orchestra.run.smithi092.stdout: libquadmath-8.5.0-21.el8.x86_64 2023-11-28T11:19:17.274 INFO:teuthology.orchestra.run.smithi092.stdout: librabbitmq-0.9.0-4.el8.x86_64 2023-11-28T11:19:17.274 INFO:teuthology.orchestra.run.smithi092.stdout: librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:17.274 INFO:teuthology.orchestra.run.smithi092.stdout: libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:17.274 INFO:teuthology.orchestra.run.smithi092.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-11-28T11:19:17.274 INFO:teuthology.orchestra.run.smithi092.stdout: librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:17.274 INFO:teuthology.orchestra.run.smithi092.stdout: libselinux-devel-2.9-8.el8.x86_64 2023-11-28T11:19:17.274 INFO:teuthology.orchestra.run.smithi092.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-11-28T11:19:17.274 INFO:teuthology.orchestra.run.smithi092.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-28T11:19:17.275 INFO:teuthology.orchestra.run.smithi092.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-28T11:19:17.275 INFO:teuthology.orchestra.run.smithi092.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2023-11-28T11:19:17.275 INFO:teuthology.orchestra.run.smithi092.stdout: libverto-devel-0.3.2-2.el8.x86_64 2023-11-28T11:19:17.275 INFO:teuthology.orchestra.run.smithi092.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2023-11-28T11:19:17.275 INFO:teuthology.orchestra.run.smithi092.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-28T11:19:17.275 INFO:teuthology.orchestra.run.smithi092.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-28T11:19:17.275 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-0.3.15-4.el8.x86_64 2023-11-28T11:19:17.275 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-threads-0.3.15-4.el8.x86_64 2023-11-28T11:19:17.275 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-devel-1:1.1.1k-11.el8.x86_64 2023-11-28T11:19:17.275 INFO:teuthology.orchestra.run.smithi092.stdout: pcre2-devel-10.32-3.el8.x86_64 2023-11-28T11:19:17.275 INFO:teuthology.orchestra.run.smithi092.stdout: pcre2-utf16-10.32-3.el8.x86_64 2023-11-28T11:19:17.276 INFO:teuthology.orchestra.run.smithi092.stdout: pcre2-utf32-10.32-3.el8.x86_64 2023-11-28T11:19:17.276 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-10.23-2.module_el8+572+929c87ac.x86_64 2023-11-28T11:19:17.276 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-contrib-10.23-2.module_el8+572+929c87ac.x86_64 2023-11-28T11:19:17.276 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-server-10.23-2.module_el8+572+929c87ac.x86_64 2023-11-28T11:19:17.276 INFO:teuthology.orchestra.run.smithi092.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-11-28T11:19:17.276 INFO:teuthology.orchestra.run.smithi092.stdout: python3-babel-2.5.1-7.el8.noarch 2023-11-28T11:19:17.276 INFO:teuthology.orchestra.run.smithi092.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-28T11:19:17.276 INFO:teuthology.orchestra.run.smithi092.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-28T11:19:17.276 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-28T11:19:17.276 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:17.276 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:17.276 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:17.277 INFO:teuthology.orchestra.run.smithi092.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-28T11:19:17.277 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cffi-1.11.5-6.el8.x86_64 2023-11-28T11:19:17.277 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-28T11:19:17.277 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-28T11:19:17.277 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2023-11-28T11:19:17.277 INFO:teuthology.orchestra.run.smithi092.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-28T11:19:17.277 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-28T11:19:17.277 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-28T11:19:17.277 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-11-28T11:19:17.277 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-28T11:19:17.277 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-28T11:19:17.278 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-28T11:19:17.278 INFO:teuthology.orchestra.run.smithi092.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-28T11:19:17.278 INFO:teuthology.orchestra.run.smithi092.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-28T11:19:17.278 INFO:teuthology.orchestra.run.smithi092.stdout: python3-mako-1.0.6-14.el8.noarch 2023-11-28T11:19:17.278 INFO:teuthology.orchestra.run.smithi092.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-28T11:19:17.278 INFO:teuthology.orchestra.run.smithi092.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-28T11:19:17.278 INFO:teuthology.orchestra.run.smithi092.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-11-28T11:19:17.278 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-11-28T11:19:17.278 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-11-28T11:19:17.278 INFO:teuthology.orchestra.run.smithi092.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-28T11:19:17.278 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-28T11:19:17.279 INFO:teuthology.orchestra.run.smithi092.stdout: python3-portend-2.6-1.el8.noarch 2023-11-28T11:19:17.279 INFO:teuthology.orchestra.run.smithi092.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-28T11:19:17.279 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-11-28T11:19:17.279 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-28T11:19:17.279 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-28T11:19:17.279 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pycparser-2.14-14.el8.noarch 2023-11-28T11:19:17.279 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pytz-2017.2-11.el8.noarch 2023-11-28T11:19:17.279 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:17.279 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:17.279 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-28T11:19:17.279 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-28T11:19:17.279 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:17.280 INFO:teuthology.orchestra.run.smithi092.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-28T11:19:17.280 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-28T11:19:17.280 INFO:teuthology.orchestra.run.smithi092.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2023-11-28T11:19:17.280 INFO:teuthology.orchestra.run.smithi092.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-28T11:19:17.280 INFO:teuthology.orchestra.run.smithi092.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-28T11:19:17.280 INFO:teuthology.orchestra.run.smithi092.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-28T11:19:17.280 INFO:teuthology.orchestra.run.smithi092.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-28T11:19:17.280 INFO:teuthology.orchestra.run.smithi092.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-28T11:19:17.280 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-28T11:19:17.280 INFO:teuthology.orchestra.run.smithi092.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-28T11:19:17.280 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-28T11:19:17.280 INFO:teuthology.orchestra.run.smithi092.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-28T11:19:17.281 INFO:teuthology.orchestra.run.smithi092.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-28T11:19:17.281 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:17.281 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:17.281 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:17.281 INFO:teuthology.orchestra.run.smithi092.stdout: socat-1.7.4.1-1.el8.x86_64 2023-11-28T11:19:17.281 INFO:teuthology.orchestra.run.smithi092.stdout: thrift-0.13.0-2.el8.x86_64 2023-11-28T11:19:17.281 INFO:teuthology.orchestra.run.smithi092.stdout: uuid-1.6.2-43.el8.x86_64 2023-11-28T11:19:17.281 INFO:teuthology.orchestra.run.smithi092.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2023-11-28T11:19:17.281 INFO:teuthology.orchestra.run.smithi092.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-28T11:19:17.282 INFO:teuthology.orchestra.run.smithi092.stdout: zlib-devel-1.2.11-21.el8.x86_64 2023-11-28T11:19:17.282 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:19:17.282 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2023-11-28T11:19:17.309 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 119/170 2023-11-28T11:19:17.571 DEBUG:teuthology.parallel:result is None 2023-11-28T11:19:17.716 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el 120/170 2023-11-28T11:19:17.902 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 121/170 2023-11-28T11:19:17.968 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 121/170 2023-11-28T11:19:18.144 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 122/170 2023-11-28T11:19:18.172 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 122/170 2023-11-28T11:19:18.365 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 123/170 2023-11-28T11:19:18.544 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libtool-2.4.6-25.el8.x86_64 124/170 2023-11-28T11:19:23.305 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 124/170 2023-11-28T11:19:24.836 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 125/170 2023-11-28T11:19:25.101 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 126/170 2023-11-28T11:19:25.163 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 126/170 2023-11-28T11:19:25.385 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 127/170 2023-11-28T11:19:25.457 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 127/170 2023-11-28T11:19:25.652 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 128/170 2023-11-28T11:19:26.056 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 128/170 2023-11-28T11:19:26.269 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 129/170 2023-11-28T11:19:26.559 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 129/170 2023-11-28T11:19:26.907 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : postgresql-contrib-10.23-2.module_el8+572+929c87 130/170 2023-11-28T11:19:28.199 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 131/170 2023-11-28T11:19:28.556 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : openssl-devel-1:1.1.1k-11.el8.x86_64 132/170 2023-11-28T11:19:28.775 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : elfutils-libelf-devel-0.190-1.el8.x86_64 133/170 2023-11-28T11:19:29.032 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 134/170 2023-11-28T11:19:29.272 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 135/170 2023-11-28T11:19:29.502 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_ 136/170 2023-11-28T11:19:29.865 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : acl-2.2.53-3.el8.x86_64 137/170 2023-11-28T11:19:30.019 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : libacl-devel-2.2.53-3.el8.x86_64 138/170 2023-11-28T11:19:31.282 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: postgresql-server-10.23-2.module_el8+572+929c87a 139/170 2023-11-28T11:19:31.559 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : postgresql-server-10.23-2.module_el8+572+929c87a 139/170 2023-11-28T11:19:31.650 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: postgresql-server-10.23-2.module_el8+572+929c87a 139/170 2023-11-28T11:19:31.914 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : xfsdump-3.1.8-7.el8.x86_64 140/170 2023-11-28T11:19:32.143 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 141/170 2023-11-28T11:19:32.376 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 142/170 2023-11-28T11:19:32.819 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 143/170 2023-11-28T11:19:33.271 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : iproute-6.2.0-5.el8.x86_64 144/170 2023-11-28T11:19:33.595 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : openssl-1:1.1.1k-11.el8.x86_64 145/170 2023-11-28T11:19:33.775 INFO:teuthology.orchestra.run.smithi169.stdout: Upgrading : NetworkManager-initscripts-updown-1:1.40.16-11.e 146/170 2023-11-28T11:19:33.807 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-11.e 146/170 2023-11-28T11:19:34.014 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 147/170 2023-11-28T11:19:34.308 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : flex-2.6.1-9.el8.x86_64 148/170 2023-11-28T11:19:34.449 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 148/170 2023-11-28T11:19:34.700 INFO:teuthology.orchestra.run.smithi169.stdout: Installing : bison-3.0.4-10.el8.x86_64 149/170 2023-11-28T11:19:34.729 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 149/170 2023-11-28T11:19:34.952 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : elfutils-debuginfod-client-0.189-1.el8.x86_64 150/170 2023-11-28T11:19:35.212 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : elfutils-libs-0.189-1.el8.x86_64 151/170 2023-11-28T11:19:35.485 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : openssl-1:1.1.1k-9.el8.x86_64 152/170 2023-11-28T11:19:35.663 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 153/170 2023-11-28T11:19:35.663 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 154/170 2023-11-28T11:19:35.955 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 154/170 2023-11-28T11:19:35.955 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 155/170 2023-11-28T11:19:36.229 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 155/170 2023-11-28T11:19:36.343 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : libacl-devel-2.2.53-1.el8.x86_64 156/170 2023-11-28T11:19:36.361 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-4.el 157/170 2023-11-28T11:19:36.514 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : NetworkManager-initscripts-updown-1:1.40.16-4.el 157/170 2023-11-28T11:19:36.514 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 158/170 2023-11-28T11:19:36.820 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 158/170 2023-11-28T11:19:37.041 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : acl-2.2.53-1.el8.x86_64 159/170 2023-11-28T11:19:37.285 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : iproute-6.2.0-1.el8.x86_64 160/170 2023-11-28T11:19:37.489 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : util-linux-user-2.32.1-41.el8.x86_64 161/170 2023-11-28T11:19:37.735 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : util-linux-2.32.1-41.el8.x86_64 162/170 2023-11-28T11:19:37.735 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : libfdisk-2.32.1-41.el8.x86_64 163/170 2023-11-28T11:19:37.970 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libfdisk-2.32.1-41.el8.x86_64 163/170 2023-11-28T11:19:37.970 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : libmount-2.32.1-41.el8.x86_64 164/170 2023-11-28T11:19:38.228 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libmount-2.32.1-41.el8.x86_64 164/170 2023-11-28T11:19:38.228 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : libblkid-2.32.1-41.el8.x86_64 165/170 2023-11-28T11:19:38.462 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libblkid-2.32.1-41.el8.x86_64 165/170 2023-11-28T11:19:38.462 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : libuuid-2.32.1-41.el8.x86_64 166/170 2023-11-28T11:19:38.678 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libuuid-2.32.1-41.el8.x86_64 166/170 2023-11-28T11:19:38.678 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : libsmartcols-2.32.1-41.el8.x86_64 167/170 2023-11-28T11:19:38.928 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: libsmartcols-2.32.1-41.el8.x86_64 167/170 2023-11-28T11:19:39.143 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : elfutils-libelf-0.189-1.el8.x86_64 168/170 2023-11-28T11:19:39.269 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : libacl-2.2.53-1.el8.x86_64 169/170 2023-11-28T11:19:39.269 INFO:teuthology.orchestra.run.smithi169.stdout: Cleanup : openssl-libs-1:1.1.1k-9.el8.x86_64 170/170 2023-11-28T11:19:39.829 INFO:teuthology.orchestra.run.smithi169.stdout: Running scriptlet: openssl-libs-1:1.1.1k-9.el8.x86_64 170/170 2023-11-28T11:19:39.829 INFO:teuthology.orchestra.run.smithi169.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-11-28T11:19:39.829 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-28T11:19:40.560 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 1/170 2023-11-28T11:19:40.560 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 2/170 2023-11-28T11:19:40.560 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 3/170 2023-11-28T11:19:40.560 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 4/170 2023-11-28T11:19:40.560 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 5/170 2023-11-28T11:19:40.560 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 6/170 2023-11-28T11:19:40.561 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 7/170 2023-11-28T11:19:40.561 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 8/170 2023-11-28T11:19:40.561 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 9/170 2023-11-28T11:19:40.561 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 10/170 2023-11-28T11:19:40.561 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 11/170 2023-11-28T11:19:40.561 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 12/170 2023-11-28T11:19:40.561 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 13/170 2023-11-28T11:19:40.561 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_ 14/170 2023-11-28T11:19:40.561 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 15/170 2023-11-28T11:19:40.561 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 16/170 2023-11-28T11:19:40.561 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_6 17/170 2023-11-28T11:19:40.561 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 18/170 2023-11-28T11:19:40.562 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 19/170 2023-11-28T11:19:40.562 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el 20/170 2023-11-28T11:19:40.562 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8. 21/170 2023-11-28T11:19:40.562 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_6 22/170 2023-11-28T11:19:40.562 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 23/170 2023-11-28T11:19:40.562 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 24/170 2023-11-28T11:19:40.562 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 25/170 2023-11-28T11:19:40.562 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 26/170 2023-11-28T11:19:40.562 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 27/170 2023-11-28T11:19:40.562 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 28/170 2023-11-28T11:19:40.576 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028. 29/170 2023-11-28T11:19:40.576 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 30/170 2023-11-28T11:19:40.576 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 31/170 2023-11-28T11:19:40.576 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 32/170 2023-11-28T11:19:40.576 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el 33/170 2023-11-28T11:19:40.576 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 34/170 2023-11-28T11:19:40.577 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.e 35/170 2023-11-28T11:19:40.577 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 36/170 2023-11-28T11:19:40.577 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 37/170 2023-11-28T11:19:40.577 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 38/170 2023-11-28T11:19:40.577 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : automake-1.16.1-8.el8.noarch 39/170 2023-11-28T11:19:40.577 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : bison-3.0.4-10.el8.x86_64 40/170 2023-11-28T11:19:40.577 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : flex-2.6.1-9.el8.x86_64 41/170 2023-11-28T11:19:40.577 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : jq-1.6-8.el8.x86_64 42/170 2023-11-28T11:19:40.577 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libpq-13.5-1.el8.x86_64 43/170 2023-11-28T11:19:40.577 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 44/170 2023-11-28T11:19:40.577 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 45/170 2023-11-28T11:19:40.577 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 46/170 2023-11-28T11:19:40.577 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 47/170 2023-11-28T11:19:40.578 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 48/170 2023-11-28T11:19:40.578 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 49/170 2023-11-28T11:19:40.578 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : postgresql-10.23-2.module_el8+572+929c87ac.x86_6 50/170 2023-11-28T11:19:40.578 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : postgresql-contrib-10.23-2.module_el8+572+929c87 51/170 2023-11-28T11:19:40.578 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : postgresql-server-10.23-2.module_el8+572+929c87a 52/170 2023-11-28T11:19:40.578 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 53/170 2023-11-28T11:19:40.578 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 54/170 2023-11-28T11:19:40.578 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 55/170 2023-11-28T11:19:40.578 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 56/170 2023-11-28T11:19:40.578 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 57/170 2023-11-28T11:19:40.578 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 58/170 2023-11-28T11:19:40.579 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 59/170 2023-11-28T11:19:40.579 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 60/170 2023-11-28T11:19:40.579 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 61/170 2023-11-28T11:19:40.579 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 62/170 2023-11-28T11:19:40.579 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 63/170 2023-11-28T11:19:40.579 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 64/170 2023-11-28T11:19:40.579 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 65/170 2023-11-28T11:19:40.579 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 66/170 2023-11-28T11:19:40.579 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 67/170 2023-11-28T11:19:40.579 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 68/170 2023-11-28T11:19:40.579 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 69/170 2023-11-28T11:19:40.579 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 70/170 2023-11-28T11:19:40.580 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : elfutils-libelf-devel-0.190-1.el8.x86_64 71/170 2023-11-28T11:19:40.580 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 72/170 2023-11-28T11:19:40.580 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 73/170 2023-11-28T11:19:40.580 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : krb5-devel-1.18.2-22.el8.x86_64 74/170 2023-11-28T11:19:40.580 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 75/170 2023-11-28T11:19:40.580 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libcom_err-devel-1.45.6-5.el8.x86_64 76/170 2023-11-28T11:19:40.580 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libgfortran-8.5.0-21.el8.x86_64 77/170 2023-11-28T11:19:40.580 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libquadmath-8.5.0-21.el8.x86_64 78/170 2023-11-28T11:19:40.580 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 79/170 2023-11-28T11:19:40.580 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libselinux-devel-2.9-8.el8.x86_64 80/170 2023-11-28T11:19:40.580 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 81/170 2023-11-28T11:19:40.581 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 82/170 2023-11-28T11:19:40.581 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libverto-devel-0.3.2-2.el8.x86_64 83/170 2023-11-28T11:19:40.581 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 84/170 2023-11-28T11:19:40.581 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openssl-devel-1:1.1.1k-11.el8.x86_64 85/170 2023-11-28T11:19:40.581 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : pcre2-devel-10.32-3.el8.x86_64 86/170 2023-11-28T11:19:40.581 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : pcre2-utf16-10.32-3.el8.x86_64 87/170 2023-11-28T11:19:40.581 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : pcre2-utf32-10.32-3.el8.x86_64 88/170 2023-11-28T11:19:40.581 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 89/170 2023-11-28T11:19:40.581 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 90/170 2023-11-28T11:19:40.581 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 91/170 2023-11-28T11:19:40.581 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 92/170 2023-11-28T11:19:40.581 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 93/170 2023-11-28T11:19:40.582 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 94/170 2023-11-28T11:19:40.582 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 95/170 2023-11-28T11:19:40.582 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : zlib-devel-1.2.11-21.el8.x86_64 96/170 2023-11-28T11:19:40.582 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 97/170 2023-11-28T11:19:40.582 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 98/170 2023-11-28T11:19:40.582 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 99/170 2023-11-28T11:19:40.582 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 100/170 2023-11-28T11:19:40.582 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 101/170 2023-11-28T11:19:40.582 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 102/170 2023-11-28T11:19:40.582 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 103/170 2023-11-28T11:19:40.582 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 104/170 2023-11-28T11:19:40.582 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 105/170 2023-11-28T11:19:40.583 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 106/170 2023-11-28T11:19:40.583 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 107/170 2023-11-28T11:19:40.583 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 108/170 2023-11-28T11:19:40.583 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 109/170 2023-11-28T11:19:40.583 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 110/170 2023-11-28T11:19:40.583 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 111/170 2023-11-28T11:19:40.583 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 112/170 2023-11-28T11:19:40.583 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 113/170 2023-11-28T11:19:40.583 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 114/170 2023-11-28T11:19:40.583 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-portend-2.6-1.el8.noarch 115/170 2023-11-28T11:19:40.583 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 116/170 2023-11-28T11:19:40.583 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 117/170 2023-11-28T11:19:40.584 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 118/170 2023-11-28T11:19:40.584 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 119/170 2023-11-28T11:19:40.584 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 120/170 2023-11-28T11:19:40.584 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 121/170 2023-11-28T11:19:40.584 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 122/170 2023-11-28T11:19:40.584 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 123/170 2023-11-28T11:19:40.584 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 124/170 2023-11-28T11:19:40.584 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 125/170 2023-11-28T11:19:40.584 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 126/170 2023-11-28T11:19:40.584 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 127/170 2023-11-28T11:19:40.584 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 128/170 2023-11-28T11:19:40.585 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 129/170 2023-11-28T11:19:40.585 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/170 2023-11-28T11:19:40.585 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 131/170 2023-11-28T11:19:40.585 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/170 2023-11-28T11:19:40.585 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-11.e 133/170 2023-11-28T11:19:40.585 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-4.el 134/170 2023-11-28T11:19:40.585 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : acl-2.2.53-3.el8.x86_64 135/170 2023-11-28T11:19:40.585 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : acl-2.2.53-1.el8.x86_64 136/170 2023-11-28T11:19:40.585 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : elfutils-debuginfod-client-0.190-1.el8.x86_64 137/170 2023-11-28T11:19:40.585 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : elfutils-debuginfod-client-0.189-1.el8.x86_64 138/170 2023-11-28T11:19:40.586 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : elfutils-libelf-0.190-1.el8.x86_64 139/170 2023-11-28T11:19:40.586 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : elfutils-libelf-0.189-1.el8.x86_64 140/170 2023-11-28T11:19:40.586 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : elfutils-libs-0.190-1.el8.x86_64 141/170 2023-11-28T11:19:40.586 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : elfutils-libs-0.189-1.el8.x86_64 142/170 2023-11-28T11:19:40.586 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : iproute-6.2.0-5.el8.x86_64 143/170 2023-11-28T11:19:40.586 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : iproute-6.2.0-1.el8.x86_64 144/170 2023-11-28T11:19:40.586 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libacl-2.2.53-3.el8.x86_64 145/170 2023-11-28T11:19:40.586 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libacl-2.2.53-1.el8.x86_64 146/170 2023-11-28T11:19:40.586 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libacl-devel-2.2.53-3.el8.x86_64 147/170 2023-11-28T11:19:40.586 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libacl-devel-2.2.53-1.el8.x86_64 148/170 2023-11-28T11:19:40.586 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 149/170 2023-11-28T11:19:40.587 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libblkid-2.32.1-41.el8.x86_64 150/170 2023-11-28T11:19:40.587 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 151/170 2023-11-28T11:19:40.587 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libfdisk-2.32.1-41.el8.x86_64 152/170 2023-11-28T11:19:40.587 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 153/170 2023-11-28T11:19:40.587 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libmount-2.32.1-41.el8.x86_64 154/170 2023-11-28T11:19:40.587 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 155/170 2023-11-28T11:19:40.587 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libsmartcols-2.32.1-41.el8.x86_64 156/170 2023-11-28T11:19:40.587 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 157/170 2023-11-28T11:19:40.587 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : libuuid-2.32.1-41.el8.x86_64 158/170 2023-11-28T11:19:40.587 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openssl-1:1.1.1k-11.el8.x86_64 159/170 2023-11-28T11:19:40.587 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openssl-1:1.1.1k-9.el8.x86_64 160/170 2023-11-28T11:19:40.588 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openssl-libs-1:1.1.1k-11.el8.x86_64 161/170 2023-11-28T11:19:40.588 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : openssl-libs-1:1.1.1k-9.el8.x86_64 162/170 2023-11-28T11:19:40.588 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 163/170 2023-11-28T11:19:40.588 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : util-linux-2.32.1-41.el8.x86_64 164/170 2023-11-28T11:19:40.588 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 165/170 2023-11-28T11:19:40.588 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : util-linux-user-2.32.1-41.el8.x86_64 166/170 2023-11-28T11:19:40.588 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : xfsdump-3.1.8-7.el8.x86_64 167/170 2023-11-28T11:19:40.588 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 168/170 2023-11-28T11:19:40.588 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 169/170 2023-11-28T11:19:44.269 INFO:teuthology.orchestra.run.smithi169.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 170/170 2023-11-28T11:19:44.270 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-28T11:19:44.270 INFO:teuthology.orchestra.run.smithi169.stdout:Upgraded: 2023-11-28T11:19:44.270 INFO:teuthology.orchestra.run.smithi169.stdout: NetworkManager-initscripts-updown-1:1.40.16-11.el8.noarch 2023-11-28T11:19:44.270 INFO:teuthology.orchestra.run.smithi169.stdout: acl-2.2.53-3.el8.x86_64 2023-11-28T11:19:44.270 INFO:teuthology.orchestra.run.smithi169.stdout: elfutils-debuginfod-client-0.190-1.el8.x86_64 2023-11-28T11:19:44.271 INFO:teuthology.orchestra.run.smithi169.stdout: elfutils-libelf-0.190-1.el8.x86_64 2023-11-28T11:19:44.271 INFO:teuthology.orchestra.run.smithi169.stdout: elfutils-libs-0.190-1.el8.x86_64 2023-11-28T11:19:44.271 INFO:teuthology.orchestra.run.smithi169.stdout: iproute-6.2.0-5.el8.x86_64 2023-11-28T11:19:44.271 INFO:teuthology.orchestra.run.smithi169.stdout: libacl-2.2.53-3.el8.x86_64 2023-11-28T11:19:44.271 INFO:teuthology.orchestra.run.smithi169.stdout: libacl-devel-2.2.53-3.el8.x86_64 2023-11-28T11:19:44.271 INFO:teuthology.orchestra.run.smithi169.stdout: libblkid-2.32.1-43.el8.x86_64 2023-11-28T11:19:44.272 INFO:teuthology.orchestra.run.smithi169.stdout: libfdisk-2.32.1-43.el8.x86_64 2023-11-28T11:19:44.272 INFO:teuthology.orchestra.run.smithi169.stdout: libmount-2.32.1-43.el8.x86_64 2023-11-28T11:19:44.272 INFO:teuthology.orchestra.run.smithi169.stdout: librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:44.272 INFO:teuthology.orchestra.run.smithi169.stdout: librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:44.272 INFO:teuthology.orchestra.run.smithi169.stdout: libsmartcols-2.32.1-43.el8.x86_64 2023-11-28T11:19:44.273 INFO:teuthology.orchestra.run.smithi169.stdout: libuuid-2.32.1-43.el8.x86_64 2023-11-28T11:19:44.273 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-1:1.1.1k-11.el8.x86_64 2023-11-28T11:19:44.273 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-libs-1:1.1.1k-11.el8.x86_64 2023-11-28T11:19:44.273 INFO:teuthology.orchestra.run.smithi169.stdout: util-linux-2.32.1-43.el8.x86_64 2023-11-28T11:19:44.273 INFO:teuthology.orchestra.run.smithi169.stdout: util-linux-user-2.32.1-43.el8.x86_64 2023-11-28T11:19:44.273 INFO:teuthology.orchestra.run.smithi169.stdout: xfsdump-3.1.8-7.el8.x86_64 2023-11-28T11:19:44.273 INFO:teuthology.orchestra.run.smithi169.stdout: xfsprogs-5.0.0-12.el8.x86_64 2023-11-28T11:19:44.273 INFO:teuthology.orchestra.run.smithi169.stdout:Installed: 2023-11-28T11:19:44.273 INFO:teuthology.orchestra.run.smithi169.stdout: automake-1.16.1-8.el8.noarch 2023-11-28T11:19:44.274 INFO:teuthology.orchestra.run.smithi169.stdout: bison-3.0.4-10.el8.x86_64 2023-11-28T11:19:44.274 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:44.274 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:44.274 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:44.274 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:44.274 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:19:44.277 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:44.277 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:44.277 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:44.278 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:19:44.278 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:19:44.278 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:19:44.278 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:19:44.278 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:19:44.279 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:44.279 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:44.279 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:19:44.279 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:44.280 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:44.280 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:44.280 INFO:teuthology.orchestra.run.smithi169.stdout: ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:19:44.280 INFO:teuthology.orchestra.run.smithi169.stdout: cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:19:44.280 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:44.281 INFO:teuthology.orchestra.run.smithi169.stdout: cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:19:44.281 INFO:teuthology.orchestra.run.smithi169.stdout: elfutils-libelf-devel-0.190-1.el8.x86_64 2023-11-28T11:19:44.281 INFO:teuthology.orchestra.run.smithi169.stdout: flex-2.6.1-9.el8.x86_64 2023-11-28T11:19:44.281 INFO:teuthology.orchestra.run.smithi169.stdout: fuse-2.9.7-17.el8.x86_64 2023-11-28T11:19:44.281 INFO:teuthology.orchestra.run.smithi169.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-28T11:19:44.281 INFO:teuthology.orchestra.run.smithi169.stdout: jq-1.6-8.el8.x86_64 2023-11-28T11:19:44.282 INFO:teuthology.orchestra.run.smithi169.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-11-28T11:19:44.282 INFO:teuthology.orchestra.run.smithi169.stdout: krb5-devel-1.18.2-22.el8.x86_64 2023-11-28T11:19:44.282 INFO:teuthology.orchestra.run.smithi169.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-28T11:19:44.282 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:44.282 INFO:teuthology.orchestra.run.smithi169.stdout: libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:44.283 INFO:teuthology.orchestra.run.smithi169.stdout: libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:44.283 INFO:teuthology.orchestra.run.smithi169.stdout: libcom_err-devel-1.45.6-5.el8.x86_64 2023-11-28T11:19:44.283 INFO:teuthology.orchestra.run.smithi169.stdout: libgfortran-8.5.0-21.el8.x86_64 2023-11-28T11:19:44.283 INFO:teuthology.orchestra.run.smithi169.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-28T11:19:44.283 INFO:teuthology.orchestra.run.smithi169.stdout: libpq-13.5-1.el8.x86_64 2023-11-28T11:19:44.283 INFO:teuthology.orchestra.run.smithi169.stdout: libquadmath-8.5.0-21.el8.x86_64 2023-11-28T11:19:44.284 INFO:teuthology.orchestra.run.smithi169.stdout: librabbitmq-0.9.0-4.el8.x86_64 2023-11-28T11:19:44.284 INFO:teuthology.orchestra.run.smithi169.stdout: librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:44.284 INFO:teuthology.orchestra.run.smithi169.stdout: libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:44.284 INFO:teuthology.orchestra.run.smithi169.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-11-28T11:19:44.284 INFO:teuthology.orchestra.run.smithi169.stdout: librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:44.285 INFO:teuthology.orchestra.run.smithi169.stdout: libselinux-devel-2.9-8.el8.x86_64 2023-11-28T11:19:44.285 INFO:teuthology.orchestra.run.smithi169.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-11-28T11:19:44.285 INFO:teuthology.orchestra.run.smithi169.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-28T11:19:44.285 INFO:teuthology.orchestra.run.smithi169.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-28T11:19:44.285 INFO:teuthology.orchestra.run.smithi169.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2023-11-28T11:19:44.286 INFO:teuthology.orchestra.run.smithi169.stdout: libverto-devel-0.3.2-2.el8.x86_64 2023-11-28T11:19:44.286 INFO:teuthology.orchestra.run.smithi169.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2023-11-28T11:19:44.286 INFO:teuthology.orchestra.run.smithi169.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-28T11:19:44.286 INFO:teuthology.orchestra.run.smithi169.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-28T11:19:44.286 INFO:teuthology.orchestra.run.smithi169.stdout: openblas-0.3.15-4.el8.x86_64 2023-11-28T11:19:44.287 INFO:teuthology.orchestra.run.smithi169.stdout: openblas-threads-0.3.15-4.el8.x86_64 2023-11-28T11:19:44.287 INFO:teuthology.orchestra.run.smithi169.stdout: openssl-devel-1:1.1.1k-11.el8.x86_64 2023-11-28T11:19:44.287 INFO:teuthology.orchestra.run.smithi169.stdout: pcre2-devel-10.32-3.el8.x86_64 2023-11-28T11:19:44.287 INFO:teuthology.orchestra.run.smithi169.stdout: pcre2-utf16-10.32-3.el8.x86_64 2023-11-28T11:19:44.288 INFO:teuthology.orchestra.run.smithi169.stdout: pcre2-utf32-10.32-3.el8.x86_64 2023-11-28T11:19:44.288 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-10.23-2.module_el8+572+929c87ac.x86_64 2023-11-28T11:19:44.288 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-contrib-10.23-2.module_el8+572+929c87ac.x86_64 2023-11-28T11:19:44.289 INFO:teuthology.orchestra.run.smithi169.stdout: postgresql-server-10.23-2.module_el8+572+929c87ac.x86_64 2023-11-28T11:19:44.289 INFO:teuthology.orchestra.run.smithi169.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-11-28T11:19:44.289 INFO:teuthology.orchestra.run.smithi169.stdout: python3-babel-2.5.1-7.el8.noarch 2023-11-28T11:19:44.289 INFO:teuthology.orchestra.run.smithi169.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-28T11:19:44.289 INFO:teuthology.orchestra.run.smithi169.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-28T11:19:44.290 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-28T11:19:44.290 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:44.290 INFO:teuthology.orchestra.run.smithi169.stdout: python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:44.290 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:44.290 INFO:teuthology.orchestra.run.smithi169.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-28T11:19:44.291 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cffi-1.11.5-6.el8.x86_64 2023-11-28T11:19:44.291 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-28T11:19:44.291 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-28T11:19:44.291 INFO:teuthology.orchestra.run.smithi169.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2023-11-28T11:19:44.291 INFO:teuthology.orchestra.run.smithi169.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-28T11:19:44.291 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-28T11:19:44.292 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-28T11:19:44.292 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-11-28T11:19:44.292 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-28T11:19:44.292 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-28T11:19:44.292 INFO:teuthology.orchestra.run.smithi169.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-28T11:19:44.293 INFO:teuthology.orchestra.run.smithi169.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-28T11:19:44.293 INFO:teuthology.orchestra.run.smithi169.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-28T11:19:44.293 INFO:teuthology.orchestra.run.smithi169.stdout: python3-mako-1.0.6-14.el8.noarch 2023-11-28T11:19:44.293 INFO:teuthology.orchestra.run.smithi169.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-28T11:19:44.293 INFO:teuthology.orchestra.run.smithi169.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-28T11:19:44.294 INFO:teuthology.orchestra.run.smithi169.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-11-28T11:19:44.294 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-11-28T11:19:44.294 INFO:teuthology.orchestra.run.smithi169.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-11-28T11:19:44.294 INFO:teuthology.orchestra.run.smithi169.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-28T11:19:44.294 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-28T11:19:44.295 INFO:teuthology.orchestra.run.smithi169.stdout: python3-portend-2.6-1.el8.noarch 2023-11-28T11:19:44.295 INFO:teuthology.orchestra.run.smithi169.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-28T11:19:44.295 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-11-28T11:19:44.295 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-28T11:19:44.295 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-28T11:19:44.295 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pycparser-2.14-14.el8.noarch 2023-11-28T11:19:44.296 INFO:teuthology.orchestra.run.smithi169.stdout: python3-pytz-2017.2-11.el8.noarch 2023-11-28T11:19:44.296 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:44.296 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:44.296 INFO:teuthology.orchestra.run.smithi169.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-28T11:19:44.296 INFO:teuthology.orchestra.run.smithi169.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-28T11:19:44.297 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:44.297 INFO:teuthology.orchestra.run.smithi169.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-28T11:19:44.297 INFO:teuthology.orchestra.run.smithi169.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-28T11:19:44.297 INFO:teuthology.orchestra.run.smithi169.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2023-11-28T11:19:44.297 INFO:teuthology.orchestra.run.smithi169.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-28T11:19:44.297 INFO:teuthology.orchestra.run.smithi169.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-28T11:19:44.298 INFO:teuthology.orchestra.run.smithi169.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-28T11:19:44.298 INFO:teuthology.orchestra.run.smithi169.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-28T11:19:44.298 INFO:teuthology.orchestra.run.smithi169.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-28T11:19:44.298 INFO:teuthology.orchestra.run.smithi169.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-28T11:19:44.298 INFO:teuthology.orchestra.run.smithi169.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-28T11:19:44.299 INFO:teuthology.orchestra.run.smithi169.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-28T11:19:44.299 INFO:teuthology.orchestra.run.smithi169.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-28T11:19:44.299 INFO:teuthology.orchestra.run.smithi169.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-28T11:19:44.299 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:44.300 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:44.300 INFO:teuthology.orchestra.run.smithi169.stdout: rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:19:44.300 INFO:teuthology.orchestra.run.smithi169.stdout: socat-1.7.4.1-1.el8.x86_64 2023-11-28T11:19:44.300 INFO:teuthology.orchestra.run.smithi169.stdout: thrift-0.13.0-2.el8.x86_64 2023-11-28T11:19:44.300 INFO:teuthology.orchestra.run.smithi169.stdout: uuid-1.6.2-43.el8.x86_64 2023-11-28T11:19:44.300 INFO:teuthology.orchestra.run.smithi169.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2023-11-28T11:19:44.301 INFO:teuthology.orchestra.run.smithi169.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-28T11:19:44.302 INFO:teuthology.orchestra.run.smithi169.stdout: zlib-devel-1.2.11-21.el8.x86_64 2023-11-28T11:19:44.302 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-28T11:19:44.302 INFO:teuthology.orchestra.run.smithi169.stdout:Complete! 2023-11-28T11:19:44.580 DEBUG:teuthology.parallel:result is None 2023-11-28T11:19:44.580 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2f5f6028b893ad41357cd06e827373f0fe2a1232 2023-11-28T11:19:44.891 DEBUG:teuthology.orchestra.run.smithi092:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-28T11:19:44.955 INFO:teuthology.orchestra.run.smithi092.stdout:18.0.0-2697.g2f5f6028.el8 2023-11-28T11:19:44.956 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-2697.g2f5f6028.el8 2023-11-28T11:19:44.956 INFO:teuthology.task.install:The correct ceph version 18.0.0-2697.g2f5f6028 is installed. 2023-11-28T11:19:44.956 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2f5f6028b893ad41357cd06e827373f0fe2a1232 2023-11-28T11:19:45.269 DEBUG:teuthology.orchestra.run.smithi169:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-28T11:19:45.328 INFO:teuthology.orchestra.run.smithi169.stdout:18.0.0-2697.g2f5f6028.el8 2023-11-28T11:19:45.328 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-2697.g2f5f6028.el8 2023-11-28T11:19:45.328 INFO:teuthology.task.install:The correct ceph version 18.0.0-2697.g2f5f6028 is installed. 2023-11-28T11:19:45.329 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-11-28T11:19:45.329 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-28T11:19:45.329 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-28T11:19:45.414 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-28T11:19:45.414 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-28T11:19:45.494 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-11-28T11:19:45.494 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-28T11:19:45.494 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/daemon-helper 2023-11-28T11:19:45.573 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-28T11:19:45.691 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-28T11:19:45.691 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/daemon-helper 2023-11-28T11:19:45.762 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-28T11:19:45.874 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-11-28T11:19:45.874 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-28T11:19:45.874 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-28T11:19:45.952 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-28T11:19:46.070 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-28T11:19:46.071 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-28T11:19:46.147 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-28T11:19:46.261 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-11-28T11:19:46.261 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-28T11:19:46.261 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/stdin-killer 2023-11-28T11:19:46.340 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-28T11:19:46.418 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-28T11:19:46.419 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/usr/bin/stdin-killer 2023-11-28T11:19:46.492 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-28T11:19:46.608 INFO:teuthology.run_tasks:Running task ceph... 2023-11-28T11:19:46.674 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-11-28T11:19:46.674 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 777 /var/log/ceph 2023-11-28T11:19:46.679 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 777 /var/log/ceph 2023-11-28T11:19:46.752 INFO:tasks.ceph:Disabling ceph logrotate... 2023-11-28T11:19:46.752 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-28T11:19:46.757 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-28T11:19:46.828 INFO:tasks.ceph:Creating extra log directories... 2023-11-28T11:19:46.828 DEBUG:teuthology.orchestra.run.smithi092:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-28T11:19:46.833 DEBUG:teuthology.orchestra.run.smithi169:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-28T11:19:46.912 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-11-28T11:19:46.912 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-11-28T11:19:46.912 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-11-28T11:19:46.912 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-28T11:19:46.913 DEBUG:teuthology.orchestra.run.smithi092:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-28T11:19:46.959 DEBUG:teuthology.orchestra.run.smithi092:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-28T11:19:47.167 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-28T11:19:47.237 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-28T11:19:47.237 DEBUG:teuthology.orchestra.run.smithi169:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-28T11:19:47.285 DEBUG:teuthology.orchestra.run.smithi169:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-28T11:19:47.506 DEBUG:teuthology.orchestra.run.smithi169:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-28T11:19:47.584 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-11-28T11:19:47.585 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-11-28T11:19:47.585 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469955', 'branch': 'wip-neesingh-testing-231123', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub}', 'email': 'neesingh@redhat.com', 'first_in_suite': False, 'job_id': '7469955', 'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '8.stream', 'overrides': {'admin_socket': {'branch': 'wip-neesingh-testing-231123'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'issue_54557', 'sha1': 'e67b67438c1e5e13734a7203ff9e57e30b8e4d32'}}, 'owner': 'scheduled_neesingh@teuthology', 'priority': 10, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 9437, 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'issue_54557', 'suite_path': '/home/teuthworker/src/github.com_neesingh-rh_ceph_e67b67438c1e5e13734a7203ff9e57e30b8e4d32/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/neesingh-rh/ceph', 'suite_sha1': 'e67b67438c1e5e13734a7203ff9e57e30b8e4d32', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi092.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLBWKDssAcujaTJGYoh0m+6KRf79YlBnyP6+4+KzFgJNnb+zlPv8t0bboGt8tbKRVck/k776TBljTkSk6kIOe1M=', 'smithi169.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBspMRSneSLchVXtu1lFxgIfmFRbU9QHxtOh73ymQiDEyhUjA07U4CTxv98VSN9qWSUXOJ7F0WXBIc5AsA4w4ss='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232'}}, {'ceph': None}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_forward_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '5ae09a5cc7064ca2bf994df19f3006b79794789a', 'timestamp': '2023-11-28_10:48:03', 'tube': 'smithi', 'user': 'neesingh', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.232157'} 2023-11-28T11:19:47.585 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-28T11:19:47.590 DEBUG:teuthology.orchestra.run.smithi169:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-28T11:19:47.639 DEBUG:teuthology.orchestra.run.smithi092:> sudo install -d -m0777 -- /var/run/ceph 2023-11-28T11:19:47.677 DEBUG:teuthology.orchestra.run.smithi169:> sudo install -d -m0777 -- /var/run/ceph 2023-11-28T11:19:47.752 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-28T11:19:47.752 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/scratch_devs of=/dev/stdout 2023-11-28T11:19:47.798 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-28T11:19:47.798 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_1 2023-11-28T11:19:47.884 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-28T11:19:47.884 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T11:19:47.884 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 448 Links: 1 2023-11-28T11:19:47.884 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T11:19:47.884 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T11:19:47.884 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2023-11-28 11:19:14.020418115 +0000 2023-11-28T11:19:47.884 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2023-11-28 11:11:22.071841739 +0000 2023-11-28T11:19:47.884 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2023-11-28 11:11:22.071841739 +0000 2023-11-28T11:19:47.884 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2023-11-28 11:11:22.042842560 +0000 2023-11-28T11:19:47.885 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-28T11:19:47.990 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2023-11-28T11:19:47.990 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2023-11-28T11:19:47.990 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.00030301 s, 1.7 MB/s 2023-11-28T11:19:47.993 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-28T11:19:48.041 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_2 2023-11-28T11:19:48.087 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-28T11:19:48.087 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T11:19:48.087 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 454 Links: 1 2023-11-28T11:19:48.087 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T11:19:48.087 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T11:19:48.087 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2023-11-28 11:19:14.020418115 +0000 2023-11-28T11:19:48.087 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2023-11-28 11:11:22.917817787 +0000 2023-11-28T11:19:48.087 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2023-11-28 11:11:22.917817787 +0000 2023-11-28T11:19:48.087 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2023-11-28 11:11:22.891818523 +0000 2023-11-28T11:19:48.088 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-28T11:19:48.192 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2023-11-28T11:19:48.193 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2023-11-28T11:19:48.193 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000271698 s, 1.9 MB/s 2023-11-28T11:19:48.195 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-28T11:19:48.243 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_3 2023-11-28T11:19:48.289 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-28T11:19:48.289 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T11:19:48.289 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 460 Links: 1 2023-11-28T11:19:48.289 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T11:19:48.289 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T11:19:48.289 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2023-11-28 11:19:14.021418086 +0000 2023-11-28T11:19:48.289 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2023-11-28 11:11:23.764793808 +0000 2023-11-28T11:19:48.289 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2023-11-28 11:11:23.764793808 +0000 2023-11-28T11:19:48.290 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2023-11-28 11:11:23.741794460 +0000 2023-11-28T11:19:48.290 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-28T11:19:48.396 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2023-11-28T11:19:48.396 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2023-11-28T11:19:48.396 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.00028466 s, 1.8 MB/s 2023-11-28T11:19:48.399 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-28T11:19:48.448 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_4 2023-11-28T11:19:48.495 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-28T11:19:48.495 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T11:19:48.496 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 466 Links: 1 2023-11-28T11:19:48.496 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T11:19:48.496 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T11:19:48.496 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2023-11-28 11:19:14.021418086 +0000 2023-11-28T11:19:48.496 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2023-11-28 11:11:24.616769689 +0000 2023-11-28T11:19:48.497 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2023-11-28 11:11:24.616769689 +0000 2023-11-28T11:19:48.497 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2023-11-28 11:11:24.591770396 +0000 2023-11-28T11:19:48.497 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-28T11:19:48.602 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2023-11-28T11:19:48.602 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2023-11-28T11:19:48.602 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.00030718 s, 1.7 MB/s 2023-11-28T11:19:48.604 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-28T11:19:48.653 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T11:19:48.653 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-28T11:19:48.653 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/scratch_devs of=/dev/stdout 2023-11-28T11:19:48.702 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-28T11:19:48.702 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_1 2023-11-28T11:19:48.750 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-28T11:19:48.750 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T11:19:48.751 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 5h/5d Inode: 448 Links: 1 2023-11-28T11:19:48.751 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T11:19:48.751 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T11:19:48.751 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2023-11-28 11:19:40.048445648 +0000 2023-11-28T11:19:48.751 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2023-11-28 11:11:21.766111957 +0000 2023-11-28T11:19:48.751 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2023-11-28 11:11:21.766111957 +0000 2023-11-28T11:19:48.751 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2023-11-28 11:11:21.740112497 +0000 2023-11-28T11:19:48.751 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-28T11:19:48.822 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2023-11-28T11:19:48.822 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2023-11-28T11:19:48.822 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000297938 s, 1.7 MB/s 2023-11-28T11:19:48.825 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-28T11:19:48.876 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_2 2023-11-28T11:19:48.923 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-28T11:19:48.923 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T11:19:48.923 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 5h/5d Inode: 454 Links: 1 2023-11-28T11:19:48.923 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T11:19:48.923 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T11:19:48.923 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2023-11-28 11:19:40.048445648 +0000 2023-11-28T11:19:48.923 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2023-11-28 11:11:22.623094168 +0000 2023-11-28T11:19:48.924 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2023-11-28 11:11:22.623094168 +0000 2023-11-28T11:19:48.924 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2023-11-28 11:11:22.598094687 +0000 2023-11-28T11:19:48.924 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-28T11:19:48.994 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2023-11-28T11:19:48.995 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2023-11-28T11:19:48.995 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000270958 s, 1.9 MB/s 2023-11-28T11:19:48.996 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-28T11:19:49.045 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_3 2023-11-28T11:19:49.094 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-28T11:19:49.094 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T11:19:49.094 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 5h/5d Inode: 460 Links: 1 2023-11-28T11:19:49.094 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T11:19:49.094 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T11:19:49.094 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2023-11-28 11:19:40.049445626 +0000 2023-11-28T11:19:49.094 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2023-11-28 11:11:23.506075839 +0000 2023-11-28T11:19:49.094 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2023-11-28 11:11:23.506075839 +0000 2023-11-28T11:19:49.095 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2023-11-28 11:11:23.479076400 +0000 2023-11-28T11:19:49.095 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-28T11:19:49.166 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2023-11-28T11:19:49.166 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2023-11-28T11:19:49.167 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000311547 s, 1.6 MB/s 2023-11-28T11:19:49.169 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-28T11:19:49.220 DEBUG:teuthology.orchestra.run.smithi169:> stat /dev/vg_nvme/lv_4 2023-11-28T11:19:49.269 INFO:teuthology.orchestra.run.smithi169.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-28T11:19:49.269 INFO:teuthology.orchestra.run.smithi169.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T11:19:49.269 INFO:teuthology.orchestra.run.smithi169.stdout:Device: 5h/5d Inode: 466 Links: 1 2023-11-28T11:19:49.269 INFO:teuthology.orchestra.run.smithi169.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T11:19:49.270 INFO:teuthology.orchestra.run.smithi169.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T11:19:49.270 INFO:teuthology.orchestra.run.smithi169.stdout:Access: 2023-11-28 11:19:40.049445626 +0000 2023-11-28T11:19:49.270 INFO:teuthology.orchestra.run.smithi169.stdout:Modify: 2023-11-28 11:11:24.376057781 +0000 2023-11-28T11:19:49.270 INFO:teuthology.orchestra.run.smithi169.stdout:Change: 2023-11-28 11:11:24.376057781 +0000 2023-11-28T11:19:49.270 INFO:teuthology.orchestra.run.smithi169.stdout: Birth: 2023-11-28 11:11:24.350058320 +0000 2023-11-28T11:19:49.271 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-28T11:19:49.339 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records in 2023-11-28T11:19:49.339 INFO:teuthology.orchestra.run.smithi169.stderr:1+0 records out 2023-11-28T11:19:49.339 INFO:teuthology.orchestra.run.smithi169.stderr:512 bytes copied, 0.000290005 s, 1.8 MB/s 2023-11-28T11:19:49.341 DEBUG:teuthology.orchestra.run.smithi169:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-28T11:19:49.391 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T11:19:49.391 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi092.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi169.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-28T11:19:49.391 INFO:tasks.ceph:Generating config... 2023-11-28T11:19:49.392 INFO:tasks.ceph:[client] client mount timeout = 600 2023-11-28T11:19:49.392 INFO:tasks.ceph:[client] debug client = 20 2023-11-28T11:19:49.392 INFO:tasks.ceph:[client] debug ms = 1 2023-11-28T11:19:49.392 INFO:tasks.ceph:[client] pid file = 2023-11-28T11:19:49.393 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-11-28T11:19:49.393 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-11-28T11:19:49.393 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-11-28T11:19:49.393 INFO:tasks.ceph:[mds] debug mds = 20 2023-11-28T11:19:49.393 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-11-28T11:19:49.393 INFO:tasks.ceph:[mds] debug ms = 1 2023-11-28T11:19:49.393 INFO:tasks.ceph:[mds] mds debug frag = True 2023-11-28T11:19:49.393 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-11-28T11:19:49.393 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-11-28T11:19:49.393 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-11-28T11:19:49.394 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-11-28T11:19:49.394 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-11-28T11:19:49.394 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-11-28T11:19:49.394 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-11-28T11:19:49.394 INFO:tasks.ceph:[mgr] debug ms = 1 2023-11-28T11:19:49.394 INFO:tasks.ceph:[mon] debug mon = 20 2023-11-28T11:19:49.394 INFO:tasks.ceph:[mon] debug ms = 1 2023-11-28T11:19:49.394 INFO:tasks.ceph:[mon] debug paxos = 20 2023-11-28T11:19:49.394 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-11-28T11:19:49.394 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-11-28T11:19:49.394 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-11-28T11:19:49.395 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-11-28T11:19:49.395 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-11-28T11:19:49.395 INFO:tasks.ceph:[osd] debug ms = 1 2023-11-28T11:19:49.395 INFO:tasks.ceph:[osd] debug osd = 20 2023-11-28T11:19:49.395 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-11-28T11:19:49.395 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-11-28T11:19:49.395 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-11-28T11:19:49.395 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-11-28T11:19:49.395 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-11-28T11:19:49.395 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-11-28T11:19:49.395 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-11-28T11:19:49.395 INFO:tasks.ceph:Setting up mon.a... 2023-11-28T11:19:49.396 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-11-28T11:19:49.506 INFO:teuthology.orchestra.run.smithi092.stdout:creating /etc/ceph/ceph.keyring 2023-11-28T11:19:49.513 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-11-28T11:19:49.613 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-28T11:19:49.681 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.92'), ('mon.b', '172.21.15.169'), ('mon.c', '[v2:172.21.15.169:3301,v1:172.21.15.169:6790]')] 2023-11-28T11:19:49.682 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.92,172.21.15.169,[v2:172.21.15.169:3301,v1:172.21.15.169:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-11-28T11:19:49.683 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-28T11:19:49.683 DEBUG:teuthology.orchestra.run.smithi092:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-11-28T11:19:49.728 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.92 --add b 172.21.15.169 --addv c '[v2:172.21.15.169:3301,v1:172.21.15.169:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:19:49.857 INFO:teuthology.orchestra.run.smithi092.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:19:49.857 INFO:teuthology.orchestra.run.smithi092.stdout:monmaptool: generated fsid 68956e0c-c16f-47cd-a220-f98364b0831d 2023-11-28T11:19:49.857 INFO:teuthology.orchestra.run.smithi092.stdout:setting min_mon_release = pacific 2023-11-28T11:19:49.857 INFO:teuthology.orchestra.run.smithi092.stdout:epoch 0 2023-11-28T11:19:49.857 INFO:teuthology.orchestra.run.smithi092.stdout:fsid 68956e0c-c16f-47cd-a220-f98364b0831d 2023-11-28T11:19:49.857 INFO:teuthology.orchestra.run.smithi092.stdout:last_changed 2023-11-28T11:19:49.857076+0000 2023-11-28T11:19:49.857 INFO:teuthology.orchestra.run.smithi092.stdout:created 2023-11-28T11:19:49.857076+0000 2023-11-28T11:19:49.858 INFO:teuthology.orchestra.run.smithi092.stdout:min_mon_release 16 (pacific) 2023-11-28T11:19:49.858 INFO:teuthology.orchestra.run.smithi092.stdout:election_strategy: 1 2023-11-28T11:19:49.858 INFO:teuthology.orchestra.run.smithi092.stdout:0: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.a 2023-11-28T11:19:49.858 INFO:teuthology.orchestra.run.smithi092.stdout:1: [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] mon.b 2023-11-28T11:19:49.858 INFO:teuthology.orchestra.run.smithi092.stdout:2: [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] mon.c 2023-11-28T11:19:49.858 INFO:teuthology.orchestra.run.smithi092.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-11-28T11:19:49.860 DEBUG:teuthology.orchestra.run.smithi092:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-11-28T11:19:49.905 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 68956e0c-c16f-47cd-a220-f98364b0831d... 2023-11-28T11:19:49.906 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-28T11:19:49.950 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-28T11:19:50.062 INFO:teuthology.orchestra.run.smithi092.stdout:[global] 2023-11-28T11:19:50.062 INFO:teuthology.orchestra.run.smithi092.stdout: chdir = "" 2023-11-28T11:19:50.062 INFO:teuthology.orchestra.run.smithi092.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-28T11:19:50.062 INFO:teuthology.orchestra.run.smithi092.stdout: auth supported = cephx 2023-11-28T11:19:50.062 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:19:50.063 INFO:teuthology.orchestra.run.smithi092.stdout: filestore xattr use omap = true 2023-11-28T11:19:50.063 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:19:50.063 INFO:teuthology.orchestra.run.smithi092.stdout: mon clock drift allowed = 1.000 2023-11-28T11:19:50.063 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:19:50.063 INFO:teuthology.orchestra.run.smithi092.stdout: osd crush chooseleaf type = 0 2023-11-28T11:19:50.063 INFO:teuthology.orchestra.run.smithi092.stdout: auth debug = true 2023-11-28T11:19:50.063 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:19:50.063 INFO:teuthology.orchestra.run.smithi092.stdout: ms die on old message = true 2023-11-28T11:19:50.063 INFO:teuthology.orchestra.run.smithi092.stdout: ms die on bug = true 2023-11-28T11:19:50.063 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:19:50.063 INFO:teuthology.orchestra.run.smithi092.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-28T11:19:50.064 INFO:teuthology.orchestra.run.smithi092.stdout: mon pg warn max object skew = 0 2023-11-28T11:19:50.064 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:19:50.064 INFO:teuthology.orchestra.run.smithi092.stdout: # disable pg_autoscaler by default for new pools 2023-11-28T11:19:50.064 INFO:teuthology.orchestra.run.smithi092.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-28T11:19:50.064 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:19:50.064 INFO:teuthology.orchestra.run.smithi092.stdout: osd pool default size = 2 2023-11-28T11:19:50.064 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:19:50.064 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd allow primary affinity = true 2023-11-28T11:19:50.064 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd allow pg remap = true 2023-11-28T11:19:50.064 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on legacy crush tunables = false 2023-11-28T11:19:50.064 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on crush straw calc version zero = false 2023-11-28T11:19:50.065 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on no sortbitwise = false 2023-11-28T11:19:50.065 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on osd down out interval zero = false 2023-11-28T11:19:50.065 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on too few osds = false 2023-11-28T11:19:50.065 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-28T11:19:50.065 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-28T11:19:50.065 INFO:teuthology.orchestra.run.smithi092.stdout: mon_allow_pool_size_one = true 2023-11-28T11:19:50.065 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:19:50.065 INFO:teuthology.orchestra.run.smithi092.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-28T11:19:50.065 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:19:50.065 INFO:teuthology.orchestra.run.smithi092.stdout: osd default data pool replay window = 5 2023-11-28T11:19:50.065 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:19:50.066 INFO:teuthology.orchestra.run.smithi092.stdout: mon allow pool delete = true 2023-11-28T11:19:50.066 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:19:50.066 INFO:teuthology.orchestra.run.smithi092.stdout: mon cluster log file level = debug 2023-11-28T11:19:50.066 INFO:teuthology.orchestra.run.smithi092.stdout: debug asserts on shutdown = true 2023-11-28T11:19:50.066 INFO:teuthology.orchestra.run.smithi092.stdout: mon health detail to clog = false 2023-11-28T11:19:50.066 INFO:teuthology.orchestra.run.smithi092.stdout: mon host = "172.21.15.92,172.21.15.169,[v2:172.21.15.169:3301,v1:172.21.15.169:6790]" 2023-11-28T11:19:50.066 INFO:teuthology.orchestra.run.smithi092.stdout: ms die on skipped message = False 2023-11-28T11:19:50.066 INFO:teuthology.orchestra.run.smithi092.stdout: fsid = 68956e0c-c16f-47cd-a220-f98364b0831d 2023-11-28T11:19:50.066 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:19:50.066 INFO:teuthology.orchestra.run.smithi092.stdout:[osd] 2023-11-28T11:19:50.067 INFO:teuthology.orchestra.run.smithi092.stdout: osd journal size = 100 2023-11-28T11:19:50.067 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:19:50.067 INFO:teuthology.orchestra.run.smithi092.stdout: osd scrub load threshold = 5.0 2023-11-28T11:19:50.067 INFO:teuthology.orchestra.run.smithi092.stdout: osd scrub max interval = 600 2023-11-28T11:19:50.067 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:19:50.067 INFO:teuthology.orchestra.run.smithi092.stdout: osd recover clone overlap = true 2023-11-28T11:19:50.067 INFO:teuthology.orchestra.run.smithi092.stdout: osd recovery max chunk = 1048576 2023-11-28T11:19:50.067 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:19:50.067 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug shutdown = true 2023-11-28T11:19:50.067 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug op order = true 2023-11-28T11:19:50.068 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug verify stray on activate = true 2023-11-28T11:19:50.068 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:19:50.068 INFO:teuthology.orchestra.run.smithi092.stdout: osd open classes on start = true 2023-11-28T11:19:50.068 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug pg log writeout = true 2023-11-28T11:19:50.068 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:19:50.068 INFO:teuthology.orchestra.run.smithi092.stdout: osd deep scrub update digest min age = 30 2023-11-28T11:19:50.068 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:19:50.068 INFO:teuthology.orchestra.run.smithi092.stdout: osd map max advance = 10 2023-11-28T11:19:50.068 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:19:50.068 INFO:teuthology.orchestra.run.smithi092.stdout: journal zero on create = true 2023-11-28T11:19:50.069 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:19:50.069 INFO:teuthology.orchestra.run.smithi092.stdout: filestore ondisk finisher threads = 3 2023-11-28T11:19:50.069 INFO:teuthology.orchestra.run.smithi092.stdout: filestore apply finisher threads = 3 2023-11-28T11:19:50.069 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:19:50.069 INFO:teuthology.orchestra.run.smithi092.stdout: bdev debug aio = true 2023-11-28T11:19:50.069 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug misdirected ops = true 2023-11-28T11:19:50.069 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore block size = 96636764160 2023-11-28T11:19:50.069 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore fsck on mount = True 2023-11-28T11:19:50.069 INFO:teuthology.orchestra.run.smithi092.stdout: debug bluefs = 20 2023-11-28T11:19:50.070 INFO:teuthology.orchestra.run.smithi092.stdout: debug bluestore = 20 2023-11-28T11:19:50.070 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2023-11-28T11:19:50.070 INFO:teuthology.orchestra.run.smithi092.stdout: debug osd = 20 2023-11-28T11:19:50.070 INFO:teuthology.orchestra.run.smithi092.stdout: debug rocksdb = 10 2023-11-28T11:19:50.070 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd backfillfull_ratio = 0.85 2023-11-28T11:19:50.070 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd full ratio = 0.9 2023-11-28T11:19:50.070 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd nearfull ratio = 0.8 2023-11-28T11:19:50.070 INFO:teuthology.orchestra.run.smithi092.stdout: osd failsafe full ratio = 0.95 2023-11-28T11:19:50.070 INFO:teuthology.orchestra.run.smithi092.stdout: osd objectstore = bluestore 2023-11-28T11:19:50.070 INFO:teuthology.orchestra.run.smithi092.stdout: osd op complaint time = 180 2023-11-28T11:19:50.070 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:19:50.070 INFO:teuthology.orchestra.run.smithi092.stdout:[mgr] 2023-11-28T11:19:50.071 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2023-11-28T11:19:50.071 INFO:teuthology.orchestra.run.smithi092.stdout: debug mgr = 20 2023-11-28T11:19:50.071 INFO:teuthology.orchestra.run.smithi092.stdout: debug mon = 20 2023-11-28T11:19:50.071 INFO:teuthology.orchestra.run.smithi092.stdout: debug auth = 20 2023-11-28T11:19:50.071 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min pgs per osd = 4 2023-11-28T11:19:50.071 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min bytes per osd = 10 2023-11-28T11:19:50.071 INFO:teuthology.orchestra.run.smithi092.stdout: mgr/telemetry/nag = false 2023-11-28T11:19:50.071 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:19:50.071 INFO:teuthology.orchestra.run.smithi092.stdout:[mon] 2023-11-28T11:19:50.071 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2023-11-28T11:19:50.071 INFO:teuthology.orchestra.run.smithi092.stdout: debug mon = 20 2023-11-28T11:19:50.071 INFO:teuthology.orchestra.run.smithi092.stdout: debug paxos = 20 2023-11-28T11:19:50.072 INFO:teuthology.orchestra.run.smithi092.stdout: debug auth = 20 2023-11-28T11:19:50.072 INFO:teuthology.orchestra.run.smithi092.stdout: mon data avail warn = 5 2023-11-28T11:19:50.072 INFO:teuthology.orchestra.run.smithi092.stdout: mon mgr mkfs grace = 240 2023-11-28T11:19:50.072 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min pgs per osd = 4 2023-11-28T11:19:50.072 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd reporter subtree level = osd 2023-11-28T11:19:50.072 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd prime pg temp = true 2023-11-28T11:19:50.072 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min bytes per osd = 10 2023-11-28T11:19:50.072 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:19:50.072 INFO:teuthology.orchestra.run.smithi092.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-28T11:19:50.072 INFO:teuthology.orchestra.run.smithi092.stdout: auth mon ticket ttl = 660 # 11m 2023-11-28T11:19:50.072 INFO:teuthology.orchestra.run.smithi092.stdout: auth service ticket ttl = 240 # 4m 2023-11-28T11:19:50.072 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:19:50.072 INFO:teuthology.orchestra.run.smithi092.stdout: # don't complain about insecure global_id in the test suite 2023-11-28T11:19:50.073 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-28T11:19:50.073 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-28T11:19:50.073 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:19:50.073 INFO:teuthology.orchestra.run.smithi092.stdout: # 1m isn't quite enough 2023-11-28T11:19:50.073 INFO:teuthology.orchestra.run.smithi092.stdout: mon_down_mkfs_grace = 2m 2023-11-28T11:19:50.073 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:19:50.073 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_filestore_osds = false 2023-11-28T11:19:50.073 INFO:teuthology.orchestra.run.smithi092.stdout: mon op complaint time = 120 2023-11-28T11:19:50.074 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:19:50.074 INFO:teuthology.orchestra.run.smithi092.stdout:[client] 2023-11-28T11:19:50.074 INFO:teuthology.orchestra.run.smithi092.stdout: rgw cache enabled = true 2023-11-28T11:19:50.074 INFO:teuthology.orchestra.run.smithi092.stdout: rgw enable ops log = true 2023-11-28T11:19:50.074 INFO:teuthology.orchestra.run.smithi092.stdout: rgw enable usage log = true 2023-11-28T11:19:50.074 INFO:teuthology.orchestra.run.smithi092.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-28T11:19:50.074 INFO:teuthology.orchestra.run.smithi092.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-28T11:19:50.074 INFO:teuthology.orchestra.run.smithi092.stdout: client mount timeout = 600 2023-11-28T11:19:50.074 INFO:teuthology.orchestra.run.smithi092.stdout: debug client = 20 2023-11-28T11:19:50.074 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2023-11-28T11:19:50.074 INFO:teuthology.orchestra.run.smithi092.stdout: pid file = "" 2023-11-28T11:19:50.074 INFO:teuthology.orchestra.run.smithi092.stdout: rados mon op timeout = 900 2023-11-28T11:19:50.075 INFO:teuthology.orchestra.run.smithi092.stdout: rados osd op timeout = 900 2023-11-28T11:19:50.075 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.a] 2023-11-28T11:19:50.075 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.b] 2023-11-28T11:19:50.075 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.c] 2023-11-28T11:19:50.075 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.a] 2023-11-28T11:19:50.075 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.c] 2023-11-28T11:19:50.075 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.b] 2023-11-28T11:19:50.075 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.d] 2023-11-28T11:19:50.075 INFO:teuthology.orchestra.run.smithi092.stdout:[mds] 2023-11-28T11:19:50.075 INFO:teuthology.orchestra.run.smithi092.stdout: debug mds = 20 2023-11-28T11:19:50.075 INFO:teuthology.orchestra.run.smithi092.stdout: debug mds balancer = 20 2023-11-28T11:19:50.075 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2023-11-28T11:19:50.075 INFO:teuthology.orchestra.run.smithi092.stdout: mds debug frag = True 2023-11-28T11:19:50.076 INFO:teuthology.orchestra.run.smithi092.stdout: mds debug scatterstat = True 2023-11-28T11:19:50.076 INFO:teuthology.orchestra.run.smithi092.stdout: mds op complaint time = 180 2023-11-28T11:19:50.076 INFO:teuthology.orchestra.run.smithi092.stdout: mds verify scatter = True 2023-11-28T11:19:50.076 INFO:teuthology.orchestra.run.smithi092.stdout: osd op complaint time = 180 2023-11-28T11:19:50.076 INFO:teuthology.orchestra.run.smithi092.stdout: rados mon op timeout = 900 2023-11-28T11:19:50.076 INFO:teuthology.orchestra.run.smithi092.stdout: rados osd op timeout = 900 2023-11-28T11:19:50.076 INFO:teuthology.orchestra.run.smithi169.stdout:[global] 2023-11-28T11:19:50.076 INFO:teuthology.orchestra.run.smithi169.stdout: chdir = "" 2023-11-28T11:19:50.076 INFO:teuthology.orchestra.run.smithi169.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-28T11:19:50.077 INFO:teuthology.orchestra.run.smithi169.stdout: auth supported = cephx 2023-11-28T11:19:50.077 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-28T11:19:50.077 INFO:teuthology.orchestra.run.smithi169.stdout: filestore xattr use omap = true 2023-11-28T11:19:50.077 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-28T11:19:50.077 INFO:teuthology.orchestra.run.smithi169.stdout: mon clock drift allowed = 1.000 2023-11-28T11:19:50.077 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-28T11:19:50.077 INFO:teuthology.orchestra.run.smithi169.stdout: osd crush chooseleaf type = 0 2023-11-28T11:19:50.077 INFO:teuthology.orchestra.run.smithi169.stdout: auth debug = true 2023-11-28T11:19:50.077 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-28T11:19:50.077 INFO:teuthology.orchestra.run.smithi169.stdout: ms die on old message = true 2023-11-28T11:19:50.077 INFO:teuthology.orchestra.run.smithi169.stdout: ms die on bug = true 2023-11-28T11:19:50.077 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-28T11:19:50.077 INFO:teuthology.orchestra.run.smithi169.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-28T11:19:50.078 INFO:teuthology.orchestra.run.smithi169.stdout: mon pg warn max object skew = 0 2023-11-28T11:19:50.078 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-28T11:19:50.078 INFO:teuthology.orchestra.run.smithi169.stdout: # disable pg_autoscaler by default for new pools 2023-11-28T11:19:50.078 INFO:teuthology.orchestra.run.smithi169.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-28T11:19:50.078 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-28T11:19:50.078 INFO:teuthology.orchestra.run.smithi169.stdout: osd pool default size = 2 2023-11-28T11:19:50.078 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-28T11:19:50.078 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd allow primary affinity = true 2023-11-28T11:19:50.078 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd allow pg remap = true 2023-11-28T11:19:50.078 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on legacy crush tunables = false 2023-11-28T11:19:50.078 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on crush straw calc version zero = false 2023-11-28T11:19:50.078 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on no sortbitwise = false 2023-11-28T11:19:50.079 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on osd down out interval zero = false 2023-11-28T11:19:50.079 INFO:teuthology.orchestra.run.smithi169.stdout: mon warn on too few osds = false 2023-11-28T11:19:50.079 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-28T11:19:50.079 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-28T11:19:50.079 INFO:teuthology.orchestra.run.smithi169.stdout: mon_allow_pool_size_one = true 2023-11-28T11:19:50.079 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-28T11:19:50.079 INFO:teuthology.orchestra.run.smithi169.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-28T11:19:50.079 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-28T11:19:50.079 INFO:teuthology.orchestra.run.smithi169.stdout: osd default data pool replay window = 5 2023-11-28T11:19:50.079 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-28T11:19:50.079 INFO:teuthology.orchestra.run.smithi169.stdout: mon allow pool delete = true 2023-11-28T11:19:50.079 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-28T11:19:50.080 INFO:teuthology.orchestra.run.smithi169.stdout: mon cluster log file level = debug 2023-11-28T11:19:50.080 INFO:teuthology.orchestra.run.smithi169.stdout: debug asserts on shutdown = true 2023-11-28T11:19:50.080 INFO:teuthology.orchestra.run.smithi169.stdout: mon health detail to clog = false 2023-11-28T11:19:50.080 INFO:teuthology.orchestra.run.smithi169.stdout: mon host = "172.21.15.92,172.21.15.169,[v2:172.21.15.169:3301,v1:172.21.15.169:6790]" 2023-11-28T11:19:50.080 INFO:teuthology.orchestra.run.smithi169.stdout: ms die on skipped message = False 2023-11-28T11:19:50.080 INFO:teuthology.orchestra.run.smithi169.stdout: fsid = 68956e0c-c16f-47cd-a220-f98364b0831d 2023-11-28T11:19:50.080 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-28T11:19:50.080 INFO:teuthology.orchestra.run.smithi169.stdout:[osd] 2023-11-28T11:19:50.080 INFO:teuthology.orchestra.run.smithi169.stdout: osd journal size = 100 2023-11-28T11:19:50.080 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-28T11:19:50.080 INFO:teuthology.orchestra.run.smithi169.stdout: osd scrub load threshold = 5.0 2023-11-28T11:19:50.081 INFO:teuthology.orchestra.run.smithi169.stdout: osd scrub max interval = 600 2023-11-28T11:19:50.081 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-28T11:19:50.081 INFO:teuthology.orchestra.run.smithi169.stdout: osd recover clone overlap = true 2023-11-28T11:19:50.081 INFO:teuthology.orchestra.run.smithi169.stdout: osd recovery max chunk = 1048576 2023-11-28T11:19:50.081 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-28T11:19:50.081 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug shutdown = true 2023-11-28T11:19:50.081 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug op order = true 2023-11-28T11:19:50.081 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug verify stray on activate = true 2023-11-28T11:19:50.081 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-28T11:19:50.081 INFO:teuthology.orchestra.run.smithi169.stdout: osd open classes on start = true 2023-11-28T11:19:50.081 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug pg log writeout = true 2023-11-28T11:19:50.081 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-28T11:19:50.082 INFO:teuthology.orchestra.run.smithi169.stdout: osd deep scrub update digest min age = 30 2023-11-28T11:19:50.082 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-28T11:19:50.082 INFO:teuthology.orchestra.run.smithi169.stdout: osd map max advance = 10 2023-11-28T11:19:50.082 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-28T11:19:50.082 INFO:teuthology.orchestra.run.smithi169.stdout: journal zero on create = true 2023-11-28T11:19:50.082 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-28T11:19:50.082 INFO:teuthology.orchestra.run.smithi169.stdout: filestore ondisk finisher threads = 3 2023-11-28T11:19:50.082 INFO:teuthology.orchestra.run.smithi169.stdout: filestore apply finisher threads = 3 2023-11-28T11:19:50.082 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-28T11:19:50.082 INFO:teuthology.orchestra.run.smithi169.stdout: bdev debug aio = true 2023-11-28T11:19:50.082 INFO:teuthology.orchestra.run.smithi169.stdout: osd debug misdirected ops = true 2023-11-28T11:19:50.082 INFO:teuthology.orchestra.run.smithi169.stdout: bluestore block size = 96636764160 2023-11-28T11:19:50.083 INFO:teuthology.orchestra.run.smithi169.stdout: bluestore fsck on mount = True 2023-11-28T11:19:50.083 INFO:teuthology.orchestra.run.smithi169.stdout: debug bluefs = 20 2023-11-28T11:19:50.083 INFO:teuthology.orchestra.run.smithi169.stdout: debug bluestore = 20 2023-11-28T11:19:50.083 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2023-11-28T11:19:50.083 INFO:teuthology.orchestra.run.smithi169.stdout: debug osd = 20 2023-11-28T11:19:50.083 INFO:teuthology.orchestra.run.smithi169.stdout: debug rocksdb = 10 2023-11-28T11:19:50.083 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd backfillfull_ratio = 0.85 2023-11-28T11:19:50.083 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd full ratio = 0.9 2023-11-28T11:19:50.083 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd nearfull ratio = 0.8 2023-11-28T11:19:50.083 INFO:teuthology.orchestra.run.smithi169.stdout: osd failsafe full ratio = 0.95 2023-11-28T11:19:50.083 INFO:teuthology.orchestra.run.smithi169.stdout: osd objectstore = bluestore 2023-11-28T11:19:50.083 INFO:teuthology.orchestra.run.smithi169.stdout: osd op complaint time = 180 2023-11-28T11:19:50.084 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-28T11:19:50.084 INFO:teuthology.orchestra.run.smithi169.stdout:[mgr] 2023-11-28T11:19:50.084 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2023-11-28T11:19:50.084 INFO:teuthology.orchestra.run.smithi169.stdout: debug mgr = 20 2023-11-28T11:19:50.084 INFO:teuthology.orchestra.run.smithi169.stdout: debug mon = 20 2023-11-28T11:19:50.084 INFO:teuthology.orchestra.run.smithi169.stdout: debug auth = 20 2023-11-28T11:19:50.084 INFO:teuthology.orchestra.run.smithi169.stdout: mon reweight min pgs per osd = 4 2023-11-28T11:19:50.084 INFO:teuthology.orchestra.run.smithi169.stdout: mon reweight min bytes per osd = 10 2023-11-28T11:19:50.084 INFO:teuthology.orchestra.run.smithi169.stdout: mgr/telemetry/nag = false 2023-11-28T11:19:50.085 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-28T11:19:50.085 INFO:teuthology.orchestra.run.smithi169.stdout:[mon] 2023-11-28T11:19:50.085 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2023-11-28T11:19:50.085 INFO:teuthology.orchestra.run.smithi169.stdout: debug mon = 20 2023-11-28T11:19:50.085 INFO:teuthology.orchestra.run.smithi169.stdout: debug paxos = 20 2023-11-28T11:19:50.085 INFO:teuthology.orchestra.run.smithi169.stdout: debug auth = 20 2023-11-28T11:19:50.085 INFO:teuthology.orchestra.run.smithi169.stdout: mon data avail warn = 5 2023-11-28T11:19:50.085 INFO:teuthology.orchestra.run.smithi169.stdout: mon mgr mkfs grace = 240 2023-11-28T11:19:50.085 INFO:teuthology.orchestra.run.smithi169.stdout: mon reweight min pgs per osd = 4 2023-11-28T11:19:50.085 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd reporter subtree level = osd 2023-11-28T11:19:50.085 INFO:teuthology.orchestra.run.smithi169.stdout: mon osd prime pg temp = true 2023-11-28T11:19:50.085 INFO:teuthology.orchestra.run.smithi169.stdout: mon reweight min bytes per osd = 10 2023-11-28T11:19:50.086 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-28T11:19:50.086 INFO:teuthology.orchestra.run.smithi169.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-28T11:19:50.086 INFO:teuthology.orchestra.run.smithi169.stdout: auth mon ticket ttl = 660 # 11m 2023-11-28T11:19:50.086 INFO:teuthology.orchestra.run.smithi169.stdout: auth service ticket ttl = 240 # 4m 2023-11-28T11:19:50.086 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-28T11:19:50.086 INFO:teuthology.orchestra.run.smithi169.stdout: # don't complain about insecure global_id in the test suite 2023-11-28T11:19:50.086 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-28T11:19:50.086 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-28T11:19:50.086 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-28T11:19:50.087 INFO:teuthology.orchestra.run.smithi169.stdout: # 1m isn't quite enough 2023-11-28T11:19:50.087 INFO:teuthology.orchestra.run.smithi169.stdout: mon_down_mkfs_grace = 2m 2023-11-28T11:19:50.087 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-28T11:19:50.087 INFO:teuthology.orchestra.run.smithi169.stdout: mon_warn_on_filestore_osds = false 2023-11-28T11:19:50.087 INFO:teuthology.orchestra.run.smithi169.stdout: mon op complaint time = 120 2023-11-28T11:19:50.087 INFO:teuthology.orchestra.run.smithi169.stdout: 2023-11-28T11:19:50.087 INFO:teuthology.orchestra.run.smithi169.stdout:[client] 2023-11-28T11:19:50.087 INFO:teuthology.orchestra.run.smithi169.stdout: rgw cache enabled = true 2023-11-28T11:19:50.087 INFO:teuthology.orchestra.run.smithi169.stdout: rgw enable ops log = true 2023-11-28T11:19:50.088 INFO:teuthology.orchestra.run.smithi169.stdout: rgw enable usage log = true 2023-11-28T11:19:50.088 INFO:teuthology.orchestra.run.smithi169.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-28T11:19:50.088 INFO:teuthology.orchestra.run.smithi169.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-28T11:19:50.088 INFO:teuthology.orchestra.run.smithi169.stdout: client mount timeout = 600 2023-11-28T11:19:50.088 INFO:teuthology.orchestra.run.smithi169.stdout: debug client = 20 2023-11-28T11:19:50.088 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2023-11-28T11:19:50.088 INFO:teuthology.orchestra.run.smithi169.stdout: pid file = "" 2023-11-28T11:19:50.088 INFO:teuthology.orchestra.run.smithi169.stdout: rados mon op timeout = 900 2023-11-28T11:19:50.088 INFO:teuthology.orchestra.run.smithi169.stdout: rados osd op timeout = 900 2023-11-28T11:19:50.088 INFO:teuthology.orchestra.run.smithi169.stdout:[mon.a] 2023-11-28T11:19:50.088 INFO:teuthology.orchestra.run.smithi169.stdout:[mon.b] 2023-11-28T11:19:50.089 INFO:teuthology.orchestra.run.smithi169.stdout:[mon.c] 2023-11-28T11:19:50.089 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.a] 2023-11-28T11:19:50.089 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.c] 2023-11-28T11:19:50.089 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.b] 2023-11-28T11:19:50.089 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.d] 2023-11-28T11:19:50.089 INFO:teuthology.orchestra.run.smithi169.stdout:[mds] 2023-11-28T11:19:50.089 INFO:teuthology.orchestra.run.smithi169.stdout: debug mds = 20 2023-11-28T11:19:50.089 INFO:teuthology.orchestra.run.smithi169.stdout: debug mds balancer = 20 2023-11-28T11:19:50.089 INFO:teuthology.orchestra.run.smithi169.stdout: debug ms = 1 2023-11-28T11:19:50.089 INFO:teuthology.orchestra.run.smithi169.stdout: mds debug frag = True 2023-11-28T11:19:50.090 INFO:teuthology.orchestra.run.smithi169.stdout: mds debug scatterstat = True 2023-11-28T11:19:50.090 INFO:teuthology.orchestra.run.smithi169.stdout: mds op complaint time = 180 2023-11-28T11:19:50.090 INFO:teuthology.orchestra.run.smithi169.stdout: mds verify scatter = True 2023-11-28T11:19:50.090 INFO:teuthology.orchestra.run.smithi169.stdout: osd op complaint time = 180 2023-11-28T11:19:50.090 INFO:teuthology.orchestra.run.smithi169.stdout: rados mon op timeout = 900 2023-11-28T11:19:50.090 INFO:teuthology.orchestra.run.smithi169.stdout: rados osd op timeout = 900 2023-11-28T11:19:50.105 INFO:tasks.ceph:Creating admin key on mon.a... 2023-11-28T11:19:50.105 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-11-28T11:19:50.216 INFO:tasks.ceph:Copying monmap to all nodes... 2023-11-28T11:19:50.217 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-28T11:19:50.217 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-11-28T11:19:50.262 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-28T11:19:50.263 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-11-28T11:19:50.348 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi092.front.sepia.ceph.com 2023-11-28T11:19:50.348 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-28T11:19:50.348 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-28T11:19:50.348 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-28T11:19:50.483 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-28T11:19:50.483 DEBUG:teuthology.orchestra.run.smithi092:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:19:50.569 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi169.front.sepia.ceph.com 2023-11-28T11:19:50.569 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-28T11:19:50.569 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-28T11:19:50.570 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-28T11:19:50.666 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-28T11:19:50.667 DEBUG:teuthology.orchestra.run.smithi169:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:19:50.752 INFO:tasks.ceph:Setting up mon nodes... 2023-11-28T11:19:50.752 INFO:tasks.ceph:Setting up mgr nodes... 2023-11-28T11:19:50.753 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-11-28T11:19:50.875 INFO:teuthology.orchestra.run.smithi092.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-11-28T11:19:50.881 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-11-28T11:19:51.019 INFO:teuthology.orchestra.run.smithi169.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-11-28T11:19:51.025 INFO:tasks.ceph:Setting up mds nodes... 2023-11-28T11:19:51.026 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-11-28T11:19:51.147 INFO:teuthology.orchestra.run.smithi092.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-11-28T11:19:51.152 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-11-28T11:19:51.221 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-11-28T11:19:51.342 INFO:teuthology.orchestra.run.smithi092.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-11-28T11:19:51.347 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-11-28T11:19:51.417 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-11-28T11:19:51.540 INFO:teuthology.orchestra.run.smithi169.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-11-28T11:19:51.545 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-11-28T11:19:51.616 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-11-28T11:19:51.738 INFO:teuthology.orchestra.run.smithi169.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-11-28T11:19:51.743 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-11-28T11:19:51.812 INFO:tasks.ceph_client:Setting up client nodes... 2023-11-28T11:19:51.813 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-11-28T11:19:51.909 INFO:teuthology.orchestra.run.smithi092.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-11-28T11:19:51.940 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-11-28T11:19:52.037 INFO:teuthology.orchestra.run.smithi092.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-11-28T11:19:52.067 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-11-28T11:19:52.164 INFO:teuthology.orchestra.run.smithi169.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-11-28T11:19:52.194 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-11-28T11:19:52.291 INFO:teuthology.orchestra.run.smithi169.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-11-28T11:19:52.321 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-11-28T11:19:52.322 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi092.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi169.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-28T11:19:52.322 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-11-28T11:19:52.392 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T11:19:52.392 INFO:tasks.ceph:role: osd.0 2023-11-28T11:19:52.392 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi092.front.sepia.ceph.com 2023-11-28T11:19:52.392 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-28T11:19:52.468 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T11:19:52.468 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T11:19:52.468 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T11:19:52.468 INFO:teuthology.orchestra.run.smithi092.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T11:19:52.469 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T11:19:52.469 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2023-11-28T11:19:52.469 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T11:19:52.469 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T11:19:52.469 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T11:19:52.469 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T11:19:52.900 INFO:teuthology.orchestra.run.smithi092.stdout:Discarding blocks...Done. 2023-11-28T11:19:52.906 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2023-11-28T11:19:52.906 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-11-28T11:19:53.095 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-11-28T11:19:53.174 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-11-28T11:19:53.245 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T11:19:53.245 INFO:tasks.ceph:role: osd.1 2023-11-28T11:19:53.245 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi092.front.sepia.ceph.com 2023-11-28T11:19:53.245 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-28T11:19:53.322 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T11:19:53.322 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T11:19:53.323 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T11:19:53.323 INFO:teuthology.orchestra.run.smithi092.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T11:19:53.323 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T11:19:53.323 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2023-11-28T11:19:53.323 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T11:19:53.323 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T11:19:53.323 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T11:19:53.323 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T11:19:53.732 INFO:teuthology.orchestra.run.smithi092.stdout:Discarding blocks...Done. 2023-11-28T11:19:53.736 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2023-11-28T11:19:53.736 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-11-28T11:19:53.882 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-11-28T11:19:53.959 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-11-28T11:19:54.028 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T11:19:54.029 INFO:tasks.ceph:role: osd.2 2023-11-28T11:19:54.029 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi092.front.sepia.ceph.com 2023-11-28T11:19:54.029 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-28T11:19:54.106 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T11:19:54.107 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T11:19:54.107 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T11:19:54.107 INFO:teuthology.orchestra.run.smithi092.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T11:19:54.107 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T11:19:54.107 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2023-11-28T11:19:54.107 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T11:19:54.108 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T11:19:54.108 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T11:19:54.108 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T11:19:54.555 INFO:teuthology.orchestra.run.smithi092.stdout:Discarding blocks...Done. 2023-11-28T11:19:54.559 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2023-11-28T11:19:54.559 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-11-28T11:19:54.745 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-11-28T11:19:54.824 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-11-28T11:19:54.893 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T11:19:54.893 INFO:tasks.ceph:role: osd.3 2023-11-28T11:19:54.894 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi092.front.sepia.ceph.com 2023-11-28T11:19:54.894 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-28T11:19:54.971 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T11:19:54.971 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T11:19:54.971 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T11:19:54.971 INFO:teuthology.orchestra.run.smithi092.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T11:19:54.971 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T11:19:54.971 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2023-11-28T11:19:54.971 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T11:19:54.971 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T11:19:54.971 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T11:19:54.971 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T11:19:55.403 INFO:teuthology.orchestra.run.smithi092.stdout:Discarding blocks...Done. 2023-11-28T11:19:55.407 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2023-11-28T11:19:55.408 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-11-28T11:19:55.534 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-11-28T11:19:55.610 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:19:55.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:19:55.727+0000 7fea31818680 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-11-28T11:19:55.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:19:55.728+0000 7fea31818680 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-11-28T11:19:55.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:19:55.728+0000 7fea31818680 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-28T11:19:55.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:19:55.728+0000 7fea31818680 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-28T11:19:55.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:19:55.728+0000 7fea31818680 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-28T11:19:55.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:19:55.730+0000 7fea31818680 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-11-28T11:19:58.616 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-11-28T11:19:58.688 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:19:58.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:19:58.794+0000 7fb49f760680 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-11-28T11:19:58.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:19:58.794+0000 7fb49f760680 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-11-28T11:19:58.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:19:58.794+0000 7fb49f760680 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-28T11:19:58.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:19:58.794+0000 7fb49f760680 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-28T11:19:58.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:19:58.794+0000 7fb49f760680 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-28T11:19:58.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:19:58.796+0000 7fb49f760680 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-11-28T11:20:01.667 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-11-28T11:20:01.746 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:20:01.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:01.870+0000 7fdbf4ecf680 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-11-28T11:20:01.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:01.870+0000 7fdbf4ecf680 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-11-28T11:20:01.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:01.870+0000 7fdbf4ecf680 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-28T11:20:01.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:01.870+0000 7fdbf4ecf680 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-28T11:20:01.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:01.870+0000 7fdbf4ecf680 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-28T11:20:01.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:01.871+0000 7fdbf4ecf680 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-11-28T11:20:04.758 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-11-28T11:20:04.837 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:20:04.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:04.954+0000 7f91bf574680 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-11-28T11:20:04.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:04.955+0000 7f91bf574680 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-11-28T11:20:04.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:04.955+0000 7f91bf574680 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-28T11:20:04.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:04.955+0000 7f91bf574680 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-28T11:20:04.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:04.955+0000 7f91bf574680 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-28T11:20:04.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:04.957+0000 7f91bf574680 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-11-28T11:20:07.838 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-11-28T11:20:07.910 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-11-28T11:20:07.989 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T11:20:07.989 INFO:tasks.ceph:role: osd.4 2023-11-28T11:20:07.989 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi169.front.sepia.ceph.com 2023-11-28T11:20:07.989 DEBUG:teuthology.orchestra.run.smithi169:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-28T11:20:08.067 INFO:teuthology.orchestra.run.smithi169.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T11:20:08.068 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T11:20:08.068 INFO:teuthology.orchestra.run.smithi169.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T11:20:08.068 INFO:teuthology.orchestra.run.smithi169.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T11:20:08.068 INFO:teuthology.orchestra.run.smithi169.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T11:20:08.068 INFO:teuthology.orchestra.run.smithi169.stdout: = sunit=0 swidth=0 blks 2023-11-28T11:20:08.068 INFO:teuthology.orchestra.run.smithi169.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T11:20:08.068 INFO:teuthology.orchestra.run.smithi169.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T11:20:08.069 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T11:20:08.069 INFO:teuthology.orchestra.run.smithi169.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T11:20:08.506 INFO:teuthology.orchestra.run.smithi169.stdout:Discarding blocks...Done. 2023-11-28T11:20:08.509 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi169.front.sepia.ceph.com -o noatime 2023-11-28T11:20:08.510 DEBUG:teuthology.orchestra.run.smithi169:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-11-28T11:20:08.713 DEBUG:teuthology.orchestra.run.smithi169:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-11-28T11:20:08.792 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-11-28T11:20:08.862 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T11:20:08.862 INFO:tasks.ceph:role: osd.5 2023-11-28T11:20:08.863 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi169.front.sepia.ceph.com 2023-11-28T11:20:08.863 DEBUG:teuthology.orchestra.run.smithi169:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-28T11:20:08.939 INFO:teuthology.orchestra.run.smithi169.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T11:20:08.939 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T11:20:08.939 INFO:teuthology.orchestra.run.smithi169.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T11:20:08.939 INFO:teuthology.orchestra.run.smithi169.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T11:20:08.939 INFO:teuthology.orchestra.run.smithi169.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T11:20:08.940 INFO:teuthology.orchestra.run.smithi169.stdout: = sunit=0 swidth=0 blks 2023-11-28T11:20:08.940 INFO:teuthology.orchestra.run.smithi169.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T11:20:08.940 INFO:teuthology.orchestra.run.smithi169.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T11:20:08.940 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T11:20:08.940 INFO:teuthology.orchestra.run.smithi169.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T11:20:09.350 INFO:teuthology.orchestra.run.smithi169.stdout:Discarding blocks...Done. 2023-11-28T11:20:09.354 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi169.front.sepia.ceph.com -o noatime 2023-11-28T11:20:09.354 DEBUG:teuthology.orchestra.run.smithi169:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-11-28T11:20:09.501 DEBUG:teuthology.orchestra.run.smithi169:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-11-28T11:20:09.579 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-11-28T11:20:09.648 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T11:20:09.648 INFO:tasks.ceph:role: osd.6 2023-11-28T11:20:09.648 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi169.front.sepia.ceph.com 2023-11-28T11:20:09.649 DEBUG:teuthology.orchestra.run.smithi169:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-28T11:20:09.725 INFO:teuthology.orchestra.run.smithi169.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T11:20:09.725 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T11:20:09.726 INFO:teuthology.orchestra.run.smithi169.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T11:20:09.726 INFO:teuthology.orchestra.run.smithi169.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T11:20:09.726 INFO:teuthology.orchestra.run.smithi169.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T11:20:09.726 INFO:teuthology.orchestra.run.smithi169.stdout: = sunit=0 swidth=0 blks 2023-11-28T11:20:09.726 INFO:teuthology.orchestra.run.smithi169.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T11:20:09.726 INFO:teuthology.orchestra.run.smithi169.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T11:20:09.726 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T11:20:09.726 INFO:teuthology.orchestra.run.smithi169.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T11:20:10.185 INFO:teuthology.orchestra.run.smithi169.stdout:Discarding blocks...Done. 2023-11-28T11:20:10.189 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi169.front.sepia.ceph.com -o noatime 2023-11-28T11:20:10.190 DEBUG:teuthology.orchestra.run.smithi169:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-11-28T11:20:10.367 DEBUG:teuthology.orchestra.run.smithi169:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-11-28T11:20:10.446 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-11-28T11:20:10.516 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T11:20:10.517 INFO:tasks.ceph:role: osd.7 2023-11-28T11:20:10.517 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi169.front.sepia.ceph.com 2023-11-28T11:20:10.517 DEBUG:teuthology.orchestra.run.smithi169:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-28T11:20:10.593 INFO:teuthology.orchestra.run.smithi169.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T11:20:10.593 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T11:20:10.593 INFO:teuthology.orchestra.run.smithi169.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T11:20:10.593 INFO:teuthology.orchestra.run.smithi169.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T11:20:10.593 INFO:teuthology.orchestra.run.smithi169.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T11:20:10.593 INFO:teuthology.orchestra.run.smithi169.stdout: = sunit=0 swidth=0 blks 2023-11-28T11:20:10.594 INFO:teuthology.orchestra.run.smithi169.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T11:20:10.594 INFO:teuthology.orchestra.run.smithi169.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T11:20:10.594 INFO:teuthology.orchestra.run.smithi169.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T11:20:10.594 INFO:teuthology.orchestra.run.smithi169.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T11:20:11.031 INFO:teuthology.orchestra.run.smithi169.stdout:Discarding blocks...Done. 2023-11-28T11:20:11.035 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi169.front.sepia.ceph.com -o noatime 2023-11-28T11:20:11.036 DEBUG:teuthology.orchestra.run.smithi169:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-11-28T11:20:11.167 DEBUG:teuthology.orchestra.run.smithi169:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-11-28T11:20:11.243 DEBUG:teuthology.orchestra.run.smithi169:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:20:11.415 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:11.413+0000 7fcae6f85680 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-11-28T11:20:11.416 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:11.414+0000 7fcae6f85680 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-11-28T11:20:11.416 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:11.414+0000 7fcae6f85680 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-28T11:20:11.416 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:11.414+0000 7fcae6f85680 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-28T11:20:11.416 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:11.414+0000 7fcae6f85680 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-28T11:20:11.418 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:11.416+0000 7fcae6f85680 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-11-28T11:20:14.322 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-11-28T11:20:14.396 DEBUG:teuthology.orchestra.run.smithi169:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:20:14.504 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:14.502+0000 7f578f884680 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-11-28T11:20:14.504 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:14.502+0000 7f578f884680 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-11-28T11:20:14.505 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:14.502+0000 7f578f884680 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-28T11:20:14.505 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:14.502+0000 7f578f884680 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-28T11:20:14.505 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:14.503+0000 7f578f884680 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-28T11:20:14.506 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:14.504+0000 7f578f884680 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-11-28T11:20:17.402 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-11-28T11:20:17.473 DEBUG:teuthology.orchestra.run.smithi169:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:20:17.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:17.578+0000 7fba99749680 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-11-28T11:20:17.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:17.579+0000 7fba99749680 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-11-28T11:20:17.580 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:17.579+0000 7fba99749680 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-28T11:20:17.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:17.579+0000 7fba99749680 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-28T11:20:17.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:17.579+0000 7fba99749680 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-28T11:20:17.581 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:17.580+0000 7fba99749680 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-11-28T11:20:17.587 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:20:17.593 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:20:20.462 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-11-28T11:20:20.533 DEBUG:teuthology.orchestra.run.smithi169:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:20:20.640 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:20.639+0000 7f9d9bd9f680 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-11-28T11:20:20.640 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:20.639+0000 7f9d9bd9f680 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-11-28T11:20:20.640 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:20.639+0000 7f9d9bd9f680 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-28T11:20:20.640 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:20.639+0000 7f9d9bd9f680 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-28T11:20:20.640 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:20.639+0000 7f9d9bd9f680 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-28T11:20:20.641 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:20.641+0000 7f9d9bd9f680 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-11-28T11:20:23.525 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-11-28T11:20:23.597 INFO:tasks.ceph:Reading keys from all nodes... 2023-11-28T11:20:23.597 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-28T11:20:23.597 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-11-28T11:20:23.675 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-28T11:20:23.675 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-11-28T11:20:23.746 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-28T11:20:23.747 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-11-28T11:20:23.856 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-28T11:20:23.856 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-11-28T11:20:23.964 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-28T11:20:23.965 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-11-28T11:20:24.073 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-28T11:20:24.073 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-11-28T11:20:24.182 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-28T11:20:24.219 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-11-28T11:20:24.291 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-28T11:20:24.291 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-11-28T11:20:24.370 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-28T11:20:24.370 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-11-28T11:20:24.443 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-28T11:20:24.443 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-11-28T11:20:24.513 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-28T11:20:24.513 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-11-28T11:20:24.623 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-28T11:20:24.623 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-11-28T11:20:24.731 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-28T11:20:24.732 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-11-28T11:20:24.841 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-28T11:20:24.841 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-11-28T11:20:24.951 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-28T11:20:24.951 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-11-28T11:20:25.001 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-28T11:20:25.001 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-11-28T11:20:25.049 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-28T11:20:25.049 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-11-28T11:20:25.095 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-28T11:20:25.095 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-11-28T11:20:25.181 INFO:tasks.ceph:Adding keys to all mons... 2023-11-28T11:20:25.181 DEBUG:teuthology.orchestra.run.smithi092:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-28T11:20:25.186 DEBUG:teuthology.orchestra.run.smithi169:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-28T11:20:25.249 INFO:teuthology.orchestra.run.smithi092.stdout:[mgr.y] 2023-11-28T11:20:25.250 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDWzGVldLQpNBAAhdEG2ocL6T0ByZkLnwic2g== 2023-11-28T11:20:25.250 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.a] 2023-11-28T11:20:25.250 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDXzGVlH/PGCBAAor2D9Zz+LcAPQsi1BhIuQA== 2023-11-28T11:20:25.250 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.c] 2023-11-28T11:20:25.250 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDXzGVlpfloFBAAVSQxpfstP6SpWbWUMhB/hg== 2023-11-28T11:20:25.250 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.0] 2023-11-28T11:20:25.250 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDbzGVl9MVvKxAA1Pd/AN+XdW47auUMbazqdw== 2023-11-28T11:20:25.250 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.1] 2023-11-28T11:20:25.250 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDezGVldH9fLxAASVpF1+s0RIj4ObUNZoLDKg== 2023-11-28T11:20:25.251 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.2] 2023-11-28T11:20:25.251 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDhzGVl+hnoMxAAXG0K3wm2UYF3BUeGjOkHhw== 2023-11-28T11:20:25.251 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.3] 2023-11-28T11:20:25.251 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDkzGVlBB0EORAAQ3HJyXBI2AFfb/I5REmALA== 2023-11-28T11:20:25.251 INFO:teuthology.orchestra.run.smithi092.stdout:[mgr.x] 2023-11-28T11:20:25.251 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDXzGVlb4AkARAAxs1XCTncmt28cyu+vL2AYA== 2023-11-28T11:20:25.251 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.b] 2023-11-28T11:20:25.251 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDXzGVlamQ0IBAAbJVb+gWu1RqSykFqIM7Hhg== 2023-11-28T11:20:25.251 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.d] 2023-11-28T11:20:25.251 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDXzGVlqQ8ALBAAESaHrVIhfnF5ocn44xdS5A== 2023-11-28T11:20:25.251 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.4] 2023-11-28T11:20:25.251 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDrzGVlgOjDGBAAqwSISduMc7UF8VWdiSNJXQ== 2023-11-28T11:20:25.252 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.5] 2023-11-28T11:20:25.252 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDuzGVl9FMMHhAAohYzehTD1xpWI+/zxipY4Q== 2023-11-28T11:20:25.252 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.6] 2023-11-28T11:20:25.252 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDxzGVl+gOMIhAA0FwY/JqNT12aDRet8sSTYw== 2023-11-28T11:20:25.252 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.7] 2023-11-28T11:20:25.252 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQD0zGVlFPIiJhAATxpSqw+VG3u46b/IIb+yqA== 2023-11-28T11:20:25.252 INFO:teuthology.orchestra.run.smithi092.stdout:[client.0] 2023-11-28T11:20:25.252 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDXzGVl8nA7NhAAGB5c8Mp6PgErGowSH4BBdw== 2023-11-28T11:20:25.252 INFO:teuthology.orchestra.run.smithi092.stdout:[client.2] 2023-11-28T11:20:25.252 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDYzGVl1oM3AhAAdrqZFbX0CRVpZ1q531/+Dg== 2023-11-28T11:20:25.252 INFO:teuthology.orchestra.run.smithi092.stdout:[client.1] 2023-11-28T11:20:25.252 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDYzGVlbl/KCRAA23f+Q5tU6l1e3gNiomDzqQ== 2023-11-28T11:20:25.253 INFO:teuthology.orchestra.run.smithi092.stdout:[client.3] 2023-11-28T11:20:25.253 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDYzGVlGZJjERAA6iq24nyDkkyLjaWb9NkrtA== 2023-11-28T11:20:25.288 INFO:teuthology.orchestra.run.smithi169.stdout:[mgr.y] 2023-11-28T11:20:25.289 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQDWzGVldLQpNBAAhdEG2ocL6T0ByZkLnwic2g== 2023-11-28T11:20:25.289 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.a] 2023-11-28T11:20:25.289 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQDXzGVlH/PGCBAAor2D9Zz+LcAPQsi1BhIuQA== 2023-11-28T11:20:25.289 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.c] 2023-11-28T11:20:25.289 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQDXzGVlpfloFBAAVSQxpfstP6SpWbWUMhB/hg== 2023-11-28T11:20:25.289 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.0] 2023-11-28T11:20:25.289 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQDbzGVl9MVvKxAA1Pd/AN+XdW47auUMbazqdw== 2023-11-28T11:20:25.289 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.1] 2023-11-28T11:20:25.289 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQDezGVldH9fLxAASVpF1+s0RIj4ObUNZoLDKg== 2023-11-28T11:20:25.289 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.2] 2023-11-28T11:20:25.289 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQDhzGVl+hnoMxAAXG0K3wm2UYF3BUeGjOkHhw== 2023-11-28T11:20:25.290 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.3] 2023-11-28T11:20:25.290 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQDkzGVlBB0EORAAQ3HJyXBI2AFfb/I5REmALA== 2023-11-28T11:20:25.290 INFO:teuthology.orchestra.run.smithi169.stdout:[mgr.x] 2023-11-28T11:20:25.290 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQDXzGVlb4AkARAAxs1XCTncmt28cyu+vL2AYA== 2023-11-28T11:20:25.290 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.b] 2023-11-28T11:20:25.290 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQDXzGVlamQ0IBAAbJVb+gWu1RqSykFqIM7Hhg== 2023-11-28T11:20:25.290 INFO:teuthology.orchestra.run.smithi169.stdout:[mds.d] 2023-11-28T11:20:25.290 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQDXzGVlqQ8ALBAAESaHrVIhfnF5ocn44xdS5A== 2023-11-28T11:20:25.290 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.4] 2023-11-28T11:20:25.290 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQDrzGVlgOjDGBAAqwSISduMc7UF8VWdiSNJXQ== 2023-11-28T11:20:25.290 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.5] 2023-11-28T11:20:25.290 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQDuzGVl9FMMHhAAohYzehTD1xpWI+/zxipY4Q== 2023-11-28T11:20:25.291 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.6] 2023-11-28T11:20:25.291 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQDxzGVl+gOMIhAA0FwY/JqNT12aDRet8sSTYw== 2023-11-28T11:20:25.291 INFO:teuthology.orchestra.run.smithi169.stdout:[osd.7] 2023-11-28T11:20:25.291 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQD0zGVlFPIiJhAATxpSqw+VG3u46b/IIb+yqA== 2023-11-28T11:20:25.291 INFO:teuthology.orchestra.run.smithi169.stdout:[client.0] 2023-11-28T11:20:25.291 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQDXzGVl8nA7NhAAGB5c8Mp6PgErGowSH4BBdw== 2023-11-28T11:20:25.291 INFO:teuthology.orchestra.run.smithi169.stdout:[client.2] 2023-11-28T11:20:25.291 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQDYzGVl1oM3AhAAdrqZFbX0CRVpZ1q531/+Dg== 2023-11-28T11:20:25.291 INFO:teuthology.orchestra.run.smithi169.stdout:[client.1] 2023-11-28T11:20:25.291 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQDYzGVlbl/KCRAA23f+Q5tU6l1e3gNiomDzqQ== 2023-11-28T11:20:25.291 INFO:teuthology.orchestra.run.smithi169.stdout:[client.3] 2023-11-28T11:20:25.291 INFO:teuthology.orchestra.run.smithi169.stdout: key = AQDYzGVlGZJjERAA6iq24nyDkkyLjaWb9NkrtA== 2023-11-28T11:20:25.292 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-28T11:20:25.298 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-28T11:20:25.442 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T11:20:25.447 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T11:20:25.560 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T11:20:25.564 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T11:20:25.677 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:20:25.682 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:20:25.791 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:20:25.797 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:20:25.908 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:20:25.913 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:20:26.023 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:20:26.028 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:20:26.139 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-28T11:20:26.143 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-28T11:20:26.252 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T11:20:26.256 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T11:20:26.370 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T11:20:26.375 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T11:20:26.488 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:20:26.492 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:20:26.602 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:20:26.606 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:20:26.716 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:20:26.721 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:20:26.830 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:20:26.835 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:20:26.943 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T11:20:26.948 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T11:20:27.062 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T11:20:27.067 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T11:20:27.181 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T11:20:27.186 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T11:20:27.299 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T11:20:27.303 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T11:20:27.416 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-11-28T11:20:27.416 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-11-28T11:20:27.485 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-28T11:20:27.609 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-11-28T11:20:27.679 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-11-28T11:20:27.751 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-28T11:20:27.875 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-11-28T11:20:27.946 DEBUG:teuthology.orchestra.run.smithi169:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-11-28T11:20:28.016 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-28T11:20:28.142 DEBUG:teuthology.orchestra.run.smithi169:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-11-28T11:20:28.212 DEBUG:teuthology.orchestra.run.smithi092:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:20:28.217 DEBUG:teuthology.orchestra.run.smithi169:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:20:28.264 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-11-28T11:20:28.264 INFO:tasks.ceph.mon.a:Restarting daemon 2023-11-28T11:20:28.264 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-11-28T11:20:28.305 INFO:tasks.ceph.mon.a:Started 2023-11-28T11:20:28.305 INFO:tasks.ceph.mon.b:Restarting daemon 2023-11-28T11:20:28.306 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-11-28T11:20:28.310 INFO:tasks.ceph.mon.b:Started 2023-11-28T11:20:28.310 INFO:tasks.ceph.mon.c:Restarting daemon 2023-11-28T11:20:28.311 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-11-28T11:20:28.315 INFO:tasks.ceph.mon.c:Started 2023-11-28T11:20:28.315 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-11-28T11:20:28.315 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-11-28T11:20:28.315 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-11-28T11:20:28.320 INFO:tasks.ceph.mgr.y:Started 2023-11-28T11:20:28.320 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-11-28T11:20:28.320 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-11-28T11:20:28.325 INFO:tasks.ceph.mgr.x:Started 2023-11-28T11:20:28.325 INFO:tasks.ceph:Setting crush tunables to default 2023-11-28T11:20:28.325 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd crush tunables default 2023-11-28T11:20:28.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.523+0000 7ff7fa3ff700 1 Processor -- start 2023-11-28T11:20:28.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.523+0000 7ff7fa3ff700 1 -- start start 2023-11-28T11:20:28.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.524+0000 7ff7fa3ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff7f40d1d70 0x7ff7f4167270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:28.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.524+0000 7ff7fa3ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7f406c790 con 0x7ff7f40cd460 2023-11-28T11:20:28.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.524+0000 7ff7fa3ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7f40b88e0 con 0x7ff7f40b8510 2023-11-28T11:20:28.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.524+0000 7ff7fa3ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7f40cdbd0 con 0x7ff7f40d1d70 2023-11-28T11:20:28.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.525+0000 7ff7f3fff700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7ff7f40b8510 0x7ff7f406b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.92:50320/0 (socket says 172.21.15.92:50320) 2023-11-28T11:20:28.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.525+0000 7ff7f899c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff7f40d1d70 0x7ff7f4167270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:28.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.525+0000 7ff7f3fff700 1 -- 172.21.15.92:0/3728748543 learned_addr learned my addr 172.21.15.92:0/3728748543 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:20:28.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.525+0000 7ff7f899c700 1 -- 172.21.15.92:0/3728748543 >> v2:172.21.15.92:3300/0 conn(0x7ff7f40d1d70 msgr2=0x7ff7f4167270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-11-28T11:20:28.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.525+0000 7ff7f899c700 1 -- 172.21.15.92:0/3728748543 >> v2:172.21.15.92:3300/0 conn(0x7ff7f40d1d70 msgr2=0x7ff7f4167270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-28T11:20:28.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.525+0000 7ff7f899c700 1 --2- 172.21.15.92:0/3728748543 >> v2:172.21.15.92:3300/0 conn(0x7ff7f40d1d70 0x7ff7f4167270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T11:20:28.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.525+0000 7ff7f899c700 1 --2- 172.21.15.92:0/3728748543 >> v2:172.21.15.92:3300/0 conn(0x7ff7f40d1d70 0x7ff7f4167270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T11:20:28.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.714+0000 7ff7f2ffd700 1 -- 172.21.15.92:0/3728748543 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2168516499 0 0) 0x7ff7f406c790 con 0x7ff7f40cd460 2023-11-28T11:20:28.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.714+0000 7ff7f2ffd700 1 -- 172.21.15.92:0/3728748543 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7dc0035b0 con 0x7ff7f40cd460 2023-11-28T11:20:28.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.714+0000 7ff7f2ffd700 1 -- 172.21.15.92:0/3728748543 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2412349391 0 0) 0x7ff7dc0035b0 con 0x7ff7f40cd460 2023-11-28T11:20:28.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.714+0000 7ff7f2ffd700 1 -- 172.21.15.92:0/3728748543 >> v2:172.21.15.92:3300/0 conn(0x7ff7f40d1d70 msgr2=0x7ff7f4167270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T11:20:28.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.714+0000 7ff7f2ffd700 1 --2- 172.21.15.92:0/3728748543 >> v2:172.21.15.92:3300/0 conn(0x7ff7f40d1d70 0x7ff7f4167270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:28.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.714+0000 7ff7f2ffd700 1 -- 172.21.15.92:0/3728748543 >> v1:172.21.15.169:6789/0 conn(0x7ff7f40b8510 legacy=0x7ff7f406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:28.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.714+0000 7ff7f2ffd700 1 -- 172.21.15.92:0/3728748543 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f40cd8f0 con 0x7ff7f40cd460 2023-11-28T11:20:28.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.714+0000 7ff7f2ffd700 1 -- 172.21.15.92:0/3728748543 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532161587 0 0) 0x7ff7e8001670 con 0x7ff7f40cd460 2023-11-28T11:20:28.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.714+0000 7ff7f2ffd700 1 -- 172.21.15.92:0/3728748543 >> v1:172.21.15.92:6789/0 conn(0x7ff7f40cd460 legacy=0x7ff7f406c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:28.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.715+0000 7ff7f2ffd700 1 --2- 172.21.15.92:0/3728748543 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7dc003a50 0x7ff7dc003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:28.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.715+0000 7ff7f2ffd700 1 --2- 172.21.15.92:0/3728748543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7dc004850 0x7ff7dc004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:28.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.715+0000 7ff7f2ffd700 1 --2- 172.21.15.92:0/3728748543 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7dc005250 0x7ff7dc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:28.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.715+0000 7ff7f2ffd700 1 -- 172.21.15.92:0/3728748543 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f40cdbd0 con 0x7ff7dc005250 2023-11-28T11:20:28.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.715+0000 7ff7f2ffd700 1 -- 172.21.15.92:0/3728748543 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7dc00c190 con 0x7ff7dc004850 2023-11-28T11:20:28.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.715+0000 7ff7f2ffd700 1 -- 172.21.15.92:0/3728748543 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7dc00c350 con 0x7ff7dc003a50 2023-11-28T11:20:28.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.715+0000 7ff7f3fff700 1 --2- 172.21.15.92:0/3728748543 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7dc003a50 0x7ff7dc003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:28.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.715+0000 7ff7f899c700 1 --2- 172.21.15.92:0/3728748543 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7dc005250 0x7ff7dc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:28.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.715+0000 7ff7f37fe700 1 --2- 172.21.15.92:0/3728748543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7dc004850 0x7ff7dc004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:28.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.715+0000 7ff7f899c700 1 -- 172.21.15.92:0/3728748543 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7dc003a50 msgr2=0x7ff7dc003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:28.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.715+0000 7ff7f899c700 1 --2- 172.21.15.92:0/3728748543 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7dc003a50 0x7ff7dc003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:28.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.715+0000 7ff7f899c700 1 -- 172.21.15.92:0/3728748543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7dc004850 msgr2=0x7ff7dc004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:28.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.715+0000 7ff7f899c700 1 --2- 172.21.15.92:0/3728748543 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7dc004850 0x7ff7dc004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:28.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.715+0000 7ff7f899c700 1 -- 172.21.15.92:0/3728748543 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7dc0035b0 con 0x7ff7dc005250 2023-11-28T11:20:28.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.715+0000 7ff7f899c700 1 -- 172.21.15.92:0/3728748543 reap_dead start 2023-11-28T11:20:28.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.716+0000 7ff7f899c700 1 --2- 172.21.15.92:0/3728748543 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7dc005250 0x7ff7dc00ba10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc005830 tx=0x7ff7dc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19e7f38b2ead250d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:28.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.716+0000 7ff7f2ffd700 1 -- 172.21.15.92:0/3728748543 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec007df0 con 0x7ff7dc005250 2023-11-28T11:20:28.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.716+0000 7ff7f2ffd700 1 -- 172.21.15.92:0/3728748543 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7ec003a20 con 0x7ff7dc005250 2023-11-28T11:20:28.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.716+0000 7ff7f2ffd700 1 -- 172.21.15.92:0/3728748543 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec003dd0 con 0x7ff7dc005250 2023-11-28T11:20:28.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.717+0000 7ff7fa3ff700 1 -- 172.21.15.92:0/3728748543 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7dc005250 msgr2=0x7ff7dc00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:28.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.717+0000 7ff7fa3ff700 1 --2- 172.21.15.92:0/3728748543 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7dc005250 0x7ff7dc00ba10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc005830 tx=0x7ff7dc005880 comp rx=0 tx=0).stop 2023-11-28T11:20:28.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.717+0000 7ff7fa3ff700 1 -- 172.21.15.92:0/3728748543 shutdown_connections 2023-11-28T11:20:28.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.717+0000 7ff7fa3ff700 1 --2- 172.21.15.92:0/3728748543 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7dc005250 0x7ff7dc00ba10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:28.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.717+0000 7ff7fa3ff700 1 -- 172.21.15.92:0/3728748543 >> 172.21.15.92:0/3728748543 conn(0x7ff7f40719a0 msgr2=0x7ff7f4071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:28.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.717+0000 7ff7fa3ff700 1 -- 172.21.15.92:0/3728748543 shutdown_connections 2023-11-28T11:20:28.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.717+0000 7ff7fa3ff700 1 -- 172.21.15.92:0/3728748543 wait complete. 2023-11-28T11:20:28.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.719+0000 7ff7fa3ff700 1 Processor -- start 2023-11-28T11:20:28.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.719+0000 7ff7fa3ff700 1 -- start start 2023-11-28T11:20:28.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.720+0000 7ff7fa3ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7dc005250 0x7ff7f4157a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:28.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.720+0000 7ff7fa3ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7f41587d0 0x7ff7f41543c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:28.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.720+0000 7ff7fa3ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7f4159fe0 0x7ff7f40b6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:28.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.720+0000 7ff7fa3ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f4154900 con 0x7ff7f4159fe0 2023-11-28T11:20:28.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.720+0000 7ff7fa3ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f4158f40 con 0x7ff7f41587d0 2023-11-28T11:20:28.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.720+0000 7ff7fa3ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f4159220 con 0x7ff7dc005250 2023-11-28T11:20:28.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.721+0000 7ff7f899c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7f4159fe0 0x7ff7f40b6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:28.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.721+0000 7ff7f899c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7f4159fe0 0x7ff7f40b6b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41512/0 (socket says 172.21.15.92:41512) 2023-11-28T11:20:28.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.721+0000 7ff7f899c700 1 -- 172.21.15.92:0/4189654146 learned_addr learned my addr 172.21.15.92:0/4189654146 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:20:28.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.721+0000 7ff7f3fff700 1 --2- 172.21.15.92:0/4189654146 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7dc005250 0x7ff7f4157a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:28.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.721+0000 7ff7f37fe700 1 --2- 172.21.15.92:0/4189654146 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7f41587d0 0x7ff7f41543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:28.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.721+0000 7ff7f899c700 1 -- 172.21.15.92:0/4189654146 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7dc005250 msgr2=0x7ff7f4157a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:28.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.721+0000 7ff7f899c700 1 --2- 172.21.15.92:0/4189654146 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7dc005250 0x7ff7f4157a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:28.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.721+0000 7ff7f899c700 1 -- 172.21.15.92:0/4189654146 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7f41587d0 msgr2=0x7ff7f41543c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:28.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.721+0000 7ff7f899c700 1 --2- 172.21.15.92:0/4189654146 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7f41587d0 0x7ff7f41543c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:28.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.721+0000 7ff7f899c700 1 -- 172.21.15.92:0/4189654146 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f40cfff0 con 0x7ff7f4159fe0 2023-11-28T11:20:28.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.721+0000 7ff7f3fff700 1 --2- 172.21.15.92:0/4189654146 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7dc005250 0x7ff7f4157a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:20:28.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.721+0000 7ff7f899c700 1 --2- 172.21.15.92:0/4189654146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7f4159fe0 0x7ff7f40b6b50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff7f40ce5e0 tx=0x7ff7ec004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:28.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.722+0000 7ff7f0ff9700 1 -- 172.21.15.92:0/4189654146 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec014d70 con 0x7ff7f4159fe0 2023-11-28T11:20:28.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.722+0000 7ff7f0ff9700 1 -- 172.21.15.92:0/4189654146 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7ec01d470 con 0x7ff7f4159fe0 2023-11-28T11:20:28.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.722+0000 7ff7fa3ff700 1 -- 172.21.15.92:0/4189654146 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7f40cfd10 con 0x7ff7f4159fe0 2023-11-28T11:20:28.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.722+0000 7ff7f0ff9700 1 -- 172.21.15.92:0/4189654146 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec01d860 con 0x7ff7f4159fe0 2023-11-28T11:20:28.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.722+0000 7ff7fa3ff700 1 -- 172.21.15.92:0/4189654146 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7f40d29c0 con 0x7ff7f4159fe0 2023-11-28T11:20:28.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.722+0000 7ff7f0ff9700 1 -- 172.21.15.92:0/4189654146 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7ff7ec012070 con 0x7ff7f4159fe0 2023-11-28T11:20:28.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.722+0000 7ff7f0ff9700 1 -- 172.21.15.92:0/4189654146 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff7ec029910 con 0x7ff7f4159fe0 2023-11-28T11:20:28.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.723+0000 7ff7fa3ff700 1 -- 172.21.15.92:0/4189654146 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7f4067880 con 0x7ff7f4159fe0 2023-11-28T11:20:28.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.727+0000 7ff7f0ff9700 1 -- 172.21.15.92:0/4189654146 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7ff7ec009bd0 con 0x7ff7f4159fe0 2023-11-28T11:20:28.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.805+0000 7ff7fa3ff700 1 -- 172.21.15.92:0/4189654146 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7ff7f40b7e60 con 0x7ff7f4159fe0 2023-11-28T11:20:28.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.811+0000 7ff7f0ff9700 1 -- 172.21.15.92:0/4189654146 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7ff7ec029460 con 0x7ff7f4159fe0 2023-11-28T11:20:28.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.812+0000 7ff7fa3ff700 1 -- 172.21.15.92:0/4189654146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7f4159fe0 msgr2=0x7ff7f40b6b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:28.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.812+0000 7ff7fa3ff700 1 --2- 172.21.15.92:0/4189654146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7f4159fe0 0x7ff7f40b6b50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff7f40ce5e0 tx=0x7ff7ec004b40 comp rx=0 tx=0).stop 2023-11-28T11:20:28.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.813+0000 7ff7fa3ff700 1 -- 172.21.15.92:0/4189654146 shutdown_connections 2023-11-28T11:20:28.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.813+0000 7ff7fa3ff700 1 --2- 172.21.15.92:0/4189654146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7f4159fe0 0x7ff7f40b6b50 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:28.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.813+0000 7ff7fa3ff700 1 --2- 172.21.15.92:0/4189654146 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7dc005250 0x7ff7f4157a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:28.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.813+0000 7ff7fa3ff700 1 --2- 172.21.15.92:0/4189654146 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7f41587d0 0x7ff7f41543c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:28.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.813+0000 7ff7fa3ff700 1 -- 172.21.15.92:0/4189654146 >> 172.21.15.92:0/4189654146 conn(0x7ff7f40719a0 msgr2=0x7ff7f415c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:28.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.813+0000 7ff7fa3ff700 1 -- 172.21.15.92:0/4189654146 shutdown_connections 2023-11-28T11:20:28.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:28.813+0000 7ff7fa3ff700 1 -- 172.21.15.92:0/4189654146 wait complete. 2023-11-28T11:20:28.815 INFO:teuthology.orchestra.run.smithi092.stderr:adjusted tunables profile to default 2023-11-28T11:20:28.831 INFO:tasks.ceph:check_enable_crimson: False 2023-11-28T11:20:28.831 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-11-28T11:20:28.831 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-28T11:20:28.832 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-11-28T11:20:28.877 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-11-28T11:20:28.877+0000 7f2166e55240 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T11:20:28.886 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-28T11:20:28.885+0000 7f8f2f6df240 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T11:20:28.900 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-28T11:20:28.900 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-11-28T11:20:29.017 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-28T11:20:29.017 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-11-28T11:20:29.096 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-11-28T11:20:29.095+0000 7f2166e55240 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T11:20:29.104 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-28T11:20:29.104+0000 7f8f2f6df240 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T11:20:29.137 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-28T11:20:29.137 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-11-28T11:20:29.246 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-28T11:20:29.246 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-11-28T11:20:29.319 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-28T11:20:29.319 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-11-28T11:20:29.430 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-28T11:20:29.430 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-11-28T11:20:29.493 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-11-28T11:20:29.492+0000 7f2166e55240 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T11:20:29.509 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-28T11:20:29.508+0000 7f8f2f6df240 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T11:20:29.540 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-28T11:20:29.540 DEBUG:teuthology.orchestra.run.smithi169:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-11-28T11:20:29.650 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new 6abe483f-0e03-4a26-b9cf-62041291747e 0 2023-11-28T11:20:29.887 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.886+0000 7f4f035ff700 1 Processor -- start 2023-11-28T11:20:29.887 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.887+0000 7f4f035ff700 1 -- start start 2023-11-28T11:20:29.889 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.888+0000 7f4f035ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4efc0b1c60 0x7f4efc0cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:29.889 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.888+0000 7f4f035ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4efc0d1950 con 0x7f4efc0b4fc0 2023-11-28T11:20:29.889 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.888+0000 7f4f035ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4efc0b2500 con 0x7f4efc0d2df0 2023-11-28T11:20:29.889 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.888+0000 7f4f035ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4efc0d3d30 con 0x7f4efc0b1c60 2023-11-28T11:20:29.889 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.888+0000 7f4f0139b700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f4efc0d2df0 0x7f4efc0d31c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.169:35842/0 (socket says 172.21.15.169:35842) 2023-11-28T11:20:29.889 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.888+0000 7f4f0139b700 1 -- 172.21.15.169:0/1728826338 learned_addr learned my addr 172.21.15.169:0/1728826338 (peer_addr_for_me v1:172.21.15.169:0/0) 2023-11-28T11:20:29.889 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.888+0000 7f4f01b9c700 1 --2- 172.21.15.169:0/1728826338 >> v2:172.21.15.92:3300/0 conn(0x7f4efc0b1c60 0x7f4efc0cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:29.891 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.888+0000 7f4ef3fff700 1 -- 172.21.15.169:0/1728826338 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3443346682 0 0) 0x7f4efc0b2500 con 0x7f4efc0d2df0 2023-11-28T11:20:29.891 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.889+0000 7f4ef3fff700 1 -- 172.21.15.169:0/1728826338 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ee80035b0 con 0x7f4efc0d2df0 2023-11-28T11:20:29.891 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.889+0000 7f4f01b9c700 1 -- 172.21.15.169:0/1728826338 >> v1:172.21.15.169:6789/0 conn(0x7f4efc0d2df0 legacy=0x7f4efc0d31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:29.891 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.889+0000 7f4f01b9c700 1 -- 172.21.15.169:0/1728826338 >> v1:172.21.15.92:6789/0 conn(0x7f4efc0b4fc0 legacy=0x7f4efc0cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:29.891 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.889+0000 7f4f01b9c700 1 -- 172.21.15.169:0/1728826338 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4efc0d3a50 con 0x7f4efc0b1c60 2023-11-28T11:20:29.891 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.889+0000 7f4f01b9c700 1 --2- 172.21.15.169:0/1728826338 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4efc0b1c60 0x7f4efc0cd620 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4ef4009d90 tx=0x7f4ef400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe63b2e40d607aa1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:29.891 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.889+0000 7f4ef3fff700 1 -- 172.21.15.169:0/1728826338 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef400e070 con 0x7f4efc0b1c60 2023-11-28T11:20:29.891 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.889+0000 7f4ef3fff700 1 -- 172.21.15.169:0/1728826338 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ef400bb20 con 0x7f4efc0b1c60 2023-11-28T11:20:29.891 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.889+0000 7f4ef3fff700 1 -- 172.21.15.169:0/1728826338 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef4012430 con 0x7f4efc0b1c60 2023-11-28T11:20:29.891 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.890+0000 7f4f035ff700 1 -- 172.21.15.169:0/1728826338 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4efc0b1c60 msgr2=0x7f4efc0cd620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:29.891 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.890+0000 7f4f035ff700 1 --2- 172.21.15.169:0/1728826338 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4efc0b1c60 0x7f4efc0cd620 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4ef4009d90 tx=0x7f4ef400b330 comp rx=0 tx=0).stop 2023-11-28T11:20:29.892 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.890+0000 7f4f035ff700 1 -- 172.21.15.169:0/1728826338 shutdown_connections 2023-11-28T11:20:29.892 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.890+0000 7f4f035ff700 1 --2- 172.21.15.169:0/1728826338 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4efc0b1c60 0x7f4efc0cd620 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:29.892 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.890+0000 7f4f035ff700 1 -- 172.21.15.169:0/1728826338 >> 172.21.15.169:0/1728826338 conn(0x7f4efc0719a0 msgr2=0x7f4efc071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:29.892 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.890+0000 7f4f035ff700 1 -- 172.21.15.169:0/1728826338 shutdown_connections 2023-11-28T11:20:29.892 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.890+0000 7f4f035ff700 1 -- 172.21.15.169:0/1728826338 wait complete. 2023-11-28T11:20:29.892 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.892+0000 7f4f035ff700 1 Processor -- start 2023-11-28T11:20:29.892 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.892+0000 7f4f035ff700 1 -- start start 2023-11-28T11:20:29.895 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.893+0000 7f4f035ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4efc0b1c60 0x7f4efc159f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:29.895 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.893+0000 7f4f035ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4efc0b4fc0 0x7f4efc153ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:29.895 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.893+0000 7f4f035ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4efc0d2df0 0x7f4efc157a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:29.895 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.893+0000 7f4f035ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4efc157f80 con 0x7f4efc0b4fc0 2023-11-28T11:20:29.895 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.893+0000 7f4f035ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4efc15dfa0 con 0x7f4efc0b1c60 2023-11-28T11:20:29.895 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.893+0000 7f4f035ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4efc1581b0 con 0x7f4efc0d2df0 2023-11-28T11:20:29.895 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.893+0000 7f4f0139b700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4efc0b1c60 0x7f4efc159f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:29.895 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.893+0000 7f4f0139b700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4efc0b1c60 0x7f4efc159f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:56498/0 (socket says 172.21.15.169:56498) 2023-11-28T11:20:29.895 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.893+0000 7f4f0139b700 1 -- 172.21.15.169:0/785383744 learned_addr learned my addr 172.21.15.169:0/785383744 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-11-28T11:20:29.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.893+0000 7f4f01b9c700 1 --2- 172.21.15.169:0/785383744 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4efc0d2df0 0x7f4efc157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:29.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.893+0000 7f4f00b9a700 1 --2- 172.21.15.169:0/785383744 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4efc0b4fc0 0x7f4efc153ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:29.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.894+0000 7f4f01b9c700 1 -- 172.21.15.169:0/785383744 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4efc0b1c60 msgr2=0x7f4efc159f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:29.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.894+0000 7f4f01b9c700 1 --2- 172.21.15.169:0/785383744 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4efc0b1c60 0x7f4efc159f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:29.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.894+0000 7f4f01b9c700 1 -- 172.21.15.169:0/785383744 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4efc0b4fc0 msgr2=0x7f4efc153ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:29.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.894+0000 7f4f01b9c700 1 --2- 172.21.15.169:0/785383744 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4efc0b4fc0 0x7f4efc153ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:29.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.894+0000 7f4f01b9c700 1 -- 172.21.15.169:0/785383744 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef4009a70 con 0x7f4efc0d2df0 2023-11-28T11:20:29.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.894+0000 7f4f0139b700 1 --2- 172.21.15.169:0/785383744 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4efc0b1c60 0x7f4efc159f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:20:29.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.894+0000 7f4f01b9c700 1 --2- 172.21.15.169:0/785383744 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4efc0d2df0 0x7f4efc157a40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4ef4016000 tx=0x7f4ef4004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:29.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.894+0000 7f4ef1ffb700 1 -- 172.21.15.169:0/785383744 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef4004880 con 0x7f4efc0d2df0 2023-11-28T11:20:29.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.894+0000 7f4ef1ffb700 1 -- 172.21.15.169:0/785383744 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ef4020070 con 0x7f4efc0d2df0 2023-11-28T11:20:29.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.894+0000 7f4f035ff700 1 -- 172.21.15.169:0/785383744 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4efc0b88f0 con 0x7f4efc0d2df0 2023-11-28T11:20:29.896 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.894+0000 7f4ef1ffb700 1 -- 172.21.15.169:0/785383744 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef40276c0 con 0x7f4efc0d2df0 2023-11-28T11:20:29.897 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.894+0000 7f4f035ff700 1 -- 172.21.15.169:0/785383744 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4efc0d15e0 con 0x7f4efc0d2df0 2023-11-28T11:20:29.897 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.895+0000 7f4ef1ffb700 1 -- 172.21.15.169:0/785383744 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f4ef402e070 con 0x7f4efc0d2df0 2023-11-28T11:20:29.897 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.895+0000 7f4ef1ffb700 1 -- 172.21.15.169:0/785383744 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4ef4022070 con 0x7f4efc0d2df0 2023-11-28T11:20:29.897 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.895+0000 7f4f035ff700 1 -- 172.21.15.169:0/785383744 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4efc0b21b0 con 0x7f4efc0d2df0 2023-11-28T11:20:29.899 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.898+0000 7f4ef1ffb700 1 -- 172.21.15.169:0/785383744 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f4ef402c060 con 0x7f4efc0d2df0 2023-11-28T11:20:29.986 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.986+0000 7f4f035ff700 1 -- 172.21.15.169:0/785383744 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd new", "uuid": "6abe483f-0e03-4a26-b9cf-62041291747e", "id": 0} v 0) v1 -- 0x7f4efc067170 con 0x7f4efc0d2df0 2023-11-28T11:20:29.993 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.992+0000 7f4ef1ffb700 1 -- 172.21.15.169:0/785383744 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6abe483f-0e03-4a26-b9cf-62041291747e", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4ef4027cf0 con 0x7f4efc0d2df0 2023-11-28T11:20:29.993 INFO:teuthology.orchestra.run.smithi169.stdout:0 2023-11-28T11:20:29.994 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.993+0000 7f4f035ff700 1 -- 172.21.15.169:0/785383744 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4efc0d2df0 msgr2=0x7f4efc157a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:29.994 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.993+0000 7f4f035ff700 1 --2- 172.21.15.169:0/785383744 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4efc0d2df0 0x7f4efc157a40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4ef4016000 tx=0x7f4ef4004ad0 comp rx=0 tx=0).stop 2023-11-28T11:20:29.994 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.993+0000 7f4f035ff700 1 -- 172.21.15.169:0/785383744 shutdown_connections 2023-11-28T11:20:29.995 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.993+0000 7f4f035ff700 1 --2- 172.21.15.169:0/785383744 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4efc0d2df0 0x7f4efc157a40 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:29.995 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.993+0000 7f4f035ff700 1 --2- 172.21.15.169:0/785383744 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4efc0b1c60 0x7f4efc159f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:29.995 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.993+0000 7f4f035ff700 1 --2- 172.21.15.169:0/785383744 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4efc0b4fc0 0x7f4efc153ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:29.995 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.993+0000 7f4f035ff700 1 -- 172.21.15.169:0/785383744 >> 172.21.15.169:0/785383744 conn(0x7f4efc0719a0 msgr2=0x7f4efc0d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:29.995 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.993+0000 7f4f035ff700 1 -- 172.21.15.169:0/785383744 shutdown_connections 2023-11-28T11:20:29.995 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:29.993+0000 7f4f035ff700 1 -- 172.21.15.169:0/785383744 wait complete. 2023-11-28T11:20:30.012 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new 13c1de73-1a8c-4d7e-8a28-9dae52dfe089 1 2023-11-28T11:20:30.125 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-28T11:20:30.124+0000 7f8f2f6df240 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T11:20:30.166 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-11-28T11:20:30.165+0000 7f2166e55240 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T11:20:30.213 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.212+0000 7f231d5ff700 1 Processor -- start 2023-11-28T11:20:30.213 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.212+0000 7f231d5ff700 1 -- start start 2023-11-28T11:20:30.214 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.213+0000 7f231d5ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f23180b8510 0x7f231806b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:30.214 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.213+0000 7f231d5ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f23180d0d60 0x7f23180d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:30.215 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.213+0000 7f231d5ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23180b88e0 con 0x7f23180cd460 2023-11-28T11:20:30.215 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.213+0000 7f231d5ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23180cdbd0 con 0x7f23180d0d60 2023-11-28T11:20:30.215 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.213+0000 7f231d5ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23180d02d0 con 0x7f23180b8510 2023-11-28T11:20:30.215 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.214+0000 7f2316ffd700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f23180b8510 0x7f231806b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:30.215 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.214+0000 7f2316ffd700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f23180b8510 0x7f231806b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:56500/0 (socket says 172.21.15.169:56500) 2023-11-28T11:20:30.215 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.214+0000 7f2316ffd700 1 -- 172.21.15.169:0/1758237948 learned_addr learned my addr 172.21.15.169:0/1758237948 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-11-28T11:20:30.215 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.214+0000 7f23177fe700 1 --2- 172.21.15.169:0/1758237948 >> v2:172.21.15.92:3300/0 conn(0x7f23180d0d60 0x7f23180d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:30.216 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.214+0000 7f2316ffd700 1 -- 172.21.15.169:0/1758237948 >> v2:172.21.15.92:3300/0 conn(0x7f23180d0d60 msgr2=0x7f23180d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:30.216 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.214+0000 7f2316ffd700 1 --2- 172.21.15.169:0/1758237948 >> v2:172.21.15.92:3300/0 conn(0x7f23180d0d60 0x7f23180d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:30.217 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.214+0000 7f2316ffd700 1 -- 172.21.15.169:0/1758237948 >> v1:172.21.15.92:6789/0 conn(0x7f23180cd460 legacy=0x7f231806beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:30.217 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.214+0000 7f2316ffd700 1 -- 172.21.15.169:0/1758237948 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23180cd8f0 con 0x7f23180b8510 2023-11-28T11:20:30.217 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.214+0000 7f2316ffd700 1 --2- 172.21.15.169:0/1758237948 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23180b8510 0x7f231806b970 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f23180b5a50 tx=0x7f2308009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a5f2c5bb336f7d5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:30.217 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.215+0000 7f23167fc700 1 -- 172.21.15.169:0/1758237948 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2308011830 con 0x7f23180b8510 2023-11-28T11:20:30.217 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.215+0000 7f23167fc700 1 -- 172.21.15.169:0/1758237948 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2308011990 con 0x7f23180b8510 2023-11-28T11:20:30.217 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.215+0000 7f23167fc700 1 -- 172.21.15.169:0/1758237948 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2308011d20 con 0x7f23180b8510 2023-11-28T11:20:30.217 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.215+0000 7f231d5ff700 1 -- 172.21.15.169:0/1758237948 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23180b8510 msgr2=0x7f231806b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:30.217 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.215+0000 7f231d5ff700 1 --2- 172.21.15.169:0/1758237948 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23180b8510 0x7f231806b970 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f23180b5a50 tx=0x7f2308009bd0 comp rx=0 tx=0).stop 2023-11-28T11:20:30.217 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.215+0000 7f231d5ff700 1 -- 172.21.15.169:0/1758237948 shutdown_connections 2023-11-28T11:20:30.217 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.215+0000 7f231d5ff700 1 --2- 172.21.15.169:0/1758237948 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23180b8510 0x7f231806b970 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:30.217 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.215+0000 7f231d5ff700 1 --2- 172.21.15.169:0/1758237948 >> v2:172.21.15.92:3300/0 conn(0x7f23180d0d60 0x7f23180d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:30.217 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.215+0000 7f231d5ff700 1 -- 172.21.15.169:0/1758237948 >> 172.21.15.169:0/1758237948 conn(0x7f23180719a0 msgr2=0x7f2318071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:30.218 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.215+0000 7f231d5ff700 1 -- 172.21.15.169:0/1758237948 shutdown_connections 2023-11-28T11:20:30.218 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.216+0000 7f231d5ff700 1 -- 172.21.15.169:0/1758237948 wait complete. 2023-11-28T11:20:30.221 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.217+0000 7f231d5ff700 1 Processor -- start 2023-11-28T11:20:30.221 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.217+0000 7f231d5ff700 1 -- start start 2023-11-28T11:20:30.221 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.218+0000 7f231d5ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23180b8510 0x7f23181e1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:30.221 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.218+0000 7f231d5ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23180cd460 0x7f23181561b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:30.221 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.218+0000 7f231d5ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23180d0d60 0x7f23181e2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:30.221 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.218+0000 7f231d5ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23181e2bd0 con 0x7f23180b8510 2023-11-28T11:20:30.221 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.218+0000 7f231d5ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23181566f0 con 0x7f23180cd460 2023-11-28T11:20:30.221 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.218+0000 7f231d5ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23181e1f50 con 0x7f23180d0d60 2023-11-28T11:20:30.221 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.218+0000 7f230ffff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23180cd460 0x7f23181561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:30.221 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.218+0000 7f23177fe700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23180d0d60 0x7f23181e2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:30.221 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.219+0000 7f2316ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23180b8510 0x7f23181e1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:30.222 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.219+0000 7f23177fe700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23180d0d60 0x7f23181e2ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.169:49760/0 (socket says 172.21.15.169:49760) 2023-11-28T11:20:30.222 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.219+0000 7f23177fe700 1 -- 172.21.15.169:0/1370139363 learned_addr learned my addr 172.21.15.169:0/1370139363 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-11-28T11:20:30.222 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.219+0000 7f230ffff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23180cd460 0x7f23181561b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:56508/0 (socket says 172.21.15.169:56508) 2023-11-28T11:20:30.222 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.219+0000 7f230ffff700 1 -- 172.21.15.169:0/1370139363 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23180d0d60 msgr2=0x7f23181e2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:30.222 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.219+0000 7f230ffff700 1 --2- 172.21.15.169:0/1370139363 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23180d0d60 0x7f23181e2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:30.222 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.219+0000 7f230ffff700 1 -- 172.21.15.169:0/1370139363 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23180b8510 msgr2=0x7f23181e1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:30.222 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.219+0000 7f230ffff700 1 --2- 172.21.15.169:0/1370139363 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23180b8510 0x7f23181e1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:30.222 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.219+0000 7f230ffff700 1 -- 172.21.15.169:0/1370139363 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2308009550 con 0x7f23180cd460 2023-11-28T11:20:30.222 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.219+0000 7f2316ffd700 1 --2- 172.21.15.169:0/1370139363 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23180b8510 0x7f23181e1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:20:30.222 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.219+0000 7f23177fe700 1 --2- 172.21.15.169:0/1370139363 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23180d0d60 0x7f23181e2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:20:30.222 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.219+0000 7f230ffff700 1 --2- 172.21.15.169:0/1370139363 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23180cd460 0x7f23181561b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2300000f60 tx=0x7f230000c730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:30.223 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.219+0000 7f230f7fe700 1 -- 172.21.15.169:0/1370139363 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23000050d0 con 0x7f23180cd460 2023-11-28T11:20:30.223 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.219+0000 7f230f7fe700 1 -- 172.21.15.169:0/1370139363 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2300005230 con 0x7f23180cd460 2023-11-28T11:20:30.223 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.220+0000 7f231d5ff700 1 -- 172.21.15.169:0/1370139363 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23180cfff0 con 0x7f23180cd460 2023-11-28T11:20:30.223 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.220+0000 7f230f7fe700 1 -- 172.21.15.169:0/1370139363 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2300003680 con 0x7f23180cd460 2023-11-28T11:20:30.223 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.220+0000 7f231d5ff700 1 -- 172.21.15.169:0/1370139363 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23180cfd10 con 0x7f23180cd460 2023-11-28T11:20:30.223 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.220+0000 7f230f7fe700 1 -- 172.21.15.169:0/1370139363 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f2300019030 con 0x7f23180cd460 2023-11-28T11:20:30.223 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.220+0000 7f230f7fe700 1 -- 172.21.15.169:0/1370139363 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f2300011710 con 0x7f23180cd460 2023-11-28T11:20:30.224 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.221+0000 7f231d5ff700 1 -- 172.21.15.169:0/1370139363 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23181e2d10 con 0x7f23180cd460 2023-11-28T11:20:30.224 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.224+0000 7f230f7fe700 1 -- 172.21.15.169:0/1370139363 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f23181e2d10 con 0x7f23180cd460 2023-11-28T11:20:30.311 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.311+0000 7f231d5ff700 1 -- 172.21.15.169:0/1370139363 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd new", "uuid": "13c1de73-1a8c-4d7e-8a28-9dae52dfe089", "id": 1} v 0) v1 -- 0x7f2318067880 con 0x7f23180cd460 2023-11-28T11:20:30.318 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.317+0000 7f230f7fe700 1 -- 172.21.15.169:0/1370139363 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "13c1de73-1a8c-4d7e-8a28-9dae52dfe089", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2300017070 con 0x7f23180cd460 2023-11-28T11:20:30.318 INFO:teuthology.orchestra.run.smithi169.stdout:1 2023-11-28T11:20:30.320 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.319+0000 7f231d5ff700 1 -- 172.21.15.169:0/1370139363 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23180cd460 msgr2=0x7f23181561b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:30.320 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.319+0000 7f231d5ff700 1 --2- 172.21.15.169:0/1370139363 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23180cd460 0x7f23181561b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2300000f60 tx=0x7f230000c730 comp rx=0 tx=0).stop 2023-11-28T11:20:30.320 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.319+0000 7f231d5ff700 1 -- 172.21.15.169:0/1370139363 shutdown_connections 2023-11-28T11:20:30.320 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.319+0000 7f231d5ff700 1 --2- 172.21.15.169:0/1370139363 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f23180d0d60 0x7f23181e2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:30.320 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.319+0000 7f231d5ff700 1 --2- 172.21.15.169:0/1370139363 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23180b8510 0x7f23181e1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:30.320 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.319+0000 7f231d5ff700 1 --2- 172.21.15.169:0/1370139363 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f23180cd460 0x7f23181561b0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:30.320 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.319+0000 7f231d5ff700 1 -- 172.21.15.169:0/1370139363 >> 172.21.15.169:0/1370139363 conn(0x7f23180719a0 msgr2=0x7f23180b3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:30.320 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.319+0000 7f231d5ff700 1 -- 172.21.15.169:0/1370139363 shutdown_connections 2023-11-28T11:20:30.320 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.319+0000 7f231d5ff700 1 -- 172.21.15.169:0/1370139363 wait complete. 2023-11-28T11:20:30.345 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new 25788a55-112f-4f00-810d-2bc37cf76778 2 2023-11-28T11:20:30.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.541+0000 7efd269ff700 1 Processor -- start 2023-11-28T11:20:30.542 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.541+0000 7efd269ff700 1 -- start start 2023-11-28T11:20:30.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.542+0000 7efd269ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7efd200b8510 0x7efd2006b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:30.543 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.542+0000 7efd269ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd200cd460 0x7efd2006beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:30.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.543+0000 7efd269ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd200b88e0 con 0x7efd2006c3f0 2023-11-28T11:20:30.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.543+0000 7efd269ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd200cdbd0 con 0x7efd200b8510 2023-11-28T11:20:30.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.543+0000 7efd269ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd200d02d0 con 0x7efd200cd460 2023-11-28T11:20:30.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.543+0000 7efd1f7fe700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd200cd460 0x7efd2006beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:30.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.543+0000 7efd1ffff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7efd200b8510 0x7efd2006b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:30.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.543+0000 7efd1f7fe700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd200cd460 0x7efd2006beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.169:49774/0 (socket says 172.21.15.169:49774) 2023-11-28T11:20:30.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.543+0000 7efd1ffff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7efd200b8510 0x7efd2006b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:56512/0 (socket says 172.21.15.169:56512) 2023-11-28T11:20:30.544 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.543+0000 7efd1f7fe700 1 -- 172.21.15.169:0/4176828353 learned_addr learned my addr 172.21.15.169:0/4176828353 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-11-28T11:20:30.547 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.543+0000 7efd1f7fe700 1 -- 172.21.15.169:0/4176828353 >> v2:172.21.15.169:3300/0 conn(0x7efd200b8510 msgr2=0x7efd2006b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:30.547 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.543+0000 7efd1f7fe700 1 --2- 172.21.15.169:0/4176828353 >> v2:172.21.15.169:3300/0 conn(0x7efd200b8510 0x7efd2006b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:30.547 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.543+0000 7efd1f7fe700 1 -- 172.21.15.169:0/4176828353 >> v1:172.21.15.169:6789/0 conn(0x7efd2006c3f0 legacy=0x7efd2015cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:30.547 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.543+0000 7efd1f7fe700 1 -- 172.21.15.169:0/4176828353 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd200cd8f0 con 0x7efd200cd460 2023-11-28T11:20:30.547 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.543+0000 7efd1effd700 1 -- 172.21.15.169:0/4176828353 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2067065056 0 0) 0x7efd200b88e0 con 0x7efd2006c3f0 2023-11-28T11:20:30.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.543+0000 7efd1ffff700 1 --2- 172.21.15.169:0/4176828353 >> v2:172.21.15.169:3300/0 conn(0x7efd200b8510 0x7efd2006b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:20:30.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.544+0000 7efd1f7fe700 1 --2- 172.21.15.169:0/4176828353 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd200cd460 0x7efd2006beb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7efd140098f0 tx=0x7efd14009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25c0e0519c8b3615 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:30.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.544+0000 7efd1effd700 1 -- 172.21.15.169:0/4176828353 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd14011830 con 0x7efd200cd460 2023-11-28T11:20:30.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.544+0000 7efd1effd700 1 -- 172.21.15.169:0/4176828353 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd14011990 con 0x7efd200cd460 2023-11-28T11:20:30.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.544+0000 7efd1effd700 1 -- 172.21.15.169:0/4176828353 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd14011d20 con 0x7efd200cd460 2023-11-28T11:20:30.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.544+0000 7efd269ff700 1 -- 172.21.15.169:0/4176828353 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd200cd460 msgr2=0x7efd2006beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:30.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.544+0000 7efd269ff700 1 --2- 172.21.15.169:0/4176828353 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd200cd460 0x7efd2006beb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7efd140098f0 tx=0x7efd14009c00 comp rx=0 tx=0).stop 2023-11-28T11:20:30.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.545+0000 7efd269ff700 1 -- 172.21.15.169:0/4176828353 shutdown_connections 2023-11-28T11:20:30.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.545+0000 7efd269ff700 1 --2- 172.21.15.169:0/4176828353 >> v2:172.21.15.169:3300/0 conn(0x7efd200b8510 0x7efd2006b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:30.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.545+0000 7efd269ff700 1 --2- 172.21.15.169:0/4176828353 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd200cd460 0x7efd2006beb0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:30.548 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.545+0000 7efd269ff700 1 -- 172.21.15.169:0/4176828353 >> 172.21.15.169:0/4176828353 conn(0x7efd200719a0 msgr2=0x7efd20071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:30.549 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.545+0000 7efd269ff700 1 -- 172.21.15.169:0/4176828353 shutdown_connections 2023-11-28T11:20:30.549 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.545+0000 7efd269ff700 1 -- 172.21.15.169:0/4176828353 wait complete. 2023-11-28T11:20:30.549 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.546+0000 7efd269ff700 1 Processor -- start 2023-11-28T11:20:30.549 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.547+0000 7efd269ff700 1 -- start start 2023-11-28T11:20:30.550 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-28T11:20:30.548+0000 7f8f2f6df240 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T11:20:30.554 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.547+0000 7efd269ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd2006c3f0 0x7efd200a9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:30.554 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.547+0000 7efd269ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd200b8510 0x7efd20155840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:30.555 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.547+0000 7efd269ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd200cd460 0x7efd200c6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:30.555 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.547+0000 7efd269ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd20155d80 con 0x7efd2006c3f0 2023-11-28T11:20:30.555 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.547+0000 7efd269ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd200c7f40 con 0x7efd200b8510 2023-11-28T11:20:30.555 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.547+0000 7efd269ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd20156780 con 0x7efd200cd460 2023-11-28T11:20:30.555 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.548+0000 7efd1f7fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd200b8510 0x7efd20155840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:30.556 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.548+0000 7efd24f9c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd200cd460 0x7efd200c6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:30.556 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.548+0000 7efd1f7fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd200b8510 0x7efd20155840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:56518/0 (socket says 172.21.15.169:56518) 2023-11-28T11:20:30.556 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.548+0000 7efd1f7fe700 1 -- 172.21.15.169:0/3005389820 learned_addr learned my addr 172.21.15.169:0/3005389820 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-11-28T11:20:30.556 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.548+0000 7efd1ffff700 1 --2- 172.21.15.169:0/3005389820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd2006c3f0 0x7efd200a9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:30.556 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.548+0000 7efd1f7fe700 1 -- 172.21.15.169:0/3005389820 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd200cd460 msgr2=0x7efd200c6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:30.557 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.548+0000 7efd1f7fe700 1 --2- 172.21.15.169:0/3005389820 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd200cd460 0x7efd200c6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:30.557 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.548+0000 7efd1f7fe700 1 -- 172.21.15.169:0/3005389820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd2006c3f0 msgr2=0x7efd200a9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:30.557 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.548+0000 7efd1f7fe700 1 --2- 172.21.15.169:0/3005389820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd2006c3f0 0x7efd200a9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:30.557 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.548+0000 7efd1f7fe700 1 -- 172.21.15.169:0/3005389820 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd100094a0 con 0x7efd200b8510 2023-11-28T11:20:30.557 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.548+0000 7efd24f9c700 1 --2- 172.21.15.169:0/3005389820 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd200cd460 0x7efd200c6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:20:30.557 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.548+0000 7efd1f7fe700 1 --2- 172.21.15.169:0/3005389820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd200b8510 0x7efd20155840 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7efd14000c00 tx=0x7efd14004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:30.558 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.549+0000 7efd1cff9700 1 -- 172.21.15.169:0/3005389820 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1401a910 con 0x7efd200b8510 2023-11-28T11:20:30.558 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.549+0000 7efd1cff9700 1 -- 172.21.15.169:0/3005389820 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd1401f070 con 0x7efd200b8510 2023-11-28T11:20:30.558 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.549+0000 7efd1cff9700 1 -- 172.21.15.169:0/3005389820 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1401acb0 con 0x7efd200b8510 2023-11-28T11:20:30.558 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.549+0000 7efd269ff700 1 -- 172.21.15.169:0/3005389820 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd14009550 con 0x7efd200b8510 2023-11-28T11:20:30.558 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.549+0000 7efd269ff700 1 -- 172.21.15.169:0/3005389820 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd200cfff0 con 0x7efd200b8510 2023-11-28T11:20:30.558 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.549+0000 7efd1cff9700 1 -- 172.21.15.169:0/3005389820 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7efd1400f050 con 0x7efd200b8510 2023-11-28T11:20:30.559 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.549+0000 7efd1cff9700 1 -- 172.21.15.169:0/3005389820 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7efd14025400 con 0x7efd200b8510 2023-11-28T11:20:30.559 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.550+0000 7efd269ff700 1 -- 172.21.15.169:0/3005389820 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd200cfd10 con 0x7efd200b8510 2023-11-28T11:20:30.559 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.553+0000 7efd1cff9700 1 -- 172.21.15.169:0/3005389820 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7efd14033060 con 0x7efd200b8510 2023-11-28T11:20:30.638 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-11-28T11:20:30.638+0000 7f2166e55240 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T11:20:30.639 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.639+0000 7efd269ff700 1 -- 172.21.15.169:0/3005389820 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd new", "uuid": "25788a55-112f-4f00-810d-2bc37cf76778", "id": 2} v 0) v1 -- 0x7efd20067880 con 0x7efd200b8510 2023-11-28T11:20:30.645 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.644+0000 7efd1cff9700 1 -- 172.21.15.169:0/3005389820 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "25788a55-112f-4f00-810d-2bc37cf76778", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7efd14021070 con 0x7efd200b8510 2023-11-28T11:20:30.645 INFO:teuthology.orchestra.run.smithi169.stdout:2 2023-11-28T11:20:30.647 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.645+0000 7efd269ff700 1 -- 172.21.15.169:0/3005389820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd200b8510 msgr2=0x7efd20155840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:30.647 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.645+0000 7efd269ff700 1 --2- 172.21.15.169:0/3005389820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd200b8510 0x7efd20155840 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7efd14000c00 tx=0x7efd14004750 comp rx=0 tx=0).stop 2023-11-28T11:20:30.647 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.646+0000 7efd269ff700 1 -- 172.21.15.169:0/3005389820 shutdown_connections 2023-11-28T11:20:30.647 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.646+0000 7efd269ff700 1 --2- 172.21.15.169:0/3005389820 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efd200cd460 0x7efd200c6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:30.647 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.646+0000 7efd269ff700 1 --2- 172.21.15.169:0/3005389820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd2006c3f0 0x7efd200a9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:30.647 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.646+0000 7efd269ff700 1 --2- 172.21.15.169:0/3005389820 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efd200b8510 0x7efd20155840 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:30.647 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.646+0000 7efd269ff700 1 -- 172.21.15.169:0/3005389820 >> 172.21.15.169:0/3005389820 conn(0x7efd200719a0 msgr2=0x7efd200b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:30.647 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.646+0000 7efd269ff700 1 -- 172.21.15.169:0/3005389820 shutdown_connections 2023-11-28T11:20:30.648 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.646+0000 7efd269ff700 1 -- 172.21.15.169:0/3005389820 wait complete. 2023-11-28T11:20:30.664 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new adf54e24-33a9-41fe-9ce3-98a5ab6c158e 3 2023-11-28T11:20:30.862 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.861+0000 7fd9acbff700 1 Processor -- start 2023-11-28T11:20:30.862 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.861+0000 7fd9acbff700 1 -- start start 2023-11-28T11:20:30.865 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.863+0000 7fd9acbff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd9a80cd460 0x7fd9a806c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:30.865 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.863+0000 7fd9acbff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9a80d0d60 0x7fd9a80d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:30.865 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.863+0000 7fd9acbff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9a80b88e0 con 0x7fd9a80b8510 2023-11-28T11:20:30.865 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.863+0000 7fd9acbff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9a80cdbd0 con 0x7fd9a80cd460 2023-11-28T11:20:30.865 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.863+0000 7fd9acbff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a80d02d0 con 0x7fd9a80d0d60 2023-11-28T11:20:30.865 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.863+0000 7fd99ffff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd9a80cd460 0x7fd9a806c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:30.865 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.863+0000 7fd9a6d9d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9a80d0d60 0x7fd9a80d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:30.865 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.863+0000 7fd99ffff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd9a80cd460 0x7fd9a806c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:56526/0 (socket says 172.21.15.169:56526) 2023-11-28T11:20:30.866 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.863+0000 7fd99ffff700 1 -- 172.21.15.169:0/3447050006 learned_addr learned my addr 172.21.15.169:0/3447050006 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-11-28T11:20:30.866 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.863+0000 7fd9a659c700 1 --1- 172.21.15.169:0/3447050006 >> v1:172.21.15.92:6789/0 conn(0x7fd9a80b8510 0x7fd9a806b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.169:43324/0 (socket says 172.21.15.169:43324) 2023-11-28T11:20:30.866 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.863+0000 7fd99ffff700 1 -- 172.21.15.169:0/3447050006 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9a80d0d60 msgr2=0x7fd9a80d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:30.866 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.863+0000 7fd99ffff700 1 --2- 172.21.15.169:0/3447050006 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9a80d0d60 0x7fd9a80d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:30.866 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.863+0000 7fd99ffff700 1 -- 172.21.15.169:0/3447050006 >> v1:172.21.15.92:6789/0 conn(0x7fd9a80b8510 legacy=0x7fd9a806b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:30.866 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.863+0000 7fd99ffff700 1 -- 172.21.15.169:0/3447050006 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a80cd8f0 con 0x7fd9a80cd460 2023-11-28T11:20:30.867 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.863+0000 7fd9a5d9b700 1 -- 172.21.15.169:0/3447050006 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3057527107 0 0) 0x7fd9a80b88e0 con 0x7fd9a80b8510 2023-11-28T11:20:30.867 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.863+0000 7fd9a6d9d700 1 --2- 172.21.15.169:0/3447050006 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9a80d0d60 0x7fd9a80d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:20:30.867 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.864+0000 7fd99ffff700 1 --2- 172.21.15.169:0/3447050006 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9a80cd460 0x7fd9a806c080 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd990009920 tx=0x7fd990009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb8d1289e7480fc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:30.867 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.864+0000 7fd9a5d9b700 1 -- 172.21.15.169:0/3447050006 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd990011830 con 0x7fd9a80cd460 2023-11-28T11:20:30.867 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.864+0000 7fd9a5d9b700 1 -- 172.21.15.169:0/3447050006 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd990011990 con 0x7fd9a80cd460 2023-11-28T11:20:30.867 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.864+0000 7fd9a5d9b700 1 -- 172.21.15.169:0/3447050006 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd990011d20 con 0x7fd9a80cd460 2023-11-28T11:20:30.870 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.865+0000 7fd9acbff700 1 -- 172.21.15.169:0/3447050006 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9a80cd460 msgr2=0x7fd9a806c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:30.871 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.865+0000 7fd9acbff700 1 --2- 172.21.15.169:0/3447050006 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9a80cd460 0x7fd9a806c080 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd990009920 tx=0x7fd990009c30 comp rx=0 tx=0).stop 2023-11-28T11:20:30.871 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.865+0000 7fd9acbff700 1 -- 172.21.15.169:0/3447050006 shutdown_connections 2023-11-28T11:20:30.871 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.865+0000 7fd9acbff700 1 --2- 172.21.15.169:0/3447050006 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9a80d0d60 0x7fd9a80d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:30.871 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.865+0000 7fd9acbff700 1 --2- 172.21.15.169:0/3447050006 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9a80cd460 0x7fd9a806c080 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:30.871 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.865+0000 7fd9acbff700 1 -- 172.21.15.169:0/3447050006 >> 172.21.15.169:0/3447050006 conn(0x7fd9a80719a0 msgr2=0x7fd9a8071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:30.871 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.865+0000 7fd9acbff700 1 -- 172.21.15.169:0/3447050006 shutdown_connections 2023-11-28T11:20:30.871 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.865+0000 7fd9acbff700 1 -- 172.21.15.169:0/3447050006 wait complete. 2023-11-28T11:20:30.871 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.867+0000 7fd9acbff700 1 Processor -- start 2023-11-28T11:20:30.871 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.867+0000 7fd9acbff700 1 -- start start 2023-11-28T11:20:30.872 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.868+0000 7fd9acbff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9a80b8510 0x7fd9a81f1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:30.872 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.868+0000 7fd9acbff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9a80cd460 0x7fd9a81f4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:30.872 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.868+0000 7fd9acbff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9a80d0d60 0x7fd9a80cc470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:30.872 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.868+0000 7fd9acbff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a8155ed0 con 0x7fd9a80cd460 2023-11-28T11:20:30.872 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.868+0000 7fd9acbff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a80cd2c0 con 0x7fd9a80b8510 2023-11-28T11:20:30.872 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.868+0000 7fd9acbff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a80ccfe0 con 0x7fd9a80d0d60 2023-11-28T11:20:30.872 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.868+0000 7fd9a659c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9a80b8510 0x7fd9a81f1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:30.872 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.868+0000 7fd9a6d9d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9a80d0d60 0x7fd9a80cc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:30.872 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.868+0000 7fd99ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9a80cd460 0x7fd9a81f4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:30.872 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.868+0000 7fd99ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9a80cd460 0x7fd9a81f4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.169:60260/0 (socket says 172.21.15.169:60260) 2023-11-28T11:20:30.873 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.868+0000 7fd99ffff700 1 -- 172.21.15.169:0/250791659 learned_addr learned my addr 172.21.15.169:0/250791659 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-11-28T11:20:30.873 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.869+0000 7fd9a6d9d700 1 -- 172.21.15.169:0/250791659 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9a80b8510 msgr2=0x7fd9a81f1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:30.873 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.869+0000 7fd9a6d9d700 1 --2- 172.21.15.169:0/250791659 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9a80b8510 0x7fd9a81f1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:30.873 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.869+0000 7fd9a6d9d700 1 -- 172.21.15.169:0/250791659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9a80cd460 msgr2=0x7fd9a81f4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:30.873 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.869+0000 7fd9a6d9d700 1 --2- 172.21.15.169:0/250791659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9a80cd460 0x7fd9a81f4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:30.873 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.869+0000 7fd9a6d9d700 1 -- 172.21.15.169:0/250791659 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a0009a70 con 0x7fd9a80d0d60 2023-11-28T11:20:30.873 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.869+0000 7fd9a659c700 1 --2- 172.21.15.169:0/250791659 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9a80b8510 0x7fd9a81f1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:20:30.873 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.869+0000 7fd9a6d9d700 1 --2- 172.21.15.169:0/250791659 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9a80d0d60 0x7fd9a80cc470 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd9a000efc0 tx=0x7fd9a000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:30.873 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.869+0000 7fd99effd700 1 -- 172.21.15.169:0/250791659 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a0009e80 con 0x7fd9a80d0d60 2023-11-28T11:20:30.874 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.869+0000 7fd99effd700 1 -- 172.21.15.169:0/250791659 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9a000f040 con 0x7fd9a80d0d60 2023-11-28T11:20:30.874 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.869+0000 7fd99effd700 1 -- 172.21.15.169:0/250791659 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a000cc70 con 0x7fd9a80d0d60 2023-11-28T11:20:30.874 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.869+0000 7fd9acbff700 1 -- 172.21.15.169:0/250791659 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd990009550 con 0x7fd9a80d0d60 2023-11-28T11:20:30.874 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.869+0000 7fd9acbff700 1 -- 172.21.15.169:0/250791659 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9a80cfff0 con 0x7fd9a80d0d60 2023-11-28T11:20:30.874 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.870+0000 7fd99effd700 1 -- 172.21.15.169:0/250791659 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fd9a0003a20 con 0x7fd9a80d0d60 2023-11-28T11:20:30.874 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.870+0000 7fd99effd700 1 -- 172.21.15.169:0/250791659 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7fd9a0004780 con 0x7fd9a80d0d60 2023-11-28T11:20:30.875 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.870+0000 7fd9acbff700 1 -- 172.21.15.169:0/250791659 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9a80cfd10 con 0x7fd9a80d0d60 2023-11-28T11:20:30.875 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.873+0000 7fd99effd700 1 -- 172.21.15.169:0/250791659 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fd9a0018030 con 0x7fd9a80d0d60 2023-11-28T11:20:30.961 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.960+0000 7fd9acbff700 1 -- 172.21.15.169:0/250791659 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd new", "uuid": "adf54e24-33a9-41fe-9ce3-98a5ab6c158e", "id": 3} v 0) v1 -- 0x7fd9a8067880 con 0x7fd9a80d0d60 2023-11-28T11:20:30.969 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.968+0000 7fd99effd700 1 -- 172.21.15.169:0/250791659 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "adf54e24-33a9-41fe-9ce3-98a5ab6c158e", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd9a0004020 con 0x7fd9a80d0d60 2023-11-28T11:20:30.969 INFO:teuthology.orchestra.run.smithi169.stdout:3 2023-11-28T11:20:30.971 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.969+0000 7fd9acbff700 1 -- 172.21.15.169:0/250791659 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9a80d0d60 msgr2=0x7fd9a80cc470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:30.971 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.969+0000 7fd9acbff700 1 --2- 172.21.15.169:0/250791659 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9a80d0d60 0x7fd9a80cc470 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd9a000efc0 tx=0x7fd9a000c3a0 comp rx=0 tx=0).stop 2023-11-28T11:20:30.971 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.969+0000 7fd9acbff700 1 -- 172.21.15.169:0/250791659 shutdown_connections 2023-11-28T11:20:30.971 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.969+0000 7fd9acbff700 1 --2- 172.21.15.169:0/250791659 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9a80d0d60 0x7fd9a80cc470 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:30.971 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.969+0000 7fd9acbff700 1 --2- 172.21.15.169:0/250791659 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9a80b8510 0x7fd9a81f1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:30.971 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.969+0000 7fd9acbff700 1 --2- 172.21.15.169:0/250791659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9a80cd460 0x7fd9a81f4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:30.971 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.969+0000 7fd9acbff700 1 -- 172.21.15.169:0/250791659 >> 172.21.15.169:0/250791659 conn(0x7fd9a80719a0 msgr2=0x7fd9a80b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:30.971 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.970+0000 7fd9acbff700 1 -- 172.21.15.169:0/250791659 shutdown_connections 2023-11-28T11:20:30.972 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:30.970+0000 7fd9acbff700 1 -- 172.21.15.169:0/250791659 wait complete. 2023-11-28T11:20:30.987 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new ccb9b747-f399-44ea-9a18-9db3152227e8 4 2023-11-28T11:20:31.181 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.180+0000 7f7251dff700 1 Processor -- start 2023-11-28T11:20:31.182 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.181+0000 7f7251dff700 1 -- start start 2023-11-28T11:20:31.183 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.182+0000 7f7251dff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f724c0b8510 0x7f724c06b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:31.183 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.182+0000 7f7251dff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f724c0b88e0 con 0x7f724c0d0d60 2023-11-28T11:20:31.183 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.182+0000 7f7251dff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f724c0cd8f0 con 0x7f724c0cd460 2023-11-28T11:20:31.183 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.182+0000 7f7251dff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f724c0cdbd0 con 0x7f724c0b8510 2023-11-28T11:20:31.183 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.182+0000 7f7243fff700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f724c0cd460 0x7f724c06beb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.169:35862/0 (socket says 172.21.15.169:35862) 2023-11-28T11:20:31.183 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.182+0000 7f7243fff700 1 -- 172.21.15.169:0/2070877557 learned_addr learned my addr 172.21.15.169:0/2070877557 (peer_addr_for_me v1:172.21.15.169:0/0) 2023-11-28T11:20:31.183 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.182+0000 7f724b7fe700 1 --2- 172.21.15.169:0/2070877557 >> v2:172.21.15.92:3300/0 conn(0x7f724c0b8510 0x7f724c06b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:31.185 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.183+0000 7f724affd700 1 -- 172.21.15.169:0/2070877557 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2029366660 0 0) 0x7f724c0cd8f0 con 0x7f724c0cd460 2023-11-28T11:20:31.185 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.183+0000 7f724affd700 1 -- 172.21.15.169:0/2070877557 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72380035b0 con 0x7f724c0cd460 2023-11-28T11:20:31.185 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.183+0000 7f724b7fe700 1 -- 172.21.15.169:0/2070877557 >> v1:172.21.15.169:6789/0 conn(0x7f724c0cd460 legacy=0x7f724c06beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:31.185 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.183+0000 7f724b7fe700 1 -- 172.21.15.169:0/2070877557 >> v1:172.21.15.92:6789/0 conn(0x7f724c0d0d60 legacy=0x7f724c0d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:31.185 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.183+0000 7f724affd700 1 -- 172.21.15.169:0/2070877557 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 713861874 0 0) 0x7f724c0b88e0 con 0x7f724c0d0d60 2023-11-28T11:20:31.185 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.183+0000 7f724b7fe700 1 -- 172.21.15.169:0/2070877557 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f724c0cfd10 con 0x7f724c0b8510 2023-11-28T11:20:31.185 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.183+0000 7f724b7fe700 1 --2- 172.21.15.169:0/2070877557 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f724c0b8510 0x7f724c06b970 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f723c009840 tx=0x7f723c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8a002fdbfcca896 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:31.186 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.183+0000 7f724affd700 1 -- 172.21.15.169:0/2070877557 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723c011830 con 0x7f724c0b8510 2023-11-28T11:20:31.186 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.183+0000 7f724affd700 1 -- 172.21.15.169:0/2070877557 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f723c011990 con 0x7f724c0b8510 2023-11-28T11:20:31.186 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.183+0000 7f724affd700 1 -- 172.21.15.169:0/2070877557 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723c011d20 con 0x7f724c0b8510 2023-11-28T11:20:31.186 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.184+0000 7f7251dff700 1 -- 172.21.15.169:0/2070877557 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f724c0b8510 msgr2=0x7f724c06b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:31.186 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.184+0000 7f7251dff700 1 --2- 172.21.15.169:0/2070877557 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f724c0b8510 0x7f724c06b970 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f723c009840 tx=0x7f723c009b50 comp rx=0 tx=0).stop 2023-11-28T11:20:31.186 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.184+0000 7f7251dff700 1 -- 172.21.15.169:0/2070877557 shutdown_connections 2023-11-28T11:20:31.186 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.184+0000 7f7251dff700 1 --2- 172.21.15.169:0/2070877557 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f724c0b8510 0x7f724c06b970 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:31.186 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.184+0000 7f7251dff700 1 -- 172.21.15.169:0/2070877557 >> 172.21.15.169:0/2070877557 conn(0x7f724c0719a0 msgr2=0x7f724c071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:31.186 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.184+0000 7f7251dff700 1 -- 172.21.15.169:0/2070877557 shutdown_connections 2023-11-28T11:20:31.186 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.184+0000 7f7251dff700 1 -- 172.21.15.169:0/2070877557 wait complete. 2023-11-28T11:20:31.187 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.187+0000 7f7251dff700 1 Processor -- start 2023-11-28T11:20:31.187 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.187+0000 7f7251dff700 1 -- start start 2023-11-28T11:20:31.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.188+0000 7f7251dff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f724c0b8510 0x7f724c15af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:31.189 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.188+0000 7f7251dff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f724c0cd460 0x7f724c1e35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:31.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.188+0000 7f7251dff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f724c0d0d60 0x7f724c1e2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:31.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.188+0000 7f7251dff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f724c1e3af0 con 0x7f724c0b8510 2023-11-28T11:20:31.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.188+0000 7f7251dff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f724c1e26a0 con 0x7f724c0cd460 2023-11-28T11:20:31.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.188+0000 7f7251dff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f724c1e4a80 con 0x7f724c0d0d60 2023-11-28T11:20:31.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.188+0000 7f724b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f724c0b8510 0x7f724c15af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:31.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.188+0000 7f7243fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f724c0cd460 0x7f724c1e35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:31.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.188+0000 7f724b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f724c0b8510 0x7f724c15af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.169:60272/0 (socket says 172.21.15.169:60272) 2023-11-28T11:20:31.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.188+0000 7f724b7fe700 1 -- 172.21.15.169:0/369877530 learned_addr learned my addr 172.21.15.169:0/369877530 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-11-28T11:20:31.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.188+0000 7f724bfff700 1 --2- 172.21.15.169:0/369877530 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f724c0d0d60 0x7f724c1e2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:31.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.189+0000 7f724b7fe700 1 -- 172.21.15.169:0/369877530 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f724c0d0d60 msgr2=0x7f724c1e2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:31.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.189+0000 7f724b7fe700 1 --2- 172.21.15.169:0/369877530 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f724c0d0d60 0x7f724c1e2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:31.190 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.189+0000 7f724b7fe700 1 -- 172.21.15.169:0/369877530 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f724c0cd460 msgr2=0x7f724c1e35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:31.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.189+0000 7f724b7fe700 1 --2- 172.21.15.169:0/369877530 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f724c0cd460 0x7f724c1e35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:31.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.189+0000 7f724b7fe700 1 -- 172.21.15.169:0/369877530 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f723c0094a0 con 0x7f724c0b8510 2023-11-28T11:20:31.191 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.189+0000 7f724b7fe700 1 --2- 172.21.15.169:0/369877530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f724c0b8510 0x7f724c15af70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f723c009b90 tx=0x7f723c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:31.192 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.189+0000 7f7248ff9700 1 -- 172.21.15.169:0/369877530 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723c01a7e0 con 0x7f724c0b8510 2023-11-28T11:20:31.192 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.189+0000 7f7248ff9700 1 -- 172.21.15.169:0/369877530 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f723c01a940 con 0x7f724c0b8510 2023-11-28T11:20:31.192 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.189+0000 7f7251dff700 1 -- 172.21.15.169:0/369877530 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f724c0d29c0 con 0x7f724c0b8510 2023-11-28T11:20:31.192 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.189+0000 7f7248ff9700 1 -- 172.21.15.169:0/369877530 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723c01acf0 con 0x7f724c0b8510 2023-11-28T11:20:31.192 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.190+0000 7f7251dff700 1 -- 172.21.15.169:0/369877530 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f724c0cfff0 con 0x7f724c0b8510 2023-11-28T11:20:31.192 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.190+0000 7f7248ff9700 1 -- 172.21.15.169:0/369877530 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f723c02b030 con 0x7f724c0b8510 2023-11-28T11:20:31.192 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.190+0000 7f7248ff9700 1 -- 172.21.15.169:0/369877530 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f723c02d8c0 con 0x7f724c0b8510 2023-11-28T11:20:31.192 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.190+0000 7f7251dff700 1 -- 172.21.15.169:0/369877530 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f724c0d2b50 con 0x7f724c0b8510 2023-11-28T11:20:31.195 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.195+0000 7f7248ff9700 1 -- 172.21.15.169:0/369877530 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f723c029070 con 0x7f724c0b8510 2023-11-28T11:20:31.282 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.281+0000 7f7251dff700 1 -- 172.21.15.169:0/369877530 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ccb9b747-f399-44ea-9a18-9db3152227e8", "id": 4} v 0) v1 -- 0x7f724c067880 con 0x7f724c0b8510 2023-11-28T11:20:31.289 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.289+0000 7f7248ff9700 1 -- 172.21.15.169:0/369877530 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ccb9b747-f399-44ea-9a18-9db3152227e8", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f723c015020 con 0x7f724c0b8510 2023-11-28T11:20:31.290 INFO:teuthology.orchestra.run.smithi169.stdout:4 2023-11-28T11:20:31.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.290+0000 7f7251dff700 1 -- 172.21.15.169:0/369877530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f724c0b8510 msgr2=0x7f724c15af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:31.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.290+0000 7f7251dff700 1 --2- 172.21.15.169:0/369877530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f724c0b8510 0x7f724c15af70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f723c009b90 tx=0x7f723c004750 comp rx=0 tx=0).stop 2023-11-28T11:20:31.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.290+0000 7f7251dff700 1 -- 172.21.15.169:0/369877530 shutdown_connections 2023-11-28T11:20:31.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.290+0000 7f7251dff700 1 --2- 172.21.15.169:0/369877530 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f724c0d0d60 0x7f724c1e2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:31.291 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.290+0000 7f7251dff700 1 --2- 172.21.15.169:0/369877530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f724c0b8510 0x7f724c15af70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:31.292 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.290+0000 7f7251dff700 1 --2- 172.21.15.169:0/369877530 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f724c0cd460 0x7f724c1e35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:31.292 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.290+0000 7f7251dff700 1 -- 172.21.15.169:0/369877530 >> 172.21.15.169:0/369877530 conn(0x7f724c0719a0 msgr2=0x7f724c0b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:31.292 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.290+0000 7f7251dff700 1 -- 172.21.15.169:0/369877530 shutdown_connections 2023-11-28T11:20:31.292 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.290+0000 7f7251dff700 1 -- 172.21.15.169:0/369877530 wait complete. 2023-11-28T11:20:31.308 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new 308a1e63-75aa-4d7d-8a7c-cecc24714fd5 5 2023-11-28T11:20:31.435 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-28T11:20:31.434+0000 7f8f2f6df240 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T11:20:31.503 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.502+0000 7f3aea1ff700 1 Processor -- start 2023-11-28T11:20:31.503 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.502+0000 7f3aea1ff700 1 -- start start 2023-11-28T11:20:31.505 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.503+0000 7f3aea1ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3ae40cd460 0x7f3ae406c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:31.505 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.503+0000 7f3aea1ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3ae40d0d60 0x7f3ae40d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:31.505 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.503+0000 7f3aea1ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ae40b88e0 con 0x7f3ae40b8510 2023-11-28T11:20:31.505 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.503+0000 7f3aea1ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ae40cdbd0 con 0x7f3ae40d0d60 2023-11-28T11:20:31.506 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.503+0000 7f3aea1ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ae40d02d0 con 0x7f3ae40cd460 2023-11-28T11:20:31.506 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.504+0000 7f3ae2ffd700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3ae40cd460 0x7f3ae406c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:31.506 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.504+0000 7f3ae37fe700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f3ae40b8510 0x7f3ae406b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.169:35868/0 (socket says 172.21.15.169:35868) 2023-11-28T11:20:31.506 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.504+0000 7f3ae37fe700 1 -- 172.21.15.169:0/2924382283 learned_addr learned my addr 172.21.15.169:0/2924382283 (peer_addr_for_me v1:172.21.15.169:0/0) 2023-11-28T11:20:31.506 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.504+0000 7f3ae3fff700 1 --2- 172.21.15.169:0/2924382283 >> v2:172.21.15.92:3300/0 conn(0x7f3ae40d0d60 0x7f3ae40d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:31.506 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.504+0000 7f3ae2ffd700 1 -- 172.21.15.169:0/2924382283 >> v2:172.21.15.92:3300/0 conn(0x7f3ae40d0d60 msgr2=0x7f3ae40d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:31.506 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.504+0000 7f3ae2ffd700 1 --2- 172.21.15.169:0/2924382283 >> v2:172.21.15.92:3300/0 conn(0x7f3ae40d0d60 0x7f3ae40d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:31.506 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.504+0000 7f3ae2ffd700 1 -- 172.21.15.169:0/2924382283 >> v1:172.21.15.169:6789/0 conn(0x7f3ae40b8510 legacy=0x7f3ae406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:31.506 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.504+0000 7f3ae2ffd700 1 -- 172.21.15.169:0/2924382283 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ae40cd8f0 con 0x7f3ae40cd460 2023-11-28T11:20:31.506 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.504+0000 7f3ae27fc700 1 -- 172.21.15.169:0/2924382283 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 789649438 0 0) 0x7f3ae40b88e0 con 0x7f3ae40b8510 2023-11-28T11:20:31.506 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.504+0000 7f3ae2ffd700 1 --2- 172.21.15.169:0/2924382283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ae40cd460 0x7f3ae406c080 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3ae40ceba0 tx=0x7f3ad8009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85df63a482924fea server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:31.506 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.505+0000 7f3ae27fc700 1 -- 172.21.15.169:0/2924382283 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad8011830 con 0x7f3ae40cd460 2023-11-28T11:20:31.507 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.505+0000 7f3ae27fc700 1 -- 172.21.15.169:0/2924382283 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ad8011990 con 0x7f3ae40cd460 2023-11-28T11:20:31.507 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.505+0000 7f3ae27fc700 1 -- 172.21.15.169:0/2924382283 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad8011d20 con 0x7f3ae40cd460 2023-11-28T11:20:31.508 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.505+0000 7f3aea1ff700 1 -- 172.21.15.169:0/2924382283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ae40cd460 msgr2=0x7f3ae406c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:31.508 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.505+0000 7f3aea1ff700 1 --2- 172.21.15.169:0/2924382283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ae40cd460 0x7f3ae406c080 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3ae40ceba0 tx=0x7f3ad8009b20 comp rx=0 tx=0).stop 2023-11-28T11:20:31.508 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.506+0000 7f3aea1ff700 1 -- 172.21.15.169:0/2924382283 shutdown_connections 2023-11-28T11:20:31.508 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.506+0000 7f3aea1ff700 1 --2- 172.21.15.169:0/2924382283 >> v2:172.21.15.92:3300/0 conn(0x7f3ae40d0d60 0x7f3ae40d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:31.508 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.506+0000 7f3aea1ff700 1 --2- 172.21.15.169:0/2924382283 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ae40cd460 0x7f3ae406c080 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:31.508 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.506+0000 7f3aea1ff700 1 -- 172.21.15.169:0/2924382283 >> 172.21.15.169:0/2924382283 conn(0x7f3ae40719a0 msgr2=0x7f3ae4071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:31.508 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.506+0000 7f3aea1ff700 1 -- 172.21.15.169:0/2924382283 shutdown_connections 2023-11-28T11:20:31.508 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.506+0000 7f3aea1ff700 1 -- 172.21.15.169:0/2924382283 wait complete. 2023-11-28T11:20:31.509 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.508+0000 7f3aea1ff700 1 Processor -- start 2023-11-28T11:20:31.509 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.508+0000 7f3aea1ff700 1 -- start start 2023-11-28T11:20:31.511 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.509+0000 7f3aea1ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ae40b8510 0x7f3ae41e66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:31.511 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.509+0000 7f3aea1ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ae40cd460 0x7f3ae41e4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:31.511 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.509+0000 7f3aea1ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ae40d0d60 0x7f3ae415a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:31.511 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.509+0000 7f3aea1ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae415b8f0 con 0x7f3ae40d0d60 2023-11-28T11:20:31.511 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.509+0000 7f3aea1ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae41e5600 con 0x7f3ae40b8510 2023-11-28T11:20:31.511 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.509+0000 7f3aea1ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae41e58e0 con 0x7f3ae40cd460 2023-11-28T11:20:31.511 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.509+0000 7f3ae37fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ae40b8510 0x7f3ae41e66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:31.512 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.509+0000 7f3ae37fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ae40b8510 0x7f3ae41e66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:56572/0 (socket says 172.21.15.169:56572) 2023-11-28T11:20:31.512 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.509+0000 7f3ae37fe700 1 -- 172.21.15.169:0/1361568871 learned_addr learned my addr 172.21.15.169:0/1361568871 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-11-28T11:20:31.512 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.509+0000 7f3ae2ffd700 1 --2- 172.21.15.169:0/1361568871 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ae40cd460 0x7f3ae41e4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:31.512 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.509+0000 7f3ae3fff700 1 --2- 172.21.15.169:0/1361568871 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ae40d0d60 0x7f3ae415a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:31.512 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.509+0000 7f3ae37fe700 1 -- 172.21.15.169:0/1361568871 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ae40cd460 msgr2=0x7f3ae41e4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:31.512 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.509+0000 7f3ae37fe700 1 --2- 172.21.15.169:0/1361568871 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ae40cd460 0x7f3ae41e4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:31.512 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.509+0000 7f3ae37fe700 1 -- 172.21.15.169:0/1361568871 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ae40d0d60 msgr2=0x7f3ae415a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:31.512 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.509+0000 7f3ae37fe700 1 --2- 172.21.15.169:0/1361568871 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ae40d0d60 0x7f3ae415a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:31.512 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.509+0000 7f3ae37fe700 1 -- 172.21.15.169:0/1361568871 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad80094a0 con 0x7f3ae40b8510 2023-11-28T11:20:31.512 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.509+0000 7f3ae3fff700 1 --2- 172.21.15.169:0/1361568871 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ae40d0d60 0x7f3ae415a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:20:31.512 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.510+0000 7f3ae37fe700 1 --2- 172.21.15.169:0/1361568871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ae40b8510 0x7f3ae41e66a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3ad40011e0 tx=0x7f3ad400ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:31.512 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.510+0000 7f3acbfff700 1 -- 172.21.15.169:0/1361568871 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad4004c80 con 0x7f3ae40b8510 2023-11-28T11:20:31.513 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.510+0000 7f3acbfff700 1 -- 172.21.15.169:0/1361568871 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ad4004e00 con 0x7f3ae40b8510 2023-11-28T11:20:31.513 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.510+0000 7f3acbfff700 1 -- 172.21.15.169:0/1361568871 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad4007310 con 0x7f3ae40b8510 2023-11-28T11:20:31.513 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.510+0000 7f3aea1ff700 1 -- 172.21.15.169:0/1361568871 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ae40cfff0 con 0x7f3ae40b8510 2023-11-28T11:20:31.513 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.510+0000 7f3aea1ff700 1 -- 172.21.15.169:0/1361568871 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ae40cfd10 con 0x7f3ae40b8510 2023-11-28T11:20:31.513 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.510+0000 7f3acbfff700 1 -- 172.21.15.169:0/1361568871 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f3ad401a020 con 0x7f3ae40b8510 2023-11-28T11:20:31.513 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.510+0000 7f3acbfff700 1 -- 172.21.15.169:0/1361568871 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f3ad401d3f0 con 0x7f3ae40b8510 2023-11-28T11:20:31.513 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.511+0000 7f3aea1ff700 1 -- 172.21.15.169:0/1361568871 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ae40b50e0 con 0x7f3ae40b8510 2023-11-28T11:20:31.515 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.515+0000 7f3acbfff700 1 -- 172.21.15.169:0/1361568871 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f3ad401d5b0 con 0x7f3ae40b8510 2023-11-28T11:20:31.526 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-28T11:20:31.525+0000 7f8f2f6df240 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T11:20:31.591 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-11-28T11:20:31.590+0000 7f2166e55240 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T11:20:31.603 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.602+0000 7f3aea1ff700 1 -- 172.21.15.169:0/1361568871 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd new", "uuid": "308a1e63-75aa-4d7d-8a7c-cecc24714fd5", "id": 5} v 0) v1 -- 0x7f3ae4067880 con 0x7f3ae40b8510 2023-11-28T11:20:31.611 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.610+0000 7f3acbfff700 1 -- 172.21.15.169:0/1361568871 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "308a1e63-75aa-4d7d-8a7c-cecc24714fd5", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3ad4007540 con 0x7f3ae40b8510 2023-11-28T11:20:31.611 INFO:teuthology.orchestra.run.smithi169.stdout:5 2023-11-28T11:20:31.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.611+0000 7f3aea1ff700 1 -- 172.21.15.169:0/1361568871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ae40b8510 msgr2=0x7f3ae41e66a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:31.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.611+0000 7f3aea1ff700 1 --2- 172.21.15.169:0/1361568871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ae40b8510 0x7f3ae41e66a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3ad40011e0 tx=0x7f3ad400ca30 comp rx=0 tx=0).stop 2023-11-28T11:20:31.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.611+0000 7f3aea1ff700 1 -- 172.21.15.169:0/1361568871 shutdown_connections 2023-11-28T11:20:31.612 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.611+0000 7f3aea1ff700 1 --2- 172.21.15.169:0/1361568871 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ae40d0d60 0x7f3ae415a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:31.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.611+0000 7f3aea1ff700 1 --2- 172.21.15.169:0/1361568871 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3ae40b8510 0x7f3ae41e66a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:31.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.611+0000 7f3aea1ff700 1 --2- 172.21.15.169:0/1361568871 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ae40cd460 0x7f3ae41e4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:31.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.611+0000 7f3aea1ff700 1 -- 172.21.15.169:0/1361568871 >> 172.21.15.169:0/1361568871 conn(0x7f3ae40719a0 msgr2=0x7f3ae40b6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:31.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.611+0000 7f3aea1ff700 1 -- 172.21.15.169:0/1361568871 shutdown_connections 2023-11-28T11:20:31.613 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.611+0000 7f3aea1ff700 1 -- 172.21.15.169:0/1361568871 wait complete. 2023-11-28T11:20:31.629 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new 9c459b7b-b821-4b4b-b91e-6aaae271457c 6 2023-11-28T11:20:31.689 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-11-28T11:20:31.689+0000 7f2166e55240 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T11:20:31.694 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-28T11:20:31.693+0000 7f8f2f6df240 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T11:20:31.784 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-28T11:20:31.783+0000 7f8f2f6df240 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T11:20:31.825 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.824+0000 7f427edff700 1 Processor -- start 2023-11-28T11:20:31.825 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.825+0000 7f427edff700 1 -- start start 2023-11-28T11:20:31.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.826+0000 7f427edff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42780cd460 0x7f427806c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:31.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.826+0000 7f427edff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f42780d0d60 0x7f42780d11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:31.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.826+0000 7f427edff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42780b88e0 con 0x7f42780b8510 2023-11-28T11:20:31.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.826+0000 7f427edff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42780cdbd0 con 0x7f42780d0d60 2023-11-28T11:20:31.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.826+0000 7f427edff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42780d02d0 con 0x7f42780cd460 2023-11-28T11:20:31.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.826+0000 7f427d39c700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f42780d0d60 0x7f42780d11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:31.829 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.826+0000 7f4277fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42780cd460 0x7f427806c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:31.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.826+0000 7f427cb9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f42780b8510 0x7f427806b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.169:43342/0 (socket says 172.21.15.169:43342) 2023-11-28T11:20:31.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.826+0000 7f427cb9b700 1 -- 172.21.15.169:0/147116454 learned_addr learned my addr 172.21.15.169:0/147116454 (peer_addr_for_me v1:172.21.15.169:0/0) 2023-11-28T11:20:31.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.826+0000 7f427d39c700 1 -- 172.21.15.169:0/147116454 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42780cd460 msgr2=0x7f427806c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:31.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.826+0000 7f427d39c700 1 --2- 172.21.15.169:0/147116454 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42780cd460 0x7f427806c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:31.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.826+0000 7f427d39c700 1 -- 172.21.15.169:0/147116454 >> v1:172.21.15.92:6789/0 conn(0x7f42780b8510 legacy=0x7f427806b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:31.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.827+0000 7f427d39c700 1 -- 172.21.15.169:0/147116454 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42780cd8f0 con 0x7f42780d0d60 2023-11-28T11:20:31.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.827+0000 7f4277fff700 1 --2- 172.21.15.169:0/147116454 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42780cd460 0x7f427806c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:20:31.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.826+0000 7f42777fe700 1 -- 172.21.15.169:0/147116454 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 509193463 0 0) 0x7f42780b88e0 con 0x7f42780b8510 2023-11-28T11:20:31.830 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.827+0000 7f427d39c700 1 --2- 172.21.15.169:0/147116454 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42780d0d60 0x7f42780d11e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4270009dc0 tx=0x7f427000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=669ca3c6d1179c60 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:31.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.827+0000 7f42777fe700 1 -- 172.21.15.169:0/147116454 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427000e070 con 0x7f42780d0d60 2023-11-28T11:20:31.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.827+0000 7f42777fe700 1 -- 172.21.15.169:0/147116454 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f427000bae0 con 0x7f42780d0d60 2023-11-28T11:20:31.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.827+0000 7f42777fe700 1 -- 172.21.15.169:0/147116454 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427000be90 con 0x7f42780d0d60 2023-11-28T11:20:31.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.828+0000 7f427edff700 1 -- 172.21.15.169:0/147116454 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42780d0d60 msgr2=0x7f42780d11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:31.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.828+0000 7f427edff700 1 --2- 172.21.15.169:0/147116454 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42780d0d60 0x7f42780d11e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4270009dc0 tx=0x7f427000b330 comp rx=0 tx=0).stop 2023-11-28T11:20:31.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.828+0000 7f427edff700 1 -- 172.21.15.169:0/147116454 shutdown_connections 2023-11-28T11:20:31.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.828+0000 7f427edff700 1 --2- 172.21.15.169:0/147116454 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42780d0d60 0x7f42780d11e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:31.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.828+0000 7f427edff700 1 --2- 172.21.15.169:0/147116454 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42780cd460 0x7f427806c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:31.834 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.828+0000 7f427edff700 1 -- 172.21.15.169:0/147116454 >> 172.21.15.169:0/147116454 conn(0x7f42780719a0 msgr2=0x7f4278071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:31.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.828+0000 7f427edff700 1 -- 172.21.15.169:0/147116454 shutdown_connections 2023-11-28T11:20:31.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.828+0000 7f427edff700 1 -- 172.21.15.169:0/147116454 wait complete. 2023-11-28T11:20:31.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.830+0000 7f427edff700 1 Processor -- start 2023-11-28T11:20:31.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.830+0000 7f427edff700 1 -- start start 2023-11-28T11:20:31.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.831+0000 7f427edff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42780b8510 0x7f427814d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:31.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.831+0000 7f427edff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42780cd460 0x7f42781577b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:31.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.831+0000 7f427edff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42780d0d60 0x7f42780cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:31.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.831+0000 7f427edff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42781561f0 con 0x7f42780cd460 2023-11-28T11:20:31.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.831+0000 7f427edff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4278155670 con 0x7f42780d0d60 2023-11-28T11:20:31.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.831+0000 7f427edff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4278155950 con 0x7f42780b8510 2023-11-28T11:20:31.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.831+0000 7f427cb9b700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42780b8510 0x7f427814d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:31.835 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.831+0000 7f427d39c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42780d0d60 0x7f42780cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:31.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.831+0000 7f427cb9b700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42780b8510 0x7f427814d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.169:49824/0 (socket says 172.21.15.169:49824) 2023-11-28T11:20:31.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.831+0000 7f427cb9b700 1 -- 172.21.15.169:0/2247097423 learned_addr learned my addr 172.21.15.169:0/2247097423 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-11-28T11:20:31.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.831+0000 7f4277fff700 1 --2- 172.21.15.169:0/2247097423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42780cd460 0x7f42781577b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:31.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.831+0000 7f427cb9b700 1 -- 172.21.15.169:0/2247097423 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42780d0d60 msgr2=0x7f42780cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:31.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.831+0000 7f427cb9b700 1 --2- 172.21.15.169:0/2247097423 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42780d0d60 0x7f42780cc6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:31.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.831+0000 7f427cb9b700 1 -- 172.21.15.169:0/2247097423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42780cd460 msgr2=0x7f42781577b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:31.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.831+0000 7f427cb9b700 1 --2- 172.21.15.169:0/2247097423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42780cd460 0x7f42781577b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:31.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.831+0000 7f427cb9b700 1 -- 172.21.15.169:0/2247097423 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42640094a0 con 0x7f42780b8510 2023-11-28T11:20:31.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.832+0000 7f427cb9b700 1 --2- 172.21.15.169:0/2247097423 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42780b8510 0x7f427814d140 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f426c0011e0 tx=0x7f426c00ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:31.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.832+0000 7f42757fa700 1 -- 172.21.15.169:0/2247097423 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c004c80 con 0x7f42780b8510 2023-11-28T11:20:31.836 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.832+0000 7f42757fa700 1 -- 172.21.15.169:0/2247097423 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f426c004e00 con 0x7f42780b8510 2023-11-28T11:20:31.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.832+0000 7f427edff700 1 -- 172.21.15.169:0/2247097423 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4270009a70 con 0x7f42780b8510 2023-11-28T11:20:31.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.832+0000 7f42757fa700 1 -- 172.21.15.169:0/2247097423 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c007310 con 0x7f42780b8510 2023-11-28T11:20:31.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.832+0000 7f427edff700 1 -- 172.21.15.169:0/2247097423 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42780cfff0 con 0x7f42780b8510 2023-11-28T11:20:31.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.832+0000 7f42757fa700 1 -- 172.21.15.169:0/2247097423 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f426c01a020 con 0x7f42780b8510 2023-11-28T11:20:31.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.832+0000 7f42757fa700 1 -- 172.21.15.169:0/2247097423 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f426c01d3f0 con 0x7f42780b8510 2023-11-28T11:20:31.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.833+0000 7f427edff700 1 -- 172.21.15.169:0/2247097423 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42780cfd10 con 0x7f42780b8510 2023-11-28T11:20:31.837 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.836+0000 7f42757fa700 1 -- 172.21.15.169:0/2247097423 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f426c01d5b0 con 0x7f42780b8510 2023-11-28T11:20:31.868 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-11-28T11:20:31.867+0000 7f2166e55240 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T11:20:31.878 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-28T11:20:31.877+0000 7f8f2f6df240 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T11:20:31.923 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.923+0000 7f427edff700 1 -- 172.21.15.169:0/2247097423 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd new", "uuid": "9c459b7b-b821-4b4b-b91e-6aaae271457c", "id": 6} v 0) v1 -- 0x7f42780ce400 con 0x7f42780b8510 2023-11-28T11:20:31.931 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.930+0000 7f42757fa700 1 -- 172.21.15.169:0/2247097423 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9c459b7b-b821-4b4b-b91e-6aaae271457c", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f426c007540 con 0x7f42780b8510 2023-11-28T11:20:31.931 INFO:teuthology.orchestra.run.smithi169.stdout:6 2023-11-28T11:20:31.933 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.932+0000 7f427edff700 1 -- 172.21.15.169:0/2247097423 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42780b8510 msgr2=0x7f427814d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:31.933 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.932+0000 7f427edff700 1 --2- 172.21.15.169:0/2247097423 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42780b8510 0x7f427814d140 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f426c0011e0 tx=0x7f426c00ca30 comp rx=0 tx=0).stop 2023-11-28T11:20:31.933 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.932+0000 7f427edff700 1 -- 172.21.15.169:0/2247097423 shutdown_connections 2023-11-28T11:20:31.933 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.932+0000 7f427edff700 1 --2- 172.21.15.169:0/2247097423 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42780d0d60 0x7f42780cc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:31.933 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.932+0000 7f427edff700 1 --2- 172.21.15.169:0/2247097423 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42780b8510 0x7f427814d140 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:31.933 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.932+0000 7f427edff700 1 --2- 172.21.15.169:0/2247097423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42780cd460 0x7f42781577b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:31.934 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.932+0000 7f427edff700 1 -- 172.21.15.169:0/2247097423 >> 172.21.15.169:0/2247097423 conn(0x7f42780719a0 msgr2=0x7f427806b970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:31.934 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.932+0000 7f427edff700 1 -- 172.21.15.169:0/2247097423 shutdown_connections 2023-11-28T11:20:31.934 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:31.932+0000 7f427edff700 1 -- 172.21.15.169:0/2247097423 wait complete. 2023-11-28T11:20:31.950 DEBUG:teuthology.orchestra.run.smithi169:> sudo ceph --cluster ceph osd new 1ca9aa4a-755d-4e28-a937-4d8dccab4df3 7 2023-11-28T11:20:31.963 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-11-28T11:20:31.963+0000 7f2166e55240 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T11:20:32.066 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-11-28T11:20:32.065+0000 7f2166e55240 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T11:20:32.102 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-28T11:20:32.101+0000 7f8f2f6df240 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T11:20:32.150 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.150+0000 7fec8efff700 1 Processor -- start 2023-11-28T11:20:32.151 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.150+0000 7fec8efff700 1 -- start start 2023-11-28T11:20:32.153 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.151+0000 7fec8efff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fec880cd460 0x7fec8806c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:32.153 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.151+0000 7fec8efff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec880b88e0 con 0x7fec880b8510 2023-11-28T11:20:32.153 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.151+0000 7fec8efff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec880cd8f0 con 0x7fec880d0d60 2023-11-28T11:20:32.153 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.151+0000 7fec8efff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec880cdbd0 con 0x7fec880cd460 2023-11-28T11:20:32.153 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.151+0000 7fec87fff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fec880cd460 0x7fec8806c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:32.153 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.152+0000 7fec87fff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fec880cd460 0x7fec8806c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.169:56592/0 (socket says 172.21.15.169:56592) 2023-11-28T11:20:32.153 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.152+0000 7fec87fff700 1 -- 172.21.15.169:0/1228002605 learned_addr learned my addr 172.21.15.169:0/1228002605 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-11-28T11:20:32.153 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.152+0000 7fec8d59c700 1 --1- 172.21.15.169:0/1228002605 >> v1:172.21.15.169:6789/0 conn(0x7fec880d0d60 0x7fec880d1200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.169:35876/0 (socket says 172.21.15.169:35876) 2023-11-28T11:20:32.153 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.152+0000 7fec87fff700 1 -- 172.21.15.169:0/1228002605 >> v1:172.21.15.169:6789/0 conn(0x7fec880d0d60 legacy=0x7fec880d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:32.153 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.152+0000 7fec87fff700 1 -- 172.21.15.169:0/1228002605 >> v1:172.21.15.92:6789/0 conn(0x7fec880b8510 legacy=0x7fec8806b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:32.153 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.152+0000 7fec87fff700 1 -- 172.21.15.169:0/1228002605 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec880cfd10 con 0x7fec880cd460 2023-11-28T11:20:32.153 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.152+0000 7fec87fff700 1 --2- 172.21.15.169:0/1228002605 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec880cd460 0x7fec8806c080 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fec740098f0 tx=0x7fec74009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=966a89abff41774 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:32.155 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.152+0000 7fec877fe700 1 -- 172.21.15.169:0/1228002605 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7400b8a0 con 0x7fec880cd460 2023-11-28T11:20:32.155 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.152+0000 7fec877fe700 1 -- 172.21.15.169:0/1228002605 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec7400ba00 con 0x7fec880cd460 2023-11-28T11:20:32.155 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.152+0000 7fec877fe700 1 -- 172.21.15.169:0/1228002605 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7400bd90 con 0x7fec880cd460 2023-11-28T11:20:32.155 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.153+0000 7fec8efff700 1 -- 172.21.15.169:0/1228002605 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec880cd460 msgr2=0x7fec8806c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:32.155 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.153+0000 7fec8efff700 1 --2- 172.21.15.169:0/1228002605 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec880cd460 0x7fec8806c080 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fec740098f0 tx=0x7fec74009c00 comp rx=0 tx=0).stop 2023-11-28T11:20:32.155 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.153+0000 7fec8efff700 1 -- 172.21.15.169:0/1228002605 shutdown_connections 2023-11-28T11:20:32.155 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.153+0000 7fec8efff700 1 --2- 172.21.15.169:0/1228002605 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec880cd460 0x7fec8806c080 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:32.155 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.153+0000 7fec8efff700 1 -- 172.21.15.169:0/1228002605 >> 172.21.15.169:0/1228002605 conn(0x7fec880719a0 msgr2=0x7fec88071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:32.155 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.153+0000 7fec8efff700 1 -- 172.21.15.169:0/1228002605 shutdown_connections 2023-11-28T11:20:32.155 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.153+0000 7fec8efff700 1 -- 172.21.15.169:0/1228002605 wait complete. 2023-11-28T11:20:32.156 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.155+0000 7fec8efff700 1 Processor -- start 2023-11-28T11:20:32.156 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.155+0000 7fec8efff700 1 -- start start 2023-11-28T11:20:32.157 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.156+0000 7fec8efff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec880b8510 0x7fec881e8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:32.157 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.156+0000 7fec8efff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec880cd460 0x7fec880b7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:32.157 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.156+0000 7fec8efff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec880d0d60 0x7fec8815a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:32.157 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.156+0000 7fec8efff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec881e1900 con 0x7fec880b8510 2023-11-28T11:20:32.158 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.156+0000 7fec8efff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec881e51e0 con 0x7fec880d0d60 2023-11-28T11:20:32.158 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.156+0000 7fec8efff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec881e5480 con 0x7fec880cd460 2023-11-28T11:20:32.158 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.157+0000 7fec87fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec880cd460 0x7fec880b7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:32.158 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.157+0000 7fec8d59c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec880d0d60 0x7fec8815a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:32.158 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.157+0000 7fec87fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec880cd460 0x7fec880b7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.169:49836/0 (socket says 172.21.15.169:49836) 2023-11-28T11:20:32.158 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.157+0000 7fec87fff700 1 -- 172.21.15.169:0/2639984349 learned_addr learned my addr 172.21.15.169:0/2639984349 (peer_addr_for_me v2:172.21.15.169:0/0) 2023-11-28T11:20:32.158 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.157+0000 7fec8cd9b700 1 --2- 172.21.15.169:0/2639984349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec880b8510 0x7fec881e8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:32.160 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.157+0000 7fec87fff700 1 -- 172.21.15.169:0/2639984349 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec880d0d60 msgr2=0x7fec8815a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:32.160 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.157+0000 7fec87fff700 1 --2- 172.21.15.169:0/2639984349 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec880d0d60 0x7fec8815a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:32.160 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.157+0000 7fec87fff700 1 -- 172.21.15.169:0/2639984349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec880b8510 msgr2=0x7fec881e8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:32.160 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.157+0000 7fec87fff700 1 --2- 172.21.15.169:0/2639984349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec880b8510 0x7fec881e8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:32.160 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.157+0000 7fec87fff700 1 -- 172.21.15.169:0/2639984349 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec74009550 con 0x7fec880cd460 2023-11-28T11:20:32.160 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.157+0000 7fec8d59c700 1 --2- 172.21.15.169:0/2639984349 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec880d0d60 0x7fec8815a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:20:32.160 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.157+0000 7fec87fff700 1 --2- 172.21.15.169:0/2639984349 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec880cd460 0x7fec880b7960 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fec74000c00 tx=0x7fec74004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:32.160 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.158+0000 7fec857fa700 1 -- 172.21.15.169:0/2639984349 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec74004380 con 0x7fec880cd460 2023-11-28T11:20:32.160 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.158+0000 7fec857fa700 1 -- 172.21.15.169:0/2639984349 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec7401f070 con 0x7fec880cd460 2023-11-28T11:20:32.160 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.158+0000 7fec857fa700 1 -- 172.21.15.169:0/2639984349 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec740036a0 con 0x7fec880cd460 2023-11-28T11:20:32.161 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.158+0000 7fec8efff700 1 -- 172.21.15.169:0/2639984349 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec880d29c0 con 0x7fec880cd460 2023-11-28T11:20:32.161 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.158+0000 7fec8efff700 1 -- 172.21.15.169:0/2639984349 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec880cfff0 con 0x7fec880cd460 2023-11-28T11:20:32.161 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.158+0000 7fec857fa700 1 -- 172.21.15.169:0/2639984349 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fec7402a020 con 0x7fec880cd460 2023-11-28T11:20:32.161 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.158+0000 7fec857fa700 1 -- 172.21.15.169:0/2639984349 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7fec7401a5e0 con 0x7fec880cd460 2023-11-28T11:20:32.161 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.159+0000 7fec8efff700 1 -- 172.21.15.169:0/2639984349 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec880d2b50 con 0x7fec880cd460 2023-11-28T11:20:32.162 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.161+0000 7fec857fa700 1 -- 172.21.15.169:0/2639984349 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fec74015020 con 0x7fec880cd460 2023-11-28T11:20:32.200 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-28T11:20:32.199+0000 7f8f2f6df240 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-28T11:20:32.248 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.248+0000 7fec8efff700 1 -- 172.21.15.169:0/2639984349 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd new", "uuid": "1ca9aa4a-755d-4e28-a937-4d8dccab4df3", "id": 7} v 0) v1 -- 0x7fec88067880 con 0x7fec880cd460 2023-11-28T11:20:32.257 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.256+0000 7fec857fa700 1 -- 172.21.15.169:0/2639984349 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1ca9aa4a-755d-4e28-a937-4d8dccab4df3", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fec740038d0 con 0x7fec880cd460 2023-11-28T11:20:32.257 INFO:teuthology.orchestra.run.smithi169.stdout:7 2023-11-28T11:20:32.258 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.257+0000 7fec8efff700 1 -- 172.21.15.169:0/2639984349 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec880cd460 msgr2=0x7fec880b7960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:32.258 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.257+0000 7fec8efff700 1 --2- 172.21.15.169:0/2639984349 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec880cd460 0x7fec880b7960 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fec74000c00 tx=0x7fec74004750 comp rx=0 tx=0).stop 2023-11-28T11:20:32.258 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.257+0000 7fec8efff700 1 -- 172.21.15.169:0/2639984349 shutdown_connections 2023-11-28T11:20:32.259 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.257+0000 7fec8efff700 1 --2- 172.21.15.169:0/2639984349 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec880d0d60 0x7fec8815a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:32.259 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.257+0000 7fec8efff700 1 --2- 172.21.15.169:0/2639984349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec880b8510 0x7fec881e8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:32.259 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.257+0000 7fec8efff700 1 --2- 172.21.15.169:0/2639984349 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec880cd460 0x7fec880b7960 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:32.259 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.258+0000 7fec8efff700 1 -- 172.21.15.169:0/2639984349 >> 172.21.15.169:0/2639984349 conn(0x7fec880719a0 msgr2=0x7fec880b6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:32.259 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.258+0000 7fec8efff700 1 -- 172.21.15.169:0/2639984349 shutdown_connections 2023-11-28T11:20:32.259 INFO:teuthology.orchestra.run.smithi169.stderr:2023-11-28T11:20:32.258+0000 7fec8efff700 1 -- 172.21.15.169:0/2639984349 wait complete. 2023-11-28T11:20:32.276 INFO:tasks.ceph.osd.0:Restarting daemon 2023-11-28T11:20:32.276 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-11-28T11:20:32.280 INFO:tasks.ceph.osd.0:Started 2023-11-28T11:20:32.280 INFO:tasks.ceph.osd.1:Restarting daemon 2023-11-28T11:20:32.280 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-11-28T11:20:32.284 INFO:tasks.ceph.osd.1:Started 2023-11-28T11:20:32.284 INFO:tasks.ceph.osd.2:Restarting daemon 2023-11-28T11:20:32.285 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-11-28T11:20:32.290 INFO:tasks.ceph.osd.2:Started 2023-11-28T11:20:32.290 INFO:tasks.ceph.osd.3:Restarting daemon 2023-11-28T11:20:32.290 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-11-28T11:20:32.295 INFO:tasks.ceph.osd.3:Started 2023-11-28T11:20:32.295 INFO:tasks.ceph.osd.4:Restarting daemon 2023-11-28T11:20:32.295 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-11-28T11:20:32.303 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-11-28T11:20:32.302+0000 7f2166e55240 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T11:20:32.308 INFO:tasks.ceph.osd.4:Started 2023-11-28T11:20:32.308 INFO:tasks.ceph.osd.5:Restarting daemon 2023-11-28T11:20:32.308 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-11-28T11:20:32.314 INFO:tasks.ceph.osd.5:Started 2023-11-28T11:20:32.314 INFO:tasks.ceph.osd.6:Restarting daemon 2023-11-28T11:20:32.314 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-11-28T11:20:32.319 INFO:tasks.ceph.osd.6:Started 2023-11-28T11:20:32.319 INFO:tasks.ceph.osd.7:Restarting daemon 2023-11-28T11:20:32.320 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-11-28T11:20:32.326 INFO:tasks.ceph.osd.7:Started 2023-11-28T11:20:32.326 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:20:32.333 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-28T11:20:32.327+0000 7f8f2f6df240 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T11:20:32.432 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-11-28T11:20:32.431+0000 7f2166e55240 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-28T11:20:32.566 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-11-28T11:20:32.565+0000 7f2166e55240 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T11:20:32.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.594+0000 7f8c303ff700 1 Processor -- start 2023-11-28T11:20:32.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.594+0000 7f8c303ff700 1 -- start start 2023-11-28T11:20:32.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.595+0000 7f8c303ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8c280d2df0 0x7f8c280cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:32.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.595+0000 7f8c303ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8c280b4fc0 0x7f8c280d31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:32.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.595+0000 7f8c303ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c280d1950 con 0x7f8c280b1c60 2023-11-28T11:20:32.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.595+0000 7f8c303ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c280d3d30 con 0x7f8c280b4fc0 2023-11-28T11:20:32.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.595+0000 7f8c303ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c280b2500 con 0x7f8c280d2df0 2023-11-28T11:20:32.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.596+0000 7f8c2d99a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8c280b4fc0 0x7f8c280d31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:32.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.596+0000 7f8c2e19b700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8c280d2df0 0x7f8c280cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:32.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.596+0000 7f8c2d99a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8c280b4fc0 0x7f8c280d31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45156/0 (socket says 172.21.15.92:45156) 2023-11-28T11:20:32.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.596+0000 7f8c2e19b700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8c280d2df0 0x7f8c280cd620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.92:45528/0 (socket says 172.21.15.92:45528) 2023-11-28T11:20:32.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.596+0000 7f8c2d99a700 1 -- 172.21.15.92:0/2468994753 learned_addr learned my addr 172.21.15.92:0/2468994753 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:20:32.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.596+0000 7f8c2e99c700 1 --1- 172.21.15.92:0/2468994753 >> v1:172.21.15.169:6789/0 conn(0x7f8c280b1c60 0x7f8c280cf010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.92:55430/0 (socket says 172.21.15.92:55430) 2023-11-28T11:20:32.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.596+0000 7f8c2d199700 1 -- 172.21.15.92:0/2468994753 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1558782075 0 0) 0x7f8c280d1950 con 0x7f8c280b1c60 2023-11-28T11:20:32.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.596+0000 7f8c2d99a700 1 -- 172.21.15.92:0/2468994753 >> v2:172.21.15.169:3300/0 conn(0x7f8c280d2df0 msgr2=0x7f8c280cd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:32.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.596+0000 7f8c2d99a700 1 --2- 172.21.15.92:0/2468994753 >> v2:172.21.15.169:3300/0 conn(0x7f8c280d2df0 0x7f8c280cd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:32.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.596+0000 7f8c2d99a700 1 -- 172.21.15.92:0/2468994753 >> v1:172.21.15.169:6789/0 conn(0x7f8c280b1c60 legacy=0x7f8c280cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:32.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.596+0000 7f8c2d99a700 1 -- 172.21.15.92:0/2468994753 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c280d3a50 con 0x7f8c280b4fc0 2023-11-28T11:20:32.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.596+0000 7f8c2e19b700 1 --2- 172.21.15.92:0/2468994753 >> v2:172.21.15.169:3300/0 conn(0x7f8c280d2df0 0x7f8c280cd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:20:32.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.596+0000 7f8c2d99a700 1 --2- 172.21.15.92:0/2468994753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c280b4fc0 0x7f8c280d31c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8c280d27f0 tx=0x7f8c18009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=860058e2e9982f9d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:32.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.597+0000 7f8c2d199700 1 -- 172.21.15.92:0/2468994753 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c18011830 con 0x7f8c280b4fc0 2023-11-28T11:20:32.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.597+0000 7f8c2d199700 1 -- 172.21.15.92:0/2468994753 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c18011990 con 0x7f8c280b4fc0 2023-11-28T11:20:32.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.597+0000 7f8c2d199700 1 -- 172.21.15.92:0/2468994753 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c18011d20 con 0x7f8c280b4fc0 2023-11-28T11:20:32.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.597+0000 7f8c303ff700 1 -- 172.21.15.92:0/2468994753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c280b4fc0 msgr2=0x7f8c280d31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:32.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.597+0000 7f8c303ff700 1 --2- 172.21.15.92:0/2468994753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c280b4fc0 0x7f8c280d31c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8c280d27f0 tx=0x7f8c18009bd0 comp rx=0 tx=0).stop 2023-11-28T11:20:32.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.597+0000 7f8c303ff700 1 -- 172.21.15.92:0/2468994753 shutdown_connections 2023-11-28T11:20:32.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.597+0000 7f8c303ff700 1 --2- 172.21.15.92:0/2468994753 >> v2:172.21.15.169:3300/0 conn(0x7f8c280d2df0 0x7f8c280cd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:32.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.597+0000 7f8c303ff700 1 --2- 172.21.15.92:0/2468994753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c280b4fc0 0x7f8c280d31c0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:32.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.597+0000 7f8c303ff700 1 -- 172.21.15.92:0/2468994753 >> 172.21.15.92:0/2468994753 conn(0x7f8c280719a0 msgr2=0x7f8c28071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:32.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.597+0000 7f8c303ff700 1 -- 172.21.15.92:0/2468994753 shutdown_connections 2023-11-28T11:20:32.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.597+0000 7f8c303ff700 1 -- 172.21.15.92:0/2468994753 wait complete. 2023-11-28T11:20:32.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.599+0000 7f8c303ff700 1 Processor -- start 2023-11-28T11:20:32.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.599+0000 7f8c303ff700 1 -- start start 2023-11-28T11:20:32.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.600+0000 7f8c303ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c280b1c60 0x7f8c280c9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:32.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.600+0000 7f8c303ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c280b4fc0 0x7f8c281513e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:32.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.600+0000 7f8c303ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c280d2df0 0x7f8c2815e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:32.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.600+0000 7f8c303ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c28156c80 con 0x7f8c280d2df0 2023-11-28T11:20:32.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.600+0000 7f8c303ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c280ccd70 con 0x7f8c280b4fc0 2023-11-28T11:20:32.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.600+0000 7f8c303ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c281f5f70 con 0x7f8c280b1c60 2023-11-28T11:20:32.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.600+0000 7f8c2e99c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c280d2df0 0x7f8c2815e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:32.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.601+0000 7f8c2e99c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c280d2df0 0x7f8c2815e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45172/0 (socket says 172.21.15.92:45172) 2023-11-28T11:20:32.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.601+0000 7f8c2e99c700 1 -- 172.21.15.92:0/3803510546 learned_addr learned my addr 172.21.15.92:0/3803510546 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:20:32.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.601+0000 7f8c2d99a700 1 --2- 172.21.15.92:0/3803510546 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c280b4fc0 0x7f8c281513e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:32.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.601+0000 7f8c2e19b700 1 --2- 172.21.15.92:0/3803510546 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c280b1c60 0x7f8c280c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:32.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.601+0000 7f8c2e19b700 1 -- 172.21.15.92:0/3803510546 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c280b4fc0 msgr2=0x7f8c281513e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:32.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.601+0000 7f8c2e19b700 1 --2- 172.21.15.92:0/3803510546 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c280b4fc0 0x7f8c281513e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:32.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.601+0000 7f8c2e19b700 1 -- 172.21.15.92:0/3803510546 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c280d2df0 msgr2=0x7f8c2815e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:32.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.601+0000 7f8c2e19b700 1 --2- 172.21.15.92:0/3803510546 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c280d2df0 0x7f8c2815e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:32.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.601+0000 7f8c2e19b700 1 -- 172.21.15.92:0/3803510546 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c240094a0 con 0x7f8c280b1c60 2023-11-28T11:20:32.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.601+0000 7f8c2e19b700 1 --2- 172.21.15.92:0/3803510546 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c280b1c60 0x7f8c280c9900 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8c2400e900 tx=0x7f8c2400ec10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:32.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.601+0000 7f8c1effd700 1 -- 172.21.15.92:0/3803510546 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c24013680 con 0x7f8c280b1c60 2023-11-28T11:20:32.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.601+0000 7f8c1effd700 1 -- 172.21.15.92:0/3803510546 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c240137e0 con 0x7f8c280b1c60 2023-11-28T11:20:32.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.602+0000 7f8c303ff700 1 -- 172.21.15.92:0/3803510546 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c18009550 con 0x7f8c280b1c60 2023-11-28T11:20:32.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.602+0000 7f8c1effd700 1 -- 172.21.15.92:0/3803510546 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c24013b70 con 0x7f8c280b1c60 2023-11-28T11:20:32.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.602+0000 7f8c303ff700 1 -- 172.21.15.92:0/3803510546 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c280b88f0 con 0x7f8c280b1c60 2023-11-28T11:20:32.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.602+0000 7f8c1effd700 1 -- 172.21.15.92:0/3803510546 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f8c24023070 con 0x7f8c280b1c60 2023-11-28T11:20:32.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.602+0000 7f8c1effd700 1 -- 172.21.15.92:0/3803510546 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f8c24013cd0 con 0x7f8c280b1c60 2023-11-28T11:20:32.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.603+0000 7f8c303ff700 1 -- 172.21.15.92:0/3803510546 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c280d15e0 con 0x7f8c280b1c60 2023-11-28T11:20:32.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.605+0000 7f8c1effd700 1 -- 172.21.15.92:0/3803510546 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f8c24021060 con 0x7f8c280b1c60 2023-11-28T11:20:32.633 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-28T11:20:32.632+0000 7f8f2f6df240 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T11:20:32.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.677+0000 7f8c303ff700 1 -- 172.21.15.92:0/3803510546 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c280b21b0 con 0x7f8c280b1c60 2023-11-28T11:20:32.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.678+0000 7f8c1effd700 1 -- 172.21.15.92:0/3803510546 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f8c24017070 con 0x7f8c280b1c60 2023-11-28T11:20:32.678 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:20:32.679 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":10,"fsid":"68956e0c-c16f-47cd-a220-f98364b0831d","created":"2023-11-28T11:20:28.697573+0000","modified":"2023-11-28T11:20:32.249767+0000","last_up_change":"0.000000","last_in_change":"2023-11-28T11:20:32.249767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6abe483f-0e03-4a26-b9cf-62041291747e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"13c1de73-1a8c-4d7e-8a28-9dae52dfe089","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"25788a55-112f-4f00-810d-2bc37cf76778","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"adf54e24-33a9-41fe-9ce3-98a5ab6c158e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ccb9b747-f399-44ea-9a18-9db3152227e8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"308a1e63-75aa-4d7d-8a7c-cecc24714fd5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"9c459b7b-b821-4b4b-b91e-6aaae271457c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"1ca9aa4a-755d-4e28-a937-4d8dccab4df3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:20:32.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.679+0000 7f8c303ff700 1 -- 172.21.15.92:0/3803510546 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c280b1c60 msgr2=0x7f8c280c9900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:32.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.679+0000 7f8c303ff700 1 --2- 172.21.15.92:0/3803510546 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c280b1c60 0x7f8c280c9900 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8c2400e900 tx=0x7f8c2400ec10 comp rx=0 tx=0).stop 2023-11-28T11:20:32.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.679+0000 7f8c303ff700 1 -- 172.21.15.92:0/3803510546 shutdown_connections 2023-11-28T11:20:32.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.679+0000 7f8c303ff700 1 --2- 172.21.15.92:0/3803510546 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c280d2df0 0x7f8c2815e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:32.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.679+0000 7f8c303ff700 1 --2- 172.21.15.92:0/3803510546 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c280b1c60 0x7f8c280c9900 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:32.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.679+0000 7f8c303ff700 1 --2- 172.21.15.92:0/3803510546 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c280b4fc0 0x7f8c281513e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:32.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.679+0000 7f8c303ff700 1 -- 172.21.15.92:0/3803510546 >> 172.21.15.92:0/3803510546 conn(0x7f8c280719a0 msgr2=0x7f8c280b26b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:32.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.679+0000 7f8c303ff700 1 -- 172.21.15.92:0/3803510546 shutdown_connections 2023-11-28T11:20:32.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:32.679+0000 7f8c303ff700 1 -- 172.21.15.92:0/3803510546 wait complete. 2023-11-28T11:20:32.716 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-11-28T11:20:32.716 INFO:tasks.ceph:Waiting for OSDs to come up 2023-11-28T11:20:32.739 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-28T11:20:32.738+0000 7f8f2f6df240 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T11:20:32.796 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-11-28T11:20:32.795+0000 7f2166e55240 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T11:20:32.900 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-11-28T11:20:32.900+0000 7f2166e55240 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T11:20:32.991 INFO:tasks.ceph.osd.2.smithi092.stderr:2023-11-28T11:20:32.990+0000 7ff0371ca680 -1 Falling back to public interface 2023-11-28T11:20:33.008 INFO:tasks.ceph.osd.5.smithi169.stderr:2023-11-28T11:20:33.007+0000 7faadeff8680 -1 Falling back to public interface 2023-11-28T11:20:33.015 INFO:tasks.ceph.osd.0.smithi092.stderr:2023-11-28T11:20:33.014+0000 7f30263f4680 -1 Falling back to public interface 2023-11-28T11:20:33.016 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-28T11:20:33.016+0000 7f8064802680 -1 Falling back to public interface 2023-11-28T11:20:33.021 INFO:tasks.ceph.osd.6.smithi169.stderr:2023-11-28T11:20:33.021+0000 7f0282557680 -1 Falling back to public interface 2023-11-28T11:20:33.037 INFO:tasks.ceph.osd.3.smithi092.stderr:2023-11-28T11:20:33.037+0000 7fb68e659680 -1 Falling back to public interface 2023-11-28T11:20:33.038 INFO:tasks.ceph.osd.4.smithi169.stderr:2023-11-28T11:20:33.038+0000 7f8d163ea680 -1 Falling back to public interface 2023-11-28T11:20:33.042 INFO:tasks.ceph.osd.7.smithi169.stderr:2023-11-28T11:20:33.042+0000 7f49097be680 -1 Falling back to public interface 2023-11-28T11:20:33.098 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-28T11:20:33.097+0000 7f8f2f6df240 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T11:20:33.187 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-28T11:20:33.186+0000 7f8f2f6df240 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T11:20:33.257 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-11-28T11:20:33.257+0000 7f2166e55240 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T11:20:33.351 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-11-28T11:20:33.350+0000 7f2166e55240 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T11:20:33.523 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-28T11:20:33.700 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.699+0000 7fb1393ff700 1 Processor -- start 2023-11-28T11:20:33.700 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.699+0000 7fb1393ff700 1 -- start start 2023-11-28T11:20:33.702 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.700+0000 7fb1393ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1340b2410 0x7fb1340cf320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:33.702 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.700+0000 7fb1393ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb1340b4c10 0x7fb1340b5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:33.703 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.701+0000 7fb1393ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1340b27e0 con 0x7fb1340b6520 2023-11-28T11:20:33.703 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.701+0000 7fb1393ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1340b5160 con 0x7fb1340b4c10 2023-11-28T11:20:33.703 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.701+0000 7fb1393ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1340b5440 con 0x7fb1340b2410 2023-11-28T11:20:33.703 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.701+0000 7fb1327fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb1340b4c10 0x7fb1340b5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:33.703 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.701+0000 7fb1337fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb1340b6520 0x7fb1341609f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:42994/0 (socket says 172.21.15.92:42994) 2023-11-28T11:20:33.703 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.701+0000 7fb1337fe700 1 -- 172.21.15.92:0/1805344889 learned_addr learned my addr 172.21.15.92:0/1805344889 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:20:33.703 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.701+0000 7fb132ffd700 1 --2- 172.21.15.92:0/1805344889 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1340b2410 0x7fb1340cf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:33.703 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.701+0000 7fb1327fc700 1 -- 172.21.15.92:0/1805344889 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1340b2410 msgr2=0x7fb1340cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:33.703 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.701+0000 7fb1327fc700 1 --2- 172.21.15.92:0/1805344889 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1340b2410 0x7fb1340cf320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:33.703 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.701+0000 7fb1327fc700 1 -- 172.21.15.92:0/1805344889 >> v1:172.21.15.92:6789/0 conn(0x7fb1340b6520 legacy=0x7fb1341609f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:33.704 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.701+0000 7fb1327fc700 1 -- 172.21.15.92:0/1805344889 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1340b5720 con 0x7fb1340b4c10 2023-11-28T11:20:33.704 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.701+0000 7fb131ffb700 1 -- 172.21.15.92:0/1805344889 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3402344298 0 0) 0x7fb1340b27e0 con 0x7fb1340b6520 2023-11-28T11:20:33.704 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.702+0000 7fb1327fc700 1 --2- 172.21.15.92:0/1805344889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1340b4c10 0x7fb1340b5fe0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb128009870 tx=0x7fb128009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fdab797a12c3c44 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:33.704 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.702+0000 7fb131ffb700 1 -- 172.21.15.92:0/1805344889 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb128011830 con 0x7fb1340b4c10 2023-11-28T11:20:33.704 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.702+0000 7fb131ffb700 1 -- 172.21.15.92:0/1805344889 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb128011990 con 0x7fb1340b4c10 2023-11-28T11:20:33.704 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.702+0000 7fb131ffb700 1 -- 172.21.15.92:0/1805344889 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb128011d20 con 0x7fb1340b4c10 2023-11-28T11:20:33.705 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.702+0000 7fb1393ff700 1 -- 172.21.15.92:0/1805344889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1340b4c10 msgr2=0x7fb1340b5fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:33.705 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.702+0000 7fb1393ff700 1 --2- 172.21.15.92:0/1805344889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1340b4c10 0x7fb1340b5fe0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb128009870 tx=0x7fb128009b80 comp rx=0 tx=0).stop 2023-11-28T11:20:33.705 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.702+0000 7fb1393ff700 1 -- 172.21.15.92:0/1805344889 shutdown_connections 2023-11-28T11:20:33.705 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.702+0000 7fb1393ff700 1 --2- 172.21.15.92:0/1805344889 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1340b2410 0x7fb1340cf320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:33.705 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.702+0000 7fb1393ff700 1 --2- 172.21.15.92:0/1805344889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1340b4c10 0x7fb1340b5fe0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:33.706 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.703+0000 7fb1393ff700 1 -- 172.21.15.92:0/1805344889 >> 172.21.15.92:0/1805344889 conn(0x7fb13408a1f0 msgr2=0x7fb13408a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:33.706 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.703+0000 7fb1393ff700 1 -- 172.21.15.92:0/1805344889 shutdown_connections 2023-11-28T11:20:33.706 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.703+0000 7fb1393ff700 1 -- 172.21.15.92:0/1805344889 wait complete. 2023-11-28T11:20:33.706 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.704+0000 7fb1393ff700 1 Processor -- start 2023-11-28T11:20:33.706 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.705+0000 7fb1393ff700 1 -- start start 2023-11-28T11:20:33.708 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.705+0000 7fb1393ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1340b2410 0x7fb134151d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:33.708 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.705+0000 7fb1393ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1340b4c10 0x7fb1340d1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:33.708 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.706+0000 7fb1393ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1340b6520 0x7fb13415a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:33.708 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.706+0000 7fb1393ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb13415d390 con 0x7fb1340b2410 2023-11-28T11:20:33.709 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.706+0000 7fb1393ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb134153220 con 0x7fb1340b4c10 2023-11-28T11:20:33.709 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.706+0000 7fb1393ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb134153500 con 0x7fb1340b6520 2023-11-28T11:20:33.709 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.706+0000 7fb132ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1340b2410 0x7fb134151d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:33.709 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.706+0000 7fb132ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1340b2410 0x7fb134151d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45186/0 (socket says 172.21.15.92:45186) 2023-11-28T11:20:33.709 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.706+0000 7fb132ffd700 1 -- 172.21.15.92:0/490205401 learned_addr learned my addr 172.21.15.92:0/490205401 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:20:33.709 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.706+0000 7fb1327fc700 1 --2- 172.21.15.92:0/490205401 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1340b4c10 0x7fb1340d1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:33.709 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.706+0000 7fb1337fe700 1 --2- 172.21.15.92:0/490205401 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1340b6520 0x7fb13415a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:33.709 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.706+0000 7fb132ffd700 1 -- 172.21.15.92:0/490205401 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1340b6520 msgr2=0x7fb13415a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:33.709 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.706+0000 7fb132ffd700 1 --2- 172.21.15.92:0/490205401 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1340b6520 0x7fb13415a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:33.709 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.706+0000 7fb132ffd700 1 -- 172.21.15.92:0/490205401 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1340b4c10 msgr2=0x7fb1340d1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:33.709 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.706+0000 7fb132ffd700 1 --2- 172.21.15.92:0/490205401 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1340b4c10 0x7fb1340d1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:33.709 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.706+0000 7fb132ffd700 1 -- 172.21.15.92:0/490205401 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1280094a0 con 0x7fb1340b2410 2023-11-28T11:20:33.709 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.706+0000 7fb1337fe700 1 --2- 172.21.15.92:0/490205401 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1340b6520 0x7fb13415a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:20:33.710 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.706+0000 7fb132ffd700 1 --2- 172.21.15.92:0/490205401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1340b2410 0x7fb134151d30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb12400e870 tx=0x7fb12400eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:33.710 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.707+0000 7fb11b7fe700 1 -- 172.21.15.92:0/490205401 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb124009590 con 0x7fb1340b2410 2023-11-28T11:20:33.710 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.707+0000 7fb11b7fe700 1 -- 172.21.15.92:0/490205401 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb124013680 con 0x7fb1340b2410 2023-11-28T11:20:33.710 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.707+0000 7fb11b7fe700 1 -- 172.21.15.92:0/490205401 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1240139f0 con 0x7fb1340b2410 2023-11-28T11:20:33.711 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.707+0000 7fb1393ff700 1 -- 172.21.15.92:0/490205401 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1341604d0 con 0x7fb1340b2410 2023-11-28T11:20:33.711 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.707+0000 7fb1393ff700 1 -- 172.21.15.92:0/490205401 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1340b5a00 con 0x7fb1340b2410 2023-11-28T11:20:33.711 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.707+0000 7fb11b7fe700 1 -- 172.21.15.92:0/490205401 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fb124019030 con 0x7fb1340b2410 2023-11-28T11:20:33.711 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.707+0000 7fb11b7fe700 1 -- 172.21.15.92:0/490205401 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fb124006790 con 0x7fb1340b2410 2023-11-28T11:20:33.711 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.708+0000 7fb1393ff700 1 -- 172.21.15.92:0/490205401 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1340cf860 con 0x7fb1340b2410 2023-11-28T11:20:33.711 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.710+0000 7fb11b7fe700 1 -- 172.21.15.92:0/490205401 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fb124021060 con 0x7fb1340b2410 2023-11-28T11:20:33.783 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.783+0000 7fb1393ff700 1 -- 172.21.15.92:0/490205401 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb13404e700 con 0x7fb1340b2410 2023-11-28T11:20:33.784 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.783+0000 7fb11b7fe700 1 -- 172.21.15.92:0/490205401 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fb124017070 con 0x7fb1340b2410 2023-11-28T11:20:33.784 INFO:teuthology.misc.health.smithi092.stdout: 2023-11-28T11:20:33.784 INFO:teuthology.misc.health.smithi092.stdout:{"epoch":10,"fsid":"68956e0c-c16f-47cd-a220-f98364b0831d","created":"2023-11-28T11:20:28.697573+0000","modified":"2023-11-28T11:20:32.249767+0000","last_up_change":"0.000000","last_in_change":"2023-11-28T11:20:32.249767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6abe483f-0e03-4a26-b9cf-62041291747e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"13c1de73-1a8c-4d7e-8a28-9dae52dfe089","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"25788a55-112f-4f00-810d-2bc37cf76778","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"adf54e24-33a9-41fe-9ce3-98a5ab6c158e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ccb9b747-f399-44ea-9a18-9db3152227e8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"308a1e63-75aa-4d7d-8a7c-cecc24714fd5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"9c459b7b-b821-4b4b-b91e-6aaae271457c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"1ca9aa4a-755d-4e28-a937-4d8dccab4df3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:20:33.785 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.784+0000 7fb1393ff700 1 -- 172.21.15.92:0/490205401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1340b2410 msgr2=0x7fb134151d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:33.785 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.784+0000 7fb1393ff700 1 --2- 172.21.15.92:0/490205401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1340b2410 0x7fb134151d30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb12400e870 tx=0x7fb12400eb80 comp rx=0 tx=0).stop 2023-11-28T11:20:33.785 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.784+0000 7fb1393ff700 1 -- 172.21.15.92:0/490205401 shutdown_connections 2023-11-28T11:20:33.786 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.784+0000 7fb1393ff700 1 --2- 172.21.15.92:0/490205401 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1340b6520 0x7fb13415a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:33.786 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.784+0000 7fb1393ff700 1 --2- 172.21.15.92:0/490205401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1340b2410 0x7fb134151d30 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:33.786 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.784+0000 7fb1393ff700 1 --2- 172.21.15.92:0/490205401 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1340b4c10 0x7fb1340d1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:33.786 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.785+0000 7fb1393ff700 1 -- 172.21.15.92:0/490205401 >> 172.21.15.92:0/490205401 conn(0x7fb13408a1f0 msgr2=0x7fb134155890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:33.786 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.785+0000 7fb1393ff700 1 -- 172.21.15.92:0/490205401 shutdown_connections 2023-11-28T11:20:33.786 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:33.785+0000 7fb1393ff700 1 -- 172.21.15.92:0/490205401 wait complete. 2023-11-28T11:20:33.800 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-11-28T11:20:33.819 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-28T11:20:33.818+0000 7f8f2f6df240 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T11:20:33.971 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-11-28T11:20:33.970+0000 7f2166e55240 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T11:20:34.007 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-28T11:20:34.007+0000 7f8f2f6df240 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T11:20:34.159 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-11-28T11:20:34.158+0000 7f2166e55240 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T11:20:34.386 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-28T11:20:34.385+0000 7f8f2f6df240 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T11:20:34.546 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-11-28T11:20:34.545+0000 7f2166e55240 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T11:20:35.048 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-28T11:20:35.047+0000 7f8f2f6df240 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T11:20:35.211 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-11-28T11:20:35.211+0000 7f2166e55240 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T11:20:35.236 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-28T11:20:35.235+0000 7f8f2f6df240 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T11:20:35.400 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-11-28T11:20:35.399+0000 7f2166e55240 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T11:20:35.688 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-28T11:20:35.687+0000 7f8f2f6df240 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-28T11:20:35.837 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-28T11:20:35.836+0000 7f8f2f6df240 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T11:20:35.921 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-11-28T11:20:35.918+0000 7f2166e55240 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-28T11:20:36.117 INFO:tasks.ceph.mgr.x.smithi169.stderr:2023-11-28T11:20:36.116+0000 7f2166e55240 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T11:20:36.510 INFO:tasks.ceph.osd.2.smithi092.stderr:2023-11-28T11:20:36.509+0000 7ff0371ca680 -1 osd.2 0 log_to_monitors true 2023-11-28T11:20:36.585 INFO:tasks.ceph.osd.6.smithi169.stderr:2023-11-28T11:20:36.585+0000 7f0282557680 -1 osd.6 0 log_to_monitors true 2023-11-28T11:20:36.597 INFO:tasks.ceph.osd.0.smithi092.stderr:2023-11-28T11:20:36.597+0000 7f30263f4680 -1 osd.0 0 log_to_monitors true 2023-11-28T11:20:36.772 INFO:tasks.ceph.osd.5.smithi169.stderr:2023-11-28T11:20:36.771+0000 7faadeff8680 -1 osd.5 0 log_to_monitors true 2023-11-28T11:20:36.832 INFO:tasks.ceph.osd.3.smithi092.stderr:2023-11-28T11:20:36.832+0000 7fb68e659680 -1 osd.3 0 log_to_monitors true 2023-11-28T11:20:36.860 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-28T11:20:36.859+0000 7f8ee8ffa700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-11-28T11:20:36.863 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-11-28T11:20:36.863+0000 7f8ee8ffa700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-11-28T11:20:36.881 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-28T11:20:36.881+0000 7f8064802680 -1 osd.1 0 log_to_monitors true 2023-11-28T11:20:36.887 INFO:tasks.ceph.osd.4.smithi169.stderr:2023-11-28T11:20:36.886+0000 7f8d163ea680 -1 osd.4 0 log_to_monitors true 2023-11-28T11:20:36.895 INFO:tasks.ceph.osd.7.smithi169.stderr:2023-11-28T11:20:36.895+0000 7f49097be680 -1 osd.7 0 log_to_monitors true 2023-11-28T11:20:40.251 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-28T11:20:40.249+0000 7f80535f7700 -1 osd.1 0 waiting for initial osdmap 2023-11-28T11:20:40.272 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-28T11:20:40.270+0000 7f804adfa700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T11:20:40.281 INFO:tasks.ceph.osd.0.smithi092.stderr:2023-11-28T11:20:40.280+0000 7f30151f7700 -1 osd.0 0 waiting for initial osdmap 2023-11-28T11:20:40.296 INFO:tasks.ceph.osd.0.smithi092.stderr:2023-11-28T11:20:40.294+0000 7f300c9fa700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T11:20:40.318 INFO:tasks.ceph.osd.7.smithi169.stderr:2023-11-28T11:20:40.315+0000 7f48f85f7700 -1 osd.7 0 waiting for initial osdmap 2023-11-28T11:20:40.322 INFO:tasks.ceph.osd.2.smithi092.stderr:2023-11-28T11:20:40.320+0000 7ff025ff7700 -1 osd.2 0 waiting for initial osdmap 2023-11-28T11:20:40.331 INFO:tasks.ceph.osd.6.smithi169.stderr:2023-11-28T11:20:40.330+0000 7f02713f7700 -1 osd.6 0 waiting for initial osdmap 2023-11-28T11:20:40.334 INFO:tasks.ceph.osd.5.smithi169.stderr:2023-11-28T11:20:40.333+0000 7faace5f8700 -1 osd.5 0 waiting for initial osdmap 2023-11-28T11:20:40.339 INFO:tasks.ceph.osd.2.smithi092.stderr:2023-11-28T11:20:40.337+0000 7ff01d7fa700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T11:20:40.341 INFO:tasks.ceph.osd.7.smithi169.stderr:2023-11-28T11:20:40.339+0000 7f48efdfa700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T11:20:40.346 INFO:tasks.ceph.osd.6.smithi169.stderr:2023-11-28T11:20:40.345+0000 7f0268bfa700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T11:20:40.352 INFO:tasks.ceph.osd.4.smithi169.stderr:2023-11-28T11:20:40.352+0000 7f8d051f7700 -1 osd.4 0 waiting for initial osdmap 2023-11-28T11:20:40.353 INFO:tasks.ceph.osd.5.smithi169.stderr:2023-11-28T11:20:40.352+0000 7faac55fa700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T11:20:40.359 INFO:tasks.ceph.osd.3.smithi092.stderr:2023-11-28T11:20:40.358+0000 7fb67d5f7700 -1 osd.3 0 waiting for initial osdmap 2023-11-28T11:20:40.370 INFO:tasks.ceph.osd.4.smithi169.stderr:2023-11-28T11:20:40.370+0000 7f8cfc9fa700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T11:20:40.376 INFO:tasks.ceph.osd.3.smithi092.stderr:2023-11-28T11:20:40.375+0000 7fb674dfa700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T11:20:40.606 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-28T11:20:40.780 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.778+0000 7fc0c15ff700 1 Processor -- start 2023-11-28T11:20:40.781 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.779+0000 7fc0c15ff700 1 -- start start 2023-11-28T11:20:40.783 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.780+0000 7fc0c15ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc0bc0d1fb0 0x7fc0bc0b24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:40.783 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.780+0000 7fc0c15ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0bc15d990 0x7fc0bc160c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:40.783 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.780+0000 7fc0c15ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0bc1604e0 con 0x7fc0bc161190 2023-11-28T11:20:40.783 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.780+0000 7fc0c15ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0bc15de00 con 0x7fc0bc0d1fb0 2023-11-28T11:20:40.783 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.780+0000 7fc0c15ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0bc0cd7d0 con 0x7fc0bc15d990 2023-11-28T11:20:40.783 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.780+0000 7fc0bb7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc0bc161190 0x7fc0bc0d2d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:41094/0 (socket says 172.21.15.92:41094) 2023-11-28T11:20:40.783 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.780+0000 7fc0bb7fe700 1 -- 172.21.15.92:0/3815051229 learned_addr learned my addr 172.21.15.92:0/3815051229 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:20:40.784 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.780+0000 7fc0b3fff700 1 --2- 172.21.15.92:0/3815051229 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0bc15d990 0x7fc0bc160c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:40.784 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.780+0000 7fc0baffd700 1 --2- 172.21.15.92:0/3815051229 >> v2:172.21.15.169:3300/0 conn(0x7fc0bc0d1fb0 0x7fc0bc0b24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:40.784 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.780+0000 7fc0ba7fc700 1 -- 172.21.15.92:0/3815051229 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4098612629 0 0) 0x7fc0bc1604e0 con 0x7fc0bc161190 2023-11-28T11:20:40.784 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.781+0000 7fc0ba7fc700 1 -- 172.21.15.92:0/3815051229 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0a80035b0 con 0x7fc0bc161190 2023-11-28T11:20:40.784 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.781+0000 7fc0b3fff700 1 -- 172.21.15.92:0/3815051229 >> v2:172.21.15.169:3300/0 conn(0x7fc0bc0d1fb0 msgr2=0x7fc0bc0b24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:40.784 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.781+0000 7fc0b3fff700 1 --2- 172.21.15.92:0/3815051229 >> v2:172.21.15.169:3300/0 conn(0x7fc0bc0d1fb0 0x7fc0bc0b24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:40.784 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.781+0000 7fc0b3fff700 1 -- 172.21.15.92:0/3815051229 >> v1:172.21.15.92:6789/0 conn(0x7fc0bc161190 legacy=0x7fc0bc0d2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:40.784 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.781+0000 7fc0b3fff700 1 -- 172.21.15.92:0/3815051229 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0bc0cdab0 con 0x7fc0bc15d990 2023-11-28T11:20:40.784 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.781+0000 7fc0baffd700 1 --2- 172.21.15.92:0/3815051229 >> v2:172.21.15.169:3300/0 conn(0x7fc0bc0d1fb0 0x7fc0bc0b24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:20:40.784 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.781+0000 7fc0b3fff700 1 --2- 172.21.15.92:0/3815051229 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0bc15d990 0x7fc0bc160c50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4009840 tx=0x7fc0a4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bf3d984dcce5af5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:40.785 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.781+0000 7fc0ba7fc700 1 -- 172.21.15.92:0/3815051229 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a4011830 con 0x7fc0bc15d990 2023-11-28T11:20:40.785 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.781+0000 7fc0ba7fc700 1 -- 172.21.15.92:0/3815051229 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0a4011990 con 0x7fc0bc15d990 2023-11-28T11:20:40.785 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.781+0000 7fc0ba7fc700 1 -- 172.21.15.92:0/3815051229 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a4011d20 con 0x7fc0bc15d990 2023-11-28T11:20:40.786 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.782+0000 7fc0c15ff700 1 -- 172.21.15.92:0/3815051229 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0bc15d990 msgr2=0x7fc0bc160c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:40.786 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.782+0000 7fc0c15ff700 1 --2- 172.21.15.92:0/3815051229 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0bc15d990 0x7fc0bc160c50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4009840 tx=0x7fc0a4009b50 comp rx=0 tx=0).stop 2023-11-28T11:20:40.786 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.782+0000 7fc0c15ff700 1 -- 172.21.15.92:0/3815051229 shutdown_connections 2023-11-28T11:20:40.786 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.782+0000 7fc0c15ff700 1 --2- 172.21.15.92:0/3815051229 >> v2:172.21.15.169:3300/0 conn(0x7fc0bc0d1fb0 0x7fc0bc0b24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:40.786 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.782+0000 7fc0c15ff700 1 --2- 172.21.15.92:0/3815051229 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0bc15d990 0x7fc0bc160c50 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:40.786 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.782+0000 7fc0c15ff700 1 -- 172.21.15.92:0/3815051229 >> 172.21.15.92:0/3815051229 conn(0x7fc0bc071950 msgr2=0x7fc0bc071d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:40.786 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.782+0000 7fc0c15ff700 1 -- 172.21.15.92:0/3815051229 shutdown_connections 2023-11-28T11:20:40.786 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.782+0000 7fc0c15ff700 1 -- 172.21.15.92:0/3815051229 wait complete. 2023-11-28T11:20:40.786 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.784+0000 7fc0c15ff700 1 Processor -- start 2023-11-28T11:20:40.787 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.784+0000 7fc0c15ff700 1 -- start start 2023-11-28T11:20:40.788 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.785+0000 7fc0c15ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0bc0d1fb0 0x7fc0bc1e6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:40.788 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.785+0000 7fc0c15ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0bc15d990 0x7fc0bc1f5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:40.789 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.785+0000 7fc0c15ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0bc161190 0x7fc0bc1f62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:40.789 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.785+0000 7fc0c15ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0bc1f56c0 con 0x7fc0bc0d1fb0 2023-11-28T11:20:40.789 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.785+0000 7fc0c15ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0bc1f67e0 con 0x7fc0bc161190 2023-11-28T11:20:40.789 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.785+0000 7fc0c15ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0bc0b53d0 con 0x7fc0bc15d990 2023-11-28T11:20:40.789 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.785+0000 7fc0baffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0bc0d1fb0 0x7fc0bc1e6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:40.789 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.785+0000 7fc0baffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0bc0d1fb0 0x7fc0bc1e6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44702/0 (socket says 172.21.15.92:44702) 2023-11-28T11:20:40.789 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.785+0000 7fc0baffd700 1 -- 172.21.15.92:0/1744372030 learned_addr learned my addr 172.21.15.92:0/1744372030 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:20:40.789 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.785+0000 7fc0bb7fe700 1 --2- 172.21.15.92:0/1744372030 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0bc161190 0x7fc0bc1f62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:40.789 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.785+0000 7fc0baffd700 1 -- 172.21.15.92:0/1744372030 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0bc15d990 msgr2=0x7fc0bc1f5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:40.789 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.785+0000 7fc0baffd700 1 --2- 172.21.15.92:0/1744372030 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0bc15d990 0x7fc0bc1f5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:40.789 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.785+0000 7fc0baffd700 1 -- 172.21.15.92:0/1744372030 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0bc161190 msgr2=0x7fc0bc1f62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:40.789 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.785+0000 7fc0baffd700 1 --2- 172.21.15.92:0/1744372030 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0bc161190 0x7fc0bc1f62a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:40.790 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.785+0000 7fc0baffd700 1 -- 172.21.15.92:0/1744372030 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0ac0094a0 con 0x7fc0bc0d1fb0 2023-11-28T11:20:40.790 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.785+0000 7fc0baffd700 1 --2- 172.21.15.92:0/1744372030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0bc0d1fb0 0x7fc0bc1e6270 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac00ebe0 tx=0x7fc0ac009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:40.790 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.786+0000 7fc0b37fe700 1 -- 172.21.15.92:0/1744372030 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac00c910 con 0x7fc0bc0d1fb0 2023-11-28T11:20:40.790 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.786+0000 7fc0b37fe700 1 -- 172.21.15.92:0/1744372030 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0ac00ca70 con 0x7fc0bc0d1fb0 2023-11-28T11:20:40.790 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.786+0000 7fc0b37fe700 1 -- 172.21.15.92:0/1744372030 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac00cde0 con 0x7fc0bc0d1fb0 2023-11-28T11:20:40.790 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.786+0000 7fc0c15ff700 1 -- 172.21.15.92:0/1744372030 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0a40094a0 con 0x7fc0bc0d1fb0 2023-11-28T11:20:40.790 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.786+0000 7fc0c15ff700 1 -- 172.21.15.92:0/1744372030 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0bc160730 con 0x7fc0bc0d1fb0 2023-11-28T11:20:40.790 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.787+0000 7fc0c15ff700 1 -- 172.21.15.92:0/1744372030 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0bc0cdd90 con 0x7fc0bc0d1fb0 2023-11-28T11:20:40.791 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.788+0000 7fc0b37fe700 1 -- 172.21.15.92:0/1744372030 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7fc0ac019030 con 0x7fc0bc0d1fb0 2023-11-28T11:20:40.791 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.788+0000 7fc0b37fe700 1 --2- 172.21.15.92:0/1744372030 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc09c0682a0 0x7fc09c06a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:40.791 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.789+0000 7fc0b37fe700 1 -- 172.21.15.92:0/1744372030 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7fc0ac084ee0 con 0x7fc0bc0d1fb0 2023-11-28T11:20:40.792 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.789+0000 7fc0b3fff700 1 --2- 172.21.15.92:0/1744372030 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc09c0682a0 0x7fc09c06a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:40.792 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.789+0000 7fc0b3fff700 1 --2- 172.21.15.92:0/1744372030 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc09c0682a0 0x7fc09c06a750 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc0b8e90 tx=0x7fc0a4010000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:40.795 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.794+0000 7fc0b37fe700 1 -- 172.21.15.92:0/1744372030 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc0ac052590 con 0x7fc0bc0d1fb0 2023-11-28T11:20:40.993 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.992+0000 7fc0c15ff700 1 -- 172.21.15.92:0/1744372030 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc0bc0d3490 con 0x7fc0bc0d1fb0 2023-11-28T11:20:40.994 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.992+0000 7fc0b37fe700 1 -- 172.21.15.92:0/1744372030 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fc0ac021090 con 0x7fc0bc0d1fb0 2023-11-28T11:20:40.994 INFO:teuthology.misc.health.smithi092.stdout: 2023-11-28T11:20:40.994 INFO:teuthology.misc.health.smithi092.stdout:{"epoch":12,"fsid":"68956e0c-c16f-47cd-a220-f98364b0831d","created":"2023-11-28T11:20:28.697573+0000","modified":"2023-11-28T11:20:37.929644+0000","last_up_change":"0.000000","last_in_change":"2023-11-28T11:20:32.249767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6abe483f-0e03-4a26-b9cf-62041291747e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"13c1de73-1a8c-4d7e-8a28-9dae52dfe089","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"25788a55-112f-4f00-810d-2bc37cf76778","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"adf54e24-33a9-41fe-9ce3-98a5ab6c158e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ccb9b747-f399-44ea-9a18-9db3152227e8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"308a1e63-75aa-4d7d-8a7c-cecc24714fd5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"9c459b7b-b821-4b4b-b91e-6aaae271457c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"1ca9aa4a-755d-4e28-a937-4d8dccab4df3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:20:40.997 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.995+0000 7fc0c15ff700 1 -- 172.21.15.92:0/1744372030 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc09c0682a0 msgr2=0x7fc09c06a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:40.997 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.995+0000 7fc0c15ff700 1 --2- 172.21.15.92:0/1744372030 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc09c0682a0 0x7fc09c06a750 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc0b8e90 tx=0x7fc0a4010000 comp rx=0 tx=0).stop 2023-11-28T11:20:40.997 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.995+0000 7fc0c15ff700 1 -- 172.21.15.92:0/1744372030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0bc0d1fb0 msgr2=0x7fc0bc1e6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:40.997 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.995+0000 7fc0c15ff700 1 --2- 172.21.15.92:0/1744372030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0bc0d1fb0 0x7fc0bc1e6270 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac00ebe0 tx=0x7fc0ac009ec0 comp rx=0 tx=0).stop 2023-11-28T11:20:40.997 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.995+0000 7fc0c15ff700 1 -- 172.21.15.92:0/1744372030 shutdown_connections 2023-11-28T11:20:40.997 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.995+0000 7fc0c15ff700 1 --2- 172.21.15.92:0/1744372030 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0bc161190 0x7fc0bc1f62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:40.997 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.995+0000 7fc0c15ff700 1 --2- 172.21.15.92:0/1744372030 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc09c0682a0 0x7fc09c06a750 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:40.997 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.995+0000 7fc0c15ff700 1 --2- 172.21.15.92:0/1744372030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0bc0d1fb0 0x7fc0bc1e6270 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:40.998 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.995+0000 7fc0c15ff700 1 --2- 172.21.15.92:0/1744372030 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0bc15d990 0x7fc0bc1f5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:40.998 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.995+0000 7fc0c15ff700 1 -- 172.21.15.92:0/1744372030 >> 172.21.15.92:0/1744372030 conn(0x7fc0bc071950 msgr2=0x7fc0bc06bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:40.998 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.995+0000 7fc0c15ff700 1 -- 172.21.15.92:0/1744372030 shutdown_connections 2023-11-28T11:20:40.998 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:40.995+0000 7fc0c15ff700 1 -- 172.21.15.92:0/1744372030 wait complete. 2023-11-28T11:20:41.012 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-11-28T11:20:47.712 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:20:47.717 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:20:47.821 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-28T11:20:48.015 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.013+0000 7f46817ff700 1 Processor -- start 2023-11-28T11:20:48.016 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.014+0000 7f46817ff700 1 -- start start 2023-11-28T11:20:48.017 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.015+0000 7f46817ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f467c0b24b0 0x7f467c15d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:48.017 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.015+0000 7f46817ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f467c1604e0 con 0x7f467c0d2d80 2023-11-28T11:20:48.017 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.015+0000 7f46817ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f467c161360 con 0x7f467c0d1fb0 2023-11-28T11:20:48.017 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.015+0000 7f46817ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f467c0cd7d0 con 0x7f467c0b24b0 2023-11-28T11:20:48.017 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.015+0000 7f4673fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f467c0b24b0 0x7f467c15d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:48.017 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.015+0000 7f467b7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f467c0d2d80 0x7f467c0d0cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:41102/0 (socket says 172.21.15.92:41102) 2023-11-28T11:20:48.017 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.015+0000 7f467b7fe700 1 -- 172.21.15.92:0/3743916961 learned_addr learned my addr 172.21.15.92:0/3743916961 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:20:48.019 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.015+0000 7f467a7fc700 1 -- 172.21.15.92:0/3743916961 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 56349986 0 0) 0x7f467c1604e0 con 0x7f467c0d2d80 2023-11-28T11:20:48.019 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.016+0000 7f467a7fc700 1 -- 172.21.15.92:0/3743916961 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46680035b0 con 0x7f467c0d2d80 2023-11-28T11:20:48.019 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.016+0000 7f4673fff700 1 -- 172.21.15.92:0/3743916961 >> v1:172.21.15.169:6789/0 conn(0x7f467c0d1fb0 legacy=0x7f467c160c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:48.019 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.016+0000 7f467a7fc700 1 -- 172.21.15.92:0/3743916961 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3345010190 0 0) 0x7f467c161360 con 0x7f467c0d1fb0 2023-11-28T11:20:48.019 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.016+0000 7f4673fff700 1 -- 172.21.15.92:0/3743916961 >> v1:172.21.15.92:6789/0 conn(0x7f467c0d2d80 legacy=0x7f467c0d0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:48.019 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.016+0000 7f4673fff700 1 -- 172.21.15.92:0/3743916961 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f467c0cdab0 con 0x7f467c0b24b0 2023-11-28T11:20:48.019 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.016+0000 7f4673fff700 1 --2- 172.21.15.92:0/3743916961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f467c0b24b0 0x7f467c15d990 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4664009840 tx=0x7f4664009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97a9dd3ec3539853 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:48.019 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.016+0000 7f467a7fc700 1 -- 172.21.15.92:0/3743916961 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4664011830 con 0x7f467c0b24b0 2023-11-28T11:20:48.019 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.016+0000 7f467a7fc700 1 -- 172.21.15.92:0/3743916961 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4664011990 con 0x7f467c0b24b0 2023-11-28T11:20:48.020 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.016+0000 7f467a7fc700 1 -- 172.21.15.92:0/3743916961 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4664011d20 con 0x7f467c0b24b0 2023-11-28T11:20:48.020 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.017+0000 7f46817ff700 1 -- 172.21.15.92:0/3743916961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f467c0b24b0 msgr2=0x7f467c15d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:48.020 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.017+0000 7f46817ff700 1 --2- 172.21.15.92:0/3743916961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f467c0b24b0 0x7f467c15d990 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4664009840 tx=0x7f4664009b50 comp rx=0 tx=0).stop 2023-11-28T11:20:48.020 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.017+0000 7f46817ff700 1 -- 172.21.15.92:0/3743916961 shutdown_connections 2023-11-28T11:20:48.020 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.017+0000 7f46817ff700 1 --2- 172.21.15.92:0/3743916961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f467c0b24b0 0x7f467c15d990 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:48.020 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.017+0000 7f46817ff700 1 -- 172.21.15.92:0/3743916961 >> 172.21.15.92:0/3743916961 conn(0x7f467c071950 msgr2=0x7f467c071d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:48.020 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.017+0000 7f46817ff700 1 -- 172.21.15.92:0/3743916961 shutdown_connections 2023-11-28T11:20:48.020 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.017+0000 7f46817ff700 1 -- 172.21.15.92:0/3743916961 wait complete. 2023-11-28T11:20:48.021 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.019+0000 7f46817ff700 1 Processor -- start 2023-11-28T11:20:48.021 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.020+0000 7f46817ff700 1 -- start start 2023-11-28T11:20:48.026 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.021+0000 7f46817ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f467c0b24b0 0x7f467c0c9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:48.026 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.021+0000 7f46817ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f467c0d1fb0 0x7f467c1e6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:48.026 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.021+0000 7f46817ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f467c0d2d80 0x7f467c0b7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:48.026 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.021+0000 7f46817ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467c0c9540 con 0x7f467c0b24b0 2023-11-28T11:20:48.026 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.021+0000 7f46817ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467c1e71e0 con 0x7f467c0d2d80 2023-11-28T11:20:48.026 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.021+0000 7f46817ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f467c152390 con 0x7f467c0d1fb0 2023-11-28T11:20:48.026 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.021+0000 7f467affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f467c0b24b0 0x7f467c0c9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:48.026 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.021+0000 7f467affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f467c0b24b0 0x7f467c0c9000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44758/0 (socket says 172.21.15.92:44758) 2023-11-28T11:20:48.026 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.021+0000 7f467affd700 1 -- 172.21.15.92:0/2692460930 learned_addr learned my addr 172.21.15.92:0/2692460930 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:20:48.027 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.021+0000 7f467affd700 1 -- 172.21.15.92:0/2692460930 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f467c0d1fb0 msgr2=0x7f467c1e6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:48.027 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.021+0000 7f467affd700 1 --2- 172.21.15.92:0/2692460930 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f467c0d1fb0 0x7f467c1e6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:48.027 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.021+0000 7f467affd700 1 -- 172.21.15.92:0/2692460930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f467c0d2d80 msgr2=0x7f467c0b7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:48.027 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.021+0000 7f467affd700 1 --2- 172.21.15.92:0/2692460930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f467c0d2d80 0x7f467c0b7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:48.027 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.021+0000 7f467affd700 1 -- 172.21.15.92:0/2692460930 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46640094a0 con 0x7f467c0b24b0 2023-11-28T11:20:48.027 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.022+0000 7f467affd700 1 --2- 172.21.15.92:0/2692460930 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f467c0b24b0 0x7f467c0c9000 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f466c00c500 tx=0x7f466c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:48.027 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.022+0000 7f46737fe700 1 -- 172.21.15.92:0/2692460930 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f466c0054e0 con 0x7f467c0b24b0 2023-11-28T11:20:48.027 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.022+0000 7f46737fe700 1 -- 172.21.15.92:0/2692460930 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f466c003680 con 0x7f467c0b24b0 2023-11-28T11:20:48.027 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.022+0000 7f46817ff700 1 -- 172.21.15.92:0/2692460930 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f467c160730 con 0x7f467c0b24b0 2023-11-28T11:20:48.027 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.022+0000 7f46737fe700 1 -- 172.21.15.92:0/2692460930 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f466c003a30 con 0x7f467c0b24b0 2023-11-28T11:20:48.027 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.022+0000 7f46817ff700 1 -- 172.21.15.92:0/2692460930 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f467c0cdd90 con 0x7f467c0b24b0 2023-11-28T11:20:48.027 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.023+0000 7f46817ff700 1 -- 172.21.15.92:0/2692460930 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f467c04e700 con 0x7f467c0b24b0 2023-11-28T11:20:48.027 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.024+0000 7f46737fe700 1 -- 172.21.15.92:0/2692460930 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f466c019030 con 0x7f467c0b24b0 2023-11-28T11:20:48.028 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.025+0000 7f46737fe700 1 --2- 172.21.15.92:0/2692460930 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f465c0682f0 0x7f465c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:48.028 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.026+0000 7f46737fe700 1 -- 172.21.15.92:0/2692460930 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f466c084c90 con 0x7f467c0b24b0 2023-11-28T11:20:48.028 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.026+0000 7f4673fff700 1 --2- 172.21.15.92:0/2692460930 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f465c0682f0 0x7f465c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:48.028 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.026+0000 7f4673fff700 1 --2- 172.21.15.92:0/2692460930 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f465c0682f0 0x7f465c06a7a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f467c15ab00 tx=0x7f4664010000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:48.032 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.030+0000 7f46737fe700 1 -- 172.21.15.92:0/2692460930 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f466c051910 con 0x7f467c0b24b0 2023-11-28T11:20:48.225 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.223+0000 7f46817ff700 1 -- 172.21.15.92:0/2692460930 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f467c071ff0 con 0x7f467c0b24b0 2023-11-28T11:20:48.226 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.224+0000 7f46737fe700 1 -- 172.21.15.92:0/2692460930 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12478 (secure 0 0 0) 0x7f466c017070 con 0x7f467c0b24b0 2023-11-28T11:20:48.229 INFO:teuthology.misc.health.smithi092.stdout: 2023-11-28T11:20:48.229 INFO:teuthology.misc.health.smithi092.stdout:{"epoch":16,"fsid":"68956e0c-c16f-47cd-a220-f98364b0831d","created":"2023-11-28T11:20:28.697573+0000","modified":"2023-11-28T11:20:44.292564+0000","last_up_change":"2023-11-28T11:20:41.257011+0000","last_in_change":"2023-11-28T11:20:32.249767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:20:41.925237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6abe483f-0e03-4a26-b9cf-62041291747e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":115823},{"type":"v1","addr":"172.21.15.92:6809","nonce":115823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":115823},{"type":"v1","addr":"172.21.15.92:6813","nonce":115823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":115823},{"type":"v1","addr":"172.21.15.92:6821","nonce":115823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":115823},{"type":"v1","addr":"172.21.15.92:6817","nonce":115823}]},"public_addr":"172.21.15.92:6809/115823","cluster_addr":"172.21.15.92:6813/115823","heartbeat_back_addr":"172.21.15.92:6821/115823","heartbeat_front_addr":"172.21.15.92:6817/115823","state":["exists","up"]},{"osd":1,"uuid":"13c1de73-1a8c-4d7e-8a28-9dae52dfe089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":115824},{"type":"v1","addr":"172.21.15.92:6812","nonce":115824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":115824},{"type":"v1","addr":"172.21.15.92:6816","nonce":115824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":115824},{"type":"v1","addr":"172.21.15.92:6823","nonce":115824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":115824},{"type":"v1","addr":"172.21.15.92:6820","nonce":115824}]},"public_addr":"172.21.15.92:6812/115824","cluster_addr":"172.21.15.92:6816/115824","heartbeat_back_addr":"172.21.15.92:6823/115824","heartbeat_front_addr":"172.21.15.92:6820/115824","state":["exists","up"]},{"osd":2,"uuid":"25788a55-112f-4f00-810d-2bc37cf76778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":115821},{"type":"v1","addr":"172.21.15.92:6801","nonce":115821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":115821},{"type":"v1","addr":"172.21.15.92:6803","nonce":115821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":115821},{"type":"v1","addr":"172.21.15.92:6807","nonce":115821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":115821},{"type":"v1","addr":"172.21.15.92:6805","nonce":115821}]},"public_addr":"172.21.15.92:6801/115821","cluster_addr":"172.21.15.92:6803/115821","heartbeat_back_addr":"172.21.15.92:6807/115821","heartbeat_front_addr":"172.21.15.92:6805/115821","state":["exists","up"]},{"osd":3,"uuid":"adf54e24-33a9-41fe-9ce3-98a5ab6c158e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":115825},{"type":"v1","addr":"172.21.15.92:6825","nonce":115825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":115825},{"type":"v1","addr":"172.21.15.92:6827","nonce":115825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":115825},{"type":"v1","addr":"172.21.15.92:6831","nonce":115825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":115825},{"type":"v1","addr":"172.21.15.92:6829","nonce":115825}]},"public_addr":"172.21.15.92:6825/115825","cluster_addr":"172.21.15.92:6827/115825","heartbeat_back_addr":"172.21.15.92:6831/115825","heartbeat_front_addr":"172.21.15.92:6829/115825","state":["exists","up"]},{"osd":4,"uuid":"ccb9b747-f399-44ea-9a18-9db3152227e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":116035},{"type":"v1","addr":"172.21.15.169:6817","nonce":116035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":116035},{"type":"v1","addr":"172.21.15.169:6819","nonce":116035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":116035},{"type":"v1","addr":"172.21.15.169:6826","nonce":116035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":116035},{"type":"v1","addr":"172.21.15.169:6822","nonce":116035}]},"public_addr":"172.21.15.169:6817/116035","cluster_addr":"172.21.15.169:6819/116035","heartbeat_back_addr":"172.21.15.169:6826/116035","heartbeat_front_addr":"172.21.15.169:6822/116035","state":["exists","up"]},{"osd":5,"uuid":"308a1e63-75aa-4d7d-8a7c-cecc24714fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":116033},{"type":"v1","addr":"172.21.15.169:6801","nonce":116033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":116033},{"type":"v1","addr":"172.21.15.169:6803","nonce":116033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":116033},{"type":"v1","addr":"172.21.15.169:6807","nonce":116033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":116033},{"type":"v1","addr":"172.21.15.169:6805","nonce":116033}]},"public_addr":"172.21.15.169:6801/116033","cluster_addr":"172.21.15.169:6803/116033","heartbeat_back_addr":"172.21.15.169:6807/116033","heartbeat_front_addr":"172.21.15.169:6805/116033","state":["exists","up"]},{"osd":6,"uuid":"9c459b7b-b821-4b4b-b91e-6aaae271457c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":116034},{"type":"v1","addr":"172.21.15.169:6809","nonce":116034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":116034},{"type":"v1","addr":"172.21.15.169:6811","nonce":116034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":116034},{"type":"v1","addr":"172.21.15.169:6815","nonce":116034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":116034},{"type":"v1","addr":"172.21.15.169:6813","nonce":116034}]},"public_addr":"172.21.15.169:6809/116034","cluster_addr":"172.21.15.169:6811/116034","heartbeat_back_addr":"172.21.15.169:6815/116034","heartbeat_front_addr":"172.21.15.169:6813/116034","state":["exists","up"]},{"osd":7,"uuid":"1ca9aa4a-755d-4e28-a937-4d8dccab4df3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":116044},{"type":"v1","addr":"172.21.15.169:6823","nonce":116044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6825","nonce":116044},{"type":"v1","addr":"172.21.15.169:6827","nonce":116044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":116044},{"type":"v1","addr":"172.21.15.169:6831","nonce":116044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":116044},{"type":"v1","addr":"172.21.15.169:6829","nonce":116044}]},"public_addr":"172.21.15.169:6823/116044","cluster_addr":"172.21.15.169:6827/116044","heartbeat_back_addr":"172.21.15.169:6831/116044","heartbeat_front_addr":"172.21.15.169:6829/116044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.586781+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.869778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.508594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.860493+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.872267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.746933+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.611520+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.871446+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:20:48.230 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.226+0000 7f46817ff700 1 -- 172.21.15.92:0/2692460930 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f465c0682f0 msgr2=0x7f465c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:48.230 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.226+0000 7f46817ff700 1 --2- 172.21.15.92:0/2692460930 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f465c0682f0 0x7f465c06a7a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f467c15ab00 tx=0x7f4664010000 comp rx=0 tx=0).stop 2023-11-28T11:20:48.230 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.226+0000 7f46817ff700 1 -- 172.21.15.92:0/2692460930 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f467c0b24b0 msgr2=0x7f467c0c9000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:48.231 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.226+0000 7f46817ff700 1 --2- 172.21.15.92:0/2692460930 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f467c0b24b0 0x7f467c0c9000 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f466c00c500 tx=0x7f466c00c8c0 comp rx=0 tx=0).stop 2023-11-28T11:20:48.232 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.227+0000 7f46817ff700 1 -- 172.21.15.92:0/2692460930 shutdown_connections 2023-11-28T11:20:48.232 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.227+0000 7f46817ff700 1 --2- 172.21.15.92:0/2692460930 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f467c0d2d80 0x7f467c0b7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:48.232 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.227+0000 7f46817ff700 1 --2- 172.21.15.92:0/2692460930 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f465c0682f0 0x7f465c06a7a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:48.232 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.227+0000 7f46817ff700 1 --2- 172.21.15.92:0/2692460930 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f467c0b24b0 0x7f467c0c9000 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:48.232 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.227+0000 7f46817ff700 1 --2- 172.21.15.92:0/2692460930 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f467c0d1fb0 0x7f467c1e6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:48.232 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.227+0000 7f46817ff700 1 -- 172.21.15.92:0/2692460930 >> 172.21.15.92:0/2692460930 conn(0x7f467c071950 msgr2=0x7f467c15f810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:48.232 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.227+0000 7f46817ff700 1 -- 172.21.15.92:0/2692460930 shutdown_connections 2023-11-28T11:20:48.232 INFO:teuthology.misc.health.smithi092.stderr:2023-11-28T11:20:48.227+0000 7f46817ff700 1 -- 172.21.15.92:0/2692460930 wait complete. 2023-11-28T11:20:48.244 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-11-28T11:20:48.244 INFO:tasks.ceph:Creating RBD pool 2023-11-28T11:20:48.244 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd pool create rbd 8 2023-11-28T11:20:48.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.435+0000 7f1d699ff700 1 Processor -- start 2023-11-28T11:20:48.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.435+0000 7f1d699ff700 1 -- start start 2023-11-28T11:20:48.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.435+0000 7f1d699ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1d640b8510 0x7f1d6406b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:48.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.435+0000 7f1d699ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d640b88e0 con 0x7f1d640cd460 2023-11-28T11:20:48.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.435+0000 7f1d699ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d640cd8f0 con 0x7f1d640d0d60 2023-11-28T11:20:48.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.435+0000 7f1d699ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d640cdbd0 con 0x7f1d640b8510 2023-11-28T11:20:48.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.435+0000 7f1d627fc700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f1d640cd460 0x7f1d6406beb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:41112/0 (socket says 172.21.15.92:41112) 2023-11-28T11:20:48.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.436+0000 7f1d627fc700 1 -- 172.21.15.92:0/2874617877 learned_addr learned my addr 172.21.15.92:0/2874617877 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:20:48.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.436+0000 7f1d62ffd700 1 --2- 172.21.15.92:0/2874617877 >> v2:172.21.15.169:3300/0 conn(0x7f1d640b8510 0x7f1d6406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:48.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.436+0000 7f1d61ffb700 1 -- 172.21.15.92:0/2874617877 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1364853751 0 0) 0x7f1d640b88e0 con 0x7f1d640cd460 2023-11-28T11:20:48.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.436+0000 7f1d61ffb700 1 -- 172.21.15.92:0/2874617877 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d4c0035b0 con 0x7f1d640cd460 2023-11-28T11:20:48.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.436+0000 7f1d61ffb700 1 -- 172.21.15.92:0/2874617877 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 16212950 0 0) 0x7f1d4c0035b0 con 0x7f1d640cd460 2023-11-28T11:20:48.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.436+0000 7f1d61ffb700 1 -- 172.21.15.92:0/2874617877 >> v2:172.21.15.169:3300/0 conn(0x7f1d640b8510 msgr2=0x7f1d6406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:48.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.436+0000 7f1d61ffb700 1 --2- 172.21.15.92:0/2874617877 >> v2:172.21.15.169:3300/0 conn(0x7f1d640b8510 0x7f1d6406b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:48.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.436+0000 7f1d61ffb700 1 -- 172.21.15.92:0/2874617877 >> v1:172.21.15.169:6789/0 conn(0x7f1d640d0d60 legacy=0x7f1d640d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:48.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.436+0000 7f1d61ffb700 1 -- 172.21.15.92:0/2874617877 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d640cfd10 con 0x7f1d640cd460 2023-11-28T11:20:48.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.436+0000 7f1d61ffb700 1 -- 172.21.15.92:0/2874617877 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532161587 0 0) 0x7f1d58001670 con 0x7f1d640cd460 2023-11-28T11:20:48.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.436+0000 7f1d61ffb700 1 -- 172.21.15.92:0/2874617877 >> v1:172.21.15.92:6789/0 conn(0x7f1d640cd460 legacy=0x7f1d6406beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:48.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.436+0000 7f1d61ffb700 1 --2- 172.21.15.92:0/2874617877 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d4c003a00 0x7f1d4c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:48.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.436+0000 7f1d61ffb700 1 --2- 172.21.15.92:0/2874617877 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d4c004850 0x7f1d4c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:48.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.436+0000 7f1d61ffb700 1 --2- 172.21.15.92:0/2874617877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d4c005200 0x7f1d4c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:48.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.436+0000 7f1d61ffb700 1 -- 172.21.15.92:0/2874617877 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d640cdbd0 con 0x7f1d4c005200 2023-11-28T11:20:48.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.436+0000 7f1d61ffb700 1 -- 172.21.15.92:0/2874617877 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4c00c080 con 0x7f1d4c003a00 2023-11-28T11:20:48.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.436+0000 7f1d61ffb700 1 -- 172.21.15.92:0/2874617877 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4c00c240 con 0x7f1d4c004850 2023-11-28T11:20:48.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.436+0000 7f1d62ffd700 1 --2- 172.21.15.92:0/2874617877 >> v2:172.21.15.169:3300/0 conn(0x7f1d640b8510 0x7f1d6406b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:20:48.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.437+0000 7f1d637fe700 1 --2- 172.21.15.92:0/2874617877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d4c005200 0x7f1d4c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:48.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.437+0000 7f1d627fc700 1 --2- 172.21.15.92:0/2874617877 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d4c004850 0x7f1d4c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:48.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.437+0000 7f1d62ffd700 1 --2- 172.21.15.92:0/2874617877 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d4c003a00 0x7f1d4c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:48.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.437+0000 7f1d637fe700 1 -- 172.21.15.92:0/2874617877 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d4c004850 msgr2=0x7f1d4c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:48.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.437+0000 7f1d637fe700 1 --2- 172.21.15.92:0/2874617877 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d4c004850 0x7f1d4c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:48.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.437+0000 7f1d637fe700 1 -- 172.21.15.92:0/2874617877 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d4c003a00 msgr2=0x7f1d4c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:48.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.437+0000 7f1d637fe700 1 --2- 172.21.15.92:0/2874617877 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d4c003a00 0x7f1d4c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:48.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.437+0000 7f1d637fe700 1 -- 172.21.15.92:0/2874617877 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d4c0035b0 con 0x7f1d4c005200 2023-11-28T11:20:48.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.437+0000 7f1d637fe700 1 -- 172.21.15.92:0/2874617877 reap_dead start 2023-11-28T11:20:48.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.437+0000 7f1d637fe700 1 --2- 172.21.15.92:0/2874617877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d4c005200 0x7f1d4c00ba10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c00c190 tx=0x7f1d4c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23898d8da932fb27 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:48.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.437+0000 7f1d61ffb700 1 -- 172.21.15.92:0/2874617877 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c001db0 con 0x7f1d4c005200 2023-11-28T11:20:48.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.438+0000 7f1d61ffb700 1 -- 172.21.15.92:0/2874617877 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d5c00ccd0 con 0x7f1d4c005200 2023-11-28T11:20:48.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.438+0000 7f1d61ffb700 1 -- 172.21.15.92:0/2874617877 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c003c40 con 0x7f1d4c005200 2023-11-28T11:20:48.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.438+0000 7f1d699ff700 1 -- 172.21.15.92:0/2874617877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d4c005200 msgr2=0x7f1d4c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:48.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.438+0000 7f1d699ff700 1 --2- 172.21.15.92:0/2874617877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d4c005200 0x7f1d4c00ba10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c00c190 tx=0x7f1d4c005880 comp rx=0 tx=0).stop 2023-11-28T11:20:48.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.438+0000 7f1d699ff700 1 -- 172.21.15.92:0/2874617877 shutdown_connections 2023-11-28T11:20:48.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.438+0000 7f1d699ff700 1 --2- 172.21.15.92:0/2874617877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d4c005200 0x7f1d4c00ba10 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:48.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.438+0000 7f1d699ff700 1 -- 172.21.15.92:0/2874617877 >> 172.21.15.92:0/2874617877 conn(0x7f1d640719a0 msgr2=0x7f1d64071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:48.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.438+0000 7f1d699ff700 1 -- 172.21.15.92:0/2874617877 shutdown_connections 2023-11-28T11:20:48.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.438+0000 7f1d699ff700 1 -- 172.21.15.92:0/2874617877 wait complete. 2023-11-28T11:20:48.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.440+0000 7f1d699ff700 1 Processor -- start 2023-11-28T11:20:48.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.440+0000 7f1d699ff700 1 -- start start 2023-11-28T11:20:48.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.441+0000 7f1d699ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d4c005200 0x7f1d64159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:48.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.441+0000 7f1d699ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d640ccd60 0x7f1d6414d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:48.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.441+0000 7f1d699ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d6415b470 0x7f1d640b5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:48.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.441+0000 7f1d699ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6415a3d0 con 0x7f1d6415b470 2023-11-28T11:20:48.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.441+0000 7f1d699ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6415a6b0 con 0x7f1d4c005200 2023-11-28T11:20:48.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.441+0000 7f1d699ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6415a990 con 0x7f1d640ccd60 2023-11-28T11:20:48.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.441+0000 7f1d637fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d6415b470 0x7f1d640b5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:48.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.441+0000 7f1d637fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d6415b470 0x7f1d640b5f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44774/0 (socket says 172.21.15.92:44774) 2023-11-28T11:20:48.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.441+0000 7f1d637fe700 1 -- 172.21.15.92:0/3493905975 learned_addr learned my addr 172.21.15.92:0/3493905975 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:20:48.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.441+0000 7f1d62ffd700 1 --2- 172.21.15.92:0/3493905975 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d4c005200 0x7f1d64159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:48.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.441+0000 7f1d627fc700 1 --2- 172.21.15.92:0/3493905975 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d640ccd60 0x7f1d6414d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:48.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.442+0000 7f1d637fe700 1 -- 172.21.15.92:0/3493905975 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d640ccd60 msgr2=0x7f1d6414d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:48.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.442+0000 7f1d637fe700 1 --2- 172.21.15.92:0/3493905975 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d640ccd60 0x7f1d6414d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:48.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.442+0000 7f1d637fe700 1 -- 172.21.15.92:0/3493905975 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d4c005200 msgr2=0x7f1d64159cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:48.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.442+0000 7f1d637fe700 1 --2- 172.21.15.92:0/3493905975 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d4c005200 0x7f1d64159cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:48.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.442+0000 7f1d637fe700 1 -- 172.21.15.92:0/3493905975 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d640d29c0 con 0x7f1d6415b470 2023-11-28T11:20:48.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.442+0000 7f1d637fe700 1 --2- 172.21.15.92:0/3493905975 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d6415b470 0x7f1d640b5f10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1d640ce610 tx=0x7f1d5c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:48.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.442+0000 7f1d4bfff700 1 -- 172.21.15.92:0/3493905975 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c003e60 con 0x7f1d6415b470 2023-11-28T11:20:48.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.442+0000 7f1d4bfff700 1 -- 172.21.15.92:0/3493905975 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d5c005070 con 0x7f1d6415b470 2023-11-28T11:20:48.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.442+0000 7f1d4bfff700 1 -- 172.21.15.92:0/3493905975 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c005220 con 0x7f1d6415b470 2023-11-28T11:20:48.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.442+0000 7f1d699ff700 1 -- 172.21.15.92:0/3493905975 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d640cfff0 con 0x7f1d6415b470 2023-11-28T11:20:48.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.442+0000 7f1d699ff700 1 -- 172.21.15.92:0/3493905975 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d640d2b50 con 0x7f1d6415b470 2023-11-28T11:20:48.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.443+0000 7f1d699ff700 1 -- 172.21.15.92:0/3493905975 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d64067880 con 0x7f1d6415b470 2023-11-28T11:20:48.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.444+0000 7f1d4bfff700 1 -- 172.21.15.92:0/3493905975 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1d5c019040 con 0x7f1d6415b470 2023-11-28T11:20:48.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.445+0000 7f1d4bfff700 1 --2- 172.21.15.92:0/3493905975 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f1d500682f0 0x7f1d5006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:48.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.445+0000 7f1d4bfff700 1 -- 172.21.15.92:0/3493905975 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1d5c086160 con 0x7f1d6415b470 2023-11-28T11:20:48.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.445+0000 7f1d62ffd700 1 --2- 172.21.15.92:0/3493905975 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f1d500682f0 0x7f1d5006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:48.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.446+0000 7f1d62ffd700 1 --2- 172.21.15.92:0/3493905975 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f1d500682f0 0x7f1d5006a7a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1d54009470 tx=0x7f1d54009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:48.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.449+0000 7f1d4bfff700 1 -- 172.21.15.92:0/3493905975 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1d5c052d20 con 0x7f1d6415b470 2023-11-28T11:20:48.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:48.642+0000 7f1d699ff700 1 -- 172.21.15.92:0/3493905975 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f1d64067880 con 0x7f1d6415b470 2023-11-28T11:20:49.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.505+0000 7f1d4bfff700 1 -- 172.21.15.92:0/3493905975 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v17) v1 ==== 109+0+0 (secure 0 0 0) 0x7f1d5c056340 con 0x7f1d6415b470 2023-11-28T11:20:49.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.508+0000 7f1d699ff700 1 -- 172.21.15.92:0/3493905975 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f1d500682f0 msgr2=0x7f1d5006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:49.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.508+0000 7f1d699ff700 1 --2- 172.21.15.92:0/3493905975 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f1d500682f0 0x7f1d5006a7a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1d54009470 tx=0x7f1d54009210 comp rx=0 tx=0).stop 2023-11-28T11:20:49.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.508+0000 7f1d699ff700 1 -- 172.21.15.92:0/3493905975 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d6415b470 msgr2=0x7f1d640b5f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:49.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.508+0000 7f1d699ff700 1 --2- 172.21.15.92:0/3493905975 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d6415b470 0x7f1d640b5f10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1d640ce610 tx=0x7f1d5c0014a0 comp rx=0 tx=0).stop 2023-11-28T11:20:49.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.508+0000 7f1d699ff700 1 -- 172.21.15.92:0/3493905975 shutdown_connections 2023-11-28T11:20:49.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.508+0000 7f1d699ff700 1 --2- 172.21.15.92:0/3493905975 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f1d500682f0 0x7f1d5006a7a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:49.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.508+0000 7f1d699ff700 1 --2- 172.21.15.92:0/3493905975 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1d6415b470 0x7f1d640b5f10 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:49.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.508+0000 7f1d699ff700 1 --2- 172.21.15.92:0/3493905975 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1d4c005200 0x7f1d64159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:49.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.508+0000 7f1d699ff700 1 --2- 172.21.15.92:0/3493905975 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1d640ccd60 0x7f1d6414d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:49.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.508+0000 7f1d699ff700 1 -- 172.21.15.92:0/3493905975 >> 172.21.15.92:0/3493905975 conn(0x7f1d640719a0 msgr2=0x7f1d640b78a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:49.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.509+0000 7f1d699ff700 1 -- 172.21.15.92:0/3493905975 shutdown_connections 2023-11-28T11:20:49.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.509+0000 7f1d699ff700 1 -- 172.21.15.92:0/3493905975 wait complete. 2023-11-28T11:20:49.511 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'rbd' created 2023-11-28T11:20:49.531 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-11-28T11:20:49.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.728+0000 7f86511ff700 1 Processor -- start 2023-11-28T11:20:49.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.728+0000 7f86511ff700 1 -- start start 2023-11-28T11:20:49.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.729+0000 7f86511ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f864c0b8790 0x7f864c0cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:49.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.729+0000 7f86511ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f864c0d00b0 con 0x7f864c06c050 2023-11-28T11:20:49.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.729+0000 7f86511ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f864c0b5240 con 0x7f864c0cd460 2023-11-28T11:20:49.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.729+0000 7f86511ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f864c0b5520 con 0x7f864c0b8790 2023-11-28T11:20:49.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.729+0000 7f864b59e700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f864c06c050 0x7f864c0d0fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51782/0 (socket says 172.21.15.92:51782) 2023-11-28T11:20:49.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.729+0000 7f864b59e700 1 -- 172.21.15.92:0/1917681406 learned_addr learned my addr 172.21.15.92:0/1917681406 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:20:49.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.729+0000 7f864ad9d700 1 --2- 172.21.15.92:0/1917681406 >> v2:172.21.15.169:3300/0 conn(0x7f864c0b8790 0x7f864c0cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:49.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.730+0000 7f8649d9b700 1 -- 172.21.15.92:0/1917681406 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2415099127 0 0) 0x7f864c0d00b0 con 0x7f864c06c050 2023-11-28T11:20:49.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.730+0000 7f8649d9b700 1 -- 172.21.15.92:0/1917681406 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86340035b0 con 0x7f864c06c050 2023-11-28T11:20:49.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.730+0000 7f864ad9d700 1 -- 172.21.15.92:0/1917681406 >> v1:172.21.15.169:6789/0 conn(0x7f864c0cd460 legacy=0x7f864c06b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:49.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.730+0000 7f864ad9d700 1 -- 172.21.15.92:0/1917681406 >> v1:172.21.15.92:6789/0 conn(0x7f864c06c050 legacy=0x7f864c0d0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:49.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.730+0000 7f864ad9d700 1 -- 172.21.15.92:0/1917681406 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f864c0d0a80 con 0x7f864c0b8790 2023-11-28T11:20:49.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.730+0000 7f864ad9d700 1 --2- 172.21.15.92:0/1917681406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f864c0b8790 0x7f864c0cfb70 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f863c009870 tx=0x7f863c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cefaa63617a875f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:49.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.730+0000 7f8649d9b700 1 -- 172.21.15.92:0/1917681406 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863c011830 con 0x7f864c0b8790 2023-11-28T11:20:49.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.730+0000 7f8649d9b700 1 -- 172.21.15.92:0/1917681406 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f863c011990 con 0x7f864c0b8790 2023-11-28T11:20:49.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.730+0000 7f8649d9b700 1 -- 172.21.15.92:0/1917681406 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863c011d20 con 0x7f864c0b8790 2023-11-28T11:20:49.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.731+0000 7f86511ff700 1 -- 172.21.15.92:0/1917681406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f864c0b8790 msgr2=0x7f864c0cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:49.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.731+0000 7f86511ff700 1 --2- 172.21.15.92:0/1917681406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f864c0b8790 0x7f864c0cfb70 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f863c009870 tx=0x7f863c009b80 comp rx=0 tx=0).stop 2023-11-28T11:20:49.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.731+0000 7f86511ff700 1 -- 172.21.15.92:0/1917681406 shutdown_connections 2023-11-28T11:20:49.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.731+0000 7f86511ff700 1 --2- 172.21.15.92:0/1917681406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f864c0b8790 0x7f864c0cfb70 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:49.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.731+0000 7f86511ff700 1 -- 172.21.15.92:0/1917681406 >> 172.21.15.92:0/1917681406 conn(0x7f864c0719a0 msgr2=0x7f864c071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:49.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.731+0000 7f86511ff700 1 -- 172.21.15.92:0/1917681406 shutdown_connections 2023-11-28T11:20:49.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.731+0000 7f86511ff700 1 -- 172.21.15.92:0/1917681406 wait complete. 2023-11-28T11:20:49.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.733+0000 7f86511ff700 1 Processor -- start 2023-11-28T11:20:49.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.733+0000 7f86511ff700 1 -- start start 2023-11-28T11:20:49.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.734+0000 7f86511ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f864c06c050 0x7f864c0b20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:49.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.734+0000 7f86511ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f864c0b8790 0x7f864c0b25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:49.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.734+0000 7f86511ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f864c0cd460 0x7f864c0c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:49.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.734+0000 7f86511ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f864c1f6470 con 0x7f864c0b8790 2023-11-28T11:20:49.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.734+0000 7f86511ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f864c1f2e60 con 0x7f864c0cd460 2023-11-28T11:20:49.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.734+0000 7f86511ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f864c1f20a0 con 0x7f864c06c050 2023-11-28T11:20:49.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.734+0000 7f864a59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f864c0b8790 0x7f864c0b25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:49.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.734+0000 7f864a59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f864c0b8790 0x7f864c0b25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41438/0 (socket says 172.21.15.92:41438) 2023-11-28T11:20:49.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.734+0000 7f864a59c700 1 -- 172.21.15.92:0/197570879 learned_addr learned my addr 172.21.15.92:0/197570879 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:20:49.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.734+0000 7f864ad9d700 1 --2- 172.21.15.92:0/197570879 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f864c06c050 0x7f864c0b20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:49.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.734+0000 7f864b59e700 1 --2- 172.21.15.92:0/197570879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f864c0cd460 0x7f864c0c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:49.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.734+0000 7f864a59c700 1 -- 172.21.15.92:0/197570879 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f864c06c050 msgr2=0x7f864c0b20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:49.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.734+0000 7f864a59c700 1 --2- 172.21.15.92:0/197570879 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f864c06c050 0x7f864c0b20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:49.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.734+0000 7f864a59c700 1 -- 172.21.15.92:0/197570879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f864c0cd460 msgr2=0x7f864c0c9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:49.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.734+0000 7f864a59c700 1 --2- 172.21.15.92:0/197570879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f864c0cd460 0x7f864c0c9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:49.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.734+0000 7f864a59c700 1 -- 172.21.15.92:0/197570879 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f863c0094a0 con 0x7f864c0b8790 2023-11-28T11:20:49.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.735+0000 7f864a59c700 1 --2- 172.21.15.92:0/197570879 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f864c0b8790 0x7f864c0b25e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f864000c450 tx=0x7f864000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:49.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.735+0000 7f86337fe700 1 -- 172.21.15.92:0/197570879 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86400052d0 con 0x7f864c0b8790 2023-11-28T11:20:49.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.735+0000 7f86337fe700 1 -- 172.21.15.92:0/197570879 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8640005430 con 0x7f864c0b8790 2023-11-28T11:20:49.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.735+0000 7f86511ff700 1 -- 172.21.15.92:0/197570879 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f864c0d16b0 con 0x7f864c0b8790 2023-11-28T11:20:49.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.735+0000 7f86337fe700 1 -- 172.21.15.92:0/197570879 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86400038c0 con 0x7f864c0b8790 2023-11-28T11:20:49.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.735+0000 7f86511ff700 1 -- 172.21.15.92:0/197570879 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f864c0b5800 con 0x7f864c0b8790 2023-11-28T11:20:49.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.736+0000 7f86511ff700 1 -- 172.21.15.92:0/197570879 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f864c0d1840 con 0x7f864c0b8790 2023-11-28T11:20:49.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.737+0000 7f86337fe700 1 -- 172.21.15.92:0/197570879 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8640019030 con 0x7f864c0b8790 2023-11-28T11:20:49.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.738+0000 7f86337fe700 1 --2- 172.21.15.92:0/197570879 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f86380682f0 0x7f863806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:49.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.739+0000 7f86337fe700 1 -- 172.21.15.92:0/197570879 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5199+0+0 (secure 0 0 0) 0x7f8640084c10 con 0x7f864c0b8790 2023-11-28T11:20:49.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.739+0000 7f864ad9d700 1 --2- 172.21.15.92:0/197570879 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f86380682f0 0x7f863806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:49.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.739+0000 7f864ad9d700 1 --2- 172.21.15.92:0/197570879 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f86380682f0 0x7f863806a7a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f863c009840 tx=0x7f863c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:49.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.744+0000 7f86337fe700 1 -- 172.21.15.92:0/197570879 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f86400516e0 con 0x7f864c0b8790 2023-11-28T11:20:49.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:49.939+0000 7f86511ff700 1 -- 172.21.15.92:0/197570879 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f864c067880 con 0x7f864c0b8790 2023-11-28T11:20:50.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.514+0000 7f86337fe700 1 -- 172.21.15.92:0/197570879 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v18) v1 ==== 173+0+0 (secure 0 0 0) 0x7f8640054d00 con 0x7f864c0b8790 2023-11-28T11:20:50.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.519+0000 7f86511ff700 1 -- 172.21.15.92:0/197570879 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f86380682f0 msgr2=0x7f863806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:50.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.519+0000 7f86511ff700 1 --2- 172.21.15.92:0/197570879 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f86380682f0 0x7f863806a7a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f863c009840 tx=0x7f863c008030 comp rx=0 tx=0).stop 2023-11-28T11:20:50.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.519+0000 7f86511ff700 1 -- 172.21.15.92:0/197570879 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f864c0b8790 msgr2=0x7f864c0b25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:50.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.519+0000 7f86511ff700 1 --2- 172.21.15.92:0/197570879 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f864c0b8790 0x7f864c0b25e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f864000c450 tx=0x7f864000c810 comp rx=0 tx=0).stop 2023-11-28T11:20:50.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.519+0000 7f86511ff700 1 -- 172.21.15.92:0/197570879 shutdown_connections 2023-11-28T11:20:50.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.519+0000 7f86511ff700 1 --2- 172.21.15.92:0/197570879 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f864c0cd460 0x7f864c0c9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:50.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.519+0000 7f86511ff700 1 --2- 172.21.15.92:0/197570879 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f864c06c050 0x7f864c0b20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:50.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.519+0000 7f86511ff700 1 --2- 172.21.15.92:0/197570879 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f86380682f0 0x7f863806a7a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:50.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.519+0000 7f86511ff700 1 --2- 172.21.15.92:0/197570879 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f864c0b8790 0x7f864c0b25e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:50.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.519+0000 7f86511ff700 1 -- 172.21.15.92:0/197570879 >> 172.21.15.92:0/197570879 conn(0x7f864c0719a0 msgr2=0x7f864c072cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:50.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.519+0000 7f86511ff700 1 -- 172.21.15.92:0/197570879 shutdown_connections 2023-11-28T11:20:50.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.519+0000 7f86511ff700 1 -- 172.21.15.92:0/197570879 wait complete. 2023-11-28T11:20:50.521 INFO:teuthology.orchestra.run.smithi092.stderr:enabled application 'rbd' on pool 'rbd' 2023-11-28T11:20:50.542 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-11-28T11:20:50.542 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T11:20:50.542 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T11:20:50.547 INFO:tasks.ceph.mds.a:Started 2023-11-28T11:20:50.547 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T11:20:50.547 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T11:20:50.551 INFO:tasks.ceph.mds.c:Started 2023-11-28T11:20:50.552 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T11:20:50.552 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T11:20:50.556 INFO:tasks.ceph.mds.b:Started 2023-11-28T11:20:50.557 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T11:20:50.557 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T11:20:50.562 INFO:tasks.ceph.mds.d:Started 2023-11-28T11:20:50.562 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-11-28T11:20:50.563 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:20:50.703 INFO:tasks.ceph.mds.c.smithi092.stdout:starting mds.c at 2023-11-28T11:20:50.704 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-11-28T11:20:50.714 INFO:tasks.ceph.mds.d.smithi169.stdout:starting mds.d at 2023-11-28T11:20:50.715 INFO:tasks.ceph.mds.b.smithi169.stdout:starting mds.b at 2023-11-28T11:20:50.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.776+0000 7f9c4fbff700 1 Processor -- start 2023-11-28T11:20:50.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.777+0000 7f9c4fbff700 1 -- start start 2023-11-28T11:20:50.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.778+0000 7f9c4fbff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c480d1d70 0x7f9c48167270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:50.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.778+0000 7f9c4fbff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c4806c790 con 0x7f9c480b8510 2023-11-28T11:20:50.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.778+0000 7f9c4fbff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c480b88e0 con 0x7f9c480cd460 2023-11-28T11:20:50.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.778+0000 7f9c4fbff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c480cdbd0 con 0x7f9c480d1d70 2023-11-28T11:20:50.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.778+0000 7f9c4d99b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f9c480b8510 0x7f9c4806b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51820/0 (socket says 172.21.15.92:51820) 2023-11-28T11:20:50.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.778+0000 7f9c4d99b700 1 -- 172.21.15.92:0/1652026526 learned_addr learned my addr 172.21.15.92:0/1652026526 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:20:50.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.778+0000 7f9c4e19c700 1 --2- 172.21.15.92:0/1652026526 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c480d1d70 0x7f9c48167270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:50.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.778+0000 7f9c4c999700 1 -- 172.21.15.92:0/1652026526 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2836576733 0 0) 0x7f9c4806c790 con 0x7f9c480b8510 2023-11-28T11:20:50.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.778+0000 7f9c4c999700 1 -- 172.21.15.92:0/1652026526 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c340035b0 con 0x7f9c480b8510 2023-11-28T11:20:50.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.778+0000 7f9c4c999700 1 -- 172.21.15.92:0/1652026526 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3048053207 0 0) 0x7f9c480b88e0 con 0x7f9c480cd460 2023-11-28T11:20:50.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.778+0000 7f9c4c999700 1 -- 172.21.15.92:0/1652026526 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c4806c790 con 0x7f9c480cd460 2023-11-28T11:20:50.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.779+0000 7f9c4e19c700 1 -- 172.21.15.92:0/1652026526 >> v1:172.21.15.169:6789/0 conn(0x7f9c480cd460 legacy=0x7f9c4806c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:50.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.779+0000 7f9c4e19c700 1 -- 172.21.15.92:0/1652026526 >> v1:172.21.15.92:6789/0 conn(0x7f9c480b8510 legacy=0x7f9c4806b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:50.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.779+0000 7f9c4e19c700 1 -- 172.21.15.92:0/1652026526 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c480cd8f0 con 0x7f9c480d1d70 2023-11-28T11:20:50.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.779+0000 7f9c4e19c700 1 --2- 172.21.15.92:0/1652026526 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c480d1d70 0x7f9c48167270 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9c40009dc0 tx=0x7f9c4000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3408fc5fa3c2639 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:50.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.779+0000 7f9c4c999700 1 -- 172.21.15.92:0/1652026526 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4000e070 con 0x7f9c480d1d70 2023-11-28T11:20:50.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.779+0000 7f9c4c999700 1 -- 172.21.15.92:0/1652026526 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c4000bb20 con 0x7f9c480d1d70 2023-11-28T11:20:50.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.779+0000 7f9c4c999700 1 -- 172.21.15.92:0/1652026526 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c40012430 con 0x7f9c480d1d70 2023-11-28T11:20:50.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.780+0000 7f9c4fbff700 1 -- 172.21.15.92:0/1652026526 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c480d1d70 msgr2=0x7f9c48167270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:50.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.780+0000 7f9c4fbff700 1 --2- 172.21.15.92:0/1652026526 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c480d1d70 0x7f9c48167270 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9c40009dc0 tx=0x7f9c4000b330 comp rx=0 tx=0).stop 2023-11-28T11:20:50.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.780+0000 7f9c4fbff700 1 -- 172.21.15.92:0/1652026526 shutdown_connections 2023-11-28T11:20:50.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.780+0000 7f9c4fbff700 1 --2- 172.21.15.92:0/1652026526 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c480d1d70 0x7f9c48167270 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:50.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.780+0000 7f9c4fbff700 1 -- 172.21.15.92:0/1652026526 >> 172.21.15.92:0/1652026526 conn(0x7f9c480719a0 msgr2=0x7f9c48071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:50.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.780+0000 7f9c4fbff700 1 -- 172.21.15.92:0/1652026526 shutdown_connections 2023-11-28T11:20:50.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.780+0000 7f9c4fbff700 1 -- 172.21.15.92:0/1652026526 wait complete. 2023-11-28T11:20:50.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.781+0000 7f9c4fbff700 1 Processor -- start 2023-11-28T11:20:50.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.782+0000 7f9c4fbff700 1 -- start start 2023-11-28T11:20:50.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.782+0000 7f9c4fbff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c480b8510 0x7f9c481f2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:50.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.782+0000 7f9c4fbff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c480cd460 0x7f9c48155a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:50.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.782+0000 7f9c4fbff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c480d1d70 0x7f9c481576e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:50.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.783+0000 7f9c4fbff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c48158620 con 0x7f9c480cd460 2023-11-28T11:20:50.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.783+0000 7f9c4fbff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c481580a0 con 0x7f9c480b8510 2023-11-28T11:20:50.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.783+0000 7f9c4fbff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c48158870 con 0x7f9c480d1d70 2023-11-28T11:20:50.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.783+0000 7f9c4d19a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c480cd460 0x7f9c48155a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:50.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.783+0000 7f9c4d19a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c480cd460 0x7f9c48155a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41472/0 (socket says 172.21.15.92:41472) 2023-11-28T11:20:50.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.783+0000 7f9c4d19a700 1 -- 172.21.15.92:0/1926117201 learned_addr learned my addr 172.21.15.92:0/1926117201 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:20:50.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.783+0000 7f9c4e19c700 1 --2- 172.21.15.92:0/1926117201 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c480d1d70 0x7f9c481576e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:50.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.783+0000 7f9c4d99b700 1 --2- 172.21.15.92:0/1926117201 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c480b8510 0x7f9c481f2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:50.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.783+0000 7f9c4d19a700 1 -- 172.21.15.92:0/1926117201 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c480d1d70 msgr2=0x7f9c481576e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:50.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.783+0000 7f9c4d19a700 1 --2- 172.21.15.92:0/1926117201 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c480d1d70 0x7f9c481576e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:50.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.783+0000 7f9c4d19a700 1 -- 172.21.15.92:0/1926117201 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c480b8510 msgr2=0x7f9c481f2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:50.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.783+0000 7f9c4d19a700 1 --2- 172.21.15.92:0/1926117201 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c480b8510 0x7f9c481f2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:50.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.783+0000 7f9c4d19a700 1 -- 172.21.15.92:0/1926117201 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c40009a70 con 0x7f9c480cd460 2023-11-28T11:20:50.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.783+0000 7f9c4d19a700 1 --2- 172.21.15.92:0/1926117201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c480cd460 0x7f9c48155a80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9c3800c500 tx=0x7f9c3800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:50.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.784+0000 7f9c3e7fc700 1 -- 172.21.15.92:0/1926117201 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c380052d0 con 0x7f9c480cd460 2023-11-28T11:20:50.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.784+0000 7f9c3e7fc700 1 -- 172.21.15.92:0/1926117201 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c38005430 con 0x7f9c480cd460 2023-11-28T11:20:50.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.784+0000 7f9c4fbff700 1 -- 172.21.15.92:0/1926117201 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c480cfff0 con 0x7f9c480cd460 2023-11-28T11:20:50.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.784+0000 7f9c3e7fc700 1 -- 172.21.15.92:0/1926117201 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c380038c0 con 0x7f9c480cd460 2023-11-28T11:20:50.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.784+0000 7f9c4fbff700 1 -- 172.21.15.92:0/1926117201 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c480cfd10 con 0x7f9c480cd460 2023-11-28T11:20:50.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.785+0000 7f9c4fbff700 1 -- 172.21.15.92:0/1926117201 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c480d26c0 con 0x7f9c480cd460 2023-11-28T11:20:50.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.786+0000 7f9c3e7fc700 1 -- 172.21.15.92:0/1926117201 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9c38019030 con 0x7f9c480cd460 2023-11-28T11:20:50.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.787+0000 7f9c3e7fc700 1 --2- 172.21.15.92:0/1926117201 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9c2c0682f0 0x7f9c2c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:50.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.787+0000 7f9c3e7fc700 1 -- 172.21.15.92:0/1926117201 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f9c38084bc0 con 0x7f9c480cd460 2023-11-28T11:20:50.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.787+0000 7f9c4d99b700 1 --2- 172.21.15.92:0/1926117201 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9c2c0682f0 0x7f9c2c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:50.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.787+0000 7f9c4d99b700 1 --2- 172.21.15.92:0/1926117201 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9c2c0682f0 0x7f9c2c06a7a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9c44007fd0 tx=0x7f9c44007dc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:50.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.791+0000 7f9c3e7fc700 1 -- 172.21.15.92:0/1926117201 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9c38051830 con 0x7f9c480cd460 2023-11-28T11:20:50.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.984+0000 7f9c4fbff700 1 -- 172.21.15.92:0/1926117201 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9c48067880 con 0x7f9c480cd460 2023-11-28T11:20:50.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.985+0000 7f9c3e7fc700 1 -- 172.21.15.92:0/1926117201 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+14277 (secure 0 0 0) 0x7f9c38054e50 con 0x7f9c480cd460 2023-11-28T11:20:50.986 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:20:50.986 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":18,"fsid":"68956e0c-c16f-47cd-a220-f98364b0831d","created":"2023-11-28T11:20:28.697573+0000","modified":"2023-11-28T11:20:50.504366+0000","last_up_change":"2023-11-28T11:20:41.257011+0000","last_in_change":"2023-11-28T11:20:32.249767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:20:41.925237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:20:48.682497+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6abe483f-0e03-4a26-b9cf-62041291747e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":115823},{"type":"v1","addr":"172.21.15.92:6809","nonce":115823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":115823},{"type":"v1","addr":"172.21.15.92:6813","nonce":115823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":115823},{"type":"v1","addr":"172.21.15.92:6821","nonce":115823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":115823},{"type":"v1","addr":"172.21.15.92:6817","nonce":115823}]},"public_addr":"172.21.15.92:6809/115823","cluster_addr":"172.21.15.92:6813/115823","heartbeat_back_addr":"172.21.15.92:6821/115823","heartbeat_front_addr":"172.21.15.92:6817/115823","state":["exists","up"]},{"osd":1,"uuid":"13c1de73-1a8c-4d7e-8a28-9dae52dfe089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":115824},{"type":"v1","addr":"172.21.15.92:6812","nonce":115824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":115824},{"type":"v1","addr":"172.21.15.92:6816","nonce":115824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":115824},{"type":"v1","addr":"172.21.15.92:6823","nonce":115824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":115824},{"type":"v1","addr":"172.21.15.92:6820","nonce":115824}]},"public_addr":"172.21.15.92:6812/115824","cluster_addr":"172.21.15.92:6816/115824","heartbeat_back_addr":"172.21.15.92:6823/115824","heartbeat_front_addr":"172.21.15.92:6820/115824","state":["exists","up"]},{"osd":2,"uuid":"25788a55-112f-4f00-810d-2bc37cf76778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":115821},{"type":"v1","addr":"172.21.15.92:6801","nonce":115821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":115821},{"type":"v1","addr":"172.21.15.92:6803","nonce":115821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":115821},{"type":"v1","addr":"172.21.15.92:6807","nonce":115821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":115821},{"type":"v1","addr":"172.21.15.92:6805","nonce":115821}]},"public_addr":"172.21.15.92:6801/115821","cluster_addr":"172.21.15.92:6803/115821","heartbeat_back_addr":"172.21.15.92:6807/115821","heartbeat_front_addr":"172.21.15.92:6805/115821","state":["exists","up"]},{"osd":3,"uuid":"adf54e24-33a9-41fe-9ce3-98a5ab6c158e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":115825},{"type":"v1","addr":"172.21.15.92:6825","nonce":115825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":115825},{"type":"v1","addr":"172.21.15.92:6827","nonce":115825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":115825},{"type":"v1","addr":"172.21.15.92:6831","nonce":115825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":115825},{"type":"v1","addr":"172.21.15.92:6829","nonce":115825}]},"public_addr":"172.21.15.92:6825/115825","cluster_addr":"172.21.15.92:6827/115825","heartbeat_back_addr":"172.21.15.92:6831/115825","heartbeat_front_addr":"172.21.15.92:6829/115825","state":["exists","up"]},{"osd":4,"uuid":"ccb9b747-f399-44ea-9a18-9db3152227e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":116035},{"type":"v1","addr":"172.21.15.169:6817","nonce":116035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":116035},{"type":"v1","addr":"172.21.15.169:6819","nonce":116035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":116035},{"type":"v1","addr":"172.21.15.169:6826","nonce":116035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":116035},{"type":"v1","addr":"172.21.15.169:6822","nonce":116035}]},"public_addr":"172.21.15.169:6817/116035","cluster_addr":"172.21.15.169:6819/116035","heartbeat_back_addr":"172.21.15.169:6826/116035","heartbeat_front_addr":"172.21.15.169:6822/116035","state":["exists","up"]},{"osd":5,"uuid":"308a1e63-75aa-4d7d-8a7c-cecc24714fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":116033},{"type":"v1","addr":"172.21.15.169:6801","nonce":116033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":116033},{"type":"v1","addr":"172.21.15.169:6803","nonce":116033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":116033},{"type":"v1","addr":"172.21.15.169:6807","nonce":116033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":116033},{"type":"v1","addr":"172.21.15.169:6805","nonce":116033}]},"public_addr":"172.21.15.169:6801/116033","cluster_addr":"172.21.15.169:6803/116033","heartbeat_back_addr":"172.21.15.169:6807/116033","heartbeat_front_addr":"172.21.15.169:6805/116033","state":["exists","up"]},{"osd":6,"uuid":"9c459b7b-b821-4b4b-b91e-6aaae271457c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":116034},{"type":"v1","addr":"172.21.15.169:6809","nonce":116034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":116034},{"type":"v1","addr":"172.21.15.169:6811","nonce":116034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":116034},{"type":"v1","addr":"172.21.15.169:6815","nonce":116034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":116034},{"type":"v1","addr":"172.21.15.169:6813","nonce":116034}]},"public_addr":"172.21.15.169:6809/116034","cluster_addr":"172.21.15.169:6811/116034","heartbeat_back_addr":"172.21.15.169:6815/116034","heartbeat_front_addr":"172.21.15.169:6813/116034","state":["exists","up"]},{"osd":7,"uuid":"1ca9aa4a-755d-4e28-a937-4d8dccab4df3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":116044},{"type":"v1","addr":"172.21.15.169:6823","nonce":116044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6825","nonce":116044},{"type":"v1","addr":"172.21.15.169:6827","nonce":116044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":116044},{"type":"v1","addr":"172.21.15.169:6831","nonce":116044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":116044},{"type":"v1","addr":"172.21.15.169:6829","nonce":116044}]},"public_addr":"172.21.15.169:6823/116044","cluster_addr":"172.21.15.169:6827/116044","heartbeat_back_addr":"172.21.15.169:6831/116044","heartbeat_front_addr":"172.21.15.169:6829/116044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.586781+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.869778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.508594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.860493+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.872267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.746933+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.611520+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.871446+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:20:50.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.987+0000 7f9c4fbff700 1 -- 172.21.15.92:0/1926117201 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9c2c0682f0 msgr2=0x7f9c2c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:50.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.987+0000 7f9c4fbff700 1 --2- 172.21.15.92:0/1926117201 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9c2c0682f0 0x7f9c2c06a7a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9c44007fd0 tx=0x7f9c44007dc0 comp rx=0 tx=0).stop 2023-11-28T11:20:50.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.987+0000 7f9c4fbff700 1 -- 172.21.15.92:0/1926117201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c480cd460 msgr2=0x7f9c48155a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:50.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.987+0000 7f9c4fbff700 1 --2- 172.21.15.92:0/1926117201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c480cd460 0x7f9c48155a80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9c3800c500 tx=0x7f9c3800c810 comp rx=0 tx=0).stop 2023-11-28T11:20:50.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.987+0000 7f9c4fbff700 1 -- 172.21.15.92:0/1926117201 shutdown_connections 2023-11-28T11:20:50.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.987+0000 7f9c4fbff700 1 --2- 172.21.15.92:0/1926117201 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c480d1d70 0x7f9c481576e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:50.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.987+0000 7f9c4fbff700 1 --2- 172.21.15.92:0/1926117201 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c480b8510 0x7f9c481f2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:50.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.987+0000 7f9c4fbff700 1 --2- 172.21.15.92:0/1926117201 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9c2c0682f0 0x7f9c2c06a7a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:50.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.987+0000 7f9c4fbff700 1 --2- 172.21.15.92:0/1926117201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c480cd460 0x7f9c48155a80 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:50.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.987+0000 7f9c4fbff700 1 -- 172.21.15.92:0/1926117201 >> 172.21.15.92:0/1926117201 conn(0x7f9c480719a0 msgr2=0x7f9c480b41b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:50.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.987+0000 7f9c4fbff700 1 -- 172.21.15.92:0/1926117201 shutdown_connections 2023-11-28T11:20:50.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:50.987+0000 7f9c4fbff700 1 -- 172.21.15.92:0/1926117201 wait complete. 2023-11-28T11:20:51.006 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:20:41.925237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:20:48.682497+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:20:51.006 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:20:51.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.205+0000 7fb5cd3ff700 1 Processor -- start 2023-11-28T11:20:51.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.206+0000 7fb5cd3ff700 1 -- start start 2023-11-28T11:20:51.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.207+0000 7fb5cd3ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb5c80d2df0 0x7fb5c80cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:51.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.207+0000 7fb5cd3ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb5c80b4fc0 0x7fb5c80d31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:51.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.207+0000 7fb5cd3ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5c80b1c60 0x7fb5c80cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:51.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.207+0000 7fb5cd3ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5c80d3d30 con 0x7fb5c80b4fc0 2023-11-28T11:20:51.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.207+0000 7fb5cd3ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5c80d1950 con 0x7fb5c80d2df0 2023-11-28T11:20:51.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.207+0000 7fb5cd3ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c80b2500 con 0x7fb5c80b1c60 2023-11-28T11:20:51.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.207+0000 7fb5bffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb5c80b4fc0 0x7fb5c80d31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:51.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.207+0000 7fb5bffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb5c80b4fc0 0x7fb5c80d31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41488/0 (socket says 172.21.15.92:41488) 2023-11-28T11:20:51.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.207+0000 7fb5bffff700 1 -- 172.21.15.92:0/1733685046 learned_addr learned my addr 172.21.15.92:0/1733685046 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:20:51.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.207+0000 7fb5c77fe700 1 --2- 172.21.15.92:0/1733685046 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5c80b1c60 0x7fb5c80cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:51.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.207+0000 7fb5c6ffd700 1 --2- 172.21.15.92:0/1733685046 >> v2:172.21.15.169:3300/0 conn(0x7fb5c80d2df0 0x7fb5c80cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:51.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.207+0000 7fb5bffff700 1 -- 172.21.15.92:0/1733685046 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5c80b1c60 msgr2=0x7fb5c80cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:51.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.207+0000 7fb5bffff700 1 --2- 172.21.15.92:0/1733685046 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5c80b1c60 0x7fb5c80cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:51.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.207+0000 7fb5bffff700 1 -- 172.21.15.92:0/1733685046 >> v2:172.21.15.169:3300/0 conn(0x7fb5c80d2df0 msgr2=0x7fb5c80cd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:51.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.207+0000 7fb5bffff700 1 --2- 172.21.15.92:0/1733685046 >> v2:172.21.15.169:3300/0 conn(0x7fb5c80d2df0 0x7fb5c80cd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:51.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.207+0000 7fb5bffff700 1 -- 172.21.15.92:0/1733685046 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c80d3a50 con 0x7fb5c80b4fc0 2023-11-28T11:20:51.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.207+0000 7fb5c6ffd700 1 --2- 172.21.15.92:0/1733685046 >> v2:172.21.15.169:3300/0 conn(0x7fb5c80d2df0 0x7fb5c80cd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:20:51.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.207+0000 7fb5c77fe700 1 --2- 172.21.15.92:0/1733685046 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5c80b1c60 0x7fb5c80cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:20:51.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.208+0000 7fb5bffff700 1 --2- 172.21.15.92:0/1733685046 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5c80b4fc0 0x7fb5c80d31c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb5c80d27f0 tx=0x7fb5b0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7500d4093ebfe782 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:51.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.208+0000 7fb5c67fc700 1 -- 172.21.15.92:0/1733685046 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b000b870 con 0x7fb5c80b4fc0 2023-11-28T11:20:51.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.208+0000 7fb5c67fc700 1 -- 172.21.15.92:0/1733685046 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5b000b9d0 con 0x7fb5c80b4fc0 2023-11-28T11:20:51.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.208+0000 7fb5c67fc700 1 -- 172.21.15.92:0/1733685046 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b000bd60 con 0x7fb5c80b4fc0 2023-11-28T11:20:51.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.208+0000 7fb5cd3ff700 1 -- 172.21.15.92:0/1733685046 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5c80b4fc0 msgr2=0x7fb5c80d31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:51.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.208+0000 7fb5cd3ff700 1 --2- 172.21.15.92:0/1733685046 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5c80b4fc0 0x7fb5c80d31c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb5c80d27f0 tx=0x7fb5b0009bd0 comp rx=0 tx=0).stop 2023-11-28T11:20:51.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.208+0000 7fb5cd3ff700 1 -- 172.21.15.92:0/1733685046 shutdown_connections 2023-11-28T11:20:51.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.208+0000 7fb5cd3ff700 1 --2- 172.21.15.92:0/1733685046 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5c80b1c60 0x7fb5c80cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:51.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.208+0000 7fb5cd3ff700 1 --2- 172.21.15.92:0/1733685046 >> v2:172.21.15.169:3300/0 conn(0x7fb5c80d2df0 0x7fb5c80cd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:51.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.208+0000 7fb5cd3ff700 1 --2- 172.21.15.92:0/1733685046 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5c80b4fc0 0x7fb5c80d31c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:51.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.208+0000 7fb5cd3ff700 1 -- 172.21.15.92:0/1733685046 >> 172.21.15.92:0/1733685046 conn(0x7fb5c80719a0 msgr2=0x7fb5c8071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:51.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.209+0000 7fb5cd3ff700 1 -- 172.21.15.92:0/1733685046 shutdown_connections 2023-11-28T11:20:51.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.209+0000 7fb5cd3ff700 1 -- 172.21.15.92:0/1733685046 wait complete. 2023-11-28T11:20:51.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.210+0000 7fb5cd3ff700 1 Processor -- start 2023-11-28T11:20:51.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.210+0000 7fb5cd3ff700 1 -- start start 2023-11-28T11:20:51.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.211+0000 7fb5cd3ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5c80b1c60 0x7fb5c80cffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:51.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.211+0000 7fb5cd3ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5c80b4fc0 0x7fb5c80aaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:51.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.211+0000 7fb5cd3ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5c80d2df0 0x7fb5c80aafe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:51.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.211+0000 7fb5cd3ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c80c9f30 con 0x7fb5c80b1c60 2023-11-28T11:20:51.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.211+0000 7fb5cd3ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c81f66c0 con 0x7fb5c80d2df0 2023-11-28T11:20:51.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.211+0000 7fb5cd3ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c80ca180 con 0x7fb5c80b4fc0 2023-11-28T11:20:51.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.212+0000 7fb5c6ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5c80b1c60 0x7fb5c80cffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:51.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.212+0000 7fb5c6ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5c80b1c60 0x7fb5c80cffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41496/0 (socket says 172.21.15.92:41496) 2023-11-28T11:20:51.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.212+0000 7fb5c6ffd700 1 -- 172.21.15.92:0/3078334434 learned_addr learned my addr 172.21.15.92:0/3078334434 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:20:51.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.212+0000 7fb5bffff700 1 --2- 172.21.15.92:0/3078334434 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5c80b4fc0 0x7fb5c80aaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:51.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.212+0000 7fb5c77fe700 1 --2- 172.21.15.92:0/3078334434 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5c80d2df0 0x7fb5c80aafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:51.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.212+0000 7fb5c6ffd700 1 -- 172.21.15.92:0/3078334434 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5c80b4fc0 msgr2=0x7fb5c80aaaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:51.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.212+0000 7fb5c6ffd700 1 --2- 172.21.15.92:0/3078334434 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5c80b4fc0 0x7fb5c80aaaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:51.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.212+0000 7fb5c6ffd700 1 -- 172.21.15.92:0/3078334434 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5c80d2df0 msgr2=0x7fb5c80aafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:51.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.212+0000 7fb5c6ffd700 1 --2- 172.21.15.92:0/3078334434 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5c80d2df0 0x7fb5c80aafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:51.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.212+0000 7fb5c6ffd700 1 -- 172.21.15.92:0/3078334434 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5b0009550 con 0x7fb5c80b1c60 2023-11-28T11:20:51.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.212+0000 7fb5bffff700 1 --2- 172.21.15.92:0/3078334434 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5c80b4fc0 0x7fb5c80aaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:20:51.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.212+0000 7fb5c6ffd700 1 --2- 172.21.15.92:0/3078334434 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5c80b1c60 0x7fb5c80cffe0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb5b800c750 tx=0x7fb5b800ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:51.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.212+0000 7fb5bf7fe700 1 -- 172.21.15.92:0/3078334434 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b8011800 con 0x7fb5c80b1c60 2023-11-28T11:20:51.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.212+0000 7fb5bf7fe700 1 -- 172.21.15.92:0/3078334434 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5b8011960 con 0x7fb5c80b1c60 2023-11-28T11:20:51.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.213+0000 7fb5cd3ff700 1 -- 172.21.15.92:0/3078334434 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5c80b88f0 con 0x7fb5c80b1c60 2023-11-28T11:20:51.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.213+0000 7fb5bf7fe700 1 -- 172.21.15.92:0/3078334434 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b8011cd0 con 0x7fb5c80b1c60 2023-11-28T11:20:51.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.213+0000 7fb5cd3ff700 1 -- 172.21.15.92:0/3078334434 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5c80d15e0 con 0x7fb5c80b1c60 2023-11-28T11:20:51.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.214+0000 7fb5cd3ff700 1 -- 172.21.15.92:0/3078334434 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5c80b21b0 con 0x7fb5c80b1c60 2023-11-28T11:20:51.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.214+0000 7fb5bf7fe700 1 -- 172.21.15.92:0/3078334434 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb5b8020070 con 0x7fb5c80b1c60 2023-11-28T11:20:51.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.215+0000 7fb5bf7fe700 1 --2- 172.21.15.92:0/3078334434 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb5a8068320 0x7fb5a806a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:51.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.215+0000 7fb5bf7fe700 1 -- 172.21.15.92:0/3078334434 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fb5b8085670 con 0x7fb5c80b1c60 2023-11-28T11:20:51.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.215+0000 7fb5bffff700 1 --2- 172.21.15.92:0/3078334434 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb5a8068320 0x7fb5a806a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:51.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.216+0000 7fb5bffff700 1 --2- 172.21.15.92:0/3078334434 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb5a8068320 0x7fb5a806a7d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0009840 tx=0x7fb5b0008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:51.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.220+0000 7fb5bf7fe700 1 -- 172.21.15.92:0/3078334434 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb5b8052130 con 0x7fb5c80b1c60 2023-11-28T11:20:51.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.410+0000 7fb5cd3ff700 1 -- 172.21.15.92:0/3078334434 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb5c80ca7b0 con 0x7fb5c80b1c60 2023-11-28T11:20:51.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.410+0000 7fb5bf7fe700 1 -- 172.21.15.92:0/3078334434 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb5b8055750 con 0x7fb5c80b1c60 2023-11-28T11:20:51.411 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-11-28T11:20:51.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.412+0000 7fb5cd3ff700 1 -- 172.21.15.92:0/3078334434 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb5a8068320 msgr2=0x7fb5a806a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:51.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.412+0000 7fb5cd3ff700 1 --2- 172.21.15.92:0/3078334434 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb5a8068320 0x7fb5a806a7d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0009840 tx=0x7fb5b0008030 comp rx=0 tx=0).stop 2023-11-28T11:20:51.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.413+0000 7fb5cd3ff700 1 -- 172.21.15.92:0/3078334434 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5c80b1c60 msgr2=0x7fb5c80cffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:51.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.413+0000 7fb5cd3ff700 1 --2- 172.21.15.92:0/3078334434 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5c80b1c60 0x7fb5c80cffe0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb5b800c750 tx=0x7fb5b800ca60 comp rx=0 tx=0).stop 2023-11-28T11:20:51.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.413+0000 7fb5cd3ff700 1 -- 172.21.15.92:0/3078334434 shutdown_connections 2023-11-28T11:20:51.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.413+0000 7fb5cd3ff700 1 --2- 172.21.15.92:0/3078334434 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb5c80d2df0 0x7fb5c80aafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:51.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.413+0000 7fb5cd3ff700 1 --2- 172.21.15.92:0/3078334434 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb5a8068320 0x7fb5a806a7d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:51.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.413+0000 7fb5cd3ff700 1 --2- 172.21.15.92:0/3078334434 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5c80b1c60 0x7fb5c80cffe0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:51.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.413+0000 7fb5cd3ff700 1 --2- 172.21.15.92:0/3078334434 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb5c80b4fc0 0x7fb5c80aaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:51.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.413+0000 7fb5cd3ff700 1 -- 172.21.15.92:0/3078334434 >> 172.21.15.92:0/3078334434 conn(0x7fb5c80719a0 msgr2=0x7fb5c80b66a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:51.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.413+0000 7fb5cd3ff700 1 -- 172.21.15.92:0/3078334434 shutdown_connections 2023-11-28T11:20:51.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.413+0000 7fb5cd3ff700 1 -- 172.21.15.92:0/3078334434 wait complete. 2023-11-28T11:20:51.432 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:20:51.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.630+0000 7fc719bff700 1 Processor -- start 2023-11-28T11:20:51.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.631+0000 7fc719bff700 1 -- start start 2023-11-28T11:20:51.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.632+0000 7fc719bff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7140cd460 0x7fc71406c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:51.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.632+0000 7fc719bff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7140b88e0 con 0x7fc7140d0d60 2023-11-28T11:20:51.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.632+0000 7fc719bff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7140cd8f0 con 0x7fc7140b8510 2023-11-28T11:20:51.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.632+0000 7fc719bff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7140cdbd0 con 0x7fc7140cd460 2023-11-28T11:20:51.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.632+0000 7fc713fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc7140d0d60 0x7fc7140d11e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51832/0 (socket says 172.21.15.92:51832) 2023-11-28T11:20:51.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.632+0000 7fc713fff700 1 -- 172.21.15.92:0/4293517299 learned_addr learned my addr 172.21.15.92:0/4293517299 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:20:51.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.632+0000 7fc70bfff700 1 --2- 172.21.15.92:0/4293517299 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7140cd460 0x7fc71406c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:51.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.632+0000 7fc712ffd700 1 -- 172.21.15.92:0/4293517299 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1333964849 0 0) 0x7fc7140b88e0 con 0x7fc7140d0d60 2023-11-28T11:20:51.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.633+0000 7fc712ffd700 1 -- 172.21.15.92:0/4293517299 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7000035b0 con 0x7fc7140d0d60 2023-11-28T11:20:51.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.633+0000 7fc712ffd700 1 -- 172.21.15.92:0/4293517299 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2540927979 0 0) 0x7fc7000035b0 con 0x7fc7140d0d60 2023-11-28T11:20:51.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.633+0000 7fc712ffd700 1 -- 172.21.15.92:0/4293517299 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7140cd460 msgr2=0x7fc71406c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:51.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.633+0000 7fc712ffd700 1 --2- 172.21.15.92:0/4293517299 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7140cd460 0x7fc71406c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:51.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.633+0000 7fc712ffd700 1 -- 172.21.15.92:0/4293517299 >> v1:172.21.15.169:6789/0 conn(0x7fc7140b8510 legacy=0x7fc71406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:51.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.633+0000 7fc712ffd700 1 -- 172.21.15.92:0/4293517299 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7140cfd10 con 0x7fc7140d0d60 2023-11-28T11:20:51.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.633+0000 7fc712ffd700 1 -- 172.21.15.92:0/4293517299 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532161587 0 0) 0x7fc70c0019f0 con 0x7fc7140d0d60 2023-11-28T11:20:51.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.633+0000 7fc712ffd700 1 -- 172.21.15.92:0/4293517299 >> v1:172.21.15.92:6789/0 conn(0x7fc7140d0d60 legacy=0x7fc7140d11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:51.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.633+0000 7fc712ffd700 1 --2- 172.21.15.92:0/4293517299 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc700003a30 0x7fc700003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:51.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.633+0000 7fc712ffd700 1 --2- 172.21.15.92:0/4293517299 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7140cd460 0x7fc700004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:51.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.633+0000 7fc712ffd700 1 --2- 172.21.15.92:0/4293517299 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc700004d90 0x7fc700005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:51.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.633+0000 7fc712ffd700 1 -- 172.21.15.92:0/4293517299 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7140cdbd0 con 0x7fc700004d90 2023-11-28T11:20:51.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.633+0000 7fc712ffd700 1 -- 172.21.15.92:0/4293517299 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70000bac0 con 0x7fc700003a30 2023-11-28T11:20:51.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.633+0000 7fc712ffd700 1 -- 172.21.15.92:0/4293517299 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc70000bc30 con 0x7fc7140cd460 2023-11-28T11:20:51.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.633+0000 7fc713fff700 1 --2- 172.21.15.92:0/4293517299 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc700004d90 0x7fc700005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:51.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.633+0000 7fc70bfff700 1 --2- 172.21.15.92:0/4293517299 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7140cd460 0x7fc700004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:51.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.633+0000 7fc7137fe700 1 --2- 172.21.15.92:0/4293517299 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc700003a30 0x7fc700003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:51.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.634+0000 7fc713fff700 1 -- 172.21.15.92:0/4293517299 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7140cd460 msgr2=0x7fc700004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:51.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.634+0000 7fc713fff700 1 --2- 172.21.15.92:0/4293517299 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7140cd460 0x7fc700004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:51.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.634+0000 7fc713fff700 1 -- 172.21.15.92:0/4293517299 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc700003a30 msgr2=0x7fc700003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:51.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.634+0000 7fc713fff700 1 --2- 172.21.15.92:0/4293517299 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc700003a30 0x7fc700003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:51.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.634+0000 7fc713fff700 1 -- 172.21.15.92:0/4293517299 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7000035b0 con 0x7fc700004d90 2023-11-28T11:20:51.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.634+0000 7fc713fff700 1 --2- 172.21.15.92:0/4293517299 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc700004d90 0x7fc700005200 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc70c00a810 tx=0x7fc700005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96bbd469b0000acd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:51.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.634+0000 7fc712ffd700 1 -- 172.21.15.92:0/4293517299 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70c00a250 con 0x7fc700004d90 2023-11-28T11:20:51.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.634+0000 7fc712ffd700 1 -- 172.21.15.92:0/4293517299 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc70c012600 con 0x7fc700004d90 2023-11-28T11:20:51.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.634+0000 7fc712ffd700 1 -- 172.21.15.92:0/4293517299 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70c0129b0 con 0x7fc700004d90 2023-11-28T11:20:51.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.634+0000 7fc719bff700 1 -- 172.21.15.92:0/4293517299 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc700004d90 msgr2=0x7fc700005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:51.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.634+0000 7fc719bff700 1 --2- 172.21.15.92:0/4293517299 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc700004d90 0x7fc700005200 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc70c00a810 tx=0x7fc700005880 comp rx=0 tx=0).stop 2023-11-28T11:20:51.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.634+0000 7fc713fff700 1 -- 172.21.15.92:0/4293517299 reap_dead start 2023-11-28T11:20:51.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.635+0000 7fc719bff700 1 -- 172.21.15.92:0/4293517299 shutdown_connections 2023-11-28T11:20:51.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.635+0000 7fc719bff700 1 -- 172.21.15.92:0/4293517299 >> 172.21.15.92:0/4293517299 conn(0x7fc7140719a0 msgr2=0x7fc714071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:51.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.635+0000 7fc719bff700 1 -- 172.21.15.92:0/4293517299 shutdown_connections 2023-11-28T11:20:51.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.635+0000 7fc719bff700 1 -- 172.21.15.92:0/4293517299 wait complete. 2023-11-28T11:20:51.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.636+0000 7fc719bff700 1 Processor -- start 2023-11-28T11:20:51.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.637+0000 7fc719bff700 1 -- start start 2023-11-28T11:20:51.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.637+0000 7fc719bff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc700004d90 0x7fc7140b1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:51.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.637+0000 7fc719bff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc714155050 0x7fc71415b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:51.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.637+0000 7fc719bff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc71415b620 0x7fc71415ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:51.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.637+0000 7fc719bff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7141f3e10 con 0x7fc714155050 2023-11-28T11:20:51.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.637+0000 7fc719bff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7141f3190 con 0x7fc700004d90 2023-11-28T11:20:51.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.638+0000 7fc719bff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7141f3470 con 0x7fc71415b620 2023-11-28T11:20:51.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.638+0000 7fc70bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc714155050 0x7fc71415b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:51.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.638+0000 7fc70bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc714155050 0x7fc71415b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41514/0 (socket says 172.21.15.92:41514) 2023-11-28T11:20:51.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.638+0000 7fc70bfff700 1 -- 172.21.15.92:0/82511756 learned_addr learned my addr 172.21.15.92:0/82511756 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:20:51.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.638+0000 7fc7137fe700 1 --2- 172.21.15.92:0/82511756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc700004d90 0x7fc7140b1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:51.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.638+0000 7fc713fff700 1 --2- 172.21.15.92:0/82511756 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc71415b620 0x7fc71415ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:51.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.638+0000 7fc70bfff700 1 -- 172.21.15.92:0/82511756 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc71415b620 msgr2=0x7fc71415ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:51.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.638+0000 7fc70bfff700 1 --2- 172.21.15.92:0/82511756 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc71415b620 0x7fc71415ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:51.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.638+0000 7fc70bfff700 1 -- 172.21.15.92:0/82511756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc700004d90 msgr2=0x7fc7140b1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:51.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.638+0000 7fc70bfff700 1 --2- 172.21.15.92:0/82511756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc700004d90 0x7fc7140b1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:51.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.638+0000 7fc70bfff700 1 -- 172.21.15.92:0/82511756 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7140d29c0 con 0x7fc714155050 2023-11-28T11:20:51.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.638+0000 7fc7137fe700 1 --2- 172.21.15.92:0/82511756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc700004d90 0x7fc7140b1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:20:51.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.638+0000 7fc70bfff700 1 --2- 172.21.15.92:0/82511756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc714155050 0x7fc71415b0e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc011aa0 tx=0x7fc6fc011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:51.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.638+0000 7fc710ff9700 1 -- 172.21.15.92:0/82511756 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc00f980 con 0x7fc714155050 2023-11-28T11:20:51.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.639+0000 7fc710ff9700 1 -- 172.21.15.92:0/82511756 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6fc00fae0 con 0x7fc714155050 2023-11-28T11:20:51.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.639+0000 7fc719bff700 1 -- 172.21.15.92:0/82511756 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7140cfff0 con 0x7fc714155050 2023-11-28T11:20:51.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.639+0000 7fc710ff9700 1 -- 172.21.15.92:0/82511756 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc00fe90 con 0x7fc714155050 2023-11-28T11:20:51.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.639+0000 7fc719bff700 1 -- 172.21.15.92:0/82511756 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7140d2b50 con 0x7fc714155050 2023-11-28T11:20:51.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.640+0000 7fc719bff700 1 -- 172.21.15.92:0/82511756 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7141f3f50 con 0x7fc714155050 2023-11-28T11:20:51.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.641+0000 7fc710ff9700 1 -- 172.21.15.92:0/82511756 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc6fc014440 con 0x7fc714155050 2023-11-28T11:20:51.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.642+0000 7fc710ff9700 1 --2- 172.21.15.92:0/82511756 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc6f40682f0 0x7fc6f406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:51.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.642+0000 7fc710ff9700 1 -- 172.21.15.92:0/82511756 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fc6fc086310 con 0x7fc714155050 2023-11-28T11:20:51.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.642+0000 7fc7137fe700 1 --2- 172.21.15.92:0/82511756 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc6f40682f0 0x7fc6f406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:51.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.643+0000 7fc7137fe700 1 --2- 172.21.15.92:0/82511756 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc6f40682f0 0x7fc6f406a7a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc704007fd0 tx=0x7fc704009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:51.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.646+0000 7fc710ff9700 1 -- 172.21.15.92:0/82511756 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc7141f3f50 con 0x7fc714155050 2023-11-28T11:20:51.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.838+0000 7fc719bff700 1 -- 172.21.15.92:0/82511756 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc7141f3f50 con 0x7fc714155050 2023-11-28T11:20:51.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.839+0000 7fc710ff9700 1 -- 172.21.15.92:0/82511756 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v19) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc7141f3f50 con 0x7fc714155050 2023-11-28T11:20:51.840 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-11-28T11:20:51.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.841+0000 7fc719bff700 1 -- 172.21.15.92:0/82511756 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc6f40682f0 msgr2=0x7fc6f406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:51.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.841+0000 7fc719bff700 1 --2- 172.21.15.92:0/82511756 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc6f40682f0 0x7fc6f406a7a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc704007fd0 tx=0x7fc704009040 comp rx=0 tx=0).stop 2023-11-28T11:20:51.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.841+0000 7fc719bff700 1 -- 172.21.15.92:0/82511756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc714155050 msgr2=0x7fc71415b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:51.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.841+0000 7fc719bff700 1 --2- 172.21.15.92:0/82511756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc714155050 0x7fc71415b0e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc011aa0 tx=0x7fc6fc011e60 comp rx=0 tx=0).stop 2023-11-28T11:20:51.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.841+0000 7fc719bff700 1 -- 172.21.15.92:0/82511756 shutdown_connections 2023-11-28T11:20:51.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.841+0000 7fc719bff700 1 --2- 172.21.15.92:0/82511756 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc71415b620 0x7fc71415ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:51.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.841+0000 7fc719bff700 1 --2- 172.21.15.92:0/82511756 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc700004d90 0x7fc7140b1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:51.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.841+0000 7fc719bff700 1 --2- 172.21.15.92:0/82511756 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc6f40682f0 0x7fc6f406a7a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:51.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.841+0000 7fc719bff700 1 --2- 172.21.15.92:0/82511756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc714155050 0x7fc71415b0e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:51.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.841+0000 7fc719bff700 1 -- 172.21.15.92:0/82511756 >> 172.21.15.92:0/82511756 conn(0x7fc7140719a0 msgr2=0x7fc7140b27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:51.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.841+0000 7fc719bff700 1 -- 172.21.15.92:0/82511756 shutdown_connections 2023-11-28T11:20:51.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:51.841+0000 7fc719bff700 1 -- 172.21.15.92:0/82511756 wait complete. 2023-11-28T11:20:51.861 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:20:52.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.056+0000 7fa84b9ff700 1 Processor -- start 2023-11-28T11:20:52.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.057+0000 7fa84b9ff700 1 -- start start 2023-11-28T11:20:52.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.058+0000 7fa84b9ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8440cd460 0x7fa84406c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:52.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.058+0000 7fa84b9ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa8440d0d60 0x7fa8440d11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:52.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.058+0000 7fa84b9ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8440b88e0 con 0x7fa8440b8510 2023-11-28T11:20:52.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.058+0000 7fa84b9ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8440cdbd0 con 0x7fa8440d0d60 2023-11-28T11:20:52.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.058+0000 7fa84b9ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8440d02d0 con 0x7fa8440cd460 2023-11-28T11:20:52.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.058+0000 7fa84979b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fa8440b8510 0x7fa84406b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51836/0 (socket says 172.21.15.92:51836) 2023-11-28T11:20:52.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.058+0000 7fa84979b700 1 -- 172.21.15.92:0/1274803274 learned_addr learned my addr 172.21.15.92:0/1274803274 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:20:52.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.058+0000 7fa849f9c700 1 --2- 172.21.15.92:0/1274803274 >> v2:172.21.15.169:3300/0 conn(0x7fa8440d0d60 0x7fa8440d11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:52.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.058+0000 7fa848f9a700 1 --2- 172.21.15.92:0/1274803274 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8440cd460 0x7fa84406c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:52.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.058+0000 7fa83bfff700 1 -- 172.21.15.92:0/1274803274 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3659192800 0 0) 0x7fa8440b88e0 con 0x7fa8440b8510 2023-11-28T11:20:52.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.058+0000 7fa83bfff700 1 -- 172.21.15.92:0/1274803274 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8300035b0 con 0x7fa8440b8510 2023-11-28T11:20:52.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.059+0000 7fa83bfff700 1 -- 172.21.15.92:0/1274803274 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1665829646 0 0) 0x7fa8300035b0 con 0x7fa8440b8510 2023-11-28T11:20:52.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.059+0000 7fa83bfff700 1 -- 172.21.15.92:0/1274803274 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8440cd460 msgr2=0x7fa84406c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:52.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.059+0000 7fa83bfff700 1 --2- 172.21.15.92:0/1274803274 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8440cd460 0x7fa84406c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:52.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.059+0000 7fa83bfff700 1 -- 172.21.15.92:0/1274803274 >> v2:172.21.15.169:3300/0 conn(0x7fa8440d0d60 msgr2=0x7fa8440d11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:52.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.059+0000 7fa83bfff700 1 --2- 172.21.15.92:0/1274803274 >> v2:172.21.15.169:3300/0 conn(0x7fa8440d0d60 0x7fa8440d11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:52.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.059+0000 7fa83bfff700 1 -- 172.21.15.92:0/1274803274 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8440cd8f0 con 0x7fa8440b8510 2023-11-28T11:20:52.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.059+0000 7fa83bfff700 1 -- 172.21.15.92:0/1274803274 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532161587 0 0) 0x7fa840001670 con 0x7fa8440b8510 2023-11-28T11:20:52.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.059+0000 7fa83bfff700 1 -- 172.21.15.92:0/1274803274 >> v1:172.21.15.92:6789/0 conn(0x7fa8440b8510 legacy=0x7fa84406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:52.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.059+0000 7fa83bfff700 1 --2- 172.21.15.92:0/1274803274 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8440cd460 0x7fa830003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:52.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.059+0000 7fa83bfff700 1 --2- 172.21.15.92:0/1274803274 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa830003fb0 0x7fa830004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:52.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.059+0000 7fa83bfff700 1 --2- 172.21.15.92:0/1274803274 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa830004d90 0x7fa830005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:52.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.059+0000 7fa83bfff700 1 -- 172.21.15.92:0/1274803274 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8440cdbd0 con 0x7fa830003fb0 2023-11-28T11:20:52.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.059+0000 7fa83bfff700 1 -- 172.21.15.92:0/1274803274 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8440d02d0 con 0x7fa830004d90 2023-11-28T11:20:52.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.059+0000 7fa83bfff700 1 -- 172.21.15.92:0/1274803274 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa83000ba10 con 0x7fa8440cd460 2023-11-28T11:20:52.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.059+0000 7fa848f9a700 1 --2- 172.21.15.92:0/1274803274 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa830003fb0 0x7fa830004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:52.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.059+0000 7fa84979b700 1 --2- 172.21.15.92:0/1274803274 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8440cd460 0x7fa830003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:52.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.059+0000 7fa849f9c700 1 --2- 172.21.15.92:0/1274803274 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa830004d90 0x7fa830005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:52.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.059+0000 7fa848f9a700 1 -- 172.21.15.92:0/1274803274 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8440cd460 msgr2=0x7fa830003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:52.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.059+0000 7fa848f9a700 1 --2- 172.21.15.92:0/1274803274 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8440cd460 0x7fa830003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:52.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.059+0000 7fa848f9a700 1 -- 172.21.15.92:0/1274803274 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa830004d90 msgr2=0x7fa830005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:52.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.059+0000 7fa848f9a700 1 --2- 172.21.15.92:0/1274803274 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa830004d90 0x7fa830005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:52.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.059+0000 7fa848f9a700 1 -- 172.21.15.92:0/1274803274 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8300035b0 con 0x7fa830003fb0 2023-11-28T11:20:52.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.059+0000 7fa849f9c700 1 --2- 172.21.15.92:0/1274803274 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa830004d90 0x7fa830005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:20:52.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.060+0000 7fa848f9a700 1 --2- 172.21.15.92:0/1274803274 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa830003fb0 0x7fa830004850 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa834009b60 tx=0x7fa830005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b137f7aede080a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:52.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.060+0000 7fa83bfff700 1 -- 172.21.15.92:0/1274803274 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83400e6c0 con 0x7fa830003fb0 2023-11-28T11:20:52.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.060+0000 7fa83bfff700 1 -- 172.21.15.92:0/1274803274 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa83400e820 con 0x7fa830003fb0 2023-11-28T11:20:52.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.060+0000 7fa83bfff700 1 -- 172.21.15.92:0/1274803274 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83400eb90 con 0x7fa830003fb0 2023-11-28T11:20:52.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.060+0000 7fa84b9ff700 1 -- 172.21.15.92:0/1274803274 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa830003fb0 msgr2=0x7fa830004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:52.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.060+0000 7fa84b9ff700 1 --2- 172.21.15.92:0/1274803274 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa830003fb0 0x7fa830004850 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa834009b60 tx=0x7fa830005880 comp rx=0 tx=0).stop 2023-11-28T11:20:52.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.060+0000 7fa849f9c700 1 -- 172.21.15.92:0/1274803274 reap_dead start 2023-11-28T11:20:52.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.061+0000 7fa84b9ff700 1 -- 172.21.15.92:0/1274803274 shutdown_connections 2023-11-28T11:20:52.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.061+0000 7fa84b9ff700 1 -- 172.21.15.92:0/1274803274 >> 172.21.15.92:0/1274803274 conn(0x7fa8440719a0 msgr2=0x7fa844071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:52.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.061+0000 7fa84b9ff700 1 -- 172.21.15.92:0/1274803274 shutdown_connections 2023-11-28T11:20:52.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.061+0000 7fa84b9ff700 1 -- 172.21.15.92:0/1274803274 wait complete. 2023-11-28T11:20:52.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.063+0000 7fa84b9ff700 1 Processor -- start 2023-11-28T11:20:52.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.063+0000 7fa84b9ff700 1 -- start start 2023-11-28T11:20:52.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.064+0000 7fa84b9ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa830003fb0 0x7fa8441e86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:52.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.064+0000 7fa84b9ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8441e8be0 0x7fa8440cca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:52.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.064+0000 7fa84b9ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8440ccfb0 0x7fa844158db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:52.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.064+0000 7fa84b9ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8440cb730 con 0x7fa8440ccfb0 2023-11-28T11:20:52.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.064+0000 7fa84b9ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8440cb9d0 con 0x7fa830003fb0 2023-11-28T11:20:52.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.064+0000 7fa84b9ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8440cbcb0 con 0x7fa8441e8be0 2023-11-28T11:20:52.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.065+0000 7fa849f9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8440ccfb0 0x7fa844158db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:52.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.065+0000 7fa849f9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8440ccfb0 0x7fa844158db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41534/0 (socket says 172.21.15.92:41534) 2023-11-28T11:20:52.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.065+0000 7fa849f9c700 1 -- 172.21.15.92:0/2688592400 learned_addr learned my addr 172.21.15.92:0/2688592400 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:20:52.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.065+0000 7fa848f9a700 1 --2- 172.21.15.92:0/2688592400 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8441e8be0 0x7fa8440cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:52.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.065+0000 7fa84979b700 1 --2- 172.21.15.92:0/2688592400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa830003fb0 0x7fa8441e86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:52.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.065+0000 7fa849f9c700 1 -- 172.21.15.92:0/2688592400 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8441e8be0 msgr2=0x7fa8440cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:52.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.065+0000 7fa849f9c700 1 --2- 172.21.15.92:0/2688592400 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8441e8be0 0x7fa8440cca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:52.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.065+0000 7fa849f9c700 1 -- 172.21.15.92:0/2688592400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa830003fb0 msgr2=0x7fa8441e86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:52.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.065+0000 7fa849f9c700 1 --2- 172.21.15.92:0/2688592400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa830003fb0 0x7fa8441e86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:52.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.065+0000 7fa849f9c700 1 -- 172.21.15.92:0/2688592400 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8440cfff0 con 0x7fa8440ccfb0 2023-11-28T11:20:52.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.065+0000 7fa849f9c700 1 --2- 172.21.15.92:0/2688592400 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8440ccfb0 0x7fa844158db0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa8440d1130 tx=0x7fa83c0104a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:52.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.066+0000 7fa839ffb700 1 -- 172.21.15.92:0/2688592400 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83c00e050 con 0x7fa8440ccfb0 2023-11-28T11:20:52.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.066+0000 7fa839ffb700 1 -- 172.21.15.92:0/2688592400 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa83c00f040 con 0x7fa8440ccfb0 2023-11-28T11:20:52.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.066+0000 7fa839ffb700 1 -- 172.21.15.92:0/2688592400 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83c007ea0 con 0x7fa8440ccfb0 2023-11-28T11:20:52.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.066+0000 7fa84b9ff700 1 -- 172.21.15.92:0/2688592400 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8440cfd10 con 0x7fa8440ccfb0 2023-11-28T11:20:52.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.066+0000 7fa84b9ff700 1 -- 172.21.15.92:0/2688592400 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8440d29c0 con 0x7fa8440ccfb0 2023-11-28T11:20:52.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.067+0000 7fa84b9ff700 1 -- 172.21.15.92:0/2688592400 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa844067880 con 0x7fa8440ccfb0 2023-11-28T11:20:52.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.068+0000 7fa839ffb700 1 -- 172.21.15.92:0/2688592400 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa83c010d00 con 0x7fa8440ccfb0 2023-11-28T11:20:52.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.068+0000 7fa839ffb700 1 --2- 172.21.15.92:0/2688592400 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa8280682f0 0x7fa82806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:52.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.068+0000 7fa839ffb700 1 -- 172.21.15.92:0/2688592400 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa83c085c90 con 0x7fa8440ccfb0 2023-11-28T11:20:52.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.069+0000 7fa84979b700 1 --2- 172.21.15.92:0/2688592400 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa8280682f0 0x7fa82806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:52.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.069+0000 7fa84979b700 1 --2- 172.21.15.92:0/2688592400 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa8280682f0 0x7fa82806a7a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa8400017a0 tx=0x7fa8400014b0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:52.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.072+0000 7fa839ffb700 1 -- 172.21.15.92:0/2688592400 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa83c052750 con 0x7fa8440ccfb0 2023-11-28T11:20:52.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.303+0000 7fa84b9ff700 1 -- 172.21.15.92:0/2688592400 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa8440b6430 con 0x7fa8440ccfb0 2023-11-28T11:20:52.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.304+0000 7fa839ffb700 1 -- 172.21.15.92:0/2688592400 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3527 (secure 0 0 0) 0x7fa83c01ade0 con 0x7fa8440ccfb0 2023-11-28T11:20:52.305 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:20:52.305 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/3257209990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3257209990},{"type":"v1","addr":"172.21.15.92:6836","nonce":3257209990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2193338949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2193338949},{"type":"v1","addr":"172.21.15.169:6834","nonce":2193338949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4308,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/471035619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":471035619},{"type":"v1","addr":"172.21.15.92:6837","nonce":471035619}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4326,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2067251916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2067251916},{"type":"v1","addr":"172.21.15.169:6835","nonce":2067251916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-11-28T11:20:52.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.306+0000 7fa84b9ff700 1 -- 172.21.15.92:0/2688592400 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa8280682f0 msgr2=0x7fa82806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:52.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.306+0000 7fa84b9ff700 1 --2- 172.21.15.92:0/2688592400 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa8280682f0 0x7fa82806a7a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa8400017a0 tx=0x7fa8400014b0 comp rx=0 tx=0).stop 2023-11-28T11:20:52.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.306+0000 7fa84b9ff700 1 -- 172.21.15.92:0/2688592400 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8440ccfb0 msgr2=0x7fa844158db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:52.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.306+0000 7fa84b9ff700 1 --2- 172.21.15.92:0/2688592400 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8440ccfb0 0x7fa844158db0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa8440d1130 tx=0x7fa83c0104a0 comp rx=0 tx=0).stop 2023-11-28T11:20:52.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.306+0000 7fa84b9ff700 1 -- 172.21.15.92:0/2688592400 shutdown_connections 2023-11-28T11:20:52.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.306+0000 7fa84b9ff700 1 --2- 172.21.15.92:0/2688592400 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa8280682f0 0x7fa82806a7a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:52.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.306+0000 7fa84b9ff700 1 --2- 172.21.15.92:0/2688592400 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8440ccfb0 0x7fa844158db0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:52.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.306+0000 7fa84b9ff700 1 --2- 172.21.15.92:0/2688592400 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa830003fb0 0x7fa8441e86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:52.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.306+0000 7fa84b9ff700 1 --2- 172.21.15.92:0/2688592400 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8441e8be0 0x7fa8440cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:52.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.306+0000 7fa84b9ff700 1 -- 172.21.15.92:0/2688592400 >> 172.21.15.92:0/2688592400 conn(0x7fa8440719a0 msgr2=0x7fa8440d2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:52.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.306+0000 7fa84b9ff700 1 -- 172.21.15.92:0/2688592400 shutdown_connections 2023-11-28T11:20:52.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.306+0000 7fa84b9ff700 1 -- 172.21.15.92:0/2688592400 wait complete. 2023-11-28T11:20:52.309 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 3 2023-11-28T11:20:52.326 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:20:52.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.521+0000 7f79d43ff700 1 Processor -- start 2023-11-28T11:20:52.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.522+0000 7f79d43ff700 1 -- start start 2023-11-28T11:20:52.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.523+0000 7f79d43ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f79cc0d2df0 0x7f79cc0cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:52.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.523+0000 7f79d43ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f79cc0b4fc0 0x7f79cc0d31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:52.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.523+0000 7f79d43ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79cc0b1c60 0x7f79cc0cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:52.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.523+0000 7f79d43ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79cc0d3d30 con 0x7f79cc0b4fc0 2023-11-28T11:20:52.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.523+0000 7f79d43ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79cc0d1950 con 0x7f79cc0d2df0 2023-11-28T11:20:52.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.523+0000 7f79d43ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79cc0b2500 con 0x7f79cc0b1c60 2023-11-28T11:20:52.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.524+0000 7f79d199a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f79cc0b4fc0 0x7f79cc0d31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:52.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.524+0000 7f79d219b700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f79cc0d2df0 0x7f79cc0cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:52.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.524+0000 7f79d199a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f79cc0b4fc0 0x7f79cc0d31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41546/0 (socket says 172.21.15.92:41546) 2023-11-28T11:20:52.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.524+0000 7f79d199a700 1 -- 172.21.15.92:0/920952280 learned_addr learned my addr 172.21.15.92:0/920952280 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:20:52.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.524+0000 7f79d299c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79cc0b1c60 0x7f79cc0cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:52.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.524+0000 7f79d199a700 1 -- 172.21.15.92:0/920952280 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79cc0b1c60 msgr2=0x7f79cc0cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:52.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.524+0000 7f79d199a700 1 --2- 172.21.15.92:0/920952280 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79cc0b1c60 0x7f79cc0cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:52.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.524+0000 7f79d199a700 1 -- 172.21.15.92:0/920952280 >> v2:172.21.15.169:3300/0 conn(0x7f79cc0d2df0 msgr2=0x7f79cc0cd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:52.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.524+0000 7f79d199a700 1 --2- 172.21.15.92:0/920952280 >> v2:172.21.15.169:3300/0 conn(0x7f79cc0d2df0 0x7f79cc0cd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:52.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.524+0000 7f79d199a700 1 -- 172.21.15.92:0/920952280 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79cc0d3a50 con 0x7f79cc0b4fc0 2023-11-28T11:20:52.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.524+0000 7f79d199a700 1 --2- 172.21.15.92:0/920952280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79cc0b4fc0 0x7f79cc0d31c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f79cc0d27f0 tx=0x7f79bc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61407b649bc9c48d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:52.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.526+0000 7f79d1199700 1 -- 172.21.15.92:0/920952280 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79bc011830 con 0x7f79cc0b4fc0 2023-11-28T11:20:52.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.526+0000 7f79d1199700 1 -- 172.21.15.92:0/920952280 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79bc011990 con 0x7f79cc0b4fc0 2023-11-28T11:20:52.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.527+0000 7f79d1199700 1 -- 172.21.15.92:0/920952280 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79bc011d20 con 0x7f79cc0b4fc0 2023-11-28T11:20:52.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.527+0000 7f79d43ff700 1 -- 172.21.15.92:0/920952280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79cc0b4fc0 msgr2=0x7f79cc0d31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:52.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.527+0000 7f79d43ff700 1 --2- 172.21.15.92:0/920952280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79cc0b4fc0 0x7f79cc0d31c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f79cc0d27f0 tx=0x7f79bc009bd0 comp rx=0 tx=0).stop 2023-11-28T11:20:52.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.527+0000 7f79d43ff700 1 -- 172.21.15.92:0/920952280 shutdown_connections 2023-11-28T11:20:52.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.527+0000 7f79d43ff700 1 --2- 172.21.15.92:0/920952280 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79cc0b1c60 0x7f79cc0cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:52.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.527+0000 7f79d43ff700 1 --2- 172.21.15.92:0/920952280 >> v2:172.21.15.169:3300/0 conn(0x7f79cc0d2df0 0x7f79cc0cd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:52.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.527+0000 7f79d43ff700 1 --2- 172.21.15.92:0/920952280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79cc0b4fc0 0x7f79cc0d31c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:52.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.527+0000 7f79d43ff700 1 -- 172.21.15.92:0/920952280 >> 172.21.15.92:0/920952280 conn(0x7f79cc0719a0 msgr2=0x7f79cc071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:52.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.527+0000 7f79d43ff700 1 -- 172.21.15.92:0/920952280 shutdown_connections 2023-11-28T11:20:52.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.527+0000 7f79d43ff700 1 -- 172.21.15.92:0/920952280 wait complete. 2023-11-28T11:20:52.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.529+0000 7f79d43ff700 1 Processor -- start 2023-11-28T11:20:52.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.529+0000 7f79d43ff700 1 -- start start 2023-11-28T11:20:52.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.530+0000 7f79d43ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79cc0b1c60 0x7f79cc14e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:52.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.530+0000 7f79d43ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79cc0b4fc0 0x7f79cc0cfc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:52.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.530+0000 7f79d43ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79cc0d2df0 0x7f79cc0d0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:52.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.530+0000 7f79d43ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79cc1e6b80 con 0x7f79cc0b4fc0 2023-11-28T11:20:52.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.530+0000 7f79d43ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79cc1e61e0 con 0x7f79cc0b1c60 2023-11-28T11:20:52.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.530+0000 7f79d43ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79cc1e64c0 con 0x7f79cc0d2df0 2023-11-28T11:20:52.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.530+0000 7f79d199a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79cc0b4fc0 0x7f79cc0cfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:52.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.530+0000 7f79d199a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79cc0b4fc0 0x7f79cc0cfc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41554/0 (socket says 172.21.15.92:41554) 2023-11-28T11:20:52.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.530+0000 7f79d199a700 1 -- 172.21.15.92:0/1073838747 learned_addr learned my addr 172.21.15.92:0/1073838747 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:20:52.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.531+0000 7f79d219b700 1 --2- 172.21.15.92:0/1073838747 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79cc0b1c60 0x7f79cc14e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:52.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.531+0000 7f79d299c700 1 --2- 172.21.15.92:0/1073838747 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79cc0d2df0 0x7f79cc0d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:52.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.531+0000 7f79d199a700 1 -- 172.21.15.92:0/1073838747 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79cc0d2df0 msgr2=0x7f79cc0d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:52.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.531+0000 7f79d199a700 1 --2- 172.21.15.92:0/1073838747 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79cc0d2df0 0x7f79cc0d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:52.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.531+0000 7f79d199a700 1 -- 172.21.15.92:0/1073838747 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79cc0b1c60 msgr2=0x7f79cc14e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:52.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.531+0000 7f79d199a700 1 --2- 172.21.15.92:0/1073838747 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79cc0b1c60 0x7f79cc14e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:52.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.531+0000 7f79d199a700 1 -- 172.21.15.92:0/1073838747 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79bc009550 con 0x7f79cc0b4fc0 2023-11-28T11:20:52.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.531+0000 7f79d199a700 1 --2- 172.21.15.92:0/1073838747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79cc0b4fc0 0x7f79cc0cfc20 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f79bc014040 tx=0x7f79bc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:52.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.531+0000 7f79c2ffd700 1 -- 172.21.15.92:0/1073838747 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79bc004450 con 0x7f79cc0b4fc0 2023-11-28T11:20:52.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.531+0000 7f79c2ffd700 1 -- 172.21.15.92:0/1073838747 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79bc01e070 con 0x7f79cc0b4fc0 2023-11-28T11:20:52.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.531+0000 7f79c2ffd700 1 -- 172.21.15.92:0/1073838747 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79bc0038e0 con 0x7f79cc0b4fc0 2023-11-28T11:20:52.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.531+0000 7f79d43ff700 1 -- 172.21.15.92:0/1073838747 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79cc0b88f0 con 0x7f79cc0b4fc0 2023-11-28T11:20:52.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.532+0000 7f79d43ff700 1 -- 172.21.15.92:0/1073838747 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79cc0d15e0 con 0x7f79cc0b4fc0 2023-11-28T11:20:52.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.533+0000 7f79d43ff700 1 -- 172.21.15.92:0/1073838747 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79cc0b21b0 con 0x7f79cc0b4fc0 2023-11-28T11:20:52.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.533+0000 7f79c2ffd700 1 -- 172.21.15.92:0/1073838747 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f79bc00f050 con 0x7f79cc0b4fc0 2023-11-28T11:20:52.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.534+0000 7f79c2ffd700 1 --2- 172.21.15.92:0/1073838747 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f79b00682f0 0x7f79b006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:52.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.534+0000 7f79c2ffd700 1 -- 172.21.15.92:0/1073838747 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f79bc0867f0 con 0x7f79cc0b4fc0 2023-11-28T11:20:52.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.534+0000 7f79d219b700 1 --2- 172.21.15.92:0/1073838747 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f79b00682f0 0x7f79b006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:52.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.535+0000 7f79d219b700 1 --2- 172.21.15.92:0/1073838747 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f79b00682f0 0x7f79b006a7a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f79c8009b60 tx=0x7f79c8009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:52.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.538+0000 7f79c2ffd700 1 -- 172.21.15.92:0/1073838747 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f79bc0532b0 con 0x7f79cc0b4fc0 2023-11-28T11:20:52.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.731+0000 7f79d43ff700 1 -- 172.21.15.92:0/1073838747 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f79cc067170 con 0x7f79cc0b4fc0 2023-11-28T11:20:52.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.732+0000 7f79c2ffd700 1 -- 172.21.15.92:0/1073838747 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+14264 (secure 0 0 0) 0x7f79bc020090 con 0x7f79cc0b4fc0 2023-11-28T11:20:52.734 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:20:52.734 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":19,"fsid":"68956e0c-c16f-47cd-a220-f98364b0831d","created":"2023-11-28T11:20:28.697573+0000","modified":"2023-11-28T11:20:51.512190+0000","last_up_change":"2023-11-28T11:20:41.257011+0000","last_in_change":"2023-11-28T11:20:32.249767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:20:41.925237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:20:48.682497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6abe483f-0e03-4a26-b9cf-62041291747e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":115823},{"type":"v1","addr":"172.21.15.92:6809","nonce":115823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":115823},{"type":"v1","addr":"172.21.15.92:6813","nonce":115823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":115823},{"type":"v1","addr":"172.21.15.92:6821","nonce":115823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":115823},{"type":"v1","addr":"172.21.15.92:6817","nonce":115823}]},"public_addr":"172.21.15.92:6809/115823","cluster_addr":"172.21.15.92:6813/115823","heartbeat_back_addr":"172.21.15.92:6821/115823","heartbeat_front_addr":"172.21.15.92:6817/115823","state":["exists","up"]},{"osd":1,"uuid":"13c1de73-1a8c-4d7e-8a28-9dae52dfe089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":115824},{"type":"v1","addr":"172.21.15.92:6812","nonce":115824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":115824},{"type":"v1","addr":"172.21.15.92:6816","nonce":115824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":115824},{"type":"v1","addr":"172.21.15.92:6823","nonce":115824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":115824},{"type":"v1","addr":"172.21.15.92:6820","nonce":115824}]},"public_addr":"172.21.15.92:6812/115824","cluster_addr":"172.21.15.92:6816/115824","heartbeat_back_addr":"172.21.15.92:6823/115824","heartbeat_front_addr":"172.21.15.92:6820/115824","state":["exists","up"]},{"osd":2,"uuid":"25788a55-112f-4f00-810d-2bc37cf76778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":115821},{"type":"v1","addr":"172.21.15.92:6801","nonce":115821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":115821},{"type":"v1","addr":"172.21.15.92:6803","nonce":115821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":115821},{"type":"v1","addr":"172.21.15.92:6807","nonce":115821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":115821},{"type":"v1","addr":"172.21.15.92:6805","nonce":115821}]},"public_addr":"172.21.15.92:6801/115821","cluster_addr":"172.21.15.92:6803/115821","heartbeat_back_addr":"172.21.15.92:6807/115821","heartbeat_front_addr":"172.21.15.92:6805/115821","state":["exists","up"]},{"osd":3,"uuid":"adf54e24-33a9-41fe-9ce3-98a5ab6c158e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":115825},{"type":"v1","addr":"172.21.15.92:6825","nonce":115825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":115825},{"type":"v1","addr":"172.21.15.92:6827","nonce":115825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":115825},{"type":"v1","addr":"172.21.15.92:6831","nonce":115825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":115825},{"type":"v1","addr":"172.21.15.92:6829","nonce":115825}]},"public_addr":"172.21.15.92:6825/115825","cluster_addr":"172.21.15.92:6827/115825","heartbeat_back_addr":"172.21.15.92:6831/115825","heartbeat_front_addr":"172.21.15.92:6829/115825","state":["exists","up"]},{"osd":4,"uuid":"ccb9b747-f399-44ea-9a18-9db3152227e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":116035},{"type":"v1","addr":"172.21.15.169:6817","nonce":116035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":116035},{"type":"v1","addr":"172.21.15.169:6819","nonce":116035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":116035},{"type":"v1","addr":"172.21.15.169:6826","nonce":116035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":116035},{"type":"v1","addr":"172.21.15.169:6822","nonce":116035}]},"public_addr":"172.21.15.169:6817/116035","cluster_addr":"172.21.15.169:6819/116035","heartbeat_back_addr":"172.21.15.169:6826/116035","heartbeat_front_addr":"172.21.15.169:6822/116035","state":["exists","up"]},{"osd":5,"uuid":"308a1e63-75aa-4d7d-8a7c-cecc24714fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":116033},{"type":"v1","addr":"172.21.15.169:6801","nonce":116033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":116033},{"type":"v1","addr":"172.21.15.169:6803","nonce":116033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":116033},{"type":"v1","addr":"172.21.15.169:6807","nonce":116033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":116033},{"type":"v1","addr":"172.21.15.169:6805","nonce":116033}]},"public_addr":"172.21.15.169:6801/116033","cluster_addr":"172.21.15.169:6803/116033","heartbeat_back_addr":"172.21.15.169:6807/116033","heartbeat_front_addr":"172.21.15.169:6805/116033","state":["exists","up"]},{"osd":6,"uuid":"9c459b7b-b821-4b4b-b91e-6aaae271457c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":116034},{"type":"v1","addr":"172.21.15.169:6809","nonce":116034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":116034},{"type":"v1","addr":"172.21.15.169:6811","nonce":116034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":116034},{"type":"v1","addr":"172.21.15.169:6815","nonce":116034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":116034},{"type":"v1","addr":"172.21.15.169:6813","nonce":116034}]},"public_addr":"172.21.15.169:6809/116034","cluster_addr":"172.21.15.169:6811/116034","heartbeat_back_addr":"172.21.15.169:6815/116034","heartbeat_front_addr":"172.21.15.169:6813/116034","state":["exists","up"]},{"osd":7,"uuid":"1ca9aa4a-755d-4e28-a937-4d8dccab4df3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":116044},{"type":"v1","addr":"172.21.15.169:6823","nonce":116044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6825","nonce":116044},{"type":"v1","addr":"172.21.15.169:6827","nonce":116044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":116044},{"type":"v1","addr":"172.21.15.169:6831","nonce":116044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":116044},{"type":"v1","addr":"172.21.15.169:6829","nonce":116044}]},"public_addr":"172.21.15.169:6823/116044","cluster_addr":"172.21.15.169:6827/116044","heartbeat_back_addr":"172.21.15.169:6831/116044","heartbeat_front_addr":"172.21.15.169:6829/116044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.586781+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.869778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.508594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.860493+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.872267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.746933+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.611520+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.871446+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:20:52.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.734+0000 7f79d43ff700 1 -- 172.21.15.92:0/1073838747 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f79b00682f0 msgr2=0x7f79b006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:52.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.734+0000 7f79d43ff700 1 --2- 172.21.15.92:0/1073838747 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f79b00682f0 0x7f79b006a7a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f79c8009b60 tx=0x7f79c8009210 comp rx=0 tx=0).stop 2023-11-28T11:20:52.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.734+0000 7f79d43ff700 1 -- 172.21.15.92:0/1073838747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79cc0b4fc0 msgr2=0x7f79cc0cfc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:52.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.734+0000 7f79d43ff700 1 --2- 172.21.15.92:0/1073838747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79cc0b4fc0 0x7f79cc0cfc20 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f79bc014040 tx=0x7f79bc004750 comp rx=0 tx=0).stop 2023-11-28T11:20:52.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.734+0000 7f79d43ff700 1 -- 172.21.15.92:0/1073838747 shutdown_connections 2023-11-28T11:20:52.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.734+0000 7f79d43ff700 1 --2- 172.21.15.92:0/1073838747 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79cc0d2df0 0x7f79cc0d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:52.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.734+0000 7f79d43ff700 1 --2- 172.21.15.92:0/1073838747 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79cc0b1c60 0x7f79cc14e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:52.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.734+0000 7f79d43ff700 1 --2- 172.21.15.92:0/1073838747 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f79b00682f0 0x7f79b006a7a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:52.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.734+0000 7f79d43ff700 1 --2- 172.21.15.92:0/1073838747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79cc0b4fc0 0x7f79cc0cfc20 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:52.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.734+0000 7f79d43ff700 1 -- 172.21.15.92:0/1073838747 >> 172.21.15.92:0/1073838747 conn(0x7f79cc0719a0 msgr2=0x7f79cc0ccd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:52.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.734+0000 7f79d43ff700 1 -- 172.21.15.92:0/1073838747 shutdown_connections 2023-11-28T11:20:52.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.734+0000 7f79d43ff700 1 -- 172.21.15.92:0/1073838747 wait complete. 2023-11-28T11:20:52.753 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:20:41.925237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:20:48.682497+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:20:52.753 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:20:52.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.950+0000 7f3fd83ff700 1 Processor -- start 2023-11-28T11:20:52.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.951+0000 7f3fd83ff700 1 -- start start 2023-11-28T11:20:52.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.952+0000 7f3fd83ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3fd00cd460 0x7f3fd006c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:52.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.952+0000 7f3fd83ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fd00b88e0 con 0x7f3fd00d0d60 2023-11-28T11:20:52.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.952+0000 7f3fd83ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fd00cd8f0 con 0x7f3fd00b8510 2023-11-28T11:20:52.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.952+0000 7f3fd83ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd00cdbd0 con 0x7f3fd00cd460 2023-11-28T11:20:52.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.952+0000 7f3fd699c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f3fd00d0d60 0x7f3fd00d11e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51852/0 (socket says 172.21.15.92:51852) 2023-11-28T11:20:52.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.952+0000 7f3fd699c700 1 -- 172.21.15.92:0/3427614758 learned_addr learned my addr 172.21.15.92:0/3427614758 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:20:52.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.952+0000 7f3fd599a700 1 --2- 172.21.15.92:0/3427614758 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3fd00cd460 0x7f3fd006c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:52.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.952+0000 7f3fd5199700 1 -- 172.21.15.92:0/3427614758 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4167644034 0 0) 0x7f3fd00b88e0 con 0x7f3fd00d0d60 2023-11-28T11:20:52.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.952+0000 7f3fd5199700 1 -- 172.21.15.92:0/3427614758 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fbc0035b0 con 0x7f3fd00d0d60 2023-11-28T11:20:52.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.953+0000 7f3fd5199700 1 -- 172.21.15.92:0/3427614758 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 69882339 0 0) 0x7f3fd00cd8f0 con 0x7f3fd00b8510 2023-11-28T11:20:52.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.953+0000 7f3fd5199700 1 -- 172.21.15.92:0/3427614758 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fd00b88e0 con 0x7f3fd00b8510 2023-11-28T11:20:52.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.953+0000 7f3fd5199700 1 -- 172.21.15.92:0/3427614758 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2349343422 0 0) 0x7f3fbc0035b0 con 0x7f3fd00d0d60 2023-11-28T11:20:52.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.953+0000 7f3fd599a700 1 -- 172.21.15.92:0/3427614758 >> v1:172.21.15.169:6789/0 conn(0x7f3fd00b8510 legacy=0x7f3fd006b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:52.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.953+0000 7f3fd599a700 1 -- 172.21.15.92:0/3427614758 >> v1:172.21.15.92:6789/0 conn(0x7f3fd00d0d60 legacy=0x7f3fd00d11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:52.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.953+0000 7f3fd599a700 1 -- 172.21.15.92:0/3427614758 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd00cfd10 con 0x7f3fd00cd460 2023-11-28T11:20:52.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.953+0000 7f3fd599a700 1 --2- 172.21.15.92:0/3427614758 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3fd00cd460 0x7f3fd006c080 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0009870 tx=0x7f3fc0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0532077e1a73aaf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:52.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.953+0000 7f3fd5199700 1 -- 172.21.15.92:0/3427614758 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc80019f0 con 0x7f3fd00cd460 2023-11-28T11:20:52.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.953+0000 7f3fd5199700 1 -- 172.21.15.92:0/3427614758 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fc0004030 con 0x7f3fd00cd460 2023-11-28T11:20:52.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.953+0000 7f3fd5199700 1 -- 172.21.15.92:0/3427614758 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc00043c0 con 0x7f3fd00cd460 2023-11-28T11:20:52.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.954+0000 7f3fd83ff700 1 -- 172.21.15.92:0/3427614758 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3fd00cd460 msgr2=0x7f3fd006c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:52.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.954+0000 7f3fd83ff700 1 --2- 172.21.15.92:0/3427614758 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3fd00cd460 0x7f3fd006c080 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0009870 tx=0x7f3fc0009b80 comp rx=0 tx=0).stop 2023-11-28T11:20:52.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.954+0000 7f3fd83ff700 1 -- 172.21.15.92:0/3427614758 shutdown_connections 2023-11-28T11:20:52.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.954+0000 7f3fd83ff700 1 --2- 172.21.15.92:0/3427614758 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3fd00cd460 0x7f3fd006c080 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:52.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.954+0000 7f3fd83ff700 1 -- 172.21.15.92:0/3427614758 >> 172.21.15.92:0/3427614758 conn(0x7f3fd00719a0 msgr2=0x7f3fd0071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:52.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.954+0000 7f3fd83ff700 1 -- 172.21.15.92:0/3427614758 shutdown_connections 2023-11-28T11:20:52.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.954+0000 7f3fd83ff700 1 -- 172.21.15.92:0/3427614758 wait complete. 2023-11-28T11:20:52.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.956+0000 7f3fd83ff700 1 Processor -- start 2023-11-28T11:20:52.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.956+0000 7f3fd83ff700 1 -- start start 2023-11-28T11:20:52.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.957+0000 7f3fd83ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3fd00b8510 0x7f3fd014d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:52.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.957+0000 7f3fd83ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3fd00cd460 0x7f3fd015b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:52.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.957+0000 7f3fd83ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3fd00d0d60 0x7f3fd01f1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:52.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.957+0000 7f3fd83ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd0157420 con 0x7f3fd00b8510 2023-11-28T11:20:52.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.957+0000 7f3fd83ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd0156810 con 0x7f3fd00d0d60 2023-11-28T11:20:52.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.957+0000 7f3fd83ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd0156af0 con 0x7f3fd00cd460 2023-11-28T11:20:52.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.957+0000 7f3fd619b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3fd00b8510 0x7f3fd014d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:52.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.957+0000 7f3fd619b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3fd00b8510 0x7f3fd014d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41564/0 (socket says 172.21.15.92:41564) 2023-11-28T11:20:52.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.957+0000 7f3fd619b700 1 -- 172.21.15.92:0/670497823 learned_addr learned my addr 172.21.15.92:0/670497823 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:20:52.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.957+0000 7f3fd599a700 1 --2- 172.21.15.92:0/670497823 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3fd00cd460 0x7f3fd015b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:52.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.957+0000 7f3fd619b700 1 -- 172.21.15.92:0/670497823 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3fd00cd460 msgr2=0x7f3fd015b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:52.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.957+0000 7f3fd619b700 1 --2- 172.21.15.92:0/670497823 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3fd00cd460 0x7f3fd015b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:52.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.957+0000 7f3fd619b700 1 -- 172.21.15.92:0/670497823 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3fd00d0d60 msgr2=0x7f3fd01f1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:52.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.957+0000 7f3fd699c700 1 --2- 172.21.15.92:0/670497823 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3fd00d0d60 0x7f3fd01f1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:52.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.957+0000 7f3fd619b700 1 --2- 172.21.15.92:0/670497823 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3fd00d0d60 0x7f3fd01f1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:52.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.957+0000 7f3fd619b700 1 -- 172.21.15.92:0/670497823 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc00094a0 con 0x7f3fd00b8510 2023-11-28T11:20:52.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.957+0000 7f3fd699c700 1 --2- 172.21.15.92:0/670497823 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3fd00d0d60 0x7f3fd01f1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:20:52.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.958+0000 7f3fd619b700 1 --2- 172.21.15.92:0/670497823 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3fd00b8510 0x7f3fd014d120 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc00c520 tx=0x7f3fcc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:52.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.958+0000 7f3fc6ffd700 1 -- 172.21.15.92:0/670497823 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc0052d0 con 0x7f3fd00b8510 2023-11-28T11:20:52.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.958+0000 7f3fc6ffd700 1 -- 172.21.15.92:0/670497823 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fcc005430 con 0x7f3fd00b8510 2023-11-28T11:20:52.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.958+0000 7f3fc6ffd700 1 -- 172.21.15.92:0/670497823 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc0038c0 con 0x7f3fd00b8510 2023-11-28T11:20:52.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.958+0000 7f3fd83ff700 1 -- 172.21.15.92:0/670497823 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fd00d29c0 con 0x7f3fd00b8510 2023-11-28T11:20:52.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.958+0000 7f3fd83ff700 1 -- 172.21.15.92:0/670497823 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fd00cfff0 con 0x7f3fd00b8510 2023-11-28T11:20:52.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.959+0000 7f3fd83ff700 1 -- 172.21.15.92:0/670497823 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fd00d2b50 con 0x7f3fd00b8510 2023-11-28T11:20:52.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.960+0000 7f3fc6ffd700 1 -- 172.21.15.92:0/670497823 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3fcc019030 con 0x7f3fd00b8510 2023-11-28T11:20:52.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.961+0000 7f3fc6ffd700 1 --2- 172.21.15.92:0/670497823 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3fb40682f0 0x7f3fb406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:52.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.961+0000 7f3fc6ffd700 1 -- 172.21.15.92:0/670497823 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f3fcc084ea0 con 0x7f3fd00b8510 2023-11-28T11:20:52.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.961+0000 7f3fd599a700 1 --2- 172.21.15.92:0/670497823 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3fb40682f0 0x7f3fb406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:52.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.961+0000 7f3fd599a700 1 --2- 172.21.15.92:0/670497823 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3fb40682f0 0x7f3fb406a7a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3fd00b38f0 tx=0x7f3fc0008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:52.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:52.965+0000 7f3fc6ffd700 1 -- 172.21.15.92:0/670497823 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3fcc051960 con 0x7f3fd00b8510 2023-11-28T11:20:53.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.153+0000 7f3fd83ff700 1 -- 172.21.15.92:0/670497823 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3fd0067880 con 0x7f3fd00b8510 2023-11-28T11:20:53.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.153+0000 7f3fc6ffd700 1 -- 172.21.15.92:0/670497823 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3fcc017020 con 0x7f3fd00b8510 2023-11-28T11:20:53.155 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-11-28T11:20:53.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.156+0000 7f3fd83ff700 1 -- 172.21.15.92:0/670497823 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3fb40682f0 msgr2=0x7f3fb406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:53.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.156+0000 7f3fd83ff700 1 --2- 172.21.15.92:0/670497823 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3fb40682f0 0x7f3fb406a7a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3fd00b38f0 tx=0x7f3fc0008030 comp rx=0 tx=0).stop 2023-11-28T11:20:53.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.156+0000 7f3fd83ff700 1 -- 172.21.15.92:0/670497823 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3fd00b8510 msgr2=0x7f3fd014d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:53.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.156+0000 7f3fd83ff700 1 --2- 172.21.15.92:0/670497823 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3fd00b8510 0x7f3fd014d120 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc00c520 tx=0x7f3fcc00c8e0 comp rx=0 tx=0).stop 2023-11-28T11:20:53.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.156+0000 7f3fd83ff700 1 -- 172.21.15.92:0/670497823 shutdown_connections 2023-11-28T11:20:53.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.156+0000 7f3fd83ff700 1 --2- 172.21.15.92:0/670497823 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3fd00d0d60 0x7f3fd01f1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:53.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.156+0000 7f3fd83ff700 1 --2- 172.21.15.92:0/670497823 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3fb40682f0 0x7f3fb406a7a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:53.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.156+0000 7f3fd83ff700 1 --2- 172.21.15.92:0/670497823 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3fd00b8510 0x7f3fd014d120 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:53.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.156+0000 7f3fd83ff700 1 --2- 172.21.15.92:0/670497823 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3fd00cd460 0x7f3fd015b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:53.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.156+0000 7f3fd83ff700 1 -- 172.21.15.92:0/670497823 >> 172.21.15.92:0/670497823 conn(0x7f3fd00719a0 msgr2=0x7f3fd00b28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:53.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.156+0000 7f3fd83ff700 1 -- 172.21.15.92:0/670497823 shutdown_connections 2023-11-28T11:20:53.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.156+0000 7f3fd83ff700 1 -- 172.21.15.92:0/670497823 wait complete. 2023-11-28T11:20:53.174 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:20:53.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.372+0000 7f1a4c5ff700 1 Processor -- start 2023-11-28T11:20:53.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.372+0000 7f1a4c5ff700 1 -- start start 2023-11-28T11:20:53.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.373+0000 7f1a4c5ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a440d2df0 0x7f1a440cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:53.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.373+0000 7f1a4c5ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1a440b4fc0 0x7f1a440d31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:53.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.373+0000 7f1a4c5ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a440d1950 con 0x7f1a440b1c60 2023-11-28T11:20:53.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.373+0000 7f1a4c5ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a440d3d30 con 0x7f1a440b4fc0 2023-11-28T11:20:53.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.373+0000 7f1a4c5ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a440b2500 con 0x7f1a440d2df0 2023-11-28T11:20:53.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.373+0000 7f1a49b9a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1a440b4fc0 0x7f1a440d31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:53.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.373+0000 7f1a49b9a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1a440b4fc0 0x7f1a440d31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41572/0 (socket says 172.21.15.92:41572) 2023-11-28T11:20:53.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.373+0000 7f1a49b9a700 1 -- 172.21.15.92:0/3920402925 learned_addr learned my addr 172.21.15.92:0/3920402925 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:20:53.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.373+0000 7f1a4a39b700 1 --2- 172.21.15.92:0/3920402925 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a440d2df0 0x7f1a440cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:53.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.374+0000 7f1a49b9a700 1 -- 172.21.15.92:0/3920402925 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a440d2df0 msgr2=0x7f1a440cd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:53.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.374+0000 7f1a49b9a700 1 --2- 172.21.15.92:0/3920402925 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a440d2df0 0x7f1a440cd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:53.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.374+0000 7f1a49b9a700 1 -- 172.21.15.92:0/3920402925 >> v1:172.21.15.92:6789/0 conn(0x7f1a440b1c60 legacy=0x7f1a440cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:53.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.374+0000 7f1a49b9a700 1 -- 172.21.15.92:0/3920402925 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a440d3a50 con 0x7f1a440b4fc0 2023-11-28T11:20:53.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.374+0000 7f1a49b9a700 1 --2- 172.21.15.92:0/3920402925 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a440b4fc0 0x7f1a440d31c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1a340098f0 tx=0x7f1a34009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a9d7fd329f245b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:53.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.374+0000 7f1a49399700 1 -- 172.21.15.92:0/3920402925 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a34011830 con 0x7f1a440b4fc0 2023-11-28T11:20:53.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.374+0000 7f1a49399700 1 -- 172.21.15.92:0/3920402925 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a34011990 con 0x7f1a440b4fc0 2023-11-28T11:20:53.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.374+0000 7f1a49399700 1 -- 172.21.15.92:0/3920402925 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a34011d20 con 0x7f1a440b4fc0 2023-11-28T11:20:53.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.375+0000 7f1a4c5ff700 1 -- 172.21.15.92:0/3920402925 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a440b4fc0 msgr2=0x7f1a440d31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:53.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.375+0000 7f1a4c5ff700 1 --2- 172.21.15.92:0/3920402925 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a440b4fc0 0x7f1a440d31c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1a340098f0 tx=0x7f1a34009c00 comp rx=0 tx=0).stop 2023-11-28T11:20:53.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.375+0000 7f1a4c5ff700 1 -- 172.21.15.92:0/3920402925 shutdown_connections 2023-11-28T11:20:53.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.375+0000 7f1a4c5ff700 1 --2- 172.21.15.92:0/3920402925 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a440d2df0 0x7f1a440cd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:53.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.375+0000 7f1a4c5ff700 1 --2- 172.21.15.92:0/3920402925 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a440b4fc0 0x7f1a440d31c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:53.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.375+0000 7f1a4c5ff700 1 -- 172.21.15.92:0/3920402925 >> 172.21.15.92:0/3920402925 conn(0x7f1a440719a0 msgr2=0x7f1a44071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:53.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.375+0000 7f1a4c5ff700 1 -- 172.21.15.92:0/3920402925 shutdown_connections 2023-11-28T11:20:53.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.375+0000 7f1a4c5ff700 1 -- 172.21.15.92:0/3920402925 wait complete. 2023-11-28T11:20:53.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.377+0000 7f1a4c5ff700 1 Processor -- start 2023-11-28T11:20:53.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.377+0000 7f1a4c5ff700 1 -- start start 2023-11-28T11:20:53.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.378+0000 7f1a4c5ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a440b1c60 0x7f1a4415a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:53.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.378+0000 7f1a4c5ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a440b4fc0 0x7f1a440cc440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:53.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.378+0000 7f1a4c5ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a440d2df0 0x7f1a440cc980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:53.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.378+0000 7f1a4c5ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a441e6750 con 0x7f1a440b1c60 2023-11-28T11:20:53.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.378+0000 7f1a4c5ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4415b970 con 0x7f1a440d2df0 2023-11-28T11:20:53.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.378+0000 7f1a4c5ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a441e5db0 con 0x7f1a440b4fc0 2023-11-28T11:20:53.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.378+0000 7f1a4a39b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a440b1c60 0x7f1a4415a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:53.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.378+0000 7f1a4a39b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a440b1c60 0x7f1a4415a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41580/0 (socket says 172.21.15.92:41580) 2023-11-28T11:20:53.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.378+0000 7f1a4a39b700 1 -- 172.21.15.92:0/406498322 learned_addr learned my addr 172.21.15.92:0/406498322 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:20:53.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.378+0000 7f1a4a39b700 1 -- 172.21.15.92:0/406498322 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a440b4fc0 msgr2=0x7f1a440cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:53.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.378+0000 7f1a4a39b700 1 --2- 172.21.15.92:0/406498322 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a440b4fc0 0x7f1a440cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:53.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.378+0000 7f1a4a39b700 1 -- 172.21.15.92:0/406498322 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a440d2df0 msgr2=0x7f1a440cc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:53.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.378+0000 7f1a4a39b700 1 --2- 172.21.15.92:0/406498322 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a440d2df0 0x7f1a440cc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:53.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.378+0000 7f1a4a39b700 1 -- 172.21.15.92:0/406498322 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a34009550 con 0x7f1a440b1c60 2023-11-28T11:20:53.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.378+0000 7f1a4a39b700 1 --2- 172.21.15.92:0/406498322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a440b1c60 0x7f1a4415a830 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1a4000e8a0 tx=0x7f1a4000ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:53.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.379+0000 7f1a3affd700 1 -- 172.21.15.92:0/406498322 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a40009590 con 0x7f1a440b1c60 2023-11-28T11:20:53.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.379+0000 7f1a3affd700 1 -- 172.21.15.92:0/406498322 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a4000eea0 con 0x7f1a440b1c60 2023-11-28T11:20:53.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.379+0000 7f1a3affd700 1 -- 172.21.15.92:0/406498322 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4000ca50 con 0x7f1a440b1c60 2023-11-28T11:20:53.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.379+0000 7f1a4c5ff700 1 -- 172.21.15.92:0/406498322 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a440b88f0 con 0x7f1a440b1c60 2023-11-28T11:20:53.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.379+0000 7f1a4c5ff700 1 -- 172.21.15.92:0/406498322 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a440d15e0 con 0x7f1a440b1c60 2023-11-28T11:20:53.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.380+0000 7f1a4c5ff700 1 -- 172.21.15.92:0/406498322 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a440b21b0 con 0x7f1a440b1c60 2023-11-28T11:20:53.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.381+0000 7f1a3affd700 1 -- 172.21.15.92:0/406498322 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1a40019030 con 0x7f1a440b1c60 2023-11-28T11:20:53.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.382+0000 7f1a3affd700 1 --2- 172.21.15.92:0/406498322 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f1a280682f0 0x7f1a2806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:53.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.382+0000 7f1a3affd700 1 -- 172.21.15.92:0/406498322 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f1a40085930 con 0x7f1a440b1c60 2023-11-28T11:20:53.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.382+0000 7f1a49b9a700 1 --2- 172.21.15.92:0/406498322 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f1a280682f0 0x7f1a2806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:53.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.383+0000 7f1a49b9a700 1 --2- 172.21.15.92:0/406498322 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f1a280682f0 0x7f1a2806a7a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1a340098f0 tx=0x7f1a34010000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:53.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.386+0000 7f1a3affd700 1 -- 172.21.15.92:0/406498322 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1a40052370 con 0x7f1a440b1c60 2023-11-28T11:20:53.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.576+0000 7f1a4c5ff700 1 -- 172.21.15.92:0/406498322 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1a44067170 con 0x7f1a440b1c60 2023-11-28T11:20:53.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.576+0000 7f1a3affd700 1 -- 172.21.15.92:0/406498322 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v19) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1a40055990 con 0x7f1a440b1c60 2023-11-28T11:20:53.578 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-11-28T11:20:53.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.579+0000 7f1a4c5ff700 1 -- 172.21.15.92:0/406498322 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f1a280682f0 msgr2=0x7f1a2806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:53.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.579+0000 7f1a4c5ff700 1 --2- 172.21.15.92:0/406498322 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f1a280682f0 0x7f1a2806a7a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1a340098f0 tx=0x7f1a34010000 comp rx=0 tx=0).stop 2023-11-28T11:20:53.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.579+0000 7f1a4c5ff700 1 -- 172.21.15.92:0/406498322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a440b1c60 msgr2=0x7f1a4415a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:53.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.579+0000 7f1a4c5ff700 1 --2- 172.21.15.92:0/406498322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a440b1c60 0x7f1a4415a830 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1a4000e8a0 tx=0x7f1a4000ec60 comp rx=0 tx=0).stop 2023-11-28T11:20:53.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.579+0000 7f1a4c5ff700 1 -- 172.21.15.92:0/406498322 shutdown_connections 2023-11-28T11:20:53.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.579+0000 7f1a4c5ff700 1 --2- 172.21.15.92:0/406498322 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1a440d2df0 0x7f1a440cc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:53.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.579+0000 7f1a4c5ff700 1 --2- 172.21.15.92:0/406498322 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f1a280682f0 0x7f1a2806a7a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:53.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.579+0000 7f1a4c5ff700 1 --2- 172.21.15.92:0/406498322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a440b1c60 0x7f1a4415a830 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:53.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.579+0000 7f1a4c5ff700 1 --2- 172.21.15.92:0/406498322 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1a440b4fc0 0x7f1a440cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:53.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.579+0000 7f1a4c5ff700 1 -- 172.21.15.92:0/406498322 >> 172.21.15.92:0/406498322 conn(0x7f1a440719a0 msgr2=0x7f1a440b7410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:53.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.579+0000 7f1a4c5ff700 1 -- 172.21.15.92:0/406498322 shutdown_connections 2023-11-28T11:20:53.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.579+0000 7f1a4c5ff700 1 -- 172.21.15.92:0/406498322 wait complete. 2023-11-28T11:20:53.597 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T11:20:53.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.795+0000 7ffab87ff700 1 Processor -- start 2023-11-28T11:20:53.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.795+0000 7ffab87ff700 1 -- start start 2023-11-28T11:20:53.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.796+0000 7ffab87ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ffab00d2df0 0x7ffab00cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:53.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.796+0000 7ffab87ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffab00b1c60 0x7ffab00cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:53.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.796+0000 7ffab87ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffab00d1950 con 0x7ffab00b4fc0 2023-11-28T11:20:53.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.796+0000 7ffab87ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffab00d3d30 con 0x7ffab00d2df0 2023-11-28T11:20:53.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.796+0000 7ffab87ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffab00b2500 con 0x7ffab00b1c60 2023-11-28T11:20:53.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.797+0000 7ffab659b700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ffab00d2df0 0x7ffab00cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:53.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.797+0000 7ffab6d9c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffab00b1c60 0x7ffab00cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:53.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.797+0000 7ffab5d9a700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7ffab00b4fc0 0x7ffab00d31c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.92:41622/0 (socket says 172.21.15.92:41622) 2023-11-28T11:20:53.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.797+0000 7ffab5d9a700 1 -- 172.21.15.92:0/4244399075 learned_addr learned my addr 172.21.15.92:0/4244399075 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:20:53.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.797+0000 7ffab5599700 1 -- 172.21.15.92:0/4244399075 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3343883220 0 0) 0x7ffab00d1950 con 0x7ffab00b4fc0 2023-11-28T11:20:53.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.797+0000 7ffab5599700 1 -- 172.21.15.92:0/4244399075 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa9c0035b0 con 0x7ffab00b4fc0 2023-11-28T11:20:53.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.797+0000 7ffab6d9c700 1 -- 172.21.15.92:0/4244399075 >> v2:172.21.15.169:3300/0 conn(0x7ffab00d2df0 msgr2=0x7ffab00cd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:53.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.797+0000 7ffab6d9c700 1 --2- 172.21.15.92:0/4244399075 >> v2:172.21.15.169:3300/0 conn(0x7ffab00d2df0 0x7ffab00cd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:53.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.797+0000 7ffab6d9c700 1 -- 172.21.15.92:0/4244399075 >> v1:172.21.15.169:6789/0 conn(0x7ffab00b4fc0 legacy=0x7ffab00d31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:53.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.797+0000 7ffab6d9c700 1 -- 172.21.15.92:0/4244399075 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffab00d3a50 con 0x7ffab00b1c60 2023-11-28T11:20:53.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.798+0000 7ffab6d9c700 1 --2- 172.21.15.92:0/4244399075 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffab00b1c60 0x7ffab00cf010 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8009d90 tx=0x7ffaa800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e49a792f20c10ce server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:53.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.798+0000 7ffab5599700 1 -- 172.21.15.92:0/4244399075 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa800e070 con 0x7ffab00b1c60 2023-11-28T11:20:53.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.798+0000 7ffab5599700 1 -- 172.21.15.92:0/4244399075 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaa800bb20 con 0x7ffab00b1c60 2023-11-28T11:20:53.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.798+0000 7ffab5599700 1 -- 172.21.15.92:0/4244399075 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa800be90 con 0x7ffab00b1c60 2023-11-28T11:20:53.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.798+0000 7ffab87ff700 1 -- 172.21.15.92:0/4244399075 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffab00b1c60 msgr2=0x7ffab00cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:53.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.799+0000 7ffab87ff700 1 --2- 172.21.15.92:0/4244399075 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffab00b1c60 0x7ffab00cf010 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8009d90 tx=0x7ffaa800b330 comp rx=0 tx=0).stop 2023-11-28T11:20:53.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.799+0000 7ffab87ff700 1 -- 172.21.15.92:0/4244399075 shutdown_connections 2023-11-28T11:20:53.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.799+0000 7ffab87ff700 1 --2- 172.21.15.92:0/4244399075 >> v2:172.21.15.169:3300/0 conn(0x7ffab00d2df0 0x7ffab00cd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:53.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.799+0000 7ffab87ff700 1 --2- 172.21.15.92:0/4244399075 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffab00b1c60 0x7ffab00cf010 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:53.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.799+0000 7ffab87ff700 1 -- 172.21.15.92:0/4244399075 >> 172.21.15.92:0/4244399075 conn(0x7ffab00719a0 msgr2=0x7ffab0071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:53.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.799+0000 7ffab87ff700 1 -- 172.21.15.92:0/4244399075 shutdown_connections 2023-11-28T11:20:53.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.799+0000 7ffab87ff700 1 -- 172.21.15.92:0/4244399075 wait complete. 2023-11-28T11:20:53.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.800+0000 7ffab87ff700 1 Processor -- start 2023-11-28T11:20:53.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.800+0000 7ffab87ff700 1 -- start start 2023-11-28T11:20:53.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.801+0000 7ffab87ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffab00b1c60 0x7ffab00ca050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:53.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.801+0000 7ffab87ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffab00b4fc0 0x7ffab0155d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:53.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.801+0000 7ffab87ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffab00d2df0 0x7ffab00cefd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:53.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.801+0000 7ffab87ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab01583f0 con 0x7ffab00b1c60 2023-11-28T11:20:53.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.801+0000 7ffab87ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab0156860 con 0x7ffab00b4fc0 2023-11-28T11:20:53.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.801+0000 7ffab87ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffab00cac30 con 0x7ffab00d2df0 2023-11-28T11:20:53.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.802+0000 7ffab659b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffab00b1c60 0x7ffab00ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:53.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.802+0000 7ffab659b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffab00b1c60 0x7ffab00ca050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41586/0 (socket says 172.21.15.92:41586) 2023-11-28T11:20:53.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.802+0000 7ffab659b700 1 -- 172.21.15.92:0/597037939 learned_addr learned my addr 172.21.15.92:0/597037939 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:20:53.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.802+0000 7ffab5d9a700 1 --2- 172.21.15.92:0/597037939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffab00b4fc0 0x7ffab0155d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:53.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.802+0000 7ffab6d9c700 1 --2- 172.21.15.92:0/597037939 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffab00d2df0 0x7ffab00cefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:53.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.802+0000 7ffab659b700 1 -- 172.21.15.92:0/597037939 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffab00d2df0 msgr2=0x7ffab00cefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:53.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.802+0000 7ffab659b700 1 --2- 172.21.15.92:0/597037939 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffab00d2df0 0x7ffab00cefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:53.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.802+0000 7ffab659b700 1 -- 172.21.15.92:0/597037939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffab00b4fc0 msgr2=0x7ffab0155d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:53.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.802+0000 7ffab659b700 1 --2- 172.21.15.92:0/597037939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffab00b4fc0 0x7ffab0155d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:53.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.802+0000 7ffab659b700 1 -- 172.21.15.92:0/597037939 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa8009a70 con 0x7ffab00b1c60 2023-11-28T11:20:53.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.802+0000 7ffab659b700 1 --2- 172.21.15.92:0/597037939 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffab00b1c60 0x7ffab00ca050 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ffaac00e870 tx=0x7ffaac00eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:53.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.802+0000 7ffaa6ffd700 1 -- 172.21.15.92:0/597037939 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaac009590 con 0x7ffab00b1c60 2023-11-28T11:20:53.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.803+0000 7ffaa6ffd700 1 -- 172.21.15.92:0/597037939 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaac013680 con 0x7ffab00b1c60 2023-11-28T11:20:53.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.803+0000 7ffab87ff700 1 -- 172.21.15.92:0/597037939 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffab00b88f0 con 0x7ffab00b1c60 2023-11-28T11:20:53.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.803+0000 7ffaa6ffd700 1 -- 172.21.15.92:0/597037939 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaac0139f0 con 0x7ffab00b1c60 2023-11-28T11:20:53.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.803+0000 7ffab87ff700 1 -- 172.21.15.92:0/597037939 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffab00d15e0 con 0x7ffab00b1c60 2023-11-28T11:20:53.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.804+0000 7ffab87ff700 1 -- 172.21.15.92:0/597037939 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffab00b21b0 con 0x7ffab00b1c60 2023-11-28T11:20:53.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.804+0000 7ffaa6ffd700 1 -- 172.21.15.92:0/597037939 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ffaac023070 con 0x7ffab00b1c60 2023-11-28T11:20:53.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.806+0000 7ffaa6ffd700 1 --2- 172.21.15.92:0/597037939 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ffa940682f0 0x7ffa9406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:53.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.806+0000 7ffaa6ffd700 1 -- 172.21.15.92:0/597037939 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7ffaac085a70 con 0x7ffab00b1c60 2023-11-28T11:20:53.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.806+0000 7ffab5d9a700 1 --2- 172.21.15.92:0/597037939 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ffa940682f0 0x7ffa9406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:53.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.806+0000 7ffab5d9a700 1 --2- 172.21.15.92:0/597037939 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ffa940682f0 0x7ffa9406a7a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ffab01f56a0 tx=0x7ffaa0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:53.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:53.810+0000 7ffaa6ffd700 1 -- 172.21.15.92:0/597037939 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ffaac0524d0 con 0x7ffab00b1c60 2023-11-28T11:20:54.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.004+0000 7ffab87ff700 1 -- 172.21.15.92:0/597037939 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ffab01586d0 con 0x7ffab00b1c60 2023-11-28T11:20:54.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.005+0000 7ffaa6ffd700 1 -- 172.21.15.92:0/597037939 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v19) v1 ==== 84+0+126 (secure 0 0 0) 0x7ffaac017090 con 0x7ffab00b1c60 2023-11-28T11:20:54.006 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:20:54.006 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-11-28T11:20:54.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:20:54.006 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 1, 2023-11-28T11:20:54.006 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": ".mgr" 2023-11-28T11:20:54.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:20:54.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:20:54.007 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 2, 2023-11-28T11:20:54.007 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": "rbd" 2023-11-28T11:20:54.007 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:20:54.007 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-11-28T11:20:54.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.007+0000 7ffab87ff700 1 -- 172.21.15.92:0/597037939 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ffa940682f0 msgr2=0x7ffa9406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:54.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.007+0000 7ffab87ff700 1 --2- 172.21.15.92:0/597037939 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ffa940682f0 0x7ffa9406a7a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ffab01f56a0 tx=0x7ffaa0009040 comp rx=0 tx=0).stop 2023-11-28T11:20:54.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.007+0000 7ffab87ff700 1 -- 172.21.15.92:0/597037939 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffab00b1c60 msgr2=0x7ffab00ca050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:54.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.007+0000 7ffab87ff700 1 --2- 172.21.15.92:0/597037939 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffab00b1c60 0x7ffab00ca050 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ffaac00e870 tx=0x7ffaac00eb80 comp rx=0 tx=0).stop 2023-11-28T11:20:54.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.007+0000 7ffab87ff700 1 -- 172.21.15.92:0/597037939 shutdown_connections 2023-11-28T11:20:54.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.007+0000 7ffab87ff700 1 --2- 172.21.15.92:0/597037939 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffab00d2df0 0x7ffab00cefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:54.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.007+0000 7ffab87ff700 1 --2- 172.21.15.92:0/597037939 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ffa940682f0 0x7ffa9406a7a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:54.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.007+0000 7ffab87ff700 1 --2- 172.21.15.92:0/597037939 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffab00b1c60 0x7ffab00ca050 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:54.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.007+0000 7ffab87ff700 1 --2- 172.21.15.92:0/597037939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffab00b4fc0 0x7ffab0155d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:54.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.007+0000 7ffab87ff700 1 -- 172.21.15.92:0/597037939 >> 172.21.15.92:0/597037939 conn(0x7ffab00719a0 msgr2=0x7ffab00b66a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:54.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.007+0000 7ffab87ff700 1 -- 172.21.15.92:0/597037939 shutdown_connections 2023-11-28T11:20:54.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.007+0000 7ffab87ff700 1 -- 172.21.15.92:0/597037939 wait complete. 2023-11-28T11:20:54.027 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T11:20:54.027 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-28T11:20:54.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.226+0000 7fdeec1ff700 1 Processor -- start 2023-11-28T11:20:54.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.226+0000 7fdeec1ff700 1 -- start start 2023-11-28T11:20:54.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.227+0000 7fdeec1ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fdee406c050 0x7fdee4167410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:54.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.228+0000 7fdeec1ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdee40b5240 con 0x7fdee40b8790 2023-11-28T11:20:54.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.228+0000 7fdeec1ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdee40b5520 con 0x7fdee40cd460 2023-11-28T11:20:54.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.228+0000 7fdeec1ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdee40d0a80 con 0x7fdee406c050 2023-11-28T11:20:54.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.228+0000 7fdee9f9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fdee40b8790 0x7fdee40cfb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51884/0 (socket says 172.21.15.92:51884) 2023-11-28T11:20:54.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.228+0000 7fdee9f9b700 1 -- 172.21.15.92:0/3476551675 learned_addr learned my addr 172.21.15.92:0/3476551675 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:20:54.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.228+0000 7fdeea79c700 1 --2- 172.21.15.92:0/3476551675 >> v2:172.21.15.169:3300/0 conn(0x7fdee406c050 0x7fdee4167410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:54.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.228+0000 7fdee8f99700 1 -- 172.21.15.92:0/3476551675 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2571433857 0 0) 0x7fdee40b5240 con 0x7fdee40b8790 2023-11-28T11:20:54.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.228+0000 7fdee8f99700 1 -- 172.21.15.92:0/3476551675 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fded00035b0 con 0x7fdee40b8790 2023-11-28T11:20:54.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.228+0000 7fdee8f99700 1 -- 172.21.15.92:0/3476551675 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1972924154 0 0) 0x7fded00035b0 con 0x7fdee40b8790 2023-11-28T11:20:54.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.228+0000 7fdee8f99700 1 -- 172.21.15.92:0/3476551675 >> v2:172.21.15.169:3300/0 conn(0x7fdee406c050 msgr2=0x7fdee4167410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:54.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.228+0000 7fdee8f99700 1 --2- 172.21.15.92:0/3476551675 >> v2:172.21.15.169:3300/0 conn(0x7fdee406c050 0x7fdee4167410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:54.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.228+0000 7fdee8f99700 1 -- 172.21.15.92:0/3476551675 >> v1:172.21.15.169:6789/0 conn(0x7fdee40cd460 legacy=0x7fdee406b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:54.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.228+0000 7fdee8f99700 1 -- 172.21.15.92:0/3476551675 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee40b5800 con 0x7fdee40b8790 2023-11-28T11:20:54.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.228+0000 7fdee8f99700 1 -- 172.21.15.92:0/3476551675 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532161587 0 0) 0x7fdee0001670 con 0x7fdee40b8790 2023-11-28T11:20:54.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.229+0000 7fdee8f99700 1 -- 172.21.15.92:0/3476551675 >> v1:172.21.15.92:6789/0 conn(0x7fdee40b8790 legacy=0x7fdee40cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:54.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.229+0000 7fdee8f99700 1 --2- 172.21.15.92:0/3476551675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fded0003a00 0x7fded0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:54.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.229+0000 7fdee8f99700 1 --2- 172.21.15.92:0/3476551675 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fded0004850 0x7fded0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:54.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.229+0000 7fdee8f99700 1 --2- 172.21.15.92:0/3476551675 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fded0005200 0x7fded000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:54.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.229+0000 7fdee8f99700 1 -- 172.21.15.92:0/3476551675 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee40d0a80 con 0x7fded0004850 2023-11-28T11:20:54.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.229+0000 7fdee8f99700 1 -- 172.21.15.92:0/3476551675 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded000c060 con 0x7fded0003a00 2023-11-28T11:20:54.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.229+0000 7fdee8f99700 1 -- 172.21.15.92:0/3476551675 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fded000c220 con 0x7fded0005200 2023-11-28T11:20:54.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.229+0000 7fdee979a700 1 --2- 172.21.15.92:0/3476551675 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fded0004850 0x7fded0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:54.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.229+0000 7fdeea79c700 1 --2- 172.21.15.92:0/3476551675 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fded0005200 0x7fded000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:54.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.229+0000 7fdee9f9b700 1 --2- 172.21.15.92:0/3476551675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fded0003a00 0x7fded0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:54.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.229+0000 7fdee979a700 1 -- 172.21.15.92:0/3476551675 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fded0005200 msgr2=0x7fded000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:54.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.229+0000 7fdee979a700 1 --2- 172.21.15.92:0/3476551675 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fded0005200 0x7fded000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:54.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.229+0000 7fdee979a700 1 -- 172.21.15.92:0/3476551675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fded0003a00 msgr2=0x7fded0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:54.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.229+0000 7fdee979a700 1 --2- 172.21.15.92:0/3476551675 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fded0003a00 0x7fded0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:54.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.229+0000 7fdee979a700 1 -- 172.21.15.92:0/3476551675 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded00035b0 con 0x7fded0004850 2023-11-28T11:20:54.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.229+0000 7fdeea79c700 1 -- 172.21.15.92:0/3476551675 reap_dead start 2023-11-28T11:20:54.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.229+0000 7fdee979a700 1 --2- 172.21.15.92:0/3476551675 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fded0004850 0x7fded0004cc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fded40018d0 tx=0x7fded0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e93e79fce47f7b85 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:54.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.230+0000 7fdee8f99700 1 -- 172.21.15.92:0/3476551675 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded400c690 con 0x7fded0004850 2023-11-28T11:20:54.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.230+0000 7fdee8f99700 1 -- 172.21.15.92:0/3476551675 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded400c7f0 con 0x7fded0004850 2023-11-28T11:20:54.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.230+0000 7fdee8f99700 1 -- 172.21.15.92:0/3476551675 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded400cba0 con 0x7fded0004850 2023-11-28T11:20:54.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.230+0000 7fdeec1ff700 1 -- 172.21.15.92:0/3476551675 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fded0004850 msgr2=0x7fded0004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:54.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.230+0000 7fdeec1ff700 1 --2- 172.21.15.92:0/3476551675 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fded0004850 0x7fded0004cc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fded40018d0 tx=0x7fded0005880 comp rx=0 tx=0).stop 2023-11-28T11:20:54.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.230+0000 7fdeec1ff700 1 -- 172.21.15.92:0/3476551675 shutdown_connections 2023-11-28T11:20:54.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.230+0000 7fdeec1ff700 1 --2- 172.21.15.92:0/3476551675 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fded0004850 0x7fded0004cc0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:54.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.230+0000 7fdeec1ff700 1 -- 172.21.15.92:0/3476551675 >> 172.21.15.92:0/3476551675 conn(0x7fdee40719a0 msgr2=0x7fdee4071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:54.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.230+0000 7fdeec1ff700 1 -- 172.21.15.92:0/3476551675 shutdown_connections 2023-11-28T11:20:54.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.230+0000 7fdeec1ff700 1 -- 172.21.15.92:0/3476551675 wait complete. 2023-11-28T11:20:54.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.232+0000 7fdeec1ff700 1 Processor -- start 2023-11-28T11:20:54.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.232+0000 7fdeec1ff700 1 -- start start 2023-11-28T11:20:54.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.233+0000 7fdeec1ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fded0004850 0x7fdee40c9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:54.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.233+0000 7fdeec1ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdee41f25f0 0x7fdee40b7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:54.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.233+0000 7fdeec1ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdee41563a0 0x7fdee4147d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:54.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.233+0000 7fdeec1ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee415ea40 con 0x7fdee41f25f0 2023-11-28T11:20:54.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.233+0000 7fdeec1ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee415e130 con 0x7fded0004850 2023-11-28T11:20:54.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.233+0000 7fdeec1ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdee40c9b70 con 0x7fdee41563a0 2023-11-28T11:20:54.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.233+0000 7fdee979a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdee41f25f0 0x7fdee40b7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:54.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.233+0000 7fdee979a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdee41f25f0 0x7fdee40b7100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41598/0 (socket says 172.21.15.92:41598) 2023-11-28T11:20:54.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.233+0000 7fdee979a700 1 -- 172.21.15.92:0/1175725925 learned_addr learned my addr 172.21.15.92:0/1175725925 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:20:54.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.233+0000 7fdee9f9b700 1 --2- 172.21.15.92:0/1175725925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fded0004850 0x7fdee40c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:54.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.233+0000 7fdeea79c700 1 --2- 172.21.15.92:0/1175725925 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdee41563a0 0x7fdee4147d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:54.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.234+0000 7fdee979a700 1 -- 172.21.15.92:0/1175725925 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdee41563a0 msgr2=0x7fdee4147d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:54.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.234+0000 7fdee979a700 1 --2- 172.21.15.92:0/1175725925 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdee41563a0 0x7fdee4147d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:54.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.234+0000 7fdee979a700 1 -- 172.21.15.92:0/1175725925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fded0004850 msgr2=0x7fdee40c9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:54.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.234+0000 7fdee979a700 1 --2- 172.21.15.92:0/1175725925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fded0004850 0x7fdee40c9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:54.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.234+0000 7fdee979a700 1 -- 172.21.15.92:0/1175725925 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee40cfd00 con 0x7fdee41f25f0 2023-11-28T11:20:54.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.234+0000 7fdee979a700 1 --2- 172.21.15.92:0/1175725925 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdee41f25f0 0x7fdee40b7100 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fded400c0e0 tx=0x7fded4001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:54.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.234+0000 7fdeda7fc700 1 -- 172.21.15.92:0/1175725925 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded400cd70 con 0x7fdee41f25f0 2023-11-28T11:20:54.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.234+0000 7fdeda7fc700 1 -- 172.21.15.92:0/1175725925 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded4020070 con 0x7fdee41f25f0 2023-11-28T11:20:54.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.234+0000 7fdeda7fc700 1 -- 172.21.15.92:0/1175725925 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded40038c0 con 0x7fdee41f25f0 2023-11-28T11:20:54.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.234+0000 7fdeec1ff700 1 -- 172.21.15.92:0/1175725925 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdee40cfb70 con 0x7fdee41f25f0 2023-11-28T11:20:54.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.234+0000 7fdeec1ff700 1 -- 172.21.15.92:0/1175725925 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdee40cfe90 con 0x7fdee41f25f0 2023-11-28T11:20:54.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.235+0000 7fdeec1ff700 1 -- 172.21.15.92:0/1175725925 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdee40733c0 con 0x7fdee41f25f0 2023-11-28T11:20:54.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.236+0000 7fdeda7fc700 1 -- 172.21.15.92:0/1175725925 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fded4010050 con 0x7fdee41f25f0 2023-11-28T11:20:54.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.237+0000 7fdeda7fc700 1 --2- 172.21.15.92:0/1175725925 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fdec80682f0 0x7fdec806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:54.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.237+0000 7fdeda7fc700 1 -- 172.21.15.92:0/1175725925 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fded4022070 con 0x7fdee41f25f0 2023-11-28T11:20:54.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.237+0000 7fdee9f9b700 1 --2- 172.21.15.92:0/1175725925 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fdec80682f0 0x7fdec806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:54.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.238+0000 7fdee9f9b700 1 --2- 172.21.15.92:0/1175725925 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fdec80682f0 0x7fdec806a7a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdee0001eb0 tx=0x7fdee0007040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:54.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.241+0000 7fdeda7fc700 1 -- 172.21.15.92:0/1175725925 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fded4053770 con 0x7fdee41f25f0 2023-11-28T11:20:54.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.432+0000 7fdeec1ff700 1 -- 172.21.15.92:0/1175725925 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fdee4067880 con 0x7fdee41f25f0 2023-11-28T11:20:54.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.732+0000 7fdeda7fc700 1 -- 172.21.15.92:0/1175725925 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v20) v1 ==== 138+0+0 (secure 0 0 0) 0x7fded402c020 con 0x7fdee41f25f0 2023-11-28T11:20:54.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.735+0000 7fdeec1ff700 1 -- 172.21.15.92:0/1175725925 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fdec80682f0 msgr2=0x7fdec806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:54.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.735+0000 7fdeec1ff700 1 --2- 172.21.15.92:0/1175725925 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fdec80682f0 0x7fdec806a7a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdee0001eb0 tx=0x7fdee0007040 comp rx=0 tx=0).stop 2023-11-28T11:20:54.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.735+0000 7fdeec1ff700 1 -- 172.21.15.92:0/1175725925 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdee41f25f0 msgr2=0x7fdee40b7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:54.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.735+0000 7fdeec1ff700 1 --2- 172.21.15.92:0/1175725925 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdee41f25f0 0x7fdee40b7100 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fded400c0e0 tx=0x7fded4001230 comp rx=0 tx=0).stop 2023-11-28T11:20:54.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.736+0000 7fdeec1ff700 1 -- 172.21.15.92:0/1175725925 shutdown_connections 2023-11-28T11:20:54.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.736+0000 7fdeec1ff700 1 --2- 172.21.15.92:0/1175725925 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdee41563a0 0x7fdee4147d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:54.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.736+0000 7fdeec1ff700 1 --2- 172.21.15.92:0/1175725925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fded0004850 0x7fdee40c9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:54.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.736+0000 7fdeec1ff700 1 --2- 172.21.15.92:0/1175725925 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fdec80682f0 0x7fdec806a7a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:54.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.736+0000 7fdeec1ff700 1 --2- 172.21.15.92:0/1175725925 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdee41f25f0 0x7fdee40b7100 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:54.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.736+0000 7fdeec1ff700 1 -- 172.21.15.92:0/1175725925 >> 172.21.15.92:0/1175725925 conn(0x7fdee40719a0 msgr2=0x7fdee40b2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:54.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.736+0000 7fdeec1ff700 1 -- 172.21.15.92:0/1175725925 shutdown_connections 2023-11-28T11:20:54.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.736+0000 7fdeec1ff700 1 -- 172.21.15.92:0/1175725925 wait complete. 2023-11-28T11:20:54.738 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' created 2023-11-28T11:20:54.758 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-11-28T11:20:54.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.960+0000 7f05f51ff700 1 Processor -- start 2023-11-28T11:20:54.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.960+0000 7f05f51ff700 1 -- start start 2023-11-28T11:20:54.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.961+0000 7f05f51ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f05f00d2e10 0x7f05f015d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:54.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.961+0000 7f05f51ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f05f00b2280 0x7f05f00d31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:54.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.961+0000 7f05f51ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05f015dee0 0x7f05f00cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:54.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.961+0000 7f05f51ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05f00cdce0 con 0x7f05f00d2e10 2023-11-28T11:20:54.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.961+0000 7f05f51ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05f00cd4d0 con 0x7f05f00b2280 2023-11-28T11:20:54.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.961+0000 7f05f51ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05f00cf470 con 0x7f05f015dee0 2023-11-28T11:20:54.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.961+0000 7f05eed9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f05f00d2e10 0x7f05f015d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:54.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.961+0000 7f05eed9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f05f00d2e10 0x7f05f015d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41614/0 (socket says 172.21.15.92:41614) 2023-11-28T11:20:54.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.961+0000 7f05eed9d700 1 -- 172.21.15.92:0/1775564493 learned_addr learned my addr 172.21.15.92:0/1775564493 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:20:54.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.962+0000 7f05ee59c700 1 --2- 172.21.15.92:0/1775564493 >> v2:172.21.15.169:3300/0 conn(0x7f05f00b2280 0x7f05f00d31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:54.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.962+0000 7f05ef59e700 1 --2- 172.21.15.92:0/1775564493 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05f015dee0 0x7f05f00cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:54.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.962+0000 7f05eed9d700 1 -- 172.21.15.92:0/1775564493 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05f015dee0 msgr2=0x7f05f00cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:54.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.962+0000 7f05eed9d700 1 --2- 172.21.15.92:0/1775564493 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05f015dee0 0x7f05f00cf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:54.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.962+0000 7f05eed9d700 1 -- 172.21.15.92:0/1775564493 >> v2:172.21.15.169:3300/0 conn(0x7f05f00b2280 msgr2=0x7f05f00d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:54.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.962+0000 7f05eed9d700 1 --2- 172.21.15.92:0/1775564493 >> v2:172.21.15.169:3300/0 conn(0x7f05f00b2280 0x7f05f00d31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:54.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.962+0000 7f05eed9d700 1 -- 172.21.15.92:0/1775564493 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f00b2700 con 0x7f05f00d2e10 2023-11-28T11:20:54.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.962+0000 7f05eed9d700 1 --2- 172.21.15.92:0/1775564493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05f00d2e10 0x7f05f015d590 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f05f00ceb60 tx=0x7f05e0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=518695f46fa4d066 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:54.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.962+0000 7f05edd9b700 1 -- 172.21.15.92:0/1775564493 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e000b870 con 0x7f05f00d2e10 2023-11-28T11:20:54.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.962+0000 7f05edd9b700 1 -- 172.21.15.92:0/1775564493 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05e000b9d0 con 0x7f05f00d2e10 2023-11-28T11:20:54.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.962+0000 7f05edd9b700 1 -- 172.21.15.92:0/1775564493 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e000bd60 con 0x7f05f00d2e10 2023-11-28T11:20:54.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.963+0000 7f05f51ff700 1 -- 172.21.15.92:0/1775564493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05f00d2e10 msgr2=0x7f05f015d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:54.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.963+0000 7f05f51ff700 1 --2- 172.21.15.92:0/1775564493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05f00d2e10 0x7f05f015d590 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f05f00ceb60 tx=0x7f05e0009bd0 comp rx=0 tx=0).stop 2023-11-28T11:20:54.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.963+0000 7f05f51ff700 1 -- 172.21.15.92:0/1775564493 shutdown_connections 2023-11-28T11:20:54.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.963+0000 7f05f51ff700 1 --2- 172.21.15.92:0/1775564493 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05f015dee0 0x7f05f00cf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:54.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.963+0000 7f05f51ff700 1 --2- 172.21.15.92:0/1775564493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05f00d2e10 0x7f05f015d590 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:54.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.963+0000 7f05f51ff700 1 --2- 172.21.15.92:0/1775564493 >> v2:172.21.15.169:3300/0 conn(0x7f05f00b2280 0x7f05f00d31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:54.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.963+0000 7f05f51ff700 1 -- 172.21.15.92:0/1775564493 >> 172.21.15.92:0/1775564493 conn(0x7f05f00719a0 msgr2=0x7f05f0071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:54.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.963+0000 7f05f51ff700 1 -- 172.21.15.92:0/1775564493 shutdown_connections 2023-11-28T11:20:54.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.963+0000 7f05f51ff700 1 -- 172.21.15.92:0/1775564493 wait complete. 2023-11-28T11:20:54.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.965+0000 7f05f51ff700 1 Processor -- start 2023-11-28T11:20:54.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.965+0000 7f05f51ff700 1 -- start start 2023-11-28T11:20:54.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.966+0000 7f05f51ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05f00b2280 0x7f05f0159c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:54.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.966+0000 7f05f51ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05f00d2e10 0x7f05f00cc820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:54.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.966+0000 7f05f51ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05f015dee0 0x7f05f015f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:54.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.966+0000 7f05f51ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f0157ff0 con 0x7f05f00b2280 2023-11-28T11:20:54.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.966+0000 7f05f51ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f014da70 con 0x7f05f00d2e10 2023-11-28T11:20:54.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.966+0000 7f05f51ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05f01573e0 con 0x7f05f015dee0 2023-11-28T11:20:54.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.966+0000 7f05eed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05f00b2280 0x7f05f0159c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:54.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.966+0000 7f05eed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05f00b2280 0x7f05f0159c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41616/0 (socket says 172.21.15.92:41616) 2023-11-28T11:20:54.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.966+0000 7f05eed9d700 1 -- 172.21.15.92:0/500044378 learned_addr learned my addr 172.21.15.92:0/500044378 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:20:54.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.966+0000 7f05eed9d700 1 -- 172.21.15.92:0/500044378 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05f015dee0 msgr2=0x7f05f015f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:54.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.966+0000 7f05eed9d700 1 --2- 172.21.15.92:0/500044378 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05f015dee0 0x7f05f015f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:54.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.966+0000 7f05eed9d700 1 -- 172.21.15.92:0/500044378 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05f00d2e10 msgr2=0x7f05f00cc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:54.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.966+0000 7f05ee59c700 1 --2- 172.21.15.92:0/500044378 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05f00d2e10 0x7f05f00cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:54.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.966+0000 7f05eed9d700 1 --2- 172.21.15.92:0/500044378 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05f00d2e10 0x7f05f00cc820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:54.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.966+0000 7f05eed9d700 1 -- 172.21.15.92:0/500044378 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e0009550 con 0x7f05f00b2280 2023-11-28T11:20:54.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.966+0000 7f05ee59c700 1 --2- 172.21.15.92:0/500044378 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05f00d2e10 0x7f05f00cc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:20:54.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.967+0000 7f05eed9d700 1 --2- 172.21.15.92:0/500044378 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05f00b2280 0x7f05f0159c30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f05e0000c00 tx=0x7f05e0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:54.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.967+0000 7f05d77fe700 1 -- 172.21.15.92:0/500044378 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e0004520 con 0x7f05f00b2280 2023-11-28T11:20:54.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.967+0000 7f05d77fe700 1 -- 172.21.15.92:0/500044378 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05e001e070 con 0x7f05f00b2280 2023-11-28T11:20:54.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.967+0000 7f05d77fe700 1 -- 172.21.15.92:0/500044378 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e00038c0 con 0x7f05f00b2280 2023-11-28T11:20:54.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.967+0000 7f05f51ff700 1 -- 172.21.15.92:0/500044378 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05f0156fa0 con 0x7f05f00b2280 2023-11-28T11:20:54.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.967+0000 7f05f51ff700 1 -- 172.21.15.92:0/500044378 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05f00d3720 con 0x7f05f00b2280 2023-11-28T11:20:54.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.968+0000 7f05f51ff700 1 -- 172.21.15.92:0/500044378 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05f00cd720 con 0x7f05f00b2280 2023-11-28T11:20:54.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.969+0000 7f05d77fe700 1 -- 172.21.15.92:0/500044378 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f05e000f050 con 0x7f05f00b2280 2023-11-28T11:20:54.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.970+0000 7f05d77fe700 1 --2- 172.21.15.92:0/500044378 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f05dc0682f0 0x7f05dc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:54.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.970+0000 7f05d77fe700 1 -- 172.21.15.92:0/500044378 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f05e0020070 con 0x7f05f00b2280 2023-11-28T11:20:54.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.970+0000 7f05ee59c700 1 --2- 172.21.15.92:0/500044378 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f05dc0682f0 0x7f05dc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:54.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.971+0000 7f05ee59c700 1 --2- 172.21.15.92:0/500044378 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f05dc0682f0 0x7f05dc06a7a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f05f015c2b0 tx=0x7f05e4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:54.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:54.974+0000 7f05d77fe700 1 -- 172.21.15.92:0/500044378 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f05e00539f0 con 0x7f05f00b2280 2023-11-28T11:20:55.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.165+0000 7f05f51ff700 1 -- 172.21.15.92:0/500044378 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f05f0067880 con 0x7f05f00b2280 2023-11-28T11:20:55.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.741+0000 7f05d77fe700 1 -- 172.21.15.92:0/500044378 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v21) v1 ==== 171+0+0 (secure 0 0 0) 0x7f05e00150d0 con 0x7f05f00b2280 2023-11-28T11:20:55.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.748+0000 7f05f51ff700 1 -- 172.21.15.92:0/500044378 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f05dc0682f0 msgr2=0x7f05dc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:55.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.748+0000 7f05f51ff700 1 --2- 172.21.15.92:0/500044378 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f05dc0682f0 0x7f05dc06a7a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f05f015c2b0 tx=0x7f05e4008040 comp rx=0 tx=0).stop 2023-11-28T11:20:55.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.748+0000 7f05f51ff700 1 -- 172.21.15.92:0/500044378 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05f00b2280 msgr2=0x7f05f0159c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:55.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.748+0000 7f05f51ff700 1 --2- 172.21.15.92:0/500044378 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05f00b2280 0x7f05f0159c30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f05e0000c00 tx=0x7f05e0004750 comp rx=0 tx=0).stop 2023-11-28T11:20:55.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.748+0000 7f05f51ff700 1 -- 172.21.15.92:0/500044378 shutdown_connections 2023-11-28T11:20:55.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.748+0000 7f05f51ff700 1 --2- 172.21.15.92:0/500044378 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f05f015dee0 0x7f05f015f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:55.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.748+0000 7f05f51ff700 1 --2- 172.21.15.92:0/500044378 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f05dc0682f0 0x7f05dc06a7a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:55.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.748+0000 7f05f51ff700 1 --2- 172.21.15.92:0/500044378 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05f00b2280 0x7f05f0159c30 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:55.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.748+0000 7f05f51ff700 1 --2- 172.21.15.92:0/500044378 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f05f00d2e10 0x7f05f00cc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:55.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.748+0000 7f05f51ff700 1 -- 172.21.15.92:0/500044378 >> 172.21.15.92:0/500044378 conn(0x7f05f00719a0 msgr2=0x7f05f00b2890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:55.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.749+0000 7f05f51ff700 1 -- 172.21.15.92:0/500044378 shutdown_connections 2023-11-28T11:20:55.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.750+0000 7f05f51ff700 1 -- 172.21.15.92:0/500044378 wait complete. 2023-11-28T11:20:55.755 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' created 2023-11-28T11:20:55.787 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T11:20:55.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.986+0000 7f8c18dff700 1 Processor -- start 2023-11-28T11:20:55.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.987+0000 7f8c18dff700 1 -- start start 2023-11-28T11:20:55.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.988+0000 7f8c18dff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8c140d2df0 0x7f8c140cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:55.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.988+0000 7f8c18dff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c140d1950 con 0x7f8c140b4fc0 2023-11-28T11:20:55.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.988+0000 7f8c18dff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c140b2500 con 0x7f8c140b1c60 2023-11-28T11:20:55.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.988+0000 7f8c18dff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c140d3d30 con 0x7f8c140d2df0 2023-11-28T11:20:55.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.988+0000 7f8c0bfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f8c140b4fc0 0x7f8c140d31c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51886/0 (socket says 172.21.15.92:51886) 2023-11-28T11:20:55.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.988+0000 7f8c0bfff700 1 -- 172.21.15.92:0/3876638498 learned_addr learned my addr 172.21.15.92:0/3876638498 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:20:55.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.988+0000 7f8c11d9b700 1 -- 172.21.15.92:0/3876638498 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3462379089 0 0) 0x7f8c140d1950 con 0x7f8c140b4fc0 2023-11-28T11:20:55.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.988+0000 7f8c11d9b700 1 -- 172.21.15.92:0/3876638498 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c000035b0 con 0x7f8c140b4fc0 2023-11-28T11:20:55.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.988+0000 7f8c1259c700 1 --2- 172.21.15.92:0/3876638498 >> v2:172.21.15.169:3300/0 conn(0x7f8c140d2df0 0x7f8c140cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:55.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.989+0000 7f8c11d9b700 1 -- 172.21.15.92:0/3876638498 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3320306951 0 0) 0x7f8c000035b0 con 0x7f8c140b4fc0 2023-11-28T11:20:55.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.989+0000 7f8c11d9b700 1 -- 172.21.15.92:0/3876638498 >> v2:172.21.15.169:3300/0 conn(0x7f8c140d2df0 msgr2=0x7f8c140cd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:55.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.989+0000 7f8c11d9b700 1 --2- 172.21.15.92:0/3876638498 >> v2:172.21.15.169:3300/0 conn(0x7f8c140d2df0 0x7f8c140cd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:55.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.989+0000 7f8c11d9b700 1 -- 172.21.15.92:0/3876638498 >> v1:172.21.15.169:6789/0 conn(0x7f8c140b1c60 legacy=0x7f8c140cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:55.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.989+0000 7f8c11d9b700 1 -- 172.21.15.92:0/3876638498 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c140d3a50 con 0x7f8c140b4fc0 2023-11-28T11:20:55.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.989+0000 7f8c11d9b700 1 -- 172.21.15.92:0/3876638498 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532161587 0 0) 0x7f8bfc001670 con 0x7f8c140b4fc0 2023-11-28T11:20:55.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.989+0000 7f8c11d9b700 1 -- 172.21.15.92:0/3876638498 >> v1:172.21.15.92:6789/0 conn(0x7f8c140b4fc0 legacy=0x7f8c140d31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:55.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.989+0000 7f8c11d9b700 1 --2- 172.21.15.92:0/3876638498 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c00003a50 0x7f8c00003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:55.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.989+0000 7f8c11d9b700 1 --2- 172.21.15.92:0/3876638498 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c00004850 0x7f8c00004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:55.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.989+0000 7f8c11d9b700 1 --2- 172.21.15.92:0/3876638498 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c00005200 0x7f8c0000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:55.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.989+0000 7f8c11d9b700 1 -- 172.21.15.92:0/3876638498 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c140d3d30 con 0x7f8c00004850 2023-11-28T11:20:55.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.989+0000 7f8c11d9b700 1 -- 172.21.15.92:0/3876638498 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0000c0e0 con 0x7f8c00005200 2023-11-28T11:20:55.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.989+0000 7f8c11d9b700 1 -- 172.21.15.92:0/3876638498 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0000c2a0 con 0x7f8c00003a50 2023-11-28T11:20:55.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.989+0000 7f8c0bfff700 1 --2- 172.21.15.92:0/3876638498 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c00004850 0x7f8c00004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:55.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.989+0000 7f8c1259c700 1 --2- 172.21.15.92:0/3876638498 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c00003a50 0x7f8c00003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:55.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.989+0000 7f8c12d9d700 1 --2- 172.21.15.92:0/3876638498 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c00005200 0x7f8c0000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:55.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.989+0000 7f8c0bfff700 1 -- 172.21.15.92:0/3876638498 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c00003a50 msgr2=0x7f8c00003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:55.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.989+0000 7f8c0bfff700 1 --2- 172.21.15.92:0/3876638498 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c00003a50 0x7f8c00003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:55.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.989+0000 7f8c0bfff700 1 -- 172.21.15.92:0/3876638498 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c00005200 msgr2=0x7f8c0000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:55.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.989+0000 7f8c0bfff700 1 --2- 172.21.15.92:0/3876638498 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c00005200 0x7f8c0000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:55.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.989+0000 7f8c0bfff700 1 -- 172.21.15.92:0/3876638498 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c000035b0 con 0x7f8c00004850 2023-11-28T11:20:55.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.990+0000 7f8c12d9d700 1 --2- 172.21.15.92:0/3876638498 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c00005200 0x7f8c0000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:20:55.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.990+0000 7f8c12d9d700 1 -- 172.21.15.92:0/3876638498 reap_dead start 2023-11-28T11:20:55.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.990+0000 7f8c0bfff700 1 --2- 172.21.15.92:0/3876638498 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c00004850 0x7f8c00004cc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc001eb0 tx=0x7f8c00005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=640939d6a3452a5c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:55.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.990+0000 7f8c11d9b700 1 -- 172.21.15.92:0/3876638498 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc007b70 con 0x7f8c00004850 2023-11-28T11:20:55.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.990+0000 7f8c11d9b700 1 -- 172.21.15.92:0/3876638498 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bfc00ccf0 con 0x7f8c00004850 2023-11-28T11:20:55.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.990+0000 7f8c11d9b700 1 -- 172.21.15.92:0/3876638498 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc005ba0 con 0x7f8c00004850 2023-11-28T11:20:55.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.990+0000 7f8c18dff700 1 -- 172.21.15.92:0/3876638498 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c00004850 msgr2=0x7f8c00004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:55.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.990+0000 7f8c18dff700 1 --2- 172.21.15.92:0/3876638498 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c00004850 0x7f8c00004cc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc001eb0 tx=0x7f8c00005880 comp rx=0 tx=0).stop 2023-11-28T11:20:55.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.990+0000 7f8c18dff700 1 -- 172.21.15.92:0/3876638498 shutdown_connections 2023-11-28T11:20:55.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.990+0000 7f8c18dff700 1 --2- 172.21.15.92:0/3876638498 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c00004850 0x7f8c00004cc0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:55.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.990+0000 7f8c18dff700 1 -- 172.21.15.92:0/3876638498 >> 172.21.15.92:0/3876638498 conn(0x7f8c140719a0 msgr2=0x7f8c14071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:55.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.991+0000 7f8c18dff700 1 -- 172.21.15.92:0/3876638498 shutdown_connections 2023-11-28T11:20:55.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.991+0000 7f8c18dff700 1 -- 172.21.15.92:0/3876638498 wait complete. 2023-11-28T11:20:55.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.992+0000 7f8c18dff700 1 Processor -- start 2023-11-28T11:20:56.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.992+0000 7f8c18dff700 1 -- start start 2023-11-28T11:20:56.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.993+0000 7f8c18dff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c00004850 0x7f8c1414a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:56.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.993+0000 7f8c18dff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c14148f30 0x7f8c1415af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:56.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.993+0000 7f8c18dff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c141493a0 0x7f8c1415b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:56.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.993+0000 7f8c18dff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1414ba10 con 0x7f8c141493a0 2023-11-28T11:20:56.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.993+0000 7f8c18dff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1415b9b0 con 0x7f8c00004850 2023-11-28T11:20:56.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.993+0000 7f8c18dff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c141498f0 con 0x7f8c14148f30 2023-11-28T11:20:56.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.994+0000 7f8c12d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c141493a0 0x7f8c1415b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:56.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.994+0000 7f8c12d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c141493a0 0x7f8c1415b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41632/0 (socket says 172.21.15.92:41632) 2023-11-28T11:20:56.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.994+0000 7f8c12d9d700 1 -- 172.21.15.92:0/4253907067 learned_addr learned my addr 172.21.15.92:0/4253907067 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:20:56.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.994+0000 7f8c0bfff700 1 --2- 172.21.15.92:0/4253907067 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c14148f30 0x7f8c1415af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:56.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.994+0000 7f8c1259c700 1 --2- 172.21.15.92:0/4253907067 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c00004850 0x7f8c1414a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:56.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.994+0000 7f8c12d9d700 1 -- 172.21.15.92:0/4253907067 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c14148f30 msgr2=0x7f8c1415af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:56.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.994+0000 7f8c12d9d700 1 --2- 172.21.15.92:0/4253907067 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c14148f30 0x7f8c1415af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:56.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.994+0000 7f8c12d9d700 1 -- 172.21.15.92:0/4253907067 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c00004850 msgr2=0x7f8c1414a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:56.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.994+0000 7f8c12d9d700 1 --2- 172.21.15.92:0/4253907067 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c00004850 0x7f8c1414a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:56.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.994+0000 7f8c12d9d700 1 -- 172.21.15.92:0/4253907067 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c140b88f0 con 0x7f8c141493a0 2023-11-28T11:20:56.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.994+0000 7f8c12d9d700 1 --2- 172.21.15.92:0/4253907067 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c141493a0 0x7f8c1415b470 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8c140ce4e0 tx=0x7f8c0c00ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:56.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.994+0000 7f8c0affd700 1 -- 172.21.15.92:0/4253907067 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c003c10 con 0x7f8c141493a0 2023-11-28T11:20:56.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.994+0000 7f8c0affd700 1 -- 172.21.15.92:0/4253907067 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c0c003d70 con 0x7f8c141493a0 2023-11-28T11:20:56.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.994+0000 7f8c0affd700 1 -- 172.21.15.92:0/4253907067 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c0052f0 con 0x7f8c141493a0 2023-11-28T11:20:56.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.995+0000 7f8c18dff700 1 -- 172.21.15.92:0/4253907067 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c140d15e0 con 0x7f8c141493a0 2023-11-28T11:20:56.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.995+0000 7f8c18dff700 1 -- 172.21.15.92:0/4253907067 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c140b21b0 con 0x7f8c141493a0 2023-11-28T11:20:56.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.996+0000 7f8c18dff700 1 -- 172.21.15.92:0/4253907067 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c14067170 con 0x7f8c141493a0 2023-11-28T11:20:56.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.997+0000 7f8c0affd700 1 -- 172.21.15.92:0/4253907067 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8c0c005450 con 0x7f8c141493a0 2023-11-28T11:20:56.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.998+0000 7f8c0affd700 1 --2- 172.21.15.92:0/4253907067 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8bf40682f0 0x7f8bf406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:56.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.998+0000 7f8c0affd700 1 -- 172.21.15.92:0/4253907067 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f8c0c085d40 con 0x7f8c141493a0 2023-11-28T11:20:56.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.998+0000 7f8c1259c700 1 --2- 172.21.15.92:0/4253907067 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8bf40682f0 0x7f8bf406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:56.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:55.998+0000 7f8c1259c700 1 --2- 172.21.15.92:0/4253907067 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8bf40682f0 0x7f8bf406a7a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8c04009c20 tx=0x7f8c04008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:56.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.002+0000 7f8c0affd700 1 -- 172.21.15.92:0/4253907067 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8c0c052600 con 0x7f8c141493a0 2023-11-28T11:20:56.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.233+0000 7f8c18dff700 1 -- 172.21.15.92:0/4253907067 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f8c140b4630 con 0x7f8c141493a0 2023-11-28T11:20:56.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.239+0000 7f8c0affd700 1 -- 172.21.15.92:0/4253907067 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f8c0c055c20 con 0x7f8c141493a0 2023-11-28T11:20:56.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.243+0000 7f8c18dff700 1 -- 172.21.15.92:0/4253907067 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8bf40682f0 msgr2=0x7f8bf406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:56.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.243+0000 7f8c18dff700 1 --2- 172.21.15.92:0/4253907067 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8bf40682f0 0x7f8bf406a7a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8c04009c20 tx=0x7f8c04008040 comp rx=0 tx=0).stop 2023-11-28T11:20:56.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.243+0000 7f8c18dff700 1 -- 172.21.15.92:0/4253907067 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c141493a0 msgr2=0x7f8c1415b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:56.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.243+0000 7f8c18dff700 1 --2- 172.21.15.92:0/4253907067 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c141493a0 0x7f8c1415b470 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8c140ce4e0 tx=0x7f8c0c00ede0 comp rx=0 tx=0).stop 2023-11-28T11:20:56.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.243+0000 7f8c18dff700 1 -- 172.21.15.92:0/4253907067 shutdown_connections 2023-11-28T11:20:56.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.243+0000 7f8c18dff700 1 --2- 172.21.15.92:0/4253907067 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8bf40682f0 0x7f8bf406a7a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:56.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.243+0000 7f8c18dff700 1 --2- 172.21.15.92:0/4253907067 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c141493a0 0x7f8c1415b470 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:56.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.243+0000 7f8c18dff700 1 --2- 172.21.15.92:0/4253907067 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8c00004850 0x7f8c1414a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:56.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.243+0000 7f8c18dff700 1 --2- 172.21.15.92:0/4253907067 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8c14148f30 0x7f8c1415af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:56.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.243+0000 7f8c18dff700 1 -- 172.21.15.92:0/4253907067 >> 172.21.15.92:0/4253907067 conn(0x7f8c140719a0 msgr2=0x7f8c140b5c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:56.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.243+0000 7f8c18dff700 1 -- 172.21.15.92:0/4253907067 shutdown_connections 2023-11-28T11:20:56.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.244+0000 7f8c18dff700 1 -- 172.21.15.92:0/4253907067 wait complete. 2023-11-28T11:20:56.250 INFO:teuthology.orchestra.run.smithi092.stderr:new fs with metadata pool 3 and data pool 4 2023-11-28T11:20:56.280 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-11-28T11:20:56.280 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-11-28T11:20:56.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.481+0000 7fc4a75ff700 1 Processor -- start 2023-11-28T11:20:56.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.481+0000 7fc4a75ff700 1 -- start start 2023-11-28T11:20:56.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.482+0000 7fc4a75ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4a015d550 0x7fc4a00d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:56.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.482+0000 7fc4a75ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc4a00d3330 0x7fc4a00d0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:56.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.482+0000 7fc4a75ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4a00d3790 con 0x7fc4a015d180 2023-11-28T11:20:56.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.482+0000 7fc4a75ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4a00cf230 con 0x7fc4a00d3330 2023-11-28T11:20:56.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.482+0000 7fc4a75ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a00cd850 con 0x7fc4a015d550 2023-11-28T11:20:56.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.483+0000 7fc4a5b9c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc4a00d3330 0x7fc4a00d0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:56.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.483+0000 7fc4a5b9c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc4a00d3330 0x7fc4a00d0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41642/0 (socket says 172.21.15.92:41642) 2023-11-28T11:20:56.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.483+0000 7fc4a5b9c700 1 -- 172.21.15.92:0/1729972814 learned_addr learned my addr 172.21.15.92:0/1729972814 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:20:56.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.483+0000 7fc4a4b9a700 1 --2- 172.21.15.92:0/1729972814 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4a015d550 0x7fc4a00d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:56.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.483+0000 7fc4a5b9c700 1 -- 172.21.15.92:0/1729972814 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4a015d550 msgr2=0x7fc4a00d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:56.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.483+0000 7fc4a5b9c700 1 --2- 172.21.15.92:0/1729972814 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4a015d550 0x7fc4a00d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:56.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.483+0000 7fc4a5b9c700 1 -- 172.21.15.92:0/1729972814 >> v1:172.21.15.169:6789/0 conn(0x7fc4a015d180 legacy=0x7fc4a00cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:56.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.483+0000 7fc4a5b9c700 1 -- 172.21.15.92:0/1729972814 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4a00cf4d0 con 0x7fc4a00d3330 2023-11-28T11:20:56.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.484+0000 7fc4a5b9c700 1 --2- 172.21.15.92:0/1729972814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4a00d3330 0x7fc4a00d0a30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc498009e50 tx=0x7fc49800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b42f692dd889b750 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:56.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.484+0000 7fc497fff700 1 -- 172.21.15.92:0/1729972814 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49800e070 con 0x7fc4a00d3330 2023-11-28T11:20:56.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.484+0000 7fc497fff700 1 -- 172.21.15.92:0/1729972814 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc49800b910 con 0x7fc4a00d3330 2023-11-28T11:20:56.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.484+0000 7fc497fff700 1 -- 172.21.15.92:0/1729972814 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49800bcc0 con 0x7fc4a00d3330 2023-11-28T11:20:56.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.484+0000 7fc4a75ff700 1 -- 172.21.15.92:0/1729972814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4a00d3330 msgr2=0x7fc4a00d0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:56.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.484+0000 7fc4a75ff700 1 --2- 172.21.15.92:0/1729972814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4a00d3330 0x7fc4a00d0a30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc498009e50 tx=0x7fc49800b330 comp rx=0 tx=0).stop 2023-11-28T11:20:56.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.484+0000 7fc4a75ff700 1 -- 172.21.15.92:0/1729972814 shutdown_connections 2023-11-28T11:20:56.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.484+0000 7fc4a75ff700 1 --2- 172.21.15.92:0/1729972814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4a00d3330 0x7fc4a00d0a30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:56.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.484+0000 7fc4a75ff700 1 --2- 172.21.15.92:0/1729972814 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4a015d550 0x7fc4a00d2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:56.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.484+0000 7fc4a75ff700 1 -- 172.21.15.92:0/1729972814 >> 172.21.15.92:0/1729972814 conn(0x7fc4a00719a0 msgr2=0x7fc4a0071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:56.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.484+0000 7fc4a75ff700 1 -- 172.21.15.92:0/1729972814 shutdown_connections 2023-11-28T11:20:56.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.484+0000 7fc4a75ff700 1 -- 172.21.15.92:0/1729972814 wait complete. 2023-11-28T11:20:56.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.486+0000 7fc4a75ff700 1 Processor -- start 2023-11-28T11:20:56.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.486+0000 7fc4a75ff700 1 -- start start 2023-11-28T11:20:56.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.487+0000 7fc4a75ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4a00d3330 0x7fc4a0152380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:56.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.487+0000 7fc4a75ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4a015d180 0x7fc4a0154660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:56.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.487+0000 7fc4a75ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4a015d550 0x7fc4a0158090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:56.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.487+0000 7fc4a75ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a015a0d0 con 0x7fc4a015d550 2023-11-28T11:20:56.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.487+0000 7fc4a75ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a01528c0 con 0x7fc4a00d3330 2023-11-28T11:20:56.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.487+0000 7fc4a75ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a0152a80 con 0x7fc4a015d180 2023-11-28T11:20:56.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.487+0000 7fc4a5b9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4a015d550 0x7fc4a0158090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:56.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.487+0000 7fc4a539b700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4a00d3330 0x7fc4a0152380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:56.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.487+0000 7fc4a4b9a700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4a015d180 0x7fc4a0154660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:56.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.487+0000 7fc4a5b9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4a015d550 0x7fc4a0158090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41654/0 (socket says 172.21.15.92:41654) 2023-11-28T11:20:56.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.487+0000 7fc4a5b9c700 1 -- 172.21.15.92:0/1622451743 learned_addr learned my addr 172.21.15.92:0/1622451743 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:20:56.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.488+0000 7fc4a5b9c700 1 -- 172.21.15.92:0/1622451743 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4a015d180 msgr2=0x7fc4a0154660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:56.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.488+0000 7fc4a5b9c700 1 --2- 172.21.15.92:0/1622451743 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4a015d180 0x7fc4a0154660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:56.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.488+0000 7fc4a5b9c700 1 -- 172.21.15.92:0/1622451743 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4a00d3330 msgr2=0x7fc4a0152380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:56.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.488+0000 7fc4a5b9c700 1 --2- 172.21.15.92:0/1622451743 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4a00d3330 0x7fc4a0152380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:56.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.488+0000 7fc4a5b9c700 1 -- 172.21.15.92:0/1622451743 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc498009b30 con 0x7fc4a015d550 2023-11-28T11:20:56.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.488+0000 7fc4a5b9c700 1 --2- 172.21.15.92:0/1622451743 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4a015d550 0x7fc4a0158090 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc4a00725a0 tx=0x7fc498004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:56.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.488+0000 7fc495ffb700 1 -- 172.21.15.92:0/1622451743 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc498004800 con 0x7fc4a015d550 2023-11-28T11:20:56.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.488+0000 7fc495ffb700 1 -- 172.21.15.92:0/1622451743 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc498004960 con 0x7fc4a015d550 2023-11-28T11:20:56.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.488+0000 7fc4a75ff700 1 -- 172.21.15.92:0/1622451743 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4a015d9c0 con 0x7fc4a015d550 2023-11-28T11:20:56.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.488+0000 7fc495ffb700 1 -- 172.21.15.92:0/1622451743 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49801b670 con 0x7fc4a015d550 2023-11-28T11:20:56.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.488+0000 7fc4a75ff700 1 -- 172.21.15.92:0/1622451743 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4a00cf7b0 con 0x7fc4a015d550 2023-11-28T11:20:56.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.489+0000 7fc4a75ff700 1 -- 172.21.15.92:0/1622451743 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4a00cd4b0 con 0x7fc4a015d550 2023-11-28T11:20:56.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.490+0000 7fc495ffb700 1 -- 172.21.15.92:0/1622451743 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc49802b070 con 0x7fc4a015d550 2023-11-28T11:20:56.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.491+0000 7fc495ffb700 1 --2- 172.21.15.92:0/1622451743 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc4840682f0 0x7fc48406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:56.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.491+0000 7fc495ffb700 1 -- 172.21.15.92:0/1622451743 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc498087480 con 0x7fc4a015d550 2023-11-28T11:20:56.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.491+0000 7fc4a539b700 1 --2- 172.21.15.92:0/1622451743 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc4840682f0 0x7fc48406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:56.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.492+0000 7fc4a539b700 1 --2- 172.21.15.92:0/1622451743 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc4840682f0 0x7fc48406a7a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0156e50 tx=0x7fc49c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:56.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.496+0000 7fc495ffb700 1 -- 172.21.15.92:0/1622451743 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc498053bf0 con 0x7fc4a015d550 2023-11-28T11:20:56.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:56.699+0000 7fc4a75ff700 1 -- 172.21.15.92:0/1622451743 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7fc4a006c7b0 con 0x7fc4a015d550 2023-11-28T11:20:57.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.245+0000 7fc495ffb700 1 -- 172.21.15.92:0/1622451743 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v23) v1 ==== 157+0+0 (secure 0 0 0) 0x7fc498057210 con 0x7fc4a015d550 2023-11-28T11:20:57.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.248+0000 7fc4a75ff700 1 -- 172.21.15.92:0/1622451743 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc4840682f0 msgr2=0x7fc48406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:57.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.248+0000 7fc4a75ff700 1 --2- 172.21.15.92:0/1622451743 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc4840682f0 0x7fc48406a7a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0156e50 tx=0x7fc49c009040 comp rx=0 tx=0).stop 2023-11-28T11:20:57.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.248+0000 7fc4a75ff700 1 -- 172.21.15.92:0/1622451743 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4a015d550 msgr2=0x7fc4a0158090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:57.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.248+0000 7fc4a75ff700 1 --2- 172.21.15.92:0/1622451743 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4a015d550 0x7fc4a0158090 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc4a00725a0 tx=0x7fc498004ad0 comp rx=0 tx=0).stop 2023-11-28T11:20:57.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.248+0000 7fc4a75ff700 1 -- 172.21.15.92:0/1622451743 shutdown_connections 2023-11-28T11:20:57.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.248+0000 7fc4a75ff700 1 --2- 172.21.15.92:0/1622451743 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc4840682f0 0x7fc48406a7a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:57.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.248+0000 7fc4a75ff700 1 --2- 172.21.15.92:0/1622451743 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4a015d550 0x7fc4a0158090 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:57.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.248+0000 7fc4a75ff700 1 --2- 172.21.15.92:0/1622451743 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4a00d3330 0x7fc4a0152380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:57.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.248+0000 7fc4a75ff700 1 --2- 172.21.15.92:0/1622451743 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4a015d180 0x7fc4a0154660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:57.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.248+0000 7fc4a75ff700 1 -- 172.21.15.92:0/1622451743 >> 172.21.15.92:0/1622451743 conn(0x7fc4a00719a0 msgr2=0x7fc4a00d2240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:57.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.248+0000 7fc4a75ff700 1 -- 172.21.15.92:0/1622451743 shutdown_connections 2023-11-28T11:20:57.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.248+0000 7fc4a75ff700 1 -- 172.21.15.92:0/1622451743 wait complete. 2023-11-28T11:20:57.268 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T11:20:57.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.472+0000 7f7b3e1ff700 1 Processor -- start 2023-11-28T11:20:57.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.472+0000 7f7b3e1ff700 1 -- start start 2023-11-28T11:20:57.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.473+0000 7f7b3e1ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7b3815c700 0x7f7b380cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:57.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.473+0000 7f7b3e1ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7b380b1d60 0x7f7b380d28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:57.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.473+0000 7f7b3e1ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b380d2e10 0x7f7b380d0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:57.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.473+0000 7f7b3e1ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b380d02b0 con 0x7f7b3815c700 2023-11-28T11:20:57.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.473+0000 7f7b3e1ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b380cd500 con 0x7f7b380b1d60 2023-11-28T11:20:57.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.473+0000 7f7b3e1ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b380b22b0 con 0x7f7b380d2e10 2023-11-28T11:20:57.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.473+0000 7f7b377fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7b3815c700 0x7f7b380cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:57.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.473+0000 7f7b377fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7b3815c700 0x7f7b380cfd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41670/0 (socket says 172.21.15.92:41670) 2023-11-28T11:20:57.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.473+0000 7f7b377fe700 1 -- 172.21.15.92:0/2474148496 learned_addr learned my addr 172.21.15.92:0/2474148496 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:20:57.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.473+0000 7f7b2ffff700 1 --2- 172.21.15.92:0/2474148496 >> v2:172.21.15.169:3300/0 conn(0x7f7b380b1d60 0x7f7b380d28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:57.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.473+0000 7f7b37fff700 1 --2- 172.21.15.92:0/2474148496 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b380d2e10 0x7f7b380d0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:57.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.474+0000 7f7b377fe700 1 -- 172.21.15.92:0/2474148496 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b380d2e10 msgr2=0x7f7b380d0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:57.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.474+0000 7f7b377fe700 1 --2- 172.21.15.92:0/2474148496 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b380d2e10 0x7f7b380d0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:57.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.474+0000 7f7b377fe700 1 -- 172.21.15.92:0/2474148496 >> v2:172.21.15.169:3300/0 conn(0x7f7b380b1d60 msgr2=0x7f7b380d28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:57.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.474+0000 7f7b377fe700 1 --2- 172.21.15.92:0/2474148496 >> v2:172.21.15.169:3300/0 conn(0x7f7b380b1d60 0x7f7b380d28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:57.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.474+0000 7f7b377fe700 1 -- 172.21.15.92:0/2474148496 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b380b2590 con 0x7f7b3815c700 2023-11-28T11:20:57.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.474+0000 7f7b37fff700 1 --2- 172.21.15.92:0/2474148496 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b380d2e10 0x7f7b380d0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:20:57.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.474+0000 7f7b377fe700 1 --2- 172.21.15.92:0/2474148496 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b3815c700 0x7f7b380cfd70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7b3806a630 tx=0x7f7b28009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b75844472a9a3e5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:57.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.474+0000 7f7b36ffd700 1 -- 172.21.15.92:0/2474148496 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b28011830 con 0x7f7b3815c700 2023-11-28T11:20:57.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.474+0000 7f7b36ffd700 1 -- 172.21.15.92:0/2474148496 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b28011990 con 0x7f7b3815c700 2023-11-28T11:20:57.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.474+0000 7f7b36ffd700 1 -- 172.21.15.92:0/2474148496 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b28011d20 con 0x7f7b3815c700 2023-11-28T11:20:57.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.475+0000 7f7b3e1ff700 1 -- 172.21.15.92:0/2474148496 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b3815c700 msgr2=0x7f7b380cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:57.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.475+0000 7f7b3e1ff700 1 --2- 172.21.15.92:0/2474148496 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b3815c700 0x7f7b380cfd70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7b3806a630 tx=0x7f7b28009bd0 comp rx=0 tx=0).stop 2023-11-28T11:20:57.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.475+0000 7f7b3e1ff700 1 -- 172.21.15.92:0/2474148496 shutdown_connections 2023-11-28T11:20:57.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.475+0000 7f7b3e1ff700 1 --2- 172.21.15.92:0/2474148496 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b380d2e10 0x7f7b380d0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:57.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.475+0000 7f7b3e1ff700 1 --2- 172.21.15.92:0/2474148496 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b3815c700 0x7f7b380cfd70 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:57.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.475+0000 7f7b3e1ff700 1 --2- 172.21.15.92:0/2474148496 >> v2:172.21.15.169:3300/0 conn(0x7f7b380b1d60 0x7f7b380d28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:57.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.475+0000 7f7b3e1ff700 1 -- 172.21.15.92:0/2474148496 >> 172.21.15.92:0/2474148496 conn(0x7f7b380719a0 msgr2=0x7f7b38071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:57.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.475+0000 7f7b3e1ff700 1 -- 172.21.15.92:0/2474148496 shutdown_connections 2023-11-28T11:20:57.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.475+0000 7f7b3e1ff700 1 -- 172.21.15.92:0/2474148496 wait complete. 2023-11-28T11:20:57.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.477+0000 7f7b3e1ff700 1 Processor -- start 2023-11-28T11:20:57.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.477+0000 7f7b3e1ff700 1 -- start start 2023-11-28T11:20:57.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.479+0000 7f7b3e1ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b380b1d60 0x7f7b3815b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:57.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.479+0000 7f7b3e1ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b380d2e10 0x7f7b380c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:57.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.479+0000 7f7b3e1ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b3815c700 0x7f7b3815ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:57.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.479+0000 7f7b3e1ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b38159880 con 0x7f7b380d2e10 2023-11-28T11:20:57.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.479+0000 7f7b3e1ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b38158fe0 con 0x7f7b380b1d60 2023-11-28T11:20:57.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.479+0000 7f7b3e1ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3815bf30 con 0x7f7b3815c700 2023-11-28T11:20:57.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.479+0000 7f7b2ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b380d2e10 0x7f7b380c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:57.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.479+0000 7f7b2ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b380d2e10 0x7f7b380c8cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41678/0 (socket says 172.21.15.92:41678) 2023-11-28T11:20:57.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.479+0000 7f7b377fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b380b1d60 0x7f7b3815b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:57.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.479+0000 7f7b2ffff700 1 -- 172.21.15.92:0/310004396 learned_addr learned my addr 172.21.15.92:0/310004396 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:20:57.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.479+0000 7f7b37fff700 1 --2- 172.21.15.92:0/310004396 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b3815c700 0x7f7b3815ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:57.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.479+0000 7f7b2ffff700 1 -- 172.21.15.92:0/310004396 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b3815c700 msgr2=0x7f7b3815ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:57.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.479+0000 7f7b2ffff700 1 --2- 172.21.15.92:0/310004396 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b3815c700 0x7f7b3815ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:57.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.479+0000 7f7b2ffff700 1 -- 172.21.15.92:0/310004396 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b380b1d60 msgr2=0x7f7b3815b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:57.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.479+0000 7f7b2ffff700 1 --2- 172.21.15.92:0/310004396 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b380b1d60 0x7f7b3815b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:57.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.479+0000 7f7b2ffff700 1 -- 172.21.15.92:0/310004396 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b28009550 con 0x7f7b380d2e10 2023-11-28T11:20:57.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.479+0000 7f7b37fff700 1 --2- 172.21.15.92:0/310004396 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b3815c700 0x7f7b3815ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:20:57.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.480+0000 7f7b2ffff700 1 --2- 172.21.15.92:0/310004396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b380d2e10 0x7f7b380c8cd0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7b2000e8a0 tx=0x7f7b2000ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:57.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.480+0000 7f7b34ff9700 1 -- 172.21.15.92:0/310004396 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b20013680 con 0x7f7b380d2e10 2023-11-28T11:20:57.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.480+0000 7f7b34ff9700 1 -- 172.21.15.92:0/310004396 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b200137e0 con 0x7f7b380d2e10 2023-11-28T11:20:57.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.480+0000 7f7b3e1ff700 1 -- 172.21.15.92:0/310004396 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b3815cad0 con 0x7f7b380d2e10 2023-11-28T11:20:57.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.480+0000 7f7b34ff9700 1 -- 172.21.15.92:0/310004396 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b20013b90 con 0x7f7b380d2e10 2023-11-28T11:20:57.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.480+0000 7f7b3e1ff700 1 -- 172.21.15.92:0/310004396 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b380cd750 con 0x7f7b380d2e10 2023-11-28T11:20:57.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.481+0000 7f7b3e1ff700 1 -- 172.21.15.92:0/310004396 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b380c9cd0 con 0x7f7b380d2e10 2023-11-28T11:20:57.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.482+0000 7f7b34ff9700 1 -- 172.21.15.92:0/310004396 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7b20019030 con 0x7f7b380d2e10 2023-11-28T11:20:57.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.483+0000 7f7b34ff9700 1 --2- 172.21.15.92:0/310004396 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7b180682f0 0x7f7b1806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:57.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.483+0000 7f7b34ff9700 1 -- 172.21.15.92:0/310004396 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6288+0+0 (secure 0 0 0) 0x7f7b20085b40 con 0x7f7b380d2e10 2023-11-28T11:20:57.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.483+0000 7f7b377fe700 1 --2- 172.21.15.92:0/310004396 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7b180682f0 0x7f7b1806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:57.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.483+0000 7f7b377fe700 1 --2- 172.21.15.92:0/310004396 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7b180682f0 0x7f7b1806a7a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7b28009c10 tx=0x7f7b28008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:57.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.487+0000 7f7b34ff9700 1 -- 172.21.15.92:0/310004396 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7b200521f0 con 0x7f7b380d2e10 2023-11-28T11:20:57.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:57.677+0000 7f7b3e1ff700 1 -- 172.21.15.92:0/310004396 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f7b380cd2e0 con 0x7f7b380d2e10 2023-11-28T11:20:59.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.266+0000 7f7b34ff9700 1 -- 172.21.15.92:0/310004396 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v25) v1 ==== 228+0+0 (secure 0 0 0) 0x7f7b20055810 con 0x7f7b380d2e10 2023-11-28T11:20:59.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.273+0000 7f7b3e1ff700 1 -- 172.21.15.92:0/310004396 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7b180682f0 msgr2=0x7f7b1806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:59.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.273+0000 7f7b3e1ff700 1 --2- 172.21.15.92:0/310004396 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7b180682f0 0x7f7b1806a7a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7b28009c10 tx=0x7f7b28008030 comp rx=0 tx=0).stop 2023-11-28T11:20:59.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.273+0000 7f7b3e1ff700 1 -- 172.21.15.92:0/310004396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b380d2e10 msgr2=0x7f7b380c8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:59.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.273+0000 7f7b3e1ff700 1 --2- 172.21.15.92:0/310004396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b380d2e10 0x7f7b380c8cd0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7b2000e8a0 tx=0x7f7b2000ebb0 comp rx=0 tx=0).stop 2023-11-28T11:20:59.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.275+0000 7f7b3e1ff700 1 -- 172.21.15.92:0/310004396 shutdown_connections 2023-11-28T11:20:59.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.275+0000 7f7b3e1ff700 1 --2- 172.21.15.92:0/310004396 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b3815c700 0x7f7b3815ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:59.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.275+0000 7f7b3e1ff700 1 --2- 172.21.15.92:0/310004396 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b380b1d60 0x7f7b3815b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:59.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.275+0000 7f7b3e1ff700 1 --2- 172.21.15.92:0/310004396 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7b180682f0 0x7f7b1806a7a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:59.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.275+0000 7f7b3e1ff700 1 --2- 172.21.15.92:0/310004396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b380d2e10 0x7f7b380c8cd0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:59.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.275+0000 7f7b3e1ff700 1 -- 172.21.15.92:0/310004396 >> 172.21.15.92:0/310004396 conn(0x7f7b380719a0 msgr2=0x7f7b38071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:59.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.278+0000 7f7b3e1ff700 1 -- 172.21.15.92:0/310004396 shutdown_connections 2023-11-28T11:20:59.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.278+0000 7f7b3e1ff700 1 -- 172.21.15.92:0/310004396 wait complete. 2023-11-28T11:20:59.286 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data_ec' created 2023-11-28T11:20:59.301 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-11-28T11:20:59.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.503+0000 7fcdc57ff700 1 Processor -- start 2023-11-28T11:20:59.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.503+0000 7fcdc57ff700 1 -- start start 2023-11-28T11:20:59.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.504+0000 7fcdc57ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcdc00cd460 0x7fcdc006b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:59.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.504+0000 7fcdc57ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdc006be80 0x7fcdc00d0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:59.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.504+0000 7fcdc57ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdc00b5240 con 0x7fcdc00b8790 2023-11-28T11:20:59.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.505+0000 7fcdc57ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdc006c300 con 0x7fcdc00cd460 2023-11-28T11:20:59.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.505+0000 7fcdc57ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc00b5520 con 0x7fcdc006be80 2023-11-28T11:20:59.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.505+0000 7fcdbeffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fcdc00b8790 0x7fcdc00cfb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46442/0 (socket says 172.21.15.92:46442) 2023-11-28T11:20:59.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.505+0000 7fcdbeffd700 1 -- 172.21.15.92:0/547127389 learned_addr learned my addr 172.21.15.92:0/547127389 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:20:59.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.505+0000 7fcdbf7fe700 1 --2- 172.21.15.92:0/547127389 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdc006be80 0x7fcdc00d0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:59.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.505+0000 7fcdb7fff700 1 --2- 172.21.15.92:0/547127389 >> v2:172.21.15.169:3300/0 conn(0x7fcdc00cd460 0x7fcdc006b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:59.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.505+0000 7fcdbe7fc700 1 -- 172.21.15.92:0/547127389 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2805209828 0 0) 0x7fcdc00b5240 con 0x7fcdc00b8790 2023-11-28T11:20:59.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.505+0000 7fcdbe7fc700 1 -- 172.21.15.92:0/547127389 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdac0035b0 con 0x7fcdc00b8790 2023-11-28T11:20:59.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.505+0000 7fcdbe7fc700 1 -- 172.21.15.92:0/547127389 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3282135609 0 0) 0x7fcdac0035b0 con 0x7fcdc00b8790 2023-11-28T11:20:59.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.505+0000 7fcdbe7fc700 1 -- 172.21.15.92:0/547127389 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdc006be80 msgr2=0x7fcdc00d0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:59.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.505+0000 7fcdbe7fc700 1 --2- 172.21.15.92:0/547127389 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdc006be80 0x7fcdc00d0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:59.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.505+0000 7fcdbe7fc700 1 -- 172.21.15.92:0/547127389 >> v2:172.21.15.169:3300/0 conn(0x7fcdc00cd460 msgr2=0x7fcdc006b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:59.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.505+0000 7fcdbe7fc700 1 --2- 172.21.15.92:0/547127389 >> v2:172.21.15.169:3300/0 conn(0x7fcdc00cd460 0x7fcdc006b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:59.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.505+0000 7fcdbe7fc700 1 -- 172.21.15.92:0/547127389 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc00d0a80 con 0x7fcdc00b8790 2023-11-28T11:20:59.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.506+0000 7fcdbe7fc700 1 -- 172.21.15.92:0/547127389 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532161587 0 0) 0x7fcdb0001670 con 0x7fcdc00b8790 2023-11-28T11:20:59.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.506+0000 7fcdbe7fc700 1 -- 172.21.15.92:0/547127389 >> v1:172.21.15.92:6789/0 conn(0x7fcdc00b8790 legacy=0x7fcdc00cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:59.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.506+0000 7fcdbe7fc700 1 --2- 172.21.15.92:0/547127389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdac003a00 0x7fcdac003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:59.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.506+0000 7fcdbe7fc700 1 --2- 172.21.15.92:0/547127389 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdc006be80 0x7fcdac004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:59.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.506+0000 7fcdbe7fc700 1 --2- 172.21.15.92:0/547127389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcdac004d90 0x7fcdac005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:59.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.506+0000 7fcdbe7fc700 1 -- 172.21.15.92:0/547127389 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc006c300 con 0x7fcdac004d90 2023-11-28T11:20:59.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.506+0000 7fcdbe7fc700 1 -- 172.21.15.92:0/547127389 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc00b5520 con 0x7fcdac003a00 2023-11-28T11:20:59.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.506+0000 7fcdbe7fc700 1 -- 172.21.15.92:0/547127389 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdac00ba10 con 0x7fcdc006be80 2023-11-28T11:20:59.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.506+0000 7fcdbeffd700 1 --2- 172.21.15.92:0/547127389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdac003a00 0x7fcdac003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:59.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.506+0000 7fcdbf7fe700 1 --2- 172.21.15.92:0/547127389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcdac004d90 0x7fcdac005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:59.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.506+0000 7fcdb7fff700 1 --2- 172.21.15.92:0/547127389 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdc006be80 0x7fcdac004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:59.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.506+0000 7fcdb7fff700 1 -- 172.21.15.92:0/547127389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdac003a00 msgr2=0x7fcdac003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:59.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.506+0000 7fcdb7fff700 1 --2- 172.21.15.92:0/547127389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdac003a00 0x7fcdac003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:59.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.506+0000 7fcdb7fff700 1 -- 172.21.15.92:0/547127389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcdac004d90 msgr2=0x7fcdac005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:20:59.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.506+0000 7fcdb7fff700 1 --2- 172.21.15.92:0/547127389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcdac004d90 0x7fcdac005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:59.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.506+0000 7fcdb7fff700 1 -- 172.21.15.92:0/547127389 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdac0035b0 con 0x7fcdc006be80 2023-11-28T11:20:59.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.506+0000 7fcdbf7fe700 1 --2- 172.21.15.92:0/547127389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcdac004d90 0x7fcdac005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:20:59.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.507+0000 7fcdb7fff700 1 --2- 172.21.15.92:0/547127389 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdc006be80 0x7fcdac004850 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcda8009b60 tx=0x7fcdac005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9864e977b0d9d7f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:59.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.507+0000 7fcdbe7fc700 1 -- 172.21.15.92:0/547127389 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda800e6c0 con 0x7fcdc006be80 2023-11-28T11:20:59.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.507+0000 7fcdbe7fc700 1 -- 172.21.15.92:0/547127389 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcda800e820 con 0x7fcdc006be80 2023-11-28T11:20:59.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.507+0000 7fcdbe7fc700 1 -- 172.21.15.92:0/547127389 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda800eb90 con 0x7fcdc006be80 2023-11-28T11:20:59.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.508+0000 7fcdc57ff700 1 -- 172.21.15.92:0/547127389 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdc006be80 msgr2=0x7fcdac004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:59.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.508+0000 7fcdc57ff700 1 --2- 172.21.15.92:0/547127389 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdc006be80 0x7fcdac004850 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcda8009b60 tx=0x7fcdac005880 comp rx=0 tx=0).stop 2023-11-28T11:20:59.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.508+0000 7fcdbf7fe700 1 -- 172.21.15.92:0/547127389 reap_dead start 2023-11-28T11:20:59.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.508+0000 7fcdc57ff700 1 -- 172.21.15.92:0/547127389 shutdown_connections 2023-11-28T11:20:59.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.508+0000 7fcdc57ff700 1 -- 172.21.15.92:0/547127389 >> 172.21.15.92:0/547127389 conn(0x7fcdc00719a0 msgr2=0x7fcdc0071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:20:59.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.508+0000 7fcdc57ff700 1 -- 172.21.15.92:0/547127389 shutdown_connections 2023-11-28T11:20:59.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.508+0000 7fcdc57ff700 1 -- 172.21.15.92:0/547127389 wait complete. 2023-11-28T11:20:59.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.509+0000 7fcdc57ff700 1 Processor -- start 2023-11-28T11:20:59.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.510+0000 7fcdc57ff700 1 -- start start 2023-11-28T11:20:59.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.510+0000 7fcdc57ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdc006be80 0x7fcdc00cc670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:59.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.510+0000 7fcdc57ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdc00c86f0 0x7fcdc0157060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:59.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.510+0000 7fcdc57ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcdc00c8b60 0x7fcdc01575a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:59.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.510+0000 7fcdc57ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc01f22f0 con 0x7fcdc00c8b60 2023-11-28T11:20:59.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.510+0000 7fcdc57ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc01f24e0 con 0x7fcdc006be80 2023-11-28T11:20:59.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.510+0000 7fcdc57ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc01f27c0 con 0x7fcdc00c86f0 2023-11-28T11:20:59.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.511+0000 7fcdbf7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcdc00c8b60 0x7fcdc01575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:59.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.511+0000 7fcdbf7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcdc00c8b60 0x7fcdc01575a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60304/0 (socket says 172.21.15.92:60304) 2023-11-28T11:20:59.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.511+0000 7fcdbf7fe700 1 -- 172.21.15.92:0/3298237746 learned_addr learned my addr 172.21.15.92:0/3298237746 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:20:59.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.511+0000 7fcdb7fff700 1 --2- 172.21.15.92:0/3298237746 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdc00c86f0 0x7fcdc0157060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:59.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.511+0000 7fcdbeffd700 1 --2- 172.21.15.92:0/3298237746 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdc006be80 0x7fcdc00cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:59.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.511+0000 7fcdbf7fe700 1 -- 172.21.15.92:0/3298237746 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdc00c86f0 msgr2=0x7fcdc0157060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:59.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.511+0000 7fcdbf7fe700 1 --2- 172.21.15.92:0/3298237746 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdc00c86f0 0x7fcdc0157060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:59.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.511+0000 7fcdbf7fe700 1 -- 172.21.15.92:0/3298237746 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdc006be80 msgr2=0x7fcdc00cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:20:59.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.511+0000 7fcdbf7fe700 1 --2- 172.21.15.92:0/3298237746 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdc006be80 0x7fcdc00cc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:20:59.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.511+0000 7fcdbf7fe700 1 -- 172.21.15.92:0/3298237746 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc00d0fa0 con 0x7fcdc00c8b60 2023-11-28T11:20:59.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.511+0000 7fcdbeffd700 1 --2- 172.21.15.92:0/3298237746 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdc006be80 0x7fcdc00cc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:20:59.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.511+0000 7fcdbf7fe700 1 --2- 172.21.15.92:0/3298237746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcdc00c8b60 0x7fcdc01575a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0072610 tx=0x7fcdb800f490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:59.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.511+0000 7fcdb77fe700 1 -- 172.21.15.92:0/3298237746 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb801c070 con 0x7fcdc00c8b60 2023-11-28T11:20:59.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.512+0000 7fcdb77fe700 1 -- 172.21.15.92:0/3298237746 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdb800f040 con 0x7fcdc00c8b60 2023-11-28T11:20:59.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.512+0000 7fcdb77fe700 1 -- 172.21.15.92:0/3298237746 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb800e040 con 0x7fcdc00c8b60 2023-11-28T11:20:59.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.512+0000 7fcdc57ff700 1 -- 172.21.15.92:0/3298237746 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdc00b5800 con 0x7fcdc00c8b60 2023-11-28T11:20:59.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.512+0000 7fcdc57ff700 1 -- 172.21.15.92:0/3298237746 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdc00d1130 con 0x7fcdc00c8b60 2023-11-28T11:20:59.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.513+0000 7fcdc57ff700 1 -- 172.21.15.92:0/3298237746 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdc0067880 con 0x7fcdc00c8b60 2023-11-28T11:20:59.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.514+0000 7fcdb77fe700 1 -- 172.21.15.92:0/3298237746 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcdb8015430 con 0x7fcdc00c8b60 2023-11-28T11:20:59.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.514+0000 7fcdb77fe700 1 --2- 172.21.15.92:0/3298237746 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fcda0068320 0x7fcda006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:20:59.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.515+0000 7fcdb77fe700 1 -- 172.21.15.92:0/3298237746 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6781+0+0 (secure 0 0 0) 0x7fcdb8085d00 con 0x7fcdc00c8b60 2023-11-28T11:20:59.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.515+0000 7fcdbeffd700 1 --2- 172.21.15.92:0/3298237746 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fcda0068320 0x7fcda006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:20:59.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.515+0000 7fcdbeffd700 1 --2- 172.21.15.92:0/3298237746 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fcda0068320 0x7fcda006a7d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0007cf0 tx=0x7fcdb0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:20:59.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.519+0000 7fcdb77fe700 1 -- 172.21.15.92:0/3298237746 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fcdb8052140 con 0x7fcdc00c8b60 2023-11-28T11:20:59.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:20:59.709+0000 7fcdc57ff700 1 -- 172.21.15.92:0/3298237746 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7fcdc00d2120 con 0x7fcdc00c8b60 2023-11-28T11:21:00.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.270+0000 7fcdb77fe700 1 -- 172.21.15.92:0/3298237746 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 5 allow_ec_overwrites to true v26) v1 ==== 169+0+0 (secure 0 0 0) 0x7fcdb8009c10 con 0x7fcdc00c8b60 2023-11-28T11:21:00.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.276+0000 7fcdc57ff700 1 -- 172.21.15.92:0/3298237746 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fcda0068320 msgr2=0x7fcda006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:00.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.276+0000 7fcdc57ff700 1 --2- 172.21.15.92:0/3298237746 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fcda0068320 0x7fcda006a7d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0007cf0 tx=0x7fcdb0009040 comp rx=0 tx=0).stop 2023-11-28T11:21:00.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.276+0000 7fcdc57ff700 1 -- 172.21.15.92:0/3298237746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcdc00c8b60 msgr2=0x7fcdc01575a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:00.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.276+0000 7fcdc57ff700 1 --2- 172.21.15.92:0/3298237746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcdc00c8b60 0x7fcdc01575a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0072610 tx=0x7fcdb800f490 comp rx=0 tx=0).stop 2023-11-28T11:21:00.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.276+0000 7fcdc57ff700 1 -- 172.21.15.92:0/3298237746 shutdown_connections 2023-11-28T11:21:00.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.276+0000 7fcdc57ff700 1 --2- 172.21.15.92:0/3298237746 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fcda0068320 0x7fcda006a7d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:00.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.276+0000 7fcdc57ff700 1 --2- 172.21.15.92:0/3298237746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcdc00c8b60 0x7fcdc01575a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:00.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.276+0000 7fcdc57ff700 1 --2- 172.21.15.92:0/3298237746 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdc006be80 0x7fcdc00cc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:00.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.276+0000 7fcdc57ff700 1 --2- 172.21.15.92:0/3298237746 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdc00c86f0 0x7fcdc0157060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:00.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.276+0000 7fcdc57ff700 1 -- 172.21.15.92:0/3298237746 >> 172.21.15.92:0/3298237746 conn(0x7fcdc00719a0 msgr2=0x7fcdc00d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:00.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.279+0000 7fcdc57ff700 1 -- 172.21.15.92:0/3298237746 shutdown_connections 2023-11-28T11:21:00.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.279+0000 7fcdc57ff700 1 -- 172.21.15.92:0/3298237746 wait complete. 2023-11-28T11:21:00.282 INFO:teuthology.orchestra.run.smithi092.stderr:set pool 5 allow_ec_overwrites to true 2023-11-28T11:21:00.320 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-11-28T11:21:00.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.516+0000 7f54525ff700 1 Processor -- start 2023-11-28T11:21:00.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.516+0000 7f54525ff700 1 -- start start 2023-11-28T11:21:00.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.517+0000 7f54525ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f544c0cd460 0x7f544c06c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:00.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.517+0000 7f54525ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f544c0d0d60 0x7f544c0d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:00.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.517+0000 7f54525ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f544c0b88e0 con 0x7f544c0b8510 2023-11-28T11:21:00.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.517+0000 7f54525ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f544c0cdbd0 con 0x7f544c0cd460 2023-11-28T11:21:00.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.517+0000 7f54525ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f544c0d02d0 con 0x7f544c0d0d60 2023-11-28T11:21:00.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.517+0000 7f544bfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f544c0b8510 0x7f544c06b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46452/0 (socket says 172.21.15.92:46452) 2023-11-28T11:21:00.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.517+0000 7f544bfff700 1 -- 172.21.15.92:0/2808802392 learned_addr learned my addr 172.21.15.92:0/2808802392 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:00.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.517+0000 7f544b7fe700 1 --2- 172.21.15.92:0/2808802392 >> v2:172.21.15.92:3300/0 conn(0x7f544c0cd460 0x7f544c06c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:00.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.517+0000 7f5450b9c700 1 --2- 172.21.15.92:0/2808802392 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f544c0d0d60 0x7f544c0d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:00.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.517+0000 7f544b7fe700 1 -- 172.21.15.92:0/2808802392 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f544c0d0d60 msgr2=0x7f544c0d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:00.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.517+0000 7f544b7fe700 1 --2- 172.21.15.92:0/2808802392 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f544c0d0d60 0x7f544c0d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:00.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.517+0000 7f544b7fe700 1 -- 172.21.15.92:0/2808802392 >> v1:172.21.15.92:6789/0 conn(0x7f544c0b8510 legacy=0x7f544c06b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:00.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.518+0000 7f544b7fe700 1 -- 172.21.15.92:0/2808802392 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f544c0cd8f0 con 0x7f544c0cd460 2023-11-28T11:21:00.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.518+0000 7f544affd700 1 -- 172.21.15.92:0/2808802392 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3018028963 0 0) 0x7f544c0b88e0 con 0x7f544c0b8510 2023-11-28T11:21:00.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.518+0000 7f5450b9c700 1 --2- 172.21.15.92:0/2808802392 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f544c0d0d60 0x7f544c0d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:21:00.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.518+0000 7f544b7fe700 1 --2- 172.21.15.92:0/2808802392 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f544c0cd460 0x7f544c06c080 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5440009870 tx=0x7f5440009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdcc51310a9eeec0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:00.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.518+0000 7f544affd700 1 -- 172.21.15.92:0/2808802392 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5440011830 con 0x7f544c0cd460 2023-11-28T11:21:00.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.518+0000 7f544affd700 1 -- 172.21.15.92:0/2808802392 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5440011990 con 0x7f544c0cd460 2023-11-28T11:21:00.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.518+0000 7f544affd700 1 -- 172.21.15.92:0/2808802392 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5440011d20 con 0x7f544c0cd460 2023-11-28T11:21:00.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.518+0000 7f54525ff700 1 -- 172.21.15.92:0/2808802392 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f544c0cd460 msgr2=0x7f544c06c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:00.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.518+0000 7f54525ff700 1 --2- 172.21.15.92:0/2808802392 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f544c0cd460 0x7f544c06c080 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5440009870 tx=0x7f5440009b80 comp rx=0 tx=0).stop 2023-11-28T11:21:00.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.519+0000 7f54525ff700 1 -- 172.21.15.92:0/2808802392 shutdown_connections 2023-11-28T11:21:00.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.519+0000 7f54525ff700 1 --2- 172.21.15.92:0/2808802392 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f544c0d0d60 0x7f544c0d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:00.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.519+0000 7f54525ff700 1 --2- 172.21.15.92:0/2808802392 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f544c0cd460 0x7f544c06c080 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:00.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.519+0000 7f54525ff700 1 -- 172.21.15.92:0/2808802392 >> 172.21.15.92:0/2808802392 conn(0x7f544c0719a0 msgr2=0x7f544c071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:00.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.519+0000 7f54525ff700 1 -- 172.21.15.92:0/2808802392 shutdown_connections 2023-11-28T11:21:00.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.519+0000 7f54525ff700 1 -- 172.21.15.92:0/2808802392 wait complete. 2023-11-28T11:21:00.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.520+0000 7f54525ff700 1 Processor -- start 2023-11-28T11:21:00.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.521+0000 7f54525ff700 1 -- start start 2023-11-28T11:21:00.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.521+0000 7f54525ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f544c0b8510 0x7f544c155ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:00.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.521+0000 7f54525ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f544c0cd460 0x7f544c0b7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:00.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.521+0000 7f54525ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f544c0d0d60 0x7f544c0b7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:00.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.521+0000 7f54525ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f544c15a340 con 0x7f544c0d0d60 2023-11-28T11:21:00.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.521+0000 7f54525ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f544c0ca460 con 0x7f544c0b8510 2023-11-28T11:21:00.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.521+0000 7f54525ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f544c159d80 con 0x7f544c0cd460 2023-11-28T11:21:00.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.522+0000 7f5450b9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f544c0d0d60 0x7f544c0b7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:00.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.522+0000 7f544b7fe700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f544c0cd460 0x7f544c0b7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:00.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.522+0000 7f5450b9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f544c0d0d60 0x7f544c0b7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60316/0 (socket says 172.21.15.92:60316) 2023-11-28T11:21:00.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.522+0000 7f5450b9c700 1 -- 172.21.15.92:0/691653273 learned_addr learned my addr 172.21.15.92:0/691653273 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:00.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.522+0000 7f544bfff700 1 --2- 172.21.15.92:0/691653273 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f544c0b8510 0x7f544c155ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:00.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.522+0000 7f5450b9c700 1 -- 172.21.15.92:0/691653273 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f544c0cd460 msgr2=0x7f544c0b7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:00.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.522+0000 7f5450b9c700 1 --2- 172.21.15.92:0/691653273 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f544c0cd460 0x7f544c0b7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:00.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.522+0000 7f5450b9c700 1 -- 172.21.15.92:0/691653273 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f544c0b8510 msgr2=0x7f544c155ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:00.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.522+0000 7f5450b9c700 1 --2- 172.21.15.92:0/691653273 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f544c0b8510 0x7f544c155ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:00.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.522+0000 7f5450b9c700 1 -- 172.21.15.92:0/691653273 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54400094a0 con 0x7f544c0d0d60 2023-11-28T11:21:00.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.522+0000 7f5450b9c700 1 --2- 172.21.15.92:0/691653273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f544c0d0d60 0x7f544c0b7550 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f544400cd50 tx=0x7f544400a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:00.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.523+0000 7f5448ff9700 1 -- 172.21.15.92:0/691653273 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544400cea0 con 0x7f544c0d0d60 2023-11-28T11:21:00.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.523+0000 7f5448ff9700 1 -- 172.21.15.92:0/691653273 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f544400f480 con 0x7f544c0d0d60 2023-11-28T11:21:00.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.523+0000 7f54525ff700 1 -- 172.21.15.92:0/691653273 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f544c0cfff0 con 0x7f544c0d0d60 2023-11-28T11:21:00.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.523+0000 7f5448ff9700 1 -- 172.21.15.92:0/691653273 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544400f830 con 0x7f544c0d0d60 2023-11-28T11:21:00.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.523+0000 7f54525ff700 1 -- 172.21.15.92:0/691653273 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f544c0cfd10 con 0x7f544c0d0d60 2023-11-28T11:21:00.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.524+0000 7f54525ff700 1 -- 172.21.15.92:0/691653273 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f544c0ca5a0 con 0x7f544c0d0d60 2023-11-28T11:21:00.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.525+0000 7f5448ff9700 1 -- 172.21.15.92:0/691653273 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5444009030 con 0x7f544c0d0d60 2023-11-28T11:21:00.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.525+0000 7f5448ff9700 1 --2- 172.21.15.92:0/691653273 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f54380682f0 0x7f543806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:00.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.525+0000 7f5448ff9700 1 -- 172.21.15.92:0/691653273 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6781+0+0 (secure 0 0 0) 0x7f5444086920 con 0x7f544c0d0d60 2023-11-28T11:21:00.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.526+0000 7f544bfff700 1 --2- 172.21.15.92:0/691653273 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f54380682f0 0x7f543806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:00.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.526+0000 7f544bfff700 1 --2- 172.21.15.92:0/691653273 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f54380682f0 0x7f543806a7a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f543c007fd0 tx=0x7f543c007de0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:00.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.530+0000 7f5448ff9700 1 -- 172.21.15.92:0/691653273 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f544c0ca5a0 con 0x7f544c0d0d60 2023-11-28T11:21:00.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.769+0000 7f54525ff700 1 -- 172.21.15.92:0/691653273 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f544c0ca5a0 con 0x7f544c0d0d60 2023-11-28T11:21:00.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.774+0000 7f5448ff9700 1 -- 172.21.15.92:0/691653273 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 5 to fsmap v7) v1 ==== 137+0+0 (secure 0 0 0) 0x7f544c0ca5a0 con 0x7f544c0d0d60 2023-11-28T11:21:00.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.777+0000 7f54525ff700 1 -- 172.21.15.92:0/691653273 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f54380682f0 msgr2=0x7f543806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:00.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.777+0000 7f54525ff700 1 --2- 172.21.15.92:0/691653273 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f54380682f0 0x7f543806a7a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f543c007fd0 tx=0x7f543c007de0 comp rx=0 tx=0).stop 2023-11-28T11:21:00.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.777+0000 7f54525ff700 1 -- 172.21.15.92:0/691653273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f544c0d0d60 msgr2=0x7f544c0b7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:00.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.777+0000 7f54525ff700 1 --2- 172.21.15.92:0/691653273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f544c0d0d60 0x7f544c0b7550 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f544400cd50 tx=0x7f544400a560 comp rx=0 tx=0).stop 2023-11-28T11:21:00.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.777+0000 7f54525ff700 1 -- 172.21.15.92:0/691653273 shutdown_connections 2023-11-28T11:21:00.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.777+0000 7f54525ff700 1 --2- 172.21.15.92:0/691653273 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f54380682f0 0x7f543806a7a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:00.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.777+0000 7f54525ff700 1 --2- 172.21.15.92:0/691653273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f544c0d0d60 0x7f544c0b7550 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:00.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.777+0000 7f54525ff700 1 --2- 172.21.15.92:0/691653273 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f544c0b8510 0x7f544c155ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:00.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.777+0000 7f54525ff700 1 --2- 172.21.15.92:0/691653273 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f544c0cd460 0x7f544c0b7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:00.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.777+0000 7f54525ff700 1 -- 172.21.15.92:0/691653273 >> 172.21.15.92:0/691653273 conn(0x7f544c0719a0 msgr2=0x7f544c0b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:00.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.777+0000 7f54525ff700 1 -- 172.21.15.92:0/691653273 shutdown_connections 2023-11-28T11:21:00.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.777+0000 7f54525ff700 1 -- 172.21.15.92:0/691653273 wait complete. 2023-11-28T11:21:00.779 INFO:teuthology.orchestra.run.smithi092.stderr:added data pool 5 to fsmap 2023-11-28T11:21:00.796 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:21:00.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.991+0000 7f324bbff700 1 Processor -- start 2023-11-28T11:21:00.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.992+0000 7f324bbff700 1 -- start start 2023-11-28T11:21:00.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.993+0000 7f324bbff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f32440b4fc0 0x7f32440cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:00.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.993+0000 7f324bbff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32440b1c60 0x7f32440cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:00.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.993+0000 7f324bbff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32440d1950 con 0x7f32440d2df0 2023-11-28T11:21:00.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.993+0000 7f324bbff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32440d3d30 con 0x7f32440b4fc0 2023-11-28T11:21:00.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.993+0000 7f324bbff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32440b2500 con 0x7f32440b1c60 2023-11-28T11:21:00.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.993+0000 7f324a19c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32440b1c60 0x7f32440cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:00.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.993+0000 7f324a19c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32440b1c60 0x7f32440cf010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.92:39214/0 (socket says 172.21.15.92:39214) 2023-11-28T11:21:00.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.993+0000 7f324a19c700 1 -- 172.21.15.92:0/1897985414 learned_addr learned my addr 172.21.15.92:0/1897985414 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:00.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.993+0000 7f324999b700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f32440d2df0 0x7f32440d31c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.92:60608/0 (socket says 172.21.15.92:60608) 2023-11-28T11:21:00.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.993+0000 7f324919a700 1 --2- 172.21.15.92:0/1897985414 >> v2:172.21.15.169:3300/0 conn(0x7f32440b4fc0 0x7f32440cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:00.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.994+0000 7f3248999700 1 -- 172.21.15.92:0/1897985414 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1313271922 0 0) 0x7f32440d1950 con 0x7f32440d2df0 2023-11-28T11:21:00.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.994+0000 7f3248999700 1 -- 172.21.15.92:0/1897985414 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32300035b0 con 0x7f32440d2df0 2023-11-28T11:21:00.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.994+0000 7f324a19c700 1 -- 172.21.15.92:0/1897985414 >> v2:172.21.15.169:3300/0 conn(0x7f32440b4fc0 msgr2=0x7f32440cd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:00.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.994+0000 7f324a19c700 1 --2- 172.21.15.92:0/1897985414 >> v2:172.21.15.169:3300/0 conn(0x7f32440b4fc0 0x7f32440cd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:00.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.994+0000 7f324a19c700 1 -- 172.21.15.92:0/1897985414 >> v1:172.21.15.169:6789/0 conn(0x7f32440d2df0 legacy=0x7f32440d31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:00.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.994+0000 7f324a19c700 1 -- 172.21.15.92:0/1897985414 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32440d3a50 con 0x7f32440b1c60 2023-11-28T11:21:00.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.994+0000 7f324a19c700 1 --2- 172.21.15.92:0/1897985414 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32440b1c60 0x7f32440cf010 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f323c009e50 tx=0x7f323c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6b8ffea3c726607 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:00.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.994+0000 7f3248999700 1 -- 172.21.15.92:0/1897985414 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323c00e070 con 0x7f32440b1c60 2023-11-28T11:21:00.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.994+0000 7f3248999700 1 -- 172.21.15.92:0/1897985414 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f323c00b910 con 0x7f32440b1c60 2023-11-28T11:21:00.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.994+0000 7f3248999700 1 -- 172.21.15.92:0/1897985414 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323c00bcc0 con 0x7f32440b1c60 2023-11-28T11:21:00.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.995+0000 7f324bbff700 1 -- 172.21.15.92:0/1897985414 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32440b1c60 msgr2=0x7f32440cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:00.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.995+0000 7f324bbff700 1 --2- 172.21.15.92:0/1897985414 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32440b1c60 0x7f32440cf010 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f323c009e50 tx=0x7f323c00b330 comp rx=0 tx=0).stop 2023-11-28T11:21:00.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.995+0000 7f324bbff700 1 -- 172.21.15.92:0/1897985414 shutdown_connections 2023-11-28T11:21:00.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.995+0000 7f324bbff700 1 --2- 172.21.15.92:0/1897985414 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32440b1c60 0x7f32440cf010 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:00.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.995+0000 7f324bbff700 1 --2- 172.21.15.92:0/1897985414 >> v2:172.21.15.169:3300/0 conn(0x7f32440b4fc0 0x7f32440cd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:00.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.995+0000 7f324bbff700 1 -- 172.21.15.92:0/1897985414 >> 172.21.15.92:0/1897985414 conn(0x7f32440719a0 msgr2=0x7f3244071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:00.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.995+0000 7f324bbff700 1 -- 172.21.15.92:0/1897985414 shutdown_connections 2023-11-28T11:21:00.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.995+0000 7f324bbff700 1 -- 172.21.15.92:0/1897985414 wait complete. 2023-11-28T11:21:01.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.997+0000 7f324bbff700 1 Processor -- start 2023-11-28T11:21:01.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.997+0000 7f324bbff700 1 -- start start 2023-11-28T11:21:01.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.998+0000 7f324bbff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32440b1c60 0x7f324415a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:01.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.998+0000 7f324bbff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32440b4fc0 0x7f32441e79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:01.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.998+0000 7f324bbff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32440d2df0 0x7f32440b68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:01.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.998+0000 7f324bbff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32441e7f00 con 0x7f32440b4fc0 2023-11-28T11:21:01.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.998+0000 7f324bbff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32441e7530 con 0x7f32440d2df0 2023-11-28T11:21:01.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.998+0000 7f324bbff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32441e6640 con 0x7f32440b1c60 2023-11-28T11:21:01.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.998+0000 7f324919a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32440b4fc0 0x7f32441e79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:01.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.998+0000 7f324919a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32440b4fc0 0x7f32441e79c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60322/0 (socket says 172.21.15.92:60322) 2023-11-28T11:21:01.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.998+0000 7f324919a700 1 -- 172.21.15.92:0/273947 learned_addr learned my addr 172.21.15.92:0/273947 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:01.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.998+0000 7f324a19c700 1 --2- 172.21.15.92:0/273947 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32440d2df0 0x7f32440b68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:01.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.998+0000 7f324999b700 1 --2- 172.21.15.92:0/273947 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32440b1c60 0x7f324415a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:01.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.998+0000 7f324919a700 1 -- 172.21.15.92:0/273947 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32440b1c60 msgr2=0x7f324415a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:01.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.998+0000 7f324919a700 1 --2- 172.21.15.92:0/273947 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32440b1c60 0x7f324415a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:01.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.998+0000 7f324919a700 1 -- 172.21.15.92:0/273947 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32440d2df0 msgr2=0x7f32440b68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:01.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.998+0000 7f324919a700 1 --2- 172.21.15.92:0/273947 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32440d2df0 0x7f32440b68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:01.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.998+0000 7f324919a700 1 -- 172.21.15.92:0/273947 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f323c009b30 con 0x7f32440b4fc0 2023-11-28T11:21:01.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.999+0000 7f324919a700 1 --2- 172.21.15.92:0/273947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32440b4fc0 0x7f32441e79c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f323400e870 tx=0x7f323400eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:01.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.999+0000 7f323a7fc700 1 -- 172.21.15.92:0/273947 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3234009590 con 0x7f32440b4fc0 2023-11-28T11:21:01.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.999+0000 7f323a7fc700 1 -- 172.21.15.92:0/273947 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3234013680 con 0x7f32440b4fc0 2023-11-28T11:21:01.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.999+0000 7f324bbff700 1 -- 172.21.15.92:0/273947 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32440b88f0 con 0x7f32440b4fc0 2023-11-28T11:21:01.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.999+0000 7f323a7fc700 1 -- 172.21.15.92:0/273947 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32340139f0 con 0x7f32440b4fc0 2023-11-28T11:21:01.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:00.999+0000 7f324bbff700 1 -- 172.21.15.92:0/273947 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32440d15e0 con 0x7f32440b4fc0 2023-11-28T11:21:01.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.000+0000 7f324bbff700 1 -- 172.21.15.92:0/273947 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32440b21b0 con 0x7f32440b4fc0 2023-11-28T11:21:01.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.001+0000 7f323a7fc700 1 -- 172.21.15.92:0/273947 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3234019030 con 0x7f32440b4fc0 2023-11-28T11:21:01.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.002+0000 7f323a7fc700 1 --2- 172.21.15.92:0/273947 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f32280682f0 0x7f322806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:01.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.002+0000 7f323a7fc700 1 -- 172.21.15.92:0/273947 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3234085d30 con 0x7f32440b4fc0 2023-11-28T11:21:01.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.003+0000 7f324999b700 1 --2- 172.21.15.92:0/273947 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f32280682f0 0x7f322806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:01.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.003+0000 7f324999b700 1 --2- 172.21.15.92:0/273947 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f32280682f0 0x7f322806a7a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3240007fd0 tx=0x7f3240007dc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:01.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.007+0000 7f323a7fc700 1 -- 172.21.15.92:0/273947 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f32340521b0 con 0x7f32440b4fc0 2023-11-28T11:21:01.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.237+0000 7f324bbff700 1 -- 172.21.15.92:0/273947 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f32441e76a0 con 0x7f32440b4fc0 2023-11-28T11:21:01.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.237+0000 7f323a7fc700 1 -- 172.21.15.92:0/273947 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4647 (secure 0 0 0) 0x7f32340557d0 con 0x7f32440b4fc0 2023-11-28T11:21:01.238 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:21:01.238 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/3257209990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3257209990},{"type":"v1","addr":"172.21.15.92:6836","nonce":3257209990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2193338949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2193338949},{"type":"v1","addr":"172.21.15.169:6834","nonce":2193338949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4308,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/471035619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":471035619},{"type":"v1","addr":"172.21.15.92:6837","nonce":471035619}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:20:56.234970+0000","modified":"2023-11-28T11:21:00.770991+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4326},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4326":{"gid":4326,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/2067251916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2067251916},{"type":"v1","addr":"172.21.15.169:6835","nonce":2067251916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-11-28T11:21:01.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.239+0000 7f324bbff700 1 -- 172.21.15.92:0/273947 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f32280682f0 msgr2=0x7f322806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:01.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.239+0000 7f324bbff700 1 --2- 172.21.15.92:0/273947 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f32280682f0 0x7f322806a7a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3240007fd0 tx=0x7f3240007dc0 comp rx=0 tx=0).stop 2023-11-28T11:21:01.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.239+0000 7f324bbff700 1 -- 172.21.15.92:0/273947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32440b4fc0 msgr2=0x7f32441e79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:01.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.239+0000 7f324bbff700 1 --2- 172.21.15.92:0/273947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32440b4fc0 0x7f32441e79c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f323400e870 tx=0x7f323400eb80 comp rx=0 tx=0).stop 2023-11-28T11:21:01.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.239+0000 7f324bbff700 1 -- 172.21.15.92:0/273947 shutdown_connections 2023-11-28T11:21:01.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.239+0000 7f324bbff700 1 --2- 172.21.15.92:0/273947 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32440d2df0 0x7f32440b68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:01.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.239+0000 7f324bbff700 1 --2- 172.21.15.92:0/273947 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32440b1c60 0x7f324415a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:01.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.239+0000 7f324bbff700 1 --2- 172.21.15.92:0/273947 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f32280682f0 0x7f322806a7a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:01.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.239+0000 7f324bbff700 1 --2- 172.21.15.92:0/273947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32440b4fc0 0x7f32441e79c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:01.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.239+0000 7f324bbff700 1 -- 172.21.15.92:0/273947 >> 172.21.15.92:0/273947 conn(0x7f32440719a0 msgr2=0x7f32440cbf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:01.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.240+0000 7f324bbff700 1 -- 172.21.15.92:0/273947 shutdown_connections 2023-11-28T11:21:01.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.240+0000 7f324bbff700 1 -- 172.21.15.92:0/273947 wait complete. 2023-11-28T11:21:01.242 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 7 2023-11-28T11:21:01.259 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:21:01.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.452+0000 7fc6a83ff700 1 Processor -- start 2023-11-28T11:21:01.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.452+0000 7fc6a83ff700 1 -- start start 2023-11-28T11:21:01.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.453+0000 7fc6a83ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6a00b8510 0x7fc6a006b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:01.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.453+0000 7fc6a83ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc6a00cd460 0x7fc6a006beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:01.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.454+0000 7fc6a83ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6a00b88e0 con 0x7fc6a006c3f0 2023-11-28T11:21:01.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.454+0000 7fc6a83ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6a00cdbd0 con 0x7fc6a00cd460 2023-11-28T11:21:01.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.454+0000 7fc6a83ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a006c830 con 0x7fc6a00b8510 2023-11-28T11:21:01.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.454+0000 7fc6a699c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc6a006c3f0 0x7fc6a015cc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46458/0 (socket says 172.21.15.92:46458) 2023-11-28T11:21:01.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.454+0000 7fc6a699c700 1 -- 172.21.15.92:0/1108070688 learned_addr learned my addr 172.21.15.92:0/1108070688 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:01.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.454+0000 7fc6a599a700 1 --2- 172.21.15.92:0/1108070688 >> v2:172.21.15.169:3300/0 conn(0x7fc6a00cd460 0x7fc6a006beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:01.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.454+0000 7fc6a619b700 1 --2- 172.21.15.92:0/1108070688 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6a00b8510 0x7fc6a006b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:01.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.454+0000 7fc6a5199700 1 -- 172.21.15.92:0/1108070688 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2944882523 0 0) 0x7fc6a00b88e0 con 0x7fc6a006c3f0 2023-11-28T11:21:01.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.454+0000 7fc6a5199700 1 -- 172.21.15.92:0/1108070688 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc68c0035b0 con 0x7fc6a006c3f0 2023-11-28T11:21:01.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.454+0000 7fc6a619b700 1 -- 172.21.15.92:0/1108070688 >> v2:172.21.15.169:3300/0 conn(0x7fc6a00cd460 msgr2=0x7fc6a006beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:01.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.454+0000 7fc6a619b700 1 --2- 172.21.15.92:0/1108070688 >> v2:172.21.15.169:3300/0 conn(0x7fc6a00cd460 0x7fc6a006beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:01.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.454+0000 7fc6a619b700 1 -- 172.21.15.92:0/1108070688 >> v1:172.21.15.92:6789/0 conn(0x7fc6a006c3f0 legacy=0x7fc6a015cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:01.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.455+0000 7fc6a5199700 1 -- 172.21.15.92:0/1108070688 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2582885942 0 0) 0x7fc68c0035b0 con 0x7fc6a006c3f0 2023-11-28T11:21:01.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.455+0000 7fc6a619b700 1 -- 172.21.15.92:0/1108070688 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a00cd8f0 con 0x7fc6a00b8510 2023-11-28T11:21:01.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.455+0000 7fc6a619b700 1 --2- 172.21.15.92:0/1108070688 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6a00b8510 0x7fc6a006b970 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc69c009840 tx=0x7fc69c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1b6c3b024fb087a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:01.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.455+0000 7fc6a5199700 1 -- 172.21.15.92:0/1108070688 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69c011830 con 0x7fc6a00b8510 2023-11-28T11:21:01.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.455+0000 7fc6a5199700 1 -- 172.21.15.92:0/1108070688 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc69c011990 con 0x7fc6a00b8510 2023-11-28T11:21:01.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.455+0000 7fc6a5199700 1 -- 172.21.15.92:0/1108070688 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69c011d20 con 0x7fc6a00b8510 2023-11-28T11:21:01.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.456+0000 7fc6a83ff700 1 -- 172.21.15.92:0/1108070688 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6a00b8510 msgr2=0x7fc6a006b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:01.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.456+0000 7fc6a83ff700 1 --2- 172.21.15.92:0/1108070688 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6a00b8510 0x7fc6a006b970 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc69c009840 tx=0x7fc69c009b50 comp rx=0 tx=0).stop 2023-11-28T11:21:01.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.456+0000 7fc6a83ff700 1 -- 172.21.15.92:0/1108070688 shutdown_connections 2023-11-28T11:21:01.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.456+0000 7fc6a83ff700 1 --2- 172.21.15.92:0/1108070688 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6a00b8510 0x7fc6a006b970 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:01.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.456+0000 7fc6a83ff700 1 --2- 172.21.15.92:0/1108070688 >> v2:172.21.15.169:3300/0 conn(0x7fc6a00cd460 0x7fc6a006beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:01.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.456+0000 7fc6a83ff700 1 -- 172.21.15.92:0/1108070688 >> 172.21.15.92:0/1108070688 conn(0x7fc6a00719a0 msgr2=0x7fc6a0071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:01.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.456+0000 7fc6a83ff700 1 -- 172.21.15.92:0/1108070688 shutdown_connections 2023-11-28T11:21:01.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.456+0000 7fc6a83ff700 1 -- 172.21.15.92:0/1108070688 wait complete. 2023-11-28T11:21:01.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.457+0000 7fc6a83ff700 1 Processor -- start 2023-11-28T11:21:01.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.458+0000 7fc6a83ff700 1 -- start start 2023-11-28T11:21:01.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.458+0000 7fc6a83ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6a006c3f0 0x7fc6a00c9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:01.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.459+0000 7fc6a83ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6a00b8510 0x7fc6a00c6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:01.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.459+0000 7fc6a83ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6a00cd460 0x7fc6a01593f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:01.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.459+0000 7fc6a83ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a00b1eb0 con 0x7fc6a00b8510 2023-11-28T11:21:01.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.459+0000 7fc6a83ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a0159930 con 0x7fc6a00cd460 2023-11-28T11:21:01.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.459+0000 7fc6a83ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a00b2100 con 0x7fc6a006c3f0 2023-11-28T11:21:01.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.459+0000 7fc6a599a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6a00b8510 0x7fc6a00c6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:01.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.459+0000 7fc6a599a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6a00b8510 0x7fc6a00c6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60324/0 (socket says 172.21.15.92:60324) 2023-11-28T11:21:01.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.459+0000 7fc6a599a700 1 -- 172.21.15.92:0/1380053803 learned_addr learned my addr 172.21.15.92:0/1380053803 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:01.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.459+0000 7fc6a619b700 1 --2- 172.21.15.92:0/1380053803 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6a006c3f0 0x7fc6a00c9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:01.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.459+0000 7fc6a699c700 1 --2- 172.21.15.92:0/1380053803 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6a00cd460 0x7fc6a01593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:01.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.459+0000 7fc6a599a700 1 -- 172.21.15.92:0/1380053803 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6a006c3f0 msgr2=0x7fc6a00c9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:01.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.459+0000 7fc6a599a700 1 --2- 172.21.15.92:0/1380053803 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6a006c3f0 0x7fc6a00c9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:01.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.459+0000 7fc6a599a700 1 -- 172.21.15.92:0/1380053803 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6a00cd460 msgr2=0x7fc6a01593f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:01.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.459+0000 7fc6a599a700 1 --2- 172.21.15.92:0/1380053803 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6a00cd460 0x7fc6a01593f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:01.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.459+0000 7fc6a599a700 1 -- 172.21.15.92:0/1380053803 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc69c0094a0 con 0x7fc6a00b8510 2023-11-28T11:21:01.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.460+0000 7fc6a599a700 1 --2- 172.21.15.92:0/1380053803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6a00b8510 0x7fc6a00c6940 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc69000e870 tx=0x7fc69000eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:01.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.460+0000 7fc696ffd700 1 -- 172.21.15.92:0/1380053803 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc690009590 con 0x7fc6a00b8510 2023-11-28T11:21:01.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.460+0000 7fc696ffd700 1 -- 172.21.15.92:0/1380053803 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc690013680 con 0x7fc6a00b8510 2023-11-28T11:21:01.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.460+0000 7fc6a83ff700 1 -- 172.21.15.92:0/1380053803 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6a00cfff0 con 0x7fc6a00b8510 2023-11-28T11:21:01.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.460+0000 7fc696ffd700 1 -- 172.21.15.92:0/1380053803 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6900139f0 con 0x7fc6a00b8510 2023-11-28T11:21:01.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.460+0000 7fc6a83ff700 1 -- 172.21.15.92:0/1380053803 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6a00cfd10 con 0x7fc6a00b8510 2023-11-28T11:21:01.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.461+0000 7fc6a83ff700 1 -- 172.21.15.92:0/1380053803 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6a00cbb70 con 0x7fc6a00b8510 2023-11-28T11:21:01.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.462+0000 7fc696ffd700 1 -- 172.21.15.92:0/1380053803 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc690019030 con 0x7fc6a00b8510 2023-11-28T11:21:01.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.462+0000 7fc696ffd700 1 --2- 172.21.15.92:0/1380053803 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc6840682f0 0x7fc68406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:01.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.463+0000 7fc696ffd700 1 -- 172.21.15.92:0/1380053803 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fc690085d70 con 0x7fc6a00b8510 2023-11-28T11:21:01.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.463+0000 7fc6a619b700 1 --2- 172.21.15.92:0/1380053803 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc6840682f0 0x7fc68406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:01.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.463+0000 7fc6a619b700 1 --2- 172.21.15.92:0/1380053803 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc6840682f0 0x7fc68406a7a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc69c0090b0 tx=0x7fc69c01c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:01.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.467+0000 7fc696ffd700 1 -- 172.21.15.92:0/1380053803 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc690052190 con 0x7fc6a00b8510 2023-11-28T11:21:01.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.658+0000 7fc6a83ff700 1 -- 172.21.15.92:0/1380053803 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc6a00cb0a0 con 0x7fc6a00b8510 2023-11-28T11:21:01.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.660+0000 7fc696ffd700 1 -- 172.21.15.92:0/1380053803 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+19748 (secure 0 0 0) 0x7fc6a00cb0a0 con 0x7fc6a00b8510 2023-11-28T11:21:01.661 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:21:01.661 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":27,"fsid":"68956e0c-c16f-47cd-a220-f98364b0831d","created":"2023-11-28T11:20:28.697573+0000","modified":"2023-11-28T11:21:00.770781+0000","last_up_change":"2023-11-28T11:20:41.257011+0000","last_in_change":"2023-11-28T11:20:32.249767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:20:41.925237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:20:48.682497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:20:54.471398+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:20:55.204585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:20:58.297307+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6abe483f-0e03-4a26-b9cf-62041291747e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":115823},{"type":"v1","addr":"172.21.15.92:6809","nonce":115823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":115823},{"type":"v1","addr":"172.21.15.92:6813","nonce":115823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":115823},{"type":"v1","addr":"172.21.15.92:6821","nonce":115823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":115823},{"type":"v1","addr":"172.21.15.92:6817","nonce":115823}]},"public_addr":"172.21.15.92:6809/115823","cluster_addr":"172.21.15.92:6813/115823","heartbeat_back_addr":"172.21.15.92:6821/115823","heartbeat_front_addr":"172.21.15.92:6817/115823","state":["exists","up"]},{"osd":1,"uuid":"13c1de73-1a8c-4d7e-8a28-9dae52dfe089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":115824},{"type":"v1","addr":"172.21.15.92:6812","nonce":115824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":115824},{"type":"v1","addr":"172.21.15.92:6816","nonce":115824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":115824},{"type":"v1","addr":"172.21.15.92:6823","nonce":115824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":115824},{"type":"v1","addr":"172.21.15.92:6820","nonce":115824}]},"public_addr":"172.21.15.92:6812/115824","cluster_addr":"172.21.15.92:6816/115824","heartbeat_back_addr":"172.21.15.92:6823/115824","heartbeat_front_addr":"172.21.15.92:6820/115824","state":["exists","up"]},{"osd":2,"uuid":"25788a55-112f-4f00-810d-2bc37cf76778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":115821},{"type":"v1","addr":"172.21.15.92:6801","nonce":115821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":115821},{"type":"v1","addr":"172.21.15.92:6803","nonce":115821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":115821},{"type":"v1","addr":"172.21.15.92:6807","nonce":115821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":115821},{"type":"v1","addr":"172.21.15.92:6805","nonce":115821}]},"public_addr":"172.21.15.92:6801/115821","cluster_addr":"172.21.15.92:6803/115821","heartbeat_back_addr":"172.21.15.92:6807/115821","heartbeat_front_addr":"172.21.15.92:6805/115821","state":["exists","up"]},{"osd":3,"uuid":"adf54e24-33a9-41fe-9ce3-98a5ab6c158e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":115825},{"type":"v1","addr":"172.21.15.92:6825","nonce":115825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":115825},{"type":"v1","addr":"172.21.15.92:6827","nonce":115825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":115825},{"type":"v1","addr":"172.21.15.92:6831","nonce":115825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":115825},{"type":"v1","addr":"172.21.15.92:6829","nonce":115825}]},"public_addr":"172.21.15.92:6825/115825","cluster_addr":"172.21.15.92:6827/115825","heartbeat_back_addr":"172.21.15.92:6831/115825","heartbeat_front_addr":"172.21.15.92:6829/115825","state":["exists","up"]},{"osd":4,"uuid":"ccb9b747-f399-44ea-9a18-9db3152227e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":116035},{"type":"v1","addr":"172.21.15.169:6817","nonce":116035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":116035},{"type":"v1","addr":"172.21.15.169:6819","nonce":116035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":116035},{"type":"v1","addr":"172.21.15.169:6826","nonce":116035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":116035},{"type":"v1","addr":"172.21.15.169:6822","nonce":116035}]},"public_addr":"172.21.15.169:6817/116035","cluster_addr":"172.21.15.169:6819/116035","heartbeat_back_addr":"172.21.15.169:6826/116035","heartbeat_front_addr":"172.21.15.169:6822/116035","state":["exists","up"]},{"osd":5,"uuid":"308a1e63-75aa-4d7d-8a7c-cecc24714fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":116033},{"type":"v1","addr":"172.21.15.169:6801","nonce":116033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":116033},{"type":"v1","addr":"172.21.15.169:6803","nonce":116033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":116033},{"type":"v1","addr":"172.21.15.169:6807","nonce":116033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":116033},{"type":"v1","addr":"172.21.15.169:6805","nonce":116033}]},"public_addr":"172.21.15.169:6801/116033","cluster_addr":"172.21.15.169:6803/116033","heartbeat_back_addr":"172.21.15.169:6807/116033","heartbeat_front_addr":"172.21.15.169:6805/116033","state":["exists","up"]},{"osd":6,"uuid":"9c459b7b-b821-4b4b-b91e-6aaae271457c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":116034},{"type":"v1","addr":"172.21.15.169:6809","nonce":116034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":116034},{"type":"v1","addr":"172.21.15.169:6811","nonce":116034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":116034},{"type":"v1","addr":"172.21.15.169:6815","nonce":116034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":116034},{"type":"v1","addr":"172.21.15.169:6813","nonce":116034}]},"public_addr":"172.21.15.169:6809/116034","cluster_addr":"172.21.15.169:6811/116034","heartbeat_back_addr":"172.21.15.169:6815/116034","heartbeat_front_addr":"172.21.15.169:6813/116034","state":["exists","up"]},{"osd":7,"uuid":"1ca9aa4a-755d-4e28-a937-4d8dccab4df3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":116044},{"type":"v1","addr":"172.21.15.169:6823","nonce":116044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6825","nonce":116044},{"type":"v1","addr":"172.21.15.169:6827","nonce":116044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":116044},{"type":"v1","addr":"172.21.15.169:6831","nonce":116044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":116044},{"type":"v1","addr":"172.21.15.169:6829","nonce":116044}]},"public_addr":"172.21.15.169:6823/116044","cluster_addr":"172.21.15.169:6827/116044","heartbeat_back_addr":"172.21.15.169:6831/116044","heartbeat_front_addr":"172.21.15.169:6829/116044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.586781+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.869778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.508594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.860493+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.872267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.746933+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.611520+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.871446+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:21:01.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.662+0000 7fc6a83ff700 1 -- 172.21.15.92:0/1380053803 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc6840682f0 msgr2=0x7fc68406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:01.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.662+0000 7fc6a83ff700 1 --2- 172.21.15.92:0/1380053803 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc6840682f0 0x7fc68406a7a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc69c0090b0 tx=0x7fc69c01c040 comp rx=0 tx=0).stop 2023-11-28T11:21:01.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.662+0000 7fc6a83ff700 1 -- 172.21.15.92:0/1380053803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6a00b8510 msgr2=0x7fc6a00c6940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:01.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.662+0000 7fc6a83ff700 1 --2- 172.21.15.92:0/1380053803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6a00b8510 0x7fc6a00c6940 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc69000e870 tx=0x7fc69000eb80 comp rx=0 tx=0).stop 2023-11-28T11:21:01.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.662+0000 7fc6a83ff700 1 -- 172.21.15.92:0/1380053803 shutdown_connections 2023-11-28T11:21:01.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.662+0000 7fc6a83ff700 1 --2- 172.21.15.92:0/1380053803 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6a00cd460 0x7fc6a01593f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:01.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.662+0000 7fc6a83ff700 1 --2- 172.21.15.92:0/1380053803 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6a006c3f0 0x7fc6a00c9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:01.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.662+0000 7fc6a83ff700 1 --2- 172.21.15.92:0/1380053803 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc6840682f0 0x7fc68406a7a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:01.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.662+0000 7fc6a83ff700 1 --2- 172.21.15.92:0/1380053803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6a00b8510 0x7fc6a00c6940 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:01.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.662+0000 7fc6a83ff700 1 -- 172.21.15.92:0/1380053803 >> 172.21.15.92:0/1380053803 conn(0x7fc6a00719a0 msgr2=0x7fc6a01e7110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:01.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.662+0000 7fc6a83ff700 1 -- 172.21.15.92:0/1380053803 shutdown_connections 2023-11-28T11:21:01.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.662+0000 7fc6a83ff700 1 -- 172.21.15.92:0/1380053803 wait complete. 2023-11-28T11:21:01.682 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:21:01.728 INFO:tasks.ceph.mds.b.smithi169.stderr:2023-11-28T11:21:01.727+0000 7ff7b7ff1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T11:21:01.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.878+0000 7f3f379ff700 1 Processor -- start 2023-11-28T11:21:01.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.878+0000 7f3f379ff700 1 -- start start 2023-11-28T11:21:01.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.880+0000 7f3f379ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3f300b8510 0x7f3f3006b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:01.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.880+0000 7f3f379ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3f300d0d60 0x7f3f300d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:01.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.880+0000 7f3f379ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f300b88e0 con 0x7f3f300cd460 2023-11-28T11:21:01.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.880+0000 7f3f379ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f300cdbd0 con 0x7f3f300b8510 2023-11-28T11:21:01.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.880+0000 7f3f379ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f300d02d0 con 0x7f3f300d0d60 2023-11-28T11:21:01.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.880+0000 7f3f3579b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3f300b8510 0x7f3f3006b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:01.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.880+0000 7f3f3579b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3f300b8510 0x7f3f3006b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60334/0 (socket says 172.21.15.92:60334) 2023-11-28T11:21:01.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.880+0000 7f3f3579b700 1 -- 172.21.15.92:0/3875512881 learned_addr learned my addr 172.21.15.92:0/3875512881 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:01.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.880+0000 7f3f35f9c700 1 --2- 172.21.15.92:0/3875512881 >> v2:172.21.15.169:3300/0 conn(0x7f3f300d0d60 0x7f3f300d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:01.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.881+0000 7f3f3579b700 1 -- 172.21.15.92:0/3875512881 >> v2:172.21.15.169:3300/0 conn(0x7f3f300d0d60 msgr2=0x7f3f300d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:01.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.881+0000 7f3f3579b700 1 --2- 172.21.15.92:0/3875512881 >> v2:172.21.15.169:3300/0 conn(0x7f3f300d0d60 0x7f3f300d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:01.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.881+0000 7f3f3579b700 1 -- 172.21.15.92:0/3875512881 >> v1:172.21.15.169:6789/0 conn(0x7f3f300cd460 legacy=0x7f3f3006beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:01.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.881+0000 7f3f27fff700 1 -- 172.21.15.92:0/3875512881 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1538958370 0 0) 0x7f3f300b88e0 con 0x7f3f300cd460 2023-11-28T11:21:01.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.881+0000 7f3f3579b700 1 -- 172.21.15.92:0/3875512881 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f300cd8f0 con 0x7f3f300b8510 2023-11-28T11:21:01.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.881+0000 7f3f3579b700 1 --2- 172.21.15.92:0/3875512881 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f300b8510 0x7f3f3006b970 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3f300ce8c0 tx=0x7f3f2c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d30ad7028d2bd730 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:01.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.881+0000 7f3f27fff700 1 -- 172.21.15.92:0/3875512881 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f2c011830 con 0x7f3f300b8510 2023-11-28T11:21:01.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.882+0000 7f3f27fff700 1 -- 172.21.15.92:0/3875512881 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f2c011990 con 0x7f3f300b8510 2023-11-28T11:21:01.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.882+0000 7f3f27fff700 1 -- 172.21.15.92:0/3875512881 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f2c011d20 con 0x7f3f300b8510 2023-11-28T11:21:01.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.882+0000 7f3f379ff700 1 -- 172.21.15.92:0/3875512881 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f300b8510 msgr2=0x7f3f3006b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:01.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.882+0000 7f3f379ff700 1 --2- 172.21.15.92:0/3875512881 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f300b8510 0x7f3f3006b970 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3f300ce8c0 tx=0x7f3f2c009c00 comp rx=0 tx=0).stop 2023-11-28T11:21:01.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.884+0000 7f3f379ff700 1 -- 172.21.15.92:0/3875512881 shutdown_connections 2023-11-28T11:21:01.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.884+0000 7f3f379ff700 1 --2- 172.21.15.92:0/3875512881 >> v2:172.21.15.169:3300/0 conn(0x7f3f300d0d60 0x7f3f300d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:01.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.884+0000 7f3f379ff700 1 --2- 172.21.15.92:0/3875512881 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f300b8510 0x7f3f3006b970 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:01.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.884+0000 7f3f379ff700 1 -- 172.21.15.92:0/3875512881 >> 172.21.15.92:0/3875512881 conn(0x7f3f300719a0 msgr2=0x7f3f30071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:01.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.884+0000 7f3f379ff700 1 -- 172.21.15.92:0/3875512881 shutdown_connections 2023-11-28T11:21:01.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.884+0000 7f3f379ff700 1 -- 172.21.15.92:0/3875512881 wait complete. 2023-11-28T11:21:01.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.886+0000 7f3f379ff700 1 Processor -- start 2023-11-28T11:21:01.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.886+0000 7f3f379ff700 1 -- start start 2023-11-28T11:21:01.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.887+0000 7f3f379ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f300b8510 0x7f3f30159070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:01.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.887+0000 7f3f379ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3f300cd460 0x7f3f301e5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:01.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.887+0000 7f3f379ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f300d0d60 0x7f3f300b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:01.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.887+0000 7f3f379ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f301595b0 con 0x7f3f300d0d60 2023-11-28T11:21:01.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.887+0000 7f3f379ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f301e5940 con 0x7f3f300b8510 2023-11-28T11:21:01.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.887+0000 7f3f379ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f301e5c20 con 0x7f3f300cd460 2023-11-28T11:21:01.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.887+0000 7f3f35f9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f300d0d60 0x7f3f300b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:01.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.887+0000 7f3f35f9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f300d0d60 0x7f3f300b31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60350/0 (socket says 172.21.15.92:60350) 2023-11-28T11:21:01.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.887+0000 7f3f35f9c700 1 -- 172.21.15.92:0/159265747 learned_addr learned my addr 172.21.15.92:0/159265747 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:01.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.887+0000 7f3f3579b700 1 --2- 172.21.15.92:0/159265747 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f300b8510 0x7f3f30159070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:01.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.887+0000 7f3f34f9a700 1 --2- 172.21.15.92:0/159265747 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3f300cd460 0x7f3f301e5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:01.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.888+0000 7f3f35f9c700 1 -- 172.21.15.92:0/159265747 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3f300cd460 msgr2=0x7f3f301e5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:01.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.888+0000 7f3f35f9c700 1 --2- 172.21.15.92:0/159265747 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3f300cd460 0x7f3f301e5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:01.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.888+0000 7f3f35f9c700 1 -- 172.21.15.92:0/159265747 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f300b8510 msgr2=0x7f3f30159070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:01.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.888+0000 7f3f35f9c700 1 --2- 172.21.15.92:0/159265747 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f300b8510 0x7f3f30159070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:01.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.888+0000 7f3f35f9c700 1 -- 172.21.15.92:0/159265747 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f2c009550 con 0x7f3f300d0d60 2023-11-28T11:21:01.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.888+0000 7f3f34f9a700 1 --2- 172.21.15.92:0/159265747 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3f300cd460 0x7f3f301e5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:21:01.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.888+0000 7f3f35f9c700 1 --2- 172.21.15.92:0/159265747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f300d0d60 0x7f3f300b31f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3f2800efc0 tx=0x7f3f2800c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:01.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.888+0000 7f3f25ffb700 1 -- 172.21.15.92:0/159265747 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f2800cb80 con 0x7f3f300d0d60 2023-11-28T11:21:01.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.888+0000 7f3f25ffb700 1 -- 172.21.15.92:0/159265747 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f2800f040 con 0x7f3f300d0d60 2023-11-28T11:21:01.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.888+0000 7f3f379ff700 1 -- 172.21.15.92:0/159265747 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f300cfff0 con 0x7f3f300d0d60 2023-11-28T11:21:01.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.888+0000 7f3f25ffb700 1 -- 172.21.15.92:0/159265747 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f28014480 con 0x7f3f300d0d60 2023-11-28T11:21:01.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.888+0000 7f3f379ff700 1 -- 172.21.15.92:0/159265747 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f300cfd10 con 0x7f3f300d0d60 2023-11-28T11:21:01.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.889+0000 7f3f379ff700 1 -- 172.21.15.92:0/159265747 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f301e6550 con 0x7f3f300d0d60 2023-11-28T11:21:01.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.890+0000 7f3f25ffb700 1 -- 172.21.15.92:0/159265747 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3f280145e0 con 0x7f3f300d0d60 2023-11-28T11:21:01.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.891+0000 7f3f25ffb700 1 --2- 172.21.15.92:0/159265747 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3f140682f0 0x7f3f1406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:01.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.891+0000 7f3f25ffb700 1 -- 172.21.15.92:0/159265747 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3f28086c70 con 0x7f3f300d0d60 2023-11-28T11:21:01.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.891+0000 7f3f3579b700 1 --2- 172.21.15.92:0/159265747 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3f140682f0 0x7f3f1406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:01.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.892+0000 7f3f3579b700 1 --2- 172.21.15.92:0/159265747 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3f140682f0 0x7f3f1406a7a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c009870 tx=0x7f3f2c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:01.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:01.895+0000 7f3f25ffb700 1 -- 172.21.15.92:0/159265747 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3f280530f0 con 0x7f3f300d0d60 2023-11-28T11:21:02.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.092+0000 7f3f379ff700 1 -- 172.21.15.92:0/159265747 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3f301e32a0 con 0x7f3f300d0d60 2023-11-28T11:21:02.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.093+0000 7f3f25ffb700 1 -- 172.21.15.92:0/159265747 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+19748 (secure 0 0 0) 0x7f3f301e32a0 con 0x7f3f300d0d60 2023-11-28T11:21:02.094 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:21:02.094 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":27,"fsid":"68956e0c-c16f-47cd-a220-f98364b0831d","created":"2023-11-28T11:20:28.697573+0000","modified":"2023-11-28T11:21:00.770781+0000","last_up_change":"2023-11-28T11:20:41.257011+0000","last_in_change":"2023-11-28T11:20:32.249767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:20:41.925237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:20:48.682497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:20:54.471398+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:20:55.204585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:20:58.297307+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6abe483f-0e03-4a26-b9cf-62041291747e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":115823},{"type":"v1","addr":"172.21.15.92:6809","nonce":115823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":115823},{"type":"v1","addr":"172.21.15.92:6813","nonce":115823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":115823},{"type":"v1","addr":"172.21.15.92:6821","nonce":115823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":115823},{"type":"v1","addr":"172.21.15.92:6817","nonce":115823}]},"public_addr":"172.21.15.92:6809/115823","cluster_addr":"172.21.15.92:6813/115823","heartbeat_back_addr":"172.21.15.92:6821/115823","heartbeat_front_addr":"172.21.15.92:6817/115823","state":["exists","up"]},{"osd":1,"uuid":"13c1de73-1a8c-4d7e-8a28-9dae52dfe089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":115824},{"type":"v1","addr":"172.21.15.92:6812","nonce":115824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":115824},{"type":"v1","addr":"172.21.15.92:6816","nonce":115824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":115824},{"type":"v1","addr":"172.21.15.92:6823","nonce":115824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":115824},{"type":"v1","addr":"172.21.15.92:6820","nonce":115824}]},"public_addr":"172.21.15.92:6812/115824","cluster_addr":"172.21.15.92:6816/115824","heartbeat_back_addr":"172.21.15.92:6823/115824","heartbeat_front_addr":"172.21.15.92:6820/115824","state":["exists","up"]},{"osd":2,"uuid":"25788a55-112f-4f00-810d-2bc37cf76778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":115821},{"type":"v1","addr":"172.21.15.92:6801","nonce":115821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":115821},{"type":"v1","addr":"172.21.15.92:6803","nonce":115821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":115821},{"type":"v1","addr":"172.21.15.92:6807","nonce":115821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":115821},{"type":"v1","addr":"172.21.15.92:6805","nonce":115821}]},"public_addr":"172.21.15.92:6801/115821","cluster_addr":"172.21.15.92:6803/115821","heartbeat_back_addr":"172.21.15.92:6807/115821","heartbeat_front_addr":"172.21.15.92:6805/115821","state":["exists","up"]},{"osd":3,"uuid":"adf54e24-33a9-41fe-9ce3-98a5ab6c158e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":115825},{"type":"v1","addr":"172.21.15.92:6825","nonce":115825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":115825},{"type":"v1","addr":"172.21.15.92:6827","nonce":115825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":115825},{"type":"v1","addr":"172.21.15.92:6831","nonce":115825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":115825},{"type":"v1","addr":"172.21.15.92:6829","nonce":115825}]},"public_addr":"172.21.15.92:6825/115825","cluster_addr":"172.21.15.92:6827/115825","heartbeat_back_addr":"172.21.15.92:6831/115825","heartbeat_front_addr":"172.21.15.92:6829/115825","state":["exists","up"]},{"osd":4,"uuid":"ccb9b747-f399-44ea-9a18-9db3152227e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":116035},{"type":"v1","addr":"172.21.15.169:6817","nonce":116035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":116035},{"type":"v1","addr":"172.21.15.169:6819","nonce":116035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":116035},{"type":"v1","addr":"172.21.15.169:6826","nonce":116035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":116035},{"type":"v1","addr":"172.21.15.169:6822","nonce":116035}]},"public_addr":"172.21.15.169:6817/116035","cluster_addr":"172.21.15.169:6819/116035","heartbeat_back_addr":"172.21.15.169:6826/116035","heartbeat_front_addr":"172.21.15.169:6822/116035","state":["exists","up"]},{"osd":5,"uuid":"308a1e63-75aa-4d7d-8a7c-cecc24714fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":116033},{"type":"v1","addr":"172.21.15.169:6801","nonce":116033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":116033},{"type":"v1","addr":"172.21.15.169:6803","nonce":116033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":116033},{"type":"v1","addr":"172.21.15.169:6807","nonce":116033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":116033},{"type":"v1","addr":"172.21.15.169:6805","nonce":116033}]},"public_addr":"172.21.15.169:6801/116033","cluster_addr":"172.21.15.169:6803/116033","heartbeat_back_addr":"172.21.15.169:6807/116033","heartbeat_front_addr":"172.21.15.169:6805/116033","state":["exists","up"]},{"osd":6,"uuid":"9c459b7b-b821-4b4b-b91e-6aaae271457c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":116034},{"type":"v1","addr":"172.21.15.169:6809","nonce":116034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":116034},{"type":"v1","addr":"172.21.15.169:6811","nonce":116034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":116034},{"type":"v1","addr":"172.21.15.169:6815","nonce":116034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":116034},{"type":"v1","addr":"172.21.15.169:6813","nonce":116034}]},"public_addr":"172.21.15.169:6809/116034","cluster_addr":"172.21.15.169:6811/116034","heartbeat_back_addr":"172.21.15.169:6815/116034","heartbeat_front_addr":"172.21.15.169:6813/116034","state":["exists","up"]},{"osd":7,"uuid":"1ca9aa4a-755d-4e28-a937-4d8dccab4df3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":116044},{"type":"v1","addr":"172.21.15.169:6823","nonce":116044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6825","nonce":116044},{"type":"v1","addr":"172.21.15.169:6827","nonce":116044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":116044},{"type":"v1","addr":"172.21.15.169:6831","nonce":116044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":116044},{"type":"v1","addr":"172.21.15.169:6829","nonce":116044}]},"public_addr":"172.21.15.169:6823/116044","cluster_addr":"172.21.15.169:6827/116044","heartbeat_back_addr":"172.21.15.169:6831/116044","heartbeat_front_addr":"172.21.15.169:6829/116044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.586781+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.869778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.508594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.860493+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.872267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.746933+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.611520+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.871446+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:21:02.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.095+0000 7f3f379ff700 1 -- 172.21.15.92:0/159265747 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3f140682f0 msgr2=0x7f3f1406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:02.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.095+0000 7f3f379ff700 1 --2- 172.21.15.92:0/159265747 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3f140682f0 0x7f3f1406a7a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c009870 tx=0x7f3f2c008030 comp rx=0 tx=0).stop 2023-11-28T11:21:02.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.095+0000 7f3f379ff700 1 -- 172.21.15.92:0/159265747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f300d0d60 msgr2=0x7f3f300b31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:02.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.095+0000 7f3f379ff700 1 --2- 172.21.15.92:0/159265747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f300d0d60 0x7f3f300b31f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3f2800efc0 tx=0x7f3f2800c0c0 comp rx=0 tx=0).stop 2023-11-28T11:21:02.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.096+0000 7f3f379ff700 1 -- 172.21.15.92:0/159265747 shutdown_connections 2023-11-28T11:21:02.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.096+0000 7f3f379ff700 1 --2- 172.21.15.92:0/159265747 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3f140682f0 0x7f3f1406a7a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:02.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.096+0000 7f3f379ff700 1 --2- 172.21.15.92:0/159265747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f300d0d60 0x7f3f300b31f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:02.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.096+0000 7f3f379ff700 1 --2- 172.21.15.92:0/159265747 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f300b8510 0x7f3f30159070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:02.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.096+0000 7f3f379ff700 1 --2- 172.21.15.92:0/159265747 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3f300cd460 0x7f3f301e5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:02.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.096+0000 7f3f379ff700 1 -- 172.21.15.92:0/159265747 >> 172.21.15.92:0/159265747 conn(0x7f3f300719a0 msgr2=0x7f3f300b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:02.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.096+0000 7f3f379ff700 1 -- 172.21.15.92:0/159265747 shutdown_connections 2023-11-28T11:21:02.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.096+0000 7f3f379ff700 1 -- 172.21.15.92:0/159265747 wait complete. 2023-11-28T11:21:02.128 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:21:02.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.324+0000 7fb1dd9ff700 1 Processor -- start 2023-11-28T11:21:02.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.325+0000 7fb1dd9ff700 1 -- start start 2023-11-28T11:21:02.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.326+0000 7fb1dd9ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb1d80d2df0 0x7fb1d80cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:02.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.326+0000 7fb1dd9ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1d80d1950 con 0x7fb1d80b4fc0 2023-11-28T11:21:02.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.326+0000 7fb1dd9ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1d80b2500 con 0x7fb1d80b1c60 2023-11-28T11:21:02.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.326+0000 7fb1dd9ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1d80d3d30 con 0x7fb1d80d2df0 2023-11-28T11:21:02.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.326+0000 7fb1d67fc700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb1d80b4fc0 0x7fb1d80d31c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46470/0 (socket says 172.21.15.92:46470) 2023-11-28T11:21:02.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.326+0000 7fb1d67fc700 1 -- 172.21.15.92:0/637104925 learned_addr learned my addr 172.21.15.92:0/637104925 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:02.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.327+0000 7fb1d6ffd700 1 --2- 172.21.15.92:0/637104925 >> v2:172.21.15.169:3300/0 conn(0x7fb1d80d2df0 0x7fb1d80cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:02.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.327+0000 7fb1d5ffb700 1 -- 172.21.15.92:0/637104925 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1817882984 0 0) 0x7fb1d80d1950 con 0x7fb1d80b4fc0 2023-11-28T11:21:02.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.327+0000 7fb1d5ffb700 1 -- 172.21.15.92:0/637104925 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1c40035b0 con 0x7fb1d80b4fc0 2023-11-28T11:21:02.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.327+0000 7fb1d5ffb700 1 -- 172.21.15.92:0/637104925 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3126050303 0 0) 0x7fb1d80b2500 con 0x7fb1d80b1c60 2023-11-28T11:21:02.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.327+0000 7fb1d5ffb700 1 -- 172.21.15.92:0/637104925 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1d80d1950 con 0x7fb1d80b1c60 2023-11-28T11:21:02.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.327+0000 7fb1d5ffb700 1 -- 172.21.15.92:0/637104925 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3115767425 0 0) 0x7fb1c40035b0 con 0x7fb1d80b4fc0 2023-11-28T11:21:02.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.327+0000 7fb1d6ffd700 1 -- 172.21.15.92:0/637104925 >> v1:172.21.15.169:6789/0 conn(0x7fb1d80b1c60 legacy=0x7fb1d80cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:02.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.327+0000 7fb1d6ffd700 1 -- 172.21.15.92:0/637104925 >> v1:172.21.15.92:6789/0 conn(0x7fb1d80b4fc0 legacy=0x7fb1d80d31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:02.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.327+0000 7fb1d6ffd700 1 -- 172.21.15.92:0/637104925 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d80d3a50 con 0x7fb1d80d2df0 2023-11-28T11:21:02.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.328+0000 7fb1d6ffd700 1 --2- 172.21.15.92:0/637104925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1d80d2df0 0x7fb1d80cd620 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8009840 tx=0x7fb1c8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2649a486c0b2b2b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:02.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.328+0000 7fb1d5ffb700 1 -- 172.21.15.92:0/637104925 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c0001670 con 0x7fb1d80d2df0 2023-11-28T11:21:02.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.328+0000 7fb1d5ffb700 1 -- 172.21.15.92:0/637104925 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1c8004030 con 0x7fb1d80d2df0 2023-11-28T11:21:02.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.328+0000 7fb1d5ffb700 1 -- 172.21.15.92:0/637104925 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c80043c0 con 0x7fb1d80d2df0 2023-11-28T11:21:02.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.328+0000 7fb1dd9ff700 1 -- 172.21.15.92:0/637104925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1d80d2df0 msgr2=0x7fb1d80cd620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:02.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.328+0000 7fb1dd9ff700 1 --2- 172.21.15.92:0/637104925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1d80d2df0 0x7fb1d80cd620 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8009840 tx=0x7fb1c8009b50 comp rx=0 tx=0).stop 2023-11-28T11:21:02.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.328+0000 7fb1dd9ff700 1 -- 172.21.15.92:0/637104925 shutdown_connections 2023-11-28T11:21:02.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.328+0000 7fb1dd9ff700 1 --2- 172.21.15.92:0/637104925 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1d80d2df0 0x7fb1d80cd620 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:02.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.328+0000 7fb1dd9ff700 1 -- 172.21.15.92:0/637104925 >> 172.21.15.92:0/637104925 conn(0x7fb1d80719a0 msgr2=0x7fb1d8071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:02.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.328+0000 7fb1dd9ff700 1 -- 172.21.15.92:0/637104925 shutdown_connections 2023-11-28T11:21:02.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.329+0000 7fb1dd9ff700 1 -- 172.21.15.92:0/637104925 wait complete. 2023-11-28T11:21:02.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.330+0000 7fb1dd9ff700 1 Processor -- start 2023-11-28T11:21:02.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.330+0000 7fb1dd9ff700 1 -- start start 2023-11-28T11:21:02.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.331+0000 7fb1dd9ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1d80b1c60 0x7fb1d80a9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:02.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.331+0000 7fb1dd9ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1d80b4fc0 0x7fb1d80b0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:02.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.331+0000 7fb1dd9ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1d80d2df0 0x7fb1d80b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:02.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.331+0000 7fb1dd9ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d80cc9d0 con 0x7fb1d80b4fc0 2023-11-28T11:21:02.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.331+0000 7fb1dd9ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d814aac0 con 0x7fb1d80b1c60 2023-11-28T11:21:02.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.331+0000 7fb1dd9ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d80b1820 con 0x7fb1d80d2df0 2023-11-28T11:21:02.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.331+0000 7fb1d67fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1d80b4fc0 0x7fb1d80b0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:02.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.332+0000 7fb1d67fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1d80b4fc0 0x7fb1d80b0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60364/0 (socket says 172.21.15.92:60364) 2023-11-28T11:21:02.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.332+0000 7fb1d67fc700 1 -- 172.21.15.92:0/56606257 learned_addr learned my addr 172.21.15.92:0/56606257 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:02.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.332+0000 7fb1d6ffd700 1 --2- 172.21.15.92:0/56606257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1d80b1c60 0x7fb1d80a9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:02.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.332+0000 7fb1d77fe700 1 --2- 172.21.15.92:0/56606257 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1d80d2df0 0x7fb1d80b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:02.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.332+0000 7fb1d67fc700 1 -- 172.21.15.92:0/56606257 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1d80d2df0 msgr2=0x7fb1d80b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:02.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.332+0000 7fb1d67fc700 1 --2- 172.21.15.92:0/56606257 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1d80d2df0 0x7fb1d80b12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:02.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.332+0000 7fb1d67fc700 1 -- 172.21.15.92:0/56606257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1d80b1c60 msgr2=0x7fb1d80a9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:02.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.332+0000 7fb1d67fc700 1 --2- 172.21.15.92:0/56606257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1d80b1c60 0x7fb1d80a9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:02.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.332+0000 7fb1d67fc700 1 -- 172.21.15.92:0/56606257 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c80094a0 con 0x7fb1d80b4fc0 2023-11-28T11:21:02.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.332+0000 7fb1d77fe700 1 --2- 172.21.15.92:0/56606257 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1d80d2df0 0x7fb1d80b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:21:02.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.332+0000 7fb1d67fc700 1 --2- 172.21.15.92:0/56606257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1d80b4fc0 0x7fb1d80b0da0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb1c000c550 tx=0x7fb1c000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:02.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.332+0000 7fb1cffff700 1 -- 172.21.15.92:0/56606257 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c0004c80 con 0x7fb1d80b4fc0 2023-11-28T11:21:02.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.332+0000 7fb1cffff700 1 -- 172.21.15.92:0/56606257 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1c0004de0 con 0x7fb1d80b4fc0 2023-11-28T11:21:02.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.332+0000 7fb1cffff700 1 -- 172.21.15.92:0/56606257 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c0007310 con 0x7fb1d80b4fc0 2023-11-28T11:21:02.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.332+0000 7fb1dd9ff700 1 -- 172.21.15.92:0/56606257 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1d80b88f0 con 0x7fb1d80b4fc0 2023-11-28T11:21:02.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.332+0000 7fb1dd9ff700 1 -- 172.21.15.92:0/56606257 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1d80d15e0 con 0x7fb1d80b4fc0 2023-11-28T11:21:02.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.334+0000 7fb1dd9ff700 1 -- 172.21.15.92:0/56606257 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1d80b21b0 con 0x7fb1d80b4fc0 2023-11-28T11:21:02.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.334+0000 7fb1cffff700 1 -- 172.21.15.92:0/56606257 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb1c001a020 con 0x7fb1d80b4fc0 2023-11-28T11:21:02.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.335+0000 7fb1cffff700 1 --2- 172.21.15.92:0/56606257 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb1b80682f0 0x7fb1b806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:02.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.335+0000 7fb1cffff700 1 -- 172.21.15.92:0/56606257 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb1c0085ac0 con 0x7fb1d80b4fc0 2023-11-28T11:21:02.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.335+0000 7fb1d6ffd700 1 --2- 172.21.15.92:0/56606257 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb1b80682f0 0x7fb1b806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:02.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.336+0000 7fb1d6ffd700 1 --2- 172.21.15.92:0/56606257 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb1b80682f0 0x7fb1b806a7a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb1c80090b0 tx=0x7fb1c8008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:02.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.339+0000 7fb1cffff700 1 -- 172.21.15.92:0/56606257 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb1c0052030 con 0x7fb1d80b4fc0 2023-11-28T11:21:02.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.568+0000 7fb1dd9ff700 1 -- 172.21.15.92:0/56606257 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb1d80d25f0 con 0x7fb1d80b4fc0 2023-11-28T11:21:02.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.568+0000 7fb1cffff700 1 -- 172.21.15.92:0/56606257 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4647 (secure 0 0 0) 0x7fb1c0055650 con 0x7fb1d80b4fc0 2023-11-28T11:21:02.569 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:21:02.569 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/3257209990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3257209990},{"type":"v1","addr":"172.21.15.92:6836","nonce":3257209990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2193338949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2193338949},{"type":"v1","addr":"172.21.15.169:6834","nonce":2193338949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4308,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/471035619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":471035619},{"type":"v1","addr":"172.21.15.92:6837","nonce":471035619}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:20:56.234970+0000","modified":"2023-11-28T11:21:00.770991+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4326},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4326":{"gid":4326,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/2067251916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2067251916},{"type":"v1","addr":"172.21.15.169:6835","nonce":2067251916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-11-28T11:21:02.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.571+0000 7fb1dd9ff700 1 -- 172.21.15.92:0/56606257 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb1b80682f0 msgr2=0x7fb1b806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:02.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.571+0000 7fb1dd9ff700 1 --2- 172.21.15.92:0/56606257 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb1b80682f0 0x7fb1b806a7a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb1c80090b0 tx=0x7fb1c8008030 comp rx=0 tx=0).stop 2023-11-28T11:21:02.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.571+0000 7fb1dd9ff700 1 -- 172.21.15.92:0/56606257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1d80b4fc0 msgr2=0x7fb1d80b0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:02.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.571+0000 7fb1dd9ff700 1 --2- 172.21.15.92:0/56606257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1d80b4fc0 0x7fb1d80b0da0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb1c000c550 tx=0x7fb1c000c860 comp rx=0 tx=0).stop 2023-11-28T11:21:02.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.571+0000 7fb1dd9ff700 1 -- 172.21.15.92:0/56606257 shutdown_connections 2023-11-28T11:21:02.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.571+0000 7fb1dd9ff700 1 --2- 172.21.15.92:0/56606257 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1d80d2df0 0x7fb1d80b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:02.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.571+0000 7fb1dd9ff700 1 --2- 172.21.15.92:0/56606257 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1d80b1c60 0x7fb1d80a9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:02.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.571+0000 7fb1dd9ff700 1 --2- 172.21.15.92:0/56606257 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb1b80682f0 0x7fb1b806a7a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:02.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.571+0000 7fb1dd9ff700 1 --2- 172.21.15.92:0/56606257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1d80b4fc0 0x7fb1d80b0da0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:02.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.571+0000 7fb1dd9ff700 1 -- 172.21.15.92:0/56606257 >> 172.21.15.92:0/56606257 conn(0x7fb1d80719a0 msgr2=0x7fb1d80b3010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:02.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.571+0000 7fb1dd9ff700 1 -- 172.21.15.92:0/56606257 shutdown_connections 2023-11-28T11:21:02.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:02.571+0000 7fb1dd9ff700 1 -- 172.21.15.92:0/56606257 wait complete. 2023-11-28T11:21:02.573 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 7 2023-11-28T11:21:02.992 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T11:20:56.234970+0000', 'modified': '2023-11-28T11:21:00.770991+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4326}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4326': {'gid': 4326, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.169:6835/2067251916', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6833', 'nonce': 2067251916}, {'type': 'v1', 'addr': '172.21.15.169:6835', 'nonce': 2067251916}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4, 5], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-11-28T11:21:02.992 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T11:21:02.993 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-28T11:21:03.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.189+0000 7fec353ff700 1 Processor -- start 2023-11-28T11:21:03.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.189+0000 7fec353ff700 1 -- start start 2023-11-28T11:21:03.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.191+0000 7fec353ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fec300b4fc0 0x7fec300cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:03.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.191+0000 7fec353ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec300b1c60 0x7fec300cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:03.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.191+0000 7fec353ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec300d1950 con 0x7fec300d2df0 2023-11-28T11:21:03.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.191+0000 7fec353ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec300d3d30 con 0x7fec300b4fc0 2023-11-28T11:21:03.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.191+0000 7fec353ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec300b2500 con 0x7fec300b1c60 2023-11-28T11:21:03.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.191+0000 7fec27fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fec300b4fc0 0x7fec300cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:03.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.191+0000 7fec2effd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fec300d2df0 0x7fec300d31c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46472/0 (socket says 172.21.15.92:46472) 2023-11-28T11:21:03.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.191+0000 7fec27fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fec300b4fc0 0x7fec300cd620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60376/0 (socket says 172.21.15.92:60376) 2023-11-28T11:21:03.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.191+0000 7fec2effd700 1 -- 172.21.15.92:0/116825718 learned_addr learned my addr 172.21.15.92:0/116825718 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:03.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.191+0000 7fec2f7fe700 1 --2- 172.21.15.92:0/116825718 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec300b1c60 0x7fec300cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:03.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.191+0000 7fec27fff700 1 -- 172.21.15.92:0/116825718 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec300b1c60 msgr2=0x7fec300cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:03.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.191+0000 7fec27fff700 1 --2- 172.21.15.92:0/116825718 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec300b1c60 0x7fec300cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:03.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.191+0000 7fec27fff700 1 -- 172.21.15.92:0/116825718 >> v1:172.21.15.92:6789/0 conn(0x7fec300d2df0 legacy=0x7fec300d31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:03.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.191+0000 7fec27fff700 1 -- 172.21.15.92:0/116825718 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec300d3a50 con 0x7fec300b4fc0 2023-11-28T11:21:03.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.192+0000 7fec27fff700 1 --2- 172.21.15.92:0/116825718 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec300b4fc0 0x7fec300cd620 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fec18009840 tx=0x7fec18009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d185eef61a26c6a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:03.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.192+0000 7fec2e7fc700 1 -- 172.21.15.92:0/116825718 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec18011830 con 0x7fec300b4fc0 2023-11-28T11:21:03.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.192+0000 7fec2e7fc700 1 -- 172.21.15.92:0/116825718 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec18011990 con 0x7fec300b4fc0 2023-11-28T11:21:03.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.192+0000 7fec2e7fc700 1 -- 172.21.15.92:0/116825718 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec18011d20 con 0x7fec300b4fc0 2023-11-28T11:21:03.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.192+0000 7fec353ff700 1 -- 172.21.15.92:0/116825718 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec300b4fc0 msgr2=0x7fec300cd620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:03.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.192+0000 7fec353ff700 1 --2- 172.21.15.92:0/116825718 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec300b4fc0 0x7fec300cd620 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fec18009840 tx=0x7fec18009b50 comp rx=0 tx=0).stop 2023-11-28T11:21:03.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.192+0000 7fec353ff700 1 -- 172.21.15.92:0/116825718 shutdown_connections 2023-11-28T11:21:03.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.192+0000 7fec353ff700 1 --2- 172.21.15.92:0/116825718 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec300b1c60 0x7fec300cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:03.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.192+0000 7fec353ff700 1 --2- 172.21.15.92:0/116825718 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec300b4fc0 0x7fec300cd620 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:03.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.192+0000 7fec353ff700 1 -- 172.21.15.92:0/116825718 >> 172.21.15.92:0/116825718 conn(0x7fec300719a0 msgr2=0x7fec30071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:03.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.192+0000 7fec353ff700 1 -- 172.21.15.92:0/116825718 shutdown_connections 2023-11-28T11:21:03.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.192+0000 7fec353ff700 1 -- 172.21.15.92:0/116825718 wait complete. 2023-11-28T11:21:03.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.194+0000 7fec353ff700 1 Processor -- start 2023-11-28T11:21:03.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.194+0000 7fec353ff700 1 -- start start 2023-11-28T11:21:03.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.195+0000 7fec353ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec300b1c60 0x7fec301e5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:03.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.195+0000 7fec353ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec300b4fc0 0x7fec3014e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:03.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.195+0000 7fec353ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec300d2df0 0x7fec301f1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:03.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.195+0000 7fec353ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec301e5ff0 con 0x7fec300b4fc0 2023-11-28T11:21:03.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.195+0000 7fec353ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec301e7380 con 0x7fec300d2df0 2023-11-28T11:21:03.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.195+0000 7fec353ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec301e7660 con 0x7fec300b1c60 2023-11-28T11:21:03.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.195+0000 7fec27fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec300b4fc0 0x7fec3014e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:03.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.195+0000 7fec27fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec300b4fc0 0x7fec3014e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60390/0 (socket says 172.21.15.92:60390) 2023-11-28T11:21:03.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.195+0000 7fec27fff700 1 -- 172.21.15.92:0/3771892175 learned_addr learned my addr 172.21.15.92:0/3771892175 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:03.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.196+0000 7fec2effd700 1 --2- 172.21.15.92:0/3771892175 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec300b1c60 0x7fec301e5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:03.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.196+0000 7fec2f7fe700 1 --2- 172.21.15.92:0/3771892175 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec300d2df0 0x7fec301f1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:03.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.196+0000 7fec27fff700 1 -- 172.21.15.92:0/3771892175 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec300b1c60 msgr2=0x7fec301e5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:03.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.196+0000 7fec27fff700 1 --2- 172.21.15.92:0/3771892175 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec300b1c60 0x7fec301e5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:03.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.196+0000 7fec27fff700 1 -- 172.21.15.92:0/3771892175 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec300d2df0 msgr2=0x7fec301f1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:03.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.196+0000 7fec27fff700 1 --2- 172.21.15.92:0/3771892175 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec300d2df0 0x7fec301f1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:03.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.196+0000 7fec27fff700 1 -- 172.21.15.92:0/3771892175 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec180094a0 con 0x7fec300b4fc0 2023-11-28T11:21:03.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.196+0000 7fec27fff700 1 --2- 172.21.15.92:0/3771892175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec300b4fc0 0x7fec3014e6e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fec180090b0 tx=0x7fec18004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:03.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.196+0000 7fec277fe700 1 -- 172.21.15.92:0/3771892175 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec180036a0 con 0x7fec300b4fc0 2023-11-28T11:21:03.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.196+0000 7fec277fe700 1 -- 172.21.15.92:0/3771892175 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec1801e070 con 0x7fec300b4fc0 2023-11-28T11:21:03.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.196+0000 7fec353ff700 1 -- 172.21.15.92:0/3771892175 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec300b88f0 con 0x7fec300b4fc0 2023-11-28T11:21:03.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.196+0000 7fec277fe700 1 -- 172.21.15.92:0/3771892175 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec18003a60 con 0x7fec300b4fc0 2023-11-28T11:21:03.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.196+0000 7fec353ff700 1 -- 172.21.15.92:0/3771892175 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec300d15e0 con 0x7fec300b4fc0 2023-11-28T11:21:03.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.198+0000 7fec277fe700 1 -- 172.21.15.92:0/3771892175 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fec1800f050 con 0x7fec300b4fc0 2023-11-28T11:21:03.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.199+0000 7fec277fe700 1 --2- 172.21.15.92:0/3771892175 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fec100682f0 0x7fec1006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:03.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.199+0000 7fec277fe700 1 -- 172.21.15.92:0/3771892175 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fec18086ff0 con 0x7fec300b4fc0 2023-11-28T11:21:03.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.199+0000 7fec2effd700 1 --2- 172.21.15.92:0/3771892175 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fec100682f0 0x7fec1006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:03.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.200+0000 7fec2effd700 1 --2- 172.21.15.92:0/3771892175 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fec100682f0 0x7fec1006a7a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fec20005910 tx=0x7fec20009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:03.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.201+0000 7fec48d2cb80 1 Processor -- start 2023-11-28T11:21:03.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.201+0000 7fec48d2cb80 1 -- start start 2023-11-28T11:21:03.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.202+0000 7fec48d2cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e22c8f8f50 0x55e22c8b29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:03.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.202+0000 7fec48d2cb80 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e22c8efac0 0x55e22c8f9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:03.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.202+0000 7fec48d2cb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e22c8db660 0x55e22c8b3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:03.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.202+0000 7fec48d2cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e22c830de0 con 0x55e22c8f8f50 2023-11-28T11:21:03.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.202+0000 7fec48d2cb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e22c79d140 con 0x55e22c8db660 2023-11-28T11:21:03.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.202+0000 7fec48d2cb80 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e22c80bdb0 con 0x55e22c8efac0 2023-11-28T11:21:03.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.202+0000 7fec2effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e22c8f8f50 0x55e22c8b29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:03.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.202+0000 7fec2effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e22c8f8f50 0x55e22c8b29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60392/0 (socket says 172.21.15.92:60392) 2023-11-28T11:21:03.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.202+0000 7fec2effd700 1 -- 172.21.15.92:0/2217136270 learned_addr learned my addr 172.21.15.92:0/2217136270 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:03.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.203+0000 7fec2f7fe700 1 --2- 172.21.15.92:0/2217136270 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e22c8db660 0x55e22c8b3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:03.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.203+0000 7fec27fff700 1 --2- 172.21.15.92:0/2217136270 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e22c8efac0 0x55e22c8f9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:03.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.203+0000 7fec2effd700 1 -- 172.21.15.92:0/2217136270 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e22c8efac0 msgr2=0x55e22c8f9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:03.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.203+0000 7fec2effd700 1 --2- 172.21.15.92:0/2217136270 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e22c8efac0 0x55e22c8f9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:03.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.203+0000 7fec2effd700 1 -- 172.21.15.92:0/2217136270 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e22c8db660 msgr2=0x55e22c8b3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:03.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.203+0000 7fec2effd700 1 --2- 172.21.15.92:0/2217136270 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e22c8db660 0x55e22c8b3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:03.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.203+0000 7fec2effd700 1 -- 172.21.15.92:0/2217136270 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e22c5f9790 con 0x55e22c8f8f50 2023-11-28T11:21:03.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.203+0000 7fec2effd700 1 --2- 172.21.15.92:0/2217136270 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e22c8f8f50 0x55e22c8b29f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fec2000f7d0 tx=0x7fec20012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd3e19ac7b1a8743 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:03.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.203+0000 7fec24ff9700 1 -- 172.21.15.92:0/2217136270 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec20014450 con 0x55e22c8f8f50 2023-11-28T11:21:03.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.203+0000 7fec24ff9700 1 -- 172.21.15.92:0/2217136270 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec200145b0 con 0x55e22c8f8f50 2023-11-28T11:21:03.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.203+0000 7fec24ff9700 1 -- 172.21.15.92:0/2217136270 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec20014920 con 0x55e22c8f8f50 2023-11-28T11:21:03.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.204+0000 7fec48d2cb80 1 -- 172.21.15.92:0/2217136270 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e22c8f8f50 msgr2=0x55e22c8b29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:03.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.204+0000 7fec48d2cb80 1 --2- 172.21.15.92:0/2217136270 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e22c8f8f50 0x55e22c8b29f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fec2000f7d0 tx=0x7fec20012000 comp rx=0 tx=0).stop 2023-11-28T11:21:03.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.204+0000 7fec48d2cb80 1 -- 172.21.15.92:0/2217136270 shutdown_connections 2023-11-28T11:21:03.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.204+0000 7fec48d2cb80 1 --2- 172.21.15.92:0/2217136270 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e22c8db660 0x55e22c8b3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:03.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.204+0000 7fec48d2cb80 1 --2- 172.21.15.92:0/2217136270 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e22c8f8f50 0x55e22c8b29f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:03.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.204+0000 7fec48d2cb80 1 --2- 172.21.15.92:0/2217136270 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e22c8efac0 0x55e22c8f9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:03.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.204+0000 7fec48d2cb80 1 -- 172.21.15.92:0/2217136270 >> 172.21.15.92:0/2217136270 conn(0x55e22c8a8d20 msgr2=0x55e22c8b1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:03.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.204+0000 7fec48d2cb80 1 -- 172.21.15.92:0/2217136270 shutdown_connections 2023-11-28T11:21:03.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.204+0000 7fec48d2cb80 1 -- 172.21.15.92:0/2217136270 wait complete. 2023-11-28T11:21:03.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.205+0000 7fec48d2cb80 1 -- start start 2023-11-28T11:21:03.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.205+0000 7fec48d2cb80 1 Processor -- start 2023-11-28T11:21:03.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.206+0000 7fec48d2cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e22c8db660 0x55e22c8d6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:03.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.206+0000 7fec48d2cb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e22c8efac0 0x55e22c8f5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:03.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.206+0000 7fec48d2cb80 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e22c8f8f50 0x55e22c8d8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:03.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.206+0000 7fec48d2cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e22c805120 con 0x55e22c8db660 2023-11-28T11:21:03.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.206+0000 7fec48d2cb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e22c5fa390 con 0x55e22c8efac0 2023-11-28T11:21:03.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.206+0000 7fec48d2cb80 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e22c79d340 con 0x55e22c8f8f50 2023-11-28T11:21:03.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.206+0000 7fec48d2cb80 10 client.0 _ll_register_callbacks cb 0x55e22c7a3400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:21:03.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.206+0000 7fec2effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e22c8db660 0x55e22c8d6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:03.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.206+0000 7fec2effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e22c8db660 0x55e22c8d6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60408/0 (socket says 172.21.15.92:60408) 2023-11-28T11:21:03.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.206+0000 7fec2effd700 1 -- 172.21.15.92:0/2860417383 learned_addr learned my addr 172.21.15.92:0/2860417383 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:03.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.206+0000 7fec2f7fe700 1 --2- 172.21.15.92:0/2860417383 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e22c8f8f50 0x55e22c8d8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:03.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.206+0000 7fec27fff700 1 --2- 172.21.15.92:0/2860417383 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e22c8efac0 0x55e22c8f5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:03.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.207+0000 7fec2effd700 1 -- 172.21.15.92:0/2860417383 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e22c8f8f50 msgr2=0x55e22c8d8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:03.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.207+0000 7fec2effd700 1 --2- 172.21.15.92:0/2860417383 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e22c8f8f50 0x55e22c8d8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:03.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.207+0000 7fec2effd700 1 -- 172.21.15.92:0/2860417383 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e22c8efac0 msgr2=0x55e22c8f5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:03.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.207+0000 7fec2effd700 1 --2- 172.21.15.92:0/2860417383 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e22c8efac0 0x55e22c8f5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:03.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.207+0000 7fec2effd700 1 -- 172.21.15.92:0/2860417383 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec300d1950 con 0x55e22c8db660 2023-11-28T11:21:03.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.207+0000 7fec2effd700 1 --2- 172.21.15.92:0/2860417383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e22c8db660 0x55e22c8d6a20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fec20000f60 tx=0x7fec20005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66a99ebc7944098a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:03.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.207+0000 7fec0dffb700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-11-28T11:21:03.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.207+0000 7fec0dffb700 1 -- 172.21.15.92:0/2860417383 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec20014450 con 0x55e22c8db660 2023-11-28T11:21:03.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.207+0000 7fec0dffb700 1 -- 172.21.15.92:0/2860417383 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fec2001c040 con 0x55e22c8db660 2023-11-28T11:21:03.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.207+0000 7fec48d2cb80 1 -- 172.21.15.92:0/2860417383 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e22c809030 con 0x55e22c8db660 2023-11-28T11:21:03.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.207+0000 7fec0dffb700 1 -- 172.21.15.92:0/2860417383 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec20007b90 con 0x55e22c8db660 2023-11-28T11:21:03.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.207+0000 7fec48d2cb80 10 client.4455 fetch_fsmap learned FSMap version 7 2023-11-28T11:21:03.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.207+0000 7fec48d2cb80 1 -- 172.21.15.92:0/2860417383 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x55e22c6e60a0 con 0x55e22c8db660 2023-11-28T11:21:03.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.208+0000 7fec0dffb700 1 -- 172.21.15.92:0/2860417383 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e22c809030 con 0x55e22c8db660 2023-11-28T11:21:03.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.208+0000 7fec0dffb700 1 -- 172.21.15.92:0/2860417383 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 7) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fec20020820 con 0x55e22c8db660 2023-11-28T11:21:03.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.208+0000 7fec48d2cb80 10 client.4455 fetch_fsmap finished waiting for FSMap version 7 2023-11-28T11:21:03.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.208+0000 7fec48d2cb80 10 client.4455 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-28T11:21:03.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.208+0000 7fec48d2cb80 20 client.4455 populate_metadata read hostname 'smithi092' 2023-11-28T11:21:03.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.208+0000 7fec48d2cb80 1 --2- 172.21.15.92:0/2860417383 >> [v2:172.21.15.169:6833/2067251916,v1:172.21.15.169:6835/2067251916] conn(0x55e22c8df3b0 0x55e22c8df780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:03.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.208+0000 7fec48d2cb80 4 client.4455 mds_command: new command op to 4326 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:21:03.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.208+0000 7fec48d2cb80 1 -- 172.21.15.92:0/2860417383 --> [v2:172.21.15.169:6833/2067251916,v1:172.21.15.169:6835/2067251916] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e22c654ef0 con 0x55e22c8df3b0 2023-11-28T11:21:03.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.208+0000 7fec27fff700 1 --2- 172.21.15.92:0/2860417383 >> [v2:172.21.15.169:6833/2067251916,v1:172.21.15.169:6835/2067251916] conn(0x55e22c8df3b0 0x55e22c8df780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:03.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.209+0000 7fec0dffb700 10 client.4455 ms_handle_connect on v2:172.21.15.169:6833/2067251916 2023-11-28T11:21:03.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.209+0000 7fec27fff700 1 --2- 172.21.15.92:0/2860417383 >> [v2:172.21.15.169:6833/2067251916,v1:172.21.15.169:6835/2067251916] conn(0x55e22c8df3b0 0x55e22c8df780 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ed26bfad61671e3 server_cookie=9225c5ee9336a3e5 in_seq=0 out_seq=0 2023-11-28T11:21:03.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.210+0000 7fec0dffb700 1 -- 172.21.15.92:0/2860417383 <== mds.0 v2:172.21.15.169:6833/2067251916 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x55e22c654ef0 con 0x55e22c8df3b0 2023-11-28T11:21:03.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.210+0000 7fec0dffb700 10 client.4455 handle_command_reply: tid=0 2023-11-28T11:21:03.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.210+0000 7fec48d2cb80 1 client.4455 shutdown 2023-11-28T11:21:03.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.210+0000 7fec48d2cb80 1 -- 172.21.15.92:0/2860417383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e22c8db660 msgr2=0x55e22c8d6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:03.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.210+0000 7fec48d2cb80 1 --2- 172.21.15.92:0/2860417383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e22c8db660 0x55e22c8d6a20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fec20000f60 tx=0x7fec20005880 comp rx=0 tx=0).stop 2023-11-28T11:21:03.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.210+0000 7fec48d2cb80 1 -- 172.21.15.92:0/2860417383 shutdown_connections 2023-11-28T11:21:03.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.210+0000 7fec48d2cb80 1 --2- 172.21.15.92:0/2860417383 >> [v2:172.21.15.169:6833/2067251916,v1:172.21.15.169:6835/2067251916] conn(0x55e22c8df3b0 0x55e22c8df780 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:03.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.210+0000 7fec48d2cb80 1 --2- 172.21.15.92:0/2860417383 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e22c8f8f50 0x55e22c8d8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:03.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.210+0000 7fec48d2cb80 1 --2- 172.21.15.92:0/2860417383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e22c8db660 0x55e22c8d6a20 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:03.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.210+0000 7fec0dffb700 0 client.4455 ms_handle_reset on v2:172.21.15.169:6833/2067251916 2023-11-28T11:21:03.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.210+0000 7fec48d2cb80 1 --2- 172.21.15.92:0/2860417383 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e22c8efac0 0x55e22c8f5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:03.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.210+0000 7fec48d2cb80 1 -- 172.21.15.92:0/2860417383 >> 172.21.15.92:0/2860417383 conn(0x55e22c8a8d20 msgr2=0x55e22c8adee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:03.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.210+0000 7fec48d2cb80 1 -- 172.21.15.92:0/2860417383 shutdown_connections 2023-11-28T11:21:03.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.210+0000 7fec48d2cb80 1 -- 172.21.15.92:0/2860417383 wait complete. 2023-11-28T11:21:03.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.210+0000 7fec48d2cb80 20 client.4455 trim_cache size 0 max 16384 2023-11-28T11:21:03.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.228+0000 7fec48d2cb80 1 Processor -- start 2023-11-28T11:21:03.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.228+0000 7fec48d2cb80 1 -- start start 2023-11-28T11:21:03.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.229+0000 7fec48d2cb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e22c8db660 0x55e22c8ee840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:03.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.229+0000 7fec48d2cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e22c8efac0 0x55e22c8eed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:03.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.229+0000 7fec48d2cb80 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e22c8f8f50 0x55e22c8d9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:03.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.229+0000 7fec48d2cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e22c8ef2c0 con 0x55e22c8efac0 2023-11-28T11:21:03.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.229+0000 7fec48d2cb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e22c8b4d80 con 0x55e22c8db660 2023-11-28T11:21:03.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.229+0000 7fec48d2cb80 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e22c8af610 con 0x55e22c8f8f50 2023-11-28T11:21:03.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.229+0000 7fec27fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e22c8efac0 0x55e22c8eed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:03.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.230+0000 7fec27fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e22c8efac0 0x55e22c8eed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60410/0 (socket says 172.21.15.92:60410) 2023-11-28T11:21:03.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.230+0000 7fec27fff700 1 -- 172.21.15.92:0/535037354 learned_addr learned my addr 172.21.15.92:0/535037354 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:03.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.230+0000 7fec2effd700 1 --2- 172.21.15.92:0/535037354 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e22c8db660 0x55e22c8ee840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:03.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.230+0000 7fec2f7fe700 1 --2- 172.21.15.92:0/535037354 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e22c8f8f50 0x55e22c8d9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:03.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.230+0000 7fec27fff700 1 -- 172.21.15.92:0/535037354 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e22c8f8f50 msgr2=0x55e22c8d9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:03.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.230+0000 7fec27fff700 1 --2- 172.21.15.92:0/535037354 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e22c8f8f50 0x55e22c8d9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:03.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.230+0000 7fec27fff700 1 -- 172.21.15.92:0/535037354 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e22c8db660 msgr2=0x55e22c8ee840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:03.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.230+0000 7fec27fff700 1 --2- 172.21.15.92:0/535037354 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e22c8db660 0x55e22c8ee840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:03.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.230+0000 7fec27fff700 1 -- 172.21.15.92:0/535037354 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e22c7cc2e0 con 0x55e22c8efac0 2023-11-28T11:21:03.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.230+0000 7fec27fff700 1 --2- 172.21.15.92:0/535037354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e22c8efac0 0x55e22c8eed80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fec18009790 tx=0x7fec18085210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=136eddeed99dc9ac server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:03.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.230+0000 7fec257fa700 1 -- 172.21.15.92:0/535037354 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec18055370 con 0x55e22c8efac0 2023-11-28T11:21:03.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.230+0000 7fec257fa700 1 -- 172.21.15.92:0/535037354 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec18055510 con 0x55e22c8efac0 2023-11-28T11:21:03.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.230+0000 7fec257fa700 1 -- 172.21.15.92:0/535037354 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec18056860 con 0x55e22c8efac0 2023-11-28T11:21:03.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.231+0000 7fec48d2cb80 1 -- 172.21.15.92:0/535037354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e22c8efac0 msgr2=0x55e22c8eed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:03.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.231+0000 7fec48d2cb80 1 --2- 172.21.15.92:0/535037354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e22c8efac0 0x55e22c8eed80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fec18009790 tx=0x7fec18085210 comp rx=0 tx=0).stop 2023-11-28T11:21:03.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.231+0000 7fec48d2cb80 1 -- 172.21.15.92:0/535037354 shutdown_connections 2023-11-28T11:21:03.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.231+0000 7fec48d2cb80 1 --2- 172.21.15.92:0/535037354 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e22c8f8f50 0x55e22c8d9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:03.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.231+0000 7fec48d2cb80 1 --2- 172.21.15.92:0/535037354 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e22c8db660 0x55e22c8ee840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:03.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.231+0000 7fec48d2cb80 1 --2- 172.21.15.92:0/535037354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e22c8efac0 0x55e22c8eed80 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:03.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.231+0000 7fec48d2cb80 1 -- 172.21.15.92:0/535037354 >> 172.21.15.92:0/535037354 conn(0x55e22c8a8d20 msgr2=0x55e22c8f29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:03.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.231+0000 7fec48d2cb80 1 -- 172.21.15.92:0/535037354 shutdown_connections 2023-11-28T11:21:03.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.231+0000 7fec48d2cb80 1 -- 172.21.15.92:0/535037354 wait complete. 2023-11-28T11:21:03.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.232+0000 7fec48d2cb80 1 -- start start 2023-11-28T11:21:03.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.232+0000 7fec48d2cb80 1 Processor -- start 2023-11-28T11:21:03.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.233+0000 7fec48d2cb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e22c8db660 0x55e22c95d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:03.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.233+0000 7fec48d2cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e22c8efac0 0x55e22c8d9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:03.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.233+0000 7fec48d2cb80 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e22c8f8f50 0x55e22c8ee840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:03.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.233+0000 7fec48d2cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e22c95dba0 con 0x55e22c8efac0 2023-11-28T11:21:03.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.233+0000 7fec48d2cb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e22c95dfd0 con 0x55e22c8db660 2023-11-28T11:21:03.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.233+0000 7fec48d2cb80 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e22c90c300 con 0x55e22c8f8f50 2023-11-28T11:21:03.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.233+0000 7fec48d2cb80 10 client.0 _ll_register_callbacks cb 0x55e22c7a3400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:21:03.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.233+0000 7fec27fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e22c8efac0 0x55e22c8d9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:03.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.233+0000 7fec27fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e22c8efac0 0x55e22c8d9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60426/0 (socket says 172.21.15.92:60426) 2023-11-28T11:21:03.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.233+0000 7fec27fff700 1 -- 172.21.15.92:0/2123285995 learned_addr learned my addr 172.21.15.92:0/2123285995 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:03.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.233+0000 7fec2f7fe700 1 --2- 172.21.15.92:0/2123285995 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e22c8f8f50 0x55e22c8ee840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:03.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.233+0000 7fec2effd700 1 --2- 172.21.15.92:0/2123285995 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e22c8db660 0x55e22c95d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:03.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.233+0000 7fec27fff700 1 -- 172.21.15.92:0/2123285995 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e22c8f8f50 msgr2=0x55e22c8ee840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:03.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.233+0000 7fec27fff700 1 --2- 172.21.15.92:0/2123285995 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e22c8f8f50 0x55e22c8ee840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:03.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.233+0000 7fec27fff700 1 -- 172.21.15.92:0/2123285995 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e22c8db660 msgr2=0x55e22c95d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:03.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.233+0000 7fec27fff700 1 --2- 172.21.15.92:0/2123285995 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e22c8db660 0x55e22c95d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:03.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.233+0000 7fec27fff700 1 -- 172.21.15.92:0/2123285995 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec300d15e0 con 0x55e22c8efac0 2023-11-28T11:21:03.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.234+0000 7fec27fff700 1 --2- 172.21.15.92:0/2123285995 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e22c8efac0 0x55e22c8d9c30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fec18004720 tx=0x7fec20015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=179a111aa94c6b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:03.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.234+0000 7fec0dffb700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-11-28T11:21:03.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.234+0000 7fec0dffb700 1 -- 172.21.15.92:0/2123285995 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec18055370 con 0x55e22c8efac0 2023-11-28T11:21:03.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.234+0000 7fec0dffb700 1 -- 172.21.15.92:0/2123285995 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fec1805ed70 con 0x55e22c8efac0 2023-11-28T11:21:03.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.234+0000 7fec48d2cb80 1 -- 172.21.15.92:0/2123285995 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fec200145b0 con 0x55e22c8efac0 2023-11-28T11:21:03.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.234+0000 7fec0dffb700 1 -- 172.21.15.92:0/2123285995 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec1806a650 con 0x55e22c8efac0 2023-11-28T11:21:03.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.234+0000 7fec48d2cb80 10 client.4461 fetch_fsmap learned FSMap version 7 2023-11-28T11:21:03.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.234+0000 7fec48d2cb80 1 -- 172.21.15.92:0/2123285995 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x55e22c65a1d0 con 0x55e22c8efac0 2023-11-28T11:21:03.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.234+0000 7fec0dffb700 1 -- 172.21.15.92:0/2123285995 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec200145b0 con 0x55e22c8efac0 2023-11-28T11:21:03.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.234+0000 7fec0dffb700 1 -- 172.21.15.92:0/2123285995 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 7) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fec18066030 con 0x55e22c8efac0 2023-11-28T11:21:03.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.234+0000 7fec48d2cb80 10 client.4461 fetch_fsmap finished waiting for FSMap version 7 2023-11-28T11:21:03.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.234+0000 7fec48d2cb80 10 client.4461 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-28T11:21:03.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.234+0000 7fec48d2cb80 20 client.4461 populate_metadata read hostname 'smithi092' 2023-11-28T11:21:03.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.234+0000 7fec48d2cb80 1 --2- 172.21.15.92:0/2123285995 >> [v2:172.21.15.169:6833/2067251916,v1:172.21.15.169:6835/2067251916] conn(0x55e22c8da170 0x55e22c8eed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:03.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.234+0000 7fec48d2cb80 4 client.4461 mds_command: new command op to 4326 tid=0[{"prefix": "status"}] 2023-11-28T11:21:03.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.234+0000 7fec48d2cb80 1 -- 172.21.15.92:0/2123285995 --> [v2:172.21.15.169:6833/2067251916,v1:172.21.15.169:6835/2067251916] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e22c68a040 con 0x55e22c8da170 2023-11-28T11:21:03.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.235+0000 7fec2effd700 1 --2- 172.21.15.92:0/2123285995 >> [v2:172.21.15.169:6833/2067251916,v1:172.21.15.169:6835/2067251916] conn(0x55e22c8da170 0x55e22c8eed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:03.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.235+0000 7fec2effd700 1 --2- 172.21.15.92:0/2123285995 >> [v2:172.21.15.169:6833/2067251916,v1:172.21.15.169:6835/2067251916] conn(0x55e22c8da170 0x55e22c8eed80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85c0b7b3c7b84920 server_cookie=716f6df04f79959f in_seq=0 out_seq=0 2023-11-28T11:21:03.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.235+0000 7fec0dffb700 10 client.4461 ms_handle_connect on v2:172.21.15.169:6833/2067251916 2023-11-28T11:21:03.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.236+0000 7fec0dffb700 1 -- 172.21.15.92:0/2123285995 <== mds.0 v2:172.21.15.169:6833/2067251916 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55e22c68a040 con 0x55e22c8da170 2023-11-28T11:21:03.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.236+0000 7fec0dffb700 10 client.4461 handle_command_reply: tid=0 2023-11-28T11:21:03.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.236+0000 7fec48d2cb80 1 client.4461 shutdown 2023-11-28T11:21:03.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.236+0000 7fec48d2cb80 1 -- 172.21.15.92:0/2123285995 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e22c8efac0 msgr2=0x55e22c8d9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:03.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.236+0000 7fec48d2cb80 1 --2- 172.21.15.92:0/2123285995 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e22c8efac0 0x55e22c8d9c30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fec18004720 tx=0x7fec20015000 comp rx=0 tx=0).stop 2023-11-28T11:21:03.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.236+0000 7fec48d2cb80 1 -- 172.21.15.92:0/2123285995 shutdown_connections 2023-11-28T11:21:03.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.236+0000 7fec48d2cb80 1 --2- 172.21.15.92:0/2123285995 >> [v2:172.21.15.169:6833/2067251916,v1:172.21.15.169:6835/2067251916] conn(0x55e22c8da170 0x55e22c8eed80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:03.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.236+0000 7fec48d2cb80 1 --2- 172.21.15.92:0/2123285995 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e22c8f8f50 0x55e22c8ee840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:03.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.236+0000 7fec48d2cb80 1 --2- 172.21.15.92:0/2123285995 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e22c8db660 0x55e22c95d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:03.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.236+0000 7fec48d2cb80 1 --2- 172.21.15.92:0/2123285995 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e22c8efac0 0x55e22c8d9c30 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:03.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.236+0000 7fec0dffb700 0 client.4461 ms_handle_reset on v2:172.21.15.169:6833/2067251916 2023-11-28T11:21:03.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.236+0000 7fec48d2cb80 1 -- 172.21.15.92:0/2123285995 >> 172.21.15.92:0/2123285995 conn(0x55e22c8a8d20 msgr2=0x55e22c8adf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:03.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.236+0000 7fec48d2cb80 1 -- 172.21.15.92:0/2123285995 shutdown_connections 2023-11-28T11:21:03.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.236+0000 7fec48d2cb80 1 -- 172.21.15.92:0/2123285995 wait complete. 2023-11-28T11:21:03.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.236+0000 7fec48d2cb80 20 client.4461 trim_cache size 0 max 16384 2023-11-28T11:21:03.242 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-28T11:21:03.242 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "68956e0c-c16f-47cd-a220-f98364b0831d", 2023-11-28T11:21:03.242 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 0, 2023-11-28T11:21:03.242 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 4326, 2023-11-28T11:21:03.243 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-11-28T11:21:03.243 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-11-28T11:21:03.243 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-11-28T11:21:03.243 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 6.9983849930000002, 2023-11-28T11:21:03.243 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 7, 2023-11-28T11:21:03.244 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 27, 2023-11-28T11:21:03.244 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 26, 2023-11-28T11:21:03.244 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 12.517280394 2023-11-28T11:21:03.244 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-28T11:21:03.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.237+0000 7fec353ff700 1 -- 172.21.15.92:0/3771892175 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fec100682f0 msgr2=0x7fec1006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:03.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.237+0000 7fec353ff700 1 --2- 172.21.15.92:0/3771892175 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fec100682f0 0x7fec1006a7a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fec20005910 tx=0x7fec20009040 comp rx=0 tx=0).stop 2023-11-28T11:21:03.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.237+0000 7fec353ff700 1 -- 172.21.15.92:0/3771892175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec300b4fc0 msgr2=0x7fec3014e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:03.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.237+0000 7fec353ff700 1 --2- 172.21.15.92:0/3771892175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec300b4fc0 0x7fec3014e6e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fec180090b0 tx=0x7fec18004750 comp rx=0 tx=0).stop 2023-11-28T11:21:03.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.237+0000 7fec353ff700 1 -- 172.21.15.92:0/3771892175 shutdown_connections 2023-11-28T11:21:03.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.237+0000 7fec353ff700 1 --2- 172.21.15.92:0/3771892175 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fec300d2df0 0x7fec301f1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:03.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.237+0000 7fec353ff700 1 --2- 172.21.15.92:0/3771892175 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fec300b1c60 0x7fec301e5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:03.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.237+0000 7fec353ff700 1 --2- 172.21.15.92:0/3771892175 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fec100682f0 0x7fec1006a7a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:03.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.237+0000 7fec353ff700 1 --2- 172.21.15.92:0/3771892175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec300b4fc0 0x7fec3014e6e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:03.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.237+0000 7fec353ff700 1 -- 172.21.15.92:0/3771892175 >> 172.21.15.92:0/3771892175 conn(0x7fec300719a0 msgr2=0x7fec300d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:03.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.237+0000 7fec353ff700 1 -- 172.21.15.92:0/3771892175 shutdown_connections 2023-11-28T11:21:03.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.237+0000 7fec353ff700 1 -- 172.21.15.92:0/3771892175 wait complete. 2023-11-28T11:21:03.255 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-28T11:21:03.255 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-11-28T11:21:03.255 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T11:21:03.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.450+0000 7f6300bff700 1 Processor -- start 2023-11-28T11:21:03.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.450+0000 7f6300bff700 1 -- start start 2023-11-28T11:21:03.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.451+0000 7f6300bff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62fc0d2df0 0x7f62fc0cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:03.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.451+0000 7f6300bff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f62fc0b4fc0 0x7f62fc0d31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:03.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.451+0000 7f6300bff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62fc0d1950 con 0x7f62fc0b1c60 2023-11-28T11:21:03.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.451+0000 7f6300bff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62fc0d3d30 con 0x7f62fc0b4fc0 2023-11-28T11:21:03.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.451+0000 7f6300bff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62fc0b2500 con 0x7f62fc0d2df0 2023-11-28T11:21:03.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.451+0000 7f62fad9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f62fc0b1c60 0x7f62fc0cf010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46476/0 (socket says 172.21.15.92:46476) 2023-11-28T11:21:03.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.451+0000 7f62fad9d700 1 -- 172.21.15.92:0/4007203640 learned_addr learned my addr 172.21.15.92:0/4007203640 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:03.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.451+0000 7f62f3fff700 1 --2- 172.21.15.92:0/4007203640 >> v2:172.21.15.169:3300/0 conn(0x7f62fc0b4fc0 0x7f62fc0d31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:03.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.451+0000 7f62fa59c700 1 --2- 172.21.15.92:0/4007203640 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62fc0d2df0 0x7f62fc0cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:03.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.452+0000 7f62f9d9b700 1 -- 172.21.15.92:0/4007203640 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1618445839 0 0) 0x7f62fc0d1950 con 0x7f62fc0b1c60 2023-11-28T11:21:03.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.452+0000 7f62f9d9b700 1 -- 172.21.15.92:0/4007203640 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62e80035b0 con 0x7f62fc0b1c60 2023-11-28T11:21:03.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.452+0000 7f62f3fff700 1 -- 172.21.15.92:0/4007203640 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62fc0d2df0 msgr2=0x7f62fc0cd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:03.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.452+0000 7f62f3fff700 1 --2- 172.21.15.92:0/4007203640 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62fc0d2df0 0x7f62fc0cd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:03.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.452+0000 7f62f3fff700 1 -- 172.21.15.92:0/4007203640 >> v1:172.21.15.92:6789/0 conn(0x7f62fc0b1c60 legacy=0x7f62fc0cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:03.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.452+0000 7f62f3fff700 1 -- 172.21.15.92:0/4007203640 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62fc0d3a50 con 0x7f62fc0b4fc0 2023-11-28T11:21:03.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.452+0000 7f62fa59c700 1 --2- 172.21.15.92:0/4007203640 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62fc0d2df0 0x7f62fc0cd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:21:03.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.452+0000 7f62f3fff700 1 --2- 172.21.15.92:0/4007203640 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62fc0b4fc0 0x7f62fc0d31c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f62e4009840 tx=0x7f62e4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bdcfeda044d9406 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:03.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.452+0000 7f62f9d9b700 1 -- 172.21.15.92:0/4007203640 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e4011830 con 0x7f62fc0b4fc0 2023-11-28T11:21:03.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.452+0000 7f62f9d9b700 1 -- 172.21.15.92:0/4007203640 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62e4011990 con 0x7f62fc0b4fc0 2023-11-28T11:21:03.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.452+0000 7f62f9d9b700 1 -- 172.21.15.92:0/4007203640 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e4011d00 con 0x7f62fc0b4fc0 2023-11-28T11:21:03.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.453+0000 7f6300bff700 1 -- 172.21.15.92:0/4007203640 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62fc0b4fc0 msgr2=0x7f62fc0d31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:03.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.453+0000 7f6300bff700 1 --2- 172.21.15.92:0/4007203640 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62fc0b4fc0 0x7f62fc0d31c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f62e4009840 tx=0x7f62e4009b50 comp rx=0 tx=0).stop 2023-11-28T11:21:03.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.453+0000 7f6300bff700 1 -- 172.21.15.92:0/4007203640 shutdown_connections 2023-11-28T11:21:03.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.453+0000 7f6300bff700 1 --2- 172.21.15.92:0/4007203640 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62fc0d2df0 0x7f62fc0cd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:03.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.453+0000 7f6300bff700 1 --2- 172.21.15.92:0/4007203640 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62fc0b4fc0 0x7f62fc0d31c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:03.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.453+0000 7f6300bff700 1 -- 172.21.15.92:0/4007203640 >> 172.21.15.92:0/4007203640 conn(0x7f62fc0719a0 msgr2=0x7f62fc071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:03.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.453+0000 7f6300bff700 1 -- 172.21.15.92:0/4007203640 shutdown_connections 2023-11-28T11:21:03.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.453+0000 7f6300bff700 1 -- 172.21.15.92:0/4007203640 wait complete. 2023-11-28T11:21:03.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.455+0000 7f6300bff700 1 Processor -- start 2023-11-28T11:21:03.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.455+0000 7f6300bff700 1 -- start start 2023-11-28T11:21:03.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.456+0000 7f6300bff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62fc0b1c60 0x7f62fc0cc530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:03.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.456+0000 7f6300bff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62fc0b4fc0 0x7f62fc15b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:03.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.456+0000 7f6300bff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62fc0d2df0 0x7f62fc1e5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:03.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.456+0000 7f6300bff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62fc1e5950 con 0x7f62fc0b4fc0 2023-11-28T11:21:03.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.456+0000 7f6300bff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62fc0cb490 con 0x7f62fc0d2df0 2023-11-28T11:21:03.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.456+0000 7f6300bff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62fc0cb770 con 0x7f62fc0b1c60 2023-11-28T11:21:03.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.456+0000 7f62f3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62fc0b4fc0 0x7f62fc15b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:03.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.456+0000 7f62f3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62fc0b4fc0 0x7f62fc15b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60430/0 (socket says 172.21.15.92:60430) 2023-11-28T11:21:03.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.456+0000 7f62f3fff700 1 -- 172.21.15.92:0/3035736114 learned_addr learned my addr 172.21.15.92:0/3035736114 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:03.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.456+0000 7f62fa59c700 1 --2- 172.21.15.92:0/3035736114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62fc0b1c60 0x7f62fc0cc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:03.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.456+0000 7f62fad9d700 1 --2- 172.21.15.92:0/3035736114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62fc0d2df0 0x7f62fc1e5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:03.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.457+0000 7f62f3fff700 1 -- 172.21.15.92:0/3035736114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62fc0b1c60 msgr2=0x7f62fc0cc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:03.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.457+0000 7f62f3fff700 1 --2- 172.21.15.92:0/3035736114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62fc0b1c60 0x7f62fc0cc530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:03.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.457+0000 7f62f3fff700 1 -- 172.21.15.92:0/3035736114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62fc0d2df0 msgr2=0x7f62fc1e5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:03.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.457+0000 7f62f3fff700 1 --2- 172.21.15.92:0/3035736114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62fc0d2df0 0x7f62fc1e5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:03.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.457+0000 7f62f3fff700 1 -- 172.21.15.92:0/3035736114 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62ec0094a0 con 0x7f62fc0b4fc0 2023-11-28T11:21:03.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.457+0000 7f62fad9d700 1 --2- 172.21.15.92:0/3035736114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62fc0d2df0 0x7f62fc1e5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:21:03.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.457+0000 7f62fa59c700 1 --2- 172.21.15.92:0/3035736114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62fc0b1c60 0x7f62fc0cc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:21:03.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.457+0000 7f62f3fff700 1 --2- 172.21.15.92:0/3035736114 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62fc0b4fc0 0x7f62fc15b250 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f62e40090b0 tx=0x7f62e4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:03.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.457+0000 7f62f2ffd700 1 -- 172.21.15.92:0/3035736114 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e40036a0 con 0x7f62fc0b4fc0 2023-11-28T11:21:03.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.457+0000 7f62f2ffd700 1 -- 172.21.15.92:0/3035736114 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62e401e070 con 0x7f62fc0b4fc0 2023-11-28T11:21:03.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.457+0000 7f6300bff700 1 -- 172.21.15.92:0/3035736114 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62e40094a0 con 0x7f62fc0b4fc0 2023-11-28T11:21:03.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.457+0000 7f62f2ffd700 1 -- 172.21.15.92:0/3035736114 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e4003a40 con 0x7f62fc0b4fc0 2023-11-28T11:21:03.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.457+0000 7f6300bff700 1 -- 172.21.15.92:0/3035736114 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62fc0b88f0 con 0x7f62fc0b4fc0 2023-11-28T11:21:03.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.458+0000 7f6300bff700 1 -- 172.21.15.92:0/3035736114 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62fc0d15e0 con 0x7f62fc0b4fc0 2023-11-28T11:21:03.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.459+0000 7f62f2ffd700 1 -- 172.21.15.92:0/3035736114 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f62e400f050 con 0x7f62fc0b4fc0 2023-11-28T11:21:03.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.460+0000 7f62f2ffd700 1 --2- 172.21.15.92:0/3035736114 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f62dc0682f0 0x7f62dc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:03.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.461+0000 7f62f2ffd700 1 -- 172.21.15.92:0/3035736114 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f62e4086fe0 con 0x7f62fc0b4fc0 2023-11-28T11:21:03.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.461+0000 7f62fa59c700 1 --2- 172.21.15.92:0/3035736114 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f62dc0682f0 0x7f62dc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:03.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.461+0000 7f62fa59c700 1 --2- 172.21.15.92:0/3035736114 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f62dc0682f0 0x7f62dc06a7a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f62fc15be10 tx=0x7f62ec008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:03.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.465+0000 7f62f2ffd700 1 -- 172.21.15.92:0/3035736114 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f62e40534a0 con 0x7f62fc0b4fc0 2023-11-28T11:21:03.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.700+0000 7f6300bff700 1 -- 172.21.15.92:0/3035736114 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f62fc0b21b0 con 0x7f62fc0b4fc0 2023-11-28T11:21:03.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.700+0000 7f62f2ffd700 1 -- 172.21.15.92:0/3035736114 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+88 (secure 0 0 0) 0x7f62e4020020 con 0x7f62fc0b4fc0 2023-11-28T11:21:03.701 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-28T11:21:03.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.703+0000 7f6300bff700 1 -- 172.21.15.92:0/3035736114 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f62dc0682f0 msgr2=0x7f62dc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:03.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.703+0000 7f6300bff700 1 --2- 172.21.15.92:0/3035736114 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f62dc0682f0 0x7f62dc06a7a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f62fc15be10 tx=0x7f62ec008040 comp rx=0 tx=0).stop 2023-11-28T11:21:03.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.703+0000 7f6300bff700 1 -- 172.21.15.92:0/3035736114 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62fc0b4fc0 msgr2=0x7f62fc15b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:03.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.703+0000 7f6300bff700 1 --2- 172.21.15.92:0/3035736114 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62fc0b4fc0 0x7f62fc15b250 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f62e40090b0 tx=0x7f62e4004750 comp rx=0 tx=0).stop 2023-11-28T11:21:03.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.703+0000 7f6300bff700 1 -- 172.21.15.92:0/3035736114 shutdown_connections 2023-11-28T11:21:03.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.703+0000 7f6300bff700 1 --2- 172.21.15.92:0/3035736114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62fc0d2df0 0x7f62fc1e5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:03.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.703+0000 7f6300bff700 1 --2- 172.21.15.92:0/3035736114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62fc0b1c60 0x7f62fc0cc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:03.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.703+0000 7f6300bff700 1 --2- 172.21.15.92:0/3035736114 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f62dc0682f0 0x7f62dc06a7a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:03.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.703+0000 7f6300bff700 1 --2- 172.21.15.92:0/3035736114 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62fc0b4fc0 0x7f62fc15b250 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:03.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.703+0000 7f6300bff700 1 -- 172.21.15.92:0/3035736114 >> 172.21.15.92:0/3035736114 conn(0x7f62fc0719a0 msgr2=0x7f62fc0cefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:03.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.703+0000 7f6300bff700 1 -- 172.21.15.92:0/3035736114 shutdown_connections 2023-11-28T11:21:03.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:03.703+0000 7f6300bff700 1 -- 172.21.15.92:0/3035736114 wait complete. 2023-11-28T11:21:03.721 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-28T11:21:03.721 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T11:21:03.721 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi092.front.sepia.ceph.com 2023-11-28T11:21:03.721 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-11-28T11:21:03.721 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-11-28T11:21:03.722 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-28T11:21:03.722 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T11:21:03.722 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T11:21:03.722 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-11-28T11:21:03.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:03.722 DEBUG:teuthology.orchestra.run.smithi092:> ip addr 2023-11-28T11:21:03.769 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:21:03.769 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:21:03.769 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:21:03.770 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:21:03.770 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-11-28T11:21:03.770 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:21:03.770 INFO:teuthology.orchestra.run.smithi092.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:21:03.770 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b2 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:21:03.770 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f0 2023-11-28T11:21:03.770 INFO:teuthology.orchestra.run.smithi092.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:21:03.770 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b3 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:21:03.770 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f1 2023-11-28T11:21:03.770 INFO:teuthology.orchestra.run.smithi092.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:21:03.770 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:44 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:21:03.770 INFO:teuthology.orchestra.run.smithi092.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T11:21:03.770 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:45 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:21:03.771 INFO:teuthology.orchestra.run.smithi092.stdout: inet 172.21.15.92/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T11:21:03.771 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft 41991sec preferred_lft 41991sec 2023-11-28T11:21:03.771 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::ec4:7aff:fe8f:cd45/64 scope link 2023-11-28T11:21:03.771 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:21:03.772 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-28T11:21:03.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:03.772 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-11-28T11:21:03.772 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-11-28T11:21:03.772 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link add name ceph-brx type bridge 2023-11-28T11:21:03.772 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip addr flush dev ceph-brx 2023-11-28T11:21:03.773 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set ceph-brx up 2023-11-28T11:21:03.773 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-28T11:21:03.773 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-11-28T11:21:04.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:04.216 DEBUG:teuthology.orchestra.run.smithi092:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T11:21:04.294 INFO:teuthology.orchestra.run.smithi092.stdout:1 2023-11-28T11:21:04.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:04.298 DEBUG:teuthology.orchestra.run.smithi092:> route 2023-11-28T11:21:04.374 INFO:teuthology.orchestra.run.smithi092.stdout:Kernel IP routing table 2023-11-28T11:21:04.375 INFO:teuthology.orchestra.run.smithi092.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T11:21:04.375 INFO:teuthology.orchestra.run.smithi092.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T11:21:04.375 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T11:21:04.375 INFO:teuthology.orchestra.run.smithi092.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T11:21:04.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:04.376 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-11-28T11:21:04.376 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-11-28T11:21:04.376 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T11:21:04.376 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T11:21:04.377 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T11:21:04.377 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-11-28T11:21:05.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:05.091 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-11-28T11:21:05.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:05.138 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list-id 2023-11-28T11:21:05.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:05.186 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-11-28T11:21:05.186 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-11-28T11:21:05.186 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-28T11:21:05.186 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-11-28T11:21:05.186 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-11-28T11:21:05.328 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-11-28T11:21:05.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:05.329 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-11-28T11:21:05.329 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-11-28T11:21:05.329 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-11-28T11:21:05.330 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-28T11:21:05.330 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-11-28T11:21:05.330 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-11-28T11:21:05.330 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-11-28T11:21:05.330 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-11-28T11:21:05.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:05.594 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-11-28T11:21:05.594 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-11-28T11:21:05.594 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set brx.0 up 2023-11-28T11:21:05.594 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set dev brx.0 master ceph-brx 2023-11-28T11:21:05.594 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-11-28T11:21:05.765 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-11-28T11:21:05.766 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:21:05.766 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-11-28T11:21:05.812 INFO:teuthology.orchestra.run.smithi092.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-11-28T11:21:05.812 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:21:05.812 DEBUG:teuthology.orchestra.run.smithi092:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-11-28T11:21:05.858 DEBUG:teuthology.orchestra.run.smithi092:> sudo modprobe fuse 2023-11-28T11:21:05.969 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-28T11:21:05.969 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-28T11:21:06.038 INFO:teuthology.orchestra.run.smithi092.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-28T11:21:06.040 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-28T11:21:06.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:06.041 DEBUG:teuthology.orchestra.run.smithi092:> ls /sys/fs/fuse/connections 2023-11-28T11:21:06.087 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-28T11:21:06.087 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_mountpoint=/ --client_fs=cephfs) 2023-11-28T11:21:06.132 DEBUG:teuthology.orchestra.run.smithi092:> sudo modprobe fuse 2023-11-28T11:21:06.203 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-28T11:21:06.203 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-28T11:21:06.271 INFO:teuthology.orchestra.run.smithi092.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-28T11:21:06.273 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-28T11:21:06.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:06.274 DEBUG:teuthology.orchestra.run.smithi092:> ls /sys/fs/fuse/connections 2023-11-28T11:21:06.300 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi092.stderr:2023-11-28T11:21:06.299+0000 7f7c8be815c0 -1 init, newargv = 0x55ec37f05c20 newargc=15 2023-11-28T11:21:06.300 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi092.stderr:ceph-fuse[123359]: starting ceph client 2023-11-28T11:21:06.311 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi092.stderr:ceph-fuse[123359]: starting fuse 2023-11-28T11:21:06.323 INFO:teuthology.orchestra.run.smithi092.stdout:43 2023-11-28T11:21:06.323 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-11-28T11:21:06.324 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T11:21:06.324 DEBUG:teuthology.orchestra.run.smithi092:> import glob 2023-11-28T11:21:06.324 DEBUG:teuthology.orchestra.run.smithi092:> import re 2023-11-28T11:21:06.324 DEBUG:teuthology.orchestra.run.smithi092:> import os 2023-11-28T11:21:06.324 DEBUG:teuthology.orchestra.run.smithi092:> import subprocess 2023-11-28T11:21:06.324 DEBUG:teuthology.orchestra.run.smithi092:> 2023-11-28T11:21:06.324 DEBUG:teuthology.orchestra.run.smithi092:> def _find_admin_socket(client_name): 2023-11-28T11:21:06.324 DEBUG:teuthology.orchestra.run.smithi092:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-11-28T11:21:06.324 DEBUG:teuthology.orchestra.run.smithi092:> files = glob.glob(asok_path) 2023-11-28T11:21:06.324 DEBUG:teuthology.orchestra.run.smithi092:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-11-28T11:21:06.324 DEBUG:teuthology.orchestra.run.smithi092:> 2023-11-28T11:21:06.325 DEBUG:teuthology.orchestra.run.smithi092:> # Given a non-glob path, it better be there 2023-11-28T11:21:06.325 DEBUG:teuthology.orchestra.run.smithi092:> if "*" not in asok_path: 2023-11-28T11:21:06.325 DEBUG:teuthology.orchestra.run.smithi092:> assert(len(files) == 1) 2023-11-28T11:21:06.325 DEBUG:teuthology.orchestra.run.smithi092:> return files[0] 2023-11-28T11:21:06.325 DEBUG:teuthology.orchestra.run.smithi092:> 2023-11-28T11:21:06.325 DEBUG:teuthology.orchestra.run.smithi092:> for f in files: 2023-11-28T11:21:06.325 DEBUG:teuthology.orchestra.run.smithi092:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-28T11:21:06.325 DEBUG:teuthology.orchestra.run.smithi092:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-28T11:21:06.325 DEBUG:teuthology.orchestra.run.smithi092:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-28T11:21:06.325 DEBUG:teuthology.orchestra.run.smithi092:> contents = proc_f.read() 2023-11-28T11:21:06.325 DEBUG:teuthology.orchestra.run.smithi092:> if mountpoint in contents: 2023-11-28T11:21:06.325 DEBUG:teuthology.orchestra.run.smithi092:> return f 2023-11-28T11:21:06.325 DEBUG:teuthology.orchestra.run.smithi092:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-28T11:21:06.325 DEBUG:teuthology.orchestra.run.smithi092:> 2023-11-28T11:21:06.326 DEBUG:teuthology.orchestra.run.smithi092:> print(_find_admin_socket("client.admin")) 2023-11-28T11:21:06.326 DEBUG:teuthology.orchestra.run.smithi092:> ' 2023-11-28T11:21:06.508 INFO:teuthology.orchestra.run.smithi092.stdout:/var/run/ceph/ceph-client.admin.123359.asok 2023-11-28T11:21:06.686 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.123359.asok 2023-11-28T11:21:06.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:06.687 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.123359.asok status 2023-11-28T11:21:06.914 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-28T11:21:06.914 INFO:teuthology.orchestra.run.smithi092.stdout: "metadata": { 2023-11-28T11:21:06.914 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_sha1": "2f5f6028b893ad41357cd06e827373f0fe2a1232", 2023-11-28T11:21:06.915 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_version": "ceph version 18.0.0-2697-g2f5f6028 (2f5f6028b893ad41357cd06e827373f0fe2a1232) reef (dev)", 2023-11-28T11:21:06.915 INFO:teuthology.orchestra.run.smithi092.stdout: "entity_id": "admin", 2023-11-28T11:21:06.915 INFO:teuthology.orchestra.run.smithi092.stdout: "hostname": "smithi092", 2023-11-28T11:21:06.915 INFO:teuthology.orchestra.run.smithi092.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-11-28T11:21:06.915 INFO:teuthology.orchestra.run.smithi092.stdout: "pid": "123359", 2023-11-28T11:21:06.915 INFO:teuthology.orchestra.run.smithi092.stdout: "root": "/" 2023-11-28T11:21:06.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:21:06.916 INFO:teuthology.orchestra.run.smithi092.stdout: "dentry_count": 0, 2023-11-28T11:21:06.916 INFO:teuthology.orchestra.run.smithi092.stdout: "dentry_pinned_count": 0, 2023-11-28T11:21:06.916 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 4426, 2023-11-28T11:21:06.916 INFO:teuthology.orchestra.run.smithi092.stdout: "inst": { 2023-11-28T11:21:06.916 INFO:teuthology.orchestra.run.smithi092.stdout: "name": { 2023-11-28T11:21:06.916 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "client", 2023-11-28T11:21:06.916 INFO:teuthology.orchestra.run.smithi092.stdout: "num": 4426 2023-11-28T11:21:06.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:21:06.916 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": { 2023-11-28T11:21:06.916 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2023-11-28T11:21:06.916 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:0", 2023-11-28T11:21:06.916 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3513898193 2023-11-28T11:21:06.917 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:21:06.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:21:06.917 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": { 2023-11-28T11:21:06.917 INFO:teuthology.orchestra.run.smithi092.stdout: "type": "v1", 2023-11-28T11:21:06.917 INFO:teuthology.orchestra.run.smithi092.stdout: "addr": "172.21.15.92:0", 2023-11-28T11:21:06.917 INFO:teuthology.orchestra.run.smithi092.stdout: "nonce": 3513898193 2023-11-28T11:21:06.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:21:06.917 INFO:teuthology.orchestra.run.smithi092.stdout: "inst_str": "client.4426 172.21.15.92:0/3513898193", 2023-11-28T11:21:06.917 INFO:teuthology.orchestra.run.smithi092.stdout: "addr_str": "172.21.15.92:0/3513898193", 2023-11-28T11:21:06.917 INFO:teuthology.orchestra.run.smithi092.stdout: "inode_count": 1, 2023-11-28T11:21:06.917 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_epoch": 7, 2023-11-28T11:21:06.918 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_epoch": 27, 2023-11-28T11:21:06.918 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_epoch_barrier": 0, 2023-11-28T11:21:06.918 INFO:teuthology.orchestra.run.smithi092.stdout: "blocklisted": false, 2023-11-28T11:21:06.918 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs" 2023-11-28T11:21:06.918 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-28T11:21:06.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:06.927 DEBUG:teuthology.orchestra.run.smithi092:> stat --file-system '--printf=%T 2023-11-28T11:21:06.928 DEBUG:teuthology.orchestra.run.smithi092:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-11-28T11:21:06.976 INFO:teuthology.orchestra.run.smithi092.stdout:fuseblk 2023-11-28T11:21:06.976 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-11-28T11:21:06.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:06.976 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-11-28T11:21:07.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:07.049 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-11-28T11:21:07.188 INFO:teuthology.orchestra.run.smithi092.stdout:# file: . 2023-11-28T11:21:07.188 INFO:teuthology.orchestra.run.smithi092.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-11-28T11:21:07.189 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:21:07.190 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-28T11:21:07.190 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:21:07.237 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:21:07.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.471+0000 7f02da9ff700 1 Processor -- start 2023-11-28T11:21:07.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.472+0000 7f02da9ff700 1 -- start start 2023-11-28T11:21:07.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.473+0000 7f02da9ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f02d40b8510 0x7f02d406b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:07.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.473+0000 7f02da9ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02d40d0d60 0x7f02d40d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:07.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.473+0000 7f02da9ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02d40b88e0 con 0x7f02d40cd460 2023-11-28T11:21:07.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.473+0000 7f02da9ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02d40cdbd0 con 0x7f02d40b8510 2023-11-28T11:21:07.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.473+0000 7f02da9ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02d40d02d0 con 0x7f02d40d0d60 2023-11-28T11:21:07.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.473+0000 7f02d3fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f02d40b8510 0x7f02d406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:07.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.473+0000 7f02d3fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f02d40b8510 0x7f02d406b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60436/0 (socket says 172.21.15.92:60436) 2023-11-28T11:21:07.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.473+0000 7f02d3fff700 1 -- 172.21.15.92:0/1451410522 learned_addr learned my addr 172.21.15.92:0/1451410522 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:07.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.473+0000 7f02d8f9c700 1 --2- 172.21.15.92:0/1451410522 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02d40d0d60 0x7f02d40d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:07.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.473+0000 7f02d3fff700 1 -- 172.21.15.92:0/1451410522 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02d40d0d60 msgr2=0x7f02d40d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:07.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.473+0000 7f02d3fff700 1 --2- 172.21.15.92:0/1451410522 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02d40d0d60 0x7f02d40d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:07.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.473+0000 7f02d3fff700 1 -- 172.21.15.92:0/1451410522 >> v1:172.21.15.92:6789/0 conn(0x7f02d40cd460 legacy=0x7f02d406beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:07.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.473+0000 7f02d3fff700 1 -- 172.21.15.92:0/1451410522 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d40cd8f0 con 0x7f02d40b8510 2023-11-28T11:21:07.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.474+0000 7f02d3fff700 1 --2- 172.21.15.92:0/1451410522 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02d40b8510 0x7f02d406b970 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f02c4009920 tx=0x7f02c4009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3545c149981335b0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:07.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.474+0000 7f02d2ffd700 1 -- 172.21.15.92:0/1451410522 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c400ba20 con 0x7f02d40b8510 2023-11-28T11:21:07.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.474+0000 7f02d2ffd700 1 -- 172.21.15.92:0/1451410522 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02c400bb80 con 0x7f02d40b8510 2023-11-28T11:21:07.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.474+0000 7f02d2ffd700 1 -- 172.21.15.92:0/1451410522 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c4009e70 con 0x7f02d40b8510 2023-11-28T11:21:07.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.474+0000 7f02da9ff700 1 -- 172.21.15.92:0/1451410522 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02d40b8510 msgr2=0x7f02d406b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:07.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.474+0000 7f02da9ff700 1 --2- 172.21.15.92:0/1451410522 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02d40b8510 0x7f02d406b970 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f02c4009920 tx=0x7f02c4009c30 comp rx=0 tx=0).stop 2023-11-28T11:21:07.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.475+0000 7f02da9ff700 1 -- 172.21.15.92:0/1451410522 shutdown_connections 2023-11-28T11:21:07.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.475+0000 7f02da9ff700 1 --2- 172.21.15.92:0/1451410522 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02d40d0d60 0x7f02d40d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:07.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.475+0000 7f02da9ff700 1 --2- 172.21.15.92:0/1451410522 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02d40b8510 0x7f02d406b970 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:07.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.475+0000 7f02da9ff700 1 -- 172.21.15.92:0/1451410522 >> 172.21.15.92:0/1451410522 conn(0x7f02d40719a0 msgr2=0x7f02d4071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:07.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.475+0000 7f02da9ff700 1 -- 172.21.15.92:0/1451410522 shutdown_connections 2023-11-28T11:21:07.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.475+0000 7f02da9ff700 1 -- 172.21.15.92:0/1451410522 wait complete. 2023-11-28T11:21:07.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.477+0000 7f02da9ff700 1 Processor -- start 2023-11-28T11:21:07.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.477+0000 7f02da9ff700 1 -- start start 2023-11-28T11:21:07.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.478+0000 7f02da9ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02d40b8510 0x7f02d414d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:07.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.478+0000 7f02da9ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02d40cd460 0x7f02d41f24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:07.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.478+0000 7f02da9ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02d40d0d60 0x7f02d40b7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:07.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.478+0000 7f02da9ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d41e5260 con 0x7f02d40b8510 2023-11-28T11:21:07.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.478+0000 7f02da9ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d41e4650 con 0x7f02d40d0d60 2023-11-28T11:21:07.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.478+0000 7f02da9ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02d41e4930 con 0x7f02d40cd460 2023-11-28T11:21:07.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.479+0000 7f02d3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02d40b8510 0x7f02d414d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:07.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.479+0000 7f02d3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02d40b8510 0x7f02d414d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60444/0 (socket says 172.21.15.92:60444) 2023-11-28T11:21:07.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.479+0000 7f02d3fff700 1 -- 172.21.15.92:0/1148038005 learned_addr learned my addr 172.21.15.92:0/1148038005 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:07.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.479+0000 7f02d37fe700 1 --2- 172.21.15.92:0/1148038005 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02d40cd460 0x7f02d41f24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:07.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.479+0000 7f02d8f9c700 1 --2- 172.21.15.92:0/1148038005 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02d40d0d60 0x7f02d40b7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:07.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.479+0000 7f02d3fff700 1 -- 172.21.15.92:0/1148038005 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02d40cd460 msgr2=0x7f02d41f24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:07.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.479+0000 7f02d3fff700 1 --2- 172.21.15.92:0/1148038005 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02d40cd460 0x7f02d41f24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:07.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.479+0000 7f02d3fff700 1 -- 172.21.15.92:0/1148038005 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02d40d0d60 msgr2=0x7f02d40b7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:07.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.479+0000 7f02d3fff700 1 --2- 172.21.15.92:0/1148038005 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02d40d0d60 0x7f02d40b7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:07.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.479+0000 7f02d3fff700 1 -- 172.21.15.92:0/1148038005 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02c4009550 con 0x7f02d40b8510 2023-11-28T11:21:07.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.479+0000 7f02d3fff700 1 --2- 172.21.15.92:0/1148038005 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02d40b8510 0x7f02d414d2e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f02c400c010 tx=0x7f02c4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:07.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.480+0000 7f02d0ff9700 1 -- 172.21.15.92:0/1148038005 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c4004300 con 0x7f02d40b8510 2023-11-28T11:21:07.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.480+0000 7f02d0ff9700 1 -- 172.21.15.92:0/1148038005 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02c401f070 con 0x7f02d40b8510 2023-11-28T11:21:07.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.480+0000 7f02d0ff9700 1 -- 172.21.15.92:0/1148038005 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c40036a0 con 0x7f02d40b8510 2023-11-28T11:21:07.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.480+0000 7f02da9ff700 1 -- 172.21.15.92:0/1148038005 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02d40cfff0 con 0x7f02d40b8510 2023-11-28T11:21:07.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.480+0000 7f02da9ff700 1 -- 172.21.15.92:0/1148038005 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02d40cfd10 con 0x7f02d40b8510 2023-11-28T11:21:07.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.481+0000 7f02da9ff700 1 -- 172.21.15.92:0/1148038005 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02d4067880 con 0x7f02d40b8510 2023-11-28T11:21:07.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.482+0000 7f02d0ff9700 1 -- 172.21.15.92:0/1148038005 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f02c402a020 con 0x7f02d40b8510 2023-11-28T11:21:07.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.482+0000 7f02d0ff9700 1 --2- 172.21.15.92:0/1148038005 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f02c00682f0 0x7f02c006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:07.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.483+0000 7f02d0ff9700 1 -- 172.21.15.92:0/1148038005 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f02c4087660 con 0x7f02d40b8510 2023-11-28T11:21:07.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.483+0000 7f02d37fe700 1 --2- 172.21.15.92:0/1148038005 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f02c00682f0 0x7f02c006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:07.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.483+0000 7f02d37fe700 1 --2- 172.21.15.92:0/1148038005 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f02c00682f0 0x7f02c006a7a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f02d41f5930 tx=0x7f02c8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:07.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.487+0000 7f02d0ff9700 1 -- 172.21.15.92:0/1148038005 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f02c4053a00 con 0x7f02d40b8510 2023-11-28T11:21:07.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.681+0000 7f02da9ff700 1 -- 172.21.15.92:0/1148038005 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f02d40d2940 con 0x7f02d40b8510 2023-11-28T11:21:07.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.682+0000 7f02d0ff9700 1 -- 172.21.15.92:0/1148038005 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+19748 (secure 0 0 0) 0x7f02c4057020 con 0x7f02d40b8510 2023-11-28T11:21:07.684 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:21:07.684 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":27,"fsid":"68956e0c-c16f-47cd-a220-f98364b0831d","created":"2023-11-28T11:20:28.697573+0000","modified":"2023-11-28T11:21:00.770781+0000","last_up_change":"2023-11-28T11:20:41.257011+0000","last_in_change":"2023-11-28T11:20:32.249767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:20:41.925237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:20:48.682497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:20:54.471398+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:20:55.204585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:20:58.297307+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6abe483f-0e03-4a26-b9cf-62041291747e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":115823},{"type":"v1","addr":"172.21.15.92:6809","nonce":115823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":115823},{"type":"v1","addr":"172.21.15.92:6813","nonce":115823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":115823},{"type":"v1","addr":"172.21.15.92:6821","nonce":115823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":115823},{"type":"v1","addr":"172.21.15.92:6817","nonce":115823}]},"public_addr":"172.21.15.92:6809/115823","cluster_addr":"172.21.15.92:6813/115823","heartbeat_back_addr":"172.21.15.92:6821/115823","heartbeat_front_addr":"172.21.15.92:6817/115823","state":["exists","up"]},{"osd":1,"uuid":"13c1de73-1a8c-4d7e-8a28-9dae52dfe089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":115824},{"type":"v1","addr":"172.21.15.92:6812","nonce":115824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":115824},{"type":"v1","addr":"172.21.15.92:6816","nonce":115824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":115824},{"type":"v1","addr":"172.21.15.92:6823","nonce":115824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":115824},{"type":"v1","addr":"172.21.15.92:6820","nonce":115824}]},"public_addr":"172.21.15.92:6812/115824","cluster_addr":"172.21.15.92:6816/115824","heartbeat_back_addr":"172.21.15.92:6823/115824","heartbeat_front_addr":"172.21.15.92:6820/115824","state":["exists","up"]},{"osd":2,"uuid":"25788a55-112f-4f00-810d-2bc37cf76778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":115821},{"type":"v1","addr":"172.21.15.92:6801","nonce":115821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":115821},{"type":"v1","addr":"172.21.15.92:6803","nonce":115821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":115821},{"type":"v1","addr":"172.21.15.92:6807","nonce":115821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":115821},{"type":"v1","addr":"172.21.15.92:6805","nonce":115821}]},"public_addr":"172.21.15.92:6801/115821","cluster_addr":"172.21.15.92:6803/115821","heartbeat_back_addr":"172.21.15.92:6807/115821","heartbeat_front_addr":"172.21.15.92:6805/115821","state":["exists","up"]},{"osd":3,"uuid":"adf54e24-33a9-41fe-9ce3-98a5ab6c158e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":115825},{"type":"v1","addr":"172.21.15.92:6825","nonce":115825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":115825},{"type":"v1","addr":"172.21.15.92:6827","nonce":115825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":115825},{"type":"v1","addr":"172.21.15.92:6831","nonce":115825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":115825},{"type":"v1","addr":"172.21.15.92:6829","nonce":115825}]},"public_addr":"172.21.15.92:6825/115825","cluster_addr":"172.21.15.92:6827/115825","heartbeat_back_addr":"172.21.15.92:6831/115825","heartbeat_front_addr":"172.21.15.92:6829/115825","state":["exists","up"]},{"osd":4,"uuid":"ccb9b747-f399-44ea-9a18-9db3152227e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":116035},{"type":"v1","addr":"172.21.15.169:6817","nonce":116035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":116035},{"type":"v1","addr":"172.21.15.169:6819","nonce":116035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":116035},{"type":"v1","addr":"172.21.15.169:6826","nonce":116035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":116035},{"type":"v1","addr":"172.21.15.169:6822","nonce":116035}]},"public_addr":"172.21.15.169:6817/116035","cluster_addr":"172.21.15.169:6819/116035","heartbeat_back_addr":"172.21.15.169:6826/116035","heartbeat_front_addr":"172.21.15.169:6822/116035","state":["exists","up"]},{"osd":5,"uuid":"308a1e63-75aa-4d7d-8a7c-cecc24714fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":116033},{"type":"v1","addr":"172.21.15.169:6801","nonce":116033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":116033},{"type":"v1","addr":"172.21.15.169:6803","nonce":116033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":116033},{"type":"v1","addr":"172.21.15.169:6807","nonce":116033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":116033},{"type":"v1","addr":"172.21.15.169:6805","nonce":116033}]},"public_addr":"172.21.15.169:6801/116033","cluster_addr":"172.21.15.169:6803/116033","heartbeat_back_addr":"172.21.15.169:6807/116033","heartbeat_front_addr":"172.21.15.169:6805/116033","state":["exists","up"]},{"osd":6,"uuid":"9c459b7b-b821-4b4b-b91e-6aaae271457c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":116034},{"type":"v1","addr":"172.21.15.169:6809","nonce":116034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":116034},{"type":"v1","addr":"172.21.15.169:6811","nonce":116034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":116034},{"type":"v1","addr":"172.21.15.169:6815","nonce":116034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":116034},{"type":"v1","addr":"172.21.15.169:6813","nonce":116034}]},"public_addr":"172.21.15.169:6809/116034","cluster_addr":"172.21.15.169:6811/116034","heartbeat_back_addr":"172.21.15.169:6815/116034","heartbeat_front_addr":"172.21.15.169:6813/116034","state":["exists","up"]},{"osd":7,"uuid":"1ca9aa4a-755d-4e28-a937-4d8dccab4df3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":116044},{"type":"v1","addr":"172.21.15.169:6823","nonce":116044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6825","nonce":116044},{"type":"v1","addr":"172.21.15.169:6827","nonce":116044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":116044},{"type":"v1","addr":"172.21.15.169:6831","nonce":116044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":116044},{"type":"v1","addr":"172.21.15.169:6829","nonce":116044}]},"public_addr":"172.21.15.169:6823/116044","cluster_addr":"172.21.15.169:6827/116044","heartbeat_back_addr":"172.21.15.169:6831/116044","heartbeat_front_addr":"172.21.15.169:6829/116044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.586781+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.869778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.508594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.860493+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.872267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.746933+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.611520+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.871446+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:21:07.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.685+0000 7f02da9ff700 1 -- 172.21.15.92:0/1148038005 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f02c00682f0 msgr2=0x7f02c006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:07.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.685+0000 7f02da9ff700 1 --2- 172.21.15.92:0/1148038005 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f02c00682f0 0x7f02c006a7a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f02d41f5930 tx=0x7f02c8009040 comp rx=0 tx=0).stop 2023-11-28T11:21:07.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.685+0000 7f02da9ff700 1 -- 172.21.15.92:0/1148038005 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02d40b8510 msgr2=0x7f02d414d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:07.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.685+0000 7f02da9ff700 1 --2- 172.21.15.92:0/1148038005 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02d40b8510 0x7f02d414d2e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f02c400c010 tx=0x7f02c4004750 comp rx=0 tx=0).stop 2023-11-28T11:21:07.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.685+0000 7f02da9ff700 1 -- 172.21.15.92:0/1148038005 shutdown_connections 2023-11-28T11:21:07.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.685+0000 7f02da9ff700 1 --2- 172.21.15.92:0/1148038005 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02d40d0d60 0x7f02d40b7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:07.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.685+0000 7f02da9ff700 1 --2- 172.21.15.92:0/1148038005 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f02c00682f0 0x7f02c006a7a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:07.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.685+0000 7f02da9ff700 1 --2- 172.21.15.92:0/1148038005 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02d40b8510 0x7f02d414d2e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:07.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.685+0000 7f02da9ff700 1 --2- 172.21.15.92:0/1148038005 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02d40cd460 0x7f02d41f24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:07.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.685+0000 7f02da9ff700 1 -- 172.21.15.92:0/1148038005 >> 172.21.15.92:0/1148038005 conn(0x7f02d40719a0 msgr2=0x7f02d40cbef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:07.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.685+0000 7f02da9ff700 1 -- 172.21.15.92:0/1148038005 shutdown_connections 2023-11-28T11:21:07.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.685+0000 7f02da9ff700 1 -- 172.21.15.92:0/1148038005 wait complete. 2023-11-28T11:21:07.704 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:20:41.925237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:20:48.682497+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:20:54.471398+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:20:55.204585+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T11:20:58.297307+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T11:21:07.704 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:21:07.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.902+0000 7f6d97fff700 1 Processor -- start 2023-11-28T11:21:07.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.902+0000 7f6d97fff700 1 -- start start 2023-11-28T11:21:07.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.903+0000 7f6d97fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d900b8510 0x7f6d9006b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:07.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.903+0000 7f6d97fff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6d900cd460 0x7f6d9006beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:07.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.904+0000 7f6d97fff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d900b88e0 con 0x7f6d9006c3f0 2023-11-28T11:21:07.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.904+0000 7f6d97fff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d900cdbd0 con 0x7f6d900cd460 2023-11-28T11:21:07.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.904+0000 7f6d97fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9006c830 con 0x7f6d900b8510 2023-11-28T11:21:07.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.904+0000 7f6d9559a700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6d900cd460 0x7f6d9006beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:07.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.904+0000 7f6d95d9b700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d900b8510 0x7f6d9006b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:07.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.904+0000 7f6d95d9b700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d900b8510 0x7f6d9006b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.92:39368/0 (socket says 172.21.15.92:39368) 2023-11-28T11:21:07.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.904+0000 7f6d9659c700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f6d9006c3f0 0x7f6d9015cc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.92:60650/0 (socket says 172.21.15.92:60650) 2023-11-28T11:21:07.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.904+0000 7f6d9559a700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6d900cd460 0x7f6d9006beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.92:53446/0 (socket says 172.21.15.92:53446) 2023-11-28T11:21:07.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.904+0000 7f6d95d9b700 1 -- 172.21.15.92:0/296508408 learned_addr learned my addr 172.21.15.92:0/296508408 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:07.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.905+0000 7f6d95d9b700 1 -- 172.21.15.92:0/296508408 >> v2:172.21.15.169:3300/0 conn(0x7f6d900cd460 msgr2=0x7f6d9006beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:07.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.905+0000 7f6d95d9b700 1 --2- 172.21.15.92:0/296508408 >> v2:172.21.15.169:3300/0 conn(0x7f6d900cd460 0x7f6d9006beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:07.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.905+0000 7f6d95d9b700 1 -- 172.21.15.92:0/296508408 >> v1:172.21.15.169:6789/0 conn(0x7f6d9006c3f0 legacy=0x7f6d9015cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:07.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.905+0000 7f6d95d9b700 1 -- 172.21.15.92:0/296508408 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d900cd8f0 con 0x7f6d900b8510 2023-11-28T11:21:07.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.905+0000 7f6d95d9b700 1 --2- 172.21.15.92:0/296508408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d900b8510 0x7f6d9006b970 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c0098f0 tx=0x7f6d8c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdc0a66b0edc3f78 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:07.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.905+0000 7f6d94d99700 1 -- 172.21.15.92:0/296508408 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c011830 con 0x7f6d900b8510 2023-11-28T11:21:07.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.905+0000 7f6d94d99700 1 -- 172.21.15.92:0/296508408 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d8c011990 con 0x7f6d900b8510 2023-11-28T11:21:07.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.905+0000 7f6d94d99700 1 -- 172.21.15.92:0/296508408 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c011d20 con 0x7f6d900b8510 2023-11-28T11:21:07.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.906+0000 7f6d97fff700 1 -- 172.21.15.92:0/296508408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d900b8510 msgr2=0x7f6d9006b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:07.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.906+0000 7f6d97fff700 1 --2- 172.21.15.92:0/296508408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d900b8510 0x7f6d9006b970 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c0098f0 tx=0x7f6d8c009c00 comp rx=0 tx=0).stop 2023-11-28T11:21:07.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.906+0000 7f6d97fff700 1 -- 172.21.15.92:0/296508408 shutdown_connections 2023-11-28T11:21:07.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.906+0000 7f6d97fff700 1 --2- 172.21.15.92:0/296508408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d900b8510 0x7f6d9006b970 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:07.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.906+0000 7f6d97fff700 1 --2- 172.21.15.92:0/296508408 >> v2:172.21.15.169:3300/0 conn(0x7f6d900cd460 0x7f6d9006beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:07.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.906+0000 7f6d97fff700 1 -- 172.21.15.92:0/296508408 >> 172.21.15.92:0/296508408 conn(0x7f6d900719a0 msgr2=0x7f6d90071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:07.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.906+0000 7f6d97fff700 1 -- 172.21.15.92:0/296508408 shutdown_connections 2023-11-28T11:21:07.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.906+0000 7f6d97fff700 1 -- 172.21.15.92:0/296508408 wait complete. 2023-11-28T11:21:07.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.908+0000 7f6d97fff700 1 Processor -- start 2023-11-28T11:21:07.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.908+0000 7f6d97fff700 1 -- start start 2023-11-28T11:21:07.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.909+0000 7f6d97fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d9006c3f0 0x7f6d900c9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:07.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.909+0000 7f6d97fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d900b8510 0x7f6d900cb740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:07.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.909+0000 7f6d97fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d900cd460 0x7f6d900c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:07.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.909+0000 7f6d97fff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d900cb2b0 con 0x7f6d900b8510 2023-11-28T11:21:07.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.909+0000 7f6d97fff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d900cbc80 con 0x7f6d9006c3f0 2023-11-28T11:21:07.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.909+0000 7f6d97fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d900ca420 con 0x7f6d900cd460 2023-11-28T11:21:07.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.909+0000 7f6d9559a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d900b8510 0x7f6d900cb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:07.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.909+0000 7f6d9559a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d900b8510 0x7f6d900cb740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60458/0 (socket says 172.21.15.92:60458) 2023-11-28T11:21:07.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.909+0000 7f6d9559a700 1 -- 172.21.15.92:0/3493407077 learned_addr learned my addr 172.21.15.92:0/3493407077 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:07.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.910+0000 7f6d9659c700 1 --2- 172.21.15.92:0/3493407077 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d900cd460 0x7f6d900c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:07.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.910+0000 7f6d95d9b700 1 --2- 172.21.15.92:0/3493407077 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d9006c3f0 0x7f6d900c9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:07.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.910+0000 7f6d9559a700 1 -- 172.21.15.92:0/3493407077 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d900cd460 msgr2=0x7f6d900c9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:07.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.910+0000 7f6d9559a700 1 --2- 172.21.15.92:0/3493407077 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d900cd460 0x7f6d900c9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:07.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.910+0000 7f6d9559a700 1 -- 172.21.15.92:0/3493407077 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d9006c3f0 msgr2=0x7f6d900c9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:07.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.910+0000 7f6d9559a700 1 --2- 172.21.15.92:0/3493407077 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d9006c3f0 0x7f6d900c9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:07.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.910+0000 7f6d9559a700 1 -- 172.21.15.92:0/3493407077 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d8c009550 con 0x7f6d900b8510 2023-11-28T11:21:07.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.910+0000 7f6d9559a700 1 --2- 172.21.15.92:0/3493407077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d900b8510 0x7f6d900cb740 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6d8000e8a0 tx=0x7f6d8000ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:07.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.910+0000 7f6d867fc700 1 -- 172.21.15.92:0/3493407077 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d80009590 con 0x7f6d900b8510 2023-11-28T11:21:07.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.910+0000 7f6d867fc700 1 -- 172.21.15.92:0/3493407077 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d80013680 con 0x7f6d900b8510 2023-11-28T11:21:07.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.910+0000 7f6d867fc700 1 -- 172.21.15.92:0/3493407077 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d800139f0 con 0x7f6d900b8510 2023-11-28T11:21:07.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.910+0000 7f6d97fff700 1 -- 172.21.15.92:0/3493407077 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d900cfff0 con 0x7f6d900b8510 2023-11-28T11:21:07.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.910+0000 7f6d97fff700 1 -- 172.21.15.92:0/3493407077 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d900cfd10 con 0x7f6d900b8510 2023-11-28T11:21:07.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.911+0000 7f6d97fff700 1 -- 172.21.15.92:0/3493407077 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d900cb460 con 0x7f6d900b8510 2023-11-28T11:21:07.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.912+0000 7f6d867fc700 1 -- 172.21.15.92:0/3493407077 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6d80019030 con 0x7f6d900b8510 2023-11-28T11:21:07.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.914+0000 7f6d867fc700 1 --2- 172.21.15.92:0/3493407077 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6d740682f0 0x7f6d7406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:07.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.914+0000 7f6d867fc700 1 -- 172.21.15.92:0/3493407077 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6d80085d70 con 0x7f6d900b8510 2023-11-28T11:21:07.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.914+0000 7f6d95d9b700 1 --2- 172.21.15.92:0/3493407077 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6d740682f0 0x7f6d7406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:07.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.914+0000 7f6d95d9b700 1 --2- 172.21.15.92:0/3493407077 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6d740682f0 0x7f6d7406a7a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c000c00 tx=0x7f6d8c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:07.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:07.918+0000 7f6d867fc700 1 -- 172.21.15.92:0/3493407077 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6d900cb460 con 0x7f6d900b8510 2023-11-28T11:21:08.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.108+0000 7f6d97fff700 1 -- 172.21.15.92:0/3493407077 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6d900cb460 con 0x7f6d900b8510 2023-11-28T11:21:08.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.108+0000 7f6d867fc700 1 -- 172.21.15.92:0/3493407077 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6d900cb460 con 0x7f6d900b8510 2023-11-28T11:21:08.110 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-11-28T11:21:08.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.111+0000 7f6d97fff700 1 -- 172.21.15.92:0/3493407077 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6d740682f0 msgr2=0x7f6d7406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:08.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.111+0000 7f6d97fff700 1 --2- 172.21.15.92:0/3493407077 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6d740682f0 0x7f6d7406a7a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c000c00 tx=0x7f6d8c008030 comp rx=0 tx=0).stop 2023-11-28T11:21:08.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.111+0000 7f6d97fff700 1 -- 172.21.15.92:0/3493407077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d900b8510 msgr2=0x7f6d900cb740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:08.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.111+0000 7f6d97fff700 1 --2- 172.21.15.92:0/3493407077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d900b8510 0x7f6d900cb740 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6d8000e8a0 tx=0x7f6d8000ebb0 comp rx=0 tx=0).stop 2023-11-28T11:21:08.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.111+0000 7f6d97fff700 1 -- 172.21.15.92:0/3493407077 shutdown_connections 2023-11-28T11:21:08.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.111+0000 7f6d97fff700 1 --2- 172.21.15.92:0/3493407077 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d900cd460 0x7f6d900c9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:08.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.111+0000 7f6d97fff700 1 --2- 172.21.15.92:0/3493407077 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d9006c3f0 0x7f6d900c9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:08.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.111+0000 7f6d97fff700 1 --2- 172.21.15.92:0/3493407077 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6d740682f0 0x7f6d7406a7a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:08.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.111+0000 7f6d97fff700 1 --2- 172.21.15.92:0/3493407077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d900b8510 0x7f6d900cb740 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:08.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.111+0000 7f6d97fff700 1 -- 172.21.15.92:0/3493407077 >> 172.21.15.92:0/3493407077 conn(0x7f6d900719a0 msgr2=0x7f6d900cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:08.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.111+0000 7f6d97fff700 1 -- 172.21.15.92:0/3493407077 shutdown_connections 2023-11-28T11:21:08.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.111+0000 7f6d97fff700 1 -- 172.21.15.92:0/3493407077 wait complete. 2023-11-28T11:21:08.131 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:21:08.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.332+0000 7fb3975ff700 1 Processor -- start 2023-11-28T11:21:08.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.332+0000 7fb3975ff700 1 -- start start 2023-11-28T11:21:08.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.334+0000 7fb3975ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3900b1c60 0x7fb3900cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:08.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.334+0000 7fb3975ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3900d1950 con 0x7fb3900b4fc0 2023-11-28T11:21:08.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.334+0000 7fb3975ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3900b2500 con 0x7fb3900d2df0 2023-11-28T11:21:08.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.334+0000 7fb3975ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3900d3d30 con 0x7fb3900b1c60 2023-11-28T11:21:08.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.334+0000 7fb394b9a700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb3900b4fc0 0x7fb3900cf010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46484/0 (socket says 172.21.15.92:46484) 2023-11-28T11:21:08.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.334+0000 7fb394b9a700 1 -- 172.21.15.92:0/4267076936 learned_addr learned my addr 172.21.15.92:0/4267076936 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:08.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.334+0000 7fb395b9c700 1 --2- 172.21.15.92:0/4267076936 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3900b1c60 0x7fb3900cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:08.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.334+0000 7fb387fff700 1 -- 172.21.15.92:0/4267076936 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2342672046 0 0) 0x7fb3900d1950 con 0x7fb3900b4fc0 2023-11-28T11:21:08.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.334+0000 7fb387fff700 1 -- 172.21.15.92:0/4267076936 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb37c0035b0 con 0x7fb3900b4fc0 2023-11-28T11:21:08.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.334+0000 7fb387fff700 1 -- 172.21.15.92:0/4267076936 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2296909531 0 0) 0x7fb3900b2500 con 0x7fb3900d2df0 2023-11-28T11:21:08.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.334+0000 7fb387fff700 1 -- 172.21.15.92:0/4267076936 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3900d1950 con 0x7fb3900d2df0 2023-11-28T11:21:08.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.335+0000 7fb387fff700 1 -- 172.21.15.92:0/4267076936 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3240406430 0 0) 0x7fb37c0035b0 con 0x7fb3900b4fc0 2023-11-28T11:21:08.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.335+0000 7fb387fff700 1 -- 172.21.15.92:0/4267076936 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3900b1c60 msgr2=0x7fb3900cd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:08.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.335+0000 7fb387fff700 1 --2- 172.21.15.92:0/4267076936 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3900b1c60 0x7fb3900cd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:08.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.335+0000 7fb387fff700 1 -- 172.21.15.92:0/4267076936 >> v1:172.21.15.169:6789/0 conn(0x7fb3900d2df0 legacy=0x7fb3900d31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:08.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.335+0000 7fb387fff700 1 -- 172.21.15.92:0/4267076936 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3900d3a50 con 0x7fb3900b4fc0 2023-11-28T11:21:08.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.335+0000 7fb387fff700 1 -- 172.21.15.92:0/4267076936 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532161587 0 0) 0x7fb380001670 con 0x7fb3900b4fc0 2023-11-28T11:21:08.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.335+0000 7fb387fff700 1 -- 172.21.15.92:0/4267076936 >> v1:172.21.15.92:6789/0 conn(0x7fb3900b4fc0 legacy=0x7fb3900cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:08.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.335+0000 7fb387fff700 1 --2- 172.21.15.92:0/4267076936 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb37c003bc0 0x7fb37c003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:08.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.335+0000 7fb387fff700 1 --2- 172.21.15.92:0/4267076936 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb37c004850 0x7fb37c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:08.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.335+0000 7fb387fff700 1 --2- 172.21.15.92:0/4267076936 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3900b1c60 0x7fb37c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:08.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.335+0000 7fb387fff700 1 -- 172.21.15.92:0/4267076936 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3900d3d30 con 0x7fb37c004850 2023-11-28T11:21:08.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.335+0000 7fb387fff700 1 -- 172.21.15.92:0/4267076936 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb37c00bac0 con 0x7fb37c003bc0 2023-11-28T11:21:08.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.335+0000 7fb387fff700 1 -- 172.21.15.92:0/4267076936 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb37c00bc30 con 0x7fb3900b1c60 2023-11-28T11:21:08.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.335+0000 7fb394b9a700 1 --2- 172.21.15.92:0/4267076936 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb37c004850 0x7fb37c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:08.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.335+0000 7fb39539b700 1 --2- 172.21.15.92:0/4267076936 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb37c003bc0 0x7fb37c003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:08.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.335+0000 7fb395b9c700 1 --2- 172.21.15.92:0/4267076936 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3900b1c60 0x7fb37c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:08.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.335+0000 7fb394b9a700 1 -- 172.21.15.92:0/4267076936 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3900b1c60 msgr2=0x7fb37c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:08.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.335+0000 7fb394b9a700 1 --2- 172.21.15.92:0/4267076936 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3900b1c60 0x7fb37c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:08.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.335+0000 7fb394b9a700 1 -- 172.21.15.92:0/4267076936 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb37c003bc0 msgr2=0x7fb37c003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:08.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.335+0000 7fb394b9a700 1 --2- 172.21.15.92:0/4267076936 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb37c003bc0 0x7fb37c003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:08.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.335+0000 7fb394b9a700 1 -- 172.21.15.92:0/4267076936 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb37c0035b0 con 0x7fb37c004850 2023-11-28T11:21:08.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.335+0000 7fb395b9c700 1 --2- 172.21.15.92:0/4267076936 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3900b1c60 0x7fb37c005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:21:08.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.336+0000 7fb394b9a700 1 --2- 172.21.15.92:0/4267076936 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb37c004850 0x7fb37c004cc0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb38000c210 tx=0x7fb37c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89eba7d9f5056c08 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:08.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.336+0000 7fb387fff700 1 -- 172.21.15.92:0/4267076936 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38000cdb0 con 0x7fb37c004850 2023-11-28T11:21:08.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.336+0000 7fb387fff700 1 -- 172.21.15.92:0/4267076936 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb380001af0 con 0x7fb37c004850 2023-11-28T11:21:08.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.336+0000 7fb387fff700 1 -- 172.21.15.92:0/4267076936 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb380005ba0 con 0x7fb37c004850 2023-11-28T11:21:08.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.336+0000 7fb3975ff700 1 -- 172.21.15.92:0/4267076936 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb37c004850 msgr2=0x7fb37c004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:08.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.336+0000 7fb3975ff700 1 --2- 172.21.15.92:0/4267076936 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb37c004850 0x7fb37c004cc0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb38000c210 tx=0x7fb37c005880 comp rx=0 tx=0).stop 2023-11-28T11:21:08.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.336+0000 7fb395b9c700 1 -- 172.21.15.92:0/4267076936 reap_dead start 2023-11-28T11:21:08.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.336+0000 7fb3975ff700 1 -- 172.21.15.92:0/4267076936 shutdown_connections 2023-11-28T11:21:08.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.336+0000 7fb3975ff700 1 -- 172.21.15.92:0/4267076936 >> 172.21.15.92:0/4267076936 conn(0x7fb3900719a0 msgr2=0x7fb390071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:08.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.336+0000 7fb3975ff700 1 -- 172.21.15.92:0/4267076936 shutdown_connections 2023-11-28T11:21:08.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.336+0000 7fb3975ff700 1 -- 172.21.15.92:0/4267076936 wait complete. 2023-11-28T11:21:08.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.338+0000 7fb3975ff700 1 Processor -- start 2023-11-28T11:21:08.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.338+0000 7fb3975ff700 1 -- start start 2023-11-28T11:21:08.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.339+0000 7fb3975ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb37c004850 0x7fb3901e7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:08.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.339+0000 7fb3975ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb390159140 0x7fb39014c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:08.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.339+0000 7fb3975ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3900b3130 0x7fb3900b35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:08.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.339+0000 7fb3975ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3900b3ae0 con 0x7fb390159140 2023-11-28T11:21:08.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.339+0000 7fb3975ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3901e80c0 con 0x7fb3900b3130 2023-11-28T11:21:08.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.339+0000 7fb3975ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3901e83a0 con 0x7fb37c004850 2023-11-28T11:21:08.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.339+0000 7fb394b9a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb390159140 0x7fb39014c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:08.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.339+0000 7fb394b9a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb390159140 0x7fb39014c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60474/0 (socket says 172.21.15.92:60474) 2023-11-28T11:21:08.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.339+0000 7fb394b9a700 1 -- 172.21.15.92:0/2174182466 learned_addr learned my addr 172.21.15.92:0/2174182466 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:08.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.340+0000 7fb395b9c700 1 --2- 172.21.15.92:0/2174182466 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3900b3130 0x7fb3900b35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:08.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.340+0000 7fb39539b700 1 --2- 172.21.15.92:0/2174182466 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb37c004850 0x7fb3901e7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:08.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.340+0000 7fb394b9a700 1 -- 172.21.15.92:0/2174182466 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb37c004850 msgr2=0x7fb3901e7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:08.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.340+0000 7fb394b9a700 1 --2- 172.21.15.92:0/2174182466 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb37c004850 0x7fb3901e7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:08.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.340+0000 7fb394b9a700 1 -- 172.21.15.92:0/2174182466 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3900b3130 msgr2=0x7fb3900b35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:08.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.340+0000 7fb394b9a700 1 --2- 172.21.15.92:0/2174182466 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3900b3130 0x7fb3900b35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:08.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.340+0000 7fb394b9a700 1 -- 172.21.15.92:0/2174182466 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3900b88f0 con 0x7fb390159140 2023-11-28T11:21:08.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.340+0000 7fb39539b700 1 --2- 172.21.15.92:0/2174182466 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb37c004850 0x7fb3901e7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:21:08.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.340+0000 7fb394b9a700 1 --2- 172.21.15.92:0/2174182466 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb390159140 0x7fb39014c060 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb37c004780 tx=0x7fb380001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:08.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.340+0000 7fb385ffb700 1 -- 172.21.15.92:0/2174182466 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3800070d0 con 0x7fb390159140 2023-11-28T11:21:08.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.340+0000 7fb385ffb700 1 -- 172.21.15.92:0/2174182466 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb380007230 con 0x7fb390159140 2023-11-28T11:21:08.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.340+0000 7fb385ffb700 1 -- 172.21.15.92:0/2174182466 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3800075e0 con 0x7fb390159140 2023-11-28T11:21:08.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.340+0000 7fb3975ff700 1 -- 172.21.15.92:0/2174182466 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3900d15e0 con 0x7fb390159140 2023-11-28T11:21:08.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.341+0000 7fb3975ff700 1 -- 172.21.15.92:0/2174182466 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3900b21b0 con 0x7fb390159140 2023-11-28T11:21:08.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.342+0000 7fb3975ff700 1 -- 172.21.15.92:0/2174182466 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb390067170 con 0x7fb390159140 2023-11-28T11:21:08.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.342+0000 7fb385ffb700 1 -- 172.21.15.92:0/2174182466 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb380010070 con 0x7fb390159140 2023-11-28T11:21:08.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.343+0000 7fb385ffb700 1 --2- 172.21.15.92:0/2174182466 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb3740682f0 0x7fb37406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:08.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.343+0000 7fb385ffb700 1 -- 172.21.15.92:0/2174182466 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb380086bc0 con 0x7fb390159140 2023-11-28T11:21:08.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.343+0000 7fb39539b700 1 --2- 172.21.15.92:0/2174182466 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb3740682f0 0x7fb37406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:08.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.344+0000 7fb39539b700 1 --2- 172.21.15.92:0/2174182466 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb3740682f0 0x7fb37406a7a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb38c007fd0 tx=0x7fb38c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:08.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.347+0000 7fb385ffb700 1 -- 172.21.15.92:0/2174182466 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb380053040 con 0x7fb390159140 2023-11-28T11:21:08.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.541+0000 7fb3975ff700 1 -- 172.21.15.92:0/2174182466 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb3900b7e50 con 0x7fb390159140 2023-11-28T11:21:08.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.541+0000 7fb385ffb700 1 -- 172.21.15.92:0/2174182466 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v27) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb380056660 con 0x7fb390159140 2023-11-28T11:21:08.545 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-11-28T11:21:08.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.544+0000 7fb3975ff700 1 -- 172.21.15.92:0/2174182466 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb3740682f0 msgr2=0x7fb37406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:08.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.544+0000 7fb3975ff700 1 --2- 172.21.15.92:0/2174182466 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb3740682f0 0x7fb37406a7a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb38c007fd0 tx=0x7fb38c009040 comp rx=0 tx=0).stop 2023-11-28T11:21:08.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.544+0000 7fb3975ff700 1 -- 172.21.15.92:0/2174182466 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb390159140 msgr2=0x7fb39014c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:08.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.544+0000 7fb3975ff700 1 --2- 172.21.15.92:0/2174182466 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb390159140 0x7fb39014c060 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb37c004780 tx=0x7fb380001980 comp rx=0 tx=0).stop 2023-11-28T11:21:08.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.544+0000 7fb3975ff700 1 -- 172.21.15.92:0/2174182466 shutdown_connections 2023-11-28T11:21:08.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.544+0000 7fb3975ff700 1 --2- 172.21.15.92:0/2174182466 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3900b3130 0x7fb3900b35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:08.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.544+0000 7fb3975ff700 1 --2- 172.21.15.92:0/2174182466 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb37c004850 0x7fb3901e7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:08.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.544+0000 7fb3975ff700 1 --2- 172.21.15.92:0/2174182466 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb3740682f0 0x7fb37406a7a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:08.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.544+0000 7fb3975ff700 1 --2- 172.21.15.92:0/2174182466 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb390159140 0x7fb39014c060 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:08.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.544+0000 7fb3975ff700 1 -- 172.21.15.92:0/2174182466 >> 172.21.15.92:0/2174182466 conn(0x7fb3900719a0 msgr2=0x7fb3900b26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:08.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.544+0000 7fb3975ff700 1 -- 172.21.15.92:0/2174182466 shutdown_connections 2023-11-28T11:21:08.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.544+0000 7fb3975ff700 1 -- 172.21.15.92:0/2174182466 wait complete. 2023-11-28T11:21:08.563 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:21:08.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.761+0000 7fa0ddfff700 1 Processor -- start 2023-11-28T11:21:08.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.762+0000 7fa0ddfff700 1 -- start start 2023-11-28T11:21:08.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.763+0000 7fa0ddfff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa0d80b8510 0x7fa0d806b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:08.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.763+0000 7fa0ddfff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0d80b88e0 con 0x7fa0d80d0d60 2023-11-28T11:21:08.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.763+0000 7fa0ddfff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0d80cd8f0 con 0x7fa0d80cd460 2023-11-28T11:21:08.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.763+0000 7fa0ddfff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0d80cdbd0 con 0x7fa0d80b8510 2023-11-28T11:21:08.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.763+0000 7fa0d7fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fa0d80d0d60 0x7fa0d80d1200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46490/0 (socket says 172.21.15.92:46490) 2023-11-28T11:21:08.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.763+0000 7fa0d7fff700 1 -- 172.21.15.92:0/1652548651 learned_addr learned my addr 172.21.15.92:0/1652548651 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:08.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.763+0000 7fa0d77fe700 1 --2- 172.21.15.92:0/1652548651 >> v2:172.21.15.169:3300/0 conn(0x7fa0d80b8510 0x7fa0d806b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:08.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.763+0000 7fa0d6ffd700 1 -- 172.21.15.92:0/1652548651 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2367464600 0 0) 0x7fa0d80b88e0 con 0x7fa0d80d0d60 2023-11-28T11:21:08.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.763+0000 7fa0d6ffd700 1 -- 172.21.15.92:0/1652548651 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0c40035b0 con 0x7fa0d80d0d60 2023-11-28T11:21:08.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.764+0000 7fa0d77fe700 1 -- 172.21.15.92:0/1652548651 >> v1:172.21.15.169:6789/0 conn(0x7fa0d80cd460 legacy=0x7fa0d806beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:08.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.764+0000 7fa0d77fe700 1 -- 172.21.15.92:0/1652548651 >> v1:172.21.15.92:6789/0 conn(0x7fa0d80d0d60 legacy=0x7fa0d80d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:08.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.764+0000 7fa0d6ffd700 1 -- 172.21.15.92:0/1652548651 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2460861686 0 0) 0x7fa0c40035b0 con 0x7fa0d80d0d60 2023-11-28T11:21:08.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.764+0000 7fa0d77fe700 1 -- 172.21.15.92:0/1652548651 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0d80cfd10 con 0x7fa0d80b8510 2023-11-28T11:21:08.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.764+0000 7fa0d77fe700 1 --2- 172.21.15.92:0/1652548651 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0d80b8510 0x7fa0d806b970 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa0c8009840 tx=0x7fa0c8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0fc3dd29c8f48ab server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:08.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.764+0000 7fa0d6ffd700 1 -- 172.21.15.92:0/1652548651 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d00019f0 con 0x7fa0d80b8510 2023-11-28T11:21:08.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.764+0000 7fa0d6ffd700 1 -- 172.21.15.92:0/1652548651 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0c8004030 con 0x7fa0d80b8510 2023-11-28T11:21:08.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.764+0000 7fa0d6ffd700 1 -- 172.21.15.92:0/1652548651 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c80043c0 con 0x7fa0d80b8510 2023-11-28T11:21:08.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.765+0000 7fa0ddfff700 1 -- 172.21.15.92:0/1652548651 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0d80b8510 msgr2=0x7fa0d806b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:08.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.765+0000 7fa0ddfff700 1 --2- 172.21.15.92:0/1652548651 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0d80b8510 0x7fa0d806b970 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa0c8009840 tx=0x7fa0c8009b50 comp rx=0 tx=0).stop 2023-11-28T11:21:08.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.765+0000 7fa0ddfff700 1 -- 172.21.15.92:0/1652548651 shutdown_connections 2023-11-28T11:21:08.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.765+0000 7fa0ddfff700 1 --2- 172.21.15.92:0/1652548651 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0d80b8510 0x7fa0d806b970 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:08.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.765+0000 7fa0ddfff700 1 -- 172.21.15.92:0/1652548651 >> 172.21.15.92:0/1652548651 conn(0x7fa0d80719a0 msgr2=0x7fa0d8071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:08.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.765+0000 7fa0ddfff700 1 -- 172.21.15.92:0/1652548651 shutdown_connections 2023-11-28T11:21:08.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.765+0000 7fa0ddfff700 1 -- 172.21.15.92:0/1652548651 wait complete. 2023-11-28T11:21:08.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.767+0000 7fa0ddfff700 1 Processor -- start 2023-11-28T11:21:08.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.767+0000 7fa0ddfff700 1 -- start start 2023-11-28T11:21:08.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.768+0000 7fa0ddfff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0d80b8510 0x7fa0d815aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:08.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.768+0000 7fa0ddfff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0d80cd460 0x7fa0d81f2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:08.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.768+0000 7fa0ddfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0d80d0d60 0x7fa0d806b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:08.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.768+0000 7fa0ddfff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d81e36e0 con 0x7fa0d80d0d60 2023-11-28T11:21:08.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.768+0000 7fa0ddfff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d81e3c10 con 0x7fa0d80cd460 2023-11-28T11:21:08.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.768+0000 7fa0ddfff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d81e3eb0 con 0x7fa0d80b8510 2023-11-28T11:21:08.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.768+0000 7fa0d7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0d80d0d60 0x7fa0d806b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:08.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.768+0000 7fa0d7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0d80d0d60 0x7fa0d806b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:60486/0 (socket says 172.21.15.92:60486) 2023-11-28T11:21:08.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.768+0000 7fa0d7fff700 1 -- 172.21.15.92:0/2943366596 learned_addr learned my addr 172.21.15.92:0/2943366596 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:08.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.768+0000 7fa0d77fe700 1 --2- 172.21.15.92:0/2943366596 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0d80b8510 0x7fa0d815aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:08.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.768+0000 7fa0cffff700 1 --2- 172.21.15.92:0/2943366596 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0d80cd460 0x7fa0d81f2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:08.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.768+0000 7fa0d7fff700 1 -- 172.21.15.92:0/2943366596 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0d80b8510 msgr2=0x7fa0d815aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:08.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.768+0000 7fa0d7fff700 1 --2- 172.21.15.92:0/2943366596 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0d80b8510 0x7fa0d815aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:08.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.768+0000 7fa0d7fff700 1 -- 172.21.15.92:0/2943366596 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0d80cd460 msgr2=0x7fa0d81f2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:08.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.768+0000 7fa0d7fff700 1 --2- 172.21.15.92:0/2943366596 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0d80cd460 0x7fa0d81f2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:08.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.769+0000 7fa0d7fff700 1 -- 172.21.15.92:0/2943366596 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c80094a0 con 0x7fa0d80d0d60 2023-11-28T11:21:08.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.769+0000 7fa0d7fff700 1 --2- 172.21.15.92:0/2943366596 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0d80d0d60 0x7fa0d806b970 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa0d000cbe0 tx=0x7fa0d000cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:08.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.769+0000 7fa0d4ff9700 1 -- 172.21.15.92:0/2943366596 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d000aa60 con 0x7fa0d80d0d60 2023-11-28T11:21:08.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.769+0000 7fa0d4ff9700 1 -- 172.21.15.92:0/2943366596 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0d000abe0 con 0x7fa0d80d0d60 2023-11-28T11:21:08.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.769+0000 7fa0ddfff700 1 -- 172.21.15.92:0/2943366596 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0d80d29c0 con 0x7fa0d80d0d60 2023-11-28T11:21:08.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.769+0000 7fa0d4ff9700 1 -- 172.21.15.92:0/2943366596 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d00070d0 con 0x7fa0d80d0d60 2023-11-28T11:21:08.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.769+0000 7fa0ddfff700 1 -- 172.21.15.92:0/2943366596 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0d80cfff0 con 0x7fa0d80d0d60 2023-11-28T11:21:08.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.770+0000 7fa0ddfff700 1 -- 172.21.15.92:0/2943366596 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0d80d2b50 con 0x7fa0d80d0d60 2023-11-28T11:21:08.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.771+0000 7fa0d4ff9700 1 -- 172.21.15.92:0/2943366596 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa0d001a030 con 0x7fa0d80d0d60 2023-11-28T11:21:08.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.772+0000 7fa0d4ff9700 1 --2- 172.21.15.92:0/2943366596 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa0b80682f0 0x7fa0b806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:08.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.772+0000 7fa0d4ff9700 1 -- 172.21.15.92:0/2943366596 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa0d0085c20 con 0x7fa0d80d0d60 2023-11-28T11:21:08.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.772+0000 7fa0d77fe700 1 --2- 172.21.15.92:0/2943366596 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa0b80682f0 0x7fa0b806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:08.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.773+0000 7fa0d77fe700 1 --2- 172.21.15.92:0/2943366596 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa0b80682f0 0x7fa0b806a7a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa0c80090b0 tx=0x7fa0c8008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:08.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.776+0000 7fa0d4ff9700 1 -- 172.21.15.92:0/2943366596 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa0d00520a0 con 0x7fa0d80d0d60 2023-11-28T11:21:08.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.966+0000 7fa0ddfff700 1 -- 172.21.15.92:0/2943366596 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa0d8067880 con 0x7fa0d80d0d60 2023-11-28T11:21:08.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.967+0000 7fa0d4ff9700 1 -- 172.21.15.92:0/2943366596 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa0d0018020 con 0x7fa0d80d0d60 2023-11-28T11:21:08.968 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-11-28T11:21:08.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.969+0000 7fa0ddfff700 1 -- 172.21.15.92:0/2943366596 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa0b80682f0 msgr2=0x7fa0b806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:08.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.969+0000 7fa0ddfff700 1 --2- 172.21.15.92:0/2943366596 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa0b80682f0 0x7fa0b806a7a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa0c80090b0 tx=0x7fa0c8008030 comp rx=0 tx=0).stop 2023-11-28T11:21:08.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.969+0000 7fa0ddfff700 1 -- 172.21.15.92:0/2943366596 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0d80d0d60 msgr2=0x7fa0d806b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:08.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.969+0000 7fa0ddfff700 1 --2- 172.21.15.92:0/2943366596 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0d80d0d60 0x7fa0d806b970 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa0d000cbe0 tx=0x7fa0d000cef0 comp rx=0 tx=0).stop 2023-11-28T11:21:08.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.969+0000 7fa0ddfff700 1 -- 172.21.15.92:0/2943366596 shutdown_connections 2023-11-28T11:21:08.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.969+0000 7fa0ddfff700 1 --2- 172.21.15.92:0/2943366596 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa0b80682f0 0x7fa0b806a7a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:08.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.969+0000 7fa0ddfff700 1 --2- 172.21.15.92:0/2943366596 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0d80d0d60 0x7fa0d806b970 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:08.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.969+0000 7fa0ddfff700 1 --2- 172.21.15.92:0/2943366596 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0d80b8510 0x7fa0d815aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:08.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.969+0000 7fa0ddfff700 1 --2- 172.21.15.92:0/2943366596 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0d80cd460 0x7fa0d81f2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:08.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.969+0000 7fa0ddfff700 1 -- 172.21.15.92:0/2943366596 >> 172.21.15.92:0/2943366596 conn(0x7fa0d80719a0 msgr2=0x7fa0d80b6bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:08.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.969+0000 7fa0ddfff700 1 -- 172.21.15.92:0/2943366596 shutdown_connections 2023-11-28T11:21:08.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:08.969+0000 7fa0ddfff700 1 -- 172.21.15.92:0/2943366596 wait complete. 2023-11-28T11:21:08.988 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:21:09.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.186+0000 7f69de9ff700 1 Processor -- start 2023-11-28T11:21:09.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.187+0000 7f69de9ff700 1 -- start start 2023-11-28T11:21:09.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.188+0000 7f69de9ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f69d80d2df0 0x7f69d80cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:09.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.188+0000 7f69de9ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f69d80b1c60 0x7f69d80cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:09.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.188+0000 7f69de9ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69d80d1950 con 0x7f69d80b4fc0 2023-11-28T11:21:09.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.188+0000 7f69de9ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69d80d3d30 con 0x7f69d80d2df0 2023-11-28T11:21:09.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.188+0000 7f69de9ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69d80b2500 con 0x7f69d80b1c60 2023-11-28T11:21:09.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.188+0000 7f69d7fff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f69d80d2df0 0x7f69d80cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:09.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.188+0000 7f69dcf9c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f69d80b1c60 0x7f69d80cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:09.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.188+0000 7f69dcf9c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f69d80b1c60 0x7f69d80cf010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.92:59148/0 (socket says 172.21.15.92:59148) 2023-11-28T11:21:09.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.188+0000 7f69d7fff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f69d80d2df0 0x7f69d80cd620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.92:49624/0 (socket says 172.21.15.92:49624) 2023-11-28T11:21:09.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.188+0000 7f69cffff700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f69d80b4fc0 0x7f69d80d31c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.92:60306/0 (socket says 172.21.15.92:60306) 2023-11-28T11:21:09.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.188+0000 7f69dcf9c700 1 -- 172.21.15.92:0/791654813 learned_addr learned my addr 172.21.15.92:0/791654813 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:09.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.189+0000 7f69d7fff700 1 -- 172.21.15.92:0/791654813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f69d80b1c60 msgr2=0x7f69d80cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:09.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.189+0000 7f69d7fff700 1 --2- 172.21.15.92:0/791654813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f69d80b1c60 0x7f69d80cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:09.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.189+0000 7f69d77fe700 1 -- 172.21.15.92:0/791654813 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1631701320 0 0) 0x7f69d80d1950 con 0x7f69d80b4fc0 2023-11-28T11:21:09.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.189+0000 7f69d7fff700 1 -- 172.21.15.92:0/791654813 >> v1:172.21.15.169:6789/0 conn(0x7f69d80b4fc0 legacy=0x7f69d80d31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:09.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.189+0000 7f69d7fff700 1 -- 172.21.15.92:0/791654813 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d80d3a50 con 0x7f69d80d2df0 2023-11-28T11:21:09.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.189+0000 7f69dcf9c700 1 --2- 172.21.15.92:0/791654813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f69d80b1c60 0x7f69d80cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:21:09.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.189+0000 7f69d7fff700 1 --2- 172.21.15.92:0/791654813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f69d80d2df0 0x7f69d80cd620 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f69c8009840 tx=0x7f69c8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef175ae01044ac22 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:09.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.189+0000 7f69d77fe700 1 -- 172.21.15.92:0/791654813 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c8011830 con 0x7f69d80d2df0 2023-11-28T11:21:09.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.189+0000 7f69d77fe700 1 -- 172.21.15.92:0/791654813 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69c8011990 con 0x7f69d80d2df0 2023-11-28T11:21:09.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.189+0000 7f69d77fe700 1 -- 172.21.15.92:0/791654813 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c8011d20 con 0x7f69d80d2df0 2023-11-28T11:21:09.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.190+0000 7f69de9ff700 1 -- 172.21.15.92:0/791654813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f69d80d2df0 msgr2=0x7f69d80cd620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:09.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.190+0000 7f69de9ff700 1 --2- 172.21.15.92:0/791654813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f69d80d2df0 0x7f69d80cd620 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f69c8009840 tx=0x7f69c8009b50 comp rx=0 tx=0).stop 2023-11-28T11:21:09.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.190+0000 7f69de9ff700 1 -- 172.21.15.92:0/791654813 shutdown_connections 2023-11-28T11:21:09.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.190+0000 7f69de9ff700 1 --2- 172.21.15.92:0/791654813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f69d80d2df0 0x7f69d80cd620 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:09.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.190+0000 7f69de9ff700 1 --2- 172.21.15.92:0/791654813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f69d80b1c60 0x7f69d80cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:09.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.190+0000 7f69de9ff700 1 -- 172.21.15.92:0/791654813 >> 172.21.15.92:0/791654813 conn(0x7f69d80719a0 msgr2=0x7f69d8071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:09.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.190+0000 7f69de9ff700 1 -- 172.21.15.92:0/791654813 shutdown_connections 2023-11-28T11:21:09.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.190+0000 7f69de9ff700 1 -- 172.21.15.92:0/791654813 wait complete. 2023-11-28T11:21:09.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.192+0000 7f69de9ff700 1 Processor -- start 2023-11-28T11:21:09.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.192+0000 7f69de9ff700 1 -- start start 2023-11-28T11:21:09.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.193+0000 7f69de9ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69d80b1c60 0x7f69d815eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:09.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.193+0000 7f69de9ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f69d80b4fc0 0x7f69d815d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:09.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.193+0000 7f69de9ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f69d80d2df0 0x7f69d815d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:09.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.193+0000 7f69de9ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d814b220 con 0x7f69d80b1c60 2023-11-28T11:21:09.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.193+0000 7f69de9ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d814b500 con 0x7f69d80d2df0 2023-11-28T11:21:09.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.193+0000 7f69de9ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69d814b7e0 con 0x7f69d80b4fc0 2023-11-28T11:21:09.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.193+0000 7f69d7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69d80b1c60 0x7f69d815eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:09.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.193+0000 7f69d7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69d80b1c60 0x7f69d815eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48824/0 (socket says 172.21.15.92:48824) 2023-11-28T11:21:09.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.193+0000 7f69d7fff700 1 -- 172.21.15.92:0/3816717992 learned_addr learned my addr 172.21.15.92:0/3816717992 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:09.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.193+0000 7f69cffff700 1 --2- 172.21.15.92:0/3816717992 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f69d80b4fc0 0x7f69d815d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:09.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.193+0000 7f69dcf9c700 1 --2- 172.21.15.92:0/3816717992 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f69d80d2df0 0x7f69d815d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:09.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.193+0000 7f69d7fff700 1 -- 172.21.15.92:0/3816717992 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f69d80b4fc0 msgr2=0x7f69d815d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:09.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.193+0000 7f69d7fff700 1 --2- 172.21.15.92:0/3816717992 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f69d80b4fc0 0x7f69d815d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:09.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.193+0000 7f69d7fff700 1 -- 172.21.15.92:0/3816717992 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f69d80d2df0 msgr2=0x7f69d815d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:09.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.193+0000 7f69d7fff700 1 --2- 172.21.15.92:0/3816717992 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f69d80d2df0 0x7f69d815d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:09.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.193+0000 7f69d7fff700 1 -- 172.21.15.92:0/3816717992 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d0009b30 con 0x7f69d80b1c60 2023-11-28T11:21:09.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.193+0000 7f69dcf9c700 1 --2- 172.21.15.92:0/3816717992 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f69d80d2df0 0x7f69d815d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:21:09.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.194+0000 7f69d7fff700 1 --2- 172.21.15.92:0/3816717992 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69d80b1c60 0x7f69d815eaf0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f69c8000c00 tx=0x7f69c8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:09.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.194+0000 7f69d57fa700 1 -- 172.21.15.92:0/3816717992 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c80038a0 con 0x7f69d80b1c60 2023-11-28T11:21:09.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.194+0000 7f69d57fa700 1 -- 172.21.15.92:0/3816717992 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69c801f070 con 0x7f69d80b1c60 2023-11-28T11:21:09.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.194+0000 7f69de9ff700 1 -- 172.21.15.92:0/3816717992 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69c80094a0 con 0x7f69d80b1c60 2023-11-28T11:21:09.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.194+0000 7f69d57fa700 1 -- 172.21.15.92:0/3816717992 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c8003c00 con 0x7f69d80b1c60 2023-11-28T11:21:09.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.194+0000 7f69de9ff700 1 -- 172.21.15.92:0/3816717992 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69d80b88f0 con 0x7f69d80b1c60 2023-11-28T11:21:09.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.195+0000 7f69de9ff700 1 -- 172.21.15.92:0/3816717992 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69d80d15e0 con 0x7f69d80b1c60 2023-11-28T11:21:09.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.196+0000 7f69d57fa700 1 -- 172.21.15.92:0/3816717992 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f69c800f050 con 0x7f69d80b1c60 2023-11-28T11:21:09.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.197+0000 7f69d57fa700 1 --2- 172.21.15.92:0/3816717992 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f69b80682f0 0x7f69b806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:09.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.197+0000 7f69d57fa700 1 -- 172.21.15.92:0/3816717992 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f69c8021070 con 0x7f69d80b1c60 2023-11-28T11:21:09.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.197+0000 7f69cffff700 1 --2- 172.21.15.92:0/3816717992 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f69b80682f0 0x7f69b806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:09.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.197+0000 7f69cffff700 1 --2- 172.21.15.92:0/3816717992 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f69b80682f0 0x7f69b806a7a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f69d815ac40 tx=0x7f69c0007c90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:09.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.201+0000 7f69d57fa700 1 -- 172.21.15.92:0/3816717992 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f69c8053e40 con 0x7f69d80b1c60 2023-11-28T11:21:09.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.393+0000 7f69de9ff700 1 -- 172.21.15.92:0/3816717992 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f69d80b21b0 con 0x7f69d80b1c60 2023-11-28T11:21:09.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.393+0000 7f69d57fa700 1 -- 172.21.15.92:0/3816717992 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f69c802b0a0 con 0x7f69d80b1c60 2023-11-28T11:21:09.395 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-11-28T11:21:09.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.395+0000 7f69de9ff700 1 -- 172.21.15.92:0/3816717992 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f69b80682f0 msgr2=0x7f69b806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:09.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.395+0000 7f69de9ff700 1 --2- 172.21.15.92:0/3816717992 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f69b80682f0 0x7f69b806a7a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f69d815ac40 tx=0x7f69c0007c90 comp rx=0 tx=0).stop 2023-11-28T11:21:09.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.395+0000 7f69de9ff700 1 -- 172.21.15.92:0/3816717992 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69d80b1c60 msgr2=0x7f69d815eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:09.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.395+0000 7f69de9ff700 1 --2- 172.21.15.92:0/3816717992 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69d80b1c60 0x7f69d815eaf0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f69c8000c00 tx=0x7f69c8004750 comp rx=0 tx=0).stop 2023-11-28T11:21:09.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.395+0000 7f69de9ff700 1 -- 172.21.15.92:0/3816717992 shutdown_connections 2023-11-28T11:21:09.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.395+0000 7f69de9ff700 1 --2- 172.21.15.92:0/3816717992 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f69d80d2df0 0x7f69d815d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:09.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.395+0000 7f69de9ff700 1 --2- 172.21.15.92:0/3816717992 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f69b80682f0 0x7f69b806a7a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:09.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.395+0000 7f69de9ff700 1 --2- 172.21.15.92:0/3816717992 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69d80b1c60 0x7f69d815eaf0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:09.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.395+0000 7f69de9ff700 1 --2- 172.21.15.92:0/3816717992 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f69d80b4fc0 0x7f69d815d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:09.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.395+0000 7f69de9ff700 1 -- 172.21.15.92:0/3816717992 >> 172.21.15.92:0/3816717992 conn(0x7f69d80719a0 msgr2=0x7f69d80cbf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:09.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.396+0000 7f69de9ff700 1 -- 172.21.15.92:0/3816717992 shutdown_connections 2023-11-28T11:21:09.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.396+0000 7f69de9ff700 1 -- 172.21.15.92:0/3816717992 wait complete. 2023-11-28T11:21:09.415 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T11:21:09.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.614+0000 7f09783ff700 1 Processor -- start 2023-11-28T11:21:09.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.614+0000 7f09783ff700 1 -- start start 2023-11-28T11:21:09.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.615+0000 7f09783ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f09700b8510 0x7f097006b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:09.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.615+0000 7f09783ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f09700d0d60 0x7f09700d11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:09.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.615+0000 7f09783ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09700b88e0 con 0x7f09700cd460 2023-11-28T11:21:09.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.615+0000 7f09783ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09700cdbd0 con 0x7f09700d0d60 2023-11-28T11:21:09.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.615+0000 7f09783ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09700d02d0 con 0x7f09700b8510 2023-11-28T11:21:09.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.616+0000 7f097599a700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f09700cd460 0x7f097006beb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46114/0 (socket says 172.21.15.92:46114) 2023-11-28T11:21:09.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.616+0000 7f097599a700 1 -- 172.21.15.92:0/3355683762 learned_addr learned my addr 172.21.15.92:0/3355683762 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:09.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.616+0000 7f097699c700 1 --2- 172.21.15.92:0/3355683762 >> v2:172.21.15.92:3300/0 conn(0x7f09700d0d60 0x7f09700d11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:09.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.616+0000 7f097619b700 1 --2- 172.21.15.92:0/3355683762 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f09700b8510 0x7f097006b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:09.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.616+0000 7f097699c700 1 -- 172.21.15.92:0/3355683762 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f09700b8510 msgr2=0x7f097006b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:09.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.616+0000 7f097699c700 1 --2- 172.21.15.92:0/3355683762 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f09700b8510 0x7f097006b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:09.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.616+0000 7f097699c700 1 -- 172.21.15.92:0/3355683762 >> v1:172.21.15.92:6789/0 conn(0x7f09700cd460 legacy=0x7f097006beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:09.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.616+0000 7f097699c700 1 -- 172.21.15.92:0/3355683762 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09700cd8f0 con 0x7f09700d0d60 2023-11-28T11:21:09.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.616+0000 7f097619b700 1 --2- 172.21.15.92:0/3355683762 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f09700b8510 0x7f097006b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:21:09.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.616+0000 7f097699c700 1 --2- 172.21.15.92:0/3355683762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09700d0d60 0x7f09700d11a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0968009dc0 tx=0x7f096800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6277359bf6e60057 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:09.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.617+0000 7f0975199700 1 -- 172.21.15.92:0/3355683762 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f096800e070 con 0x7f09700d0d60 2023-11-28T11:21:09.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.617+0000 7f0975199700 1 -- 172.21.15.92:0/3355683762 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f096800bb90 con 0x7f09700d0d60 2023-11-28T11:21:09.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.617+0000 7f0975199700 1 -- 172.21.15.92:0/3355683762 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0968012430 con 0x7f09700d0d60 2023-11-28T11:21:09.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.617+0000 7f09783ff700 1 -- 172.21.15.92:0/3355683762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09700d0d60 msgr2=0x7f09700d11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:09.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.617+0000 7f09783ff700 1 --2- 172.21.15.92:0/3355683762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09700d0d60 0x7f09700d11a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0968009dc0 tx=0x7f096800b330 comp rx=0 tx=0).stop 2023-11-28T11:21:09.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.617+0000 7f09783ff700 1 -- 172.21.15.92:0/3355683762 shutdown_connections 2023-11-28T11:21:09.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.617+0000 7f09783ff700 1 --2- 172.21.15.92:0/3355683762 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f09700b8510 0x7f097006b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:09.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.617+0000 7f09783ff700 1 --2- 172.21.15.92:0/3355683762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09700d0d60 0x7f09700d11a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:09.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.617+0000 7f09783ff700 1 -- 172.21.15.92:0/3355683762 >> 172.21.15.92:0/3355683762 conn(0x7f09700719a0 msgr2=0x7f0970071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:09.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.617+0000 7f09783ff700 1 -- 172.21.15.92:0/3355683762 shutdown_connections 2023-11-28T11:21:09.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.617+0000 7f09783ff700 1 -- 172.21.15.92:0/3355683762 wait complete. 2023-11-28T11:21:09.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.619+0000 7f09783ff700 1 Processor -- start 2023-11-28T11:21:09.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.619+0000 7f09783ff700 1 -- start start 2023-11-28T11:21:09.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.620+0000 7f09783ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09700b8510 0x7f09701f1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:09.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.620+0000 7f09783ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f09700cd460 0x7f0970156500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:09.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.620+0000 7f09783ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09700d0d60 0x7f09701f5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:09.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.620+0000 7f09783ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0970156070 con 0x7f09700b8510 2023-11-28T11:21:09.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.620+0000 7f09783ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0970155460 con 0x7f09700d0d60 2023-11-28T11:21:09.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.620+0000 7f09783ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0970155740 con 0x7f09700cd460 2023-11-28T11:21:09.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.620+0000 7f097619b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09700b8510 0x7f09701f1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:09.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.620+0000 7f097619b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09700b8510 0x7f09701f1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48842/0 (socket says 172.21.15.92:48842) 2023-11-28T11:21:09.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.620+0000 7f097619b700 1 -- 172.21.15.92:0/1796523339 learned_addr learned my addr 172.21.15.92:0/1796523339 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:09.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.621+0000 7f097619b700 1 -- 172.21.15.92:0/1796523339 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f09700cd460 msgr2=0x7f0970156500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:09.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.621+0000 7f097599a700 1 --2- 172.21.15.92:0/1796523339 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f09700cd460 0x7f0970156500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:09.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.621+0000 7f097699c700 1 --2- 172.21.15.92:0/1796523339 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09700d0d60 0x7f09701f5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:09.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.621+0000 7f097619b700 1 --2- 172.21.15.92:0/1796523339 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f09700cd460 0x7f0970156500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:09.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.621+0000 7f097619b700 1 -- 172.21.15.92:0/1796523339 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09700d0d60 msgr2=0x7f09701f5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:09.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.621+0000 7f097619b700 1 --2- 172.21.15.92:0/1796523339 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09700d0d60 0x7f09701f5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:09.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.621+0000 7f097619b700 1 -- 172.21.15.92:0/1796523339 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0968009a70 con 0x7f09700b8510 2023-11-28T11:21:09.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.621+0000 7f097599a700 1 --2- 172.21.15.92:0/1796523339 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f09700cd460 0x7f0970156500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:21:09.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.621+0000 7f097699c700 1 --2- 172.21.15.92:0/1796523339 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09700d0d60 0x7f09701f5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:21:09.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.621+0000 7f097619b700 1 --2- 172.21.15.92:0/1796523339 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09700b8510 0x7f09701f1f60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f096c00e810 tx=0x7f096c00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:09.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.621+0000 7f0966ffd700 1 -- 172.21.15.92:0/1796523339 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f096c004230 con 0x7f09700b8510 2023-11-28T11:21:09.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.621+0000 7f0966ffd700 1 -- 172.21.15.92:0/1796523339 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f096c004390 con 0x7f09700b8510 2023-11-28T11:21:09.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.621+0000 7f09783ff700 1 -- 172.21.15.92:0/1796523339 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09700cfff0 con 0x7f09700b8510 2023-11-28T11:21:09.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.621+0000 7f0966ffd700 1 -- 172.21.15.92:0/1796523339 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f096c0036a0 con 0x7f09700b8510 2023-11-28T11:21:09.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.621+0000 7f09783ff700 1 -- 172.21.15.92:0/1796523339 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09700cfd10 con 0x7f09700b8510 2023-11-28T11:21:09.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.622+0000 7f09783ff700 1 -- 172.21.15.92:0/1796523339 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0970067880 con 0x7f09700b8510 2023-11-28T11:21:09.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.623+0000 7f0966ffd700 1 -- 172.21.15.92:0/1796523339 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f096c022070 con 0x7f09700b8510 2023-11-28T11:21:09.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.625+0000 7f0966ffd700 1 --2- 172.21.15.92:0/1796523339 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f09540682f0 0x7f095406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:09.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.625+0000 7f0966ffd700 1 -- 172.21.15.92:0/1796523339 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f096c085ba0 con 0x7f09700b8510 2023-11-28T11:21:09.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.625+0000 7f097599a700 1 --2- 172.21.15.92:0/1796523339 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f09540682f0 0x7f095406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:09.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.625+0000 7f097599a700 1 --2- 172.21.15.92:0/1796523339 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f09540682f0 0x7f095406a7a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f097006c3e0 tx=0x7f0960009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:09.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.629+0000 7f0966ffd700 1 -- 172.21.15.92:0/1796523339 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f096c051fc0 con 0x7f09700b8510 2023-11-28T11:21:09.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.821+0000 7f09783ff700 1 -- 172.21.15.92:0/1796523339 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f09700d2940 con 0x7f09700b8510 2023-11-28T11:21:09.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.821+0000 7f0966ffd700 1 -- 172.21.15.92:0/1796523339 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f096c0555e0 con 0x7f09700b8510 2023-11-28T11:21:09.822 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-11-28T11:21:09.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.824+0000 7f09783ff700 1 -- 172.21.15.92:0/1796523339 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f09540682f0 msgr2=0x7f095406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:09.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.824+0000 7f09783ff700 1 --2- 172.21.15.92:0/1796523339 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f09540682f0 0x7f095406a7a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f097006c3e0 tx=0x7f0960009040 comp rx=0 tx=0).stop 2023-11-28T11:21:09.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.824+0000 7f09783ff700 1 -- 172.21.15.92:0/1796523339 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09700b8510 msgr2=0x7f09701f1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:09.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.824+0000 7f09783ff700 1 --2- 172.21.15.92:0/1796523339 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09700b8510 0x7f09701f1f60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f096c00e810 tx=0x7f096c00ebd0 comp rx=0 tx=0).stop 2023-11-28T11:21:09.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.824+0000 7f09783ff700 1 -- 172.21.15.92:0/1796523339 shutdown_connections 2023-11-28T11:21:09.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.824+0000 7f09783ff700 1 --2- 172.21.15.92:0/1796523339 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09700d0d60 0x7f09701f5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:09.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.824+0000 7f09783ff700 1 --2- 172.21.15.92:0/1796523339 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f09540682f0 0x7f095406a7a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:09.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.824+0000 7f09783ff700 1 --2- 172.21.15.92:0/1796523339 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09700b8510 0x7f09701f1f60 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:09.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.824+0000 7f09783ff700 1 --2- 172.21.15.92:0/1796523339 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f09700cd460 0x7f0970156500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:09.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.824+0000 7f09783ff700 1 -- 172.21.15.92:0/1796523339 >> 172.21.15.92:0/1796523339 conn(0x7f09700719a0 msgr2=0x7f09700b27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:09.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.824+0000 7f09783ff700 1 -- 172.21.15.92:0/1796523339 shutdown_connections 2023-11-28T11:21:09.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:09.824+0000 7f09783ff700 1 -- 172.21.15.92:0/1796523339 wait complete. 2023-11-28T11:21:09.843 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T11:21:10.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.039+0000 7f3c859ff700 1 Processor -- start 2023-11-28T11:21:10.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.039+0000 7f3c859ff700 1 -- start start 2023-11-28T11:21:10.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.040+0000 7f3c859ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3c800d2df0 0x7f3c800cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:10.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.040+0000 7f3c859ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3c800b4fc0 0x7f3c800d31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:10.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.040+0000 7f3c859ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3c800b1c60 0x7f3c800cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:10.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.040+0000 7f3c859ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c800d3d30 con 0x7f3c800d2df0 2023-11-28T11:21:10.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.040+0000 7f3c859ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c800d1950 con 0x7f3c800b1c60 2023-11-28T11:21:10.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.040+0000 7f3c859ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c800b2500 con 0x7f3c800b4fc0 2023-11-28T11:21:10.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.040+0000 7f3c7effd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3c800d2df0 0x7f3c800cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:10.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.040+0000 7f3c7effd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3c800d2df0 0x7f3c800cd620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48852/0 (socket says 172.21.15.92:48852) 2023-11-28T11:21:10.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.040+0000 7f3c7effd700 1 -- 172.21.15.92:0/2572703551 learned_addr learned my addr 172.21.15.92:0/2572703551 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:10.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.041+0000 7f3c7e7fc700 1 --2- 172.21.15.92:0/2572703551 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3c800b4fc0 0x7f3c800d31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:10.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.041+0000 7f3c7f7fe700 1 --2- 172.21.15.92:0/2572703551 >> v2:172.21.15.169:3300/0 conn(0x7f3c800b1c60 0x7f3c800cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:10.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.041+0000 7f3c7effd700 1 -- 172.21.15.92:0/2572703551 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3c800b4fc0 msgr2=0x7f3c800d31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:10.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.041+0000 7f3c7effd700 1 --2- 172.21.15.92:0/2572703551 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3c800b4fc0 0x7f3c800d31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:10.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.041+0000 7f3c7effd700 1 -- 172.21.15.92:0/2572703551 >> v2:172.21.15.169:3300/0 conn(0x7f3c800b1c60 msgr2=0x7f3c800cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:10.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.041+0000 7f3c7effd700 1 --2- 172.21.15.92:0/2572703551 >> v2:172.21.15.169:3300/0 conn(0x7f3c800b1c60 0x7f3c800cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:10.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.041+0000 7f3c7effd700 1 -- 172.21.15.92:0/2572703551 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c800d3a50 con 0x7f3c800d2df0 2023-11-28T11:21:10.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.041+0000 7f3c7f7fe700 1 --2- 172.21.15.92:0/2572703551 >> v2:172.21.15.169:3300/0 conn(0x7f3c800b1c60 0x7f3c800cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:21:10.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.041+0000 7f3c7e7fc700 1 --2- 172.21.15.92:0/2572703551 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3c800b4fc0 0x7f3c800d31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:21:10.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.041+0000 7f3c7effd700 1 --2- 172.21.15.92:0/2572703551 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c800d2df0 0x7f3c800cd620 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3c8008c2e0 tx=0x7f3c70009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fd41034f79999af server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:10.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.041+0000 7f3c7dffb700 1 -- 172.21.15.92:0/2572703551 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c7000b870 con 0x7f3c800d2df0 2023-11-28T11:21:10.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.041+0000 7f3c7dffb700 1 -- 172.21.15.92:0/2572703551 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c7000b9d0 con 0x7f3c800d2df0 2023-11-28T11:21:10.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.041+0000 7f3c7dffb700 1 -- 172.21.15.92:0/2572703551 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c7000bd60 con 0x7f3c800d2df0 2023-11-28T11:21:10.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.042+0000 7f3c859ff700 1 -- 172.21.15.92:0/2572703551 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c800d2df0 msgr2=0x7f3c800cd620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:10.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.042+0000 7f3c859ff700 1 --2- 172.21.15.92:0/2572703551 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c800d2df0 0x7f3c800cd620 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3c8008c2e0 tx=0x7f3c70009bd0 comp rx=0 tx=0).stop 2023-11-28T11:21:10.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.042+0000 7f3c859ff700 1 -- 172.21.15.92:0/2572703551 shutdown_connections 2023-11-28T11:21:10.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.042+0000 7f3c859ff700 1 --2- 172.21.15.92:0/2572703551 >> v2:172.21.15.169:3300/0 conn(0x7f3c800b1c60 0x7f3c800cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:10.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.042+0000 7f3c859ff700 1 --2- 172.21.15.92:0/2572703551 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c800d2df0 0x7f3c800cd620 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:10.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.042+0000 7f3c859ff700 1 --2- 172.21.15.92:0/2572703551 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3c800b4fc0 0x7f3c800d31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:10.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.042+0000 7f3c859ff700 1 -- 172.21.15.92:0/2572703551 >> 172.21.15.92:0/2572703551 conn(0x7f3c800719a0 msgr2=0x7f3c80071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:10.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.042+0000 7f3c859ff700 1 -- 172.21.15.92:0/2572703551 shutdown_connections 2023-11-28T11:21:10.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.042+0000 7f3c859ff700 1 -- 172.21.15.92:0/2572703551 wait complete. 2023-11-28T11:21:10.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.044+0000 7f3c859ff700 1 Processor -- start 2023-11-28T11:21:10.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.044+0000 7f3c859ff700 1 -- start start 2023-11-28T11:21:10.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.045+0000 7f3c859ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c800b1c60 0x7f3c801f22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:10.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.045+0000 7f3c859ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c800b4fc0 0x7f3c80158570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:10.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.045+0000 7f3c859ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3c800d2df0 0x7f3c801e4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:10.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.045+0000 7f3c859ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c801f27e0 con 0x7f3c800b1c60 2023-11-28T11:21:10.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.045+0000 7f3c859ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c80158ab0 con 0x7f3c800b4fc0 2023-11-28T11:21:10.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.045+0000 7f3c859ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8015ddc0 con 0x7f3c800d2df0 2023-11-28T11:21:10.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.045+0000 7f3c7effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c800b1c60 0x7f3c801f22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:10.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.045+0000 7f3c7effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c800b1c60 0x7f3c801f22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48866/0 (socket says 172.21.15.92:48866) 2023-11-28T11:21:10.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.045+0000 7f3c7effd700 1 -- 172.21.15.92:0/1070299911 learned_addr learned my addr 172.21.15.92:0/1070299911 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:10.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.045+0000 7f3c7e7fc700 1 --2- 172.21.15.92:0/1070299911 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c800b4fc0 0x7f3c80158570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:10.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.045+0000 7f3c7effd700 1 -- 172.21.15.92:0/1070299911 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3c800d2df0 msgr2=0x7f3c801e4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:10.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.045+0000 7f3c7f7fe700 1 --2- 172.21.15.92:0/1070299911 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3c800d2df0 0x7f3c801e4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:10.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.045+0000 7f3c7effd700 1 --2- 172.21.15.92:0/1070299911 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3c800d2df0 0x7f3c801e4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:10.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.045+0000 7f3c7effd700 1 -- 172.21.15.92:0/1070299911 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c800b4fc0 msgr2=0x7f3c80158570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:10.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.045+0000 7f3c7effd700 1 --2- 172.21.15.92:0/1070299911 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c800b4fc0 0x7f3c80158570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:10.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.045+0000 7f3c7effd700 1 -- 172.21.15.92:0/1070299911 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c70009550 con 0x7f3c800b1c60 2023-11-28T11:21:10.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.045+0000 7f3c7e7fc700 1 --2- 172.21.15.92:0/1070299911 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c800b4fc0 0x7f3c80158570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:21:10.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.045+0000 7f3c7f7fe700 1 --2- 172.21.15.92:0/1070299911 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3c800d2df0 0x7f3c801e4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:21:10.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.046+0000 7f3c7effd700 1 --2- 172.21.15.92:0/1070299911 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c800b1c60 0x7f3c801f22a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3c70014040 tx=0x7f3c70004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:10.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.046+0000 7f3c67fff700 1 -- 172.21.15.92:0/1070299911 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c70004520 con 0x7f3c800b1c60 2023-11-28T11:21:10.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.046+0000 7f3c67fff700 1 -- 172.21.15.92:0/1070299911 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c7001e070 con 0x7f3c800b1c60 2023-11-28T11:21:10.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.046+0000 7f3c859ff700 1 -- 172.21.15.92:0/1070299911 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c800b88f0 con 0x7f3c800b1c60 2023-11-28T11:21:10.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.046+0000 7f3c67fff700 1 -- 172.21.15.92:0/1070299911 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c700038e0 con 0x7f3c800b1c60 2023-11-28T11:21:10.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.046+0000 7f3c859ff700 1 -- 172.21.15.92:0/1070299911 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c800d15e0 con 0x7f3c800b1c60 2023-11-28T11:21:10.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.047+0000 7f3c859ff700 1 -- 172.21.15.92:0/1070299911 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c800b21b0 con 0x7f3c800b1c60 2023-11-28T11:21:10.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.048+0000 7f3c67fff700 1 -- 172.21.15.92:0/1070299911 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3c7000f050 con 0x7f3c800b1c60 2023-11-28T11:21:10.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.049+0000 7f3c67fff700 1 --2- 172.21.15.92:0/1070299911 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3c6c0682f0 0x7f3c6c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:10.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.049+0000 7f3c67fff700 1 -- 172.21.15.92:0/1070299911 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3c70020070 con 0x7f3c800b1c60 2023-11-28T11:21:10.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.049+0000 7f3c7e7fc700 1 --2- 172.21.15.92:0/1070299911 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3c6c0682f0 0x7f3c6c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:10.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.049+0000 7f3c7e7fc700 1 --2- 172.21.15.92:0/1070299911 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3c6c0682f0 0x7f3c6c06a7a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3c800c96f0 tx=0x7f3c74008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:10.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.053+0000 7f3c67fff700 1 -- 172.21.15.92:0/1070299911 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3c70053b70 con 0x7f3c800b1c60 2023-11-28T11:21:10.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.258+0000 7f3c859ff700 1 -- 172.21.15.92:0/1070299911 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3c80067170 con 0x7f3c800b1c60 2023-11-28T11:21:10.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.258+0000 7f3c67fff700 1 -- 172.21.15.92:0/1070299911 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f3c70057190 con 0x7f3c800b1c60 2023-11-28T11:21:10.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.260+0000 7f3c859ff700 1 -- 172.21.15.92:0/1070299911 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3c6c0682f0 msgr2=0x7f3c6c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:10.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.260+0000 7f3c859ff700 1 --2- 172.21.15.92:0/1070299911 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3c6c0682f0 0x7f3c6c06a7a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3c800c96f0 tx=0x7f3c74008040 comp rx=0 tx=0).stop 2023-11-28T11:21:10.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.260+0000 7f3c859ff700 1 -- 172.21.15.92:0/1070299911 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c800b1c60 msgr2=0x7f3c801f22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:10.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.260+0000 7f3c859ff700 1 --2- 172.21.15.92:0/1070299911 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c800b1c60 0x7f3c801f22a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3c70014040 tx=0x7f3c70004750 comp rx=0 tx=0).stop 2023-11-28T11:21:10.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.260+0000 7f3c859ff700 1 -- 172.21.15.92:0/1070299911 shutdown_connections 2023-11-28T11:21:10.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.260+0000 7f3c859ff700 1 --2- 172.21.15.92:0/1070299911 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3c800d2df0 0x7f3c801e4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:10.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.260+0000 7f3c859ff700 1 --2- 172.21.15.92:0/1070299911 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3c6c0682f0 0x7f3c6c06a7a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:10.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.260+0000 7f3c859ff700 1 --2- 172.21.15.92:0/1070299911 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c800b1c60 0x7f3c801f22a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:10.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.260+0000 7f3c859ff700 1 --2- 172.21.15.92:0/1070299911 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3c800b4fc0 0x7f3c80158570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:10.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.260+0000 7f3c859ff700 1 -- 172.21.15.92:0/1070299911 >> 172.21.15.92:0/1070299911 conn(0x7f3c800719a0 msgr2=0x7f3c800cefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:10.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.260+0000 7f3c859ff700 1 -- 172.21.15.92:0/1070299911 shutdown_connections 2023-11-28T11:21:10.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.260+0000 7f3c859ff700 1 -- 172.21.15.92:0/1070299911 wait complete. 2023-11-28T11:21:10.306 INFO:teuthology.orchestra.run.smithi092.stderr:listed 0 entries 2023-11-28T11:21:10.307 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-28T11:21:10.307 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:21:10.354 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:21:10.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.592+0000 7f90e07ff700 1 Processor -- start 2023-11-28T11:21:10.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.592+0000 7f90e07ff700 1 -- start start 2023-11-28T11:21:10.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.593+0000 7f90e07ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f90d80cd460 0x7f90d806c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:10.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.593+0000 7f90e07ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90d80b88e0 con 0x7f90d80d0d60 2023-11-28T11:21:10.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.593+0000 7f90e07ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90d80cd8f0 con 0x7f90d80b8510 2023-11-28T11:21:10.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.593+0000 7f90e07ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90d80cdbd0 con 0x7f90d80cd460 2023-11-28T11:21:10.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.593+0000 7f90ddd9a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f90d80cd460 0x7f90d806c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:10.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.593+0000 7f90ddd9a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f90d80cd460 0x7f90d806c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48878/0 (socket says 172.21.15.92:48878) 2023-11-28T11:21:10.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.593+0000 7f90ddd9a700 1 -- 172.21.15.92:0/1459397811 learned_addr learned my addr 172.21.15.92:0/1459397811 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:10.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.594+0000 7f90ddd9a700 1 -- 172.21.15.92:0/1459397811 >> v1:172.21.15.169:6789/0 conn(0x7f90d80b8510 legacy=0x7f90d806b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:10.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.594+0000 7f90ddd9a700 1 -- 172.21.15.92:0/1459397811 >> v1:172.21.15.92:6789/0 conn(0x7f90d80d0d60 legacy=0x7f90d80d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:10.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.594+0000 7f90ddd9a700 1 -- 172.21.15.92:0/1459397811 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d80cfd10 con 0x7f90d80cd460 2023-11-28T11:21:10.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.594+0000 7f90ddd9a700 1 --2- 172.21.15.92:0/1459397811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90d80cd460 0x7f90d806c080 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f90c80098f0 tx=0x7f90c8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=393cdfcbb21ed2fb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:10.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.594+0000 7f90dd599700 1 -- 172.21.15.92:0/1459397811 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c8004230 con 0x7f90d80cd460 2023-11-28T11:21:10.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.594+0000 7f90dd599700 1 -- 172.21.15.92:0/1459397811 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90c8004390 con 0x7f90d80cd460 2023-11-28T11:21:10.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.594+0000 7f90dd599700 1 -- 172.21.15.92:0/1459397811 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c80036a0 con 0x7f90d80cd460 2023-11-28T11:21:10.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.595+0000 7f90e07ff700 1 -- 172.21.15.92:0/1459397811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90d80cd460 msgr2=0x7f90d806c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:10.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.595+0000 7f90e07ff700 1 --2- 172.21.15.92:0/1459397811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90d80cd460 0x7f90d806c080 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f90c80098f0 tx=0x7f90c8009c00 comp rx=0 tx=0).stop 2023-11-28T11:21:10.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.595+0000 7f90e07ff700 1 -- 172.21.15.92:0/1459397811 shutdown_connections 2023-11-28T11:21:10.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.595+0000 7f90e07ff700 1 --2- 172.21.15.92:0/1459397811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90d80cd460 0x7f90d806c080 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:10.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.595+0000 7f90e07ff700 1 -- 172.21.15.92:0/1459397811 >> 172.21.15.92:0/1459397811 conn(0x7f90d80719a0 msgr2=0x7f90d8071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:10.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.595+0000 7f90e07ff700 1 -- 172.21.15.92:0/1459397811 shutdown_connections 2023-11-28T11:21:10.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.595+0000 7f90e07ff700 1 -- 172.21.15.92:0/1459397811 wait complete. 2023-11-28T11:21:10.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.596+0000 7f90e07ff700 1 Processor -- start 2023-11-28T11:21:10.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.597+0000 7f90e07ff700 1 -- start start 2023-11-28T11:21:10.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.597+0000 7f90e07ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90d80b8510 0x7f90d80cbad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:10.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.597+0000 7f90e07ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90d80cd460 0x7f90d80c9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:10.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.598+0000 7f90e07ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90d80d0d60 0x7f90d815aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:10.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.598+0000 7f90e07ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d80cd050 con 0x7f90d80cd460 2023-11-28T11:21:10.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.598+0000 7f90e07ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d80cc3d0 con 0x7f90d80b8510 2023-11-28T11:21:10.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.598+0000 7f90e07ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90d80cc6b0 con 0x7f90d80d0d60 2023-11-28T11:21:10.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.598+0000 7f90ddd9a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90d80cd460 0x7f90d80c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:10.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.598+0000 7f90ddd9a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90d80cd460 0x7f90d80c9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48890/0 (socket says 172.21.15.92:48890) 2023-11-28T11:21:10.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.598+0000 7f90ddd9a700 1 -- 172.21.15.92:0/3559989464 learned_addr learned my addr 172.21.15.92:0/3559989464 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:10.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.598+0000 7f90de59b700 1 --2- 172.21.15.92:0/3559989464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90d80b8510 0x7f90d80cbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:10.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.598+0000 7f90ded9c700 1 --2- 172.21.15.92:0/3559989464 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90d80d0d60 0x7f90d815aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:10.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.598+0000 7f90ddd9a700 1 -- 172.21.15.92:0/3559989464 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90d80d0d60 msgr2=0x7f90d815aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:10.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.598+0000 7f90ddd9a700 1 --2- 172.21.15.92:0/3559989464 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90d80d0d60 0x7f90d815aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:10.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.598+0000 7f90ddd9a700 1 -- 172.21.15.92:0/3559989464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90d80b8510 msgr2=0x7f90d80cbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:10.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.598+0000 7f90ddd9a700 1 --2- 172.21.15.92:0/3559989464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90d80b8510 0x7f90d80cbad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:10.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.598+0000 7f90ddd9a700 1 -- 172.21.15.92:0/3559989464 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90c8009550 con 0x7f90d80cd460 2023-11-28T11:21:10.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.598+0000 7f90de59b700 1 --2- 172.21.15.92:0/3559989464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90d80b8510 0x7f90d80cbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:21:10.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.598+0000 7f90ddd9a700 1 --2- 172.21.15.92:0/3559989464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90d80cd460 0x7f90d80c9580 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f90c8004000 tx=0x7f90c8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:10.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.599+0000 7f90ceffd700 1 -- 172.21.15.92:0/3559989464 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c8003dc0 con 0x7f90d80cd460 2023-11-28T11:21:10.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.599+0000 7f90ceffd700 1 -- 172.21.15.92:0/3559989464 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90c801f070 con 0x7f90d80cd460 2023-11-28T11:21:10.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.599+0000 7f90e07ff700 1 -- 172.21.15.92:0/3559989464 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90d80d29c0 con 0x7f90d80cd460 2023-11-28T11:21:10.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.599+0000 7f90ceffd700 1 -- 172.21.15.92:0/3559989464 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c800b8a0 con 0x7f90d80cd460 2023-11-28T11:21:10.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.599+0000 7f90e07ff700 1 -- 172.21.15.92:0/3559989464 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90d80cfff0 con 0x7f90d80cd460 2023-11-28T11:21:10.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.600+0000 7f90e07ff700 1 -- 172.21.15.92:0/3559989464 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90d80d2b50 con 0x7f90d80cd460 2023-11-28T11:21:10.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.601+0000 7f90ceffd700 1 -- 172.21.15.92:0/3559989464 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f90c802a020 con 0x7f90d80cd460 2023-11-28T11:21:10.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.602+0000 7f90ceffd700 1 --2- 172.21.15.92:0/3559989464 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f90bc0682f0 0x7f90bc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:10.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.602+0000 7f90ceffd700 1 -- 172.21.15.92:0/3559989464 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f90c80878e0 con 0x7f90d80cd460 2023-11-28T11:21:10.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.602+0000 7f90de59b700 1 --2- 172.21.15.92:0/3559989464 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f90bc0682f0 0x7f90bc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:10.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.603+0000 7f90de59b700 1 --2- 172.21.15.92:0/3559989464 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f90bc0682f0 0x7f90bc06a7a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f90d4003fc0 tx=0x7f90d4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:10.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.606+0000 7f90ceffd700 1 -- 172.21.15.92:0/3559989464 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f90c8053d00 con 0x7f90d80cd460 2023-11-28T11:21:10.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.799+0000 7f90e07ff700 1 -- 172.21.15.92:0/3559989464 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f90d80d2b50 con 0x7f90d80cd460 2023-11-28T11:21:10.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.800+0000 7f90ceffd700 1 -- 172.21.15.92:0/3559989464 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+19748 (secure 0 0 0) 0x7f90c8057320 con 0x7f90d80cd460 2023-11-28T11:21:10.801 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:21:10.802 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":27,"fsid":"68956e0c-c16f-47cd-a220-f98364b0831d","created":"2023-11-28T11:20:28.697573+0000","modified":"2023-11-28T11:21:00.770781+0000","last_up_change":"2023-11-28T11:20:41.257011+0000","last_in_change":"2023-11-28T11:20:32.249767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:20:41.925237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:20:48.682497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:20:54.471398+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:20:55.204585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:20:58.297307+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6abe483f-0e03-4a26-b9cf-62041291747e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":115823},{"type":"v1","addr":"172.21.15.92:6809","nonce":115823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":115823},{"type":"v1","addr":"172.21.15.92:6813","nonce":115823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":115823},{"type":"v1","addr":"172.21.15.92:6821","nonce":115823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":115823},{"type":"v1","addr":"172.21.15.92:6817","nonce":115823}]},"public_addr":"172.21.15.92:6809/115823","cluster_addr":"172.21.15.92:6813/115823","heartbeat_back_addr":"172.21.15.92:6821/115823","heartbeat_front_addr":"172.21.15.92:6817/115823","state":["exists","up"]},{"osd":1,"uuid":"13c1de73-1a8c-4d7e-8a28-9dae52dfe089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":115824},{"type":"v1","addr":"172.21.15.92:6812","nonce":115824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":115824},{"type":"v1","addr":"172.21.15.92:6816","nonce":115824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":115824},{"type":"v1","addr":"172.21.15.92:6823","nonce":115824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":115824},{"type":"v1","addr":"172.21.15.92:6820","nonce":115824}]},"public_addr":"172.21.15.92:6812/115824","cluster_addr":"172.21.15.92:6816/115824","heartbeat_back_addr":"172.21.15.92:6823/115824","heartbeat_front_addr":"172.21.15.92:6820/115824","state":["exists","up"]},{"osd":2,"uuid":"25788a55-112f-4f00-810d-2bc37cf76778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":115821},{"type":"v1","addr":"172.21.15.92:6801","nonce":115821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":115821},{"type":"v1","addr":"172.21.15.92:6803","nonce":115821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":115821},{"type":"v1","addr":"172.21.15.92:6807","nonce":115821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":115821},{"type":"v1","addr":"172.21.15.92:6805","nonce":115821}]},"public_addr":"172.21.15.92:6801/115821","cluster_addr":"172.21.15.92:6803/115821","heartbeat_back_addr":"172.21.15.92:6807/115821","heartbeat_front_addr":"172.21.15.92:6805/115821","state":["exists","up"]},{"osd":3,"uuid":"adf54e24-33a9-41fe-9ce3-98a5ab6c158e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":115825},{"type":"v1","addr":"172.21.15.92:6825","nonce":115825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":115825},{"type":"v1","addr":"172.21.15.92:6827","nonce":115825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":115825},{"type":"v1","addr":"172.21.15.92:6831","nonce":115825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":115825},{"type":"v1","addr":"172.21.15.92:6829","nonce":115825}]},"public_addr":"172.21.15.92:6825/115825","cluster_addr":"172.21.15.92:6827/115825","heartbeat_back_addr":"172.21.15.92:6831/115825","heartbeat_front_addr":"172.21.15.92:6829/115825","state":["exists","up"]},{"osd":4,"uuid":"ccb9b747-f399-44ea-9a18-9db3152227e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":116035},{"type":"v1","addr":"172.21.15.169:6817","nonce":116035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":116035},{"type":"v1","addr":"172.21.15.169:6819","nonce":116035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":116035},{"type":"v1","addr":"172.21.15.169:6826","nonce":116035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":116035},{"type":"v1","addr":"172.21.15.169:6822","nonce":116035}]},"public_addr":"172.21.15.169:6817/116035","cluster_addr":"172.21.15.169:6819/116035","heartbeat_back_addr":"172.21.15.169:6826/116035","heartbeat_front_addr":"172.21.15.169:6822/116035","state":["exists","up"]},{"osd":5,"uuid":"308a1e63-75aa-4d7d-8a7c-cecc24714fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":116033},{"type":"v1","addr":"172.21.15.169:6801","nonce":116033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":116033},{"type":"v1","addr":"172.21.15.169:6803","nonce":116033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":116033},{"type":"v1","addr":"172.21.15.169:6807","nonce":116033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":116033},{"type":"v1","addr":"172.21.15.169:6805","nonce":116033}]},"public_addr":"172.21.15.169:6801/116033","cluster_addr":"172.21.15.169:6803/116033","heartbeat_back_addr":"172.21.15.169:6807/116033","heartbeat_front_addr":"172.21.15.169:6805/116033","state":["exists","up"]},{"osd":6,"uuid":"9c459b7b-b821-4b4b-b91e-6aaae271457c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":116034},{"type":"v1","addr":"172.21.15.169:6809","nonce":116034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":116034},{"type":"v1","addr":"172.21.15.169:6811","nonce":116034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":116034},{"type":"v1","addr":"172.21.15.169:6815","nonce":116034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":116034},{"type":"v1","addr":"172.21.15.169:6813","nonce":116034}]},"public_addr":"172.21.15.169:6809/116034","cluster_addr":"172.21.15.169:6811/116034","heartbeat_back_addr":"172.21.15.169:6815/116034","heartbeat_front_addr":"172.21.15.169:6813/116034","state":["exists","up"]},{"osd":7,"uuid":"1ca9aa4a-755d-4e28-a937-4d8dccab4df3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":116044},{"type":"v1","addr":"172.21.15.169:6823","nonce":116044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6825","nonce":116044},{"type":"v1","addr":"172.21.15.169:6827","nonce":116044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":116044},{"type":"v1","addr":"172.21.15.169:6831","nonce":116044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":116044},{"type":"v1","addr":"172.21.15.169:6829","nonce":116044}]},"public_addr":"172.21.15.169:6823/116044","cluster_addr":"172.21.15.169:6827/116044","heartbeat_back_addr":"172.21.15.169:6831/116044","heartbeat_front_addr":"172.21.15.169:6829/116044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.586781+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.869778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.508594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.860493+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.872267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.746933+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.611520+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.871446+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:21:10.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.802+0000 7f90e07ff700 1 -- 172.21.15.92:0/3559989464 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f90bc0682f0 msgr2=0x7f90bc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:10.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.802+0000 7f90e07ff700 1 --2- 172.21.15.92:0/3559989464 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f90bc0682f0 0x7f90bc06a7a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f90d4003fc0 tx=0x7f90d4009040 comp rx=0 tx=0).stop 2023-11-28T11:21:10.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.802+0000 7f90e07ff700 1 -- 172.21.15.92:0/3559989464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90d80cd460 msgr2=0x7f90d80c9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:10.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.802+0000 7f90e07ff700 1 --2- 172.21.15.92:0/3559989464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90d80cd460 0x7f90d80c9580 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f90c8004000 tx=0x7f90c8004750 comp rx=0 tx=0).stop 2023-11-28T11:21:10.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.803+0000 7f90e07ff700 1 -- 172.21.15.92:0/3559989464 shutdown_connections 2023-11-28T11:21:10.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.803+0000 7f90e07ff700 1 --2- 172.21.15.92:0/3559989464 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f90d80d0d60 0x7f90d815aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:10.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.803+0000 7f90e07ff700 1 --2- 172.21.15.92:0/3559989464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f90d80b8510 0x7f90d80cbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:10.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.803+0000 7f90e07ff700 1 --2- 172.21.15.92:0/3559989464 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f90bc0682f0 0x7f90bc06a7a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:10.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.803+0000 7f90e07ff700 1 --2- 172.21.15.92:0/3559989464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90d80cd460 0x7f90d80c9580 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:10.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.803+0000 7f90e07ff700 1 -- 172.21.15.92:0/3559989464 >> 172.21.15.92:0/3559989464 conn(0x7f90d80719a0 msgr2=0x7f90d806b970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:10.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.803+0000 7f90e07ff700 1 -- 172.21.15.92:0/3559989464 shutdown_connections 2023-11-28T11:21:10.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:10.803+0000 7f90e07ff700 1 -- 172.21.15.92:0/3559989464 wait complete. 2023-11-28T11:21:10.822 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:20:41.925237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:20:48.682497+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:20:54.471398+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:20:55.204585+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T11:20:58.297307+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T11:21:10.822 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:21:11.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.021+0000 7f88fb1ff700 1 Processor -- start 2023-11-28T11:21:11.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.021+0000 7f88fb1ff700 1 -- start start 2023-11-28T11:21:11.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.022+0000 7f88fb1ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f88f40cd460 0x7f88f406c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:11.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.022+0000 7f88fb1ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88f40d0d60 0x7f88f40d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:11.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.022+0000 7f88fb1ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88f40b88e0 con 0x7f88f40b8510 2023-11-28T11:21:11.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.022+0000 7f88fb1ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88f40cdbd0 con 0x7f88f40cd460 2023-11-28T11:21:11.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.022+0000 7f88fb1ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88f40d02d0 con 0x7f88f40d0d60 2023-11-28T11:21:11.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.023+0000 7f88ebfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f88f40cd460 0x7f88f406c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:11.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.023+0000 7f88ebfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f88f40cd460 0x7f88f406c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48892/0 (socket says 172.21.15.92:48892) 2023-11-28T11:21:11.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.023+0000 7f88ebfff700 1 -- 172.21.15.92:0/2522743068 learned_addr learned my addr 172.21.15.92:0/2522743068 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:11.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.023+0000 7f88f979c700 1 --2- 172.21.15.92:0/2522743068 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88f40d0d60 0x7f88f40d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:11.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.023+0000 7f88ebfff700 1 -- 172.21.15.92:0/2522743068 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88f40d0d60 msgr2=0x7f88f40d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:11.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.023+0000 7f88ebfff700 1 --2- 172.21.15.92:0/2522743068 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88f40d0d60 0x7f88f40d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:11.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.023+0000 7f88ebfff700 1 -- 172.21.15.92:0/2522743068 >> v1:172.21.15.169:6789/0 conn(0x7f88f40b8510 legacy=0x7f88f406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:11.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.023+0000 7f88ebfff700 1 -- 172.21.15.92:0/2522743068 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88f40cd8f0 con 0x7f88f40cd460 2023-11-28T11:21:11.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.023+0000 7f88f979c700 1 --2- 172.21.15.92:0/2522743068 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88f40d0d60 0x7f88f40d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:21:11.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.023+0000 7f88ebfff700 1 --2- 172.21.15.92:0/2522743068 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88f40cd460 0x7f88f406c080 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f88e4009920 tx=0x7f88e4009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91fac54d85758ef server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:11.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.024+0000 7f88eb7fe700 1 -- 172.21.15.92:0/2522743068 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e4004230 con 0x7f88f40cd460 2023-11-28T11:21:11.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.024+0000 7f88eb7fe700 1 -- 172.21.15.92:0/2522743068 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88e4004390 con 0x7f88f40cd460 2023-11-28T11:21:11.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.024+0000 7f88eb7fe700 1 -- 172.21.15.92:0/2522743068 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e4009e70 con 0x7f88f40cd460 2023-11-28T11:21:11.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.024+0000 7f88fb1ff700 1 -- 172.21.15.92:0/2522743068 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88f40cd460 msgr2=0x7f88f406c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:11.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.024+0000 7f88fb1ff700 1 --2- 172.21.15.92:0/2522743068 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88f40cd460 0x7f88f406c080 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f88e4009920 tx=0x7f88e4009c30 comp rx=0 tx=0).stop 2023-11-28T11:21:11.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.024+0000 7f88fb1ff700 1 -- 172.21.15.92:0/2522743068 shutdown_connections 2023-11-28T11:21:11.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.024+0000 7f88fb1ff700 1 --2- 172.21.15.92:0/2522743068 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88f40d0d60 0x7f88f40d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:11.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.024+0000 7f88fb1ff700 1 --2- 172.21.15.92:0/2522743068 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88f40cd460 0x7f88f406c080 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:11.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.024+0000 7f88fb1ff700 1 -- 172.21.15.92:0/2522743068 >> 172.21.15.92:0/2522743068 conn(0x7f88f40719a0 msgr2=0x7f88f4071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:11.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.024+0000 7f88fb1ff700 1 -- 172.21.15.92:0/2522743068 shutdown_connections 2023-11-28T11:21:11.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.024+0000 7f88fb1ff700 1 -- 172.21.15.92:0/2522743068 wait complete. 2023-11-28T11:21:11.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.026+0000 7f88fb1ff700 1 Processor -- start 2023-11-28T11:21:11.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.026+0000 7f88fb1ff700 1 -- start start 2023-11-28T11:21:11.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.027+0000 7f88fb1ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88f40b8510 0x7f88f40ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:11.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.027+0000 7f88fb1ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88f40cd460 0x7f88f4155e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:11.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.027+0000 7f88fb1ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88f40d0d60 0x7f88f4158e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:11.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.027+0000 7f88fb1ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f40cd220 con 0x7f88f40b8510 2023-11-28T11:21:11.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.027+0000 7f88fb1ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f40ca8f0 con 0x7f88f40cd460 2023-11-28T11:21:11.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.027+0000 7f88fb1ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88f40cab00 con 0x7f88f40d0d60 2023-11-28T11:21:11.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.027+0000 7f88f8f9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88f40b8510 0x7f88f40ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:11.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.027+0000 7f88f8f9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88f40b8510 0x7f88f40ccce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48908/0 (socket says 172.21.15.92:48908) 2023-11-28T11:21:11.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.027+0000 7f88f8f9b700 1 -- 172.21.15.92:0/2577444183 learned_addr learned my addr 172.21.15.92:0/2577444183 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:11.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.028+0000 7f88ebfff700 1 --2- 172.21.15.92:0/2577444183 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88f40cd460 0x7f88f4155e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:11.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.028+0000 7f88f979c700 1 --2- 172.21.15.92:0/2577444183 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88f40d0d60 0x7f88f4158e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:11.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.028+0000 7f88f8f9b700 1 -- 172.21.15.92:0/2577444183 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88f40d0d60 msgr2=0x7f88f4158e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:11.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.028+0000 7f88f8f9b700 1 --2- 172.21.15.92:0/2577444183 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88f40d0d60 0x7f88f4158e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:11.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.028+0000 7f88f8f9b700 1 -- 172.21.15.92:0/2577444183 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88f40cd460 msgr2=0x7f88f4155e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:11.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.028+0000 7f88f8f9b700 1 --2- 172.21.15.92:0/2577444183 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88f40cd460 0x7f88f4155e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:11.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.028+0000 7f88f8f9b700 1 -- 172.21.15.92:0/2577444183 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e4009550 con 0x7f88f40b8510 2023-11-28T11:21:11.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.028+0000 7f88f8f9b700 1 --2- 172.21.15.92:0/2577444183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88f40b8510 0x7f88f40ccce0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f88f000c4d0 tx=0x7f88f000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:11.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.028+0000 7f88e97fa700 1 -- 172.21.15.92:0/2577444183 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f0011790 con 0x7f88f40b8510 2023-11-28T11:21:11.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.028+0000 7f88e97fa700 1 -- 172.21.15.92:0/2577444183 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88f00118f0 con 0x7f88f40b8510 2023-11-28T11:21:11.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.028+0000 7f88fb1ff700 1 -- 172.21.15.92:0/2577444183 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88f40cfff0 con 0x7f88f40b8510 2023-11-28T11:21:11.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.028+0000 7f88e97fa700 1 -- 172.21.15.92:0/2577444183 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f0011ca0 con 0x7f88f40b8510 2023-11-28T11:21:11.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.028+0000 7f88fb1ff700 1 -- 172.21.15.92:0/2577444183 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88f40cfd10 con 0x7f88f40b8510 2023-11-28T11:21:11.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.029+0000 7f88fb1ff700 1 -- 172.21.15.92:0/2577444183 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88f4067880 con 0x7f88f40b8510 2023-11-28T11:21:11.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.030+0000 7f88e97fa700 1 -- 172.21.15.92:0/2577444183 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f88f0010050 con 0x7f88f40b8510 2023-11-28T11:21:11.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.031+0000 7f88e97fa700 1 --2- 172.21.15.92:0/2577444183 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f88e00682f0 0x7f88e006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:11.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.031+0000 7f88e97fa700 1 -- 172.21.15.92:0/2577444183 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f88f0085c50 con 0x7f88f40b8510 2023-11-28T11:21:11.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.031+0000 7f88ebfff700 1 --2- 172.21.15.92:0/2577444183 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f88e00682f0 0x7f88e006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:11.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.032+0000 7f88ebfff700 1 --2- 172.21.15.92:0/2577444183 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f88e00682f0 0x7f88e006a7a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f88f40b1de0 tx=0x7f88e4008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:11.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.035+0000 7f88e97fa700 1 -- 172.21.15.92:0/2577444183 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f88f0052130 con 0x7f88f40b8510 2023-11-28T11:21:11.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.224+0000 7f88fb1ff700 1 -- 172.21.15.92:0/2577444183 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f88f40d2940 con 0x7f88f40b8510 2023-11-28T11:21:11.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.224+0000 7f88e97fa700 1 -- 172.21.15.92:0/2577444183 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f88f0055750 con 0x7f88f40b8510 2023-11-28T11:21:11.225 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-11-28T11:21:11.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.227+0000 7f88fb1ff700 1 -- 172.21.15.92:0/2577444183 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f88e00682f0 msgr2=0x7f88e006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:11.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.227+0000 7f88fb1ff700 1 --2- 172.21.15.92:0/2577444183 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f88e00682f0 0x7f88e006a7a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f88f40b1de0 tx=0x7f88e4008030 comp rx=0 tx=0).stop 2023-11-28T11:21:11.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.227+0000 7f88fb1ff700 1 -- 172.21.15.92:0/2577444183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88f40b8510 msgr2=0x7f88f40ccce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:11.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.227+0000 7f88fb1ff700 1 --2- 172.21.15.92:0/2577444183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88f40b8510 0x7f88f40ccce0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f88f000c4d0 tx=0x7f88f000c7e0 comp rx=0 tx=0).stop 2023-11-28T11:21:11.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.227+0000 7f88fb1ff700 1 -- 172.21.15.92:0/2577444183 shutdown_connections 2023-11-28T11:21:11.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.227+0000 7f88fb1ff700 1 --2- 172.21.15.92:0/2577444183 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f88f40d0d60 0x7f88f4158e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:11.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.227+0000 7f88fb1ff700 1 --2- 172.21.15.92:0/2577444183 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f88e00682f0 0x7f88e006a7a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:11.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.227+0000 7f88fb1ff700 1 --2- 172.21.15.92:0/2577444183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88f40b8510 0x7f88f40ccce0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:11.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.227+0000 7f88fb1ff700 1 --2- 172.21.15.92:0/2577444183 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f88f40cd460 0x7f88f4155e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:11.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.227+0000 7f88fb1ff700 1 -- 172.21.15.92:0/2577444183 >> 172.21.15.92:0/2577444183 conn(0x7f88f40719a0 msgr2=0x7f88f40b4180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:11.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.227+0000 7f88fb1ff700 1 -- 172.21.15.92:0/2577444183 shutdown_connections 2023-11-28T11:21:11.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.227+0000 7f88fb1ff700 1 -- 172.21.15.92:0/2577444183 wait complete. 2023-11-28T11:21:11.246 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:21:11.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.443+0000 7f12973ff700 1 Processor -- start 2023-11-28T11:21:11.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.443+0000 7f12973ff700 1 -- start start 2023-11-28T11:21:11.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.445+0000 7f12973ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f12900d2df0 0x7f12900cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:11.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.445+0000 7f12973ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f12900b4fc0 0x7f12900d31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:11.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.445+0000 7f12973ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12900d1950 con 0x7f12900b1c60 2023-11-28T11:21:11.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.445+0000 7f12973ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12900d3d30 con 0x7f12900d2df0 2023-11-28T11:21:11.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.445+0000 7f12973ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12900b2500 con 0x7f12900b4fc0 2023-11-28T11:21:11.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.445+0000 7f129519b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f12900d2df0 0x7f12900cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:11.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.445+0000 7f129519b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f12900d2df0 0x7f12900cd620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48914/0 (socket says 172.21.15.92:48914) 2023-11-28T11:21:11.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.445+0000 7f129519b700 1 -- 172.21.15.92:0/1801340985 learned_addr learned my addr 172.21.15.92:0/1801340985 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:11.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.445+0000 7f129499a700 1 --2- 172.21.15.92:0/1801340985 >> v2:172.21.15.169:3300/0 conn(0x7f12900b4fc0 0x7f12900d31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:11.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.445+0000 7f129519b700 1 -- 172.21.15.92:0/1801340985 >> v2:172.21.15.169:3300/0 conn(0x7f12900b4fc0 msgr2=0x7f12900d31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:11.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.445+0000 7f129519b700 1 --2- 172.21.15.92:0/1801340985 >> v2:172.21.15.169:3300/0 conn(0x7f12900b4fc0 0x7f12900d31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:11.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.445+0000 7f129519b700 1 -- 172.21.15.92:0/1801340985 >> v1:172.21.15.169:6789/0 conn(0x7f12900b1c60 legacy=0x7f12900cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:11.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.445+0000 7f129519b700 1 -- 172.21.15.92:0/1801340985 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12900d3a50 con 0x7f12900d2df0 2023-11-28T11:21:11.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.446+0000 7f129519b700 1 --2- 172.21.15.92:0/1801340985 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12900d2df0 0x7f12900cd620 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f12901601b0 tx=0x7f128c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c8d356ba32a31d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:11.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.446+0000 7f1287fff700 1 -- 172.21.15.92:0/1801340985 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128c011830 con 0x7f12900d2df0 2023-11-28T11:21:11.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.446+0000 7f1287fff700 1 -- 172.21.15.92:0/1801340985 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f128c011990 con 0x7f12900d2df0 2023-11-28T11:21:11.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.446+0000 7f1287fff700 1 -- 172.21.15.92:0/1801340985 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128c011d20 con 0x7f12900d2df0 2023-11-28T11:21:11.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.446+0000 7f12973ff700 1 -- 172.21.15.92:0/1801340985 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12900d2df0 msgr2=0x7f12900cd620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:11.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.446+0000 7f12973ff700 1 --2- 172.21.15.92:0/1801340985 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12900d2df0 0x7f12900cd620 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f12901601b0 tx=0x7f128c009bd0 comp rx=0 tx=0).stop 2023-11-28T11:21:11.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.447+0000 7f12973ff700 1 -- 172.21.15.92:0/1801340985 shutdown_connections 2023-11-28T11:21:11.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.447+0000 7f12973ff700 1 --2- 172.21.15.92:0/1801340985 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12900d2df0 0x7f12900cd620 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:11.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.447+0000 7f12973ff700 1 --2- 172.21.15.92:0/1801340985 >> v2:172.21.15.169:3300/0 conn(0x7f12900b4fc0 0x7f12900d31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:11.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.447+0000 7f12973ff700 1 -- 172.21.15.92:0/1801340985 >> 172.21.15.92:0/1801340985 conn(0x7f12900719a0 msgr2=0x7f1290071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:11.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.447+0000 7f12973ff700 1 -- 172.21.15.92:0/1801340985 shutdown_connections 2023-11-28T11:21:11.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.447+0000 7f12973ff700 1 -- 172.21.15.92:0/1801340985 wait complete. 2023-11-28T11:21:11.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.448+0000 7f12973ff700 1 Processor -- start 2023-11-28T11:21:11.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.449+0000 7f12973ff700 1 -- start start 2023-11-28T11:21:11.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.449+0000 7f12973ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12900b1c60 0x7f12901e7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:11.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.449+0000 7f12973ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12900b4fc0 0x7f129015ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:11.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.449+0000 7f12973ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12900d2df0 0x7f129015f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:11.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.449+0000 7f12973ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1290148300 con 0x7f12900d2df0 2023-11-28T11:21:11.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.449+0000 7f12973ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f129014b1c0 con 0x7f12900b1c60 2023-11-28T11:21:11.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.449+0000 7f12973ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12901485c0 con 0x7f12900b4fc0 2023-11-28T11:21:11.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.450+0000 7f129519b700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12900b1c60 0x7f12901e7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:11.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.450+0000 7f129599c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12900d2df0 0x7f129015f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:11.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.450+0000 7f129519b700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12900b1c60 0x7f12901e7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.92:49690/0 (socket says 172.21.15.92:49690) 2023-11-28T11:21:11.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.450+0000 7f129499a700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12900b4fc0 0x7f129015ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:11.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.450+0000 7f129519b700 1 -- 172.21.15.92:0/91021047 learned_addr learned my addr 172.21.15.92:0/91021047 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:11.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.450+0000 7f129599c700 1 -- 172.21.15.92:0/91021047 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12900b4fc0 msgr2=0x7f129015ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:11.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.450+0000 7f129599c700 1 --2- 172.21.15.92:0/91021047 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12900b4fc0 0x7f129015ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:11.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.450+0000 7f129599c700 1 -- 172.21.15.92:0/91021047 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12900b1c60 msgr2=0x7f12901e7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:11.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.450+0000 7f129599c700 1 --2- 172.21.15.92:0/91021047 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12900b1c60 0x7f12901e7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:11.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.450+0000 7f129599c700 1 -- 172.21.15.92:0/91021047 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f128c009550 con 0x7f12900d2df0 2023-11-28T11:21:11.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.450+0000 7f129599c700 1 --2- 172.21.15.92:0/91021047 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12900d2df0 0x7f129015f3c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f128800cae0 tx=0x7f128800cdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:11.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.451+0000 7f1285ffb700 1 -- 172.21.15.92:0/91021047 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1288003c00 con 0x7f12900d2df0 2023-11-28T11:21:11.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.451+0000 7f1285ffb700 1 -- 172.21.15.92:0/91021047 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1288003d60 con 0x7f12900d2df0 2023-11-28T11:21:11.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.451+0000 7f12973ff700 1 -- 172.21.15.92:0/91021047 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12900b88f0 con 0x7f12900d2df0 2023-11-28T11:21:11.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.451+0000 7f1285ffb700 1 -- 172.21.15.92:0/91021047 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128800aa20 con 0x7f12900d2df0 2023-11-28T11:21:11.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.451+0000 7f12973ff700 1 -- 172.21.15.92:0/91021047 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12900d15e0 con 0x7f12900d2df0 2023-11-28T11:21:11.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.452+0000 7f12973ff700 1 -- 172.21.15.92:0/91021047 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12900b21b0 con 0x7f12900d2df0 2023-11-28T11:21:11.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.453+0000 7f1285ffb700 1 -- 172.21.15.92:0/91021047 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1288010050 con 0x7f12900d2df0 2023-11-28T11:21:11.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.453+0000 7f1285ffb700 1 --2- 172.21.15.92:0/91021047 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f1274068320 0x7f127406a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:11.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.454+0000 7f1285ffb700 1 -- 172.21.15.92:0/91021047 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1288085e00 con 0x7f12900d2df0 2023-11-28T11:21:11.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.454+0000 7f129519b700 1 --2- 172.21.15.92:0/91021047 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f1274068320 0x7f127406a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:11.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.454+0000 7f129519b700 1 --2- 172.21.15.92:0/91021047 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f1274068320 0x7f127406a7d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f128c009fd0 tx=0x7f128c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:11.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.457+0000 7f1285ffb700 1 -- 172.21.15.92:0/91021047 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f12880521f0 con 0x7f12900d2df0 2023-11-28T11:21:11.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.648+0000 7f12973ff700 1 -- 172.21.15.92:0/91021047 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1290067170 con 0x7f12900d2df0 2023-11-28T11:21:11.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.648+0000 7f1285ffb700 1 -- 172.21.15.92:0/91021047 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v27) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1288055810 con 0x7f12900d2df0 2023-11-28T11:21:11.649 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-11-28T11:21:11.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.650+0000 7f12973ff700 1 -- 172.21.15.92:0/91021047 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f1274068320 msgr2=0x7f127406a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:11.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.650+0000 7f12973ff700 1 --2- 172.21.15.92:0/91021047 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f1274068320 0x7f127406a7d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f128c009fd0 tx=0x7f128c008030 comp rx=0 tx=0).stop 2023-11-28T11:21:11.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.650+0000 7f12973ff700 1 -- 172.21.15.92:0/91021047 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12900d2df0 msgr2=0x7f129015f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:11.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.650+0000 7f12973ff700 1 --2- 172.21.15.92:0/91021047 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12900d2df0 0x7f129015f3c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f128800cae0 tx=0x7f128800cdf0 comp rx=0 tx=0).stop 2023-11-28T11:21:11.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.650+0000 7f12973ff700 1 -- 172.21.15.92:0/91021047 shutdown_connections 2023-11-28T11:21:11.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.650+0000 7f12973ff700 1 --2- 172.21.15.92:0/91021047 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f1274068320 0x7f127406a7d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:11.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.650+0000 7f12973ff700 1 --2- 172.21.15.92:0/91021047 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12900d2df0 0x7f129015f3c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:11.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.650+0000 7f12973ff700 1 --2- 172.21.15.92:0/91021047 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f12900b1c60 0x7f12901e7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:11.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.650+0000 7f12973ff700 1 --2- 172.21.15.92:0/91021047 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f12900b4fc0 0x7f129015ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:11.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.650+0000 7f12973ff700 1 -- 172.21.15.92:0/91021047 >> 172.21.15.92:0/91021047 conn(0x7f12900719a0 msgr2=0x7f12900b7270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:11.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.651+0000 7f12973ff700 1 -- 172.21.15.92:0/91021047 shutdown_connections 2023-11-28T11:21:11.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.651+0000 7f12973ff700 1 -- 172.21.15.92:0/91021047 wait complete. 2023-11-28T11:21:11.670 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:21:11.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.871+0000 7f47b45ff700 1 Processor -- start 2023-11-28T11:21:11.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.871+0000 7f47b45ff700 1 -- start start 2023-11-28T11:21:11.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.872+0000 7f47b45ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f47ac0b8510 0x7f47ac06b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:11.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.872+0000 7f47b45ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f47ac0cd460 0x7f47ac06beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:11.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.873+0000 7f47b45ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47ac0b88e0 con 0x7f47ac06c3f0 2023-11-28T11:21:11.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.873+0000 7f47b45ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47ac0cdbd0 con 0x7f47ac0b8510 2023-11-28T11:21:11.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.873+0000 7f47b45ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47ac0d02d0 con 0x7f47ac0cd460 2023-11-28T11:21:11.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.873+0000 7f47b239b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f47ac0b8510 0x7f47ac06b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:11.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.873+0000 7f47b239b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f47ac0b8510 0x7f47ac06b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48924/0 (socket says 172.21.15.92:48924) 2023-11-28T11:21:11.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.873+0000 7f47b239b700 1 -- 172.21.15.92:0/3167754048 learned_addr learned my addr 172.21.15.92:0/3167754048 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:11.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.873+0000 7f47b1b9a700 1 --2- 172.21.15.92:0/3167754048 >> v2:172.21.15.169:3300/0 conn(0x7f47ac0cd460 0x7f47ac06beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:11.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.873+0000 7f47b239b700 1 -- 172.21.15.92:0/3167754048 >> v2:172.21.15.169:3300/0 conn(0x7f47ac0cd460 msgr2=0x7f47ac06beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:11.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.873+0000 7f47b239b700 1 --2- 172.21.15.92:0/3167754048 >> v2:172.21.15.169:3300/0 conn(0x7f47ac0cd460 0x7f47ac06beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:11.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.873+0000 7f47b239b700 1 -- 172.21.15.92:0/3167754048 >> v1:172.21.15.92:6789/0 conn(0x7f47ac06c3f0 legacy=0x7f47ac15cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:11.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.873+0000 7f47b239b700 1 -- 172.21.15.92:0/3167754048 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47ac0cd8f0 con 0x7f47ac0b8510 2023-11-28T11:21:11.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.873+0000 7f47b239b700 1 --2- 172.21.15.92:0/3167754048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47ac0b8510 0x7f47ac06b970 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f47a8009920 tx=0x7f47a8009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=136ef13b63838349 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:11.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.874+0000 7f47b1399700 1 -- 172.21.15.92:0/3167754048 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a800b9e0 con 0x7f47ac0b8510 2023-11-28T11:21:11.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.874+0000 7f47b1399700 1 -- 172.21.15.92:0/3167754048 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47a800bb40 con 0x7f47ac0b8510 2023-11-28T11:21:11.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.874+0000 7f47b1399700 1 -- 172.21.15.92:0/3167754048 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a8009e70 con 0x7f47ac0b8510 2023-11-28T11:21:11.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.874+0000 7f47b45ff700 1 -- 172.21.15.92:0/3167754048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47ac0b8510 msgr2=0x7f47ac06b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:11.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.874+0000 7f47b45ff700 1 --2- 172.21.15.92:0/3167754048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47ac0b8510 0x7f47ac06b970 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f47a8009920 tx=0x7f47a8009c30 comp rx=0 tx=0).stop 2023-11-28T11:21:11.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.874+0000 7f47b45ff700 1 -- 172.21.15.92:0/3167754048 shutdown_connections 2023-11-28T11:21:11.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.874+0000 7f47b45ff700 1 --2- 172.21.15.92:0/3167754048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47ac0b8510 0x7f47ac06b970 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:11.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.874+0000 7f47b45ff700 1 --2- 172.21.15.92:0/3167754048 >> v2:172.21.15.169:3300/0 conn(0x7f47ac0cd460 0x7f47ac06beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:11.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.874+0000 7f47b45ff700 1 -- 172.21.15.92:0/3167754048 >> 172.21.15.92:0/3167754048 conn(0x7f47ac0719a0 msgr2=0x7f47ac071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:11.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.874+0000 7f47b45ff700 1 -- 172.21.15.92:0/3167754048 shutdown_connections 2023-11-28T11:21:11.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.874+0000 7f47b45ff700 1 -- 172.21.15.92:0/3167754048 wait complete. 2023-11-28T11:21:11.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.876+0000 7f47b45ff700 1 Processor -- start 2023-11-28T11:21:11.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.876+0000 7f47b45ff700 1 -- start start 2023-11-28T11:21:11.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.877+0000 7f47b45ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47ac06c3f0 0x7f47ac0c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:11.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.877+0000 7f47b45ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47ac0b8510 0x7f47ac0cb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:11.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.877+0000 7f47b45ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47ac0cd460 0x7f47ac0d10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:11.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.877+0000 7f47b45ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47ac0ccb40 con 0x7f47ac0b8510 2023-11-28T11:21:11.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.877+0000 7f47b45ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47ac0c9300 con 0x7f47ac06c3f0 2023-11-28T11:21:11.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.877+0000 7f47b45ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47ac0cbec0 con 0x7f47ac0cd460 2023-11-28T11:21:11.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.877+0000 7f47b1b9a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47ac0b8510 0x7f47ac0cb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:11.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.878+0000 7f47b1b9a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47ac0b8510 0x7f47ac0cb5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48926/0 (socket says 172.21.15.92:48926) 2023-11-28T11:21:11.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.878+0000 7f47b1b9a700 1 -- 172.21.15.92:0/1588237933 learned_addr learned my addr 172.21.15.92:0/1588237933 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:11.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.878+0000 7f47b2b9c700 1 --2- 172.21.15.92:0/1588237933 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47ac0cd460 0x7f47ac0d10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:11.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.878+0000 7f47b239b700 1 --2- 172.21.15.92:0/1588237933 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47ac06c3f0 0x7f47ac0c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:11.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.878+0000 7f47b1b9a700 1 -- 172.21.15.92:0/1588237933 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47ac0cd460 msgr2=0x7f47ac0d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:11.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.878+0000 7f47b1b9a700 1 --2- 172.21.15.92:0/1588237933 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47ac0cd460 0x7f47ac0d10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:11.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.878+0000 7f47b1b9a700 1 -- 172.21.15.92:0/1588237933 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47ac06c3f0 msgr2=0x7f47ac0c8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:11.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.878+0000 7f47b1b9a700 1 --2- 172.21.15.92:0/1588237933 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47ac06c3f0 0x7f47ac0c8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:11.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.878+0000 7f47b1b9a700 1 -- 172.21.15.92:0/1588237933 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a8009550 con 0x7f47ac0b8510 2023-11-28T11:21:11.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.878+0000 7f47b2b9c700 1 --2- 172.21.15.92:0/1588237933 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47ac0cd460 0x7f47ac0d10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:21:11.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.878+0000 7f47b239b700 1 --2- 172.21.15.92:0/1588237933 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47ac06c3f0 0x7f47ac0c8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:21:11.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.878+0000 7f47b1b9a700 1 --2- 172.21.15.92:0/1588237933 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47ac0b8510 0x7f47ac0cb5c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f479c00d7d0 tx=0x7f479c00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:11.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.878+0000 7f47a2ffd700 1 -- 172.21.15.92:0/1588237933 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479c0125f0 con 0x7f47ac0b8510 2023-11-28T11:21:11.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.878+0000 7f47a2ffd700 1 -- 172.21.15.92:0/1588237933 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f479c012770 con 0x7f47ac0b8510 2023-11-28T11:21:11.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.878+0000 7f47b45ff700 1 -- 172.21.15.92:0/1588237933 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47ac0cfff0 con 0x7f47ac0b8510 2023-11-28T11:21:11.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.878+0000 7f47a2ffd700 1 -- 172.21.15.92:0/1588237933 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479c012b20 con 0x7f47ac0b8510 2023-11-28T11:21:11.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.879+0000 7f47b45ff700 1 -- 172.21.15.92:0/1588237933 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47ac0cfd10 con 0x7f47ac0b8510 2023-11-28T11:21:11.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.879+0000 7f47b45ff700 1 -- 172.21.15.92:0/1588237933 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47ac067880 con 0x7f47ac0b8510 2023-11-28T11:21:11.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.880+0000 7f47a2ffd700 1 -- 172.21.15.92:0/1588237933 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f479c012c80 con 0x7f47ac0b8510 2023-11-28T11:21:11.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.881+0000 7f47a2ffd700 1 --2- 172.21.15.92:0/1588237933 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f47900682f0 0x7f479006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:11.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.881+0000 7f47a2ffd700 1 -- 172.21.15.92:0/1588237933 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f479c086540 con 0x7f47ac0b8510 2023-11-28T11:21:11.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.881+0000 7f47b239b700 1 --2- 172.21.15.92:0/1588237933 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f47900682f0 0x7f479006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:11.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.882+0000 7f47b239b700 1 --2- 172.21.15.92:0/1588237933 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f47900682f0 0x7f479006a7a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f47a8009c70 tx=0x7f47a8008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:11.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:11.886+0000 7f47a2ffd700 1 -- 172.21.15.92:0/1588237933 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f479c0529c0 con 0x7f47ac0b8510 2023-11-28T11:21:12.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.078+0000 7f47b45ff700 1 -- 172.21.15.92:0/1588237933 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f47ac0d2940 con 0x7f47ac0b8510 2023-11-28T11:21:12.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.078+0000 7f47a2ffd700 1 -- 172.21.15.92:0/1588237933 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f479c055fe0 con 0x7f47ac0b8510 2023-11-28T11:21:12.079 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-11-28T11:21:12.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.081+0000 7f47b45ff700 1 -- 172.21.15.92:0/1588237933 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f47900682f0 msgr2=0x7f479006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:12.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.081+0000 7f47b45ff700 1 --2- 172.21.15.92:0/1588237933 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f47900682f0 0x7f479006a7a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f47a8009c70 tx=0x7f47a8008030 comp rx=0 tx=0).stop 2023-11-28T11:21:12.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.081+0000 7f47b45ff700 1 -- 172.21.15.92:0/1588237933 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47ac0b8510 msgr2=0x7f47ac0cb5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:12.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.081+0000 7f47b45ff700 1 --2- 172.21.15.92:0/1588237933 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47ac0b8510 0x7f47ac0cb5c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f479c00d7d0 tx=0x7f479c00db90 comp rx=0 tx=0).stop 2023-11-28T11:21:12.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.081+0000 7f47b45ff700 1 -- 172.21.15.92:0/1588237933 shutdown_connections 2023-11-28T11:21:12.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.081+0000 7f47b45ff700 1 --2- 172.21.15.92:0/1588237933 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47ac0cd460 0x7f47ac0d10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:12.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.081+0000 7f47b45ff700 1 --2- 172.21.15.92:0/1588237933 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47ac06c3f0 0x7f47ac0c8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:12.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.081+0000 7f47b45ff700 1 --2- 172.21.15.92:0/1588237933 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f47900682f0 0x7f479006a7a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:12.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.081+0000 7f47b45ff700 1 --2- 172.21.15.92:0/1588237933 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47ac0b8510 0x7f47ac0cb5c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:12.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.081+0000 7f47b45ff700 1 -- 172.21.15.92:0/1588237933 >> 172.21.15.92:0/1588237933 conn(0x7f47ac0719a0 msgr2=0x7f47ac0c96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:12.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.081+0000 7f47b45ff700 1 -- 172.21.15.92:0/1588237933 shutdown_connections 2023-11-28T11:21:12.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.081+0000 7f47b45ff700 1 -- 172.21.15.92:0/1588237933 wait complete. 2023-11-28T11:21:12.100 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:21:12.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.302+0000 7f29285ff700 1 Processor -- start 2023-11-28T11:21:12.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.303+0000 7f29285ff700 1 -- start start 2023-11-28T11:21:12.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.304+0000 7f29285ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f29200b8510 0x7f292006b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:12.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.304+0000 7f29285ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29200b88e0 con 0x7f29200cd460 2023-11-28T11:21:12.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.304+0000 7f29285ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29200cd8f0 con 0x7f29200d0d60 2023-11-28T11:21:12.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.304+0000 7f29285ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29200cdbd0 con 0x7f29200b8510 2023-11-28T11:21:12.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.304+0000 7f292639b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f29200b8510 0x7f292006b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:12.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.304+0000 7f2925b9a700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f29200cd460 0x7f292006beb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46144/0 (socket says 172.21.15.92:46144) 2023-11-28T11:21:12.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.304+0000 7f2925b9a700 1 -- 172.21.15.92:0/1504846140 learned_addr learned my addr 172.21.15.92:0/1504846140 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:12.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.305+0000 7f292639b700 1 -- 172.21.15.92:0/1504846140 >> v1:172.21.15.169:6789/0 conn(0x7f29200d0d60 legacy=0x7f29200d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:12.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.305+0000 7f292639b700 1 -- 172.21.15.92:0/1504846140 >> v1:172.21.15.92:6789/0 conn(0x7f29200cd460 legacy=0x7f292006beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:12.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.305+0000 7f292639b700 1 -- 172.21.15.92:0/1504846140 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29200cfd10 con 0x7f29200b8510 2023-11-28T11:21:12.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.305+0000 7f2925399700 1 -- 172.21.15.92:0/1504846140 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4177443670 0 0) 0x7f29200b88e0 con 0x7f29200cd460 2023-11-28T11:21:12.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.305+0000 7f292639b700 1 --2- 172.21.15.92:0/1504846140 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29200b8510 0x7f292006b970 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f291c009840 tx=0x7f291c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c756b662fe7929 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:12.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.306+0000 7f2925399700 1 -- 172.21.15.92:0/1504846140 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291c011830 con 0x7f29200b8510 2023-11-28T11:21:12.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.306+0000 7f2925399700 1 -- 172.21.15.92:0/1504846140 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f291c011990 con 0x7f29200b8510 2023-11-28T11:21:12.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.306+0000 7f2925399700 1 -- 172.21.15.92:0/1504846140 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291c011d20 con 0x7f29200b8510 2023-11-28T11:21:12.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.306+0000 7f29285ff700 1 -- 172.21.15.92:0/1504846140 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29200b8510 msgr2=0x7f292006b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:12.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.306+0000 7f29285ff700 1 --2- 172.21.15.92:0/1504846140 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29200b8510 0x7f292006b970 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f291c009840 tx=0x7f291c009b50 comp rx=0 tx=0).stop 2023-11-28T11:21:12.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.306+0000 7f29285ff700 1 -- 172.21.15.92:0/1504846140 shutdown_connections 2023-11-28T11:21:12.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.306+0000 7f29285ff700 1 --2- 172.21.15.92:0/1504846140 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29200b8510 0x7f292006b970 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:12.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.306+0000 7f29285ff700 1 -- 172.21.15.92:0/1504846140 >> 172.21.15.92:0/1504846140 conn(0x7f29200719a0 msgr2=0x7f2920071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:12.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.306+0000 7f29285ff700 1 -- 172.21.15.92:0/1504846140 shutdown_connections 2023-11-28T11:21:12.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.306+0000 7f29285ff700 1 -- 172.21.15.92:0/1504846140 wait complete. 2023-11-28T11:21:12.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.308+0000 7f29285ff700 1 Processor -- start 2023-11-28T11:21:12.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.308+0000 7f29285ff700 1 -- start start 2023-11-28T11:21:12.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.309+0000 7f29285ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29200b8510 0x7f292015abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:12.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.309+0000 7f29285ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29200cd460 0x7f29201e1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:12.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.309+0000 7f29285ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f29200d0d60 0x7f29200b7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:12.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.309+0000 7f29285ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29201e30b0 con 0x7f29200cd460 2023-11-28T11:21:12.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.309+0000 7f29285ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29201e24a0 con 0x7f29200b8510 2023-11-28T11:21:12.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.309+0000 7f29285ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29201e2780 con 0x7f29200d0d60 2023-11-28T11:21:12.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.309+0000 7f2925b9a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29200cd460 0x7f29201e1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:12.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.309+0000 7f2925b9a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29200cd460 0x7f29201e1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48948/0 (socket says 172.21.15.92:48948) 2023-11-28T11:21:12.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.309+0000 7f2925b9a700 1 -- 172.21.15.92:0/288991704 learned_addr learned my addr 172.21.15.92:0/288991704 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:12.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.309+0000 7f292639b700 1 --2- 172.21.15.92:0/288991704 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29200b8510 0x7f292015abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:12.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.309+0000 7f2926b9c700 1 --2- 172.21.15.92:0/288991704 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f29200d0d60 0x7f29200b7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:12.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.310+0000 7f2925b9a700 1 -- 172.21.15.92:0/288991704 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f29200d0d60 msgr2=0x7f29200b7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:12.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.310+0000 7f2925b9a700 1 --2- 172.21.15.92:0/288991704 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f29200d0d60 0x7f29200b7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:12.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.310+0000 7f2925b9a700 1 -- 172.21.15.92:0/288991704 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29200b8510 msgr2=0x7f292015abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:12.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.310+0000 7f2925b9a700 1 --2- 172.21.15.92:0/288991704 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29200b8510 0x7f292015abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:12.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.310+0000 7f2925b9a700 1 -- 172.21.15.92:0/288991704 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f291c0094a0 con 0x7f29200cd460 2023-11-28T11:21:12.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.310+0000 7f2926b9c700 1 --2- 172.21.15.92:0/288991704 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f29200d0d60 0x7f29200b7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:21:12.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.310+0000 7f2925b9a700 1 --2- 172.21.15.92:0/288991704 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29200cd460 0x7f29201e1a20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f291000c550 tx=0x7f291000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:12.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.310+0000 7f2916ffd700 1 -- 172.21.15.92:0/288991704 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2910004c80 con 0x7f29200cd460 2023-11-28T11:21:12.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.310+0000 7f2916ffd700 1 -- 172.21.15.92:0/288991704 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2910004e20 con 0x7f29200cd460 2023-11-28T11:21:12.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.310+0000 7f29285ff700 1 -- 172.21.15.92:0/288991704 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29200d29c0 con 0x7f29200cd460 2023-11-28T11:21:12.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.310+0000 7f2916ffd700 1 -- 172.21.15.92:0/288991704 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2910007310 con 0x7f29200cd460 2023-11-28T11:21:12.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.310+0000 7f29285ff700 1 -- 172.21.15.92:0/288991704 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29200cfff0 con 0x7f29200cd460 2023-11-28T11:21:12.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.311+0000 7f29285ff700 1 -- 172.21.15.92:0/288991704 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29200d2b50 con 0x7f29200cd460 2023-11-28T11:21:12.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.312+0000 7f2916ffd700 1 -- 172.21.15.92:0/288991704 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f291001a020 con 0x7f29200cd460 2023-11-28T11:21:12.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.313+0000 7f2916ffd700 1 --2- 172.21.15.92:0/288991704 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f29040682f0 0x7f290406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:12.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.313+0000 7f2916ffd700 1 -- 172.21.15.92:0/288991704 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2910085bc0 con 0x7f29200cd460 2023-11-28T11:21:12.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.313+0000 7f292639b700 1 --2- 172.21.15.92:0/288991704 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f29040682f0 0x7f290406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:12.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.314+0000 7f292639b700 1 --2- 172.21.15.92:0/288991704 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f29040682f0 0x7f290406a7a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f291c0090b0 tx=0x7f291c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:12.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.317+0000 7f2916ffd700 1 -- 172.21.15.92:0/288991704 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f29100520f0 con 0x7f29200cd460 2023-11-28T11:21:12.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.506+0000 7f29285ff700 1 -- 172.21.15.92:0/288991704 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2920067880 con 0x7f29200cd460 2023-11-28T11:21:12.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.506+0000 7f2916ffd700 1 -- 172.21.15.92:0/288991704 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2910055710 con 0x7f29200cd460 2023-11-28T11:21:12.508 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-11-28T11:21:12.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.508+0000 7f29285ff700 1 -- 172.21.15.92:0/288991704 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f29040682f0 msgr2=0x7f290406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:12.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.509+0000 7f29285ff700 1 --2- 172.21.15.92:0/288991704 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f29040682f0 0x7f290406a7a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f291c0090b0 tx=0x7f291c008030 comp rx=0 tx=0).stop 2023-11-28T11:21:12.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.509+0000 7f29285ff700 1 -- 172.21.15.92:0/288991704 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29200cd460 msgr2=0x7f29201e1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:12.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.509+0000 7f29285ff700 1 --2- 172.21.15.92:0/288991704 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29200cd460 0x7f29201e1a20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f291000c550 tx=0x7f291000c860 comp rx=0 tx=0).stop 2023-11-28T11:21:12.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.509+0000 7f29285ff700 1 -- 172.21.15.92:0/288991704 shutdown_connections 2023-11-28T11:21:12.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.509+0000 7f29285ff700 1 --2- 172.21.15.92:0/288991704 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f29200d0d60 0x7f29200b7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:12.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.509+0000 7f29285ff700 1 --2- 172.21.15.92:0/288991704 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f29200b8510 0x7f292015abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:12.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.509+0000 7f29285ff700 1 --2- 172.21.15.92:0/288991704 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f29040682f0 0x7f290406a7a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:12.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.509+0000 7f29285ff700 1 --2- 172.21.15.92:0/288991704 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29200cd460 0x7f29201e1a20 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:12.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.509+0000 7f29285ff700 1 -- 172.21.15.92:0/288991704 >> 172.21.15.92:0/288991704 conn(0x7f29200719a0 msgr2=0x7f29200cbef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:12.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.509+0000 7f29285ff700 1 -- 172.21.15.92:0/288991704 shutdown_connections 2023-11-28T11:21:12.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.509+0000 7f29285ff700 1 -- 172.21.15.92:0/288991704 wait complete. 2023-11-28T11:21:12.528 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T11:21:12.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.730+0000 7f483a7ff700 1 Processor -- start 2023-11-28T11:21:12.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.730+0000 7f483a7ff700 1 -- start start 2023-11-28T11:21:12.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.731+0000 7f483a7ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f48340b8510 0x7f483406b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:12.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.731+0000 7f483a7ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f48340cd460 0x7f483406beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:12.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.731+0000 7f483a7ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48340b88e0 con 0x7f483406c3f0 2023-11-28T11:21:12.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.731+0000 7f483a7ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48340cdbd0 con 0x7f48340b8510 2023-11-28T11:21:12.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.731+0000 7f483a7ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48340d02d0 con 0x7f48340cd460 2023-11-28T11:21:12.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.731+0000 7f4838d9c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f483406c3f0 0x7f483415cc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46148/0 (socket says 172.21.15.92:46148) 2023-11-28T11:21:12.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.731+0000 7f4838d9c700 1 -- 172.21.15.92:0/3539398367 learned_addr learned my addr 172.21.15.92:0/3539398367 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:12.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.732+0000 7f4833fff700 1 --2- 172.21.15.92:0/3539398367 >> v2:172.21.15.169:3300/0 conn(0x7f48340b8510 0x7f483406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:12.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.732+0000 7f48337fe700 1 --2- 172.21.15.92:0/3539398367 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f48340cd460 0x7f483406beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:12.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.732+0000 7f4832ffd700 1 -- 172.21.15.92:0/3539398367 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 444457369 0 0) 0x7f48340b88e0 con 0x7f483406c3f0 2023-11-28T11:21:12.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.732+0000 7f4832ffd700 1 -- 172.21.15.92:0/3539398367 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f481c0035b0 con 0x7f483406c3f0 2023-11-28T11:21:12.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.732+0000 7f4832ffd700 1 -- 172.21.15.92:0/3539398367 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3033694903 0 0) 0x7f481c0035b0 con 0x7f483406c3f0 2023-11-28T11:21:12.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.732+0000 7f4832ffd700 1 -- 172.21.15.92:0/3539398367 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f48340cd460 msgr2=0x7f483406beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:12.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.732+0000 7f4832ffd700 1 --2- 172.21.15.92:0/3539398367 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f48340cd460 0x7f483406beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:12.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.732+0000 7f4832ffd700 1 -- 172.21.15.92:0/3539398367 >> v2:172.21.15.169:3300/0 conn(0x7f48340b8510 msgr2=0x7f483406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:12.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.732+0000 7f4832ffd700 1 --2- 172.21.15.92:0/3539398367 >> v2:172.21.15.169:3300/0 conn(0x7f48340b8510 0x7f483406b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:12.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.732+0000 7f4832ffd700 1 -- 172.21.15.92:0/3539398367 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48340cd8f0 con 0x7f483406c3f0 2023-11-28T11:21:12.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.732+0000 7f4832ffd700 1 -- 172.21.15.92:0/3539398367 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532161587 0 0) 0x7f482c0019f0 con 0x7f483406c3f0 2023-11-28T11:21:12.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.732+0000 7f4832ffd700 1 -- 172.21.15.92:0/3539398367 >> v1:172.21.15.92:6789/0 conn(0x7f483406c3f0 legacy=0x7f483415cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:12.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.732+0000 7f4832ffd700 1 --2- 172.21.15.92:0/3539398367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f481c003a30 0x7f481c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:12.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.732+0000 7f4832ffd700 1 --2- 172.21.15.92:0/3539398367 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f48340cd460 0x7f481c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:12.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.732+0000 7f4832ffd700 1 --2- 172.21.15.92:0/3539398367 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f481c004d90 0x7f481c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:12.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.732+0000 7f4832ffd700 1 -- 172.21.15.92:0/3539398367 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48340cdbd0 con 0x7f481c003a30 2023-11-28T11:21:12.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.732+0000 7f4832ffd700 1 -- 172.21.15.92:0/3539398367 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48340d02d0 con 0x7f481c004d90 2023-11-28T11:21:12.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.732+0000 7f4832ffd700 1 -- 172.21.15.92:0/3539398367 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f481c00ba10 con 0x7f48340cd460 2023-11-28T11:21:12.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.732+0000 7f4833fff700 1 --2- 172.21.15.92:0/3539398367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f481c003a30 0x7f481c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:12.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.733+0000 7f4838d9c700 1 --2- 172.21.15.92:0/3539398367 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f481c004d90 0x7f481c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:12.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.733+0000 7f48337fe700 1 --2- 172.21.15.92:0/3539398367 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f48340cd460 0x7f481c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:12.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.733+0000 7f4833fff700 1 -- 172.21.15.92:0/3539398367 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f48340cd460 msgr2=0x7f481c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:12.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.733+0000 7f4833fff700 1 --2- 172.21.15.92:0/3539398367 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f48340cd460 0x7f481c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:12.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.733+0000 7f4833fff700 1 -- 172.21.15.92:0/3539398367 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f481c004d90 msgr2=0x7f481c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:12.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.733+0000 7f4833fff700 1 --2- 172.21.15.92:0/3539398367 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f481c004d90 0x7f481c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:12.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.733+0000 7f4833fff700 1 -- 172.21.15.92:0/3539398367 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f481c0035b0 con 0x7f481c003a30 2023-11-28T11:21:12.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.733+0000 7f48337fe700 1 --2- 172.21.15.92:0/3539398367 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f48340cd460 0x7f481c004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:21:12.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.733+0000 7f4833fff700 1 --2- 172.21.15.92:0/3539398367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f481c003a30 0x7f481c003e40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4824009b60 tx=0x7f481c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91811c9020f2526b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:12.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.733+0000 7f4832ffd700 1 -- 172.21.15.92:0/3539398367 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482400e660 con 0x7f481c003a30 2023-11-28T11:21:12.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.733+0000 7f4832ffd700 1 -- 172.21.15.92:0/3539398367 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f482400e7c0 con 0x7f481c003a30 2023-11-28T11:21:12.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.733+0000 7f4832ffd700 1 -- 172.21.15.92:0/3539398367 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482400eb30 con 0x7f481c003a30 2023-11-28T11:21:12.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.734+0000 7f483a7ff700 1 -- 172.21.15.92:0/3539398367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f481c003a30 msgr2=0x7f481c003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:12.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.734+0000 7f483a7ff700 1 --2- 172.21.15.92:0/3539398367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f481c003a30 0x7f481c003e40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4824009b60 tx=0x7f481c005880 comp rx=0 tx=0).stop 2023-11-28T11:21:12.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.734+0000 7f4838d9c700 1 -- 172.21.15.92:0/3539398367 reap_dead start 2023-11-28T11:21:12.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.734+0000 7f483a7ff700 1 -- 172.21.15.92:0/3539398367 shutdown_connections 2023-11-28T11:21:12.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.734+0000 7f483a7ff700 1 -- 172.21.15.92:0/3539398367 >> 172.21.15.92:0/3539398367 conn(0x7f48340719a0 msgr2=0x7f4834071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:12.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.734+0000 7f483a7ff700 1 -- 172.21.15.92:0/3539398367 shutdown_connections 2023-11-28T11:21:12.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.734+0000 7f483a7ff700 1 -- 172.21.15.92:0/3539398367 wait complete. 2023-11-28T11:21:12.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.736+0000 7f483a7ff700 1 Processor -- start 2023-11-28T11:21:12.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.736+0000 7f483a7ff700 1 -- start start 2023-11-28T11:21:12.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.737+0000 7f483a7ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f481c003a30 0x7f4834149bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:12.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.737+0000 7f483a7ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f48340d1c60 0x7f48340d20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:12.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.737+0000 7f483a7ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4834147900 0x7f4834147d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:12.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.737+0000 7f483a7ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4834147470 con 0x7f4834147900 2023-11-28T11:21:12.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.737+0000 7f483a7ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f483414a130 con 0x7f481c003a30 2023-11-28T11:21:12.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.737+0000 7f483a7ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48340d2610 con 0x7f48340d1c60 2023-11-28T11:21:12.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.737+0000 7f4838d9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4834147900 0x7f4834147d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:12.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.737+0000 7f4838d9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4834147900 0x7f4834147d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48968/0 (socket says 172.21.15.92:48968) 2023-11-28T11:21:12.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.737+0000 7f4838d9c700 1 -- 172.21.15.92:0/395936304 learned_addr learned my addr 172.21.15.92:0/395936304 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:12.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.737+0000 7f4833fff700 1 --2- 172.21.15.92:0/395936304 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f481c003a30 0x7f4834149bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:12.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.737+0000 7f48337fe700 1 --2- 172.21.15.92:0/395936304 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f48340d1c60 0x7f48340d20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:12.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.737+0000 7f4838d9c700 1 -- 172.21.15.92:0/395936304 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f48340d1c60 msgr2=0x7f48340d20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:12.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.737+0000 7f4838d9c700 1 --2- 172.21.15.92:0/395936304 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f48340d1c60 0x7f48340d20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:12.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.737+0000 7f4838d9c700 1 -- 172.21.15.92:0/395936304 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f481c003a30 msgr2=0x7f4834149bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:12.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.737+0000 7f4838d9c700 1 --2- 172.21.15.92:0/395936304 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f481c003a30 0x7f4834149bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:12.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.737+0000 7f4838d9c700 1 -- 172.21.15.92:0/395936304 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48340cfff0 con 0x7f4834147900 2023-11-28T11:21:12.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.738+0000 7f4838d9c700 1 --2- 172.21.15.92:0/395936304 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4834147900 0x7f4834147d70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f48340ce8c0 tx=0x7f482c00a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:12.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.738+0000 7f4830ff9700 1 -- 172.21.15.92:0/395936304 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482c00ad30 con 0x7f4834147900 2023-11-28T11:21:12.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.738+0000 7f4830ff9700 1 -- 172.21.15.92:0/395936304 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f482c0133f0 con 0x7f4834147900 2023-11-28T11:21:12.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.738+0000 7f4830ff9700 1 -- 172.21.15.92:0/395936304 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482c0137a0 con 0x7f4834147900 2023-11-28T11:21:12.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.738+0000 7f483a7ff700 1 -- 172.21.15.92:0/395936304 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48340cfd10 con 0x7f4834147900 2023-11-28T11:21:12.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.738+0000 7f483a7ff700 1 -- 172.21.15.92:0/395936304 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48340d29c0 con 0x7f4834147900 2023-11-28T11:21:12.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.739+0000 7f481a7fc700 1 -- 172.21.15.92:0/395936304 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4834147620 con 0x7f4834147900 2023-11-28T11:21:12.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.740+0000 7f4830ff9700 1 -- 172.21.15.92:0/395936304 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f482c013900 con 0x7f4834147900 2023-11-28T11:21:12.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.741+0000 7f4830ff9700 1 --2- 172.21.15.92:0/395936304 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f48200682f0 0x7f482006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:12.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.741+0000 7f4830ff9700 1 -- 172.21.15.92:0/395936304 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f482c0869b0 con 0x7f4834147900 2023-11-28T11:21:12.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.741+0000 7f4833fff700 1 --2- 172.21.15.92:0/395936304 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f48200682f0 0x7f482006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:12.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.742+0000 7f4833fff700 1 --2- 172.21.15.92:0/395936304 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f48200682f0 0x7f482006a7a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f48240090b0 tx=0x7f4824008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:12.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.745+0000 7f4830ff9700 1 -- 172.21.15.92:0/395936304 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4834147620 con 0x7f4834147900 2023-11-28T11:21:12.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.935+0000 7f481a7fc700 1 -- 172.21.15.92:0/395936304 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f4834147620 con 0x7f4834147900 2023-11-28T11:21:12.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.935+0000 7f4830ff9700 1 -- 172.21.15.92:0/395936304 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f4834147620 con 0x7f4834147900 2023-11-28T11:21:12.936 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-11-28T11:21:12.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.938+0000 7f481a7fc700 1 -- 172.21.15.92:0/395936304 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f48200682f0 msgr2=0x7f482006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:12.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.938+0000 7f481a7fc700 1 --2- 172.21.15.92:0/395936304 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f48200682f0 0x7f482006a7a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f48240090b0 tx=0x7f4824008030 comp rx=0 tx=0).stop 2023-11-28T11:21:12.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.938+0000 7f481a7fc700 1 -- 172.21.15.92:0/395936304 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4834147900 msgr2=0x7f4834147d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:12.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.938+0000 7f481a7fc700 1 --2- 172.21.15.92:0/395936304 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4834147900 0x7f4834147d70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f48340ce8c0 tx=0x7f482c00a5f0 comp rx=0 tx=0).stop 2023-11-28T11:21:12.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.938+0000 7f481a7fc700 1 -- 172.21.15.92:0/395936304 shutdown_connections 2023-11-28T11:21:12.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.938+0000 7f481a7fc700 1 --2- 172.21.15.92:0/395936304 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f48200682f0 0x7f482006a7a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:12.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.938+0000 7f481a7fc700 1 --2- 172.21.15.92:0/395936304 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4834147900 0x7f4834147d70 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:12.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.938+0000 7f481a7fc700 1 --2- 172.21.15.92:0/395936304 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f481c003a30 0x7f4834149bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:12.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.938+0000 7f481a7fc700 1 --2- 172.21.15.92:0/395936304 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f48340d1c60 0x7f48340d20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:12.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.938+0000 7f481a7fc700 1 -- 172.21.15.92:0/395936304 >> 172.21.15.92:0/395936304 conn(0x7f48340719a0 msgr2=0x7f48340b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:12.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.938+0000 7f481a7fc700 1 -- 172.21.15.92:0/395936304 shutdown_connections 2023-11-28T11:21:12.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:12.938+0000 7f481a7fc700 1 -- 172.21.15.92:0/395936304 wait complete. 2023-11-28T11:21:12.957 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T11:21:13.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.153+0000 7fe00a3ff700 1 Processor -- start 2023-11-28T11:21:13.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.154+0000 7fe00a3ff700 1 -- start start 2023-11-28T11:21:13.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.155+0000 7fe00a3ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe0040d2df0 0x7fe0040cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:13.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.155+0000 7fe00a3ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0040b1c60 0x7fe0040cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:13.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.155+0000 7fe00a3ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0040d1950 con 0x7fe0040b4fc0 2023-11-28T11:21:13.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.155+0000 7fe00a3ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0040d3d30 con 0x7fe0040d2df0 2023-11-28T11:21:13.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.155+0000 7fe00a3ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0040b2500 con 0x7fe0040b1c60 2023-11-28T11:21:13.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.155+0000 7fe003fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe0040d2df0 0x7fe0040cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:13.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.155+0000 7fe003fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe0040d2df0 0x7fe0040cd620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48982/0 (socket says 172.21.15.92:48982) 2023-11-28T11:21:13.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.155+0000 7fe003fff700 1 -- 172.21.15.92:0/4182462569 learned_addr learned my addr 172.21.15.92:0/4182462569 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:13.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.155+0000 7fe00899c700 1 --2- 172.21.15.92:0/4182462569 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0040b1c60 0x7fe0040cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:13.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.155+0000 7fe003fff700 1 -- 172.21.15.92:0/4182462569 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0040b1c60 msgr2=0x7fe0040cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:13.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.155+0000 7fe003fff700 1 --2- 172.21.15.92:0/4182462569 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0040b1c60 0x7fe0040cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:13.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.155+0000 7fe003fff700 1 -- 172.21.15.92:0/4182462569 >> v1:172.21.15.169:6789/0 conn(0x7fe0040b4fc0 legacy=0x7fe0040d31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:13.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.155+0000 7fe003fff700 1 -- 172.21.15.92:0/4182462569 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0040d3a50 con 0x7fe0040d2df0 2023-11-28T11:21:13.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.156+0000 7fe003fff700 1 --2- 172.21.15.92:0/4182462569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0040d2df0 0x7fe0040cd620 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdff40098f0 tx=0x7fdff4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa474129aef3ce59 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:13.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.156+0000 7fe002ffd700 1 -- 172.21.15.92:0/4182462569 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff4004230 con 0x7fe0040d2df0 2023-11-28T11:21:13.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.156+0000 7fe002ffd700 1 -- 172.21.15.92:0/4182462569 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdff4004390 con 0x7fe0040d2df0 2023-11-28T11:21:13.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.156+0000 7fe002ffd700 1 -- 172.21.15.92:0/4182462569 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff40036a0 con 0x7fe0040d2df0 2023-11-28T11:21:13.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.156+0000 7fe00a3ff700 1 -- 172.21.15.92:0/4182462569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0040d2df0 msgr2=0x7fe0040cd620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:13.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.156+0000 7fe00a3ff700 1 --2- 172.21.15.92:0/4182462569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0040d2df0 0x7fe0040cd620 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdff40098f0 tx=0x7fdff4009c00 comp rx=0 tx=0).stop 2023-11-28T11:21:13.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.156+0000 7fe00a3ff700 1 -- 172.21.15.92:0/4182462569 shutdown_connections 2023-11-28T11:21:13.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.156+0000 7fe00a3ff700 1 --2- 172.21.15.92:0/4182462569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0040d2df0 0x7fe0040cd620 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:13.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.156+0000 7fe00a3ff700 1 --2- 172.21.15.92:0/4182462569 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0040b1c60 0x7fe0040cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:13.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.156+0000 7fe00a3ff700 1 -- 172.21.15.92:0/4182462569 >> 172.21.15.92:0/4182462569 conn(0x7fe0040719a0 msgr2=0x7fe004071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:13.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.157+0000 7fe00a3ff700 1 -- 172.21.15.92:0/4182462569 shutdown_connections 2023-11-28T11:21:13.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.157+0000 7fe00a3ff700 1 -- 172.21.15.92:0/4182462569 wait complete. 2023-11-28T11:21:13.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.158+0000 7fe00a3ff700 1 Processor -- start 2023-11-28T11:21:13.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.158+0000 7fe00a3ff700 1 -- start start 2023-11-28T11:21:13.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.159+0000 7fe00a3ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0040b1c60 0x7fe00415ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:13.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.159+0000 7fe00a3ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0040b4fc0 0x7fe004156330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:13.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.159+0000 7fe00a3ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0040d2df0 0x7fe0041e5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:13.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.159+0000 7fe00a3ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe00415d1f0 con 0x7fe0040b1c60 2023-11-28T11:21:13.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.159+0000 7fe00a3ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe004157620 con 0x7fe0040b4fc0 2023-11-28T11:21:13.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.159+0000 7fe00a3ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe004157900 con 0x7fe0040d2df0 2023-11-28T11:21:13.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.159+0000 7fe003fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0040b1c60 0x7fe00415ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:13.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.159+0000 7fe003fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0040b1c60 0x7fe00415ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48994/0 (socket says 172.21.15.92:48994) 2023-11-28T11:21:13.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.160+0000 7fe003fff700 1 -- 172.21.15.92:0/3616000737 learned_addr learned my addr 172.21.15.92:0/3616000737 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:13.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.160+0000 7fe00899c700 1 --2- 172.21.15.92:0/3616000737 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0040d2df0 0x7fe0041e5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:13.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.160+0000 7fe0037fe700 1 --2- 172.21.15.92:0/3616000737 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0040b4fc0 0x7fe004156330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:13.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.160+0000 7fe003fff700 1 -- 172.21.15.92:0/3616000737 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0040d2df0 msgr2=0x7fe0041e5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:13.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.160+0000 7fe003fff700 1 --2- 172.21.15.92:0/3616000737 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0040d2df0 0x7fe0041e5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:13.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.160+0000 7fe003fff700 1 -- 172.21.15.92:0/3616000737 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0040b4fc0 msgr2=0x7fe004156330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:13.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.160+0000 7fe003fff700 1 --2- 172.21.15.92:0/3616000737 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0040b4fc0 0x7fe004156330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:13.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.160+0000 7fe003fff700 1 -- 172.21.15.92:0/3616000737 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff4009550 con 0x7fe0040b1c60 2023-11-28T11:21:13.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.160+0000 7fe003fff700 1 --2- 172.21.15.92:0/3616000737 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0040b1c60 0x7fe00415ccb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdff4009840 tx=0x7fdff4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:13.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.160+0000 7fe000ff9700 1 -- 172.21.15.92:0/3616000737 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff400b660 con 0x7fe0040b1c60 2023-11-28T11:21:13.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.160+0000 7fe000ff9700 1 -- 172.21.15.92:0/3616000737 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdff400b7c0 con 0x7fe0040b1c60 2023-11-28T11:21:13.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.160+0000 7fe00a3ff700 1 -- 172.21.15.92:0/3616000737 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0040b88f0 con 0x7fe0040b1c60 2023-11-28T11:21:13.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.160+0000 7fe000ff9700 1 -- 172.21.15.92:0/3616000737 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff400bb70 con 0x7fe0040b1c60 2023-11-28T11:21:13.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.161+0000 7fe00a3ff700 1 -- 172.21.15.92:0/3616000737 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0040d15e0 con 0x7fe0040b1c60 2023-11-28T11:21:13.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.162+0000 7fe00a3ff700 1 -- 172.21.15.92:0/3616000737 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0040b21b0 con 0x7fe0040b1c60 2023-11-28T11:21:13.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.162+0000 7fe000ff9700 1 -- 172.21.15.92:0/3616000737 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdff4017020 con 0x7fe0040b1c60 2023-11-28T11:21:13.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.163+0000 7fe000ff9700 1 --2- 172.21.15.92:0/3616000737 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fdff00682f0 0x7fdff006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:13.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.163+0000 7fe000ff9700 1 -- 172.21.15.92:0/3616000737 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fdff4087840 con 0x7fe0040b1c60 2023-11-28T11:21:13.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.164+0000 7fe0037fe700 1 --2- 172.21.15.92:0/3616000737 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fdff00682f0 0x7fdff006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:13.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.164+0000 7fe0037fe700 1 --2- 172.21.15.92:0/3616000737 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fdff00682f0 0x7fdff006a7a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe0040ccf30 tx=0x7fdff8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:13.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.168+0000 7fe000ff9700 1 -- 172.21.15.92:0/3616000737 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fdff4053cc0 con 0x7fe0040b1c60 2023-11-28T11:21:13.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.370+0000 7fe00a3ff700 1 -- 172.21.15.92:0/3616000737 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe004067170 con 0x7fe0040b1c60 2023-11-28T11:21:13.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.371+0000 7fe000ff9700 1 -- 172.21.15.92:0/3616000737 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7fdff40572e0 con 0x7fe0040b1c60 2023-11-28T11:21:13.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.373+0000 7fe00a3ff700 1 -- 172.21.15.92:0/3616000737 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fdff00682f0 msgr2=0x7fdff006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:13.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.373+0000 7fe00a3ff700 1 --2- 172.21.15.92:0/3616000737 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fdff00682f0 0x7fdff006a7a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe0040ccf30 tx=0x7fdff8008040 comp rx=0 tx=0).stop 2023-11-28T11:21:13.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.373+0000 7fe00a3ff700 1 -- 172.21.15.92:0/3616000737 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0040b1c60 msgr2=0x7fe00415ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:13.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.373+0000 7fe00a3ff700 1 --2- 172.21.15.92:0/3616000737 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0040b1c60 0x7fe00415ccb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdff4009840 tx=0x7fdff4004750 comp rx=0 tx=0).stop 2023-11-28T11:21:13.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.373+0000 7fe00a3ff700 1 -- 172.21.15.92:0/3616000737 shutdown_connections 2023-11-28T11:21:13.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.373+0000 7fe00a3ff700 1 --2- 172.21.15.92:0/3616000737 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0040d2df0 0x7fe0041e5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:13.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.373+0000 7fe00a3ff700 1 --2- 172.21.15.92:0/3616000737 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fdff00682f0 0x7fdff006a7a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:13.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.373+0000 7fe00a3ff700 1 --2- 172.21.15.92:0/3616000737 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0040b1c60 0x7fe00415ccb0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:13.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.373+0000 7fe00a3ff700 1 --2- 172.21.15.92:0/3616000737 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0040b4fc0 0x7fe004156330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:13.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.373+0000 7fe00a3ff700 1 -- 172.21.15.92:0/3616000737 >> 172.21.15.92:0/3616000737 conn(0x7fe0040719a0 msgr2=0x7fe0040b74d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:13.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.373+0000 7fe00a3ff700 1 -- 172.21.15.92:0/3616000737 shutdown_connections 2023-11-28T11:21:13.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.373+0000 7fe00a3ff700 1 -- 172.21.15.92:0/3616000737 wait complete. 2023-11-28T11:21:13.376 INFO:teuthology.orchestra.run.smithi092.stderr:listed 0 entries 2023-11-28T11:21:13.392 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi092.front.sepia.ceph.com... 2023-11-28T11:21:13.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:13.392 DEBUG:teuthology.orchestra.run.smithi092:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-11-28T11:21:13.459 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi092.stderr:ceph-fuse[123359]: fuse finished with error 0 and tester_r 0 2023-11-28T11:21:13.464 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:21:13.489 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2023-11-28T11:21:13.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:13.489 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-11-28T11:21:13.584 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:21:13.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.779+0000 7fd9fd5ff700 1 Processor -- start 2023-11-28T11:21:13.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.779+0000 7fd9fd5ff700 1 -- start start 2023-11-28T11:21:13.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.780+0000 7fd9fd5ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9f80cd460 0x7fd9f806c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:13.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.780+0000 7fd9fd5ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd9f80d0d60 0x7fd9f80d11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:13.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.781+0000 7fd9fd5ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9f80b88e0 con 0x7fd9f80b8510 2023-11-28T11:21:13.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.781+0000 7fd9fd5ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9f80cdbd0 con 0x7fd9f80d0d60 2023-11-28T11:21:13.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.781+0000 7fd9fd5ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f80d02d0 con 0x7fd9f80cd460 2023-11-28T11:21:13.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.781+0000 7fd9f6ffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd9f80b8510 0x7fd9f806b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46160/0 (socket says 172.21.15.92:46160) 2023-11-28T11:21:13.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.781+0000 7fd9f6ffd700 1 -- 172.21.15.92:0/307656228 learned_addr learned my addr 172.21.15.92:0/307656228 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:13.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.781+0000 7fd9f77fe700 1 --2- 172.21.15.92:0/307656228 >> v2:172.21.15.169:3300/0 conn(0x7fd9f80d0d60 0x7fd9f80d11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:13.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.781+0000 7fd9effff700 1 --2- 172.21.15.92:0/307656228 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9f80cd460 0x7fd9f806c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:13.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.781+0000 7fd9f67fc700 1 -- 172.21.15.92:0/307656228 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3997996780 0 0) 0x7fd9f80b88e0 con 0x7fd9f80b8510 2023-11-28T11:21:13.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.781+0000 7fd9f67fc700 1 -- 172.21.15.92:0/307656228 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9e40035b0 con 0x7fd9f80b8510 2023-11-28T11:21:13.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.781+0000 7fd9effff700 1 -- 172.21.15.92:0/307656228 >> v2:172.21.15.169:3300/0 conn(0x7fd9f80d0d60 msgr2=0x7fd9f80d11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:13.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.781+0000 7fd9effff700 1 --2- 172.21.15.92:0/307656228 >> v2:172.21.15.169:3300/0 conn(0x7fd9f80d0d60 0x7fd9f80d11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:13.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.781+0000 7fd9effff700 1 -- 172.21.15.92:0/307656228 >> v1:172.21.15.92:6789/0 conn(0x7fd9f80b8510 legacy=0x7fd9f806b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:13.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.781+0000 7fd9effff700 1 -- 172.21.15.92:0/307656228 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f80cd8f0 con 0x7fd9f80cd460 2023-11-28T11:21:13.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.781+0000 7fd9f77fe700 1 --2- 172.21.15.92:0/307656228 >> v2:172.21.15.169:3300/0 conn(0x7fd9f80d0d60 0x7fd9f80d11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:21:13.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.782+0000 7fd9effff700 1 --2- 172.21.15.92:0/307656228 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9f80cd460 0x7fd9f806c080 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd9e0009840 tx=0x7fd9e0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0fbcbb302094411 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:13.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.782+0000 7fd9f67fc700 1 -- 172.21.15.92:0/307656228 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e0011830 con 0x7fd9f80cd460 2023-11-28T11:21:13.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.782+0000 7fd9f67fc700 1 -- 172.21.15.92:0/307656228 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9e0011990 con 0x7fd9f80cd460 2023-11-28T11:21:13.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.782+0000 7fd9f67fc700 1 -- 172.21.15.92:0/307656228 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e0011d00 con 0x7fd9f80cd460 2023-11-28T11:21:13.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.783+0000 7fd9fd5ff700 1 -- 172.21.15.92:0/307656228 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9f80cd460 msgr2=0x7fd9f806c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:13.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.783+0000 7fd9fd5ff700 1 --2- 172.21.15.92:0/307656228 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9f80cd460 0x7fd9f806c080 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd9e0009840 tx=0x7fd9e0009b50 comp rx=0 tx=0).stop 2023-11-28T11:21:13.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.783+0000 7fd9fd5ff700 1 -- 172.21.15.92:0/307656228 shutdown_connections 2023-11-28T11:21:13.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.783+0000 7fd9fd5ff700 1 --2- 172.21.15.92:0/307656228 >> v2:172.21.15.169:3300/0 conn(0x7fd9f80d0d60 0x7fd9f80d11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:13.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.783+0000 7fd9fd5ff700 1 --2- 172.21.15.92:0/307656228 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9f80cd460 0x7fd9f806c080 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:13.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.783+0000 7fd9fd5ff700 1 -- 172.21.15.92:0/307656228 >> 172.21.15.92:0/307656228 conn(0x7fd9f80719a0 msgr2=0x7fd9f8071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:13.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.783+0000 7fd9fd5ff700 1 -- 172.21.15.92:0/307656228 shutdown_connections 2023-11-28T11:21:13.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.783+0000 7fd9fd5ff700 1 -- 172.21.15.92:0/307656228 wait complete. 2023-11-28T11:21:13.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.785+0000 7fd9fd5ff700 1 Processor -- start 2023-11-28T11:21:13.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.785+0000 7fd9fd5ff700 1 -- start start 2023-11-28T11:21:13.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.786+0000 7fd9fd5ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9f80b8510 0x7fd9f8156fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:13.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.786+0000 7fd9fd5ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9f80cd460 0x7fd9f80cc280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:13.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.786+0000 7fd9fd5ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9f80d0d60 0x7fd9f8156770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:13.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.786+0000 7fd9fd5ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f80cbdf0 con 0x7fd9f80d0d60 2023-11-28T11:21:13.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.786+0000 7fd9fd5ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f80cb1e0 con 0x7fd9f80b8510 2023-11-28T11:21:13.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.786+0000 7fd9fd5ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f80cb4c0 con 0x7fd9f80cd460 2023-11-28T11:21:13.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.786+0000 7fd9f77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9f80d0d60 0x7fd9f8156770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:13.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.786+0000 7fd9f77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9f80d0d60 0x7fd9f8156770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49004/0 (socket says 172.21.15.92:49004) 2023-11-28T11:21:13.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.786+0000 7fd9f77fe700 1 -- 172.21.15.92:0/2134623117 learned_addr learned my addr 172.21.15.92:0/2134623117 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:13.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.786+0000 7fd9effff700 1 --2- 172.21.15.92:0/2134623117 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9f80cd460 0x7fd9f80cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:13.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.786+0000 7fd9f6ffd700 1 --2- 172.21.15.92:0/2134623117 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9f80b8510 0x7fd9f8156fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:13.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.787+0000 7fd9f77fe700 1 -- 172.21.15.92:0/2134623117 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9f80cd460 msgr2=0x7fd9f80cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:13.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.787+0000 7fd9f77fe700 1 --2- 172.21.15.92:0/2134623117 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9f80cd460 0x7fd9f80cc280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:13.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.787+0000 7fd9f77fe700 1 -- 172.21.15.92:0/2134623117 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9f80b8510 msgr2=0x7fd9f8156fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:13.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.787+0000 7fd9f77fe700 1 --2- 172.21.15.92:0/2134623117 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9f80b8510 0x7fd9f8156fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:13.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.787+0000 7fd9f77fe700 1 -- 172.21.15.92:0/2134623117 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f0009a70 con 0x7fd9f80d0d60 2023-11-28T11:21:13.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.787+0000 7fd9f77fe700 1 --2- 172.21.15.92:0/2134623117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9f80d0d60 0x7fd9f8156770 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd9f000efc0 tx=0x7fd9f000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:13.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.787+0000 7fd9ef7fe700 1 -- 172.21.15.92:0/2134623117 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f000ca90 con 0x7fd9f80d0d60 2023-11-28T11:21:13.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.787+0000 7fd9ef7fe700 1 -- 172.21.15.92:0/2134623117 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f000f040 con 0x7fd9f80d0d60 2023-11-28T11:21:13.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.787+0000 7fd9fd5ff700 1 -- 172.21.15.92:0/2134623117 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9e00094a0 con 0x7fd9f80d0d60 2023-11-28T11:21:13.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.787+0000 7fd9ef7fe700 1 -- 172.21.15.92:0/2134623117 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f000ce30 con 0x7fd9f80d0d60 2023-11-28T11:21:13.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.787+0000 7fd9fd5ff700 1 -- 172.21.15.92:0/2134623117 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9f80cfff0 con 0x7fd9f80d0d60 2023-11-28T11:21:13.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.788+0000 7fd9fd5ff700 1 -- 172.21.15.92:0/2134623117 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9f80cfd10 con 0x7fd9f80d0d60 2023-11-28T11:21:13.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.789+0000 7fd9ef7fe700 1 -- 172.21.15.92:0/2134623117 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd9f0014480 con 0x7fd9f80d0d60 2023-11-28T11:21:13.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.790+0000 7fd9ef7fe700 1 --2- 172.21.15.92:0/2134623117 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd9d80682f0 0x7fd9d806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:13.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.790+0000 7fd9ef7fe700 1 -- 172.21.15.92:0/2134623117 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd9f0086800 con 0x7fd9f80d0d60 2023-11-28T11:21:13.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.790+0000 7fd9f6ffd700 1 --2- 172.21.15.92:0/2134623117 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd9d80682f0 0x7fd9d806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:13.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.791+0000 7fd9f6ffd700 1 --2- 172.21.15.92:0/2134623117 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd9d80682f0 0x7fd9d806a7a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd9e8007fd0 tx=0x7fd9e8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:13.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.794+0000 7fd9ef7fe700 1 -- 172.21.15.92:0/2134623117 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd9f0052c20 con 0x7fd9f80d0d60 2023-11-28T11:21:13.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.988+0000 7fd9fd5ff700 1 -- 172.21.15.92:0/2134623117 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd9f8067880 con 0x7fd9f80d0d60 2023-11-28T11:21:13.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.990+0000 7fd9ef7fe700 1 -- 172.21.15.92:0/2134623117 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+19748 (secure 0 0 0) 0x7fd9f0009cd0 con 0x7fd9f80d0d60 2023-11-28T11:21:13.992 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:21:13.992 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":27,"fsid":"68956e0c-c16f-47cd-a220-f98364b0831d","created":"2023-11-28T11:20:28.697573+0000","modified":"2023-11-28T11:21:00.770781+0000","last_up_change":"2023-11-28T11:20:41.257011+0000","last_in_change":"2023-11-28T11:20:32.249767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:20:41.925237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:20:48.682497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:20:54.471398+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:20:55.204585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:20:58.297307+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6abe483f-0e03-4a26-b9cf-62041291747e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":115823},{"type":"v1","addr":"172.21.15.92:6809","nonce":115823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":115823},{"type":"v1","addr":"172.21.15.92:6813","nonce":115823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":115823},{"type":"v1","addr":"172.21.15.92:6821","nonce":115823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":115823},{"type":"v1","addr":"172.21.15.92:6817","nonce":115823}]},"public_addr":"172.21.15.92:6809/115823","cluster_addr":"172.21.15.92:6813/115823","heartbeat_back_addr":"172.21.15.92:6821/115823","heartbeat_front_addr":"172.21.15.92:6817/115823","state":["exists","up"]},{"osd":1,"uuid":"13c1de73-1a8c-4d7e-8a28-9dae52dfe089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":115824},{"type":"v1","addr":"172.21.15.92:6812","nonce":115824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":115824},{"type":"v1","addr":"172.21.15.92:6816","nonce":115824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":115824},{"type":"v1","addr":"172.21.15.92:6823","nonce":115824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":115824},{"type":"v1","addr":"172.21.15.92:6820","nonce":115824}]},"public_addr":"172.21.15.92:6812/115824","cluster_addr":"172.21.15.92:6816/115824","heartbeat_back_addr":"172.21.15.92:6823/115824","heartbeat_front_addr":"172.21.15.92:6820/115824","state":["exists","up"]},{"osd":2,"uuid":"25788a55-112f-4f00-810d-2bc37cf76778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":115821},{"type":"v1","addr":"172.21.15.92:6801","nonce":115821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":115821},{"type":"v1","addr":"172.21.15.92:6803","nonce":115821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":115821},{"type":"v1","addr":"172.21.15.92:6807","nonce":115821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":115821},{"type":"v1","addr":"172.21.15.92:6805","nonce":115821}]},"public_addr":"172.21.15.92:6801/115821","cluster_addr":"172.21.15.92:6803/115821","heartbeat_back_addr":"172.21.15.92:6807/115821","heartbeat_front_addr":"172.21.15.92:6805/115821","state":["exists","up"]},{"osd":3,"uuid":"adf54e24-33a9-41fe-9ce3-98a5ab6c158e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":115825},{"type":"v1","addr":"172.21.15.92:6825","nonce":115825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":115825},{"type":"v1","addr":"172.21.15.92:6827","nonce":115825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":115825},{"type":"v1","addr":"172.21.15.92:6831","nonce":115825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":115825},{"type":"v1","addr":"172.21.15.92:6829","nonce":115825}]},"public_addr":"172.21.15.92:6825/115825","cluster_addr":"172.21.15.92:6827/115825","heartbeat_back_addr":"172.21.15.92:6831/115825","heartbeat_front_addr":"172.21.15.92:6829/115825","state":["exists","up"]},{"osd":4,"uuid":"ccb9b747-f399-44ea-9a18-9db3152227e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":116035},{"type":"v1","addr":"172.21.15.169:6817","nonce":116035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":116035},{"type":"v1","addr":"172.21.15.169:6819","nonce":116035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":116035},{"type":"v1","addr":"172.21.15.169:6826","nonce":116035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":116035},{"type":"v1","addr":"172.21.15.169:6822","nonce":116035}]},"public_addr":"172.21.15.169:6817/116035","cluster_addr":"172.21.15.169:6819/116035","heartbeat_back_addr":"172.21.15.169:6826/116035","heartbeat_front_addr":"172.21.15.169:6822/116035","state":["exists","up"]},{"osd":5,"uuid":"308a1e63-75aa-4d7d-8a7c-cecc24714fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":116033},{"type":"v1","addr":"172.21.15.169:6801","nonce":116033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":116033},{"type":"v1","addr":"172.21.15.169:6803","nonce":116033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":116033},{"type":"v1","addr":"172.21.15.169:6807","nonce":116033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":116033},{"type":"v1","addr":"172.21.15.169:6805","nonce":116033}]},"public_addr":"172.21.15.169:6801/116033","cluster_addr":"172.21.15.169:6803/116033","heartbeat_back_addr":"172.21.15.169:6807/116033","heartbeat_front_addr":"172.21.15.169:6805/116033","state":["exists","up"]},{"osd":6,"uuid":"9c459b7b-b821-4b4b-b91e-6aaae271457c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":116034},{"type":"v1","addr":"172.21.15.169:6809","nonce":116034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":116034},{"type":"v1","addr":"172.21.15.169:6811","nonce":116034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":116034},{"type":"v1","addr":"172.21.15.169:6815","nonce":116034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":116034},{"type":"v1","addr":"172.21.15.169:6813","nonce":116034}]},"public_addr":"172.21.15.169:6809/116034","cluster_addr":"172.21.15.169:6811/116034","heartbeat_back_addr":"172.21.15.169:6815/116034","heartbeat_front_addr":"172.21.15.169:6813/116034","state":["exists","up"]},{"osd":7,"uuid":"1ca9aa4a-755d-4e28-a937-4d8dccab4df3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":116044},{"type":"v1","addr":"172.21.15.169:6823","nonce":116044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6825","nonce":116044},{"type":"v1","addr":"172.21.15.169:6827","nonce":116044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":116044},{"type":"v1","addr":"172.21.15.169:6831","nonce":116044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":116044},{"type":"v1","addr":"172.21.15.169:6829","nonce":116044}]},"public_addr":"172.21.15.169:6823/116044","cluster_addr":"172.21.15.169:6827/116044","heartbeat_back_addr":"172.21.15.169:6831/116044","heartbeat_front_addr":"172.21.15.169:6829/116044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.586781+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.869778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.508594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.860493+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.872267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.746933+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.611520+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.871446+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:21:13.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.992+0000 7fd9fd5ff700 1 -- 172.21.15.92:0/2134623117 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd9d80682f0 msgr2=0x7fd9d806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:13.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.992+0000 7fd9fd5ff700 1 --2- 172.21.15.92:0/2134623117 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd9d80682f0 0x7fd9d806a7a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd9e8007fd0 tx=0x7fd9e8009040 comp rx=0 tx=0).stop 2023-11-28T11:21:13.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.992+0000 7fd9fd5ff700 1 -- 172.21.15.92:0/2134623117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9f80d0d60 msgr2=0x7fd9f8156770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:13.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.992+0000 7fd9fd5ff700 1 --2- 172.21.15.92:0/2134623117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9f80d0d60 0x7fd9f8156770 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd9f000efc0 tx=0x7fd9f000c3a0 comp rx=0 tx=0).stop 2023-11-28T11:21:13.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.992+0000 7fd9fd5ff700 1 -- 172.21.15.92:0/2134623117 shutdown_connections 2023-11-28T11:21:13.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.992+0000 7fd9fd5ff700 1 --2- 172.21.15.92:0/2134623117 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd9d80682f0 0x7fd9d806a7a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:13.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.992+0000 7fd9fd5ff700 1 --2- 172.21.15.92:0/2134623117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9f80d0d60 0x7fd9f8156770 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:13.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.992+0000 7fd9fd5ff700 1 --2- 172.21.15.92:0/2134623117 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd9f80b8510 0x7fd9f8156fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:13.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.992+0000 7fd9fd5ff700 1 --2- 172.21.15.92:0/2134623117 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd9f80cd460 0x7fd9f80cc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:13.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.992+0000 7fd9fd5ff700 1 -- 172.21.15.92:0/2134623117 >> 172.21.15.92:0/2134623117 conn(0x7fd9f80719a0 msgr2=0x7fd9f80b3370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:13.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.992+0000 7fd9fd5ff700 1 -- 172.21.15.92:0/2134623117 shutdown_connections 2023-11-28T11:21:13.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:13.992+0000 7fd9fd5ff700 1 -- 172.21.15.92:0/2134623117 wait complete. 2023-11-28T11:21:14.012 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:21:14.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.206+0000 7f9b4b3ff700 1 Processor -- start 2023-11-28T11:21:14.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.206+0000 7f9b4b3ff700 1 -- start start 2023-11-28T11:21:14.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.207+0000 7f9b4b3ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9b440cd460 0x7f9b4406c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:14.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.208+0000 7f9b4b3ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9b440d0d60 0x7f9b440d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:14.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.208+0000 7f9b4b3ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b440b88e0 con 0x7f9b440b8510 2023-11-28T11:21:14.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.208+0000 7f9b4b3ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b440cdbd0 con 0x7f9b440d0d60 2023-11-28T11:21:14.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.208+0000 7f9b4b3ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b440d02d0 con 0x7f9b440cd460 2023-11-28T11:21:14.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.208+0000 7f9b4999c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9b440d0d60 0x7f9b440d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:14.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.208+0000 7f9b4999c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9b440d0d60 0x7f9b440d1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49018/0 (socket says 172.21.15.92:49018) 2023-11-28T11:21:14.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.208+0000 7f9b4899a700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9b440cd460 0x7f9b4406c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:14.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.208+0000 7f9b4999c700 1 -- 172.21.15.92:0/1625503404 learned_addr learned my addr 172.21.15.92:0/1625503404 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:14.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.208+0000 7f9b4999c700 1 -- 172.21.15.92:0/1625503404 >> v2:172.21.15.169:3300/0 conn(0x7f9b440cd460 msgr2=0x7f9b4406c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:14.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.208+0000 7f9b4999c700 1 --2- 172.21.15.92:0/1625503404 >> v2:172.21.15.169:3300/0 conn(0x7f9b440cd460 0x7f9b4406c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:14.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.208+0000 7f9b4999c700 1 -- 172.21.15.92:0/1625503404 >> v1:172.21.15.169:6789/0 conn(0x7f9b440b8510 legacy=0x7f9b4406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:14.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.208+0000 7f9b4999c700 1 -- 172.21.15.92:0/1625503404 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b440cd8f0 con 0x7f9b440d0d60 2023-11-28T11:21:14.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.208+0000 7f9b3bfff700 1 -- 172.21.15.92:0/1625503404 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1713337852 0 0) 0x7f9b440b88e0 con 0x7f9b440b8510 2023-11-28T11:21:14.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.209+0000 7f9b4999c700 1 --2- 172.21.15.92:0/1625503404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b440d0d60 0x7f9b440d1200 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9b440ceba0 tx=0x7f9b3c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93c09d0f460eb815 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:14.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.209+0000 7f9b3bfff700 1 -- 172.21.15.92:0/1625503404 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c00e070 con 0x7f9b440d0d60 2023-11-28T11:21:14.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.209+0000 7f9b3bfff700 1 -- 172.21.15.92:0/1625503404 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b3c00b8d0 con 0x7f9b440d0d60 2023-11-28T11:21:14.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.209+0000 7f9b3bfff700 1 -- 172.21.15.92:0/1625503404 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c00bc80 con 0x7f9b440d0d60 2023-11-28T11:21:14.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.209+0000 7f9b4b3ff700 1 -- 172.21.15.92:0/1625503404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b440d0d60 msgr2=0x7f9b440d1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:14.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.209+0000 7f9b4b3ff700 1 --2- 172.21.15.92:0/1625503404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b440d0d60 0x7f9b440d1200 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9b440ceba0 tx=0x7f9b3c00b330 comp rx=0 tx=0).stop 2023-11-28T11:21:14.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.210+0000 7f9b4b3ff700 1 -- 172.21.15.92:0/1625503404 shutdown_connections 2023-11-28T11:21:14.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.210+0000 7f9b4b3ff700 1 --2- 172.21.15.92:0/1625503404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b440d0d60 0x7f9b440d1200 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:14.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.210+0000 7f9b4b3ff700 1 --2- 172.21.15.92:0/1625503404 >> v2:172.21.15.169:3300/0 conn(0x7f9b440cd460 0x7f9b4406c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:14.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.210+0000 7f9b4b3ff700 1 -- 172.21.15.92:0/1625503404 >> 172.21.15.92:0/1625503404 conn(0x7f9b440719a0 msgr2=0x7f9b44071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:14.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.210+0000 7f9b4b3ff700 1 -- 172.21.15.92:0/1625503404 shutdown_connections 2023-11-28T11:21:14.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.210+0000 7f9b4b3ff700 1 -- 172.21.15.92:0/1625503404 wait complete. 2023-11-28T11:21:14.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.211+0000 7f9b4b3ff700 1 Processor -- start 2023-11-28T11:21:14.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.212+0000 7f9b4b3ff700 1 -- start start 2023-11-28T11:21:14.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.212+0000 7f9b4b3ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b440b8510 0x7f9b4415ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:14.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.212+0000 7f9b4b3ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b440cd460 0x7f9b441e7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:14.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.212+0000 7f9b4b3ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b440d0d60 0x7f9b441e75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:14.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.213+0000 7f9b4b3ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b441f5250 con 0x7f9b440b8510 2023-11-28T11:21:14.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.213+0000 7f9b4b3ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b441e4390 con 0x7f9b440d0d60 2023-11-28T11:21:14.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.213+0000 7f9b4b3ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b441e3780 con 0x7f9b440cd460 2023-11-28T11:21:14.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.213+0000 7f9b4919b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b440b8510 0x7f9b4415ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:14.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.213+0000 7f9b4919b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b440b8510 0x7f9b4415ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49028/0 (socket says 172.21.15.92:49028) 2023-11-28T11:21:14.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.213+0000 7f9b4919b700 1 -- 172.21.15.92:0/2628061635 learned_addr learned my addr 172.21.15.92:0/2628061635 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:14.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.213+0000 7f9b4899a700 1 --2- 172.21.15.92:0/2628061635 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b440cd460 0x7f9b441e7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:14.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.213+0000 7f9b4999c700 1 --2- 172.21.15.92:0/2628061635 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b440d0d60 0x7f9b441e75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:14.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.213+0000 7f9b4919b700 1 -- 172.21.15.92:0/2628061635 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b440cd460 msgr2=0x7f9b441e7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:14.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.213+0000 7f9b4919b700 1 --2- 172.21.15.92:0/2628061635 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b440cd460 0x7f9b441e7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:14.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.213+0000 7f9b4919b700 1 -- 172.21.15.92:0/2628061635 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b440d0d60 msgr2=0x7f9b441e75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:14.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.213+0000 7f9b4919b700 1 --2- 172.21.15.92:0/2628061635 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b440d0d60 0x7f9b441e75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:14.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.213+0000 7f9b4919b700 1 -- 172.21.15.92:0/2628061635 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b3c009b30 con 0x7f9b440b8510 2023-11-28T11:21:14.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.213+0000 7f9b4919b700 1 --2- 172.21.15.92:0/2628061635 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b440b8510 0x7f9b4415ad40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9b40001eb0 tx=0x7f9b4000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:14.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.214+0000 7f9b39ffb700 1 -- 172.21.15.92:0/2628061635 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b400054e0 con 0x7f9b440b8510 2023-11-28T11:21:14.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.214+0000 7f9b39ffb700 1 -- 172.21.15.92:0/2628061635 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b40003680 con 0x7f9b440b8510 2023-11-28T11:21:14.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.214+0000 7f9b39ffb700 1 -- 172.21.15.92:0/2628061635 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b400039f0 con 0x7f9b440b8510 2023-11-28T11:21:14.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.214+0000 7f9b4b3ff700 1 -- 172.21.15.92:0/2628061635 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b440cfff0 con 0x7f9b440b8510 2023-11-28T11:21:14.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.214+0000 7f9b4b3ff700 1 -- 172.21.15.92:0/2628061635 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b440cfd10 con 0x7f9b440b8510 2023-11-28T11:21:14.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.215+0000 7f9b4b3ff700 1 -- 172.21.15.92:0/2628061635 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b44067880 con 0x7f9b440b8510 2023-11-28T11:21:14.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.216+0000 7f9b39ffb700 1 -- 172.21.15.92:0/2628061635 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9b40019030 con 0x7f9b440b8510 2023-11-28T11:21:14.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.217+0000 7f9b39ffb700 1 --2- 172.21.15.92:0/2628061635 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9b280682f0 0x7f9b2806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:14.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.217+0000 7f9b39ffb700 1 -- 172.21.15.92:0/2628061635 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f9b40085100 con 0x7f9b440b8510 2023-11-28T11:21:14.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.217+0000 7f9b4899a700 1 --2- 172.21.15.92:0/2628061635 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9b280682f0 0x7f9b2806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:14.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.218+0000 7f9b4899a700 1 --2- 172.21.15.92:0/2628061635 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9b280682f0 0x7f9b2806a7a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9b440b8070 tx=0x7f9b34009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:14.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.221+0000 7f9b39ffb700 1 -- 172.21.15.92:0/2628061635 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9b4004b5e0 con 0x7f9b440b8510 2023-11-28T11:21:14.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.416+0000 7f9b4b3ff700 1 -- 172.21.15.92:0/2628061635 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9b440d2940 con 0x7f9b440b8510 2023-11-28T11:21:14.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.417+0000 7f9b39ffb700 1 -- 172.21.15.92:0/2628061635 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+19748 (secure 0 0 0) 0x7f9b40017090 con 0x7f9b440b8510 2023-11-28T11:21:14.420 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:21:14.420 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":27,"fsid":"68956e0c-c16f-47cd-a220-f98364b0831d","created":"2023-11-28T11:20:28.697573+0000","modified":"2023-11-28T11:21:00.770781+0000","last_up_change":"2023-11-28T11:20:41.257011+0000","last_in_change":"2023-11-28T11:20:32.249767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:20:41.925237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:20:48.682497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:20:54.471398+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:20:55.204585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:20:58.297307+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6abe483f-0e03-4a26-b9cf-62041291747e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":115823},{"type":"v1","addr":"172.21.15.92:6809","nonce":115823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":115823},{"type":"v1","addr":"172.21.15.92:6813","nonce":115823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":115823},{"type":"v1","addr":"172.21.15.92:6821","nonce":115823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":115823},{"type":"v1","addr":"172.21.15.92:6817","nonce":115823}]},"public_addr":"172.21.15.92:6809/115823","cluster_addr":"172.21.15.92:6813/115823","heartbeat_back_addr":"172.21.15.92:6821/115823","heartbeat_front_addr":"172.21.15.92:6817/115823","state":["exists","up"]},{"osd":1,"uuid":"13c1de73-1a8c-4d7e-8a28-9dae52dfe089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":115824},{"type":"v1","addr":"172.21.15.92:6812","nonce":115824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":115824},{"type":"v1","addr":"172.21.15.92:6816","nonce":115824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":115824},{"type":"v1","addr":"172.21.15.92:6823","nonce":115824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":115824},{"type":"v1","addr":"172.21.15.92:6820","nonce":115824}]},"public_addr":"172.21.15.92:6812/115824","cluster_addr":"172.21.15.92:6816/115824","heartbeat_back_addr":"172.21.15.92:6823/115824","heartbeat_front_addr":"172.21.15.92:6820/115824","state":["exists","up"]},{"osd":2,"uuid":"25788a55-112f-4f00-810d-2bc37cf76778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":115821},{"type":"v1","addr":"172.21.15.92:6801","nonce":115821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":115821},{"type":"v1","addr":"172.21.15.92:6803","nonce":115821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":115821},{"type":"v1","addr":"172.21.15.92:6807","nonce":115821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":115821},{"type":"v1","addr":"172.21.15.92:6805","nonce":115821}]},"public_addr":"172.21.15.92:6801/115821","cluster_addr":"172.21.15.92:6803/115821","heartbeat_back_addr":"172.21.15.92:6807/115821","heartbeat_front_addr":"172.21.15.92:6805/115821","state":["exists","up"]},{"osd":3,"uuid":"adf54e24-33a9-41fe-9ce3-98a5ab6c158e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":115825},{"type":"v1","addr":"172.21.15.92:6825","nonce":115825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":115825},{"type":"v1","addr":"172.21.15.92:6827","nonce":115825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":115825},{"type":"v1","addr":"172.21.15.92:6831","nonce":115825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":115825},{"type":"v1","addr":"172.21.15.92:6829","nonce":115825}]},"public_addr":"172.21.15.92:6825/115825","cluster_addr":"172.21.15.92:6827/115825","heartbeat_back_addr":"172.21.15.92:6831/115825","heartbeat_front_addr":"172.21.15.92:6829/115825","state":["exists","up"]},{"osd":4,"uuid":"ccb9b747-f399-44ea-9a18-9db3152227e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":116035},{"type":"v1","addr":"172.21.15.169:6817","nonce":116035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":116035},{"type":"v1","addr":"172.21.15.169:6819","nonce":116035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":116035},{"type":"v1","addr":"172.21.15.169:6826","nonce":116035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":116035},{"type":"v1","addr":"172.21.15.169:6822","nonce":116035}]},"public_addr":"172.21.15.169:6817/116035","cluster_addr":"172.21.15.169:6819/116035","heartbeat_back_addr":"172.21.15.169:6826/116035","heartbeat_front_addr":"172.21.15.169:6822/116035","state":["exists","up"]},{"osd":5,"uuid":"308a1e63-75aa-4d7d-8a7c-cecc24714fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":116033},{"type":"v1","addr":"172.21.15.169:6801","nonce":116033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":116033},{"type":"v1","addr":"172.21.15.169:6803","nonce":116033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":116033},{"type":"v1","addr":"172.21.15.169:6807","nonce":116033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":116033},{"type":"v1","addr":"172.21.15.169:6805","nonce":116033}]},"public_addr":"172.21.15.169:6801/116033","cluster_addr":"172.21.15.169:6803/116033","heartbeat_back_addr":"172.21.15.169:6807/116033","heartbeat_front_addr":"172.21.15.169:6805/116033","state":["exists","up"]},{"osd":6,"uuid":"9c459b7b-b821-4b4b-b91e-6aaae271457c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":116034},{"type":"v1","addr":"172.21.15.169:6809","nonce":116034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":116034},{"type":"v1","addr":"172.21.15.169:6811","nonce":116034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":116034},{"type":"v1","addr":"172.21.15.169:6815","nonce":116034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":116034},{"type":"v1","addr":"172.21.15.169:6813","nonce":116034}]},"public_addr":"172.21.15.169:6809/116034","cluster_addr":"172.21.15.169:6811/116034","heartbeat_back_addr":"172.21.15.169:6815/116034","heartbeat_front_addr":"172.21.15.169:6813/116034","state":["exists","up"]},{"osd":7,"uuid":"1ca9aa4a-755d-4e28-a937-4d8dccab4df3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":116044},{"type":"v1","addr":"172.21.15.169:6823","nonce":116044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6825","nonce":116044},{"type":"v1","addr":"172.21.15.169:6827","nonce":116044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":116044},{"type":"v1","addr":"172.21.15.169:6831","nonce":116044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":116044},{"type":"v1","addr":"172.21.15.169:6829","nonce":116044}]},"public_addr":"172.21.15.169:6823/116044","cluster_addr":"172.21.15.169:6827/116044","heartbeat_back_addr":"172.21.15.169:6831/116044","heartbeat_front_addr":"172.21.15.169:6829/116044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.586781+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.869778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.508594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.860493+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.872267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.746933+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.611520+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.871446+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:21:14.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.419+0000 7f9b4b3ff700 1 -- 172.21.15.92:0/2628061635 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9b280682f0 msgr2=0x7f9b2806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:14.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.419+0000 7f9b4b3ff700 1 --2- 172.21.15.92:0/2628061635 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9b280682f0 0x7f9b2806a7a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9b440b8070 tx=0x7f9b34009210 comp rx=0 tx=0).stop 2023-11-28T11:21:14.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.419+0000 7f9b4b3ff700 1 -- 172.21.15.92:0/2628061635 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b440b8510 msgr2=0x7f9b4415ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:14.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.419+0000 7f9b4b3ff700 1 --2- 172.21.15.92:0/2628061635 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b440b8510 0x7f9b4415ad40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9b40001eb0 tx=0x7f9b4000c800 comp rx=0 tx=0).stop 2023-11-28T11:21:14.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.419+0000 7f9b4b3ff700 1 -- 172.21.15.92:0/2628061635 shutdown_connections 2023-11-28T11:21:14.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.419+0000 7f9b4b3ff700 1 --2- 172.21.15.92:0/2628061635 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9b440d0d60 0x7f9b441e75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:14.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.419+0000 7f9b4b3ff700 1 --2- 172.21.15.92:0/2628061635 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9b280682f0 0x7f9b2806a7a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:14.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.419+0000 7f9b4b3ff700 1 --2- 172.21.15.92:0/2628061635 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b440b8510 0x7f9b4415ad40 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:14.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.419+0000 7f9b4b3ff700 1 --2- 172.21.15.92:0/2628061635 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9b440cd460 0x7f9b441e7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:14.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.419+0000 7f9b4b3ff700 1 -- 172.21.15.92:0/2628061635 >> 172.21.15.92:0/2628061635 conn(0x7f9b440719a0 msgr2=0x7f9b440b3f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:14.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.420+0000 7f9b4b3ff700 1 -- 172.21.15.92:0/2628061635 shutdown_connections 2023-11-28T11:21:14.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.420+0000 7f9b4b3ff700 1 -- 172.21.15.92:0/2628061635 wait complete. 2023-11-28T11:21:14.439 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T11:21:14.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.639+0000 7fa2c03ff700 1 Processor -- start 2023-11-28T11:21:14.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.639+0000 7fa2c03ff700 1 -- start start 2023-11-28T11:21:14.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.640+0000 7fa2c03ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2b80b8510 0x7fa2b806b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:14.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.640+0000 7fa2c03ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa2b80cd460 0x7fa2b806beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:14.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.641+0000 7fa2c03ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2b80b88e0 con 0x7fa2b806c3f0 2023-11-28T11:21:14.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.641+0000 7fa2c03ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2b80cdbd0 con 0x7fa2b80cd460 2023-11-28T11:21:14.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.641+0000 7fa2c03ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b806c830 con 0x7fa2b80b8510 2023-11-28T11:21:14.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.641+0000 7fa2be19b700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2b80b8510 0x7fa2b806b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:14.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.641+0000 7fa2bd99a700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa2b80cd460 0x7fa2b806beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:14.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.641+0000 7fa2be99c700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fa2b806c3f0 0x7fa2b815cc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.92:60356/0 (socket says 172.21.15.92:60356) 2023-11-28T11:21:14.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.641+0000 7fa2be99c700 1 -- 172.21.15.92:0/1965296636 learned_addr learned my addr 172.21.15.92:0/1965296636 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:14.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.641+0000 7fa2bd199700 1 -- 172.21.15.92:0/1965296636 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1224301179 0 0) 0x7fa2b80b88e0 con 0x7fa2b806c3f0 2023-11-28T11:21:14.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.642+0000 7fa2bd199700 1 -- 172.21.15.92:0/1965296636 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2a40035b0 con 0x7fa2b806c3f0 2023-11-28T11:21:14.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.642+0000 7fa2be19b700 1 -- 172.21.15.92:0/1965296636 >> v2:172.21.15.169:3300/0 conn(0x7fa2b80cd460 msgr2=0x7fa2b806beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:14.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.642+0000 7fa2be19b700 1 --2- 172.21.15.92:0/1965296636 >> v2:172.21.15.169:3300/0 conn(0x7fa2b80cd460 0x7fa2b806beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:14.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.642+0000 7fa2be19b700 1 -- 172.21.15.92:0/1965296636 >> v1:172.21.15.169:6789/0 conn(0x7fa2b806c3f0 legacy=0x7fa2b815cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:14.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.642+0000 7fa2be19b700 1 -- 172.21.15.92:0/1965296636 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b80cd8f0 con 0x7fa2b80b8510 2023-11-28T11:21:14.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.642+0000 7fa2bd99a700 1 --2- 172.21.15.92:0/1965296636 >> v2:172.21.15.169:3300/0 conn(0x7fa2b80cd460 0x7fa2b806beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:21:14.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.642+0000 7fa2be19b700 1 --2- 172.21.15.92:0/1965296636 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2b80b8510 0x7fa2b806b970 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4009840 tx=0x7fa2b4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4d2f083b60fd8d4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:14.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.642+0000 7fa2bd199700 1 -- 172.21.15.92:0/1965296636 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b4011830 con 0x7fa2b80b8510 2023-11-28T11:21:14.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.642+0000 7fa2bd199700 1 -- 172.21.15.92:0/1965296636 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2b4011990 con 0x7fa2b80b8510 2023-11-28T11:21:14.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.642+0000 7fa2bd199700 1 -- 172.21.15.92:0/1965296636 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b4011d00 con 0x7fa2b80b8510 2023-11-28T11:21:14.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.643+0000 7fa2c03ff700 1 -- 172.21.15.92:0/1965296636 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2b80b8510 msgr2=0x7fa2b806b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:14.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.643+0000 7fa2c03ff700 1 --2- 172.21.15.92:0/1965296636 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2b80b8510 0x7fa2b806b970 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4009840 tx=0x7fa2b4009b50 comp rx=0 tx=0).stop 2023-11-28T11:21:14.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.643+0000 7fa2c03ff700 1 -- 172.21.15.92:0/1965296636 shutdown_connections 2023-11-28T11:21:14.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.643+0000 7fa2c03ff700 1 --2- 172.21.15.92:0/1965296636 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2b80b8510 0x7fa2b806b970 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:14.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.643+0000 7fa2c03ff700 1 --2- 172.21.15.92:0/1965296636 >> v2:172.21.15.169:3300/0 conn(0x7fa2b80cd460 0x7fa2b806beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:14.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.643+0000 7fa2c03ff700 1 -- 172.21.15.92:0/1965296636 >> 172.21.15.92:0/1965296636 conn(0x7fa2b80719a0 msgr2=0x7fa2b8071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:14.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.643+0000 7fa2c03ff700 1 -- 172.21.15.92:0/1965296636 shutdown_connections 2023-11-28T11:21:14.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.643+0000 7fa2c03ff700 1 -- 172.21.15.92:0/1965296636 wait complete. 2023-11-28T11:21:14.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.645+0000 7fa2c03ff700 1 Processor -- start 2023-11-28T11:21:14.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.645+0000 7fa2c03ff700 1 -- start start 2023-11-28T11:21:14.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.646+0000 7fa2c03ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2b806c3f0 0x7fa2b8158380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:14.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.646+0000 7fa2c03ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2b80b8510 0x7fa2b80c93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:14.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.646+0000 7fa2c03ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2b80cd460 0x7fa2b80c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:14.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.646+0000 7fa2c03ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b80c9930 con 0x7fa2b806c3f0 2023-11-28T11:21:14.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.646+0000 7fa2c03ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b8158c40 con 0x7fa2b80b8510 2023-11-28T11:21:14.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.646+0000 7fa2c03ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b81e5bb0 con 0x7fa2b80cd460 2023-11-28T11:21:14.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.646+0000 7fa2be19b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2b806c3f0 0x7fa2b8158380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:14.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.646+0000 7fa2be19b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2b806c3f0 0x7fa2b8158380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49042/0 (socket says 172.21.15.92:49042) 2023-11-28T11:21:14.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.646+0000 7fa2be19b700 1 -- 172.21.15.92:0/3311283197 learned_addr learned my addr 172.21.15.92:0/3311283197 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:14.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.646+0000 7fa2bd99a700 1 --2- 172.21.15.92:0/3311283197 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2b80b8510 0x7fa2b80c93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:14.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.646+0000 7fa2be99c700 1 --2- 172.21.15.92:0/3311283197 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2b80cd460 0x7fa2b80c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:14.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.646+0000 7fa2be19b700 1 -- 172.21.15.92:0/3311283197 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2b80cd460 msgr2=0x7fa2b80c9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:14.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.647+0000 7fa2be19b700 1 --2- 172.21.15.92:0/3311283197 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2b80cd460 0x7fa2b80c9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:14.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.647+0000 7fa2be19b700 1 -- 172.21.15.92:0/3311283197 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2b80b8510 msgr2=0x7fa2b80c93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:14.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.647+0000 7fa2be19b700 1 --2- 172.21.15.92:0/3311283197 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2b80b8510 0x7fa2b80c93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:14.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.647+0000 7fa2be19b700 1 -- 172.21.15.92:0/3311283197 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a80094a0 con 0x7fa2b806c3f0 2023-11-28T11:21:14.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.647+0000 7fa2be19b700 1 --2- 172.21.15.92:0/3311283197 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2b806c3f0 0x7fa2b8158380 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa2b40090b0 tx=0x7fa2b4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:14.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.647+0000 7fa2aeffd700 1 -- 172.21.15.92:0/3311283197 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b4004550 con 0x7fa2b806c3f0 2023-11-28T11:21:14.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.647+0000 7fa2aeffd700 1 -- 172.21.15.92:0/3311283197 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2b401f070 con 0x7fa2b806c3f0 2023-11-28T11:21:14.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.647+0000 7fa2aeffd700 1 -- 172.21.15.92:0/3311283197 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b40038c0 con 0x7fa2b806c3f0 2023-11-28T11:21:14.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.647+0000 7fa2c03ff700 1 -- 172.21.15.92:0/3311283197 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2b40094a0 con 0x7fa2b806c3f0 2023-11-28T11:21:14.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.647+0000 7fa2c03ff700 1 -- 172.21.15.92:0/3311283197 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2b80cfff0 con 0x7fa2b806c3f0 2023-11-28T11:21:14.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.648+0000 7fa2c03ff700 1 -- 172.21.15.92:0/3311283197 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2b80cfd10 con 0x7fa2b806c3f0 2023-11-28T11:21:14.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.649+0000 7fa2aeffd700 1 -- 172.21.15.92:0/3311283197 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa2b400f050 con 0x7fa2b806c3f0 2023-11-28T11:21:14.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.650+0000 7fa2aeffd700 1 --2- 172.21.15.92:0/3311283197 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa29c0682f0 0x7fa29c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:14.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.650+0000 7fa2aeffd700 1 -- 172.21.15.92:0/3311283197 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa2b4021070 con 0x7fa2b806c3f0 2023-11-28T11:21:14.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.650+0000 7fa2bd99a700 1 --2- 172.21.15.92:0/3311283197 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa29c0682f0 0x7fa29c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:14.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.651+0000 7fa2bd99a700 1 --2- 172.21.15.92:0/3311283197 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa29c0682f0 0x7fa29c06a7a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa2b81f5aa0 tx=0x7fa2a8009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:14.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.654+0000 7fa2aeffd700 1 -- 172.21.15.92:0/3311283197 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa2b4053c60 con 0x7fa2b806c3f0 2023-11-28T11:21:14.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:14.887+0000 7fa2c03ff700 1 -- 172.21.15.92:0/3311283197 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fa2b8067880 con 0x7fa2b806c3f0 2023-11-28T11:21:15.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.540+0000 7fa2aeffd700 1 -- 172.21.15.92:0/3311283197 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) v1 ==== 118+0+0 (secure 0 0 0) 0x7fa2b4057280 con 0x7fa2b806c3f0 2023-11-28T11:21:15.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.543+0000 7fa2c03ff700 1 -- 172.21.15.92:0/3311283197 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa29c0682f0 msgr2=0x7fa29c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:15.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.543+0000 7fa2c03ff700 1 --2- 172.21.15.92:0/3311283197 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa29c0682f0 0x7fa29c06a7a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa2b81f5aa0 tx=0x7fa2a8009210 comp rx=0 tx=0).stop 2023-11-28T11:21:15.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.543+0000 7fa2c03ff700 1 -- 172.21.15.92:0/3311283197 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2b806c3f0 msgr2=0x7fa2b8158380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:15.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.543+0000 7fa2c03ff700 1 --2- 172.21.15.92:0/3311283197 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2b806c3f0 0x7fa2b8158380 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa2b40090b0 tx=0x7fa2b4004750 comp rx=0 tx=0).stop 2023-11-28T11:21:15.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.543+0000 7fa2c03ff700 1 -- 172.21.15.92:0/3311283197 shutdown_connections 2023-11-28T11:21:15.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.543+0000 7fa2c03ff700 1 --2- 172.21.15.92:0/3311283197 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa2b80cd460 0x7fa2b80c9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:15.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.543+0000 7fa2c03ff700 1 --2- 172.21.15.92:0/3311283197 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa29c0682f0 0x7fa29c06a7a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:15.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.543+0000 7fa2c03ff700 1 --2- 172.21.15.92:0/3311283197 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2b806c3f0 0x7fa2b8158380 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:15.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.543+0000 7fa2c03ff700 1 --2- 172.21.15.92:0/3311283197 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa2b80b8510 0x7fa2b80c93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:15.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.543+0000 7fa2c03ff700 1 -- 172.21.15.92:0/3311283197 >> 172.21.15.92:0/3311283197 conn(0x7fa2b80719a0 msgr2=0x7fa2b8072e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:15.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.544+0000 7fa2c03ff700 1 -- 172.21.15.92:0/3311283197 shutdown_connections 2023-11-28T11:21:15.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.544+0000 7fa2c03ff700 1 -- 172.21.15.92:0/3311283197 wait complete. 2023-11-28T11:21:15.566 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2023-11-28T11:21:15.566 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T11:21:15.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.779+0000 7f9144bff700 1 Processor -- start 2023-11-28T11:21:15.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.780+0000 7f9144bff700 1 -- start start 2023-11-28T11:21:15.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.781+0000 7f9144bff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f91400d1d70 0x7f9140167270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:15.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.781+0000 7f9144bff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f914006c790 con 0x7f91400cd460 2023-11-28T11:21:15.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.781+0000 7f9144bff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91400b88e0 con 0x7f91400b8510 2023-11-28T11:21:15.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.781+0000 7f9144bff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91400cdbd0 con 0x7f91400d1d70 2023-11-28T11:21:15.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.781+0000 7f9137fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f91400cd460 0x7f914006c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46172/0 (socket says 172.21.15.92:46172) 2023-11-28T11:21:15.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.781+0000 7f9137fff700 1 -- 172.21.15.92:0/3424509718 learned_addr learned my addr 172.21.15.92:0/3424509718 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:15.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.781+0000 7f913ed9d700 1 --2- 172.21.15.92:0/3424509718 >> v2:172.21.15.92:3300/0 conn(0x7f91400d1d70 0x7f9140167270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:15.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.782+0000 7f913ed9d700 1 -- 172.21.15.92:0/3424509718 >> v1:172.21.15.169:6789/0 conn(0x7f91400b8510 legacy=0x7f914006b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:15.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.782+0000 7f913ed9d700 1 -- 172.21.15.92:0/3424509718 >> v1:172.21.15.92:6789/0 conn(0x7f91400cd460 legacy=0x7f914006c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:15.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.782+0000 7f913ed9d700 1 -- 172.21.15.92:0/3424509718 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91400cd8f0 con 0x7f91400d1d70 2023-11-28T11:21:15.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.782+0000 7f913dd9b700 1 -- 172.21.15.92:0/3424509718 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1347381840 0 0) 0x7f914006c790 con 0x7f91400cd460 2023-11-28T11:21:15.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.782+0000 7f913ed9d700 1 --2- 172.21.15.92:0/3424509718 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91400d1d70 0x7f9140167270 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9138009d90 tx=0x7f913800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14a878f9ae9b3d1c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:15.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.782+0000 7f913dd9b700 1 -- 172.21.15.92:0/3424509718 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913800e070 con 0x7f91400d1d70 2023-11-28T11:21:15.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.782+0000 7f913dd9b700 1 -- 172.21.15.92:0/3424509718 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f913800bbd0 con 0x7f91400d1d70 2023-11-28T11:21:15.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.782+0000 7f913dd9b700 1 -- 172.21.15.92:0/3424509718 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9138012430 con 0x7f91400d1d70 2023-11-28T11:21:15.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.783+0000 7f9144bff700 1 -- 172.21.15.92:0/3424509718 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91400d1d70 msgr2=0x7f9140167270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:15.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.783+0000 7f9144bff700 1 --2- 172.21.15.92:0/3424509718 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91400d1d70 0x7f9140167270 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9138009d90 tx=0x7f913800b330 comp rx=0 tx=0).stop 2023-11-28T11:21:15.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.783+0000 7f9144bff700 1 -- 172.21.15.92:0/3424509718 shutdown_connections 2023-11-28T11:21:15.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.783+0000 7f9144bff700 1 --2- 172.21.15.92:0/3424509718 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91400d1d70 0x7f9140167270 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:15.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.783+0000 7f9144bff700 1 -- 172.21.15.92:0/3424509718 >> 172.21.15.92:0/3424509718 conn(0x7f91400719a0 msgr2=0x7f9140071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:15.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.783+0000 7f9144bff700 1 -- 172.21.15.92:0/3424509718 shutdown_connections 2023-11-28T11:21:15.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.783+0000 7f9144bff700 1 -- 172.21.15.92:0/3424509718 wait complete. 2023-11-28T11:21:15.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.785+0000 7f9144bff700 1 Processor -- start 2023-11-28T11:21:15.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.785+0000 7f9144bff700 1 -- start start 2023-11-28T11:21:15.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.786+0000 7f9144bff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91400b8510 0x7f91401e6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:15.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.786+0000 7f9144bff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91400cd460 0x7f914015b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:15.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.786+0000 7f9144bff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91400d1d70 0x7f91401e5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:15.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.786+0000 7f9144bff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f914015b7d0 con 0x7f91400b8510 2023-11-28T11:21:15.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.786+0000 7f9144bff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91401e4250 con 0x7f91400d1d70 2023-11-28T11:21:15.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.786+0000 7f9144bff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91401e3490 con 0x7f91400cd460 2023-11-28T11:21:15.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.786+0000 7f913e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91400b8510 0x7f91401e6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:15.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.786+0000 7f913e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91400b8510 0x7f91401e6790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49052/0 (socket says 172.21.15.92:49052) 2023-11-28T11:21:15.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.786+0000 7f913e59c700 1 -- 172.21.15.92:0/3355650251 learned_addr learned my addr 172.21.15.92:0/3355650251 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:15.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.786+0000 7f9137fff700 1 --2- 172.21.15.92:0/3355650251 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91400cd460 0x7f914015b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:15.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.786+0000 7f913ed9d700 1 --2- 172.21.15.92:0/3355650251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91400d1d70 0x7f91401e5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:15.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.786+0000 7f913e59c700 1 -- 172.21.15.92:0/3355650251 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91400cd460 msgr2=0x7f914015b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:15.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.786+0000 7f913e59c700 1 --2- 172.21.15.92:0/3355650251 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91400cd460 0x7f914015b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:15.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.786+0000 7f913e59c700 1 -- 172.21.15.92:0/3355650251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91400d1d70 msgr2=0x7f91401e5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:15.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.786+0000 7f913e59c700 1 --2- 172.21.15.92:0/3355650251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91400d1d70 0x7f91401e5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:15.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.786+0000 7f913e59c700 1 -- 172.21.15.92:0/3355650251 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9138009a70 con 0x7f91400b8510 2023-11-28T11:21:15.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.787+0000 7f913e59c700 1 --2- 172.21.15.92:0/3355650251 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91400b8510 0x7f91401e6790 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f913000c4e0 tx=0x7f913000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:15.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.787+0000 7f9136ffd700 1 -- 172.21.15.92:0/3355650251 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91300052d0 con 0x7f91400b8510 2023-11-28T11:21:15.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.787+0000 7f9136ffd700 1 -- 172.21.15.92:0/3355650251 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9130005430 con 0x7f91400b8510 2023-11-28T11:21:15.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.787+0000 7f9144bff700 1 -- 172.21.15.92:0/3355650251 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91400cfff0 con 0x7f91400b8510 2023-11-28T11:21:15.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.787+0000 7f9136ffd700 1 -- 172.21.15.92:0/3355650251 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91300038c0 con 0x7f91400b8510 2023-11-28T11:21:15.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.787+0000 7f9144bff700 1 -- 172.21.15.92:0/3355650251 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91400cfd10 con 0x7f91400b8510 2023-11-28T11:21:15.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.788+0000 7f9144bff700 1 -- 172.21.15.92:0/3355650251 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9140067880 con 0x7f91400b8510 2023-11-28T11:21:15.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.790+0000 7f9136ffd700 1 -- 172.21.15.92:0/3355650251 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9130019030 con 0x7f91400b8510 2023-11-28T11:21:15.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.791+0000 7f9136ffd700 1 --2- 172.21.15.92:0/3355650251 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f91200682f0 0x7f912006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:15.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.791+0000 7f9136ffd700 1 -- 172.21.15.92:0/3355650251 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f9130085710 con 0x7f91400b8510 2023-11-28T11:21:15.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.791+0000 7f9137fff700 1 --2- 172.21.15.92:0/3355650251 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f91200682f0 0x7f912006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:15.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.791+0000 7f9137fff700 1 --2- 172.21.15.92:0/3355650251 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f91200682f0 0x7f912006a7a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9128007b10 tx=0x7f9128007a60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:15.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:15.793+0000 7f9136ffd700 1 -- 172.21.15.92:0/3355650251 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9130051c10 con 0x7f91400b8510 2023-11-28T11:21:16.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.028+0000 7f9144bff700 1 -- 172.21.15.92:0/3355650251 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f91400d2940 con 0x7f91400b8510 2023-11-28T11:21:16.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.549+0000 7f9136ffd700 1 -- 172.21.15.92:0/3355650251 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9130055230 con 0x7f91400b8510 2023-11-28T11:21:16.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.552+0000 7f9144bff700 1 -- 172.21.15.92:0/3355650251 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f91200682f0 msgr2=0x7f912006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:16.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.552+0000 7f9144bff700 1 --2- 172.21.15.92:0/3355650251 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f91200682f0 0x7f912006a7a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9128007b10 tx=0x7f9128007a60 comp rx=0 tx=0).stop 2023-11-28T11:21:16.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.552+0000 7f9144bff700 1 -- 172.21.15.92:0/3355650251 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91400b8510 msgr2=0x7f91401e6790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:16.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.552+0000 7f9144bff700 1 --2- 172.21.15.92:0/3355650251 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91400b8510 0x7f91401e6790 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f913000c4e0 tx=0x7f913000c7f0 comp rx=0 tx=0).stop 2023-11-28T11:21:16.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.552+0000 7f9144bff700 1 -- 172.21.15.92:0/3355650251 shutdown_connections 2023-11-28T11:21:16.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.552+0000 7f9144bff700 1 --2- 172.21.15.92:0/3355650251 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91400d1d70 0x7f91401e5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:16.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.552+0000 7f9144bff700 1 --2- 172.21.15.92:0/3355650251 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f91200682f0 0x7f912006a7a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:16.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.552+0000 7f9144bff700 1 --2- 172.21.15.92:0/3355650251 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91400b8510 0x7f91401e6790 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:16.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.552+0000 7f9144bff700 1 --2- 172.21.15.92:0/3355650251 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91400cd460 0x7f914015b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:16.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.552+0000 7f9144bff700 1 -- 172.21.15.92:0/3355650251 >> 172.21.15.92:0/3355650251 conn(0x7f91400719a0 msgr2=0x7f91400cbef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:16.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.552+0000 7f9144bff700 1 -- 172.21.15.92:0/3355650251 shutdown_connections 2023-11-28T11:21:16.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.552+0000 7f9144bff700 1 -- 172.21.15.92:0/3355650251 wait complete. 2023-11-28T11:21:16.573 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T11:21:16.573 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:21:16.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.769+0000 7f10a47ff700 1 Processor -- start 2023-11-28T11:21:16.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.769+0000 7f10a47ff700 1 -- start start 2023-11-28T11:21:16.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.770+0000 7f10a47ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f109c0cd460 0x7f109c06c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:16.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.770+0000 7f10a47ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f109c0d0d60 0x7f109c0d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:16.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.770+0000 7f10a47ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f109c0b88e0 con 0x7f109c0b8510 2023-11-28T11:21:16.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.770+0000 7f10a47ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f109c0cdbd0 con 0x7f109c0cd460 2023-11-28T11:21:16.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.770+0000 7f10a47ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f109c0d02d0 con 0x7f109c0d0d60 2023-11-28T11:21:16.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.771+0000 7f10a2d9c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f109c0d0d60 0x7f109c0d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:16.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.771+0000 7f10a1d9a700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f109c0cd460 0x7f109c06c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:16.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.771+0000 7f10a2d9c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f109c0d0d60 0x7f109c0d1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.92:59322/0 (socket says 172.21.15.92:59322) 2023-11-28T11:21:16.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.771+0000 7f10a1d9a700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f109c0cd460 0x7f109c06c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.92:49824/0 (socket says 172.21.15.92:49824) 2023-11-28T11:21:16.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.771+0000 7f10a2d9c700 1 -- 172.21.15.92:0/118558632 learned_addr learned my addr 172.21.15.92:0/118558632 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:16.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.771+0000 7f10a259b700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f109c0b8510 0x7f109c06b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.92:60370/0 (socket says 172.21.15.92:60370) 2023-11-28T11:21:16.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.771+0000 7f10a2d9c700 1 -- 172.21.15.92:0/118558632 >> v2:172.21.15.169:3300/0 conn(0x7f109c0cd460 msgr2=0x7f109c06c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:16.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.771+0000 7f10a2d9c700 1 --2- 172.21.15.92:0/118558632 >> v2:172.21.15.169:3300/0 conn(0x7f109c0cd460 0x7f109c06c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:16.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.771+0000 7f10a2d9c700 1 -- 172.21.15.92:0/118558632 >> v1:172.21.15.169:6789/0 conn(0x7f109c0b8510 legacy=0x7f109c06b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:16.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.771+0000 7f10a2d9c700 1 -- 172.21.15.92:0/118558632 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f109c0cd8f0 con 0x7f109c0d0d60 2023-11-28T11:21:16.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.772+0000 7f10a2d9c700 1 --2- 172.21.15.92:0/118558632 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f109c0d0d60 0x7f109c0d1200 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1094009e50 tx=0x7f109400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d91fd58f8d7f992f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:16.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.772+0000 7f10a1599700 1 -- 172.21.15.92:0/118558632 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109400e070 con 0x7f109c0d0d60 2023-11-28T11:21:16.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.772+0000 7f10a1599700 1 -- 172.21.15.92:0/118558632 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f109400b910 con 0x7f109c0d0d60 2023-11-28T11:21:16.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.772+0000 7f10a1599700 1 -- 172.21.15.92:0/118558632 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109400bcc0 con 0x7f109c0d0d60 2023-11-28T11:21:16.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.772+0000 7f10a47ff700 1 -- 172.21.15.92:0/118558632 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f109c0d0d60 msgr2=0x7f109c0d1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:16.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.772+0000 7f10a47ff700 1 --2- 172.21.15.92:0/118558632 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f109c0d0d60 0x7f109c0d1200 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1094009e50 tx=0x7f109400b330 comp rx=0 tx=0).stop 2023-11-28T11:21:16.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.773+0000 7f10a47ff700 1 -- 172.21.15.92:0/118558632 shutdown_connections 2023-11-28T11:21:16.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.773+0000 7f10a47ff700 1 --2- 172.21.15.92:0/118558632 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f109c0d0d60 0x7f109c0d1200 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:16.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.773+0000 7f10a47ff700 1 --2- 172.21.15.92:0/118558632 >> v2:172.21.15.169:3300/0 conn(0x7f109c0cd460 0x7f109c06c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:16.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.773+0000 7f10a47ff700 1 -- 172.21.15.92:0/118558632 >> 172.21.15.92:0/118558632 conn(0x7f109c0719a0 msgr2=0x7f109c071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:16.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.773+0000 7f10a47ff700 1 -- 172.21.15.92:0/118558632 shutdown_connections 2023-11-28T11:21:16.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.773+0000 7f10a47ff700 1 -- 172.21.15.92:0/118558632 wait complete. 2023-11-28T11:21:16.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.774+0000 7f10a47ff700 1 Processor -- start 2023-11-28T11:21:16.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.774+0000 7f10a47ff700 1 -- start start 2023-11-28T11:21:16.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.775+0000 7f10a47ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f109c0b8510 0x7f109c0c95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:16.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.775+0000 7f10a47ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f109c0cd460 0x7f109c1e73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:16.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.775+0000 7f10a47ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f109c0d0d60 0x7f109c1e78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:16.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.775+0000 7f10a47ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109c1e7e30 con 0x7f109c0d0d60 2023-11-28T11:21:16.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.775+0000 7f10a47ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109c0c9b00 con 0x7f109c0cd460 2023-11-28T11:21:16.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.775+0000 7f10a47ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f109c14b060 con 0x7f109c0b8510 2023-11-28T11:21:16.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.776+0000 7f10a2d9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f109c0d0d60 0x7f109c1e78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:16.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.776+0000 7f10a2d9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f109c0d0d60 0x7f109c1e78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49056/0 (socket says 172.21.15.92:49056) 2023-11-28T11:21:16.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.776+0000 7f10a2d9c700 1 -- 172.21.15.92:0/3860596745 learned_addr learned my addr 172.21.15.92:0/3860596745 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:16.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.776+0000 7f10a259b700 1 --2- 172.21.15.92:0/3860596745 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f109c0b8510 0x7f109c0c95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:16.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.776+0000 7f10a1d9a700 1 --2- 172.21.15.92:0/3860596745 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f109c0cd460 0x7f109c1e73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:16.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.776+0000 7f10a2d9c700 1 -- 172.21.15.92:0/3860596745 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f109c0b8510 msgr2=0x7f109c0c95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:16.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.776+0000 7f10a2d9c700 1 --2- 172.21.15.92:0/3860596745 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f109c0b8510 0x7f109c0c95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:16.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.776+0000 7f10a2d9c700 1 -- 172.21.15.92:0/3860596745 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f109c0cd460 msgr2=0x7f109c1e73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:16.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.776+0000 7f10a2d9c700 1 --2- 172.21.15.92:0/3860596745 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f109c0cd460 0x7f109c1e73b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:16.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.776+0000 7f10a2d9c700 1 -- 172.21.15.92:0/3860596745 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1094009b30 con 0x7f109c0d0d60 2023-11-28T11:21:16.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.776+0000 7f10a2d9c700 1 --2- 172.21.15.92:0/3860596745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f109c0d0d60 0x7f109c1e78f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1094004270 tx=0x7f1094004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:16.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.777+0000 7f1092ffd700 1 -- 172.21.15.92:0/3860596745 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10940099b0 con 0x7f109c0d0d60 2023-11-28T11:21:16.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.777+0000 7f1092ffd700 1 -- 172.21.15.92:0/3860596745 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10940092e0 con 0x7f109c0d0d60 2023-11-28T11:21:16.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.777+0000 7f1092ffd700 1 -- 172.21.15.92:0/3860596745 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109401c660 con 0x7f109c0d0d60 2023-11-28T11:21:16.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.777+0000 7f10a47ff700 1 -- 172.21.15.92:0/3860596745 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f109c0cfff0 con 0x7f109c0d0d60 2023-11-28T11:21:16.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.777+0000 7f10a47ff700 1 -- 172.21.15.92:0/3860596745 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f109c0cfd10 con 0x7f109c0d0d60 2023-11-28T11:21:16.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.778+0000 7f10a47ff700 1 -- 172.21.15.92:0/3860596745 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f109c067880 con 0x7f109c0d0d60 2023-11-28T11:21:16.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.779+0000 7f1092ffd700 1 -- 172.21.15.92:0/3860596745 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1094022030 con 0x7f109c0d0d60 2023-11-28T11:21:16.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.780+0000 7f1092ffd700 1 --2- 172.21.15.92:0/3860596745 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f10800682f0 0x7f108006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:16.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.780+0000 7f1092ffd700 1 -- 172.21.15.92:0/3860596745 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f10940878f0 con 0x7f109c0d0d60 2023-11-28T11:21:16.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.780+0000 7f10a259b700 1 --2- 172.21.15.92:0/3860596745 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f10800682f0 0x7f108006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:16.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.780+0000 7f10a259b700 1 --2- 172.21.15.92:0/3860596745 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f10800682f0 0x7f108006a7a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1098000f60 tx=0x7f1098007d10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:16.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:16.784+0000 7f1092ffd700 1 -- 172.21.15.92:0/3860596745 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1094053e20 con 0x7f109c0d0d60 2023-11-28T11:21:17.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.018+0000 7f10a47ff700 1 -- 172.21.15.92:0/3860596745 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f109c0d2940 con 0x7f109c0d0d60 2023-11-28T11:21:17.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.019+0000 7f1092ffd700 1 -- 172.21.15.92:0/3860596745 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4647 (secure 0 0 0) 0x7f1094057440 con 0x7f109c0d0d60 2023-11-28T11:21:17.021 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:21:17.021 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/3257209990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3257209990},{"type":"v1","addr":"172.21.15.92:6836","nonce":3257209990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2193338949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2193338949},{"type":"v1","addr":"172.21.15.169:6834","nonce":2193338949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4308,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/471035619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":471035619},{"type":"v1","addr":"172.21.15.92:6837","nonce":471035619}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:20:56.234970+0000","modified":"2023-11-28T11:21:16.544352+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4326},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4326":{"gid":4326,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/2067251916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2067251916},{"type":"v1","addr":"172.21.15.169:6835","nonce":2067251916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T11:21:17.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.021+0000 7f10a47ff700 1 -- 172.21.15.92:0/3860596745 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f10800682f0 msgr2=0x7f108006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:17.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.021+0000 7f10a47ff700 1 --2- 172.21.15.92:0/3860596745 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f10800682f0 0x7f108006a7a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1098000f60 tx=0x7f1098007d10 comp rx=0 tx=0).stop 2023-11-28T11:21:17.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.021+0000 7f10a47ff700 1 -- 172.21.15.92:0/3860596745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f109c0d0d60 msgr2=0x7f109c1e78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:17.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.021+0000 7f10a47ff700 1 --2- 172.21.15.92:0/3860596745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f109c0d0d60 0x7f109c1e78f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1094004270 tx=0x7f1094004ad0 comp rx=0 tx=0).stop 2023-11-28T11:21:17.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.021+0000 7f10a47ff700 1 -- 172.21.15.92:0/3860596745 shutdown_connections 2023-11-28T11:21:17.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.021+0000 7f10a47ff700 1 --2- 172.21.15.92:0/3860596745 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f10800682f0 0x7f108006a7a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:17.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.021+0000 7f10a47ff700 1 --2- 172.21.15.92:0/3860596745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f109c0d0d60 0x7f109c1e78f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:17.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.021+0000 7f10a47ff700 1 --2- 172.21.15.92:0/3860596745 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f109c0b8510 0x7f109c0c95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:17.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.021+0000 7f10a47ff700 1 --2- 172.21.15.92:0/3860596745 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f109c0cd460 0x7f109c1e73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:17.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.021+0000 7f10a47ff700 1 -- 172.21.15.92:0/3860596745 >> 172.21.15.92:0/3860596745 conn(0x7f109c0719a0 msgr2=0x7f109c0b4180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:17.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.022+0000 7f10a47ff700 1 -- 172.21.15.92:0/3860596745 shutdown_connections 2023-11-28T11:21:17.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.022+0000 7f10a47ff700 1 -- 172.21.15.92:0/3860596745 wait complete. 2023-11-28T11:21:17.025 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 9 2023-11-28T11:21:17.041 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:21:17.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.234+0000 7f79703ff700 1 Processor -- start 2023-11-28T11:21:17.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.234+0000 7f79703ff700 1 -- start start 2023-11-28T11:21:17.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.235+0000 7f79703ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79680b8510 0x7f796806b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:17.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.235+0000 7f79703ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f79680d0d60 0x7f79680d11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:17.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.235+0000 7f79703ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79680b88e0 con 0x7f79680cd460 2023-11-28T11:21:17.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.235+0000 7f79703ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79680cdbd0 con 0x7f79680d0d60 2023-11-28T11:21:17.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.235+0000 7f79703ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79680d02d0 con 0x7f79680b8510 2023-11-28T11:21:17.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.236+0000 7f796e19b700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79680b8510 0x7f796806b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:17.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.236+0000 7f796e99c700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f79680d0d60 0x7f79680d11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:17.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.236+0000 7f796e99c700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f79680d0d60 0x7f79680d11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.92:49844/0 (socket says 172.21.15.92:49844) 2023-11-28T11:21:17.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.236+0000 7f796e99c700 1 -- 172.21.15.92:0/205608771 learned_addr learned my addr 172.21.15.92:0/205608771 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:17.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.236+0000 7f796e19b700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79680b8510 0x7f796806b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.92:59350/0 (socket says 172.21.15.92:59350) 2023-11-28T11:21:17.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.236+0000 7f796d99a700 1 --1- 172.21.15.92:0/205608771 >> v1:172.21.15.169:6789/0 conn(0x7f79680cd460 0x7f796806beb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.92:60384/0 (socket says 172.21.15.92:60384) 2023-11-28T11:21:17.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.236+0000 7f796e99c700 1 -- 172.21.15.92:0/205608771 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79680b8510 msgr2=0x7f796806b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:17.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.236+0000 7f796e99c700 1 --2- 172.21.15.92:0/205608771 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79680b8510 0x7f796806b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:17.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.236+0000 7f796e99c700 1 -- 172.21.15.92:0/205608771 >> v1:172.21.15.169:6789/0 conn(0x7f79680cd460 legacy=0x7f796806beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:17.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.236+0000 7f796e99c700 1 -- 172.21.15.92:0/205608771 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79680cd8f0 con 0x7f79680d0d60 2023-11-28T11:21:17.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.236+0000 7f796e19b700 1 --2- 172.21.15.92:0/205608771 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79680b8510 0x7f796806b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:21:17.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.237+0000 7f796e99c700 1 --2- 172.21.15.92:0/205608771 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79680d0d60 0x7f79680d11a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7960009e80 tx=0x7f796000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea29c3508f086919 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:17.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.237+0000 7f796d199700 1 -- 172.21.15.92:0/205608771 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796000e070 con 0x7f79680d0d60 2023-11-28T11:21:17.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.237+0000 7f796d199700 1 -- 172.21.15.92:0/205608771 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f796000b8d0 con 0x7f79680d0d60 2023-11-28T11:21:17.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.237+0000 7f796d199700 1 -- 172.21.15.92:0/205608771 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796000bc80 con 0x7f79680d0d60 2023-11-28T11:21:17.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.237+0000 7f79703ff700 1 -- 172.21.15.92:0/205608771 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79680d0d60 msgr2=0x7f79680d11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:17.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.237+0000 7f79703ff700 1 --2- 172.21.15.92:0/205608771 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79680d0d60 0x7f79680d11a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7960009e80 tx=0x7f796000b330 comp rx=0 tx=0).stop 2023-11-28T11:21:17.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.238+0000 7f79703ff700 1 -- 172.21.15.92:0/205608771 shutdown_connections 2023-11-28T11:21:17.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.238+0000 7f79703ff700 1 --2- 172.21.15.92:0/205608771 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79680d0d60 0x7f79680d11a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:17.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.238+0000 7f79703ff700 1 --2- 172.21.15.92:0/205608771 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79680b8510 0x7f796806b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:17.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.238+0000 7f79703ff700 1 -- 172.21.15.92:0/205608771 >> 172.21.15.92:0/205608771 conn(0x7f79680719a0 msgr2=0x7f7968071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:17.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.238+0000 7f79703ff700 1 -- 172.21.15.92:0/205608771 shutdown_connections 2023-11-28T11:21:17.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.238+0000 7f79703ff700 1 -- 172.21.15.92:0/205608771 wait complete. 2023-11-28T11:21:17.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.240+0000 7f79703ff700 1 Processor -- start 2023-11-28T11:21:17.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.240+0000 7f79703ff700 1 -- start start 2023-11-28T11:21:17.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.241+0000 7f79703ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79680b8510 0x7f79681f1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:17.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.241+0000 7f79703ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79680cd460 0x7f79680b2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:17.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.242+0000 7f79703ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79680d0d60 0x7f79680b2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:17.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.242+0000 7f79703ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79681f1970 con 0x7f79680b8510 2023-11-28T11:21:17.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.242+0000 7f79703ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7968154c00 con 0x7f79680cd460 2023-11-28T11:21:17.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.242+0000 7f79703ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7968155310 con 0x7f79680d0d60 2023-11-28T11:21:17.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.242+0000 7f796e19b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79680b8510 0x7f79681f1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:17.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.242+0000 7f796e19b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79680b8510 0x7f79681f1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49070/0 (socket says 172.21.15.92:49070) 2023-11-28T11:21:17.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.242+0000 7f796e19b700 1 -- 172.21.15.92:0/3940011060 learned_addr learned my addr 172.21.15.92:0/3940011060 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:17.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.242+0000 7f796d99a700 1 --2- 172.21.15.92:0/3940011060 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79680cd460 0x7f79680b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:17.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.242+0000 7f796e99c700 1 --2- 172.21.15.92:0/3940011060 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79680d0d60 0x7f79680b2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:17.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.242+0000 7f796e19b700 1 -- 172.21.15.92:0/3940011060 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79680d0d60 msgr2=0x7f79680b2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:17.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.242+0000 7f796e19b700 1 --2- 172.21.15.92:0/3940011060 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79680d0d60 0x7f79680b2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:17.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.242+0000 7f796e19b700 1 -- 172.21.15.92:0/3940011060 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79680cd460 msgr2=0x7f79680b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:17.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.242+0000 7f796e19b700 1 --2- 172.21.15.92:0/3940011060 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79680cd460 0x7f79680b2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:17.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.242+0000 7f796e19b700 1 -- 172.21.15.92:0/3940011060 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79640094a0 con 0x7f79680b8510 2023-11-28T11:21:17.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.242+0000 7f796e19b700 1 --2- 172.21.15.92:0/3940011060 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79680b8510 0x7f79681f1430 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f796400ebe0 tx=0x7f796400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:17.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.243+0000 7f795effd700 1 -- 172.21.15.92:0/3940011060 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796400c850 con 0x7f79680b8510 2023-11-28T11:21:17.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.243+0000 7f795effd700 1 -- 172.21.15.92:0/3940011060 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f796400c9b0 con 0x7f79680b8510 2023-11-28T11:21:17.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.243+0000 7f79703ff700 1 -- 172.21.15.92:0/3940011060 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7960009b30 con 0x7f79680b8510 2023-11-28T11:21:17.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.243+0000 7f795effd700 1 -- 172.21.15.92:0/3940011060 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796400cd40 con 0x7f79680b8510 2023-11-28T11:21:17.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.243+0000 7f79703ff700 1 -- 172.21.15.92:0/3940011060 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79680cfff0 con 0x7f79680b8510 2023-11-28T11:21:17.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.244+0000 7f79703ff700 1 -- 172.21.15.92:0/3940011060 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79680cfd10 con 0x7f79680b8510 2023-11-28T11:21:17.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.245+0000 7f795effd700 1 -- 172.21.15.92:0/3940011060 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7964019030 con 0x7f79680b8510 2023-11-28T11:21:17.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.246+0000 7f795effd700 1 --2- 172.21.15.92:0/3940011060 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f794c0682f0 0x7f794c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:17.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.246+0000 7f795effd700 1 -- 172.21.15.92:0/3940011060 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7964085f10 con 0x7f79680b8510 2023-11-28T11:21:17.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.246+0000 7f796d99a700 1 --2- 172.21.15.92:0/3940011060 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f794c0682f0 0x7f794c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:17.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.247+0000 7f796d99a700 1 --2- 172.21.15.92:0/3940011060 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f794c0682f0 0x7f794c06a7a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f79680c7130 tx=0x7f7958007ed0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:17.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.250+0000 7f795effd700 1 -- 172.21.15.92:0/3940011060 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7964052420 con 0x7f79680b8510 2023-11-28T11:21:17.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.441+0000 7f79703ff700 1 -- 172.21.15.92:0/3940011060 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7968067880 con 0x7f79680b8510 2023-11-28T11:21:17.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.443+0000 7f795effd700 1 -- 172.21.15.92:0/3940011060 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+19748 (secure 0 0 0) 0x7f7964017090 con 0x7f79680b8510 2023-11-28T11:21:17.445 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:21:17.445 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":27,"fsid":"68956e0c-c16f-47cd-a220-f98364b0831d","created":"2023-11-28T11:20:28.697573+0000","modified":"2023-11-28T11:21:00.770781+0000","last_up_change":"2023-11-28T11:20:41.257011+0000","last_in_change":"2023-11-28T11:20:32.249767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:20:41.925237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:20:48.682497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:20:54.471398+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:20:55.204585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:20:58.297307+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6abe483f-0e03-4a26-b9cf-62041291747e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":115823},{"type":"v1","addr":"172.21.15.92:6809","nonce":115823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":115823},{"type":"v1","addr":"172.21.15.92:6813","nonce":115823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":115823},{"type":"v1","addr":"172.21.15.92:6821","nonce":115823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":115823},{"type":"v1","addr":"172.21.15.92:6817","nonce":115823}]},"public_addr":"172.21.15.92:6809/115823","cluster_addr":"172.21.15.92:6813/115823","heartbeat_back_addr":"172.21.15.92:6821/115823","heartbeat_front_addr":"172.21.15.92:6817/115823","state":["exists","up"]},{"osd":1,"uuid":"13c1de73-1a8c-4d7e-8a28-9dae52dfe089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":115824},{"type":"v1","addr":"172.21.15.92:6812","nonce":115824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":115824},{"type":"v1","addr":"172.21.15.92:6816","nonce":115824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":115824},{"type":"v1","addr":"172.21.15.92:6823","nonce":115824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":115824},{"type":"v1","addr":"172.21.15.92:6820","nonce":115824}]},"public_addr":"172.21.15.92:6812/115824","cluster_addr":"172.21.15.92:6816/115824","heartbeat_back_addr":"172.21.15.92:6823/115824","heartbeat_front_addr":"172.21.15.92:6820/115824","state":["exists","up"]},{"osd":2,"uuid":"25788a55-112f-4f00-810d-2bc37cf76778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":115821},{"type":"v1","addr":"172.21.15.92:6801","nonce":115821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":115821},{"type":"v1","addr":"172.21.15.92:6803","nonce":115821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":115821},{"type":"v1","addr":"172.21.15.92:6807","nonce":115821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":115821},{"type":"v1","addr":"172.21.15.92:6805","nonce":115821}]},"public_addr":"172.21.15.92:6801/115821","cluster_addr":"172.21.15.92:6803/115821","heartbeat_back_addr":"172.21.15.92:6807/115821","heartbeat_front_addr":"172.21.15.92:6805/115821","state":["exists","up"]},{"osd":3,"uuid":"adf54e24-33a9-41fe-9ce3-98a5ab6c158e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":115825},{"type":"v1","addr":"172.21.15.92:6825","nonce":115825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":115825},{"type":"v1","addr":"172.21.15.92:6827","nonce":115825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":115825},{"type":"v1","addr":"172.21.15.92:6831","nonce":115825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":115825},{"type":"v1","addr":"172.21.15.92:6829","nonce":115825}]},"public_addr":"172.21.15.92:6825/115825","cluster_addr":"172.21.15.92:6827/115825","heartbeat_back_addr":"172.21.15.92:6831/115825","heartbeat_front_addr":"172.21.15.92:6829/115825","state":["exists","up"]},{"osd":4,"uuid":"ccb9b747-f399-44ea-9a18-9db3152227e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":116035},{"type":"v1","addr":"172.21.15.169:6817","nonce":116035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":116035},{"type":"v1","addr":"172.21.15.169:6819","nonce":116035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":116035},{"type":"v1","addr":"172.21.15.169:6826","nonce":116035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":116035},{"type":"v1","addr":"172.21.15.169:6822","nonce":116035}]},"public_addr":"172.21.15.169:6817/116035","cluster_addr":"172.21.15.169:6819/116035","heartbeat_back_addr":"172.21.15.169:6826/116035","heartbeat_front_addr":"172.21.15.169:6822/116035","state":["exists","up"]},{"osd":5,"uuid":"308a1e63-75aa-4d7d-8a7c-cecc24714fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":116033},{"type":"v1","addr":"172.21.15.169:6801","nonce":116033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":116033},{"type":"v1","addr":"172.21.15.169:6803","nonce":116033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":116033},{"type":"v1","addr":"172.21.15.169:6807","nonce":116033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":116033},{"type":"v1","addr":"172.21.15.169:6805","nonce":116033}]},"public_addr":"172.21.15.169:6801/116033","cluster_addr":"172.21.15.169:6803/116033","heartbeat_back_addr":"172.21.15.169:6807/116033","heartbeat_front_addr":"172.21.15.169:6805/116033","state":["exists","up"]},{"osd":6,"uuid":"9c459b7b-b821-4b4b-b91e-6aaae271457c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":116034},{"type":"v1","addr":"172.21.15.169:6809","nonce":116034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":116034},{"type":"v1","addr":"172.21.15.169:6811","nonce":116034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":116034},{"type":"v1","addr":"172.21.15.169:6815","nonce":116034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":116034},{"type":"v1","addr":"172.21.15.169:6813","nonce":116034}]},"public_addr":"172.21.15.169:6809/116034","cluster_addr":"172.21.15.169:6811/116034","heartbeat_back_addr":"172.21.15.169:6815/116034","heartbeat_front_addr":"172.21.15.169:6813/116034","state":["exists","up"]},{"osd":7,"uuid":"1ca9aa4a-755d-4e28-a937-4d8dccab4df3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":116044},{"type":"v1","addr":"172.21.15.169:6823","nonce":116044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6825","nonce":116044},{"type":"v1","addr":"172.21.15.169:6827","nonce":116044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":116044},{"type":"v1","addr":"172.21.15.169:6831","nonce":116044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":116044},{"type":"v1","addr":"172.21.15.169:6829","nonce":116044}]},"public_addr":"172.21.15.169:6823/116044","cluster_addr":"172.21.15.169:6827/116044","heartbeat_back_addr":"172.21.15.169:6831/116044","heartbeat_front_addr":"172.21.15.169:6829/116044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.586781+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.869778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.508594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.860493+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.872267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.746933+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.611520+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.871446+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:21:17.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.445+0000 7f79703ff700 1 -- 172.21.15.92:0/3940011060 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f794c0682f0 msgr2=0x7f794c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:17.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.445+0000 7f79703ff700 1 --2- 172.21.15.92:0/3940011060 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f794c0682f0 0x7f794c06a7a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f79680c7130 tx=0x7f7958007ed0 comp rx=0 tx=0).stop 2023-11-28T11:21:17.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.445+0000 7f79703ff700 1 -- 172.21.15.92:0/3940011060 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79680b8510 msgr2=0x7f79681f1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:17.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.445+0000 7f79703ff700 1 --2- 172.21.15.92:0/3940011060 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79680b8510 0x7f79681f1430 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f796400ebe0 tx=0x7f796400eef0 comp rx=0 tx=0).stop 2023-11-28T11:21:17.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.445+0000 7f79703ff700 1 -- 172.21.15.92:0/3940011060 shutdown_connections 2023-11-28T11:21:17.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.445+0000 7f79703ff700 1 --2- 172.21.15.92:0/3940011060 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79680d0d60 0x7f79680b2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:17.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.445+0000 7f79703ff700 1 --2- 172.21.15.92:0/3940011060 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f794c0682f0 0x7f794c06a7a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:17.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.445+0000 7f79703ff700 1 --2- 172.21.15.92:0/3940011060 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79680b8510 0x7f79681f1430 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:17.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.445+0000 7f79703ff700 1 --2- 172.21.15.92:0/3940011060 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79680cd460 0x7f79680b2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:17.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.445+0000 7f79703ff700 1 -- 172.21.15.92:0/3940011060 >> 172.21.15.92:0/3940011060 conn(0x7f79680719a0 msgr2=0x7f79680b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:17.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.445+0000 7f79703ff700 1 -- 172.21.15.92:0/3940011060 shutdown_connections 2023-11-28T11:21:17.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.445+0000 7f79703ff700 1 -- 172.21.15.92:0/3940011060 wait complete. 2023-11-28T11:21:17.466 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T11:21:17.466 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:21:17.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.662+0000 7faf07bff700 1 Processor -- start 2023-11-28T11:21:17.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.662+0000 7faf07bff700 1 -- start start 2023-11-28T11:21:17.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.663+0000 7faf07bff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf000b8510 0x7faf0006b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:17.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.663+0000 7faf07bff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7faf000d0d60 0x7faf000d11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:17.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.663+0000 7faf07bff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf000b88e0 con 0x7faf000cd460 2023-11-28T11:21:17.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.663+0000 7faf07bff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf000cdbd0 con 0x7faf000d0d60 2023-11-28T11:21:17.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.663+0000 7faf07bff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf000d02d0 con 0x7faf000b8510 2023-11-28T11:21:17.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.663+0000 7faf0619c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7faf000d0d60 0x7faf000d11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:17.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.663+0000 7faf0619c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7faf000d0d60 0x7faf000d11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49080/0 (socket says 172.21.15.92:49080) 2023-11-28T11:21:17.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.663+0000 7faf0619c700 1 -- 172.21.15.92:0/2233935398 learned_addr learned my addr 172.21.15.92:0/2233935398 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:17.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.664+0000 7faf0599b700 1 --2- 172.21.15.92:0/2233935398 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf000b8510 0x7faf0006b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:17.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.664+0000 7faf0619c700 1 -- 172.21.15.92:0/2233935398 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf000b8510 msgr2=0x7faf0006b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:17.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.664+0000 7faf0619c700 1 --2- 172.21.15.92:0/2233935398 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf000b8510 0x7faf0006b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:17.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.664+0000 7faf0619c700 1 -- 172.21.15.92:0/2233935398 >> v1:172.21.15.92:6789/0 conn(0x7faf000cd460 legacy=0x7faf0006beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:17.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.664+0000 7faf0619c700 1 -- 172.21.15.92:0/2233935398 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf000cd8f0 con 0x7faf000d0d60 2023-11-28T11:21:17.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.664+0000 7faf0619c700 1 --2- 172.21.15.92:0/2233935398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf000d0d60 0x7faf000d11a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7faef8009e80 tx=0x7faef800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1332e70098eff42 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:17.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.664+0000 7faf04999700 1 -- 172.21.15.92:0/2233935398 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef800e070 con 0x7faf000d0d60 2023-11-28T11:21:17.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.664+0000 7faf04999700 1 -- 172.21.15.92:0/2233935398 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faef800b980 con 0x7faf000d0d60 2023-11-28T11:21:17.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.664+0000 7faf04999700 1 -- 172.21.15.92:0/2233935398 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef800bd30 con 0x7faf000d0d60 2023-11-28T11:21:17.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.665+0000 7faf07bff700 1 -- 172.21.15.92:0/2233935398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf000d0d60 msgr2=0x7faf000d11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:17.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.665+0000 7faf07bff700 1 --2- 172.21.15.92:0/2233935398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf000d0d60 0x7faf000d11a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7faef8009e80 tx=0x7faef800b330 comp rx=0 tx=0).stop 2023-11-28T11:21:17.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.665+0000 7faf07bff700 1 -- 172.21.15.92:0/2233935398 shutdown_connections 2023-11-28T11:21:17.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.665+0000 7faf07bff700 1 --2- 172.21.15.92:0/2233935398 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf000b8510 0x7faf0006b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:17.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.665+0000 7faf07bff700 1 --2- 172.21.15.92:0/2233935398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf000d0d60 0x7faf000d11a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:17.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.665+0000 7faf07bff700 1 -- 172.21.15.92:0/2233935398 >> 172.21.15.92:0/2233935398 conn(0x7faf000719a0 msgr2=0x7faf00071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:17.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.665+0000 7faf07bff700 1 -- 172.21.15.92:0/2233935398 shutdown_connections 2023-11-28T11:21:17.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.665+0000 7faf07bff700 1 -- 172.21.15.92:0/2233935398 wait complete. 2023-11-28T11:21:17.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.667+0000 7faf07bff700 1 Processor -- start 2023-11-28T11:21:17.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.667+0000 7faf07bff700 1 -- start start 2023-11-28T11:21:17.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.668+0000 7faf07bff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf000b8510 0x7faf0014d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:17.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.668+0000 7faf07bff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf000cd460 0x7faf0014a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:17.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.668+0000 7faf07bff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf000d0d60 0x7faf00159750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:17.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.668+0000 7faf07bff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0014a7f0 con 0x7faf000d0d60 2023-11-28T11:21:17.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.668+0000 7faf07bff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0014c610 con 0x7faf000cd460 2023-11-28T11:21:17.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.668+0000 7faf07bff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf0014c8f0 con 0x7faf000b8510 2023-11-28T11:21:17.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.668+0000 7faf0619c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf000d0d60 0x7faf00159750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:17.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.668+0000 7faf0619c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf000d0d60 0x7faf00159750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49092/0 (socket says 172.21.15.92:49092) 2023-11-28T11:21:17.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.668+0000 7faf0619c700 1 -- 172.21.15.92:0/1840439116 learned_addr learned my addr 172.21.15.92:0/1840439116 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:17.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.668+0000 7faf0599b700 1 --2- 172.21.15.92:0/1840439116 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf000b8510 0x7faf0014d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:17.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.668+0000 7faf0519a700 1 --2- 172.21.15.92:0/1840439116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf000cd460 0x7faf0014a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:17.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.668+0000 7faf0619c700 1 -- 172.21.15.92:0/1840439116 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf000b8510 msgr2=0x7faf0014d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:17.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.668+0000 7faf0619c700 1 --2- 172.21.15.92:0/1840439116 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf000b8510 0x7faf0014d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:17.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.668+0000 7faf0619c700 1 -- 172.21.15.92:0/1840439116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf000cd460 msgr2=0x7faf0014a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:17.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.668+0000 7faf0619c700 1 --2- 172.21.15.92:0/1840439116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf000cd460 0x7faf0014a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:17.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.668+0000 7faf0619c700 1 -- 172.21.15.92:0/1840439116 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faef8009b30 con 0x7faf000d0d60 2023-11-28T11:21:17.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.668+0000 7faf0519a700 1 --2- 172.21.15.92:0/1840439116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf000cd460 0x7faf0014a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:21:17.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.669+0000 7faf0619c700 1 --2- 172.21.15.92:0/1840439116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf000d0d60 0x7faf00159750 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7faf00072610 tx=0x7faef8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:17.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.669+0000 7faef67fc700 1 -- 172.21.15.92:0/1840439116 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef8004960 con 0x7faf000d0d60 2023-11-28T11:21:17.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.669+0000 7faef67fc700 1 -- 172.21.15.92:0/1840439116 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faef80099b0 con 0x7faf000d0d60 2023-11-28T11:21:17.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.669+0000 7faf07bff700 1 -- 172.21.15.92:0/1840439116 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf000cfff0 con 0x7faf000d0d60 2023-11-28T11:21:17.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.669+0000 7faef67fc700 1 -- 172.21.15.92:0/1840439116 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef801b670 con 0x7faf000d0d60 2023-11-28T11:21:17.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.669+0000 7faf07bff700 1 -- 172.21.15.92:0/1840439116 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf000cfd10 con 0x7faf000d0d60 2023-11-28T11:21:17.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.670+0000 7faf07bff700 1 -- 172.21.15.92:0/1840439116 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf00067880 con 0x7faf000d0d60 2023-11-28T11:21:17.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.671+0000 7faef67fc700 1 -- 172.21.15.92:0/1840439116 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7faef8021030 con 0x7faf000d0d60 2023-11-28T11:21:17.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.672+0000 7faef67fc700 1 --2- 172.21.15.92:0/1840439116 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7faee40682f0 0x7faee406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:17.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.672+0000 7faef67fc700 1 -- 172.21.15.92:0/1840439116 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7faef8087610 con 0x7faf000d0d60 2023-11-28T11:21:17.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.672+0000 7faf0599b700 1 --2- 172.21.15.92:0/1840439116 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7faee40682f0 0x7faee406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:17.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.673+0000 7faf0599b700 1 --2- 172.21.15.92:0/1840439116 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7faee40682f0 0x7faee406a7a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7faf001594d0 tx=0x7faefc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:17.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.676+0000 7faef67fc700 1 -- 172.21.15.92:0/1840439116 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7faef8053a90 con 0x7faf000d0d60 2023-11-28T11:21:17.823 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:21:17.828 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:21:17.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.868+0000 7faf07bff700 1 -- 172.21.15.92:0/1840439116 --> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faf000d2940 con 0x7faee40682f0 2023-11-28T11:21:17.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.882+0000 7faef67fc700 1 -- 172.21.15.92:0/1840439116 <== mgr.4103 v2:172.21.15.92:6832/115482 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367403 (secure 0 0 0) 0x7faf000d2940 con 0x7faee40682f0 2023-11-28T11:21:17.892 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:21:17.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.887+0000 7faf07bff700 1 -- 172.21.15.92:0/1840439116 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7faee40682f0 msgr2=0x7faee406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:17.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.887+0000 7faf07bff700 1 --2- 172.21.15.92:0/1840439116 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7faee40682f0 0x7faee406a7a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7faf001594d0 tx=0x7faefc008040 comp rx=0 tx=0).stop 2023-11-28T11:21:17.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.887+0000 7faf07bff700 1 -- 172.21.15.92:0/1840439116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf000d0d60 msgr2=0x7faf00159750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:17.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.887+0000 7faf07bff700 1 --2- 172.21.15.92:0/1840439116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf000d0d60 0x7faf00159750 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7faf00072610 tx=0x7faef8004ad0 comp rx=0 tx=0).stop 2023-11-28T11:21:17.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.887+0000 7faf07bff700 1 -- 172.21.15.92:0/1840439116 shutdown_connections 2023-11-28T11:21:17.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.887+0000 7faf07bff700 1 --2- 172.21.15.92:0/1840439116 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7faee40682f0 0x7faee406a7a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:17.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.887+0000 7faf07bff700 1 --2- 172.21.15.92:0/1840439116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf000d0d60 0x7faf00159750 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:17.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.887+0000 7faf07bff700 1 --2- 172.21.15.92:0/1840439116 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf000b8510 0x7faf0014d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:17.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.887+0000 7faf07bff700 1 --2- 172.21.15.92:0/1840439116 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf000cd460 0x7faf0014a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:17.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.887+0000 7faf07bff700 1 -- 172.21.15.92:0/1840439116 >> 172.21.15.92:0/1840439116 conn(0x7faf000719a0 msgr2=0x7faf000b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:17.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.887+0000 7faf07bff700 1 -- 172.21.15.92:0/1840439116 shutdown_connections 2023-11-28T11:21:17.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:17.887+0000 7faf07bff700 1 -- 172.21.15.92:0/1840439116 wait complete. 2023-11-28T11:21:17.898 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-11-28T11:21:17.908 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2023-11-28T11:21:17.864979+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594896,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":67492,"kb_used_data":3492,"kb_used_omap":0,"kb_used_meta":63936,"kb_avail":754907228,"statfs":{"total":773094113280,"available":773025001472,"internally_reserved":0,"allocated":3575808,"data_stored":2148378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65470464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2198,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002663"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.282064+0000","last_change":"2023-11-28T11:20:56.269261+0000","last_active":"2023-11-28T11:21:00.282064+0000","last_peered":"2023-11-28T11:21:00.282064+0000","last_clean":"2023-11-28T11:21:00.282064+0000","last_became_active":"2023-11-28T11:20:56.268959+0000","last_became_peered":"2023-11-28T11:20:56.268959+0000","last_unstale":"2023-11-28T11:21:00.282064+0000","last_undegraded":"2023-11-28T11:21:00.282064+0000","last_fullsized":"2023-11-28T11:21:00.282064+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:39:28.472955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.273172+0000","last_change":"2023-11-28T11:20:56.258503+0000","last_active":"2023-11-28T11:21:00.273172+0000","last_peered":"2023-11-28T11:21:00.273172+0000","last_clean":"2023-11-28T11:21:00.273172+0000","last_became_active":"2023-11-28T11:20:56.258101+0000","last_became_peered":"2023-11-28T11:20:56.258101+0000","last_unstale":"2023-11-28T11:21:00.273172+0000","last_undegraded":"2023-11-28T11:21:00.273172+0000","last_fullsized":"2023-11-28T11:21:00.273172+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:14:34.876987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.274259+0000","last_change":"2023-11-28T11:20:56.269696+0000","last_active":"2023-11-28T11:21:00.274259+0000","last_peered":"2023-11-28T11:21:00.274259+0000","last_clean":"2023-11-28T11:21:00.274259+0000","last_became_active":"2023-11-28T11:20:56.264650+0000","last_became_peered":"2023-11-28T11:20:56.264650+0000","last_unstale":"2023-11-28T11:21:00.274259+0000","last_undegraded":"2023-11-28T11:21:00.274259+0000","last_fullsized":"2023-11-28T11:21:00.274259+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:25:52.242410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.286447+0000","last_change":"2023-11-28T11:20:56.269817+0000","last_active":"2023-11-28T11:21:00.286447+0000","last_peered":"2023-11-28T11:21:00.286447+0000","last_clean":"2023-11-28T11:21:00.286447+0000","last_became_active":"2023-11-28T11:20:56.269514+0000","last_became_peered":"2023-11-28T11:20:56.269514+0000","last_unstale":"2023-11-28T11:21:00.286447+0000","last_undegraded":"2023-11-28T11:21:00.286447+0000","last_fullsized":"2023-11-28T11:21:00.286447+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:09:41.753917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.289921+0000","last_change":"2023-11-28T11:20:56.274319+0000","last_active":"2023-11-28T11:21:00.289921+0000","last_peered":"2023-11-28T11:21:00.289921+0000","last_clean":"2023-11-28T11:21:00.289921+0000","last_became_active":"2023-11-28T11:20:56.273667+0000","last_became_peered":"2023-11-28T11:20:56.273667+0000","last_unstale":"2023-11-28T11:21:00.289921+0000","last_undegraded":"2023-11-28T11:21:00.289921+0000","last_fullsized":"2023-11-28T11:21:00.289921+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:31:26.827850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.274908+0000","last_change":"2023-11-28T11:20:56.263963+0000","last_active":"2023-11-28T11:21:00.274908+0000","last_peered":"2023-11-28T11:21:00.274908+0000","last_clean":"2023-11-28T11:21:00.274908+0000","last_became_active":"2023-11-28T11:20:56.260761+0000","last_became_peered":"2023-11-28T11:20:56.260761+0000","last_unstale":"2023-11-28T11:21:00.274908+0000","last_undegraded":"2023-11-28T11:21:00.274908+0000","last_fullsized":"2023-11-28T11:21:00.274908+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:13:46.683000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.288334+0000","last_change":"2023-11-28T11:20:56.261054+0000","last_active":"2023-11-28T11:21:00.288334+0000","last_peered":"2023-11-28T11:21:00.288334+0000","last_clean":"2023-11-28T11:21:00.288334+0000","last_became_active":"2023-11-28T11:20:56.260774+0000","last_became_peered":"2023-11-28T11:20:56.260774+0000","last_unstale":"2023-11-28T11:21:00.288334+0000","last_undegraded":"2023-11-28T11:21:00.288334+0000","last_fullsized":"2023-11-28T11:21:00.288334+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:08.079216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.273485+0000","last_change":"2023-11-28T11:20:56.252717+0000","last_active":"2023-11-28T11:21:00.273485+0000","last_peered":"2023-11-28T11:21:00.273485+0000","last_clean":"2023-11-28T11:21:00.273485+0000","last_became_active":"2023-11-28T11:20:56.252066+0000","last_became_peered":"2023-11-28T11:20:56.252066+0000","last_unstale":"2023-11-28T11:21:00.273485+0000","last_undegraded":"2023-11-28T11:21:00.273485+0000","last_fullsized":"2023-11-28T11:21:00.273485+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:42:57.864956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.291971+0000","last_change":"2023-11-28T11:20:56.273810+0000","last_active":"2023-11-28T11:21:00.291971+0000","last_peered":"2023-11-28T11:21:00.291971+0000","last_clean":"2023-11-28T11:21:00.291971+0000","last_became_active":"2023-11-28T11:20:56.273477+0000","last_became_peered":"2023-11-28T11:20:56.273477+0000","last_unstale":"2023-11-28T11:21:00.291971+0000","last_undegraded":"2023-11-28T11:21:00.291971+0000","last_fullsized":"2023-11-28T11:21:00.291971+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:50:45.228070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.277322+0000","last_change":"2023-11-28T11:20:56.256667+0000","last_active":"2023-11-28T11:21:00.277322+0000","last_peered":"2023-11-28T11:21:00.277322+0000","last_clean":"2023-11-28T11:21:00.277322+0000","last_became_active":"2023-11-28T11:20:56.256387+0000","last_became_peered":"2023-11-28T11:20:56.256387+0000","last_unstale":"2023-11-28T11:21:00.277322+0000","last_undegraded":"2023-11-28T11:21:00.277322+0000","last_fullsized":"2023-11-28T11:21:00.277322+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:24:07.384730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.274154+0000","last_change":"2023-11-28T11:20:56.281728+0000","last_active":"2023-11-28T11:21:00.274154+0000","last_peered":"2023-11-28T11:21:00.274154+0000","last_clean":"2023-11-28T11:21:00.274154+0000","last_became_active":"2023-11-28T11:20:56.281437+0000","last_became_peered":"2023-11-28T11:20:56.281437+0000","last_unstale":"2023-11-28T11:21:00.274154+0000","last_undegraded":"2023-11-28T11:21:00.274154+0000","last_fullsized":"2023-11-28T11:21:00.274154+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:01:57.936285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.280518+0000","last_change":"2023-11-28T11:20:56.256016+0000","last_active":"2023-11-28T11:21:00.280518+0000","last_peered":"2023-11-28T11:21:00.280518+0000","last_clean":"2023-11-28T11:21:00.280518+0000","last_became_active":"2023-11-28T11:20:56.255480+0000","last_became_peered":"2023-11-28T11:20:56.255480+0000","last_unstale":"2023-11-28T11:21:00.280518+0000","last_undegraded":"2023-11-28T11:21:00.280518+0000","last_fullsized":"2023-11-28T11:21:00.280518+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:45:09.586958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.280455+0000","last_change":"2023-11-28T11:20:56.265497+0000","last_active":"2023-11-28T11:21:00.280455+0000","last_peered":"2023-11-28T11:21:00.280455+0000","last_clean":"2023-11-28T11:21:00.280455+0000","last_became_active":"2023-11-28T11:20:56.265236+0000","last_became_peered":"2023-11-28T11:20:56.265236+0000","last_unstale":"2023-11-28T11:21:00.280455+0000","last_undegraded":"2023-11-28T11:21:00.280455+0000","last_fullsized":"2023-11-28T11:21:00.280455+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:47:39.108512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.274722+0000","last_change":"2023-11-28T11:20:56.262165+0000","last_active":"2023-11-28T11:21:00.274722+0000","last_peered":"2023-11-28T11:21:00.274722+0000","last_clean":"2023-11-28T11:21:00.274722+0000","last_became_active":"2023-11-28T11:20:56.261947+0000","last_became_peered":"2023-11-28T11:20:56.261947+0000","last_unstale":"2023-11-28T11:21:00.274722+0000","last_undegraded":"2023-11-28T11:21:00.274722+0000","last_fullsized":"2023-11-28T11:21:00.274722+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:45:14.478416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.288123+0000","last_change":"2023-11-28T11:20:56.264861+0000","last_active":"2023-11-28T11:21:00.288123+0000","last_peered":"2023-11-28T11:21:00.288123+0000","last_clean":"2023-11-28T11:21:00.288123+0000","last_became_active":"2023-11-28T11:20:56.264538+0000","last_became_peered":"2023-11-28T11:20:56.264538+0000","last_unstale":"2023-11-28T11:21:00.288123+0000","last_undegraded":"2023-11-28T11:21:00.288123+0000","last_fullsized":"2023-11-28T11:21:00.288123+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:40:42.059656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.273219+0000","last_change":"2023-11-28T11:20:56.258923+0000","last_active":"2023-11-28T11:21:00.273219+0000","last_peered":"2023-11-28T11:21:00.273219+0000","last_clean":"2023-11-28T11:21:00.273219+0000","last_became_active":"2023-11-28T11:20:56.258633+0000","last_became_peered":"2023-11-28T11:20:56.258633+0000","last_unstale":"2023-11-28T11:21:00.273219+0000","last_undegraded":"2023-11-28T11:21:00.273219+0000","last_fullsized":"2023-11-28T11:21:00.273219+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:26:32.343329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.292050+0000","last_change":"2023-11-28T11:20:56.271537+0000","last_active":"2023-11-28T11:21:00.292050+0000","last_peered":"2023-11-28T11:21:00.292050+0000","last_clean":"2023-11-28T11:21:00.292050+0000","last_became_active":"2023-11-28T11:20:56.270556+0000","last_became_peered":"2023-11-28T11:20:56.270556+0000","last_unstale":"2023-11-28T11:21:00.292050+0000","last_undegraded":"2023-11-28T11:21:00.292050+0000","last_fullsized":"2023-11-28T11:21:00.292050+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:11:36.091056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.274876+0000","last_change":"2023-11-28T11:20:56.274810+0000","last_active":"2023-11-28T11:21:00.274876+0000","last_peered":"2023-11-28T11:21:00.274876+0000","last_clean":"2023-11-28T11:21:00.274876+0000","last_became_active":"2023-11-28T11:20:56.274523+0000","last_became_peered":"2023-11-28T11:20:56.274523+0000","last_unstale":"2023-11-28T11:21:00.274876+0000","last_undegraded":"2023-11-28T11:21:00.274876+0000","last_fullsized":"2023-11-28T11:21:00.274876+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:42:57.864956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.281099+0000","last_change":"2023-11-28T11:20:56.258601+0000","last_active":"2023-11-28T11:21:00.281099+0000","last_peered":"2023-11-28T11:21:00.281099+0000","last_clean":"2023-11-28T11:21:00.281099+0000","last_became_active":"2023-11-28T11:20:56.258282+0000","last_became_peered":"2023-11-28T11:20:56.258282+0000","last_unstale":"2023-11-28T11:21:00.281099+0000","last_undegraded":"2023-11-28T11:21:00.281099+0000","last_fullsized":"2023-11-28T11:21:00.281099+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:58:47.731976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.275898+0000","last_change":"2023-11-28T11:20:56.263948+0000","last_active":"2023-11-28T11:21:00.275898+0000","last_peered":"2023-11-28T11:21:00.275898+0000","last_clean":"2023-11-28T11:21:00.275898+0000","last_became_active":"2023-11-28T11:20:56.263627+0000","last_became_peered":"2023-11-28T11:20:56.263627+0000","last_unstale":"2023-11-28T11:21:00.275898+0000","last_undegraded":"2023-11-28T11:21:00.275898+0000","last_fullsized":"2023-11-28T11:21:00.275898+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:08.079216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.300168+0000","last_change":"2023-11-28T11:20:56.274116+0000","last_active":"2023-11-28T11:21:00.300168+0000","last_peered":"2023-11-28T11:21:00.300168+0000","last_clean":"2023-11-28T11:21:00.300168+0000","last_became_active":"2023-11-28T11:20:56.273403+0000","last_became_peered":"2023-11-28T11:20:56.273403+0000","last_unstale":"2023-11-28T11:21:00.300168+0000","last_undegraded":"2023-11-28T11:21:00.300168+0000","last_fullsized":"2023-11-28T11:21:00.300168+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:45:14.478416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.291080+0000","last_change":"2023-11-28T11:20:56.259411+0000","last_active":"2023-11-28T11:21:00.291080+0000","last_peered":"2023-11-28T11:21:00.291080+0000","last_clean":"2023-11-28T11:21:00.291080+0000","last_became_active":"2023-11-28T11:20:56.259189+0000","last_became_peered":"2023-11-28T11:20:56.259189+0000","last_unstale":"2023-11-28T11:21:00.291080+0000","last_undegraded":"2023-11-28T11:21:00.291080+0000","last_fullsized":"2023-11-28T11:21:00.291080+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:24:07.384730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.270889+0000","last_change":"2023-11-28T11:20:56.257033+0000","last_active":"2023-11-28T11:21:00.270889+0000","last_peered":"2023-11-28T11:21:00.270889+0000","last_clean":"2023-11-28T11:21:00.270889+0000","last_became_active":"2023-11-28T11:20:56.256549+0000","last_became_peered":"2023-11-28T11:20:56.256549+0000","last_unstale":"2023-11-28T11:21:00.270889+0000","last_undegraded":"2023-11-28T11:21:00.270889+0000","last_fullsized":"2023-11-28T11:21:00.270889+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:24:07.384730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.286598+0000","last_change":"2023-11-28T11:20:56.270979+0000","last_active":"2023-11-28T11:21:00.286598+0000","last_peered":"2023-11-28T11:21:00.286598+0000","last_clean":"2023-11-28T11:21:00.286598+0000","last_became_active":"2023-11-28T11:20:56.270710+0000","last_became_peered":"2023-11-28T11:20:56.270710+0000","last_unstale":"2023-11-28T11:21:00.286598+0000","last_undegraded":"2023-11-28T11:21:00.286598+0000","last_fullsized":"2023-11-28T11:21:00.286598+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:42:57.864956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.290274+0000","last_change":"2023-11-28T11:20:56.262561+0000","last_active":"2023-11-28T11:21:00.290274+0000","last_peered":"2023-11-28T11:21:00.290274+0000","last_clean":"2023-11-28T11:21:00.290274+0000","last_became_active":"2023-11-28T11:20:56.262251+0000","last_became_peered":"2023-11-28T11:20:56.262251+0000","last_unstale":"2023-11-28T11:21:00.290274+0000","last_undegraded":"2023-11-28T11:21:00.290274+0000","last_fullsized":"2023-11-28T11:21:00.290274+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:11:36.091056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.292352+0000","last_change":"2023-11-28T11:20:56.261421+0000","last_active":"2023-11-28T11:21:00.292352+0000","last_peered":"2023-11-28T11:21:00.292352+0000","last_clean":"2023-11-28T11:21:00.292352+0000","last_became_active":"2023-11-28T11:20:56.259655+0000","last_became_peered":"2023-11-28T11:20:56.259655+0000","last_unstale":"2023-11-28T11:21:00.292352+0000","last_undegraded":"2023-11-28T11:21:00.292352+0000","last_fullsized":"2023-11-28T11:21:00.292352+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:01:57.936285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.273393+0000","last_change":"2023-11-28T11:20:56.258358+0000","last_active":"2023-11-28T11:21:00.273393+0000","last_peered":"2023-11-28T11:21:00.273393+0000","last_clean":"2023-11-28T11:21:00.273393+0000","last_became_active":"2023-11-28T11:20:56.257895+0000","last_became_peered":"2023-11-28T11:20:56.257895+0000","last_unstale":"2023-11-28T11:21:00.273393+0000","last_undegraded":"2023-11-28T11:21:00.273393+0000","last_fullsized":"2023-11-28T11:21:00.273393+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:08.079216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.777639+0000","last_change":"2023-11-28T11:21:00.306498+0000","last_active":"2023-11-28T11:21:00.777639+0000","last_peered":"2023-11-28T11:21:00.777639+0000","last_clean":"2023-11-28T11:21:00.777639+0000","last_became_active":"2023-11-28T11:21:00.305632+0000","last_became_peered":"2023-11-28T11:21:00.305632+0000","last_unstale":"2023-11-28T11:21:00.777639+0000","last_undegraded":"2023-11-28T11:21:00.777639+0000","last_fullsized":"2023-11-28T11:21:00.777639+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:08:42.028668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.274765+0000","last_change":"2023-11-28T11:20:55.761598+0000","last_active":"2023-11-28T11:21:00.274765+0000","last_peered":"2023-11-28T11:21:00.274765+0000","last_clean":"2023-11-28T11:21:00.274765+0000","last_became_active":"2023-11-28T11:20:55.761257+0000","last_became_peered":"2023-11-28T11:20:55.761257+0000","last_unstale":"2023-11-28T11:21:00.274765+0000","last_undegraded":"2023-11-28T11:21:00.274765+0000","last_fullsized":"2023-11-28T11:21:00.274765+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:13:32.428410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.271441+0000","last_change":"2023-11-28T11:20:56.269300+0000","last_active":"2023-11-28T11:21:00.271441+0000","last_peered":"2023-11-28T11:21:00.271441+0000","last_clean":"2023-11-28T11:21:00.271441+0000","last_became_active":"2023-11-28T11:20:56.269068+0000","last_became_peered":"2023-11-28T11:20:56.269068+0000","last_unstale":"2023-11-28T11:21:00.271441+0000","last_undegraded":"2023-11-28T11:21:00.271441+0000","last_fullsized":"2023-11-28T11:21:00.271441+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:42:57.864956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.779188+0000","last_change":"2023-11-28T11:21:00.303150+0000","last_active":"2023-11-28T11:21:00.779188+0000","last_peered":"2023-11-28T11:21:00.779188+0000","last_clean":"2023-11-28T11:21:00.779188+0000","last_became_active":"2023-11-28T11:21:00.302669+0000","last_became_peered":"2023-11-28T11:21:00.302669+0000","last_unstale":"2023-11-28T11:21:00.779188+0000","last_undegraded":"2023-11-28T11:21:00.779188+0000","last_fullsized":"2023-11-28T11:21:00.779188+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:26:35.867941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.284078+0000","last_change":"2023-11-28T11:20:55.769854+0000","last_active":"2023-11-28T11:21:00.284078+0000","last_peered":"2023-11-28T11:21:00.284078+0000","last_clean":"2023-11-28T11:21:00.284078+0000","last_became_active":"2023-11-28T11:20:55.769322+0000","last_became_peered":"2023-11-28T11:20:55.769322+0000","last_unstale":"2023-11-28T11:21:00.284078+0000","last_undegraded":"2023-11-28T11:21:00.284078+0000","last_fullsized":"2023-11-28T11:21:00.284078+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:37:05.627049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.291807+0000","last_change":"2023-11-28T11:20:56.265137+0000","last_active":"2023-11-28T11:21:00.291807+0000","last_peered":"2023-11-28T11:21:00.291807+0000","last_clean":"2023-11-28T11:21:00.291807+0000","last_became_active":"2023-11-28T11:20:56.264228+0000","last_became_peered":"2023-11-28T11:20:56.264228+0000","last_unstale":"2023-11-28T11:21:00.291807+0000","last_undegraded":"2023-11-28T11:21:00.291807+0000","last_fullsized":"2023-11-28T11:21:00.291807+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:42:57.864956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.884947+0000","last_change":"2023-11-28T11:21:00.314686+0000","last_active":"2023-11-28T11:21:01.884947+0000","last_peered":"2023-11-28T11:21:01.884947+0000","last_clean":"2023-11-28T11:21:01.884947+0000","last_became_active":"2023-11-28T11:21:00.314281+0000","last_became_peered":"2023-11-28T11:21:00.314281+0000","last_unstale":"2023-11-28T11:21:01.884947+0000","last_undegraded":"2023-11-28T11:21:01.884947+0000","last_fullsized":"2023-11-28T11:21:01.884947+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:29:41.143865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.271248+0000","last_change":"2023-11-28T11:20:55.768237+0000","last_active":"2023-11-28T11:21:00.271248+0000","last_peered":"2023-11-28T11:21:00.271248+0000","last_clean":"2023-11-28T11:21:00.271248+0000","last_became_active":"2023-11-28T11:20:55.767948+0000","last_became_peered":"2023-11-28T11:20:55.767948+0000","last_unstale":"2023-11-28T11:21:00.271248+0000","last_undegraded":"2023-11-28T11:21:00.271248+0000","last_fullsized":"2023-11-28T11:21:00.271248+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:23:36.801918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.286831+0000","last_change":"2023-11-28T11:20:56.260663+0000","last_active":"2023-11-28T11:21:00.286831+0000","last_peered":"2023-11-28T11:21:00.286831+0000","last_clean":"2023-11-28T11:21:00.286831+0000","last_became_active":"2023-11-28T11:20:56.259825+0000","last_became_peered":"2023-11-28T11:20:56.259825+0000","last_unstale":"2023-11-28T11:21:00.286831+0000","last_undegraded":"2023-11-28T11:21:00.286831+0000","last_fullsized":"2023-11-28T11:21:00.286831+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:45:09.586958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.778585+0000","last_change":"2023-11-28T11:21:00.303720+0000","last_active":"2023-11-28T11:21:00.778585+0000","last_peered":"2023-11-28T11:21:00.778585+0000","last_clean":"2023-11-28T11:21:00.778585+0000","last_became_active":"2023-11-28T11:21:00.302585+0000","last_became_peered":"2023-11-28T11:21:00.302585+0000","last_unstale":"2023-11-28T11:21:00.778585+0000","last_undegraded":"2023-11-28T11:21:00.778585+0000","last_fullsized":"2023-11-28T11:21:00.778585+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:28:33.645294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.292130+0000","last_change":"2023-11-28T11:20:55.764526+0000","last_active":"2023-11-28T11:21:00.292130+0000","last_peered":"2023-11-28T11:21:00.292130+0000","last_clean":"2023-11-28T11:21:00.292130+0000","last_became_active":"2023-11-28T11:20:55.764278+0000","last_became_peered":"2023-11-28T11:20:55.764278+0000","last_unstale":"2023-11-28T11:21:00.292130+0000","last_undegraded":"2023-11-28T11:21:00.292130+0000","last_fullsized":"2023-11-28T11:21:00.292130+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:23:36.801918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.271155+0000","last_change":"2023-11-28T11:20:56.265473+0000","last_active":"2023-11-28T11:21:00.271155+0000","last_peered":"2023-11-28T11:21:00.271155+0000","last_clean":"2023-11-28T11:21:00.271155+0000","last_became_active":"2023-11-28T11:20:56.262049+0000","last_became_peered":"2023-11-28T11:20:56.262049+0000","last_unstale":"2023-11-28T11:21:00.271155+0000","last_undegraded":"2023-11-28T11:21:00.271155+0000","last_fullsized":"2023-11-28T11:21:00.271155+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:08.079216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.806510+0000","last_change":"2023-11-28T11:21:00.306448+0000","last_active":"2023-11-28T11:21:01.806510+0000","last_peered":"2023-11-28T11:21:01.806510+0000","last_clean":"2023-11-28T11:21:01.806510+0000","last_became_active":"2023-11-28T11:21:00.305892+0000","last_became_peered":"2023-11-28T11:21:00.305892+0000","last_unstale":"2023-11-28T11:21:01.806510+0000","last_undegraded":"2023-11-28T11:21:01.806510+0000","last_fullsized":"2023-11-28T11:21:01.806510+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:41.700110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.277223+0000","last_change":"2023-11-28T11:20:55.764162+0000","last_active":"2023-11-28T11:21:00.277223+0000","last_peered":"2023-11-28T11:21:00.277223+0000","last_clean":"2023-11-28T11:21:00.277223+0000","last_became_active":"2023-11-28T11:20:55.760490+0000","last_became_peered":"2023-11-28T11:20:55.760490+0000","last_unstale":"2023-11-28T11:21:00.277223+0000","last_undegraded":"2023-11-28T11:21:00.277223+0000","last_fullsized":"2023-11-28T11:21:00.277223+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:39:27.467031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.271241+0000","last_change":"2023-11-28T11:20:56.269435+0000","last_active":"2023-11-28T11:21:00.271241+0000","last_peered":"2023-11-28T11:21:00.271241+0000","last_clean":"2023-11-28T11:21:00.271241+0000","last_became_active":"2023-11-28T11:20:56.269179+0000","last_became_peered":"2023-11-28T11:20:56.269179+0000","last_unstale":"2023-11-28T11:21:00.271241+0000","last_undegraded":"2023-11-28T11:21:00.271241+0000","last_fullsized":"2023-11-28T11:21:00.271241+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:50:45.228070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.844101+0000","last_change":"2023-11-28T11:21:00.292120+0000","last_active":"2023-11-28T11:21:01.844101+0000","last_peered":"2023-11-28T11:21:01.844101+0000","last_clean":"2023-11-28T11:21:01.844101+0000","last_became_active":"2023-11-28T11:21:00.289430+0000","last_became_peered":"2023-11-28T11:21:00.289430+0000","last_unstale":"2023-11-28T11:21:01.844101+0000","last_undegraded":"2023-11-28T11:21:01.844101+0000","last_fullsized":"2023-11-28T11:21:01.844101+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:41.700110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.298184+0000","last_change":"2023-11-28T11:20:55.771668+0000","last_active":"2023-11-28T11:21:00.298184+0000","last_peered":"2023-11-28T11:21:00.298184+0000","last_clean":"2023-11-28T11:21:00.298184+0000","last_became_active":"2023-11-28T11:20:55.771400+0000","last_became_peered":"2023-11-28T11:20:55.771400+0000","last_unstale":"2023-11-28T11:21:00.298184+0000","last_undegraded":"2023-11-28T11:21:00.298184+0000","last_fullsized":"2023-11-28T11:21:00.298184+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:27:11.291583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.290416+0000","last_change":"2023-11-28T11:20:56.263283+0000","last_active":"2023-11-28T11:21:00.290416+0000","last_peered":"2023-11-28T11:21:00.290416+0000","last_clean":"2023-11-28T11:21:00.290416+0000","last_became_active":"2023-11-28T11:20:56.262787+0000","last_became_peered":"2023-11-28T11:20:56.262787+0000","last_unstale":"2023-11-28T11:21:00.290416+0000","last_undegraded":"2023-11-28T11:21:00.290416+0000","last_fullsized":"2023-11-28T11:21:00.290416+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:40:42.059656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.883480+0000","last_change":"2023-11-28T11:21:00.297947+0000","last_active":"2023-11-28T11:21:01.883480+0000","last_peered":"2023-11-28T11:21:01.883480+0000","last_clean":"2023-11-28T11:21:01.883480+0000","last_became_active":"2023-11-28T11:21:00.297320+0000","last_became_peered":"2023-11-28T11:21:00.297320+0000","last_unstale":"2023-11-28T11:21:01.883480+0000","last_undegraded":"2023-11-28T11:21:01.883480+0000","last_fullsized":"2023-11-28T11:21:01.883480+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:56:39.135021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"27'10","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:13.462618+0000","last_change":"2023-11-28T11:20:55.761529+0000","last_active":"2023-11-28T11:21:13.462618+0000","last_peered":"2023-11-28T11:21:13.462618+0000","last_clean":"2023-11-28T11:21:13.462618+0000","last_became_active":"2023-11-28T11:20:55.759570+0000","last_became_peered":"2023-11-28T11:20:55.759570+0000","last_unstale":"2023-11-28T11:21:13.462618+0000","last_undegraded":"2023-11-28T11:21:13.462618+0000","last_fullsized":"2023-11-28T11:21:13.462618+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:54:56.683286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3746,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.272939+0000","last_change":"2023-11-28T11:20:56.280987+0000","last_active":"2023-11-28T11:21:00.272939+0000","last_peered":"2023-11-28T11:21:00.272939+0000","last_clean":"2023-11-28T11:21:00.272939+0000","last_became_active":"2023-11-28T11:20:56.279820+0000","last_became_peered":"2023-11-28T11:20:56.279820+0000","last_unstale":"2023-11-28T11:21:00.272939+0000","last_undegraded":"2023-11-28T11:21:00.272939+0000","last_fullsized":"2023-11-28T11:21:00.272939+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:50:45.228070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.383399+0000","last_change":"2023-11-28T11:21:00.294226+0000","last_active":"2023-11-28T11:21:01.383399+0000","last_peered":"2023-11-28T11:21:01.383399+0000","last_clean":"2023-11-28T11:21:01.383399+0000","last_became_active":"2023-11-28T11:21:00.293015+0000","last_became_peered":"2023-11-28T11:21:00.293015+0000","last_unstale":"2023-11-28T11:21:01.383399+0000","last_undegraded":"2023-11-28T11:21:01.383399+0000","last_fullsized":"2023-11-28T11:21:01.383399+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:35:00.472959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.298041+0000","last_change":"2023-11-28T11:20:55.766180+0000","last_active":"2023-11-28T11:21:00.298041+0000","last_peered":"2023-11-28T11:21:00.298041+0000","last_clean":"2023-11-28T11:21:00.298041+0000","last_became_active":"2023-11-28T11:20:55.765845+0000","last_became_peered":"2023-11-28T11:20:55.765845+0000","last_unstale":"2023-11-28T11:21:00.298041+0000","last_undegraded":"2023-11-28T11:21:00.298041+0000","last_fullsized":"2023-11-28T11:21:00.298041+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:39:27.467031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.271449+0000","last_change":"2023-11-28T11:20:56.268499+0000","last_active":"2023-11-28T11:21:00.271449+0000","last_peered":"2023-11-28T11:21:00.271449+0000","last_clean":"2023-11-28T11:21:00.271449+0000","last_became_active":"2023-11-28T11:20:56.267802+0000","last_became_peered":"2023-11-28T11:20:56.267802+0000","last_unstale":"2023-11-28T11:21:00.271449+0000","last_undegraded":"2023-11-28T11:21:00.271449+0000","last_fullsized":"2023-11-28T11:21:00.271449+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:11:36.091056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.844455+0000","last_change":"2023-11-28T11:21:00.302400+0000","last_active":"2023-11-28T11:21:01.844455+0000","last_peered":"2023-11-28T11:21:01.844455+0000","last_clean":"2023-11-28T11:21:01.844455+0000","last_became_active":"2023-11-28T11:21:00.301539+0000","last_became_peered":"2023-11-28T11:21:00.301539+0000","last_unstale":"2023-11-28T11:21:01.844455+0000","last_undegraded":"2023-11-28T11:21:01.844455+0000","last_fullsized":"2023-11-28T11:21:01.844455+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:53:42.920160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.290955+0000","last_change":"2023-11-28T11:20:55.766030+0000","last_active":"2023-11-28T11:21:00.290955+0000","last_peered":"2023-11-28T11:21:00.290955+0000","last_clean":"2023-11-28T11:21:00.290955+0000","last_became_active":"2023-11-28T11:20:55.765787+0000","last_became_peered":"2023-11-28T11:20:55.765787+0000","last_unstale":"2023-11-28T11:21:00.290955+0000","last_undegraded":"2023-11-28T11:21:00.290955+0000","last_fullsized":"2023-11-28T11:21:00.290955+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:58:46.726052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.273291+0000","last_change":"2023-11-28T11:20:56.258188+0000","last_active":"2023-11-28T11:21:00.273291+0000","last_peered":"2023-11-28T11:21:00.273291+0000","last_clean":"2023-11-28T11:21:00.273291+0000","last_became_active":"2023-11-28T11:20:56.257558+0000","last_became_peered":"2023-11-28T11:20:56.257558+0000","last_unstale":"2023-11-28T11:21:00.273291+0000","last_undegraded":"2023-11-28T11:21:00.273291+0000","last_fullsized":"2023-11-28T11:21:00.273291+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:11:36.091056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.844327+0000","last_change":"2023-11-28T11:21:00.307457+0000","last_active":"2023-11-28T11:21:01.844327+0000","last_peered":"2023-11-28T11:21:01.844327+0000","last_clean":"2023-11-28T11:21:01.844327+0000","last_became_active":"2023-11-28T11:21:00.307044+0000","last_became_peered":"2023-11-28T11:21:00.307044+0000","last_unstale":"2023-11-28T11:21:01.844327+0000","last_undegraded":"2023-11-28T11:21:01.844327+0000","last_fullsized":"2023-11-28T11:21:01.844327+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:35:00.472959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.288230+0000","last_change":"2023-11-28T11:20:55.765803+0000","last_active":"2023-11-28T11:21:00.288230+0000","last_peered":"2023-11-28T11:21:00.288230+0000","last_clean":"2023-11-28T11:21:00.288230+0000","last_became_active":"2023-11-28T11:20:55.765501+0000","last_became_peered":"2023-11-28T11:20:55.765501+0000","last_unstale":"2023-11-28T11:21:00.288230+0000","last_undegraded":"2023-11-28T11:21:00.288230+0000","last_fullsized":"2023-11-28T11:21:00.288230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:09:40.747993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.291211+0000","last_change":"2023-11-28T11:20:56.268222+0000","last_active":"2023-11-28T11:21:00.291211+0000","last_peered":"2023-11-28T11:21:00.291211+0000","last_clean":"2023-11-28T11:21:00.291211+0000","last_became_active":"2023-11-28T11:20:56.267973+0000","last_became_peered":"2023-11-28T11:20:56.267973+0000","last_unstale":"2023-11-28T11:21:00.291211+0000","last_undegraded":"2023-11-28T11:21:00.291211+0000","last_fullsized":"2023-11-28T11:21:00.291211+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:40:42.059656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.383115+0000","last_change":"2023-11-28T11:21:00.291222+0000","last_active":"2023-11-28T11:21:01.383115+0000","last_peered":"2023-11-28T11:21:01.383115+0000","last_clean":"2023-11-28T11:21:01.383115+0000","last_became_active":"2023-11-28T11:21:00.290768+0000","last_became_peered":"2023-11-28T11:21:00.290768+0000","last_unstale":"2023-11-28T11:21:01.383115+0000","last_undegraded":"2023-11-28T11:21:01.383115+0000","last_fullsized":"2023-11-28T11:21:01.383115+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:56:39.135021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.286149+0000","last_change":"2023-11-28T11:20:55.766427+0000","last_active":"2023-11-28T11:21:00.286149+0000","last_peered":"2023-11-28T11:21:00.286149+0000","last_clean":"2023-11-28T11:21:00.286149+0000","last_became_active":"2023-11-28T11:20:55.764723+0000","last_became_peered":"2023-11-28T11:20:55.764723+0000","last_unstale":"2023-11-28T11:21:00.286149+0000","last_undegraded":"2023-11-28T11:21:00.286149+0000","last_fullsized":"2023-11-28T11:21:00.286149+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:54:56.683286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.272205+0000","last_change":"2023-11-28T11:20:56.274667+0000","last_active":"2023-11-28T11:21:00.272205+0000","last_peered":"2023-11-28T11:21:00.272205+0000","last_clean":"2023-11-28T11:21:00.272205+0000","last_became_active":"2023-11-28T11:20:56.274325+0000","last_became_peered":"2023-11-28T11:20:56.274325+0000","last_unstale":"2023-11-28T11:21:00.272205+0000","last_undegraded":"2023-11-28T11:21:00.272205+0000","last_fullsized":"2023-11-28T11:21:00.272205+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:33:54.639487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.807008+0000","last_change":"2023-11-28T11:21:00.301042+0000","last_active":"2023-11-28T11:21:01.807008+0000","last_peered":"2023-11-28T11:21:01.807008+0000","last_clean":"2023-11-28T11:21:01.807008+0000","last_became_active":"2023-11-28T11:21:00.300581+0000","last_became_peered":"2023-11-28T11:21:00.300581+0000","last_unstale":"2023-11-28T11:21:01.807008+0000","last_undegraded":"2023-11-28T11:21:01.807008+0000","last_fullsized":"2023-11-28T11:21:01.807008+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:21:12.545633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.298437+0000","last_change":"2023-11-28T11:20:55.774510+0000","last_active":"2023-11-28T11:21:00.298437+0000","last_peered":"2023-11-28T11:21:00.298437+0000","last_clean":"2023-11-28T11:21:00.298437+0000","last_became_active":"2023-11-28T11:20:55.774211+0000","last_became_peered":"2023-11-28T11:20:55.774211+0000","last_unstale":"2023-11-28T11:21:00.298437+0000","last_undegraded":"2023-11-28T11:21:00.298437+0000","last_fullsized":"2023-11-28T11:21:00.298437+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:47:23.879025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.273372+0000","last_change":"2023-11-28T11:20:56.263432+0000","last_active":"2023-11-28T11:21:00.273372+0000","last_peered":"2023-11-28T11:21:00.273372+0000","last_clean":"2023-11-28T11:21:00.273372+0000","last_became_active":"2023-11-28T11:20:56.263125+0000","last_became_peered":"2023-11-28T11:20:56.263125+0000","last_unstale":"2023-11-28T11:21:00.273372+0000","last_undegraded":"2023-11-28T11:21:00.273372+0000","last_fullsized":"2023-11-28T11:21:00.273372+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:40:42.059656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.774944+0000","last_change":"2023-11-28T11:21:00.291421+0000","last_active":"2023-11-28T11:21:00.774944+0000","last_peered":"2023-11-28T11:21:00.774944+0000","last_clean":"2023-11-28T11:21:00.774944+0000","last_became_active":"2023-11-28T11:21:00.291077+0000","last_became_peered":"2023-11-28T11:21:00.291077+0000","last_unstale":"2023-11-28T11:21:00.774944+0000","last_undegraded":"2023-11-28T11:21:00.774944+0000","last_fullsized":"2023-11-28T11:21:00.774944+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:26:35.867941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.298743+0000","last_change":"2023-11-28T11:20:55.774653+0000","last_active":"2023-11-28T11:21:00.298743+0000","last_peered":"2023-11-28T11:21:00.298743+0000","last_clean":"2023-11-28T11:21:00.298743+0000","last_became_active":"2023-11-28T11:20:55.774352+0000","last_became_peered":"2023-11-28T11:20:55.774352+0000","last_unstale":"2023-11-28T11:21:00.298743+0000","last_undegraded":"2023-11-28T11:21:00.298743+0000","last_fullsized":"2023-11-28T11:21:00.298743+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:33:55.577608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.274658+0000","last_change":"2023-11-28T11:20:56.266409+0000","last_active":"2023-11-28T11:21:00.274658+0000","last_peered":"2023-11-28T11:21:00.274658+0000","last_clean":"2023-11-28T11:21:00.274658+0000","last_became_active":"2023-11-28T11:20:56.266091+0000","last_became_peered":"2023-11-28T11:20:56.266091+0000","last_unstale":"2023-11-28T11:21:00.274658+0000","last_undegraded":"2023-11-28T11:21:00.274658+0000","last_fullsized":"2023-11-28T11:21:00.274658+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:28:30.120683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.775175+0000","last_change":"2023-11-28T11:21:00.311439+0000","last_active":"2023-11-28T11:21:00.775175+0000","last_peered":"2023-11-28T11:21:00.775175+0000","last_clean":"2023-11-28T11:21:00.775175+0000","last_became_active":"2023-11-28T11:21:00.308732+0000","last_became_peered":"2023-11-28T11:21:00.308732+0000","last_unstale":"2023-11-28T11:21:00.775175+0000","last_undegraded":"2023-11-28T11:21:00.775175+0000","last_fullsized":"2023-11-28T11:21:00.775175+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:53:19.245981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.298398+0000","last_change":"2023-11-28T11:20:55.771105+0000","last_active":"2023-11-28T11:21:00.298398+0000","last_peered":"2023-11-28T11:21:00.298398+0000","last_clean":"2023-11-28T11:21:00.298398+0000","last_became_active":"2023-11-28T11:20:55.770583+0000","last_became_peered":"2023-11-28T11:20:55.770583+0000","last_unstale":"2023-11-28T11:21:00.298398+0000","last_undegraded":"2023-11-28T11:21:00.298398+0000","last_fullsized":"2023-11-28T11:21:00.298398+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:06:01.854141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.286992+0000","last_change":"2023-11-28T11:20:56.263719+0000","last_active":"2023-11-28T11:21:00.286992+0000","last_peered":"2023-11-28T11:21:00.286992+0000","last_clean":"2023-11-28T11:21:00.286992+0000","last_became_active":"2023-11-28T11:20:56.263444+0000","last_became_peered":"2023-11-28T11:20:56.263444+0000","last_unstale":"2023-11-28T11:21:00.286992+0000","last_undegraded":"2023-11-28T11:21:00.286992+0000","last_fullsized":"2023-11-28T11:21:00.286992+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:33:56.583532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.775522+0000","last_change":"2023-11-28T11:21:00.294534+0000","last_active":"2023-11-28T11:21:00.775522+0000","last_peered":"2023-11-28T11:21:00.775522+0000","last_clean":"2023-11-28T11:21:00.775522+0000","last_became_active":"2023-11-28T11:21:00.294055+0000","last_became_peered":"2023-11-28T11:21:00.294055+0000","last_unstale":"2023-11-28T11:21:00.775522+0000","last_undegraded":"2023-11-28T11:21:00.775522+0000","last_fullsized":"2023-11-28T11:21:00.775522+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:21:24.577104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.275166+0000","last_change":"2023-11-28T11:20:55.763501+0000","last_active":"2023-11-28T11:21:00.275166+0000","last_peered":"2023-11-28T11:21:00.275166+0000","last_clean":"2023-11-28T11:21:00.275166+0000","last_became_active":"2023-11-28T11:20:55.763188+0000","last_became_peered":"2023-11-28T11:20:55.763188+0000","last_unstale":"2023-11-28T11:21:00.275166+0000","last_undegraded":"2023-11-28T11:21:00.275166+0000","last_fullsized":"2023-11-28T11:21:00.275166+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:58:46.726052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.290571+0000","last_change":"2023-11-28T11:20:56.260904+0000","last_active":"2023-11-28T11:21:00.290571+0000","last_peered":"2023-11-28T11:21:00.290571+0000","last_clean":"2023-11-28T11:21:00.290571+0000","last_became_active":"2023-11-28T11:20:56.260584+0000","last_became_peered":"2023-11-28T11:20:56.260584+0000","last_unstale":"2023-11-28T11:21:00.290571+0000","last_undegraded":"2023-11-28T11:21:00.290571+0000","last_fullsized":"2023-11-28T11:21:00.290571+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:08.079216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.775218+0000","last_change":"2023-11-28T11:21:00.299500+0000","last_active":"2023-11-28T11:21:00.775218+0000","last_peered":"2023-11-28T11:21:00.775218+0000","last_clean":"2023-11-28T11:21:00.775218+0000","last_became_active":"2023-11-28T11:21:00.299065+0000","last_became_peered":"2023-11-28T11:21:00.299065+0000","last_unstale":"2023-11-28T11:21:00.775218+0000","last_undegraded":"2023-11-28T11:21:00.775218+0000","last_fullsized":"2023-11-28T11:21:00.775218+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:59:57.725780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.271583+0000","last_change":"2023-11-28T11:20:55.767855+0000","last_active":"2023-11-28T11:21:00.271583+0000","last_peered":"2023-11-28T11:21:00.271583+0000","last_clean":"2023-11-28T11:21:00.271583+0000","last_became_active":"2023-11-28T11:20:55.767418+0000","last_became_peered":"2023-11-28T11:20:55.767418+0000","last_unstale":"2023-11-28T11:21:00.271583+0000","last_undegraded":"2023-11-28T11:21:00.271583+0000","last_fullsized":"2023-11-28T11:21:00.271583+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:06:11.286885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.275680+0000","last_change":"2023-11-28T11:20:56.282762+0000","last_active":"2023-11-28T11:21:00.275680+0000","last_peered":"2023-11-28T11:21:00.275680+0000","last_clean":"2023-11-28T11:21:00.275680+0000","last_became_active":"2023-11-28T11:20:56.280729+0000","last_became_peered":"2023-11-28T11:20:56.280729+0000","last_unstale":"2023-11-28T11:21:00.275680+0000","last_undegraded":"2023-11-28T11:21:00.275680+0000","last_fullsized":"2023-11-28T11:21:00.275680+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:53:15.721370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.883819+0000","last_change":"2023-11-28T11:21:00.305663+0000","last_active":"2023-11-28T11:21:01.883819+0000","last_peered":"2023-11-28T11:21:01.883819+0000","last_clean":"2023-11-28T11:21:01.883819+0000","last_became_active":"2023-11-28T11:21:00.305308+0000","last_became_peered":"2023-11-28T11:21:00.305308+0000","last_unstale":"2023-11-28T11:21:01.883819+0000","last_undegraded":"2023-11-28T11:21:01.883819+0000","last_fullsized":"2023-11-28T11:21:01.883819+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:05:27.287399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.287091+0000","last_change":"2023-11-28T11:20:55.774271+0000","last_active":"2023-11-28T11:21:00.287091+0000","last_peered":"2023-11-28T11:21:00.287091+0000","last_clean":"2023-11-28T11:21:00.287091+0000","last_became_active":"2023-11-28T11:20:55.773041+0000","last_became_peered":"2023-11-28T11:20:55.773041+0000","last_unstale":"2023-11-28T11:21:00.287091+0000","last_undegraded":"2023-11-28T11:21:00.287091+0000","last_fullsized":"2023-11-28T11:21:00.287091+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:06:11.286885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.273977+0000","last_change":"2023-11-28T11:20:56.269860+0000","last_active":"2023-11-28T11:21:00.273977+0000","last_peered":"2023-11-28T11:21:00.273977+0000","last_clean":"2023-11-28T11:21:00.273977+0000","last_became_active":"2023-11-28T11:20:56.269355+0000","last_became_peered":"2023-11-28T11:20:56.269355+0000","last_unstale":"2023-11-28T11:21:00.273977+0000","last_undegraded":"2023-11-28T11:21:00.273977+0000","last_fullsized":"2023-11-28T11:21:00.273977+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:24:07.384730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.383366+0000","last_change":"2023-11-28T11:21:00.297532+0000","last_active":"2023-11-28T11:21:01.383366+0000","last_peered":"2023-11-28T11:21:01.383366+0000","last_clean":"2023-11-28T11:21:01.383366+0000","last_became_active":"2023-11-28T11:21:00.295523+0000","last_became_peered":"2023-11-28T11:21:00.295523+0000","last_unstale":"2023-11-28T11:21:01.383366+0000","last_undegraded":"2023-11-28T11:21:01.383366+0000","last_fullsized":"2023-11-28T11:21:01.383366+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:41.700110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.274453+0000","last_change":"2023-11-28T11:20:55.764095+0000","last_active":"2023-11-28T11:21:00.274453+0000","last_peered":"2023-11-28T11:21:00.274453+0000","last_clean":"2023-11-28T11:21:00.274453+0000","last_became_active":"2023-11-28T11:20:55.762342+0000","last_became_peered":"2023-11-28T11:20:55.762342+0000","last_unstale":"2023-11-28T11:21:00.274453+0000","last_undegraded":"2023-11-28T11:21:00.274453+0000","last_fullsized":"2023-11-28T11:21:00.274453+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:25:25.914486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.274835+0000","last_change":"2023-11-28T11:20:56.268448+0000","last_active":"2023-11-28T11:21:00.274835+0000","last_peered":"2023-11-28T11:21:00.274835+0000","last_clean":"2023-11-28T11:21:00.274835+0000","last_became_active":"2023-11-28T11:20:56.268150+0000","last_became_peered":"2023-11-28T11:20:56.268150+0000","last_unstale":"2023-11-28T11:21:00.274835+0000","last_undegraded":"2023-11-28T11:21:00.274835+0000","last_fullsized":"2023-11-28T11:21:00.274835+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:21:21.052493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.777908+0000","last_change":"2023-11-28T11:21:00.306271+0000","last_active":"2023-11-28T11:21:00.777908+0000","last_peered":"2023-11-28T11:21:00.777908+0000","last_clean":"2023-11-28T11:21:00.777908+0000","last_became_active":"2023-11-28T11:21:00.305835+0000","last_became_peered":"2023-11-28T11:21:00.305835+0000","last_unstale":"2023-11-28T11:21:00.777908+0000","last_undegraded":"2023-11-28T11:21:00.777908+0000","last_fullsized":"2023-11-28T11:21:00.777908+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:04:07.374394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.270977+0000","last_change":"2023-11-28T11:20:55.762830+0000","last_active":"2023-11-28T11:21:00.270977+0000","last_peered":"2023-11-28T11:21:00.270977+0000","last_clean":"2023-11-28T11:21:00.270977+0000","last_became_active":"2023-11-28T11:20:55.760841+0000","last_became_peered":"2023-11-28T11:20:55.760841+0000","last_unstale":"2023-11-28T11:21:00.270977+0000","last_undegraded":"2023-11-28T11:21:00.270977+0000","last_fullsized":"2023-11-28T11:21:00.270977+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:54:56.683286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.290757+0000","last_change":"2023-11-28T11:20:56.271356+0000","last_active":"2023-11-28T11:21:00.290757+0000","last_peered":"2023-11-28T11:21:00.290757+0000","last_clean":"2023-11-28T11:21:00.290757+0000","last_became_active":"2023-11-28T11:20:56.271110+0000","last_became_peered":"2023-11-28T11:20:56.271110+0000","last_unstale":"2023-11-28T11:21:00.290757+0000","last_undegraded":"2023-11-28T11:21:00.290757+0000","last_fullsized":"2023-11-28T11:21:00.290757+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:08.079216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.884172+0000","last_change":"2023-11-28T11:21:00.295227+0000","last_active":"2023-11-28T11:21:01.884172+0000","last_peered":"2023-11-28T11:21:01.884172+0000","last_clean":"2023-11-28T11:21:01.884172+0000","last_became_active":"2023-11-28T11:21:00.293463+0000","last_became_peered":"2023-11-28T11:21:00.293463+0000","last_unstale":"2023-11-28T11:21:01.884172+0000","last_undegraded":"2023-11-28T11:21:01.884172+0000","last_fullsized":"2023-11-28T11:21:01.884172+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:41.700110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.288397+0000","last_change":"2023-11-28T11:20:51.020016+0000","last_active":"2023-11-28T11:21:00.288397+0000","last_peered":"2023-11-28T11:21:00.288397+0000","last_clean":"2023-11-28T11:21:00.288397+0000","last_became_active":"2023-11-28T11:20:51.019492+0000","last_became_peered":"2023-11-28T11:20:51.019492+0000","last_unstale":"2023-11-28T11:21:00.288397+0000","last_undegraded":"2023-11-28T11:21:00.288397+0000","last_fullsized":"2023-11-28T11:21:00.288397+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:27.884538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.273192+0000","last_change":"2023-11-28T11:20:55.770921+0000","last_active":"2023-11-28T11:21:00.273192+0000","last_peered":"2023-11-28T11:21:00.273192+0000","last_clean":"2023-11-28T11:21:00.273192+0000","last_became_active":"2023-11-28T11:20:55.770693+0000","last_became_peered":"2023-11-28T11:20:55.770693+0000","last_unstale":"2023-11-28T11:21:00.273192+0000","last_undegraded":"2023-11-28T11:21:00.273192+0000","last_fullsized":"2023-11-28T11:21:00.273192+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:37:05.627049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.285261+0000","last_change":"2023-11-28T11:20:56.257635+0000","last_active":"2023-11-28T11:21:00.285261+0000","last_peered":"2023-11-28T11:21:00.285261+0000","last_clean":"2023-11-28T11:21:00.285261+0000","last_became_active":"2023-11-28T11:20:56.255047+0000","last_became_peered":"2023-11-28T11:20:56.255047+0000","last_unstale":"2023-11-28T11:21:00.285261+0000","last_undegraded":"2023-11-28T11:21:00.285261+0000","last_fullsized":"2023-11-28T11:21:00.285261+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:27:12.297507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.884415+0000","last_change":"2023-11-28T11:21:00.311287+0000","last_active":"2023-11-28T11:21:01.884415+0000","last_peered":"2023-11-28T11:21:01.884415+0000","last_clean":"2023-11-28T11:21:01.884415+0000","last_became_active":"2023-11-28T11:21:00.305808+0000","last_became_peered":"2023-11-28T11:21:00.305808+0000","last_unstale":"2023-11-28T11:21:01.884415+0000","last_undegraded":"2023-11-28T11:21:01.884415+0000","last_fullsized":"2023-11-28T11:21:01.884415+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:56:39.135021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.276695+0000","last_change":"2023-11-28T11:20:50.521908+0000","last_active":"2023-11-28T11:21:00.276695+0000","last_peered":"2023-11-28T11:21:00.276695+0000","last_clean":"2023-11-28T11:21:00.276695+0000","last_became_active":"2023-11-28T11:20:50.521059+0000","last_became_peered":"2023-11-28T11:20:50.521059+0000","last_unstale":"2023-11-28T11:21:00.276695+0000","last_undegraded":"2023-11-28T11:21:00.276695+0000","last_fullsized":"2023-11-28T11:21:00.276695+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:05.302498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.273078+0000","last_change":"2023-11-28T11:20:55.767339+0000","last_active":"2023-11-28T11:21:00.273078+0000","last_peered":"2023-11-28T11:21:00.273078+0000","last_clean":"2023-11-28T11:21:00.273078+0000","last_became_active":"2023-11-28T11:20:55.766065+0000","last_became_peered":"2023-11-28T11:20:55.766065+0000","last_unstale":"2023-11-28T11:21:00.273078+0000","last_undegraded":"2023-11-28T11:21:00.273078+0000","last_fullsized":"2023-11-28T11:21:00.273078+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:06:11.286885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.289236+0000","last_change":"2023-11-28T11:20:56.261611+0000","last_active":"2023-11-28T11:21:00.289236+0000","last_peered":"2023-11-28T11:21:00.289236+0000","last_clean":"2023-11-28T11:21:00.289236+0000","last_became_active":"2023-11-28T11:20:56.260557+0000","last_became_peered":"2023-11-28T11:20:56.260557+0000","last_unstale":"2023-11-28T11:21:00.289236+0000","last_undegraded":"2023-11-28T11:21:00.289236+0000","last_fullsized":"2023-11-28T11:21:00.289236+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:14:34.876987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.775674+0000","last_change":"2023-11-28T11:21:00.301087+0000","last_active":"2023-11-28T11:21:00.775674+0000","last_peered":"2023-11-28T11:21:00.775674+0000","last_clean":"2023-11-28T11:21:00.775674+0000","last_became_active":"2023-11-28T11:21:00.300674+0000","last_became_peered":"2023-11-28T11:21:00.300674+0000","last_unstale":"2023-11-28T11:21:00.775674+0000","last_undegraded":"2023-11-28T11:21:00.775674+0000","last_fullsized":"2023-11-28T11:21:00.775674+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:13:50.207611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.282488+0000","last_change":"2023-11-28T11:20:50.619004+0000","last_active":"2023-11-28T11:21:00.282488+0000","last_peered":"2023-11-28T11:21:00.282488+0000","last_clean":"2023-11-28T11:21:00.282488+0000","last_became_active":"2023-11-28T11:20:50.618315+0000","last_became_peered":"2023-11-28T11:20:50.618315+0000","last_unstale":"2023-11-28T11:21:00.282488+0000","last_undegraded":"2023-11-28T11:21:00.282488+0000","last_fullsized":"2023-11-28T11:21:00.282488+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:27.884538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.283998+0000","last_change":"2023-11-28T11:20:55.758384+0000","last_active":"2023-11-28T11:21:00.283998+0000","last_peered":"2023-11-28T11:21:00.283998+0000","last_clean":"2023-11-28T11:21:00.283998+0000","last_became_active":"2023-11-28T11:20:55.758125+0000","last_became_peered":"2023-11-28T11:20:55.758125+0000","last_unstale":"2023-11-28T11:21:00.283998+0000","last_undegraded":"2023-11-28T11:21:00.283998+0000","last_fullsized":"2023-11-28T11:21:00.283998+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:47:38.102588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.290667+0000","last_change":"2023-11-28T11:20:56.253940+0000","last_active":"2023-11-28T11:21:00.290667+0000","last_peered":"2023-11-28T11:21:00.290667+0000","last_clean":"2023-11-28T11:21:00.290667+0000","last_became_active":"2023-11-28T11:20:56.253635+0000","last_became_peered":"2023-11-28T11:20:56.253635+0000","last_unstale":"2023-11-28T11:21:00.290667+0000","last_undegraded":"2023-11-28T11:21:00.290667+0000","last_fullsized":"2023-11-28T11:21:00.290667+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:00:17.958443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.806956+0000","last_change":"2023-11-28T11:21:00.311172+0000","last_active":"2023-11-28T11:21:01.806956+0000","last_peered":"2023-11-28T11:21:01.806956+0000","last_clean":"2023-11-28T11:21:01.806956+0000","last_became_active":"2023-11-28T11:21:00.310707+0000","last_became_peered":"2023-11-28T11:21:00.310707+0000","last_unstale":"2023-11-28T11:21:01.806956+0000","last_undegraded":"2023-11-28T11:21:01.806956+0000","last_fullsized":"2023-11-28T11:21:01.806956+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:41:46.264594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.273954+0000","last_change":"2023-11-28T11:20:50.522582+0000","last_active":"2023-11-28T11:21:00.273954+0000","last_peered":"2023-11-28T11:21:00.273954+0000","last_clean":"2023-11-28T11:21:00.273954+0000","last_became_active":"2023-11-28T11:20:50.522157+0000","last_became_peered":"2023-11-28T11:20:50.522157+0000","last_unstale":"2023-11-28T11:21:00.273954+0000","last_undegraded":"2023-11-28T11:21:00.273954+0000","last_fullsized":"2023-11-28T11:21:00.273954+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:27.884538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.286501+0000","last_change":"2023-11-28T11:20:43.294845+0000","last_active":"2023-11-28T11:21:00.286501+0000","last_peered":"2023-11-28T11:21:00.286501+0000","last_clean":"2023-11-28T11:21:00.286501+0000","last_became_active":"2023-11-28T11:20:43.294483+0000","last_became_peered":"2023-11-28T11:20:43.294483+0000","last_unstale":"2023-11-28T11:21:00.286501+0000","last_undegraded":"2023-11-28T11:21:00.286501+0000","last_fullsized":"2023-11-28T11:21:00.286501+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:42.270512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:42.270512+0000","last_clean_scrub_stamp":"2023-11-28T11:20:42.270512+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:17:03.376680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.270966+0000","last_change":"2023-11-28T11:20:55.767420+0000","last_active":"2023-11-28T11:21:00.270966+0000","last_peered":"2023-11-28T11:21:00.270966+0000","last_clean":"2023-11-28T11:21:00.270966+0000","last_became_active":"2023-11-28T11:20:55.766809+0000","last_became_peered":"2023-11-28T11:20:55.766809+0000","last_unstale":"2023-11-28T11:21:00.270966+0000","last_undegraded":"2023-11-28T11:21:00.270966+0000","last_fullsized":"2023-11-28T11:21:00.270966+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:37:05.627049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.286972+0000","last_change":"2023-11-28T11:20:56.276228+0000","last_active":"2023-11-28T11:21:00.286972+0000","last_peered":"2023-11-28T11:21:00.286972+0000","last_clean":"2023-11-28T11:21:00.286972+0000","last_became_active":"2023-11-28T11:20:56.275905+0000","last_became_peered":"2023-11-28T11:20:56.275905+0000","last_unstale":"2023-11-28T11:21:00.286972+0000","last_undegraded":"2023-11-28T11:21:00.286972+0000","last_fullsized":"2023-11-28T11:21:00.286972+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:47:24.884949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.383236+0000","last_change":"2023-11-28T11:21:00.305959+0000","last_active":"2023-11-28T11:21:01.383236+0000","last_peered":"2023-11-28T11:21:01.383236+0000","last_clean":"2023-11-28T11:21:01.383236+0000","last_became_active":"2023-11-28T11:21:00.305530+0000","last_became_peered":"2023-11-28T11:21:00.305530+0000","last_unstale":"2023-11-28T11:21:01.383236+0000","last_undegraded":"2023-11-28T11:21:01.383236+0000","last_fullsized":"2023-11-28T11:21:01.383236+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:22:04.224877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.286683+0000","last_change":"2023-11-28T11:20:50.526865+0000","last_active":"2023-11-28T11:21:00.286683+0000","last_peered":"2023-11-28T11:21:00.286683+0000","last_clean":"2023-11-28T11:21:00.286683+0000","last_became_active":"2023-11-28T11:20:50.526576+0000","last_became_peered":"2023-11-28T11:20:50.526576+0000","last_unstale":"2023-11-28T11:21:00.286683+0000","last_undegraded":"2023-11-28T11:21:00.286683+0000","last_fullsized":"2023-11-28T11:21:00.286683+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:09:18.460112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.289849+0000","last_change":"2023-11-28T11:20:55.772865+0000","last_active":"2023-11-28T11:21:00.289849+0000","last_peered":"2023-11-28T11:21:00.289849+0000","last_clean":"2023-11-28T11:21:00.289849+0000","last_became_active":"2023-11-28T11:20:55.772559+0000","last_became_peered":"2023-11-28T11:20:55.772559+0000","last_unstale":"2023-11-28T11:21:00.289849+0000","last_undegraded":"2023-11-28T11:21:00.289849+0000","last_fullsized":"2023-11-28T11:21:00.289849+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:37:05.627049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.290171+0000","last_change":"2023-11-28T11:20:56.268570+0000","last_active":"2023-11-28T11:21:00.290171+0000","last_peered":"2023-11-28T11:21:00.290171+0000","last_clean":"2023-11-28T11:21:00.290171+0000","last_became_active":"2023-11-28T11:20:56.264598+0000","last_became_peered":"2023-11-28T11:20:56.264598+0000","last_unstale":"2023-11-28T11:21:00.290171+0000","last_undegraded":"2023-11-28T11:21:00.290171+0000","last_fullsized":"2023-11-28T11:21:00.290171+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:14:34.876987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.778849+0000","last_change":"2023-11-28T11:21:00.311861+0000","last_active":"2023-11-28T11:21:00.778849+0000","last_peered":"2023-11-28T11:21:00.778849+0000","last_clean":"2023-11-28T11:21:00.778849+0000","last_became_active":"2023-11-28T11:21:00.311394+0000","last_became_peered":"2023-11-28T11:21:00.311394+0000","last_unstale":"2023-11-28T11:21:00.778849+0000","last_undegraded":"2023-11-28T11:21:00.778849+0000","last_fullsized":"2023-11-28T11:21:00.778849+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:25:55.767021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.298282+0000","last_change":"2023-11-28T11:20:51.020043+0000","last_active":"2023-11-28T11:21:00.298282+0000","last_peered":"2023-11-28T11:21:00.298282+0000","last_clean":"2023-11-28T11:21:00.298282+0000","last_became_active":"2023-11-28T11:20:51.019702+0000","last_became_peered":"2023-11-28T11:20:51.019702+0000","last_unstale":"2023-11-28T11:21:00.298282+0000","last_undegraded":"2023-11-28T11:21:00.298282+0000","last_fullsized":"2023-11-28T11:21:00.298282+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:26:20.246873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.288664+0000","last_change":"2023-11-28T11:20:55.760221+0000","last_active":"2023-11-28T11:21:00.288664+0000","last_peered":"2023-11-28T11:21:00.288664+0000","last_clean":"2023-11-28T11:21:00.288664+0000","last_became_active":"2023-11-28T11:20:55.759960+0000","last_became_peered":"2023-11-28T11:20:55.759960+0000","last_unstale":"2023-11-28T11:21:00.288664+0000","last_undegraded":"2023-11-28T11:21:00.288664+0000","last_fullsized":"2023-11-28T11:21:00.288664+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:54:56.683286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.290072+0000","last_change":"2023-11-28T11:20:56.251820+0000","last_active":"2023-11-28T11:21:00.290072+0000","last_peered":"2023-11-28T11:21:00.290072+0000","last_clean":"2023-11-28T11:21:00.290072+0000","last_became_active":"2023-11-28T11:20:56.251550+0000","last_became_peered":"2023-11-28T11:20:56.251550+0000","last_unstale":"2023-11-28T11:21:00.290072+0000","last_undegraded":"2023-11-28T11:21:00.290072+0000","last_fullsized":"2023-11-28T11:21:00.290072+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:45:14.478416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.777012+0000","last_change":"2023-11-28T11:21:00.298541+0000","last_active":"2023-11-28T11:21:00.777012+0000","last_peered":"2023-11-28T11:21:00.777012+0000","last_clean":"2023-11-28T11:21:00.777012+0000","last_became_active":"2023-11-28T11:21:00.298160+0000","last_became_peered":"2023-11-28T11:21:00.298160+0000","last_unstale":"2023-11-28T11:21:00.777012+0000","last_undegraded":"2023-11-28T11:21:00.777012+0000","last_fullsized":"2023-11-28T11:21:00.777012+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:16:48.936607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.272276+0000","last_change":"2023-11-28T11:20:55.766385+0000","last_active":"2023-11-28T11:21:00.272276+0000","last_peered":"2023-11-28T11:21:00.272276+0000","last_clean":"2023-11-28T11:21:00.272276+0000","last_became_active":"2023-11-28T11:20:55.766148+0000","last_became_peered":"2023-11-28T11:20:55.766148+0000","last_unstale":"2023-11-28T11:21:00.272276+0000","last_undegraded":"2023-11-28T11:21:00.272276+0000","last_fullsized":"2023-11-28T11:21:00.272276+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:33:02.765966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.288602+0000","last_change":"2023-11-28T11:20:50.524251+0000","last_active":"2023-11-28T11:21:00.288602+0000","last_peered":"2023-11-28T11:21:00.288602+0000","last_clean":"2023-11-28T11:21:00.288602+0000","last_became_active":"2023-11-28T11:20:50.524012+0000","last_became_peered":"2023-11-28T11:20:50.524012+0000","last_unstale":"2023-11-28T11:21:00.288602+0000","last_undegraded":"2023-11-28T11:21:00.288602+0000","last_fullsized":"2023-11-28T11:21:00.288602+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:05.302498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.270885+0000","last_change":"2023-11-28T11:20:56.266167+0000","last_active":"2023-11-28T11:21:00.270885+0000","last_peered":"2023-11-28T11:21:00.270885+0000","last_clean":"2023-11-28T11:21:00.270885+0000","last_became_active":"2023-11-28T11:20:56.264890+0000","last_became_peered":"2023-11-28T11:20:56.264890+0000","last_unstale":"2023-11-28T11:21:00.270885+0000","last_undegraded":"2023-11-28T11:21:00.270885+0000","last_fullsized":"2023-11-28T11:21:00.270885+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:31:26.827850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.843672+0000","last_change":"2023-11-28T11:21:00.295193+0000","last_active":"2023-11-28T11:21:01.843672+0000","last_peered":"2023-11-28T11:21:01.843672+0000","last_clean":"2023-11-28T11:21:01.843672+0000","last_became_active":"2023-11-28T11:21:00.294668+0000","last_became_peered":"2023-11-28T11:21:00.294668+0000","last_unstale":"2023-11-28T11:21:01.843672+0000","last_undegraded":"2023-11-28T11:21:01.843672+0000","last_fullsized":"2023-11-28T11:21:01.843672+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:53:58.267145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.287437+0000","last_change":"2023-11-28T11:20:50.524838+0000","last_active":"2023-11-28T11:21:00.287437+0000","last_peered":"2023-11-28T11:21:00.287437+0000","last_clean":"2023-11-28T11:21:00.287437+0000","last_became_active":"2023-11-28T11:20:50.524402+0000","last_became_peered":"2023-11-28T11:20:50.524402+0000","last_unstale":"2023-11-28T11:21:00.287437+0000","last_undegraded":"2023-11-28T11:21:00.287437+0000","last_fullsized":"2023-11-28T11:21:00.287437+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:22.699447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"26'1","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.725299+0000","last_change":"2023-11-28T11:20:55.765528+0000","last_active":"2023-11-28T11:21:00.725299+0000","last_peered":"2023-11-28T11:21:00.725299+0000","last_clean":"2023-11-28T11:21:00.725299+0000","last_became_active":"2023-11-28T11:20:55.765244+0000","last_became_peered":"2023-11-28T11:20:55.765244+0000","last_unstale":"2023-11-28T11:21:00.725299+0000","last_undegraded":"2023-11-28T11:21:00.725299+0000","last_fullsized":"2023-11-28T11:21:00.725299+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:47:38.102588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.274546+0000","last_change":"2023-11-28T11:20:56.275241+0000","last_active":"2023-11-28T11:21:00.274546+0000","last_peered":"2023-11-28T11:21:00.274546+0000","last_clean":"2023-11-28T11:21:00.274546+0000","last_became_active":"2023-11-28T11:20:56.271190+0000","last_became_peered":"2023-11-28T11:20:56.271190+0000","last_unstale":"2023-11-28T11:21:00.274546+0000","last_undegraded":"2023-11-28T11:21:00.274546+0000","last_fullsized":"2023-11-28T11:21:00.274546+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:45:14.478416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.883997+0000","last_change":"2023-11-28T11:21:00.297029+0000","last_active":"2023-11-28T11:21:01.883997+0000","last_peered":"2023-11-28T11:21:01.883997+0000","last_clean":"2023-11-28T11:21:01.883997+0000","last_became_active":"2023-11-28T11:21:00.296497+0000","last_became_peered":"2023-11-28T11:21:00.296497+0000","last_unstale":"2023-11-28T11:21:01.883997+0000","last_undegraded":"2023-11-28T11:21:01.883997+0000","last_fullsized":"2023-11-28T11:21:01.883997+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:41.700110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.290180+0000","last_change":"2023-11-28T11:20:55.773378+0000","last_active":"2023-11-28T11:21:00.290180+0000","last_peered":"2023-11-28T11:21:00.290180+0000","last_clean":"2023-11-28T11:21:00.290180+0000","last_became_active":"2023-11-28T11:20:55.773095+0000","last_became_peered":"2023-11-28T11:20:55.773095+0000","last_unstale":"2023-11-28T11:21:00.290180+0000","last_undegraded":"2023-11-28T11:21:00.290180+0000","last_fullsized":"2023-11-28T11:21:00.290180+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:54:56.683286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.280606+0000","last_change":"2023-11-28T11:20:56.258626+0000","last_active":"2023-11-28T11:21:00.280606+0000","last_peered":"2023-11-28T11:21:00.280606+0000","last_clean":"2023-11-28T11:21:00.280606+0000","last_became_active":"2023-11-28T11:20:56.258350+0000","last_became_peered":"2023-11-28T11:20:56.258350+0000","last_unstale":"2023-11-28T11:21:00.280606+0000","last_undegraded":"2023-11-28T11:21:00.280606+0000","last_fullsized":"2023-11-28T11:21:00.280606+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:33:03.771889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.806818+0000","last_change":"2023-11-28T11:21:00.306341+0000","last_active":"2023-11-28T11:21:01.806818+0000","last_peered":"2023-11-28T11:21:01.806818+0000","last_clean":"2023-11-28T11:21:01.806818+0000","last_became_active":"2023-11-28T11:21:00.304545+0000","last_became_peered":"2023-11-28T11:21:00.304545+0000","last_unstale":"2023-11-28T11:21:01.806818+0000","last_undegraded":"2023-11-28T11:21:01.806818+0000","last_fullsized":"2023-11-28T11:21:01.806818+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:53:58.267145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.275266+0000","last_change":"2023-11-28T11:20:55.766530+0000","last_active":"2023-11-28T11:21:00.275266+0000","last_peered":"2023-11-28T11:21:00.275266+0000","last_clean":"2023-11-28T11:21:00.275266+0000","last_became_active":"2023-11-28T11:20:55.766205+0000","last_became_peered":"2023-11-28T11:20:55.766205+0000","last_unstale":"2023-11-28T11:21:00.275266+0000","last_undegraded":"2023-11-28T11:21:00.275266+0000","last_fullsized":"2023-11-28T11:21:00.275266+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:33:02.765966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.273248+0000","last_change":"2023-11-28T11:20:56.279673+0000","last_active":"2023-11-28T11:21:00.273248+0000","last_peered":"2023-11-28T11:21:00.273248+0000","last_clean":"2023-11-28T11:21:00.273248+0000","last_became_active":"2023-11-28T11:20:56.276338+0000","last_became_peered":"2023-11-28T11:20:56.276338+0000","last_unstale":"2023-11-28T11:21:00.273248+0000","last_undegraded":"2023-11-28T11:21:00.273248+0000","last_fullsized":"2023-11-28T11:21:00.273248+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:59:54.201169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.382912+0000","last_change":"2023-11-28T11:21:00.308516+0000","last_active":"2023-11-28T11:21:01.382912+0000","last_peered":"2023-11-28T11:21:01.382912+0000","last_clean":"2023-11-28T11:21:01.382912+0000","last_became_active":"2023-11-28T11:21:00.307909+0000","last_became_peered":"2023-11-28T11:21:00.307909+0000","last_unstale":"2023-11-28T11:21:01.382912+0000","last_undegraded":"2023-11-28T11:21:01.382912+0000","last_fullsized":"2023-11-28T11:21:01.382912+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:53:42.920160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.273087+0000","last_change":"2023-11-28T11:20:55.772607+0000","last_active":"2023-11-28T11:21:00.273087+0000","last_peered":"2023-11-28T11:21:00.273087+0000","last_clean":"2023-11-28T11:21:00.273087+0000","last_became_active":"2023-11-28T11:20:55.772335+0000","last_became_peered":"2023-11-28T11:20:55.772335+0000","last_unstale":"2023-11-28T11:21:00.273087+0000","last_undegraded":"2023-11-28T11:21:00.273087+0000","last_fullsized":"2023-11-28T11:21:00.273087+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:23:36.801918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.274556+0000","last_change":"2023-11-28T11:20:56.279846+0000","last_active":"2023-11-28T11:21:00.274556+0000","last_peered":"2023-11-28T11:21:00.274556+0000","last_clean":"2023-11-28T11:21:00.274556+0000","last_became_active":"2023-11-28T11:20:56.276529+0000","last_became_peered":"2023-11-28T11:20:56.276529+0000","last_unstale":"2023-11-28T11:21:00.274556+0000","last_undegraded":"2023-11-28T11:21:00.274556+0000","last_fullsized":"2023-11-28T11:21:00.274556+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:29:57.585501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.776665+0000","last_change":"2023-11-28T11:21:00.294235+0000","last_active":"2023-11-28T11:21:00.776665+0000","last_peered":"2023-11-28T11:21:00.776665+0000","last_clean":"2023-11-28T11:21:00.776665+0000","last_became_active":"2023-11-28T11:21:00.293813+0000","last_became_peered":"2023-11-28T11:21:00.293813+0000","last_unstale":"2023-11-28T11:21:00.776665+0000","last_undegraded":"2023-11-28T11:21:00.776665+0000","last_fullsized":"2023-11-28T11:21:00.776665+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:44:23.280744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.271358+0000","last_change":"2023-11-28T11:20:55.767813+0000","last_active":"2023-11-28T11:21:00.271358+0000","last_peered":"2023-11-28T11:21:00.271358+0000","last_clean":"2023-11-28T11:21:00.271358+0000","last_became_active":"2023-11-28T11:20:55.765782+0000","last_became_peered":"2023-11-28T11:20:55.765782+0000","last_unstale":"2023-11-28T11:21:00.271358+0000","last_undegraded":"2023-11-28T11:21:00.271358+0000","last_fullsized":"2023-11-28T11:21:00.271358+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:47:38.102588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.271000+0000","last_change":"2023-11-28T11:20:56.265782+0000","last_active":"2023-11-28T11:21:00.271000+0000","last_peered":"2023-11-28T11:21:00.271000+0000","last_clean":"2023-11-28T11:21:00.271000+0000","last_became_active":"2023-11-28T11:20:56.265102+0000","last_became_peered":"2023-11-28T11:20:56.265102+0000","last_unstale":"2023-11-28T11:21:00.271000+0000","last_undegraded":"2023-11-28T11:21:00.271000+0000","last_fullsized":"2023-11-28T11:21:00.271000+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:45:14.478416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.883772+0000","last_change":"2023-11-28T11:21:00.297751+0000","last_active":"2023-11-28T11:21:01.883772+0000","last_peered":"2023-11-28T11:21:01.883772+0000","last_clean":"2023-11-28T11:21:01.883772+0000","last_became_active":"2023-11-28T11:21:00.297281+0000","last_became_peered":"2023-11-28T11:21:00.297281+0000","last_unstale":"2023-11-28T11:21:01.883772+0000","last_undegraded":"2023-11-28T11:21:01.883772+0000","last_fullsized":"2023-11-28T11:21:01.883772+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:42:41.996906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.292230+0000","last_change":"2023-11-28T11:20:55.764439+0000","last_active":"2023-11-28T11:21:00.292230+0000","last_peered":"2023-11-28T11:21:00.292230+0000","last_clean":"2023-11-28T11:21:00.292230+0000","last_became_active":"2023-11-28T11:20:55.764165+0000","last_became_peered":"2023-11-28T11:20:55.764165+0000","last_unstale":"2023-11-28T11:21:00.292230+0000","last_undegraded":"2023-11-28T11:21:00.292230+0000","last_fullsized":"2023-11-28T11:21:00.292230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:33:02.765966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.273131+0000","last_change":"2023-11-28T11:20:56.265074+0000","last_active":"2023-11-28T11:21:00.273131+0000","last_peered":"2023-11-28T11:21:00.273131+0000","last_clean":"2023-11-28T11:21:00.273131+0000","last_became_active":"2023-11-28T11:20:56.264495+0000","last_became_peered":"2023-11-28T11:20:56.264495+0000","last_unstale":"2023-11-28T11:21:00.273131+0000","last_undegraded":"2023-11-28T11:21:00.273131+0000","last_fullsized":"2023-11-28T11:21:00.273131+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:31:26.827850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.286246+0000","last_change":"2023-11-28T11:20:56.260972+0000","last_active":"2023-11-28T11:21:00.286246+0000","last_peered":"2023-11-28T11:21:00.286246+0000","last_clean":"2023-11-28T11:21:00.286246+0000","last_became_active":"2023-11-28T11:20:56.259491+0000","last_became_peered":"2023-11-28T11:20:56.259491+0000","last_unstale":"2023-11-28T11:21:00.286246+0000","last_undegraded":"2023-11-28T11:21:00.286246+0000","last_fullsized":"2023-11-28T11:21:00.286246+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:06:02.860064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.271158+0000","last_change":"2023-11-28T11:20:56.263917+0000","last_active":"2023-11-28T11:21:00.271158+0000","last_peered":"2023-11-28T11:21:00.271158+0000","last_clean":"2023-11-28T11:21:00.271158+0000","last_became_active":"2023-11-28T11:20:56.261946+0000","last_became_peered":"2023-11-28T11:20:56.261946+0000","last_unstale":"2023-11-28T11:21:00.271158+0000","last_undegraded":"2023-11-28T11:21:00.271158+0000","last_fullsized":"2023-11-28T11:21:00.271158+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:40:42.059656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.274852+0000","last_change":"2023-11-28T11:20:56.272089+0000","last_active":"2023-11-28T11:21:00.274852+0000","last_peered":"2023-11-28T11:21:00.274852+0000","last_clean":"2023-11-28T11:21:00.274852+0000","last_became_active":"2023-11-28T11:20:56.271704+0000","last_became_peered":"2023-11-28T11:20:56.271704+0000","last_unstale":"2023-11-28T11:21:00.274852+0000","last_undegraded":"2023-11-28T11:21:00.274852+0000","last_fullsized":"2023-11-28T11:21:00.274852+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:33:54.639487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.270968+0000","last_change":"2023-11-28T11:20:56.270543+0000","last_active":"2023-11-28T11:21:00.270968+0000","last_peered":"2023-11-28T11:21:00.270968+0000","last_clean":"2023-11-28T11:21:00.270968+0000","last_became_active":"2023-11-28T11:20:56.270310+0000","last_became_peered":"2023-11-28T11:20:56.270310+0000","last_unstale":"2023-11-28T11:21:00.270968+0000","last_undegraded":"2023-11-28T11:21:00.270968+0000","last_fullsized":"2023-11-28T11:21:00.270968+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:21:21.052493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.288589+0000","last_change":"2023-11-28T11:20:56.262145+0000","last_active":"2023-11-28T11:21:00.288589+0000","last_peered":"2023-11-28T11:21:00.288589+0000","last_clean":"2023-11-28T11:21:00.288589+0000","last_became_active":"2023-11-28T11:20:56.260881+0000","last_became_peered":"2023-11-28T11:20:56.260881+0000","last_unstale":"2023-11-28T11:21:00.288589+0000","last_undegraded":"2023-11-28T11:21:00.288589+0000","last_fullsized":"2023-11-28T11:21:00.288589+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:32:52.386018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4528,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9280,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362560,"statfs":{"total":96636764160,"available":96627261440,"internally_reserved":0,"allocated":909312,"data_stored":713609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73999999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8100,"kb_used_data":284,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363740,"statfs":{"total":96636764160,"available":96628469760,"internally_reserved":0,"allocated":290816,"data_stored":119901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81200000000000006}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88800000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8164,"kb_used_data":284,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363676,"statfs":{"total":96636764160,"available":96628404224,"internally_reserved":0,"allocated":290816,"data_stored":119901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9260,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362580,"statfs":{"total":96636764160,"available":96627281920,"internally_reserved":0,"allocated":888832,"data_stored":710835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8100,"kb_used_data":284,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363740,"statfs":{"total":96636764160,"available":96628469760,"internally_reserved":0,"allocated":290816,"data_stored":119901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79700000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73999999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8168,"kb_used_data":288,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363672,"statfs":{"total":96636764160,"available":96628400128,"internally_reserved":0,"allocated":294912,"data_stored":119991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8248,"kb_used_data":304,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363592,"statfs":{"total":96636764160,"available":96628318208,"internally_reserved":0,"allocated":311296,"data_stored":123703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8172,"kb_used_data":292,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363668,"statfs":{"total":96636764160,"available":96628396032,"internally_reserved":0,"allocated":299008,"data_stored":120537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:21:17.920 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:21:18.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.160+0000 7f63b61ff700 1 Processor -- start 2023-11-28T11:21:18.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.160+0000 7f63b61ff700 1 -- start start 2023-11-28T11:21:18.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.161+0000 7f63b61ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63b015a110 0x7f63b0160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:18.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.161+0000 7f63b61ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f63b00cfec0 0x7f63b00b19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:18.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.161+0000 7f63b61ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63b0161070 con 0x7f63b015d820 2023-11-28T11:21:18.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.161+0000 7f63b61ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63b00cdae0 con 0x7f63b00cfec0 2023-11-28T11:21:18.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.161+0000 7f63b61ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63b00ce4f0 con 0x7f63b015a110 2023-11-28T11:21:18.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.162+0000 7f63aeffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f63b015d820 0x7f63b00cf7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46190/0 (socket says 172.21.15.92:46190) 2023-11-28T11:21:18.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.162+0000 7f63aeffd700 1 -- 172.21.15.92:0/2190122776 learned_addr learned my addr 172.21.15.92:0/2190122776 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:18.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.162+0000 7f63affff700 1 --2- 172.21.15.92:0/2190122776 >> v2:172.21.15.92:3300/0 conn(0x7f63b00cfec0 0x7f63b00b19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:18.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.162+0000 7f63af7fe700 1 --2- 172.21.15.92:0/2190122776 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63b015a110 0x7f63b0160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:18.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.162+0000 7f63affff700 1 -- 172.21.15.92:0/2190122776 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63b015a110 msgr2=0x7f63b0160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:18.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.162+0000 7f63affff700 1 --2- 172.21.15.92:0/2190122776 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63b015a110 0x7f63b0160b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:18.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.162+0000 7f63affff700 1 -- 172.21.15.92:0/2190122776 >> v1:172.21.15.92:6789/0 conn(0x7f63b015d820 legacy=0x7f63b00cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:18.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.162+0000 7f63affff700 1 -- 172.21.15.92:0/2190122776 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b00cf2d0 con 0x7f63b00cfec0 2023-11-28T11:21:18.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.162+0000 7f63affff700 1 --2- 172.21.15.92:0/2190122776 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63b00cfec0 0x7f63b00b19b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f63a8009d90 tx=0x7f63a800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=143c23ef9fc64823 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:18.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.163+0000 7f63ae7fc700 1 -- 172.21.15.92:0/2190122776 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a800e070 con 0x7f63b00cfec0 2023-11-28T11:21:18.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.163+0000 7f63ae7fc700 1 -- 172.21.15.92:0/2190122776 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63a800bbd0 con 0x7f63b00cfec0 2023-11-28T11:21:18.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.163+0000 7f63ae7fc700 1 -- 172.21.15.92:0/2190122776 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a8012430 con 0x7f63b00cfec0 2023-11-28T11:21:18.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.163+0000 7f63b61ff700 1 -- 172.21.15.92:0/2190122776 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63b00cfec0 msgr2=0x7f63b00b19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:18.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.163+0000 7f63b61ff700 1 --2- 172.21.15.92:0/2190122776 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63b00cfec0 0x7f63b00b19b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f63a8009d90 tx=0x7f63a800b330 comp rx=0 tx=0).stop 2023-11-28T11:21:18.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.163+0000 7f63b61ff700 1 -- 172.21.15.92:0/2190122776 shutdown_connections 2023-11-28T11:21:18.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.163+0000 7f63b61ff700 1 --2- 172.21.15.92:0/2190122776 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63b015a110 0x7f63b0160b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:18.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.163+0000 7f63b61ff700 1 --2- 172.21.15.92:0/2190122776 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63b00cfec0 0x7f63b00b19b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:18.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.163+0000 7f63b61ff700 1 -- 172.21.15.92:0/2190122776 >> 172.21.15.92:0/2190122776 conn(0x7f63b00719a0 msgr2=0x7f63b0071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:18.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.163+0000 7f63b61ff700 1 -- 172.21.15.92:0/2190122776 shutdown_connections 2023-11-28T11:21:18.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.163+0000 7f63b61ff700 1 -- 172.21.15.92:0/2190122776 wait complete. 2023-11-28T11:21:18.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.166+0000 7f63b61ff700 1 Processor -- start 2023-11-28T11:21:18.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.166+0000 7f63b61ff700 1 -- start start 2023-11-28T11:21:18.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.167+0000 7f63b61ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63b00cfec0 0x7f63b0153640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:18.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.167+0000 7f63b61ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63b015a110 0x7f63b00b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:18.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.167+0000 7f63b61ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63b015d820 0x7f63b00b2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:18.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.167+0000 7f63b61ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b00ca5c0 con 0x7f63b015d820 2023-11-28T11:21:18.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.167+0000 7f63b61ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b00cb0f0 con 0x7f63b00cfec0 2023-11-28T11:21:18.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.167+0000 7f63b61ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63b00cb370 con 0x7f63b015a110 2023-11-28T11:21:18.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.167+0000 7f63affff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63b015d820 0x7f63b00b2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:18.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.167+0000 7f63affff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63b015d820 0x7f63b00b2890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49114/0 (socket says 172.21.15.92:49114) 2023-11-28T11:21:18.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.167+0000 7f63affff700 1 -- 172.21.15.92:0/3161269349 learned_addr learned my addr 172.21.15.92:0/3161269349 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:18.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.168+0000 7f63aeffd700 1 --2- 172.21.15.92:0/3161269349 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63b015a110 0x7f63b00b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:18.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.168+0000 7f63af7fe700 1 --2- 172.21.15.92:0/3161269349 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63b00cfec0 0x7f63b0153640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:18.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.168+0000 7f63affff700 1 -- 172.21.15.92:0/3161269349 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63b015a110 msgr2=0x7f63b00b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:18.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.168+0000 7f63affff700 1 --2- 172.21.15.92:0/3161269349 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63b015a110 0x7f63b00b46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:18.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.168+0000 7f63affff700 1 -- 172.21.15.92:0/3161269349 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63b00cfec0 msgr2=0x7f63b0153640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:18.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.168+0000 7f63affff700 1 --2- 172.21.15.92:0/3161269349 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63b00cfec0 0x7f63b0153640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:18.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.168+0000 7f63affff700 1 -- 172.21.15.92:0/3161269349 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a8009a70 con 0x7f63b015d820 2023-11-28T11:21:18.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.168+0000 7f63affff700 1 --2- 172.21.15.92:0/3161269349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63b015d820 0x7f63b00b2890 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f63a8014040 tx=0x7f63a8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:18.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.168+0000 7f6397fff700 1 -- 172.21.15.92:0/3161269349 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a8012430 con 0x7f63b015d820 2023-11-28T11:21:18.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.168+0000 7f6397fff700 1 -- 172.21.15.92:0/3161269349 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63a8020070 con 0x7f63b015d820 2023-11-28T11:21:18.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.168+0000 7f63b61ff700 1 -- 172.21.15.92:0/3161269349 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63b015cd40 con 0x7f63b015d820 2023-11-28T11:21:18.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.168+0000 7f6397fff700 1 -- 172.21.15.92:0/3161269349 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a8004280 con 0x7f63b015d820 2023-11-28T11:21:18.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.168+0000 7f63b61ff700 1 -- 172.21.15.92:0/3161269349 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63b00cd800 con 0x7f63b015d820 2023-11-28T11:21:18.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.169+0000 7f63b61ff700 1 -- 172.21.15.92:0/3161269349 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63b015d020 con 0x7f63b015d820 2023-11-28T11:21:18.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.170+0000 7f6397fff700 1 -- 172.21.15.92:0/3161269349 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f63a802e070 con 0x7f63b015d820 2023-11-28T11:21:18.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.171+0000 7f6397fff700 1 --2- 172.21.15.92:0/3161269349 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f639c0682f0 0x7f639c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:18.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.171+0000 7f6397fff700 1 -- 172.21.15.92:0/3161269349 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f63a8022070 con 0x7f63b015d820 2023-11-28T11:21:18.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.171+0000 7f63af7fe700 1 --2- 172.21.15.92:0/3161269349 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f639c0682f0 0x7f639c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:18.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.172+0000 7f63af7fe700 1 --2- 172.21.15.92:0/3161269349 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f639c0682f0 0x7f639c06a7a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f63a0009930 tx=0x7f63a0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:18.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.175+0000 7f6397fff700 1 -- 172.21.15.92:0/3161269349 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f63a8054290 con 0x7f63b015d820 2023-11-28T11:21:18.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.365+0000 7f63b61ff700 1 -- 172.21.15.92:0/3161269349 --> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f63b0067920 con 0x7f639c0682f0 2023-11-28T11:21:18.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.375+0000 7f6397fff700 1 -- 172.21.15.92:0/3161269349 <== mgr.4103 v2:172.21.15.92:6832/115482 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367403 (secure 0 0 0) 0x7f63b0067920 con 0x7f639c0682f0 2023-11-28T11:21:18.382 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:21:18.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.380+0000 7f63b61ff700 1 -- 172.21.15.92:0/3161269349 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f639c0682f0 msgr2=0x7f639c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:18.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.380+0000 7f63b61ff700 1 --2- 172.21.15.92:0/3161269349 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f639c0682f0 0x7f639c06a7a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f63a0009930 tx=0x7f63a0008040 comp rx=0 tx=0).stop 2023-11-28T11:21:18.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.380+0000 7f63b61ff700 1 -- 172.21.15.92:0/3161269349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63b015d820 msgr2=0x7f63b00b2890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:18.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.380+0000 7f63b61ff700 1 --2- 172.21.15.92:0/3161269349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63b015d820 0x7f63b00b2890 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f63a8014040 tx=0x7f63a8004ad0 comp rx=0 tx=0).stop 2023-11-28T11:21:18.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.380+0000 7f63b61ff700 1 -- 172.21.15.92:0/3161269349 shutdown_connections 2023-11-28T11:21:18.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.380+0000 7f63b61ff700 1 --2- 172.21.15.92:0/3161269349 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f639c0682f0 0x7f639c06a7a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:18.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.380+0000 7f63b61ff700 1 --2- 172.21.15.92:0/3161269349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63b015d820 0x7f63b00b2890 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:18.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.380+0000 7f63b61ff700 1 --2- 172.21.15.92:0/3161269349 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f63b00cfec0 0x7f63b0153640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:18.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.380+0000 7f63b61ff700 1 --2- 172.21.15.92:0/3161269349 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f63b015a110 0x7f63b00b46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:18.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.380+0000 7f63b61ff700 1 -- 172.21.15.92:0/3161269349 >> 172.21.15.92:0/3161269349 conn(0x7f63b00719a0 msgr2=0x7f63b00d2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:18.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.380+0000 7f63b61ff700 1 -- 172.21.15.92:0/3161269349 shutdown_connections 2023-11-28T11:21:18.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.380+0000 7f63b61ff700 1 -- 172.21.15.92:0/3161269349 wait complete. 2023-11-28T11:21:18.386 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-11-28T11:21:18.400 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2023-11-28T11:21:17.864979+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594896,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":67492,"kb_used_data":3492,"kb_used_omap":0,"kb_used_meta":63936,"kb_avail":754907228,"statfs":{"total":773094113280,"available":773025001472,"internally_reserved":0,"allocated":3575808,"data_stored":2148378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65470464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2198,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002663"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.282064+0000","last_change":"2023-11-28T11:20:56.269261+0000","last_active":"2023-11-28T11:21:00.282064+0000","last_peered":"2023-11-28T11:21:00.282064+0000","last_clean":"2023-11-28T11:21:00.282064+0000","last_became_active":"2023-11-28T11:20:56.268959+0000","last_became_peered":"2023-11-28T11:20:56.268959+0000","last_unstale":"2023-11-28T11:21:00.282064+0000","last_undegraded":"2023-11-28T11:21:00.282064+0000","last_fullsized":"2023-11-28T11:21:00.282064+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:39:28.472955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.273172+0000","last_change":"2023-11-28T11:20:56.258503+0000","last_active":"2023-11-28T11:21:00.273172+0000","last_peered":"2023-11-28T11:21:00.273172+0000","last_clean":"2023-11-28T11:21:00.273172+0000","last_became_active":"2023-11-28T11:20:56.258101+0000","last_became_peered":"2023-11-28T11:20:56.258101+0000","last_unstale":"2023-11-28T11:21:00.273172+0000","last_undegraded":"2023-11-28T11:21:00.273172+0000","last_fullsized":"2023-11-28T11:21:00.273172+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:14:34.876987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.274259+0000","last_change":"2023-11-28T11:20:56.269696+0000","last_active":"2023-11-28T11:21:00.274259+0000","last_peered":"2023-11-28T11:21:00.274259+0000","last_clean":"2023-11-28T11:21:00.274259+0000","last_became_active":"2023-11-28T11:20:56.264650+0000","last_became_peered":"2023-11-28T11:20:56.264650+0000","last_unstale":"2023-11-28T11:21:00.274259+0000","last_undegraded":"2023-11-28T11:21:00.274259+0000","last_fullsized":"2023-11-28T11:21:00.274259+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:25:52.242410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.286447+0000","last_change":"2023-11-28T11:20:56.269817+0000","last_active":"2023-11-28T11:21:00.286447+0000","last_peered":"2023-11-28T11:21:00.286447+0000","last_clean":"2023-11-28T11:21:00.286447+0000","last_became_active":"2023-11-28T11:20:56.269514+0000","last_became_peered":"2023-11-28T11:20:56.269514+0000","last_unstale":"2023-11-28T11:21:00.286447+0000","last_undegraded":"2023-11-28T11:21:00.286447+0000","last_fullsized":"2023-11-28T11:21:00.286447+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:09:41.753917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.289921+0000","last_change":"2023-11-28T11:20:56.274319+0000","last_active":"2023-11-28T11:21:00.289921+0000","last_peered":"2023-11-28T11:21:00.289921+0000","last_clean":"2023-11-28T11:21:00.289921+0000","last_became_active":"2023-11-28T11:20:56.273667+0000","last_became_peered":"2023-11-28T11:20:56.273667+0000","last_unstale":"2023-11-28T11:21:00.289921+0000","last_undegraded":"2023-11-28T11:21:00.289921+0000","last_fullsized":"2023-11-28T11:21:00.289921+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:31:26.827850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.274908+0000","last_change":"2023-11-28T11:20:56.263963+0000","last_active":"2023-11-28T11:21:00.274908+0000","last_peered":"2023-11-28T11:21:00.274908+0000","last_clean":"2023-11-28T11:21:00.274908+0000","last_became_active":"2023-11-28T11:20:56.260761+0000","last_became_peered":"2023-11-28T11:20:56.260761+0000","last_unstale":"2023-11-28T11:21:00.274908+0000","last_undegraded":"2023-11-28T11:21:00.274908+0000","last_fullsized":"2023-11-28T11:21:00.274908+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:13:46.683000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.288334+0000","last_change":"2023-11-28T11:20:56.261054+0000","last_active":"2023-11-28T11:21:00.288334+0000","last_peered":"2023-11-28T11:21:00.288334+0000","last_clean":"2023-11-28T11:21:00.288334+0000","last_became_active":"2023-11-28T11:20:56.260774+0000","last_became_peered":"2023-11-28T11:20:56.260774+0000","last_unstale":"2023-11-28T11:21:00.288334+0000","last_undegraded":"2023-11-28T11:21:00.288334+0000","last_fullsized":"2023-11-28T11:21:00.288334+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:08.079216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.273485+0000","last_change":"2023-11-28T11:20:56.252717+0000","last_active":"2023-11-28T11:21:00.273485+0000","last_peered":"2023-11-28T11:21:00.273485+0000","last_clean":"2023-11-28T11:21:00.273485+0000","last_became_active":"2023-11-28T11:20:56.252066+0000","last_became_peered":"2023-11-28T11:20:56.252066+0000","last_unstale":"2023-11-28T11:21:00.273485+0000","last_undegraded":"2023-11-28T11:21:00.273485+0000","last_fullsized":"2023-11-28T11:21:00.273485+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:42:57.864956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.291971+0000","last_change":"2023-11-28T11:20:56.273810+0000","last_active":"2023-11-28T11:21:00.291971+0000","last_peered":"2023-11-28T11:21:00.291971+0000","last_clean":"2023-11-28T11:21:00.291971+0000","last_became_active":"2023-11-28T11:20:56.273477+0000","last_became_peered":"2023-11-28T11:20:56.273477+0000","last_unstale":"2023-11-28T11:21:00.291971+0000","last_undegraded":"2023-11-28T11:21:00.291971+0000","last_fullsized":"2023-11-28T11:21:00.291971+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:50:45.228070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.277322+0000","last_change":"2023-11-28T11:20:56.256667+0000","last_active":"2023-11-28T11:21:00.277322+0000","last_peered":"2023-11-28T11:21:00.277322+0000","last_clean":"2023-11-28T11:21:00.277322+0000","last_became_active":"2023-11-28T11:20:56.256387+0000","last_became_peered":"2023-11-28T11:20:56.256387+0000","last_unstale":"2023-11-28T11:21:00.277322+0000","last_undegraded":"2023-11-28T11:21:00.277322+0000","last_fullsized":"2023-11-28T11:21:00.277322+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:24:07.384730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.274154+0000","last_change":"2023-11-28T11:20:56.281728+0000","last_active":"2023-11-28T11:21:00.274154+0000","last_peered":"2023-11-28T11:21:00.274154+0000","last_clean":"2023-11-28T11:21:00.274154+0000","last_became_active":"2023-11-28T11:20:56.281437+0000","last_became_peered":"2023-11-28T11:20:56.281437+0000","last_unstale":"2023-11-28T11:21:00.274154+0000","last_undegraded":"2023-11-28T11:21:00.274154+0000","last_fullsized":"2023-11-28T11:21:00.274154+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:01:57.936285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.280518+0000","last_change":"2023-11-28T11:20:56.256016+0000","last_active":"2023-11-28T11:21:00.280518+0000","last_peered":"2023-11-28T11:21:00.280518+0000","last_clean":"2023-11-28T11:21:00.280518+0000","last_became_active":"2023-11-28T11:20:56.255480+0000","last_became_peered":"2023-11-28T11:20:56.255480+0000","last_unstale":"2023-11-28T11:21:00.280518+0000","last_undegraded":"2023-11-28T11:21:00.280518+0000","last_fullsized":"2023-11-28T11:21:00.280518+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:45:09.586958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.280455+0000","last_change":"2023-11-28T11:20:56.265497+0000","last_active":"2023-11-28T11:21:00.280455+0000","last_peered":"2023-11-28T11:21:00.280455+0000","last_clean":"2023-11-28T11:21:00.280455+0000","last_became_active":"2023-11-28T11:20:56.265236+0000","last_became_peered":"2023-11-28T11:20:56.265236+0000","last_unstale":"2023-11-28T11:21:00.280455+0000","last_undegraded":"2023-11-28T11:21:00.280455+0000","last_fullsized":"2023-11-28T11:21:00.280455+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:47:39.108512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.274722+0000","last_change":"2023-11-28T11:20:56.262165+0000","last_active":"2023-11-28T11:21:00.274722+0000","last_peered":"2023-11-28T11:21:00.274722+0000","last_clean":"2023-11-28T11:21:00.274722+0000","last_became_active":"2023-11-28T11:20:56.261947+0000","last_became_peered":"2023-11-28T11:20:56.261947+0000","last_unstale":"2023-11-28T11:21:00.274722+0000","last_undegraded":"2023-11-28T11:21:00.274722+0000","last_fullsized":"2023-11-28T11:21:00.274722+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:45:14.478416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.288123+0000","last_change":"2023-11-28T11:20:56.264861+0000","last_active":"2023-11-28T11:21:00.288123+0000","last_peered":"2023-11-28T11:21:00.288123+0000","last_clean":"2023-11-28T11:21:00.288123+0000","last_became_active":"2023-11-28T11:20:56.264538+0000","last_became_peered":"2023-11-28T11:20:56.264538+0000","last_unstale":"2023-11-28T11:21:00.288123+0000","last_undegraded":"2023-11-28T11:21:00.288123+0000","last_fullsized":"2023-11-28T11:21:00.288123+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:40:42.059656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.273219+0000","last_change":"2023-11-28T11:20:56.258923+0000","last_active":"2023-11-28T11:21:00.273219+0000","last_peered":"2023-11-28T11:21:00.273219+0000","last_clean":"2023-11-28T11:21:00.273219+0000","last_became_active":"2023-11-28T11:20:56.258633+0000","last_became_peered":"2023-11-28T11:20:56.258633+0000","last_unstale":"2023-11-28T11:21:00.273219+0000","last_undegraded":"2023-11-28T11:21:00.273219+0000","last_fullsized":"2023-11-28T11:21:00.273219+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:26:32.343329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.292050+0000","last_change":"2023-11-28T11:20:56.271537+0000","last_active":"2023-11-28T11:21:00.292050+0000","last_peered":"2023-11-28T11:21:00.292050+0000","last_clean":"2023-11-28T11:21:00.292050+0000","last_became_active":"2023-11-28T11:20:56.270556+0000","last_became_peered":"2023-11-28T11:20:56.270556+0000","last_unstale":"2023-11-28T11:21:00.292050+0000","last_undegraded":"2023-11-28T11:21:00.292050+0000","last_fullsized":"2023-11-28T11:21:00.292050+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:11:36.091056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.274876+0000","last_change":"2023-11-28T11:20:56.274810+0000","last_active":"2023-11-28T11:21:00.274876+0000","last_peered":"2023-11-28T11:21:00.274876+0000","last_clean":"2023-11-28T11:21:00.274876+0000","last_became_active":"2023-11-28T11:20:56.274523+0000","last_became_peered":"2023-11-28T11:20:56.274523+0000","last_unstale":"2023-11-28T11:21:00.274876+0000","last_undegraded":"2023-11-28T11:21:00.274876+0000","last_fullsized":"2023-11-28T11:21:00.274876+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:42:57.864956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.281099+0000","last_change":"2023-11-28T11:20:56.258601+0000","last_active":"2023-11-28T11:21:00.281099+0000","last_peered":"2023-11-28T11:21:00.281099+0000","last_clean":"2023-11-28T11:21:00.281099+0000","last_became_active":"2023-11-28T11:20:56.258282+0000","last_became_peered":"2023-11-28T11:20:56.258282+0000","last_unstale":"2023-11-28T11:21:00.281099+0000","last_undegraded":"2023-11-28T11:21:00.281099+0000","last_fullsized":"2023-11-28T11:21:00.281099+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:58:47.731976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.275898+0000","last_change":"2023-11-28T11:20:56.263948+0000","last_active":"2023-11-28T11:21:00.275898+0000","last_peered":"2023-11-28T11:21:00.275898+0000","last_clean":"2023-11-28T11:21:00.275898+0000","last_became_active":"2023-11-28T11:20:56.263627+0000","last_became_peered":"2023-11-28T11:20:56.263627+0000","last_unstale":"2023-11-28T11:21:00.275898+0000","last_undegraded":"2023-11-28T11:21:00.275898+0000","last_fullsized":"2023-11-28T11:21:00.275898+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:08.079216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.300168+0000","last_change":"2023-11-28T11:20:56.274116+0000","last_active":"2023-11-28T11:21:00.300168+0000","last_peered":"2023-11-28T11:21:00.300168+0000","last_clean":"2023-11-28T11:21:00.300168+0000","last_became_active":"2023-11-28T11:20:56.273403+0000","last_became_peered":"2023-11-28T11:20:56.273403+0000","last_unstale":"2023-11-28T11:21:00.300168+0000","last_undegraded":"2023-11-28T11:21:00.300168+0000","last_fullsized":"2023-11-28T11:21:00.300168+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:45:14.478416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.291080+0000","last_change":"2023-11-28T11:20:56.259411+0000","last_active":"2023-11-28T11:21:00.291080+0000","last_peered":"2023-11-28T11:21:00.291080+0000","last_clean":"2023-11-28T11:21:00.291080+0000","last_became_active":"2023-11-28T11:20:56.259189+0000","last_became_peered":"2023-11-28T11:20:56.259189+0000","last_unstale":"2023-11-28T11:21:00.291080+0000","last_undegraded":"2023-11-28T11:21:00.291080+0000","last_fullsized":"2023-11-28T11:21:00.291080+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:24:07.384730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.270889+0000","last_change":"2023-11-28T11:20:56.257033+0000","last_active":"2023-11-28T11:21:00.270889+0000","last_peered":"2023-11-28T11:21:00.270889+0000","last_clean":"2023-11-28T11:21:00.270889+0000","last_became_active":"2023-11-28T11:20:56.256549+0000","last_became_peered":"2023-11-28T11:20:56.256549+0000","last_unstale":"2023-11-28T11:21:00.270889+0000","last_undegraded":"2023-11-28T11:21:00.270889+0000","last_fullsized":"2023-11-28T11:21:00.270889+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:24:07.384730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.286598+0000","last_change":"2023-11-28T11:20:56.270979+0000","last_active":"2023-11-28T11:21:00.286598+0000","last_peered":"2023-11-28T11:21:00.286598+0000","last_clean":"2023-11-28T11:21:00.286598+0000","last_became_active":"2023-11-28T11:20:56.270710+0000","last_became_peered":"2023-11-28T11:20:56.270710+0000","last_unstale":"2023-11-28T11:21:00.286598+0000","last_undegraded":"2023-11-28T11:21:00.286598+0000","last_fullsized":"2023-11-28T11:21:00.286598+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:42:57.864956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.290274+0000","last_change":"2023-11-28T11:20:56.262561+0000","last_active":"2023-11-28T11:21:00.290274+0000","last_peered":"2023-11-28T11:21:00.290274+0000","last_clean":"2023-11-28T11:21:00.290274+0000","last_became_active":"2023-11-28T11:20:56.262251+0000","last_became_peered":"2023-11-28T11:20:56.262251+0000","last_unstale":"2023-11-28T11:21:00.290274+0000","last_undegraded":"2023-11-28T11:21:00.290274+0000","last_fullsized":"2023-11-28T11:21:00.290274+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:11:36.091056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.292352+0000","last_change":"2023-11-28T11:20:56.261421+0000","last_active":"2023-11-28T11:21:00.292352+0000","last_peered":"2023-11-28T11:21:00.292352+0000","last_clean":"2023-11-28T11:21:00.292352+0000","last_became_active":"2023-11-28T11:20:56.259655+0000","last_became_peered":"2023-11-28T11:20:56.259655+0000","last_unstale":"2023-11-28T11:21:00.292352+0000","last_undegraded":"2023-11-28T11:21:00.292352+0000","last_fullsized":"2023-11-28T11:21:00.292352+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:01:57.936285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.273393+0000","last_change":"2023-11-28T11:20:56.258358+0000","last_active":"2023-11-28T11:21:00.273393+0000","last_peered":"2023-11-28T11:21:00.273393+0000","last_clean":"2023-11-28T11:21:00.273393+0000","last_became_active":"2023-11-28T11:20:56.257895+0000","last_became_peered":"2023-11-28T11:20:56.257895+0000","last_unstale":"2023-11-28T11:21:00.273393+0000","last_undegraded":"2023-11-28T11:21:00.273393+0000","last_fullsized":"2023-11-28T11:21:00.273393+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:08.079216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.777639+0000","last_change":"2023-11-28T11:21:00.306498+0000","last_active":"2023-11-28T11:21:00.777639+0000","last_peered":"2023-11-28T11:21:00.777639+0000","last_clean":"2023-11-28T11:21:00.777639+0000","last_became_active":"2023-11-28T11:21:00.305632+0000","last_became_peered":"2023-11-28T11:21:00.305632+0000","last_unstale":"2023-11-28T11:21:00.777639+0000","last_undegraded":"2023-11-28T11:21:00.777639+0000","last_fullsized":"2023-11-28T11:21:00.777639+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:08:42.028668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.274765+0000","last_change":"2023-11-28T11:20:55.761598+0000","last_active":"2023-11-28T11:21:00.274765+0000","last_peered":"2023-11-28T11:21:00.274765+0000","last_clean":"2023-11-28T11:21:00.274765+0000","last_became_active":"2023-11-28T11:20:55.761257+0000","last_became_peered":"2023-11-28T11:20:55.761257+0000","last_unstale":"2023-11-28T11:21:00.274765+0000","last_undegraded":"2023-11-28T11:21:00.274765+0000","last_fullsized":"2023-11-28T11:21:00.274765+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:13:32.428410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.271441+0000","last_change":"2023-11-28T11:20:56.269300+0000","last_active":"2023-11-28T11:21:00.271441+0000","last_peered":"2023-11-28T11:21:00.271441+0000","last_clean":"2023-11-28T11:21:00.271441+0000","last_became_active":"2023-11-28T11:20:56.269068+0000","last_became_peered":"2023-11-28T11:20:56.269068+0000","last_unstale":"2023-11-28T11:21:00.271441+0000","last_undegraded":"2023-11-28T11:21:00.271441+0000","last_fullsized":"2023-11-28T11:21:00.271441+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:42:57.864956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.779188+0000","last_change":"2023-11-28T11:21:00.303150+0000","last_active":"2023-11-28T11:21:00.779188+0000","last_peered":"2023-11-28T11:21:00.779188+0000","last_clean":"2023-11-28T11:21:00.779188+0000","last_became_active":"2023-11-28T11:21:00.302669+0000","last_became_peered":"2023-11-28T11:21:00.302669+0000","last_unstale":"2023-11-28T11:21:00.779188+0000","last_undegraded":"2023-11-28T11:21:00.779188+0000","last_fullsized":"2023-11-28T11:21:00.779188+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:26:35.867941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.284078+0000","last_change":"2023-11-28T11:20:55.769854+0000","last_active":"2023-11-28T11:21:00.284078+0000","last_peered":"2023-11-28T11:21:00.284078+0000","last_clean":"2023-11-28T11:21:00.284078+0000","last_became_active":"2023-11-28T11:20:55.769322+0000","last_became_peered":"2023-11-28T11:20:55.769322+0000","last_unstale":"2023-11-28T11:21:00.284078+0000","last_undegraded":"2023-11-28T11:21:00.284078+0000","last_fullsized":"2023-11-28T11:21:00.284078+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:37:05.627049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.291807+0000","last_change":"2023-11-28T11:20:56.265137+0000","last_active":"2023-11-28T11:21:00.291807+0000","last_peered":"2023-11-28T11:21:00.291807+0000","last_clean":"2023-11-28T11:21:00.291807+0000","last_became_active":"2023-11-28T11:20:56.264228+0000","last_became_peered":"2023-11-28T11:20:56.264228+0000","last_unstale":"2023-11-28T11:21:00.291807+0000","last_undegraded":"2023-11-28T11:21:00.291807+0000","last_fullsized":"2023-11-28T11:21:00.291807+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:42:57.864956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.884947+0000","last_change":"2023-11-28T11:21:00.314686+0000","last_active":"2023-11-28T11:21:01.884947+0000","last_peered":"2023-11-28T11:21:01.884947+0000","last_clean":"2023-11-28T11:21:01.884947+0000","last_became_active":"2023-11-28T11:21:00.314281+0000","last_became_peered":"2023-11-28T11:21:00.314281+0000","last_unstale":"2023-11-28T11:21:01.884947+0000","last_undegraded":"2023-11-28T11:21:01.884947+0000","last_fullsized":"2023-11-28T11:21:01.884947+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:29:41.143865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.271248+0000","last_change":"2023-11-28T11:20:55.768237+0000","last_active":"2023-11-28T11:21:00.271248+0000","last_peered":"2023-11-28T11:21:00.271248+0000","last_clean":"2023-11-28T11:21:00.271248+0000","last_became_active":"2023-11-28T11:20:55.767948+0000","last_became_peered":"2023-11-28T11:20:55.767948+0000","last_unstale":"2023-11-28T11:21:00.271248+0000","last_undegraded":"2023-11-28T11:21:00.271248+0000","last_fullsized":"2023-11-28T11:21:00.271248+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:23:36.801918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.286831+0000","last_change":"2023-11-28T11:20:56.260663+0000","last_active":"2023-11-28T11:21:00.286831+0000","last_peered":"2023-11-28T11:21:00.286831+0000","last_clean":"2023-11-28T11:21:00.286831+0000","last_became_active":"2023-11-28T11:20:56.259825+0000","last_became_peered":"2023-11-28T11:20:56.259825+0000","last_unstale":"2023-11-28T11:21:00.286831+0000","last_undegraded":"2023-11-28T11:21:00.286831+0000","last_fullsized":"2023-11-28T11:21:00.286831+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:45:09.586958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.778585+0000","last_change":"2023-11-28T11:21:00.303720+0000","last_active":"2023-11-28T11:21:00.778585+0000","last_peered":"2023-11-28T11:21:00.778585+0000","last_clean":"2023-11-28T11:21:00.778585+0000","last_became_active":"2023-11-28T11:21:00.302585+0000","last_became_peered":"2023-11-28T11:21:00.302585+0000","last_unstale":"2023-11-28T11:21:00.778585+0000","last_undegraded":"2023-11-28T11:21:00.778585+0000","last_fullsized":"2023-11-28T11:21:00.778585+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:28:33.645294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.292130+0000","last_change":"2023-11-28T11:20:55.764526+0000","last_active":"2023-11-28T11:21:00.292130+0000","last_peered":"2023-11-28T11:21:00.292130+0000","last_clean":"2023-11-28T11:21:00.292130+0000","last_became_active":"2023-11-28T11:20:55.764278+0000","last_became_peered":"2023-11-28T11:20:55.764278+0000","last_unstale":"2023-11-28T11:21:00.292130+0000","last_undegraded":"2023-11-28T11:21:00.292130+0000","last_fullsized":"2023-11-28T11:21:00.292130+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:23:36.801918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.271155+0000","last_change":"2023-11-28T11:20:56.265473+0000","last_active":"2023-11-28T11:21:00.271155+0000","last_peered":"2023-11-28T11:21:00.271155+0000","last_clean":"2023-11-28T11:21:00.271155+0000","last_became_active":"2023-11-28T11:20:56.262049+0000","last_became_peered":"2023-11-28T11:20:56.262049+0000","last_unstale":"2023-11-28T11:21:00.271155+0000","last_undegraded":"2023-11-28T11:21:00.271155+0000","last_fullsized":"2023-11-28T11:21:00.271155+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:08.079216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.806510+0000","last_change":"2023-11-28T11:21:00.306448+0000","last_active":"2023-11-28T11:21:01.806510+0000","last_peered":"2023-11-28T11:21:01.806510+0000","last_clean":"2023-11-28T11:21:01.806510+0000","last_became_active":"2023-11-28T11:21:00.305892+0000","last_became_peered":"2023-11-28T11:21:00.305892+0000","last_unstale":"2023-11-28T11:21:01.806510+0000","last_undegraded":"2023-11-28T11:21:01.806510+0000","last_fullsized":"2023-11-28T11:21:01.806510+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:41.700110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.277223+0000","last_change":"2023-11-28T11:20:55.764162+0000","last_active":"2023-11-28T11:21:00.277223+0000","last_peered":"2023-11-28T11:21:00.277223+0000","last_clean":"2023-11-28T11:21:00.277223+0000","last_became_active":"2023-11-28T11:20:55.760490+0000","last_became_peered":"2023-11-28T11:20:55.760490+0000","last_unstale":"2023-11-28T11:21:00.277223+0000","last_undegraded":"2023-11-28T11:21:00.277223+0000","last_fullsized":"2023-11-28T11:21:00.277223+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:39:27.467031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.271241+0000","last_change":"2023-11-28T11:20:56.269435+0000","last_active":"2023-11-28T11:21:00.271241+0000","last_peered":"2023-11-28T11:21:00.271241+0000","last_clean":"2023-11-28T11:21:00.271241+0000","last_became_active":"2023-11-28T11:20:56.269179+0000","last_became_peered":"2023-11-28T11:20:56.269179+0000","last_unstale":"2023-11-28T11:21:00.271241+0000","last_undegraded":"2023-11-28T11:21:00.271241+0000","last_fullsized":"2023-11-28T11:21:00.271241+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:50:45.228070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.844101+0000","last_change":"2023-11-28T11:21:00.292120+0000","last_active":"2023-11-28T11:21:01.844101+0000","last_peered":"2023-11-28T11:21:01.844101+0000","last_clean":"2023-11-28T11:21:01.844101+0000","last_became_active":"2023-11-28T11:21:00.289430+0000","last_became_peered":"2023-11-28T11:21:00.289430+0000","last_unstale":"2023-11-28T11:21:01.844101+0000","last_undegraded":"2023-11-28T11:21:01.844101+0000","last_fullsized":"2023-11-28T11:21:01.844101+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:41.700110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.298184+0000","last_change":"2023-11-28T11:20:55.771668+0000","last_active":"2023-11-28T11:21:00.298184+0000","last_peered":"2023-11-28T11:21:00.298184+0000","last_clean":"2023-11-28T11:21:00.298184+0000","last_became_active":"2023-11-28T11:20:55.771400+0000","last_became_peered":"2023-11-28T11:20:55.771400+0000","last_unstale":"2023-11-28T11:21:00.298184+0000","last_undegraded":"2023-11-28T11:21:00.298184+0000","last_fullsized":"2023-11-28T11:21:00.298184+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:27:11.291583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.290416+0000","last_change":"2023-11-28T11:20:56.263283+0000","last_active":"2023-11-28T11:21:00.290416+0000","last_peered":"2023-11-28T11:21:00.290416+0000","last_clean":"2023-11-28T11:21:00.290416+0000","last_became_active":"2023-11-28T11:20:56.262787+0000","last_became_peered":"2023-11-28T11:20:56.262787+0000","last_unstale":"2023-11-28T11:21:00.290416+0000","last_undegraded":"2023-11-28T11:21:00.290416+0000","last_fullsized":"2023-11-28T11:21:00.290416+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:40:42.059656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.883480+0000","last_change":"2023-11-28T11:21:00.297947+0000","last_active":"2023-11-28T11:21:01.883480+0000","last_peered":"2023-11-28T11:21:01.883480+0000","last_clean":"2023-11-28T11:21:01.883480+0000","last_became_active":"2023-11-28T11:21:00.297320+0000","last_became_peered":"2023-11-28T11:21:00.297320+0000","last_unstale":"2023-11-28T11:21:01.883480+0000","last_undegraded":"2023-11-28T11:21:01.883480+0000","last_fullsized":"2023-11-28T11:21:01.883480+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:56:39.135021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"27'10","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:13.462618+0000","last_change":"2023-11-28T11:20:55.761529+0000","last_active":"2023-11-28T11:21:13.462618+0000","last_peered":"2023-11-28T11:21:13.462618+0000","last_clean":"2023-11-28T11:21:13.462618+0000","last_became_active":"2023-11-28T11:20:55.759570+0000","last_became_peered":"2023-11-28T11:20:55.759570+0000","last_unstale":"2023-11-28T11:21:13.462618+0000","last_undegraded":"2023-11-28T11:21:13.462618+0000","last_fullsized":"2023-11-28T11:21:13.462618+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:54:56.683286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3746,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.272939+0000","last_change":"2023-11-28T11:20:56.280987+0000","last_active":"2023-11-28T11:21:00.272939+0000","last_peered":"2023-11-28T11:21:00.272939+0000","last_clean":"2023-11-28T11:21:00.272939+0000","last_became_active":"2023-11-28T11:20:56.279820+0000","last_became_peered":"2023-11-28T11:20:56.279820+0000","last_unstale":"2023-11-28T11:21:00.272939+0000","last_undegraded":"2023-11-28T11:21:00.272939+0000","last_fullsized":"2023-11-28T11:21:00.272939+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:50:45.228070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.383399+0000","last_change":"2023-11-28T11:21:00.294226+0000","last_active":"2023-11-28T11:21:01.383399+0000","last_peered":"2023-11-28T11:21:01.383399+0000","last_clean":"2023-11-28T11:21:01.383399+0000","last_became_active":"2023-11-28T11:21:00.293015+0000","last_became_peered":"2023-11-28T11:21:00.293015+0000","last_unstale":"2023-11-28T11:21:01.383399+0000","last_undegraded":"2023-11-28T11:21:01.383399+0000","last_fullsized":"2023-11-28T11:21:01.383399+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:35:00.472959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.298041+0000","last_change":"2023-11-28T11:20:55.766180+0000","last_active":"2023-11-28T11:21:00.298041+0000","last_peered":"2023-11-28T11:21:00.298041+0000","last_clean":"2023-11-28T11:21:00.298041+0000","last_became_active":"2023-11-28T11:20:55.765845+0000","last_became_peered":"2023-11-28T11:20:55.765845+0000","last_unstale":"2023-11-28T11:21:00.298041+0000","last_undegraded":"2023-11-28T11:21:00.298041+0000","last_fullsized":"2023-11-28T11:21:00.298041+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:39:27.467031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.271449+0000","last_change":"2023-11-28T11:20:56.268499+0000","last_active":"2023-11-28T11:21:00.271449+0000","last_peered":"2023-11-28T11:21:00.271449+0000","last_clean":"2023-11-28T11:21:00.271449+0000","last_became_active":"2023-11-28T11:20:56.267802+0000","last_became_peered":"2023-11-28T11:20:56.267802+0000","last_unstale":"2023-11-28T11:21:00.271449+0000","last_undegraded":"2023-11-28T11:21:00.271449+0000","last_fullsized":"2023-11-28T11:21:00.271449+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:11:36.091056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.844455+0000","last_change":"2023-11-28T11:21:00.302400+0000","last_active":"2023-11-28T11:21:01.844455+0000","last_peered":"2023-11-28T11:21:01.844455+0000","last_clean":"2023-11-28T11:21:01.844455+0000","last_became_active":"2023-11-28T11:21:00.301539+0000","last_became_peered":"2023-11-28T11:21:00.301539+0000","last_unstale":"2023-11-28T11:21:01.844455+0000","last_undegraded":"2023-11-28T11:21:01.844455+0000","last_fullsized":"2023-11-28T11:21:01.844455+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:53:42.920160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.290955+0000","last_change":"2023-11-28T11:20:55.766030+0000","last_active":"2023-11-28T11:21:00.290955+0000","last_peered":"2023-11-28T11:21:00.290955+0000","last_clean":"2023-11-28T11:21:00.290955+0000","last_became_active":"2023-11-28T11:20:55.765787+0000","last_became_peered":"2023-11-28T11:20:55.765787+0000","last_unstale":"2023-11-28T11:21:00.290955+0000","last_undegraded":"2023-11-28T11:21:00.290955+0000","last_fullsized":"2023-11-28T11:21:00.290955+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:58:46.726052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.273291+0000","last_change":"2023-11-28T11:20:56.258188+0000","last_active":"2023-11-28T11:21:00.273291+0000","last_peered":"2023-11-28T11:21:00.273291+0000","last_clean":"2023-11-28T11:21:00.273291+0000","last_became_active":"2023-11-28T11:20:56.257558+0000","last_became_peered":"2023-11-28T11:20:56.257558+0000","last_unstale":"2023-11-28T11:21:00.273291+0000","last_undegraded":"2023-11-28T11:21:00.273291+0000","last_fullsized":"2023-11-28T11:21:00.273291+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:11:36.091056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.844327+0000","last_change":"2023-11-28T11:21:00.307457+0000","last_active":"2023-11-28T11:21:01.844327+0000","last_peered":"2023-11-28T11:21:01.844327+0000","last_clean":"2023-11-28T11:21:01.844327+0000","last_became_active":"2023-11-28T11:21:00.307044+0000","last_became_peered":"2023-11-28T11:21:00.307044+0000","last_unstale":"2023-11-28T11:21:01.844327+0000","last_undegraded":"2023-11-28T11:21:01.844327+0000","last_fullsized":"2023-11-28T11:21:01.844327+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:35:00.472959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.288230+0000","last_change":"2023-11-28T11:20:55.765803+0000","last_active":"2023-11-28T11:21:00.288230+0000","last_peered":"2023-11-28T11:21:00.288230+0000","last_clean":"2023-11-28T11:21:00.288230+0000","last_became_active":"2023-11-28T11:20:55.765501+0000","last_became_peered":"2023-11-28T11:20:55.765501+0000","last_unstale":"2023-11-28T11:21:00.288230+0000","last_undegraded":"2023-11-28T11:21:00.288230+0000","last_fullsized":"2023-11-28T11:21:00.288230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:09:40.747993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.291211+0000","last_change":"2023-11-28T11:20:56.268222+0000","last_active":"2023-11-28T11:21:00.291211+0000","last_peered":"2023-11-28T11:21:00.291211+0000","last_clean":"2023-11-28T11:21:00.291211+0000","last_became_active":"2023-11-28T11:20:56.267973+0000","last_became_peered":"2023-11-28T11:20:56.267973+0000","last_unstale":"2023-11-28T11:21:00.291211+0000","last_undegraded":"2023-11-28T11:21:00.291211+0000","last_fullsized":"2023-11-28T11:21:00.291211+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:40:42.059656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.383115+0000","last_change":"2023-11-28T11:21:00.291222+0000","last_active":"2023-11-28T11:21:01.383115+0000","last_peered":"2023-11-28T11:21:01.383115+0000","last_clean":"2023-11-28T11:21:01.383115+0000","last_became_active":"2023-11-28T11:21:00.290768+0000","last_became_peered":"2023-11-28T11:21:00.290768+0000","last_unstale":"2023-11-28T11:21:01.383115+0000","last_undegraded":"2023-11-28T11:21:01.383115+0000","last_fullsized":"2023-11-28T11:21:01.383115+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:56:39.135021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.286149+0000","last_change":"2023-11-28T11:20:55.766427+0000","last_active":"2023-11-28T11:21:00.286149+0000","last_peered":"2023-11-28T11:21:00.286149+0000","last_clean":"2023-11-28T11:21:00.286149+0000","last_became_active":"2023-11-28T11:20:55.764723+0000","last_became_peered":"2023-11-28T11:20:55.764723+0000","last_unstale":"2023-11-28T11:21:00.286149+0000","last_undegraded":"2023-11-28T11:21:00.286149+0000","last_fullsized":"2023-11-28T11:21:00.286149+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:54:56.683286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.272205+0000","last_change":"2023-11-28T11:20:56.274667+0000","last_active":"2023-11-28T11:21:00.272205+0000","last_peered":"2023-11-28T11:21:00.272205+0000","last_clean":"2023-11-28T11:21:00.272205+0000","last_became_active":"2023-11-28T11:20:56.274325+0000","last_became_peered":"2023-11-28T11:20:56.274325+0000","last_unstale":"2023-11-28T11:21:00.272205+0000","last_undegraded":"2023-11-28T11:21:00.272205+0000","last_fullsized":"2023-11-28T11:21:00.272205+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:33:54.639487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.807008+0000","last_change":"2023-11-28T11:21:00.301042+0000","last_active":"2023-11-28T11:21:01.807008+0000","last_peered":"2023-11-28T11:21:01.807008+0000","last_clean":"2023-11-28T11:21:01.807008+0000","last_became_active":"2023-11-28T11:21:00.300581+0000","last_became_peered":"2023-11-28T11:21:00.300581+0000","last_unstale":"2023-11-28T11:21:01.807008+0000","last_undegraded":"2023-11-28T11:21:01.807008+0000","last_fullsized":"2023-11-28T11:21:01.807008+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:21:12.545633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.298437+0000","last_change":"2023-11-28T11:20:55.774510+0000","last_active":"2023-11-28T11:21:00.298437+0000","last_peered":"2023-11-28T11:21:00.298437+0000","last_clean":"2023-11-28T11:21:00.298437+0000","last_became_active":"2023-11-28T11:20:55.774211+0000","last_became_peered":"2023-11-28T11:20:55.774211+0000","last_unstale":"2023-11-28T11:21:00.298437+0000","last_undegraded":"2023-11-28T11:21:00.298437+0000","last_fullsized":"2023-11-28T11:21:00.298437+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:47:23.879025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.273372+0000","last_change":"2023-11-28T11:20:56.263432+0000","last_active":"2023-11-28T11:21:00.273372+0000","last_peered":"2023-11-28T11:21:00.273372+0000","last_clean":"2023-11-28T11:21:00.273372+0000","last_became_active":"2023-11-28T11:20:56.263125+0000","last_became_peered":"2023-11-28T11:20:56.263125+0000","last_unstale":"2023-11-28T11:21:00.273372+0000","last_undegraded":"2023-11-28T11:21:00.273372+0000","last_fullsized":"2023-11-28T11:21:00.273372+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:40:42.059656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.774944+0000","last_change":"2023-11-28T11:21:00.291421+0000","last_active":"2023-11-28T11:21:00.774944+0000","last_peered":"2023-11-28T11:21:00.774944+0000","last_clean":"2023-11-28T11:21:00.774944+0000","last_became_active":"2023-11-28T11:21:00.291077+0000","last_became_peered":"2023-11-28T11:21:00.291077+0000","last_unstale":"2023-11-28T11:21:00.774944+0000","last_undegraded":"2023-11-28T11:21:00.774944+0000","last_fullsized":"2023-11-28T11:21:00.774944+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:26:35.867941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.298743+0000","last_change":"2023-11-28T11:20:55.774653+0000","last_active":"2023-11-28T11:21:00.298743+0000","last_peered":"2023-11-28T11:21:00.298743+0000","last_clean":"2023-11-28T11:21:00.298743+0000","last_became_active":"2023-11-28T11:20:55.774352+0000","last_became_peered":"2023-11-28T11:20:55.774352+0000","last_unstale":"2023-11-28T11:21:00.298743+0000","last_undegraded":"2023-11-28T11:21:00.298743+0000","last_fullsized":"2023-11-28T11:21:00.298743+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:33:55.577608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.274658+0000","last_change":"2023-11-28T11:20:56.266409+0000","last_active":"2023-11-28T11:21:00.274658+0000","last_peered":"2023-11-28T11:21:00.274658+0000","last_clean":"2023-11-28T11:21:00.274658+0000","last_became_active":"2023-11-28T11:20:56.266091+0000","last_became_peered":"2023-11-28T11:20:56.266091+0000","last_unstale":"2023-11-28T11:21:00.274658+0000","last_undegraded":"2023-11-28T11:21:00.274658+0000","last_fullsized":"2023-11-28T11:21:00.274658+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:28:30.120683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.775175+0000","last_change":"2023-11-28T11:21:00.311439+0000","last_active":"2023-11-28T11:21:00.775175+0000","last_peered":"2023-11-28T11:21:00.775175+0000","last_clean":"2023-11-28T11:21:00.775175+0000","last_became_active":"2023-11-28T11:21:00.308732+0000","last_became_peered":"2023-11-28T11:21:00.308732+0000","last_unstale":"2023-11-28T11:21:00.775175+0000","last_undegraded":"2023-11-28T11:21:00.775175+0000","last_fullsized":"2023-11-28T11:21:00.775175+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:53:19.245981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.298398+0000","last_change":"2023-11-28T11:20:55.771105+0000","last_active":"2023-11-28T11:21:00.298398+0000","last_peered":"2023-11-28T11:21:00.298398+0000","last_clean":"2023-11-28T11:21:00.298398+0000","last_became_active":"2023-11-28T11:20:55.770583+0000","last_became_peered":"2023-11-28T11:20:55.770583+0000","last_unstale":"2023-11-28T11:21:00.298398+0000","last_undegraded":"2023-11-28T11:21:00.298398+0000","last_fullsized":"2023-11-28T11:21:00.298398+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:06:01.854141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.286992+0000","last_change":"2023-11-28T11:20:56.263719+0000","last_active":"2023-11-28T11:21:00.286992+0000","last_peered":"2023-11-28T11:21:00.286992+0000","last_clean":"2023-11-28T11:21:00.286992+0000","last_became_active":"2023-11-28T11:20:56.263444+0000","last_became_peered":"2023-11-28T11:20:56.263444+0000","last_unstale":"2023-11-28T11:21:00.286992+0000","last_undegraded":"2023-11-28T11:21:00.286992+0000","last_fullsized":"2023-11-28T11:21:00.286992+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:33:56.583532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.775522+0000","last_change":"2023-11-28T11:21:00.294534+0000","last_active":"2023-11-28T11:21:00.775522+0000","last_peered":"2023-11-28T11:21:00.775522+0000","last_clean":"2023-11-28T11:21:00.775522+0000","last_became_active":"2023-11-28T11:21:00.294055+0000","last_became_peered":"2023-11-28T11:21:00.294055+0000","last_unstale":"2023-11-28T11:21:00.775522+0000","last_undegraded":"2023-11-28T11:21:00.775522+0000","last_fullsized":"2023-11-28T11:21:00.775522+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:21:24.577104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.275166+0000","last_change":"2023-11-28T11:20:55.763501+0000","last_active":"2023-11-28T11:21:00.275166+0000","last_peered":"2023-11-28T11:21:00.275166+0000","last_clean":"2023-11-28T11:21:00.275166+0000","last_became_active":"2023-11-28T11:20:55.763188+0000","last_became_peered":"2023-11-28T11:20:55.763188+0000","last_unstale":"2023-11-28T11:21:00.275166+0000","last_undegraded":"2023-11-28T11:21:00.275166+0000","last_fullsized":"2023-11-28T11:21:00.275166+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:58:46.726052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.290571+0000","last_change":"2023-11-28T11:20:56.260904+0000","last_active":"2023-11-28T11:21:00.290571+0000","last_peered":"2023-11-28T11:21:00.290571+0000","last_clean":"2023-11-28T11:21:00.290571+0000","last_became_active":"2023-11-28T11:20:56.260584+0000","last_became_peered":"2023-11-28T11:20:56.260584+0000","last_unstale":"2023-11-28T11:21:00.290571+0000","last_undegraded":"2023-11-28T11:21:00.290571+0000","last_fullsized":"2023-11-28T11:21:00.290571+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:08.079216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.775218+0000","last_change":"2023-11-28T11:21:00.299500+0000","last_active":"2023-11-28T11:21:00.775218+0000","last_peered":"2023-11-28T11:21:00.775218+0000","last_clean":"2023-11-28T11:21:00.775218+0000","last_became_active":"2023-11-28T11:21:00.299065+0000","last_became_peered":"2023-11-28T11:21:00.299065+0000","last_unstale":"2023-11-28T11:21:00.775218+0000","last_undegraded":"2023-11-28T11:21:00.775218+0000","last_fullsized":"2023-11-28T11:21:00.775218+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:59:57.725780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.271583+0000","last_change":"2023-11-28T11:20:55.767855+0000","last_active":"2023-11-28T11:21:00.271583+0000","last_peered":"2023-11-28T11:21:00.271583+0000","last_clean":"2023-11-28T11:21:00.271583+0000","last_became_active":"2023-11-28T11:20:55.767418+0000","last_became_peered":"2023-11-28T11:20:55.767418+0000","last_unstale":"2023-11-28T11:21:00.271583+0000","last_undegraded":"2023-11-28T11:21:00.271583+0000","last_fullsized":"2023-11-28T11:21:00.271583+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:06:11.286885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.275680+0000","last_change":"2023-11-28T11:20:56.282762+0000","last_active":"2023-11-28T11:21:00.275680+0000","last_peered":"2023-11-28T11:21:00.275680+0000","last_clean":"2023-11-28T11:21:00.275680+0000","last_became_active":"2023-11-28T11:20:56.280729+0000","last_became_peered":"2023-11-28T11:20:56.280729+0000","last_unstale":"2023-11-28T11:21:00.275680+0000","last_undegraded":"2023-11-28T11:21:00.275680+0000","last_fullsized":"2023-11-28T11:21:00.275680+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:53:15.721370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.883819+0000","last_change":"2023-11-28T11:21:00.305663+0000","last_active":"2023-11-28T11:21:01.883819+0000","last_peered":"2023-11-28T11:21:01.883819+0000","last_clean":"2023-11-28T11:21:01.883819+0000","last_became_active":"2023-11-28T11:21:00.305308+0000","last_became_peered":"2023-11-28T11:21:00.305308+0000","last_unstale":"2023-11-28T11:21:01.883819+0000","last_undegraded":"2023-11-28T11:21:01.883819+0000","last_fullsized":"2023-11-28T11:21:01.883819+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:05:27.287399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.287091+0000","last_change":"2023-11-28T11:20:55.774271+0000","last_active":"2023-11-28T11:21:00.287091+0000","last_peered":"2023-11-28T11:21:00.287091+0000","last_clean":"2023-11-28T11:21:00.287091+0000","last_became_active":"2023-11-28T11:20:55.773041+0000","last_became_peered":"2023-11-28T11:20:55.773041+0000","last_unstale":"2023-11-28T11:21:00.287091+0000","last_undegraded":"2023-11-28T11:21:00.287091+0000","last_fullsized":"2023-11-28T11:21:00.287091+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:06:11.286885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.273977+0000","last_change":"2023-11-28T11:20:56.269860+0000","last_active":"2023-11-28T11:21:00.273977+0000","last_peered":"2023-11-28T11:21:00.273977+0000","last_clean":"2023-11-28T11:21:00.273977+0000","last_became_active":"2023-11-28T11:20:56.269355+0000","last_became_peered":"2023-11-28T11:20:56.269355+0000","last_unstale":"2023-11-28T11:21:00.273977+0000","last_undegraded":"2023-11-28T11:21:00.273977+0000","last_fullsized":"2023-11-28T11:21:00.273977+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:24:07.384730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.383366+0000","last_change":"2023-11-28T11:21:00.297532+0000","last_active":"2023-11-28T11:21:01.383366+0000","last_peered":"2023-11-28T11:21:01.383366+0000","last_clean":"2023-11-28T11:21:01.383366+0000","last_became_active":"2023-11-28T11:21:00.295523+0000","last_became_peered":"2023-11-28T11:21:00.295523+0000","last_unstale":"2023-11-28T11:21:01.383366+0000","last_undegraded":"2023-11-28T11:21:01.383366+0000","last_fullsized":"2023-11-28T11:21:01.383366+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:41.700110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.274453+0000","last_change":"2023-11-28T11:20:55.764095+0000","last_active":"2023-11-28T11:21:00.274453+0000","last_peered":"2023-11-28T11:21:00.274453+0000","last_clean":"2023-11-28T11:21:00.274453+0000","last_became_active":"2023-11-28T11:20:55.762342+0000","last_became_peered":"2023-11-28T11:20:55.762342+0000","last_unstale":"2023-11-28T11:21:00.274453+0000","last_undegraded":"2023-11-28T11:21:00.274453+0000","last_fullsized":"2023-11-28T11:21:00.274453+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:25:25.914486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.274835+0000","last_change":"2023-11-28T11:20:56.268448+0000","last_active":"2023-11-28T11:21:00.274835+0000","last_peered":"2023-11-28T11:21:00.274835+0000","last_clean":"2023-11-28T11:21:00.274835+0000","last_became_active":"2023-11-28T11:20:56.268150+0000","last_became_peered":"2023-11-28T11:20:56.268150+0000","last_unstale":"2023-11-28T11:21:00.274835+0000","last_undegraded":"2023-11-28T11:21:00.274835+0000","last_fullsized":"2023-11-28T11:21:00.274835+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:21:21.052493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.777908+0000","last_change":"2023-11-28T11:21:00.306271+0000","last_active":"2023-11-28T11:21:00.777908+0000","last_peered":"2023-11-28T11:21:00.777908+0000","last_clean":"2023-11-28T11:21:00.777908+0000","last_became_active":"2023-11-28T11:21:00.305835+0000","last_became_peered":"2023-11-28T11:21:00.305835+0000","last_unstale":"2023-11-28T11:21:00.777908+0000","last_undegraded":"2023-11-28T11:21:00.777908+0000","last_fullsized":"2023-11-28T11:21:00.777908+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:04:07.374394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.270977+0000","last_change":"2023-11-28T11:20:55.762830+0000","last_active":"2023-11-28T11:21:00.270977+0000","last_peered":"2023-11-28T11:21:00.270977+0000","last_clean":"2023-11-28T11:21:00.270977+0000","last_became_active":"2023-11-28T11:20:55.760841+0000","last_became_peered":"2023-11-28T11:20:55.760841+0000","last_unstale":"2023-11-28T11:21:00.270977+0000","last_undegraded":"2023-11-28T11:21:00.270977+0000","last_fullsized":"2023-11-28T11:21:00.270977+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:54:56.683286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.290757+0000","last_change":"2023-11-28T11:20:56.271356+0000","last_active":"2023-11-28T11:21:00.290757+0000","last_peered":"2023-11-28T11:21:00.290757+0000","last_clean":"2023-11-28T11:21:00.290757+0000","last_became_active":"2023-11-28T11:20:56.271110+0000","last_became_peered":"2023-11-28T11:20:56.271110+0000","last_unstale":"2023-11-28T11:21:00.290757+0000","last_undegraded":"2023-11-28T11:21:00.290757+0000","last_fullsized":"2023-11-28T11:21:00.290757+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:08.079216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.884172+0000","last_change":"2023-11-28T11:21:00.295227+0000","last_active":"2023-11-28T11:21:01.884172+0000","last_peered":"2023-11-28T11:21:01.884172+0000","last_clean":"2023-11-28T11:21:01.884172+0000","last_became_active":"2023-11-28T11:21:00.293463+0000","last_became_peered":"2023-11-28T11:21:00.293463+0000","last_unstale":"2023-11-28T11:21:01.884172+0000","last_undegraded":"2023-11-28T11:21:01.884172+0000","last_fullsized":"2023-11-28T11:21:01.884172+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:41.700110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.288397+0000","last_change":"2023-11-28T11:20:51.020016+0000","last_active":"2023-11-28T11:21:00.288397+0000","last_peered":"2023-11-28T11:21:00.288397+0000","last_clean":"2023-11-28T11:21:00.288397+0000","last_became_active":"2023-11-28T11:20:51.019492+0000","last_became_peered":"2023-11-28T11:20:51.019492+0000","last_unstale":"2023-11-28T11:21:00.288397+0000","last_undegraded":"2023-11-28T11:21:00.288397+0000","last_fullsized":"2023-11-28T11:21:00.288397+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:27.884538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.273192+0000","last_change":"2023-11-28T11:20:55.770921+0000","last_active":"2023-11-28T11:21:00.273192+0000","last_peered":"2023-11-28T11:21:00.273192+0000","last_clean":"2023-11-28T11:21:00.273192+0000","last_became_active":"2023-11-28T11:20:55.770693+0000","last_became_peered":"2023-11-28T11:20:55.770693+0000","last_unstale":"2023-11-28T11:21:00.273192+0000","last_undegraded":"2023-11-28T11:21:00.273192+0000","last_fullsized":"2023-11-28T11:21:00.273192+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:37:05.627049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.285261+0000","last_change":"2023-11-28T11:20:56.257635+0000","last_active":"2023-11-28T11:21:00.285261+0000","last_peered":"2023-11-28T11:21:00.285261+0000","last_clean":"2023-11-28T11:21:00.285261+0000","last_became_active":"2023-11-28T11:20:56.255047+0000","last_became_peered":"2023-11-28T11:20:56.255047+0000","last_unstale":"2023-11-28T11:21:00.285261+0000","last_undegraded":"2023-11-28T11:21:00.285261+0000","last_fullsized":"2023-11-28T11:21:00.285261+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:27:12.297507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.884415+0000","last_change":"2023-11-28T11:21:00.311287+0000","last_active":"2023-11-28T11:21:01.884415+0000","last_peered":"2023-11-28T11:21:01.884415+0000","last_clean":"2023-11-28T11:21:01.884415+0000","last_became_active":"2023-11-28T11:21:00.305808+0000","last_became_peered":"2023-11-28T11:21:00.305808+0000","last_unstale":"2023-11-28T11:21:01.884415+0000","last_undegraded":"2023-11-28T11:21:01.884415+0000","last_fullsized":"2023-11-28T11:21:01.884415+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:56:39.135021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.276695+0000","last_change":"2023-11-28T11:20:50.521908+0000","last_active":"2023-11-28T11:21:00.276695+0000","last_peered":"2023-11-28T11:21:00.276695+0000","last_clean":"2023-11-28T11:21:00.276695+0000","last_became_active":"2023-11-28T11:20:50.521059+0000","last_became_peered":"2023-11-28T11:20:50.521059+0000","last_unstale":"2023-11-28T11:21:00.276695+0000","last_undegraded":"2023-11-28T11:21:00.276695+0000","last_fullsized":"2023-11-28T11:21:00.276695+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:05.302498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.273078+0000","last_change":"2023-11-28T11:20:55.767339+0000","last_active":"2023-11-28T11:21:00.273078+0000","last_peered":"2023-11-28T11:21:00.273078+0000","last_clean":"2023-11-28T11:21:00.273078+0000","last_became_active":"2023-11-28T11:20:55.766065+0000","last_became_peered":"2023-11-28T11:20:55.766065+0000","last_unstale":"2023-11-28T11:21:00.273078+0000","last_undegraded":"2023-11-28T11:21:00.273078+0000","last_fullsized":"2023-11-28T11:21:00.273078+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:06:11.286885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.289236+0000","last_change":"2023-11-28T11:20:56.261611+0000","last_active":"2023-11-28T11:21:00.289236+0000","last_peered":"2023-11-28T11:21:00.289236+0000","last_clean":"2023-11-28T11:21:00.289236+0000","last_became_active":"2023-11-28T11:20:56.260557+0000","last_became_peered":"2023-11-28T11:20:56.260557+0000","last_unstale":"2023-11-28T11:21:00.289236+0000","last_undegraded":"2023-11-28T11:21:00.289236+0000","last_fullsized":"2023-11-28T11:21:00.289236+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:14:34.876987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.775674+0000","last_change":"2023-11-28T11:21:00.301087+0000","last_active":"2023-11-28T11:21:00.775674+0000","last_peered":"2023-11-28T11:21:00.775674+0000","last_clean":"2023-11-28T11:21:00.775674+0000","last_became_active":"2023-11-28T11:21:00.300674+0000","last_became_peered":"2023-11-28T11:21:00.300674+0000","last_unstale":"2023-11-28T11:21:00.775674+0000","last_undegraded":"2023-11-28T11:21:00.775674+0000","last_fullsized":"2023-11-28T11:21:00.775674+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:13:50.207611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.282488+0000","last_change":"2023-11-28T11:20:50.619004+0000","last_active":"2023-11-28T11:21:00.282488+0000","last_peered":"2023-11-28T11:21:00.282488+0000","last_clean":"2023-11-28T11:21:00.282488+0000","last_became_active":"2023-11-28T11:20:50.618315+0000","last_became_peered":"2023-11-28T11:20:50.618315+0000","last_unstale":"2023-11-28T11:21:00.282488+0000","last_undegraded":"2023-11-28T11:21:00.282488+0000","last_fullsized":"2023-11-28T11:21:00.282488+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:27.884538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.283998+0000","last_change":"2023-11-28T11:20:55.758384+0000","last_active":"2023-11-28T11:21:00.283998+0000","last_peered":"2023-11-28T11:21:00.283998+0000","last_clean":"2023-11-28T11:21:00.283998+0000","last_became_active":"2023-11-28T11:20:55.758125+0000","last_became_peered":"2023-11-28T11:20:55.758125+0000","last_unstale":"2023-11-28T11:21:00.283998+0000","last_undegraded":"2023-11-28T11:21:00.283998+0000","last_fullsized":"2023-11-28T11:21:00.283998+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:47:38.102588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.290667+0000","last_change":"2023-11-28T11:20:56.253940+0000","last_active":"2023-11-28T11:21:00.290667+0000","last_peered":"2023-11-28T11:21:00.290667+0000","last_clean":"2023-11-28T11:21:00.290667+0000","last_became_active":"2023-11-28T11:20:56.253635+0000","last_became_peered":"2023-11-28T11:20:56.253635+0000","last_unstale":"2023-11-28T11:21:00.290667+0000","last_undegraded":"2023-11-28T11:21:00.290667+0000","last_fullsized":"2023-11-28T11:21:00.290667+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:00:17.958443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.806956+0000","last_change":"2023-11-28T11:21:00.311172+0000","last_active":"2023-11-28T11:21:01.806956+0000","last_peered":"2023-11-28T11:21:01.806956+0000","last_clean":"2023-11-28T11:21:01.806956+0000","last_became_active":"2023-11-28T11:21:00.310707+0000","last_became_peered":"2023-11-28T11:21:00.310707+0000","last_unstale":"2023-11-28T11:21:01.806956+0000","last_undegraded":"2023-11-28T11:21:01.806956+0000","last_fullsized":"2023-11-28T11:21:01.806956+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:41:46.264594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.273954+0000","last_change":"2023-11-28T11:20:50.522582+0000","last_active":"2023-11-28T11:21:00.273954+0000","last_peered":"2023-11-28T11:21:00.273954+0000","last_clean":"2023-11-28T11:21:00.273954+0000","last_became_active":"2023-11-28T11:20:50.522157+0000","last_became_peered":"2023-11-28T11:20:50.522157+0000","last_unstale":"2023-11-28T11:21:00.273954+0000","last_undegraded":"2023-11-28T11:21:00.273954+0000","last_fullsized":"2023-11-28T11:21:00.273954+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:27.884538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.286501+0000","last_change":"2023-11-28T11:20:43.294845+0000","last_active":"2023-11-28T11:21:00.286501+0000","last_peered":"2023-11-28T11:21:00.286501+0000","last_clean":"2023-11-28T11:21:00.286501+0000","last_became_active":"2023-11-28T11:20:43.294483+0000","last_became_peered":"2023-11-28T11:20:43.294483+0000","last_unstale":"2023-11-28T11:21:00.286501+0000","last_undegraded":"2023-11-28T11:21:00.286501+0000","last_fullsized":"2023-11-28T11:21:00.286501+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:42.270512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:42.270512+0000","last_clean_scrub_stamp":"2023-11-28T11:20:42.270512+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:17:03.376680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.270966+0000","last_change":"2023-11-28T11:20:55.767420+0000","last_active":"2023-11-28T11:21:00.270966+0000","last_peered":"2023-11-28T11:21:00.270966+0000","last_clean":"2023-11-28T11:21:00.270966+0000","last_became_active":"2023-11-28T11:20:55.766809+0000","last_became_peered":"2023-11-28T11:20:55.766809+0000","last_unstale":"2023-11-28T11:21:00.270966+0000","last_undegraded":"2023-11-28T11:21:00.270966+0000","last_fullsized":"2023-11-28T11:21:00.270966+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:37:05.627049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.286972+0000","last_change":"2023-11-28T11:20:56.276228+0000","last_active":"2023-11-28T11:21:00.286972+0000","last_peered":"2023-11-28T11:21:00.286972+0000","last_clean":"2023-11-28T11:21:00.286972+0000","last_became_active":"2023-11-28T11:20:56.275905+0000","last_became_peered":"2023-11-28T11:20:56.275905+0000","last_unstale":"2023-11-28T11:21:00.286972+0000","last_undegraded":"2023-11-28T11:21:00.286972+0000","last_fullsized":"2023-11-28T11:21:00.286972+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:47:24.884949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.383236+0000","last_change":"2023-11-28T11:21:00.305959+0000","last_active":"2023-11-28T11:21:01.383236+0000","last_peered":"2023-11-28T11:21:01.383236+0000","last_clean":"2023-11-28T11:21:01.383236+0000","last_became_active":"2023-11-28T11:21:00.305530+0000","last_became_peered":"2023-11-28T11:21:00.305530+0000","last_unstale":"2023-11-28T11:21:01.383236+0000","last_undegraded":"2023-11-28T11:21:01.383236+0000","last_fullsized":"2023-11-28T11:21:01.383236+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:22:04.224877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.286683+0000","last_change":"2023-11-28T11:20:50.526865+0000","last_active":"2023-11-28T11:21:00.286683+0000","last_peered":"2023-11-28T11:21:00.286683+0000","last_clean":"2023-11-28T11:21:00.286683+0000","last_became_active":"2023-11-28T11:20:50.526576+0000","last_became_peered":"2023-11-28T11:20:50.526576+0000","last_unstale":"2023-11-28T11:21:00.286683+0000","last_undegraded":"2023-11-28T11:21:00.286683+0000","last_fullsized":"2023-11-28T11:21:00.286683+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:09:18.460112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.289849+0000","last_change":"2023-11-28T11:20:55.772865+0000","last_active":"2023-11-28T11:21:00.289849+0000","last_peered":"2023-11-28T11:21:00.289849+0000","last_clean":"2023-11-28T11:21:00.289849+0000","last_became_active":"2023-11-28T11:20:55.772559+0000","last_became_peered":"2023-11-28T11:20:55.772559+0000","last_unstale":"2023-11-28T11:21:00.289849+0000","last_undegraded":"2023-11-28T11:21:00.289849+0000","last_fullsized":"2023-11-28T11:21:00.289849+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:37:05.627049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.290171+0000","last_change":"2023-11-28T11:20:56.268570+0000","last_active":"2023-11-28T11:21:00.290171+0000","last_peered":"2023-11-28T11:21:00.290171+0000","last_clean":"2023-11-28T11:21:00.290171+0000","last_became_active":"2023-11-28T11:20:56.264598+0000","last_became_peered":"2023-11-28T11:20:56.264598+0000","last_unstale":"2023-11-28T11:21:00.290171+0000","last_undegraded":"2023-11-28T11:21:00.290171+0000","last_fullsized":"2023-11-28T11:21:00.290171+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:14:34.876987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.778849+0000","last_change":"2023-11-28T11:21:00.311861+0000","last_active":"2023-11-28T11:21:00.778849+0000","last_peered":"2023-11-28T11:21:00.778849+0000","last_clean":"2023-11-28T11:21:00.778849+0000","last_became_active":"2023-11-28T11:21:00.311394+0000","last_became_peered":"2023-11-28T11:21:00.311394+0000","last_unstale":"2023-11-28T11:21:00.778849+0000","last_undegraded":"2023-11-28T11:21:00.778849+0000","last_fullsized":"2023-11-28T11:21:00.778849+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:25:55.767021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.298282+0000","last_change":"2023-11-28T11:20:51.020043+0000","last_active":"2023-11-28T11:21:00.298282+0000","last_peered":"2023-11-28T11:21:00.298282+0000","last_clean":"2023-11-28T11:21:00.298282+0000","last_became_active":"2023-11-28T11:20:51.019702+0000","last_became_peered":"2023-11-28T11:20:51.019702+0000","last_unstale":"2023-11-28T11:21:00.298282+0000","last_undegraded":"2023-11-28T11:21:00.298282+0000","last_fullsized":"2023-11-28T11:21:00.298282+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:26:20.246873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.288664+0000","last_change":"2023-11-28T11:20:55.760221+0000","last_active":"2023-11-28T11:21:00.288664+0000","last_peered":"2023-11-28T11:21:00.288664+0000","last_clean":"2023-11-28T11:21:00.288664+0000","last_became_active":"2023-11-28T11:20:55.759960+0000","last_became_peered":"2023-11-28T11:20:55.759960+0000","last_unstale":"2023-11-28T11:21:00.288664+0000","last_undegraded":"2023-11-28T11:21:00.288664+0000","last_fullsized":"2023-11-28T11:21:00.288664+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:54:56.683286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.290072+0000","last_change":"2023-11-28T11:20:56.251820+0000","last_active":"2023-11-28T11:21:00.290072+0000","last_peered":"2023-11-28T11:21:00.290072+0000","last_clean":"2023-11-28T11:21:00.290072+0000","last_became_active":"2023-11-28T11:20:56.251550+0000","last_became_peered":"2023-11-28T11:20:56.251550+0000","last_unstale":"2023-11-28T11:21:00.290072+0000","last_undegraded":"2023-11-28T11:21:00.290072+0000","last_fullsized":"2023-11-28T11:21:00.290072+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:45:14.478416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.777012+0000","last_change":"2023-11-28T11:21:00.298541+0000","last_active":"2023-11-28T11:21:00.777012+0000","last_peered":"2023-11-28T11:21:00.777012+0000","last_clean":"2023-11-28T11:21:00.777012+0000","last_became_active":"2023-11-28T11:21:00.298160+0000","last_became_peered":"2023-11-28T11:21:00.298160+0000","last_unstale":"2023-11-28T11:21:00.777012+0000","last_undegraded":"2023-11-28T11:21:00.777012+0000","last_fullsized":"2023-11-28T11:21:00.777012+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:16:48.936607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.272276+0000","last_change":"2023-11-28T11:20:55.766385+0000","last_active":"2023-11-28T11:21:00.272276+0000","last_peered":"2023-11-28T11:21:00.272276+0000","last_clean":"2023-11-28T11:21:00.272276+0000","last_became_active":"2023-11-28T11:20:55.766148+0000","last_became_peered":"2023-11-28T11:20:55.766148+0000","last_unstale":"2023-11-28T11:21:00.272276+0000","last_undegraded":"2023-11-28T11:21:00.272276+0000","last_fullsized":"2023-11-28T11:21:00.272276+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:33:02.765966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.288602+0000","last_change":"2023-11-28T11:20:50.524251+0000","last_active":"2023-11-28T11:21:00.288602+0000","last_peered":"2023-11-28T11:21:00.288602+0000","last_clean":"2023-11-28T11:21:00.288602+0000","last_became_active":"2023-11-28T11:20:50.524012+0000","last_became_peered":"2023-11-28T11:20:50.524012+0000","last_unstale":"2023-11-28T11:21:00.288602+0000","last_undegraded":"2023-11-28T11:21:00.288602+0000","last_fullsized":"2023-11-28T11:21:00.288602+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:05.302498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.270885+0000","last_change":"2023-11-28T11:20:56.266167+0000","last_active":"2023-11-28T11:21:00.270885+0000","last_peered":"2023-11-28T11:21:00.270885+0000","last_clean":"2023-11-28T11:21:00.270885+0000","last_became_active":"2023-11-28T11:20:56.264890+0000","last_became_peered":"2023-11-28T11:20:56.264890+0000","last_unstale":"2023-11-28T11:21:00.270885+0000","last_undegraded":"2023-11-28T11:21:00.270885+0000","last_fullsized":"2023-11-28T11:21:00.270885+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:31:26.827850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.843672+0000","last_change":"2023-11-28T11:21:00.295193+0000","last_active":"2023-11-28T11:21:01.843672+0000","last_peered":"2023-11-28T11:21:01.843672+0000","last_clean":"2023-11-28T11:21:01.843672+0000","last_became_active":"2023-11-28T11:21:00.294668+0000","last_became_peered":"2023-11-28T11:21:00.294668+0000","last_unstale":"2023-11-28T11:21:01.843672+0000","last_undegraded":"2023-11-28T11:21:01.843672+0000","last_fullsized":"2023-11-28T11:21:01.843672+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:53:58.267145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.287437+0000","last_change":"2023-11-28T11:20:50.524838+0000","last_active":"2023-11-28T11:21:00.287437+0000","last_peered":"2023-11-28T11:21:00.287437+0000","last_clean":"2023-11-28T11:21:00.287437+0000","last_became_active":"2023-11-28T11:20:50.524402+0000","last_became_peered":"2023-11-28T11:20:50.524402+0000","last_unstale":"2023-11-28T11:21:00.287437+0000","last_undegraded":"2023-11-28T11:21:00.287437+0000","last_fullsized":"2023-11-28T11:21:00.287437+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:22.699447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"26'1","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.725299+0000","last_change":"2023-11-28T11:20:55.765528+0000","last_active":"2023-11-28T11:21:00.725299+0000","last_peered":"2023-11-28T11:21:00.725299+0000","last_clean":"2023-11-28T11:21:00.725299+0000","last_became_active":"2023-11-28T11:20:55.765244+0000","last_became_peered":"2023-11-28T11:20:55.765244+0000","last_unstale":"2023-11-28T11:21:00.725299+0000","last_undegraded":"2023-11-28T11:21:00.725299+0000","last_fullsized":"2023-11-28T11:21:00.725299+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:47:38.102588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.274546+0000","last_change":"2023-11-28T11:20:56.275241+0000","last_active":"2023-11-28T11:21:00.274546+0000","last_peered":"2023-11-28T11:21:00.274546+0000","last_clean":"2023-11-28T11:21:00.274546+0000","last_became_active":"2023-11-28T11:20:56.271190+0000","last_became_peered":"2023-11-28T11:20:56.271190+0000","last_unstale":"2023-11-28T11:21:00.274546+0000","last_undegraded":"2023-11-28T11:21:00.274546+0000","last_fullsized":"2023-11-28T11:21:00.274546+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:45:14.478416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.883997+0000","last_change":"2023-11-28T11:21:00.297029+0000","last_active":"2023-11-28T11:21:01.883997+0000","last_peered":"2023-11-28T11:21:01.883997+0000","last_clean":"2023-11-28T11:21:01.883997+0000","last_became_active":"2023-11-28T11:21:00.296497+0000","last_became_peered":"2023-11-28T11:21:00.296497+0000","last_unstale":"2023-11-28T11:21:01.883997+0000","last_undegraded":"2023-11-28T11:21:01.883997+0000","last_fullsized":"2023-11-28T11:21:01.883997+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:41.700110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.290180+0000","last_change":"2023-11-28T11:20:55.773378+0000","last_active":"2023-11-28T11:21:00.290180+0000","last_peered":"2023-11-28T11:21:00.290180+0000","last_clean":"2023-11-28T11:21:00.290180+0000","last_became_active":"2023-11-28T11:20:55.773095+0000","last_became_peered":"2023-11-28T11:20:55.773095+0000","last_unstale":"2023-11-28T11:21:00.290180+0000","last_undegraded":"2023-11-28T11:21:00.290180+0000","last_fullsized":"2023-11-28T11:21:00.290180+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:54:56.683286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.280606+0000","last_change":"2023-11-28T11:20:56.258626+0000","last_active":"2023-11-28T11:21:00.280606+0000","last_peered":"2023-11-28T11:21:00.280606+0000","last_clean":"2023-11-28T11:21:00.280606+0000","last_became_active":"2023-11-28T11:20:56.258350+0000","last_became_peered":"2023-11-28T11:20:56.258350+0000","last_unstale":"2023-11-28T11:21:00.280606+0000","last_undegraded":"2023-11-28T11:21:00.280606+0000","last_fullsized":"2023-11-28T11:21:00.280606+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:33:03.771889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.806818+0000","last_change":"2023-11-28T11:21:00.306341+0000","last_active":"2023-11-28T11:21:01.806818+0000","last_peered":"2023-11-28T11:21:01.806818+0000","last_clean":"2023-11-28T11:21:01.806818+0000","last_became_active":"2023-11-28T11:21:00.304545+0000","last_became_peered":"2023-11-28T11:21:00.304545+0000","last_unstale":"2023-11-28T11:21:01.806818+0000","last_undegraded":"2023-11-28T11:21:01.806818+0000","last_fullsized":"2023-11-28T11:21:01.806818+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:53:58.267145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.275266+0000","last_change":"2023-11-28T11:20:55.766530+0000","last_active":"2023-11-28T11:21:00.275266+0000","last_peered":"2023-11-28T11:21:00.275266+0000","last_clean":"2023-11-28T11:21:00.275266+0000","last_became_active":"2023-11-28T11:20:55.766205+0000","last_became_peered":"2023-11-28T11:20:55.766205+0000","last_unstale":"2023-11-28T11:21:00.275266+0000","last_undegraded":"2023-11-28T11:21:00.275266+0000","last_fullsized":"2023-11-28T11:21:00.275266+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:33:02.765966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.273248+0000","last_change":"2023-11-28T11:20:56.279673+0000","last_active":"2023-11-28T11:21:00.273248+0000","last_peered":"2023-11-28T11:21:00.273248+0000","last_clean":"2023-11-28T11:21:00.273248+0000","last_became_active":"2023-11-28T11:20:56.276338+0000","last_became_peered":"2023-11-28T11:20:56.276338+0000","last_unstale":"2023-11-28T11:21:00.273248+0000","last_undegraded":"2023-11-28T11:21:00.273248+0000","last_fullsized":"2023-11-28T11:21:00.273248+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:59:54.201169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.382912+0000","last_change":"2023-11-28T11:21:00.308516+0000","last_active":"2023-11-28T11:21:01.382912+0000","last_peered":"2023-11-28T11:21:01.382912+0000","last_clean":"2023-11-28T11:21:01.382912+0000","last_became_active":"2023-11-28T11:21:00.307909+0000","last_became_peered":"2023-11-28T11:21:00.307909+0000","last_unstale":"2023-11-28T11:21:01.382912+0000","last_undegraded":"2023-11-28T11:21:01.382912+0000","last_fullsized":"2023-11-28T11:21:01.382912+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:53:42.920160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.273087+0000","last_change":"2023-11-28T11:20:55.772607+0000","last_active":"2023-11-28T11:21:00.273087+0000","last_peered":"2023-11-28T11:21:00.273087+0000","last_clean":"2023-11-28T11:21:00.273087+0000","last_became_active":"2023-11-28T11:20:55.772335+0000","last_became_peered":"2023-11-28T11:20:55.772335+0000","last_unstale":"2023-11-28T11:21:00.273087+0000","last_undegraded":"2023-11-28T11:21:00.273087+0000","last_fullsized":"2023-11-28T11:21:00.273087+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:23:36.801918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.274556+0000","last_change":"2023-11-28T11:20:56.279846+0000","last_active":"2023-11-28T11:21:00.274556+0000","last_peered":"2023-11-28T11:21:00.274556+0000","last_clean":"2023-11-28T11:21:00.274556+0000","last_became_active":"2023-11-28T11:20:56.276529+0000","last_became_peered":"2023-11-28T11:20:56.276529+0000","last_unstale":"2023-11-28T11:21:00.274556+0000","last_undegraded":"2023-11-28T11:21:00.274556+0000","last_fullsized":"2023-11-28T11:21:00.274556+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:29:57.585501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.776665+0000","last_change":"2023-11-28T11:21:00.294235+0000","last_active":"2023-11-28T11:21:00.776665+0000","last_peered":"2023-11-28T11:21:00.776665+0000","last_clean":"2023-11-28T11:21:00.776665+0000","last_became_active":"2023-11-28T11:21:00.293813+0000","last_became_peered":"2023-11-28T11:21:00.293813+0000","last_unstale":"2023-11-28T11:21:00.776665+0000","last_undegraded":"2023-11-28T11:21:00.776665+0000","last_fullsized":"2023-11-28T11:21:00.776665+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:44:23.280744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.271358+0000","last_change":"2023-11-28T11:20:55.767813+0000","last_active":"2023-11-28T11:21:00.271358+0000","last_peered":"2023-11-28T11:21:00.271358+0000","last_clean":"2023-11-28T11:21:00.271358+0000","last_became_active":"2023-11-28T11:20:55.765782+0000","last_became_peered":"2023-11-28T11:20:55.765782+0000","last_unstale":"2023-11-28T11:21:00.271358+0000","last_undegraded":"2023-11-28T11:21:00.271358+0000","last_fullsized":"2023-11-28T11:21:00.271358+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:47:38.102588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.271000+0000","last_change":"2023-11-28T11:20:56.265782+0000","last_active":"2023-11-28T11:21:00.271000+0000","last_peered":"2023-11-28T11:21:00.271000+0000","last_clean":"2023-11-28T11:21:00.271000+0000","last_became_active":"2023-11-28T11:20:56.265102+0000","last_became_peered":"2023-11-28T11:20:56.265102+0000","last_unstale":"2023-11-28T11:21:00.271000+0000","last_undegraded":"2023-11-28T11:21:00.271000+0000","last_fullsized":"2023-11-28T11:21:00.271000+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:45:14.478416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.883772+0000","last_change":"2023-11-28T11:21:00.297751+0000","last_active":"2023-11-28T11:21:01.883772+0000","last_peered":"2023-11-28T11:21:01.883772+0000","last_clean":"2023-11-28T11:21:01.883772+0000","last_became_active":"2023-11-28T11:21:00.297281+0000","last_became_peered":"2023-11-28T11:21:00.297281+0000","last_unstale":"2023-11-28T11:21:01.883772+0000","last_undegraded":"2023-11-28T11:21:01.883772+0000","last_fullsized":"2023-11-28T11:21:01.883772+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:42:41.996906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.292230+0000","last_change":"2023-11-28T11:20:55.764439+0000","last_active":"2023-11-28T11:21:00.292230+0000","last_peered":"2023-11-28T11:21:00.292230+0000","last_clean":"2023-11-28T11:21:00.292230+0000","last_became_active":"2023-11-28T11:20:55.764165+0000","last_became_peered":"2023-11-28T11:20:55.764165+0000","last_unstale":"2023-11-28T11:21:00.292230+0000","last_undegraded":"2023-11-28T11:21:00.292230+0000","last_fullsized":"2023-11-28T11:21:00.292230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:33:02.765966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.273131+0000","last_change":"2023-11-28T11:20:56.265074+0000","last_active":"2023-11-28T11:21:00.273131+0000","last_peered":"2023-11-28T11:21:00.273131+0000","last_clean":"2023-11-28T11:21:00.273131+0000","last_became_active":"2023-11-28T11:20:56.264495+0000","last_became_peered":"2023-11-28T11:20:56.264495+0000","last_unstale":"2023-11-28T11:21:00.273131+0000","last_undegraded":"2023-11-28T11:21:00.273131+0000","last_fullsized":"2023-11-28T11:21:00.273131+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:31:26.827850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.286246+0000","last_change":"2023-11-28T11:20:56.260972+0000","last_active":"2023-11-28T11:21:00.286246+0000","last_peered":"2023-11-28T11:21:00.286246+0000","last_clean":"2023-11-28T11:21:00.286246+0000","last_became_active":"2023-11-28T11:20:56.259491+0000","last_became_peered":"2023-11-28T11:20:56.259491+0000","last_unstale":"2023-11-28T11:21:00.286246+0000","last_undegraded":"2023-11-28T11:21:00.286246+0000","last_fullsized":"2023-11-28T11:21:00.286246+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:06:02.860064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.271158+0000","last_change":"2023-11-28T11:20:56.263917+0000","last_active":"2023-11-28T11:21:00.271158+0000","last_peered":"2023-11-28T11:21:00.271158+0000","last_clean":"2023-11-28T11:21:00.271158+0000","last_became_active":"2023-11-28T11:20:56.261946+0000","last_became_peered":"2023-11-28T11:20:56.261946+0000","last_unstale":"2023-11-28T11:21:00.271158+0000","last_undegraded":"2023-11-28T11:21:00.271158+0000","last_fullsized":"2023-11-28T11:21:00.271158+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:40:42.059656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.274852+0000","last_change":"2023-11-28T11:20:56.272089+0000","last_active":"2023-11-28T11:21:00.274852+0000","last_peered":"2023-11-28T11:21:00.274852+0000","last_clean":"2023-11-28T11:21:00.274852+0000","last_became_active":"2023-11-28T11:20:56.271704+0000","last_became_peered":"2023-11-28T11:20:56.271704+0000","last_unstale":"2023-11-28T11:21:00.274852+0000","last_undegraded":"2023-11-28T11:21:00.274852+0000","last_fullsized":"2023-11-28T11:21:00.274852+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:33:54.639487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.270968+0000","last_change":"2023-11-28T11:20:56.270543+0000","last_active":"2023-11-28T11:21:00.270968+0000","last_peered":"2023-11-28T11:21:00.270968+0000","last_clean":"2023-11-28T11:21:00.270968+0000","last_became_active":"2023-11-28T11:20:56.270310+0000","last_became_peered":"2023-11-28T11:20:56.270310+0000","last_unstale":"2023-11-28T11:21:00.270968+0000","last_undegraded":"2023-11-28T11:21:00.270968+0000","last_fullsized":"2023-11-28T11:21:00.270968+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:21:21.052493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.288589+0000","last_change":"2023-11-28T11:20:56.262145+0000","last_active":"2023-11-28T11:21:00.288589+0000","last_peered":"2023-11-28T11:21:00.288589+0000","last_clean":"2023-11-28T11:21:00.288589+0000","last_became_active":"2023-11-28T11:20:56.260881+0000","last_became_peered":"2023-11-28T11:20:56.260881+0000","last_unstale":"2023-11-28T11:21:00.288589+0000","last_undegraded":"2023-11-28T11:21:00.288589+0000","last_fullsized":"2023-11-28T11:21:00.288589+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:32:52.386018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4528,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9280,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362560,"statfs":{"total":96636764160,"available":96627261440,"internally_reserved":0,"allocated":909312,"data_stored":713609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73999999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8100,"kb_used_data":284,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363740,"statfs":{"total":96636764160,"available":96628469760,"internally_reserved":0,"allocated":290816,"data_stored":119901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81200000000000006}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88800000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8164,"kb_used_data":284,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363676,"statfs":{"total":96636764160,"available":96628404224,"internally_reserved":0,"allocated":290816,"data_stored":119901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9260,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362580,"statfs":{"total":96636764160,"available":96627281920,"internally_reserved":0,"allocated":888832,"data_stored":710835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8100,"kb_used_data":284,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363740,"statfs":{"total":96636764160,"available":96628469760,"internally_reserved":0,"allocated":290816,"data_stored":119901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79700000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73999999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8168,"kb_used_data":288,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363672,"statfs":{"total":96636764160,"available":96628400128,"internally_reserved":0,"allocated":294912,"data_stored":119991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8248,"kb_used_data":304,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363592,"statfs":{"total":96636764160,"available":96628318208,"internally_reserved":0,"allocated":311296,"data_stored":123703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8172,"kb_used_data":292,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363668,"statfs":{"total":96636764160,"available":96628396032,"internally_reserved":0,"allocated":299008,"data_stored":120537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:21:18.406 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T11:21:18.406 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-11-28T11:21:18.406 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-11-28T11:21:18.407 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-11-28T11:21:18.407 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-11-28T11:21:18.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.635+0000 7f1df49ff700 1 Processor -- start 2023-11-28T11:21:18.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.636+0000 7f1df49ff700 1 -- start start 2023-11-28T11:21:18.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.637+0000 7f1df49ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1df00cd460 0x7f1df006c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:18.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.637+0000 7f1df49ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1df00d0d60 0x7f1df00d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:18.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.637+0000 7f1df49ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1df00b88e0 con 0x7f1df00b8510 2023-11-28T11:21:18.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.637+0000 7f1df49ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1df00cdbd0 con 0x7f1df00cd460 2023-11-28T11:21:18.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.637+0000 7f1df49ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1df00d02d0 con 0x7f1df00d0d60 2023-11-28T11:21:18.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.637+0000 7f1dee59c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f1df00b8510 0x7f1df006b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46200/0 (socket says 172.21.15.92:46200) 2023-11-28T11:21:18.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.637+0000 7f1dee59c700 1 -- 172.21.15.92:0/2753311405 learned_addr learned my addr 172.21.15.92:0/2753311405 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:18.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.637+0000 7f1deed9d700 1 --2- 172.21.15.92:0/2753311405 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1df00d0d60 0x7f1df00d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:18.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.637+0000 7f1de7fff700 1 --2- 172.21.15.92:0/2753311405 >> v2:172.21.15.169:3300/0 conn(0x7f1df00cd460 0x7f1df006c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:18.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.637+0000 7f1dedd9b700 1 -- 172.21.15.92:0/2753311405 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1637885019 0 0) 0x7f1df00b88e0 con 0x7f1df00b8510 2023-11-28T11:21:18.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.637+0000 7f1dedd9b700 1 -- 172.21.15.92:0/2753311405 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ddc0035b0 con 0x7f1df00b8510 2023-11-28T11:21:18.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.638+0000 7f1deed9d700 1 -- 172.21.15.92:0/2753311405 >> v2:172.21.15.169:3300/0 conn(0x7f1df00cd460 msgr2=0x7f1df006c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:18.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.638+0000 7f1deed9d700 1 --2- 172.21.15.92:0/2753311405 >> v2:172.21.15.169:3300/0 conn(0x7f1df00cd460 0x7f1df006c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:18.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.638+0000 7f1deed9d700 1 -- 172.21.15.92:0/2753311405 >> v1:172.21.15.92:6789/0 conn(0x7f1df00b8510 legacy=0x7f1df006b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:18.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.638+0000 7f1deed9d700 1 -- 172.21.15.92:0/2753311405 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df00cd8f0 con 0x7f1df00d0d60 2023-11-28T11:21:18.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.638+0000 7f1de7fff700 1 --2- 172.21.15.92:0/2753311405 >> v2:172.21.15.169:3300/0 conn(0x7f1df00cd460 0x7f1df006c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:21:18.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.638+0000 7f1deed9d700 1 --2- 172.21.15.92:0/2753311405 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1df00d0d60 0x7f1df00d1200 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1de8009d90 tx=0x7f1de800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22fca06de31f0f56 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:18.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.638+0000 7f1dedd9b700 1 -- 172.21.15.92:0/2753311405 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de800e070 con 0x7f1df00d0d60 2023-11-28T11:21:18.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.638+0000 7f1dedd9b700 1 -- 172.21.15.92:0/2753311405 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1de800bb20 con 0x7f1df00d0d60 2023-11-28T11:21:18.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.638+0000 7f1dedd9b700 1 -- 172.21.15.92:0/2753311405 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de8012430 con 0x7f1df00d0d60 2023-11-28T11:21:18.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.639+0000 7f1df49ff700 1 -- 172.21.15.92:0/2753311405 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1df00d0d60 msgr2=0x7f1df00d1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:18.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.639+0000 7f1df49ff700 1 --2- 172.21.15.92:0/2753311405 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1df00d0d60 0x7f1df00d1200 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1de8009d90 tx=0x7f1de800b330 comp rx=0 tx=0).stop 2023-11-28T11:21:18.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.639+0000 7f1df49ff700 1 -- 172.21.15.92:0/2753311405 shutdown_connections 2023-11-28T11:21:18.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.639+0000 7f1df49ff700 1 --2- 172.21.15.92:0/2753311405 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1df00d0d60 0x7f1df00d1200 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:18.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.639+0000 7f1df49ff700 1 --2- 172.21.15.92:0/2753311405 >> v2:172.21.15.169:3300/0 conn(0x7f1df00cd460 0x7f1df006c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:18.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.639+0000 7f1df49ff700 1 -- 172.21.15.92:0/2753311405 >> 172.21.15.92:0/2753311405 conn(0x7f1df00719a0 msgr2=0x7f1df0071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:18.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.639+0000 7f1df49ff700 1 -- 172.21.15.92:0/2753311405 shutdown_connections 2023-11-28T11:21:18.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.639+0000 7f1df49ff700 1 -- 172.21.15.92:0/2753311405 wait complete. 2023-11-28T11:21:18.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.641+0000 7f1df49ff700 1 Processor -- start 2023-11-28T11:21:18.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.641+0000 7f1df49ff700 1 -- start start 2023-11-28T11:21:18.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.641+0000 7f1df49ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1df00b8510 0x7f1df01e3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:18.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.642+0000 7f1df49ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1df00cd460 0x7f1df015a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:18.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.642+0000 7f1df49ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1df00d0d60 0x7f1df0149750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:18.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.642+0000 7f1df49ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df01e4150 con 0x7f1df00b8510 2023-11-28T11:21:18.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.642+0000 7f1df49ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df0148f70 con 0x7f1df00d0d60 2023-11-28T11:21:18.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.642+0000 7f1df49ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1df01e43d0 con 0x7f1df00cd460 2023-11-28T11:21:18.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.642+0000 7f1dee59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1df00b8510 0x7f1df01e3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:18.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.642+0000 7f1dee59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1df00b8510 0x7f1df01e3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49128/0 (socket says 172.21.15.92:49128) 2023-11-28T11:21:18.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.642+0000 7f1dee59c700 1 -- 172.21.15.92:0/2123963042 learned_addr learned my addr 172.21.15.92:0/2123963042 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:18.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.642+0000 7f1de7fff700 1 --2- 172.21.15.92:0/2123963042 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1df00cd460 0x7f1df015a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:18.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.642+0000 7f1deed9d700 1 --2- 172.21.15.92:0/2123963042 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1df00d0d60 0x7f1df0149750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:18.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.642+0000 7f1dee59c700 1 -- 172.21.15.92:0/2123963042 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1df00cd460 msgr2=0x7f1df015a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:18.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.642+0000 7f1dee59c700 1 --2- 172.21.15.92:0/2123963042 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1df00cd460 0x7f1df015a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:18.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.642+0000 7f1dee59c700 1 -- 172.21.15.92:0/2123963042 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1df00d0d60 msgr2=0x7f1df0149750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:18.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.642+0000 7f1dee59c700 1 --2- 172.21.15.92:0/2123963042 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1df00d0d60 0x7f1df0149750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:18.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.642+0000 7f1dee59c700 1 -- 172.21.15.92:0/2123963042 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd80094a0 con 0x7f1df00b8510 2023-11-28T11:21:18.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.642+0000 7f1deed9d700 1 --2- 172.21.15.92:0/2123963042 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1df00d0d60 0x7f1df0149750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:21:18.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.642+0000 7f1dee59c700 1 --2- 172.21.15.92:0/2123963042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1df00b8510 0x7f1df01e3c10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1de000c700 tx=0x7f1de000ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:18.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.643+0000 7f1de6ffd700 1 -- 172.21.15.92:0/2123963042 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de0005d70 con 0x7f1df00b8510 2023-11-28T11:21:18.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.643+0000 7f1de6ffd700 1 -- 172.21.15.92:0/2123963042 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1de00070d0 con 0x7f1df00b8510 2023-11-28T11:21:18.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.643+0000 7f1de6ffd700 1 -- 172.21.15.92:0/2123963042 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de0007460 con 0x7f1df00b8510 2023-11-28T11:21:18.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.643+0000 7f1df49ff700 1 -- 172.21.15.92:0/2123963042 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1de8009a70 con 0x7f1df00b8510 2023-11-28T11:21:18.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.643+0000 7f1df49ff700 1 -- 172.21.15.92:0/2123963042 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1df00cfff0 con 0x7f1df00b8510 2023-11-28T11:21:18.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.644+0000 7f1df49ff700 1 -- 172.21.15.92:0/2123963042 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1df00cfd10 con 0x7f1df00b8510 2023-11-28T11:21:18.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.645+0000 7f1de6ffd700 1 -- 172.21.15.92:0/2123963042 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1de0019030 con 0x7f1df00b8510 2023-11-28T11:21:18.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.646+0000 7f1de6ffd700 1 --2- 172.21.15.92:0/2123963042 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f1dd0068320 0x7f1dd006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:18.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.646+0000 7f1de6ffd700 1 -- 172.21.15.92:0/2123963042 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1de00854a0 con 0x7f1df00b8510 2023-11-28T11:21:18.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.646+0000 7f1de7fff700 1 --2- 172.21.15.92:0/2123963042 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f1dd0068320 0x7f1dd006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:18.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.647+0000 7f1de7fff700 1 --2- 172.21.15.92:0/2123963042 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f1dd0068320 0x7f1dd006a7d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1df0146db0 tx=0x7f1dd8009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:18.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.651+0000 7f1de6ffd700 1 -- 172.21.15.92:0/2123963042 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1de0051890 con 0x7f1df00b8510 2023-11-28T11:21:18.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.895+0000 7f1df49ff700 1 -- 172.21.15.92:0/2123963042 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f1df0067880 con 0x7f1df00b8510 2023-11-28T11:21:18.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.898+0000 7f1de6ffd700 1 -- 172.21.15.92:0/2123963042 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+166377 (secure 0 0 0) 0x7f1de0017090 con 0x7f1df00b8510 2023-11-28T11:21:18.903 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:21:18.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.901+0000 7f1df49ff700 1 -- 172.21.15.92:0/2123963042 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f1dd0068320 msgr2=0x7f1dd006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:18.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.902+0000 7f1df49ff700 1 --2- 172.21.15.92:0/2123963042 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f1dd0068320 0x7f1dd006a7d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1df0146db0 tx=0x7f1dd8009210 comp rx=0 tx=0).stop 2023-11-28T11:21:18.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.902+0000 7f1df49ff700 1 -- 172.21.15.92:0/2123963042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1df00b8510 msgr2=0x7f1df01e3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:18.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.902+0000 7f1df49ff700 1 --2- 172.21.15.92:0/2123963042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1df00b8510 0x7f1df01e3c10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1de000c700 tx=0x7f1de000ca10 comp rx=0 tx=0).stop 2023-11-28T11:21:18.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.902+0000 7f1df49ff700 1 -- 172.21.15.92:0/2123963042 shutdown_connections 2023-11-28T11:21:18.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.902+0000 7f1df49ff700 1 --2- 172.21.15.92:0/2123963042 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1df00d0d60 0x7f1df0149750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:18.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.902+0000 7f1df49ff700 1 --2- 172.21.15.92:0/2123963042 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f1dd0068320 0x7f1dd006a7d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:18.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.902+0000 7f1df49ff700 1 --2- 172.21.15.92:0/2123963042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1df00b8510 0x7f1df01e3c10 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:18.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.902+0000 7f1df49ff700 1 --2- 172.21.15.92:0/2123963042 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1df00cd460 0x7f1df015a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:18.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.902+0000 7f1df49ff700 1 -- 172.21.15.92:0/2123963042 >> 172.21.15.92:0/2123963042 conn(0x7f1df00719a0 msgr2=0x7f1df00d1160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:18.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.902+0000 7f1df49ff700 1 -- 172.21.15.92:0/2123963042 shutdown_connections 2023-11-28T11:21:18.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:18.902+0000 7f1df49ff700 1 -- 172.21.15.92:0/2123963042 wait complete. 2023-11-28T11:21:18.922 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":5,"active_gid":4103,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":115482},{"type":"v1","addr":"172.21.15.92:6833","nonce":115482}]},"active_addr":"172.21.15.92:6833/115482","active_change":"2023-11-28T11:20:35.842029+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4106,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":632176302}]},{"addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":4113822571}]},{"addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":732175340}]},{"addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":3971606688}]}]} 2023-11-28T11:21:18.929 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-11-28T11:21:18.929 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-11-28T11:21:18.929 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:21:19.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.125+0000 7fb87bbff700 1 Processor -- start 2023-11-28T11:21:19.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.126+0000 7fb87bbff700 1 -- start start 2023-11-28T11:21:19.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.127+0000 7fb87bbff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb8740cd460 0x7fb87406c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:19.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.127+0000 7fb87bbff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8740b88e0 con 0x7fb8740b8510 2023-11-28T11:21:19.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.127+0000 7fb87bbff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8740cd8f0 con 0x7fb8740d0d60 2023-11-28T11:21:19.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.127+0000 7fb87bbff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8740cdbd0 con 0x7fb8740cd460 2023-11-28T11:21:19.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.127+0000 7fb87999b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb8740b8510 0x7fb87406b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47404/0 (socket says 172.21.15.92:47404) 2023-11-28T11:21:19.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.127+0000 7fb87999b700 1 -- 172.21.15.92:0/1534564632 learned_addr learned my addr 172.21.15.92:0/1534564632 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:19.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.127+0000 7fb87919a700 1 --2- 172.21.15.92:0/1534564632 >> v2:172.21.15.92:3300/0 conn(0x7fb8740cd460 0x7fb87406c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:19.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.128+0000 7fb87919a700 1 -- 172.21.15.92:0/1534564632 >> v1:172.21.15.169:6789/0 conn(0x7fb8740d0d60 legacy=0x7fb8740d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:19.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.128+0000 7fb87919a700 1 -- 172.21.15.92:0/1534564632 >> v1:172.21.15.92:6789/0 conn(0x7fb8740b8510 legacy=0x7fb87406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:19.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.128+0000 7fb87919a700 1 -- 172.21.15.92:0/1534564632 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8740cfd10 con 0x7fb8740cd460 2023-11-28T11:21:19.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.128+0000 7fb87919a700 1 --2- 172.21.15.92:0/1534564632 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8740cd460 0x7fb87406c080 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb864009840 tx=0x7fb864009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3db0f253615a915 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:19.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.128+0000 7fb878999700 1 -- 172.21.15.92:0/1534564632 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb864011830 con 0x7fb8740cd460 2023-11-28T11:21:19.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.128+0000 7fb878999700 1 -- 172.21.15.92:0/1534564632 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb864011990 con 0x7fb8740cd460 2023-11-28T11:21:19.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.128+0000 7fb878999700 1 -- 172.21.15.92:0/1534564632 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb864011d20 con 0x7fb8740cd460 2023-11-28T11:21:19.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.129+0000 7fb87bbff700 1 -- 172.21.15.92:0/1534564632 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8740cd460 msgr2=0x7fb87406c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:19.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.129+0000 7fb87bbff700 1 --2- 172.21.15.92:0/1534564632 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8740cd460 0x7fb87406c080 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb864009840 tx=0x7fb864009b50 comp rx=0 tx=0).stop 2023-11-28T11:21:19.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.129+0000 7fb87bbff700 1 -- 172.21.15.92:0/1534564632 shutdown_connections 2023-11-28T11:21:19.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.129+0000 7fb87bbff700 1 --2- 172.21.15.92:0/1534564632 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8740cd460 0x7fb87406c080 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:19.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.129+0000 7fb87bbff700 1 -- 172.21.15.92:0/1534564632 >> 172.21.15.92:0/1534564632 conn(0x7fb8740719a0 msgr2=0x7fb874071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:19.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.129+0000 7fb87bbff700 1 -- 172.21.15.92:0/1534564632 shutdown_connections 2023-11-28T11:21:19.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.129+0000 7fb87bbff700 1 -- 172.21.15.92:0/1534564632 wait complete. 2023-11-28T11:21:19.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.130+0000 7fb87bbff700 1 Processor -- start 2023-11-28T11:21:19.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.131+0000 7fb87bbff700 1 -- start start 2023-11-28T11:21:19.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.131+0000 7fb87bbff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8740b8510 0x7fb8741e55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:19.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.131+0000 7fb87bbff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8740cd460 0x7fb8741e6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:19.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.131+0000 7fb87bbff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8740d0d60 0x7fb8741f3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:19.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.131+0000 7fb87bbff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8741e6880 con 0x7fb8740d0d60 2023-11-28T11:21:19.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.131+0000 7fb87bbff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8741e5c00 con 0x7fb8740cd460 2023-11-28T11:21:19.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.131+0000 7fb87bbff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8741e5ee0 con 0x7fb8740b8510 2023-11-28T11:21:19.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.132+0000 7fb87a19c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8740d0d60 0x7fb8741f3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:19.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.132+0000 7fb87a19c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8740d0d60 0x7fb8741f3460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48692/0 (socket says 172.21.15.92:48692) 2023-11-28T11:21:19.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.132+0000 7fb87a19c700 1 -- 172.21.15.92:0/3988699122 learned_addr learned my addr 172.21.15.92:0/3988699122 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:19.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.132+0000 7fb87999b700 1 --2- 172.21.15.92:0/3988699122 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8740b8510 0x7fb8741e55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:19.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.132+0000 7fb87919a700 1 --2- 172.21.15.92:0/3988699122 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8740cd460 0x7fb8741e6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:19.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.132+0000 7fb87a19c700 1 -- 172.21.15.92:0/3988699122 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8740b8510 msgr2=0x7fb8741e55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:19.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.132+0000 7fb87a19c700 1 --2- 172.21.15.92:0/3988699122 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8740b8510 0x7fb8741e55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:19.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.132+0000 7fb87a19c700 1 -- 172.21.15.92:0/3988699122 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8740cd460 msgr2=0x7fb8741e6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:19.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.132+0000 7fb87a19c700 1 --2- 172.21.15.92:0/3988699122 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8740cd460 0x7fb8741e6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:19.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.132+0000 7fb87a19c700 1 -- 172.21.15.92:0/3988699122 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8640094a0 con 0x7fb8740d0d60 2023-11-28T11:21:19.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.132+0000 7fb87a19c700 1 --2- 172.21.15.92:0/3988699122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8740d0d60 0x7fb8741f3460 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb86c00cb50 tx=0x7fb86c00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:19.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.133+0000 7fb86a7fc700 1 -- 172.21.15.92:0/3988699122 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86c003c00 con 0x7fb8740d0d60 2023-11-28T11:21:19.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.133+0000 7fb86a7fc700 1 -- 172.21.15.92:0/3988699122 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb86c003d60 con 0x7fb8740d0d60 2023-11-28T11:21:19.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.133+0000 7fb86a7fc700 1 -- 172.21.15.92:0/3988699122 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86c00aa20 con 0x7fb8740d0d60 2023-11-28T11:21:19.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.133+0000 7fb87bbff700 1 -- 172.21.15.92:0/3988699122 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8740d29c0 con 0x7fb8740d0d60 2023-11-28T11:21:19.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.133+0000 7fb87bbff700 1 -- 172.21.15.92:0/3988699122 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8740cfff0 con 0x7fb8740d0d60 2023-11-28T11:21:19.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.134+0000 7fb87bbff700 1 -- 172.21.15.92:0/3988699122 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8740d2b50 con 0x7fb8740d0d60 2023-11-28T11:21:19.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.135+0000 7fb86a7fc700 1 -- 172.21.15.92:0/3988699122 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb86c010050 con 0x7fb8740d0d60 2023-11-28T11:21:19.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.136+0000 7fb86a7fc700 1 --2- 172.21.15.92:0/3988699122 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb8580682f0 0x7fb85806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:19.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.136+0000 7fb86a7fc700 1 -- 172.21.15.92:0/3988699122 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb86c085f00 con 0x7fb8740d0d60 2023-11-28T11:21:19.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.136+0000 7fb87999b700 1 --2- 172.21.15.92:0/3988699122 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb8580682f0 0x7fb85806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:19.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.137+0000 7fb87999b700 1 --2- 172.21.15.92:0/3988699122 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb8580682f0 0x7fb85806a7a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb870005910 tx=0x7fb870009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:19.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.140+0000 7fb86a7fc700 1 -- 172.21.15.92:0/3988699122 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb86c052380 con 0x7fb8740d0d60 2023-11-28T11:21:19.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.330+0000 7fb87bbff700 1 -- 172.21.15.92:0/3988699122 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb8741e69c0 con 0x7fb8740d0d60 2023-11-28T11:21:19.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.331+0000 7fb86a7fc700 1 -- 172.21.15.92:0/3988699122 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+19748 (secure 0 0 0) 0x7fb8741e69c0 con 0x7fb8740d0d60 2023-11-28T11:21:19.334 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:21:19.335 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":27,"fsid":"68956e0c-c16f-47cd-a220-f98364b0831d","created":"2023-11-28T11:20:28.697573+0000","modified":"2023-11-28T11:21:00.770781+0000","last_up_change":"2023-11-28T11:20:41.257011+0000","last_in_change":"2023-11-28T11:20:32.249767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:20:41.925237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:20:48.682497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:20:54.471398+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:20:55.204585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:20:58.297307+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6abe483f-0e03-4a26-b9cf-62041291747e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":115823},{"type":"v1","addr":"172.21.15.92:6809","nonce":115823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":115823},{"type":"v1","addr":"172.21.15.92:6813","nonce":115823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":115823},{"type":"v1","addr":"172.21.15.92:6821","nonce":115823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":115823},{"type":"v1","addr":"172.21.15.92:6817","nonce":115823}]},"public_addr":"172.21.15.92:6809/115823","cluster_addr":"172.21.15.92:6813/115823","heartbeat_back_addr":"172.21.15.92:6821/115823","heartbeat_front_addr":"172.21.15.92:6817/115823","state":["exists","up"]},{"osd":1,"uuid":"13c1de73-1a8c-4d7e-8a28-9dae52dfe089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":115824},{"type":"v1","addr":"172.21.15.92:6812","nonce":115824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":115824},{"type":"v1","addr":"172.21.15.92:6816","nonce":115824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":115824},{"type":"v1","addr":"172.21.15.92:6823","nonce":115824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":115824},{"type":"v1","addr":"172.21.15.92:6820","nonce":115824}]},"public_addr":"172.21.15.92:6812/115824","cluster_addr":"172.21.15.92:6816/115824","heartbeat_back_addr":"172.21.15.92:6823/115824","heartbeat_front_addr":"172.21.15.92:6820/115824","state":["exists","up"]},{"osd":2,"uuid":"25788a55-112f-4f00-810d-2bc37cf76778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":115821},{"type":"v1","addr":"172.21.15.92:6801","nonce":115821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":115821},{"type":"v1","addr":"172.21.15.92:6803","nonce":115821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":115821},{"type":"v1","addr":"172.21.15.92:6807","nonce":115821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":115821},{"type":"v1","addr":"172.21.15.92:6805","nonce":115821}]},"public_addr":"172.21.15.92:6801/115821","cluster_addr":"172.21.15.92:6803/115821","heartbeat_back_addr":"172.21.15.92:6807/115821","heartbeat_front_addr":"172.21.15.92:6805/115821","state":["exists","up"]},{"osd":3,"uuid":"adf54e24-33a9-41fe-9ce3-98a5ab6c158e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":115825},{"type":"v1","addr":"172.21.15.92:6825","nonce":115825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":115825},{"type":"v1","addr":"172.21.15.92:6827","nonce":115825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":115825},{"type":"v1","addr":"172.21.15.92:6831","nonce":115825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":115825},{"type":"v1","addr":"172.21.15.92:6829","nonce":115825}]},"public_addr":"172.21.15.92:6825/115825","cluster_addr":"172.21.15.92:6827/115825","heartbeat_back_addr":"172.21.15.92:6831/115825","heartbeat_front_addr":"172.21.15.92:6829/115825","state":["exists","up"]},{"osd":4,"uuid":"ccb9b747-f399-44ea-9a18-9db3152227e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":116035},{"type":"v1","addr":"172.21.15.169:6817","nonce":116035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":116035},{"type":"v1","addr":"172.21.15.169:6819","nonce":116035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":116035},{"type":"v1","addr":"172.21.15.169:6826","nonce":116035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":116035},{"type":"v1","addr":"172.21.15.169:6822","nonce":116035}]},"public_addr":"172.21.15.169:6817/116035","cluster_addr":"172.21.15.169:6819/116035","heartbeat_back_addr":"172.21.15.169:6826/116035","heartbeat_front_addr":"172.21.15.169:6822/116035","state":["exists","up"]},{"osd":5,"uuid":"308a1e63-75aa-4d7d-8a7c-cecc24714fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":116033},{"type":"v1","addr":"172.21.15.169:6801","nonce":116033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":116033},{"type":"v1","addr":"172.21.15.169:6803","nonce":116033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":116033},{"type":"v1","addr":"172.21.15.169:6807","nonce":116033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":116033},{"type":"v1","addr":"172.21.15.169:6805","nonce":116033}]},"public_addr":"172.21.15.169:6801/116033","cluster_addr":"172.21.15.169:6803/116033","heartbeat_back_addr":"172.21.15.169:6807/116033","heartbeat_front_addr":"172.21.15.169:6805/116033","state":["exists","up"]},{"osd":6,"uuid":"9c459b7b-b821-4b4b-b91e-6aaae271457c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":116034},{"type":"v1","addr":"172.21.15.169:6809","nonce":116034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":116034},{"type":"v1","addr":"172.21.15.169:6811","nonce":116034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":116034},{"type":"v1","addr":"172.21.15.169:6815","nonce":116034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":116034},{"type":"v1","addr":"172.21.15.169:6813","nonce":116034}]},"public_addr":"172.21.15.169:6809/116034","cluster_addr":"172.21.15.169:6811/116034","heartbeat_back_addr":"172.21.15.169:6815/116034","heartbeat_front_addr":"172.21.15.169:6813/116034","state":["exists","up"]},{"osd":7,"uuid":"1ca9aa4a-755d-4e28-a937-4d8dccab4df3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":116044},{"type":"v1","addr":"172.21.15.169:6823","nonce":116044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6825","nonce":116044},{"type":"v1","addr":"172.21.15.169:6827","nonce":116044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":116044},{"type":"v1","addr":"172.21.15.169:6831","nonce":116044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":116044},{"type":"v1","addr":"172.21.15.169:6829","nonce":116044}]},"public_addr":"172.21.15.169:6823/116044","cluster_addr":"172.21.15.169:6827/116044","heartbeat_back_addr":"172.21.15.169:6831/116044","heartbeat_front_addr":"172.21.15.169:6829/116044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.586781+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.869778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.508594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.860493+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.872267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.746933+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.611520+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.871446+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:21:19.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.334+0000 7fb87bbff700 1 -- 172.21.15.92:0/3988699122 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb8580682f0 msgr2=0x7fb85806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:19.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.334+0000 7fb87bbff700 1 --2- 172.21.15.92:0/3988699122 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb8580682f0 0x7fb85806a7a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb870005910 tx=0x7fb870009040 comp rx=0 tx=0).stop 2023-11-28T11:21:19.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.334+0000 7fb87bbff700 1 -- 172.21.15.92:0/3988699122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8740d0d60 msgr2=0x7fb8741f3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:19.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.334+0000 7fb87bbff700 1 --2- 172.21.15.92:0/3988699122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8740d0d60 0x7fb8741f3460 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb86c00cb50 tx=0x7fb86c00ce60 comp rx=0 tx=0).stop 2023-11-28T11:21:19.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.334+0000 7fb87bbff700 1 -- 172.21.15.92:0/3988699122 shutdown_connections 2023-11-28T11:21:19.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.334+0000 7fb87bbff700 1 --2- 172.21.15.92:0/3988699122 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb8580682f0 0x7fb85806a7a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:19.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.334+0000 7fb87bbff700 1 --2- 172.21.15.92:0/3988699122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8740d0d60 0x7fb8741f3460 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:19.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.334+0000 7fb87bbff700 1 --2- 172.21.15.92:0/3988699122 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb8740b8510 0x7fb8741e55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:19.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.334+0000 7fb87bbff700 1 --2- 172.21.15.92:0/3988699122 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb8740cd460 0x7fb8741e6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:19.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.334+0000 7fb87bbff700 1 -- 172.21.15.92:0/3988699122 >> 172.21.15.92:0/3988699122 conn(0x7fb8740719a0 msgr2=0x7fb8740b29f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:19.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.334+0000 7fb87bbff700 1 -- 172.21.15.92:0/3988699122 shutdown_connections 2023-11-28T11:21:19.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.334+0000 7fb87bbff700 1 -- 172.21.15.92:0/3988699122 wait complete. 2023-11-28T11:21:19.355 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-11-28T11:21:19.355 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:21:19.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.549+0000 7fb1c5fff700 1 Processor -- start 2023-11-28T11:21:19.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.549+0000 7fb1c5fff700 1 -- start start 2023-11-28T11:21:19.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.550+0000 7fb1c5fff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb1c00cd460 0x7fb1c006c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:19.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.550+0000 7fb1c5fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1c00d0d60 0x7fb1c00d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:19.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.550+0000 7fb1c5fff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1c00b88e0 con 0x7fb1c00b8510 2023-11-28T11:21:19.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.550+0000 7fb1c5fff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1c00cdbd0 con 0x7fb1c00cd460 2023-11-28T11:21:19.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.550+0000 7fb1c5fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c00d02d0 con 0x7fb1c00d0d60 2023-11-28T11:21:19.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.551+0000 7fb1beffd700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb1c00cd460 0x7fb1c006c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:19.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.551+0000 7fb1bffff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1c00d0d60 0x7fb1c00d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:19.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.551+0000 7fb1bf7fe700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fb1c00b8510 0x7fb1c006b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.92:60728/0 (socket says 172.21.15.92:60728) 2023-11-28T11:21:19.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.551+0000 7fb1bf7fe700 1 -- 172.21.15.92:0/347740537 learned_addr learned my addr 172.21.15.92:0/347740537 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:19.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.551+0000 7fb1beffd700 1 -- 172.21.15.92:0/347740537 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1c00d0d60 msgr2=0x7fb1c00d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:19.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.551+0000 7fb1beffd700 1 --2- 172.21.15.92:0/347740537 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1c00d0d60 0x7fb1c00d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:19.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.551+0000 7fb1beffd700 1 -- 172.21.15.92:0/347740537 >> v1:172.21.15.169:6789/0 conn(0x7fb1c00b8510 legacy=0x7fb1c006b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:19.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.551+0000 7fb1beffd700 1 -- 172.21.15.92:0/347740537 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c00cd8f0 con 0x7fb1c00cd460 2023-11-28T11:21:19.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.552+0000 7fb1beffd700 1 --2- 172.21.15.92:0/347740537 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1c00cd460 0x7fb1c006c080 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb1b4009870 tx=0x7fb1b4009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=794c7d2cb42bfaad server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:19.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.552+0000 7fb1be7fc700 1 -- 172.21.15.92:0/347740537 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b4011830 con 0x7fb1c00cd460 2023-11-28T11:21:19.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.552+0000 7fb1be7fc700 1 -- 172.21.15.92:0/347740537 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b4011990 con 0x7fb1c00cd460 2023-11-28T11:21:19.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.552+0000 7fb1be7fc700 1 -- 172.21.15.92:0/347740537 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b4011d20 con 0x7fb1c00cd460 2023-11-28T11:21:19.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.553+0000 7fb1c5fff700 1 -- 172.21.15.92:0/347740537 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1c00cd460 msgr2=0x7fb1c006c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:19.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.553+0000 7fb1c5fff700 1 --2- 172.21.15.92:0/347740537 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1c00cd460 0x7fb1c006c080 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb1b4009870 tx=0x7fb1b4009b80 comp rx=0 tx=0).stop 2023-11-28T11:21:19.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.553+0000 7fb1c5fff700 1 -- 172.21.15.92:0/347740537 shutdown_connections 2023-11-28T11:21:19.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.553+0000 7fb1c5fff700 1 --2- 172.21.15.92:0/347740537 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1c00d0d60 0x7fb1c00d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:19.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.553+0000 7fb1c5fff700 1 --2- 172.21.15.92:0/347740537 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1c00cd460 0x7fb1c006c080 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:19.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.553+0000 7fb1c5fff700 1 -- 172.21.15.92:0/347740537 >> 172.21.15.92:0/347740537 conn(0x7fb1c00719a0 msgr2=0x7fb1c0071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:19.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.553+0000 7fb1c5fff700 1 -- 172.21.15.92:0/347740537 shutdown_connections 2023-11-28T11:21:19.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.553+0000 7fb1c5fff700 1 -- 172.21.15.92:0/347740537 wait complete. 2023-11-28T11:21:19.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.555+0000 7fb1c5fff700 1 Processor -- start 2023-11-28T11:21:19.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.555+0000 7fb1c5fff700 1 -- start start 2023-11-28T11:21:19.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.556+0000 7fb1c5fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1c00b8510 0x7fb1c01e5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:19.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.556+0000 7fb1c5fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1c00cd460 0x7fb1c014e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:19.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.556+0000 7fb1c5fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1c00d0d60 0x7fb1c006b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:19.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.556+0000 7fb1c5fff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c01e5bb0 con 0x7fb1c00d0d60 2023-11-28T11:21:19.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.556+0000 7fb1c5fff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c015b700 con 0x7fb1c00cd460 2023-11-28T11:21:19.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.556+0000 7fb1c5fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c01e4f30 con 0x7fb1c00b8510 2023-11-28T11:21:19.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.556+0000 7fb1bffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1c00d0d60 0x7fb1c006b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:19.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.556+0000 7fb1bffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1c00d0d60 0x7fb1c006b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48708/0 (socket says 172.21.15.92:48708) 2023-11-28T11:21:19.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.556+0000 7fb1bf7fe700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1c00b8510 0x7fb1c01e5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:19.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.556+0000 7fb1bffff700 1 -- 172.21.15.92:0/3650101532 learned_addr learned my addr 172.21.15.92:0/3650101532 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:19.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.557+0000 7fb1beffd700 1 --2- 172.21.15.92:0/3650101532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1c00cd460 0x7fb1c014e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:19.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.557+0000 7fb1bffff700 1 -- 172.21.15.92:0/3650101532 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1c00b8510 msgr2=0x7fb1c01e5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:19.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.557+0000 7fb1bffff700 1 --2- 172.21.15.92:0/3650101532 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1c00b8510 0x7fb1c01e5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:19.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.557+0000 7fb1bffff700 1 -- 172.21.15.92:0/3650101532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1c00cd460 msgr2=0x7fb1c014e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:19.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.557+0000 7fb1bffff700 1 --2- 172.21.15.92:0/3650101532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1c00cd460 0x7fb1c014e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:19.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.557+0000 7fb1bffff700 1 -- 172.21.15.92:0/3650101532 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b40094a0 con 0x7fb1c00d0d60 2023-11-28T11:21:19.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.557+0000 7fb1bffff700 1 --2- 172.21.15.92:0/3650101532 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1c00d0d60 0x7fb1c006b970 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb1b800efc0 tx=0x7fb1b800c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:19.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.557+0000 7fb1a7fff700 1 -- 172.21.15.92:0/3650101532 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b800cad0 con 0x7fb1c00d0d60 2023-11-28T11:21:19.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.557+0000 7fb1a7fff700 1 -- 172.21.15.92:0/3650101532 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b800f040 con 0x7fb1c00d0d60 2023-11-28T11:21:19.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.557+0000 7fb1c5fff700 1 -- 172.21.15.92:0/3650101532 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1c00cfff0 con 0x7fb1c00d0d60 2023-11-28T11:21:19.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.557+0000 7fb1a7fff700 1 -- 172.21.15.92:0/3650101532 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b800ce70 con 0x7fb1c00d0d60 2023-11-28T11:21:19.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.557+0000 7fb1c5fff700 1 -- 172.21.15.92:0/3650101532 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1c00cfd10 con 0x7fb1c00d0d60 2023-11-28T11:21:19.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.559+0000 7fb1c5fff700 1 -- 172.21.15.92:0/3650101532 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1c01e5cf0 con 0x7fb1c00d0d60 2023-11-28T11:21:19.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.559+0000 7fb1a7fff700 1 -- 172.21.15.92:0/3650101532 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb1b8014480 con 0x7fb1c00d0d60 2023-11-28T11:21:19.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.561+0000 7fb1a7fff700 1 --2- 172.21.15.92:0/3650101532 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb1ac0682f0 0x7fb1ac06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:19.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.561+0000 7fb1a7fff700 1 -- 172.21.15.92:0/3650101532 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb1b8086990 con 0x7fb1c00d0d60 2023-11-28T11:21:19.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.561+0000 7fb1bf7fe700 1 --2- 172.21.15.92:0/3650101532 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb1ac0682f0 0x7fb1ac06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:19.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.561+0000 7fb1bf7fe700 1 --2- 172.21.15.92:0/3650101532 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb1ac0682f0 0x7fb1ac06a7a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0005910 tx=0x7fb1b0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:19.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.565+0000 7fb1a7fff700 1 -- 172.21.15.92:0/3650101532 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb1c01e5cf0 con 0x7fb1c00d0d60 2023-11-28T11:21:19.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.757+0000 7fb1c5fff700 1 -- 172.21.15.92:0/3650101532 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb1c01e5cf0 con 0x7fb1c00d0d60 2023-11-28T11:21:19.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.758+0000 7fb1a7fff700 1 -- 172.21.15.92:0/3650101532 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+19748 (secure 0 0 0) 0x7fb1c01e5cf0 con 0x7fb1c00d0d60 2023-11-28T11:21:19.761 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:21:19.761 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":27,"fsid":"68956e0c-c16f-47cd-a220-f98364b0831d","created":"2023-11-28T11:20:28.697573+0000","modified":"2023-11-28T11:21:00.770781+0000","last_up_change":"2023-11-28T11:20:41.257011+0000","last_in_change":"2023-11-28T11:20:32.249767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:20:41.925237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:20:48.682497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:20:54.471398+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:20:55.204585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:20:58.297307+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6abe483f-0e03-4a26-b9cf-62041291747e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":115823},{"type":"v1","addr":"172.21.15.92:6809","nonce":115823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":115823},{"type":"v1","addr":"172.21.15.92:6813","nonce":115823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":115823},{"type":"v1","addr":"172.21.15.92:6821","nonce":115823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":115823},{"type":"v1","addr":"172.21.15.92:6817","nonce":115823}]},"public_addr":"172.21.15.92:6809/115823","cluster_addr":"172.21.15.92:6813/115823","heartbeat_back_addr":"172.21.15.92:6821/115823","heartbeat_front_addr":"172.21.15.92:6817/115823","state":["exists","up"]},{"osd":1,"uuid":"13c1de73-1a8c-4d7e-8a28-9dae52dfe089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":115824},{"type":"v1","addr":"172.21.15.92:6812","nonce":115824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":115824},{"type":"v1","addr":"172.21.15.92:6816","nonce":115824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":115824},{"type":"v1","addr":"172.21.15.92:6823","nonce":115824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":115824},{"type":"v1","addr":"172.21.15.92:6820","nonce":115824}]},"public_addr":"172.21.15.92:6812/115824","cluster_addr":"172.21.15.92:6816/115824","heartbeat_back_addr":"172.21.15.92:6823/115824","heartbeat_front_addr":"172.21.15.92:6820/115824","state":["exists","up"]},{"osd":2,"uuid":"25788a55-112f-4f00-810d-2bc37cf76778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":115821},{"type":"v1","addr":"172.21.15.92:6801","nonce":115821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":115821},{"type":"v1","addr":"172.21.15.92:6803","nonce":115821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":115821},{"type":"v1","addr":"172.21.15.92:6807","nonce":115821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":115821},{"type":"v1","addr":"172.21.15.92:6805","nonce":115821}]},"public_addr":"172.21.15.92:6801/115821","cluster_addr":"172.21.15.92:6803/115821","heartbeat_back_addr":"172.21.15.92:6807/115821","heartbeat_front_addr":"172.21.15.92:6805/115821","state":["exists","up"]},{"osd":3,"uuid":"adf54e24-33a9-41fe-9ce3-98a5ab6c158e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":115825},{"type":"v1","addr":"172.21.15.92:6825","nonce":115825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":115825},{"type":"v1","addr":"172.21.15.92:6827","nonce":115825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":115825},{"type":"v1","addr":"172.21.15.92:6831","nonce":115825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":115825},{"type":"v1","addr":"172.21.15.92:6829","nonce":115825}]},"public_addr":"172.21.15.92:6825/115825","cluster_addr":"172.21.15.92:6827/115825","heartbeat_back_addr":"172.21.15.92:6831/115825","heartbeat_front_addr":"172.21.15.92:6829/115825","state":["exists","up"]},{"osd":4,"uuid":"ccb9b747-f399-44ea-9a18-9db3152227e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":116035},{"type":"v1","addr":"172.21.15.169:6817","nonce":116035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":116035},{"type":"v1","addr":"172.21.15.169:6819","nonce":116035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":116035},{"type":"v1","addr":"172.21.15.169:6826","nonce":116035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":116035},{"type":"v1","addr":"172.21.15.169:6822","nonce":116035}]},"public_addr":"172.21.15.169:6817/116035","cluster_addr":"172.21.15.169:6819/116035","heartbeat_back_addr":"172.21.15.169:6826/116035","heartbeat_front_addr":"172.21.15.169:6822/116035","state":["exists","up"]},{"osd":5,"uuid":"308a1e63-75aa-4d7d-8a7c-cecc24714fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":116033},{"type":"v1","addr":"172.21.15.169:6801","nonce":116033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":116033},{"type":"v1","addr":"172.21.15.169:6803","nonce":116033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":116033},{"type":"v1","addr":"172.21.15.169:6807","nonce":116033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":116033},{"type":"v1","addr":"172.21.15.169:6805","nonce":116033}]},"public_addr":"172.21.15.169:6801/116033","cluster_addr":"172.21.15.169:6803/116033","heartbeat_back_addr":"172.21.15.169:6807/116033","heartbeat_front_addr":"172.21.15.169:6805/116033","state":["exists","up"]},{"osd":6,"uuid":"9c459b7b-b821-4b4b-b91e-6aaae271457c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":116034},{"type":"v1","addr":"172.21.15.169:6809","nonce":116034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":116034},{"type":"v1","addr":"172.21.15.169:6811","nonce":116034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":116034},{"type":"v1","addr":"172.21.15.169:6815","nonce":116034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":116034},{"type":"v1","addr":"172.21.15.169:6813","nonce":116034}]},"public_addr":"172.21.15.169:6809/116034","cluster_addr":"172.21.15.169:6811/116034","heartbeat_back_addr":"172.21.15.169:6815/116034","heartbeat_front_addr":"172.21.15.169:6813/116034","state":["exists","up"]},{"osd":7,"uuid":"1ca9aa4a-755d-4e28-a937-4d8dccab4df3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":116044},{"type":"v1","addr":"172.21.15.169:6823","nonce":116044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6825","nonce":116044},{"type":"v1","addr":"172.21.15.169:6827","nonce":116044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":116044},{"type":"v1","addr":"172.21.15.169:6831","nonce":116044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":116044},{"type":"v1","addr":"172.21.15.169:6829","nonce":116044}]},"public_addr":"172.21.15.169:6823/116044","cluster_addr":"172.21.15.169:6827/116044","heartbeat_back_addr":"172.21.15.169:6831/116044","heartbeat_front_addr":"172.21.15.169:6829/116044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.586781+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.869778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.508594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.860493+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.872267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.746933+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.611520+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.871446+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:21:19.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.761+0000 7fb1c5fff700 1 -- 172.21.15.92:0/3650101532 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb1ac0682f0 msgr2=0x7fb1ac06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:19.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.761+0000 7fb1c5fff700 1 --2- 172.21.15.92:0/3650101532 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb1ac0682f0 0x7fb1ac06a7a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0005910 tx=0x7fb1b0009040 comp rx=0 tx=0).stop 2023-11-28T11:21:19.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.761+0000 7fb1c5fff700 1 -- 172.21.15.92:0/3650101532 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1c00d0d60 msgr2=0x7fb1c006b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:19.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.761+0000 7fb1c5fff700 1 --2- 172.21.15.92:0/3650101532 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1c00d0d60 0x7fb1c006b970 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb1b800efc0 tx=0x7fb1b800c0c0 comp rx=0 tx=0).stop 2023-11-28T11:21:19.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.761+0000 7fb1c5fff700 1 -- 172.21.15.92:0/3650101532 shutdown_connections 2023-11-28T11:21:19.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.761+0000 7fb1c5fff700 1 --2- 172.21.15.92:0/3650101532 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb1ac0682f0 0x7fb1ac06a7a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:19.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.761+0000 7fb1c5fff700 1 --2- 172.21.15.92:0/3650101532 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1c00d0d60 0x7fb1c006b970 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:19.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.761+0000 7fb1c5fff700 1 --2- 172.21.15.92:0/3650101532 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1c00b8510 0x7fb1c01e5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:19.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.761+0000 7fb1c5fff700 1 --2- 172.21.15.92:0/3650101532 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1c00cd460 0x7fb1c014e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:19.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.761+0000 7fb1c5fff700 1 -- 172.21.15.92:0/3650101532 >> 172.21.15.92:0/3650101532 conn(0x7fb1c00719a0 msgr2=0x7fb1c00b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:19.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.761+0000 7fb1c5fff700 1 -- 172.21.15.92:0/3650101532 shutdown_connections 2023-11-28T11:21:19.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:19.761+0000 7fb1c5fff700 1 -- 172.21.15.92:0/3650101532 wait complete. 2023-11-28T11:21:19.783 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-11-28T11:21:19.783 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-11-28T11:21:19.783 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-11-28T11:21:19.784 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-11-28T11:21:19.784 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-11-28T11:21:19.784 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-11-28T11:21:19.785 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-11-28T11:21:19.785 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-11-28T11:21:20.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.138+0000 7f4f555ff700 1 Processor -- start 2023-11-28T11:21:20.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.139+0000 7f65d3fff700 1 Processor -- start 2023-11-28T11:21:20.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.139+0000 7f65d3fff700 1 -- start start 2023-11-28T11:21:20.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.138+0000 7f4f555ff700 1 -- start start 2023-11-28T11:21:20.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.140+0000 7f4f555ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f50160d30 0x7f4f500d21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.140+0000 7f4f555ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4f500b6260 0x7f4f5006a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.140+0000 7f4f555ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4f5006a7b0 0x7f4f50070910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.140+0000 7f4f555ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f50161100 con 0x7f4f5006a7b0 2023-11-28T11:21:20.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.140+0000 7f4f555ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f50165560 con 0x7f4f500b6260 2023-11-28T11:21:20.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.140+0000 7f4f555ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f500b6a50 con 0x7f4f50160d30 2023-11-28T11:21:20.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.142+0000 7f4f4effd700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f50160d30 0x7f4f500d21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.142+0000 7f4f4effd700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f50160d30 0x7f4f500d21f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.92:33928/0 (socket says 172.21.15.92:33928) 2023-11-28T11:21:20.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.142+0000 7f4f4effd700 1 -- 172.21.15.92:0/258903195 learned_addr learned my addr 172.21.15.92:0/258903195 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:20.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.142+0000 7f4f467fc700 1 --2- 172.21.15.92:0/258903195 >> v2:172.21.15.169:3300/0 conn(0x7f4f500b6260 0x7f4f5006a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.143+0000 7f4f4f7fe700 1 --2- 172.21.15.92:0/258903195 >> v2:172.21.15.92:3300/0 conn(0x7f4f5006a7b0 0x7f4f50070910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.143+0000 7f4f4effd700 1 -- 172.21.15.92:0/258903195 >> v2:172.21.15.169:3300/0 conn(0x7f4f500b6260 msgr2=0x7f4f5006a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:20.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.143+0000 7f4f4effd700 1 --2- 172.21.15.92:0/258903195 >> v2:172.21.15.169:3300/0 conn(0x7f4f500b6260 0x7f4f5006a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.143+0000 7f4f4effd700 1 -- 172.21.15.92:0/258903195 >> v2:172.21.15.92:3300/0 conn(0x7f4f5006a7b0 msgr2=0x7f4f50070910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:20.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.143+0000 7f4f4effd700 1 --2- 172.21.15.92:0/258903195 >> v2:172.21.15.92:3300/0 conn(0x7f4f5006a7b0 0x7f4f50070910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.143+0000 7f4f4effd700 1 -- 172.21.15.92:0/258903195 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f5006abb0 con 0x7f4f50160d30 2023-11-28T11:21:20.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.143+0000 7f4f4effd700 1 --2- 172.21.15.92:0/258903195 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f50160d30 0x7f4f500d21f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4f50072800 tx=0x7f4f4000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db9d28d0e75537d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:20.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.144+0000 7f4f4e7fc700 1 -- 172.21.15.92:0/258903195 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4000e070 con 0x7f4f50160d30 2023-11-28T11:21:20.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.144+0000 7f4f4e7fc700 1 -- 172.21.15.92:0/258903195 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f4000b9c0 con 0x7f4f50160d30 2023-11-28T11:21:20.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.144+0000 7f4f4e7fc700 1 -- 172.21.15.92:0/258903195 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4000bd70 con 0x7f4f50160d30 2023-11-28T11:21:20.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.144+0000 7f65d3fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f65cc0c9ed0 0x7f65cc06a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.144+0000 7f65d3fff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65cc06ab80 con 0x7f65cc06a270 2023-11-28T11:21:20.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.144+0000 7f65d3fff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65cc0ccb60 con 0x7f65cc0cdee0 2023-11-28T11:21:20.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.144+0000 7f65d3fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65cc0cce40 con 0x7f65cc0c9ed0 2023-11-28T11:21:20.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.145+0000 7f65d159a700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f65cc0c9ed0 0x7f65cc06a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.145+0000 7f65d159a700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f65cc0c9ed0 0x7f65cc06a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.92:33930/0 (socket says 172.21.15.92:33930) 2023-11-28T11:21:20.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.145+0000 7f65d159a700 1 -- 172.21.15.92:0/4031875672 learned_addr learned my addr 172.21.15.92:0/4031875672 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:20.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.146+0000 7f4f555ff700 1 -- 172.21.15.92:0/258903195 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f50160d30 msgr2=0x7f4f500d21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.146+0000 7f4f555ff700 1 --2- 172.21.15.92:0/258903195 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f50160d30 0x7f4f500d21f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4f50072800 tx=0x7f4f4000b330 comp rx=0 tx=0).stop 2023-11-28T11:21:20.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.146+0000 7f65d159a700 1 -- 172.21.15.92:0/4031875672 >> v1:172.21.15.169:6789/0 conn(0x7f65cc0cdee0 legacy=0x7f65cc06c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.146+0000 7f65d159a700 1 -- 172.21.15.92:0/4031875672 >> v1:172.21.15.92:6789/0 conn(0x7f65cc06a270 legacy=0x7f65cc15ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.146+0000 7f65d159a700 1 -- 172.21.15.92:0/4031875672 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65cc0cd120 con 0x7f65cc0c9ed0 2023-11-28T11:21:20.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.146+0000 7f65d0d99700 1 -- 172.21.15.92:0/4031875672 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1252636620 0 0) 0x7f65cc0ccb60 con 0x7f65cc0cdee0 2023-11-28T11:21:20.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.146+0000 7f4f555ff700 1 -- 172.21.15.92:0/258903195 shutdown_connections 2023-11-28T11:21:20.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.146+0000 7f4f555ff700 1 --2- 172.21.15.92:0/258903195 >> v2:172.21.15.92:3300/0 conn(0x7f4f5006a7b0 0x7f4f50070910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.146+0000 7f4f555ff700 1 --2- 172.21.15.92:0/258903195 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f50160d30 0x7f4f500d21f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.146+0000 7f4f555ff700 1 --2- 172.21.15.92:0/258903195 >> v2:172.21.15.169:3300/0 conn(0x7f4f500b6260 0x7f4f5006a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.146+0000 7f4f555ff700 1 -- 172.21.15.92:0/258903195 >> 172.21.15.92:0/258903195 conn(0x7f4f500d3ca0 msgr2=0x7f4f500d4070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:20.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.146+0000 7f4f555ff700 1 -- 172.21.15.92:0/258903195 shutdown_connections 2023-11-28T11:21:20.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.146+0000 7f0ba83ff700 1 Processor -- start 2023-11-28T11:21:20.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.146+0000 7f0ba83ff700 1 -- start start 2023-11-28T11:21:20.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.146+0000 7f65d159a700 1 --2- 172.21.15.92:0/4031875672 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f65cc0c9ed0 0x7f65cc06a640 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f65c400d3a0 tx=0x7f65c400d6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e78c08a796fdea0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:20.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.148+0000 7f0ba83ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ba0142e80 0x7f0ba006a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.149+0000 7f65d0d99700 1 -- 172.21.15.92:0/4031875672 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c4010070 con 0x7f65cc0c9ed0 2023-11-28T11:21:20.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.149+0000 7f65d0d99700 1 -- 172.21.15.92:0/4031875672 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c400dd40 con 0x7f65cc0c9ed0 2023-11-28T11:21:20.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.149+0000 7f65d0d99700 1 -- 172.21.15.92:0/4031875672 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c4007d70 con 0x7f65cc0c9ed0 2023-11-28T11:21:20.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.150+0000 7f65d3fff700 1 -- 172.21.15.92:0/4031875672 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f65cc0c9ed0 msgr2=0x7f65cc06a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.149+0000 7f4f555ff700 1 -- 172.21.15.92:0/258903195 wait complete. 2023-11-28T11:21:20.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.150+0000 7f65d3fff700 1 --2- 172.21.15.92:0/4031875672 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f65cc0c9ed0 0x7f65cc06a640 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f65c400d3a0 tx=0x7f65c400d6b0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.150+0000 7f4f555ff700 1 Processor -- start 2023-11-28T11:21:20.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.149+0000 7f0ba619b700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ba0142e80 0x7f0ba006a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.149+0000 7f0ba619b700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ba0142e80 0x7f0ba006a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.92:33942/0 (socket says 172.21.15.92:33942) 2023-11-28T11:21:20.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.149+0000 7f0ba619b700 1 -- 172.21.15.92:0/739223527 learned_addr learned my addr 172.21.15.92:0/739223527 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:20.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.151+0000 7f65d3fff700 1 -- 172.21.15.92:0/4031875672 shutdown_connections 2023-11-28T11:21:20.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.151+0000 7f65d3fff700 1 --2- 172.21.15.92:0/4031875672 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f65cc0c9ed0 0x7f65cc06a640 secure :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f65c400d3a0 tx=0x7f65c400d6b0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.151+0000 7f65d3fff700 1 -- 172.21.15.92:0/4031875672 >> 172.21.15.92:0/4031875672 conn(0x7f65cc0cb490 msgr2=0x7f65cc0cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:20.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.151+0000 7f4f555ff700 1 -- start start 2023-11-28T11:21:20.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.151+0000 7f0ba83ff700 1 -- 172.21.15.92:0/739223527 --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ba006ac00 con 0x7f0ba00ca600 2023-11-28T11:21:20.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.151+0000 7f0ba83ff700 1 -- 172.21.15.92:0/739223527 --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ba0159b10 con 0x7f0ba006a7b0 2023-11-28T11:21:20.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.151+0000 7f0ba83ff700 1 -- 172.21.15.92:0/739223527 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba00c9380 con 0x7f0ba0142e80 2023-11-28T11:21:20.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.152+0000 7f0ba5199700 1 -- 172.21.15.92:0/739223527 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2456435254 0 0) 0x7f0b98001110 con 0x7f0ba00ca600 2023-11-28T11:21:20.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.152+0000 7f0ba5199700 1 -- 172.21.15.92:0/739223527 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b8c0035b0 con 0x7f0ba00ca600 2023-11-28T11:21:20.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.152+0000 7f0ba619b700 1 -- 172.21.15.92:0/739223527 >> v1:172.21.15.169:6789/0 conn(0x7f0ba006a7b0 legacy=0x7f0ba00c9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.152+0000 7f0ba619b700 1 -- 172.21.15.92:0/739223527 >> v1:172.21.15.92:6789/0 conn(0x7f0ba00ca600 legacy=0x7f0ba00cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.152+0000 7f0ba619b700 1 -- 172.21.15.92:0/739223527 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ba0143250 con 0x7f0ba0142e80 2023-11-28T11:21:20.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.152+0000 7f65d3fff700 1 -- 172.21.15.92:0/4031875672 shutdown_connections 2023-11-28T11:21:20.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.152+0000 7f4f555ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f5006a7b0 0x7f4f5014b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.152+0000 7f65d3fff700 1 -- 172.21.15.92:0/4031875672 wait complete. 2023-11-28T11:21:20.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.153+0000 7f4f555ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f500b6260 0x7f4f500cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.153+0000 7f4f4effd700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f5006a7b0 0x7f4f5014b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.153+0000 7f4f4effd700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f5006a7b0 0x7f4f5014b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.92:57002/0 (socket says 172.21.15.92:57002) 2023-11-28T11:21:20.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.153+0000 7f4f4effd700 1 -- 172.21.15.92:0/3358853134 learned_addr learned my addr 172.21.15.92:0/3358853134 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:20.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.153+0000 7f4f555ff700 1 --2- 172.21.15.92:0/3358853134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f50160d30 0x7f4f50148470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.153+0000 7f4f467fc700 1 --2- 172.21.15.92:0/3358853134 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f500b6260 0x7f4f500cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.154+0000 7f4f555ff700 1 -- 172.21.15.92:0/3358853134 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f500bbfe0 con 0x7f4f50160d30 2023-11-28T11:21:20.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.154+0000 7f4f4f7fe700 1 --2- 172.21.15.92:0/3358853134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f50160d30 0x7f4f50148470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.154+0000 7f4f555ff700 1 -- 172.21.15.92:0/3358853134 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f500bb3d0 con 0x7f4f5006a7b0 2023-11-28T11:21:20.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.154+0000 7f4f555ff700 1 -- 172.21.15.92:0/3358853134 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f500bb6b0 con 0x7f4f500b6260 2023-11-28T11:21:20.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.154+0000 7f0ba619b700 1 --2- 172.21.15.92:0/739223527 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ba0142e80 0x7f0ba006a270 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c009e80 tx=0x7f0b9c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=683d96b925efefa5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:20.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.155+0000 7f401b7ff700 1 Processor -- start 2023-11-28T11:21:20.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.155+0000 7f0ba5199700 1 -- 172.21.15.92:0/739223527 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c00e070 con 0x7f0ba0142e80 2023-11-28T11:21:20.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.155+0000 7f0ba5199700 1 -- 172.21.15.92:0/739223527 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b9c003a20 con 0x7f0ba0142e80 2023-11-28T11:21:20.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.155+0000 7f0ba5199700 1 -- 172.21.15.92:0/739223527 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c003db0 con 0x7f0ba0142e80 2023-11-28T11:21:20.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.155+0000 7f65d3fff700 1 Processor -- start 2023-11-28T11:21:20.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.155+0000 7f4f4f7fe700 1 -- 172.21.15.92:0/3358853134 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f500b6260 msgr2=0x7f4f500cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.155+0000 7f4f4f7fe700 1 --2- 172.21.15.92:0/3358853134 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f500b6260 0x7f4f500cfc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.155+0000 7f4f4f7fe700 1 -- 172.21.15.92:0/3358853134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f5006a7b0 msgr2=0x7f4f5014b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.155+0000 7f4f4f7fe700 1 --2- 172.21.15.92:0/3358853134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f5006a7b0 0x7f4f5014b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.155+0000 7f4f4f7fe700 1 -- 172.21.15.92:0/3358853134 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f40009b30 con 0x7f4f50160d30 2023-11-28T11:21:20.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.156+0000 7f0ba83ff700 1 -- 172.21.15.92:0/739223527 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ba0142e80 msgr2=0x7f0ba006a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.156+0000 7f4f4f7fe700 1 --2- 172.21.15.92:0/3358853134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f50160d30 0x7f4f50148470 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4f4800e560 tx=0x7f4f4800e920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:20.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.156+0000 7f0ba83ff700 1 --2- 172.21.15.92:0/739223527 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ba0142e80 0x7f0ba006a270 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c009e80 tx=0x7f0b9c00b330 comp rx=0 tx=0).stop 2023-11-28T11:21:20.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.156+0000 7f4f467fc700 1 --2- 172.21.15.92:0/3358853134 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4f500b6260 0x7f4f500cfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:21:20.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.155+0000 7f65d3fff700 1 -- start start 2023-11-28T11:21:20.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.156+0000 7f4f4effd700 1 --2- 172.21.15.92:0/3358853134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4f5006a7b0 0x7f4f5014b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:21:20.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.156+0000 7f4f47fff700 1 -- 172.21.15.92:0/3358853134 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4800c070 con 0x7f4f50160d30 2023-11-28T11:21:20.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.157+0000 7f4f47fff700 1 -- 172.21.15.92:0/3358853134 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f48004020 con 0x7f4f50160d30 2023-11-28T11:21:20.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.157+0000 7f4f555ff700 1 -- 172.21.15.92:0/3358853134 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f50164c30 con 0x7f4f50160d30 2023-11-28T11:21:20.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.157+0000 7f4f555ff700 1 -- 172.21.15.92:0/3358853134 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f50164950 con 0x7f4f50160d30 2023-11-28T11:21:20.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.158+0000 7f4f47fff700 1 -- 172.21.15.92:0/3358853134 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f480043b0 con 0x7f4f50160d30 2023-11-28T11:21:20.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.158+0000 7f4f555ff700 1 -- 172.21.15.92:0/3358853134 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4f34000fc0 con 0x7f4f50160d30 2023-11-28T11:21:20.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.157+0000 7f65d3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65cc06a270 0x7f65cc153c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.157+0000 7f65d3fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f65cc0cdee0 0x7f65cc0c2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.157+0000 7f65d3fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65cc1583c0 0x7f65cc158860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.157+0000 7f65d3fff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65cc158da0 con 0x7f65cc06a270 2023-11-28T11:21:20.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.157+0000 7f65d3fff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65cc155160 con 0x7f65cc1583c0 2023-11-28T11:21:20.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.157+0000 7f65d3fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65cc1553b0 con 0x7f65cc0cdee0 2023-11-28T11:21:20.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.158+0000 7f65d1d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65cc06a270 0x7f65cc153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.158+0000 7f65d159a700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f65cc0cdee0 0x7f65cc0c2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.158+0000 7f65d159a700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f65cc0cdee0 0x7f65cc0c2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.92:33954/0 (socket says 172.21.15.92:33954) 2023-11-28T11:21:20.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.158+0000 7f65d159a700 1 -- 172.21.15.92:0/4037736487 learned_addr learned my addr 172.21.15.92:0/4037736487 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:20.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.157+0000 7f0ba83ff700 1 -- 172.21.15.92:0/739223527 shutdown_connections 2023-11-28T11:21:20.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.157+0000 7f0ba83ff700 1 --2- 172.21.15.92:0/739223527 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ba0142e80 0x7f0ba006a270 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.157+0000 7f0ba83ff700 1 -- 172.21.15.92:0/739223527 >> 172.21.15.92:0/739223527 conn(0x7f0ba00cb490 msgr2=0x7f0ba00cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:20.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.155+0000 7f401b7ff700 1 -- start start 2023-11-28T11:21:20.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.157+0000 7f401b7ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40140d2df0 0x7f40140cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.157+0000 7f401b7ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f40140b1c60 0x7f40140cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.157+0000 7f401b7ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40140d1950 con 0x7f40140b4fc0 2023-11-28T11:21:20.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.157+0000 7f401b7ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40140d3d30 con 0x7f40140b1c60 2023-11-28T11:21:20.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.157+0000 7f401b7ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40140b2500 con 0x7f40140d2df0 2023-11-28T11:21:20.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.158+0000 7f4019d9c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f40140b1c60 0x7f40140cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.158+0000 7f4019d9c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f40140b1c60 0x7f40140cf010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48762/0 (socket says 172.21.15.92:48762) 2023-11-28T11:21:20.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.158+0000 7f4019d9c700 1 -- 172.21.15.92:0/3951479326 learned_addr learned my addr 172.21.15.92:0/3951479326 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:20.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.158+0000 7f0ba83ff700 1 -- 172.21.15.92:0/739223527 shutdown_connections 2023-11-28T11:21:20.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.158+0000 7f65d259c700 1 --2- 172.21.15.92:0/4037736487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65cc1583c0 0x7f65cc158860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.159+0000 7f65d259c700 1 -- 172.21.15.92:0/4037736487 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f65cc0cdee0 msgr2=0x7f65cc0c2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.159+0000 7f65d259c700 1 --2- 172.21.15.92:0/4037736487 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f65cc0cdee0 0x7f65cc0c2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.159+0000 7f65d259c700 1 -- 172.21.15.92:0/4037736487 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65cc06a270 msgr2=0x7f65cc153c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.159+0000 7f65d259c700 1 --2- 172.21.15.92:0/4037736487 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65cc06a270 0x7f65cc153c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.159+0000 7f65d259c700 1 -- 172.21.15.92:0/4037736487 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c400d050 con 0x7f65cc1583c0 2023-11-28T11:21:20.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.159+0000 7f0ba83ff700 1 -- 172.21.15.92:0/739223527 wait complete. 2023-11-28T11:21:20.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.159+0000 7f4019d9c700 1 -- 172.21.15.92:0/3951479326 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40140d2df0 msgr2=0x7f40140cd620 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T11:21:20.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.159+0000 7f6c5cdff700 1 Processor -- start 2023-11-28T11:21:20.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.160+0000 7f401959b700 1 --2- 172.21.15.92:0/3951479326 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40140d2df0 0x7f40140cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.160+0000 7f4019d9c700 1 --2- 172.21.15.92:0/3951479326 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40140d2df0 0x7f40140cd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.160+0000 7f4019d9c700 1 -- 172.21.15.92:0/3951479326 >> v1:172.21.15.169:6789/0 conn(0x7f40140b4fc0 legacy=0x7f40140d31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:20.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.160+0000 7f4019d9c700 1 -- 172.21.15.92:0/3951479326 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40140d3a50 con 0x7f40140b1c60 2023-11-28T11:21:20.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.160+0000 7f401959b700 1 --2- 172.21.15.92:0/3951479326 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40140d2df0 0x7f40140cd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:21:20.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.160+0000 7f6c5cdff700 1 -- start start 2023-11-28T11:21:20.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.160+0000 7f4019d9c700 1 --2- 172.21.15.92:0/3951479326 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40140b1c60 0x7f40140cf010 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f400c009e50 tx=0x7f400c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac3d8508c4de8bd2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:20.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.161+0000 7f400bfff700 1 -- 172.21.15.92:0/3951479326 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400c00e070 con 0x7f40140b1c60 2023-11-28T11:21:20.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.161+0000 7f400bfff700 1 -- 172.21.15.92:0/3951479326 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f400c003a20 con 0x7f40140b1c60 2023-11-28T11:21:20.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.161+0000 7f400bfff700 1 -- 172.21.15.92:0/3951479326 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400c003dd0 con 0x7f40140b1c60 2023-11-28T11:21:20.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.161+0000 7f65d259c700 1 --2- 172.21.15.92:0/4037736487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65cc1583c0 0x7f65cc158860 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f65bc00c390 tx=0x7f65bc00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:20.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.161+0000 7f65c27fc700 1 -- 172.21.15.92:0/4037736487 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65bc0054e0 con 0x7f65cc1583c0 2023-11-28T11:21:20.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.161+0000 7f65c27fc700 1 -- 172.21.15.92:0/4037736487 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65bc00ce50 con 0x7f65cc1583c0 2023-11-28T11:21:20.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.161+0000 7f65c27fc700 1 -- 172.21.15.92:0/4037736487 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65bc015ba0 con 0x7f65cc1583c0 2023-11-28T11:21:20.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.162+0000 7f4f47fff700 1 -- 172.21.15.92:0/3358853134 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4f480036a0 con 0x7f4f50160d30 2023-11-28T11:21:20.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.161+0000 7f0ba83ff700 1 Processor -- start 2023-11-28T11:21:20.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.162+0000 7f0ba83ff700 1 -- start start 2023-11-28T11:21:20.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.162+0000 7f6c5cdff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6c58160f40 0x7f6c580ada50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.162+0000 7f6c5cdff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c580adf90 0x7f6c5806a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.162+0000 7f6c5cdff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c581657d0 con 0x7f6c5806a7b0 2023-11-28T11:21:20.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.162+0000 7f6c5cdff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c5806ac30 con 0x7f6c58160f40 2023-11-28T11:21:20.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.162+0000 7f6c5cdff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c58164c50 con 0x7f6c580adf90 2023-11-28T11:21:20.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.163+0000 7f401b7ff700 1 -- 172.21.15.92:0/3951479326 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40140b1c60 msgr2=0x7f40140cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.162+0000 7f6c55d9b700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c580adf90 0x7f6c5806a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.163+0000 7f6c55d9b700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c580adf90 0x7f6c5806a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.92:33972/0 (socket says 172.21.15.92:33972) 2023-11-28T11:21:20.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.163+0000 7f6c55d9b700 1 -- 172.21.15.92:0/1766927381 learned_addr learned my addr 172.21.15.92:0/1766927381 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:20.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.163+0000 7f401b7ff700 1 --2- 172.21.15.92:0/3951479326 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40140b1c60 0x7f40140cf010 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f400c009e50 tx=0x7f400c00b330 comp rx=0 tx=0).stop 2023-11-28T11:21:20.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.163+0000 7f6c5659c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6c58160f40 0x7f6c580ada50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.163+0000 7f65d3fff700 1 -- 172.21.15.92:0/4037736487 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65cc0cd6e0 con 0x7f65cc1583c0 2023-11-28T11:21:20.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.163+0000 7f401b7ff700 1 -- 172.21.15.92:0/3951479326 shutdown_connections 2023-11-28T11:21:20.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.164+0000 7f401b7ff700 1 --2- 172.21.15.92:0/3951479326 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40140b1c60 0x7f40140cf010 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.164+0000 7f4f47fff700 1 --2- 172.21.15.92:0/3358853134 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f4f300682f0 0x7f4f3006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.164+0000 7f401b7ff700 1 --2- 172.21.15.92:0/3951479326 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40140d2df0 0x7f40140cd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.164+0000 7f6c55d9b700 1 -- 172.21.15.92:0/1766927381 >> v2:172.21.15.92:3300/0 conn(0x7f6c58160f40 msgr2=0x7f6c580ada50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:20.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.164+0000 7f6c55d9b700 1 --2- 172.21.15.92:0/1766927381 >> v2:172.21.15.92:3300/0 conn(0x7f6c58160f40 0x7f6c580ada50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.164+0000 7f6c55d9b700 1 -- 172.21.15.92:0/1766927381 >> v1:172.21.15.169:6789/0 conn(0x7f6c5806a7b0 legacy=0x7f6c58070910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.164+0000 7f6c5559a700 1 -- 172.21.15.92:0/1766927381 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1280740230 0 0) 0x7f6c581657d0 con 0x7f6c5806a7b0 2023-11-28T11:21:20.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.164+0000 7f65d3fff700 1 -- 172.21.15.92:0/4037736487 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65cc0cd400 con 0x7f65cc1583c0 2023-11-28T11:21:20.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.164+0000 7f401b7ff700 1 -- 172.21.15.92:0/3951479326 >> 172.21.15.92:0/3951479326 conn(0x7f40140719a0 msgr2=0x7f4014071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:20.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.164+0000 7f6c55d9b700 1 -- 172.21.15.92:0/1766927381 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c58164f30 con 0x7f6c580adf90 2023-11-28T11:21:20.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.165+0000 7f401b7ff700 1 -- 172.21.15.92:0/3951479326 shutdown_connections 2023-11-28T11:21:20.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.165+0000 7f4f47fff700 1 -- 172.21.15.92:0/3358853134 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f4f48086580 con 0x7f4f50160d30 2023-11-28T11:21:20.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.165+0000 7f4f4effd700 1 --2- 172.21.15.92:0/3358853134 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f4f300682f0 0x7f4f3006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.165+0000 7f4f47fff700 1 --2- 172.21.15.92:0/3358853134 >> [v2:172.21.15.92:6808/115823,v1:172.21.15.92:6809/115823] conn(0x7f4f30071f40 0x7f4f30074370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.165+0000 7f4f47fff700 1 -- 172.21.15.92:0/3358853134 --> [v2:172.21.15.92:6808/115823,v1:172.21.15.92:6809/115823] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4f30074a40 con 0x7f4f30071f40 2023-11-28T11:21:20.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.165+0000 7f4f47fff700 1 -- 172.21.15.92:0/3358853134 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f480868f0 con 0x7f4f50160d30 2023-11-28T11:21:20.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.165+0000 7f0ba83ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ba006a7b0 0x7f0ba01edd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.165+0000 7f0ba83ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ba00ca600 0x7f0ba015b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.165+0000 7f0ba83ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ba0142e80 0x7f0ba015b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.165+0000 7f0ba83ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba015c4d0 con 0x7f0ba00ca600 2023-11-28T11:21:20.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.165+0000 7f0ba83ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba00b4810 con 0x7f0ba0142e80 2023-11-28T11:21:20.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.165+0000 7f0ba83ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba00b4000 con 0x7f0ba006a7b0 2023-11-28T11:21:20.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.166+0000 7f0ba699c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ba0142e80 0x7f0ba015b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.166+0000 7f0ba699c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ba0142e80 0x7f0ba015b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.92:57026/0 (socket says 172.21.15.92:57026) 2023-11-28T11:21:20.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.166+0000 7f0ba699c700 1 -- 172.21.15.92:0/1154011601 learned_addr learned my addr 172.21.15.92:0/1154011601 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:20.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.166+0000 7f0ba599a700 1 --2- 172.21.15.92:0/1154011601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ba00ca600 0x7f0ba015b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.166+0000 7f0ba699c700 1 -- 172.21.15.92:0/1154011601 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ba006a7b0 msgr2=0x7f0ba01edd50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T11:21:20.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.166+0000 7f0ba699c700 1 --2- 172.21.15.92:0/1154011601 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0ba006a7b0 0x7f0ba01edd50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.167+0000 7f0ba699c700 1 -- 172.21.15.92:0/1154011601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ba00ca600 msgr2=0x7f0ba015b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.165+0000 7f6c55d9b700 1 --2- 172.21.15.92:0/1766927381 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c580adf90 0x7f6c5806a270 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6c480098f0 tx=0x7f6c48009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6628f25a7bdc0eb2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:20.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.166+0000 7f6c5559a700 1 -- 172.21.15.92:0/1766927381 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4800b8a0 con 0x7f6c580adf90 2023-11-28T11:21:20.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.166+0000 7f6c5559a700 1 -- 172.21.15.92:0/1766927381 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c4800ba00 con 0x7f6c580adf90 2023-11-28T11:21:20.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.167+0000 7f6c5559a700 1 -- 172.21.15.92:0/1766927381 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4800b8a0 con 0x7f6c580adf90 2023-11-28T11:21:20.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.167+0000 7f4f467fc700 1 --2- 172.21.15.92:0/3358853134 >> [v2:172.21.15.92:6808/115823,v1:172.21.15.92:6809/115823] conn(0x7f4f30071f40 0x7f4f30074370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.166+0000 7f65d3fff700 1 -- 172.21.15.92:0/4037736487 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f65b4000fc0 con 0x7f65cc1583c0 2023-11-28T11:21:20.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.167+0000 7f65c27fc700 1 -- 172.21.15.92:0/4037736487 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f65bc025070 con 0x7f65cc1583c0 2023-11-28T11:21:20.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.167+0000 7f0ba699c700 1 --2- 172.21.15.92:0/1154011601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ba00ca600 0x7f0ba015b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.167+0000 7f0ba699c700 1 -- 172.21.15.92:0/1154011601 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b9c009b30 con 0x7f0ba0142e80 2023-11-28T11:21:20.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.167+0000 7f6c5cdff700 1 -- 172.21.15.92:0/1766927381 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c580adf90 msgr2=0x7f6c5806a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.167+0000 7f0ba599a700 1 --2- 172.21.15.92:0/1154011601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ba00ca600 0x7f0ba015b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:21:20.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.168+0000 7f6c5cdff700 1 --2- 172.21.15.92:0/1766927381 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c580adf90 0x7f6c5806a270 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6c480098f0 tx=0x7f6c48009c00 comp rx=0 tx=0).stop 2023-11-28T11:21:20.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.168+0000 7f6c5cdff700 1 -- 172.21.15.92:0/1766927381 shutdown_connections 2023-11-28T11:21:20.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.168+0000 7f6c5cdff700 1 --2- 172.21.15.92:0/1766927381 >> v2:172.21.15.92:3300/0 conn(0x7f6c58160f40 0x7f6c580ada50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.168+0000 7f6c5cdff700 1 --2- 172.21.15.92:0/1766927381 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c580adf90 0x7f6c5806a270 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.168+0000 7f6c5cdff700 1 -- 172.21.15.92:0/1766927381 >> 172.21.15.92:0/1766927381 conn(0x7f6c580cb490 msgr2=0x7f6c580cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:20.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.169+0000 7f4f467fc700 1 --2- 172.21.15.92:0/3358853134 >> [v2:172.21.15.92:6808/115823,v1:172.21.15.92:6809/115823] conn(0x7f4f30071f40 0x7f4f30074370 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:20.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.169+0000 7f4f4effd700 1 --2- 172.21.15.92:0/3358853134 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f4f300682f0 0x7f4f3006a7a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4f40009fd0 tx=0x7f4f40009000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:20.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.169+0000 7f0ba699c700 1 --2- 172.21.15.92:0/1154011601 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ba0142e80 0x7f0ba015b8a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0b9800c780 tx=0x7f0b9800ca90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:20.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.166+0000 7f401b7ff700 1 -- 172.21.15.92:0/3951479326 wait complete. 2023-11-28T11:21:20.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.169+0000 7f401b7ff700 1 Processor -- start 2023-11-28T11:21:20.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.168+0000 7f65c27fc700 1 --2- 172.21.15.92:0/4037736487 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f65b0068340 0x7f65b006a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.169+0000 7f65c27fc700 1 -- 172.21.15.92:0/4037736487 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f65bc08d820 con 0x7f65cc1583c0 2023-11-28T11:21:20.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.169+0000 7f65c27fc700 1 --2- 172.21.15.92:0/4037736487 >> [v2:172.21.15.92:6810/115824,v1:172.21.15.92:6812/115824] conn(0x7f65b0072010 0x7f65b0074440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.169+0000 7f65c27fc700 1 -- 172.21.15.92:0/4037736487 --> [v2:172.21.15.92:6810/115824,v1:172.21.15.92:6812/115824] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f65b0074b10 con 0x7f65b0072010 2023-11-28T11:21:20.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.169+0000 7f65c27fc700 1 -- 172.21.15.92:0/4037736487 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65bc08fe00 con 0x7f65cc1583c0 2023-11-28T11:21:20.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.169+0000 7f6c5cdff700 1 -- 172.21.15.92:0/1766927381 shutdown_connections 2023-11-28T11:21:20.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.169+0000 7f6c5cdff700 1 -- 172.21.15.92:0/1766927381 wait complete. 2023-11-28T11:21:20.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.169+0000 7f401b7ff700 1 -- start start 2023-11-28T11:21:20.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.169+0000 7f0b96ffd700 1 -- 172.21.15.92:0/1154011601 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b980054e0 con 0x7f0ba0142e80 2023-11-28T11:21:20.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.170+0000 7f78629ff700 1 Processor -- start 2023-11-28T11:21:20.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.170+0000 7f0ba83ff700 1 -- 172.21.15.92:0/1154011601 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ba00cc5f0 con 0x7f0ba0142e80 2023-11-28T11:21:20.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.170+0000 7f0b96ffd700 1 -- 172.21.15.92:0/1154011601 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b98003680 con 0x7f0ba0142e80 2023-11-28T11:21:20.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.170+0000 7f0b96ffd700 1 -- 172.21.15.92:0/1154011601 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b98003a30 con 0x7f0ba0142e80 2023-11-28T11:21:20.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.170+0000 7f4f47fff700 1 -- 172.21.15.92:0/3358853134 <== osd.0 v2:172.21.15.92:6808/115823 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f4f30074a40 con 0x7f4f30071f40 2023-11-28T11:21:20.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.170+0000 7f65d1d9b700 1 --2- 172.21.15.92:0/4037736487 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f65b0068340 0x7f65b006a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.170+0000 7f78629ff700 1 -- start start 2023-11-28T11:21:20.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.171+0000 7f0ba83ff700 1 -- 172.21.15.92:0/1154011601 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ba00cab40 con 0x7f0ba0142e80 2023-11-28T11:21:20.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.171+0000 7f65d159a700 1 --2- 172.21.15.92:0/4037736487 >> [v2:172.21.15.92:6810/115824,v1:172.21.15.92:6812/115824] conn(0x7f65b0072010 0x7f65b0074440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.172+0000 7f935c9ff700 1 Processor -- start 2023-11-28T11:21:20.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.172+0000 7f6c5cdff700 1 Processor -- start 2023-11-28T11:21:20.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.172+0000 7f401b7ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40140b1c60 0x7f40140cca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.172+0000 7f401b7ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40140b4fc0 0x7f401415f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.172+0000 7f0ba83ff700 1 -- 172.21.15.92:0/1154011601 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0ba00cc8d0 con 0x7f0ba0142e80 2023-11-28T11:21:20.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.172+0000 7f6c5cdff700 1 -- start start 2023-11-28T11:21:20.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.172+0000 7f78629ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f785c06a270 0x7f785c06a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.172+0000 7f401b7ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40140d2df0 0x7f40141f2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.172+0000 7f401b7ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40141f22d0 con 0x7f40140b1c60 2023-11-28T11:21:20.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.172+0000 7f401b7ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40140cbd30 con 0x7f40140d2df0 2023-11-28T11:21:20.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.172+0000 7f401b7ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40140cbfb0 con 0x7f40140b4fc0 2023-11-28T11:21:20.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.172+0000 7f401959b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40140b1c60 0x7f40140cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.172+0000 7f78629ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f785c0cced0 0x7f785c0cd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.172+0000 7f78629ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f785c1657d0 con 0x7f785c160f40 2023-11-28T11:21:20.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.173+0000 7f401959b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40140b1c60 0x7f40140cca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48798/0 (socket says 172.21.15.92:48798) 2023-11-28T11:21:20.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.172+0000 7f78629ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f785c06ac20 con 0x7f785c0cced0 2023-11-28T11:21:20.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.172+0000 7f78629ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f785c164c50 con 0x7f785c06a270 2023-11-28T11:21:20.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.173+0000 7f401959b700 1 -- 172.21.15.92:0/2008247054 learned_addr learned my addr 172.21.15.92:0/2008247054 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:20.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.173+0000 7f4019d9c700 1 --2- 172.21.15.92:0/2008247054 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40140d2df0 0x7f40141f2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.173+0000 7f785bfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f785c160f40 0x7f785c0ada50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47426/0 (socket says 172.21.15.92:47426) 2023-11-28T11:21:20.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.173+0000 7f785bfff700 1 -- 172.21.15.92:0/3193811152 learned_addr learned my addr 172.21.15.92:0/3193811152 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:20.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.173+0000 7f65d1d9b700 1 --2- 172.21.15.92:0/4037736487 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f65b0068340 0x7f65b006a7f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f65cc159000 tx=0x7f65c8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:20.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.173+0000 7f65d159a700 1 --2- 172.21.15.92:0/4037736487 >> [v2:172.21.15.92:6810/115824,v1:172.21.15.92:6812/115824] conn(0x7f65b0072010 0x7f65b0074440 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:20.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.173+0000 7f401959b700 1 -- 172.21.15.92:0/2008247054 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40140b4fc0 msgr2=0x7f401415f6f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T11:21:20.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.173+0000 7f401959b700 1 --2- 172.21.15.92:0/2008247054 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40140b4fc0 0x7f401415f6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.173+0000 7f401959b700 1 -- 172.21.15.92:0/2008247054 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40140d2df0 msgr2=0x7f40141f2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.173+0000 7f401959b700 1 --2- 172.21.15.92:0/2008247054 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f40140d2df0 0x7f40141f2d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.173+0000 7f401959b700 1 -- 172.21.15.92:0/2008247054 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f400c009b30 con 0x7f40140b1c60 2023-11-28T11:21:20.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.172+0000 7f935c9ff700 1 -- start start 2023-11-28T11:21:20.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.174+0000 7f935c9ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f935806a270 0x7f93580c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.174+0000 7f935c9ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f935815ce70 0x7f935806a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.174+0000 7f935c9ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f93580cdee0 0x7f935806c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.174+0000 7f935c9ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f935806ab80 con 0x7f935815ce70 2023-11-28T11:21:20.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.174+0000 7f935c9ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93580ccb60 con 0x7f935806a270 2023-11-28T11:21:20.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.174+0000 7f935c9ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93580cce40 con 0x7f93580cdee0 2023-11-28T11:21:20.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.174+0000 7f401959b700 1 --2- 172.21.15.92:0/2008247054 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40140b1c60 0x7f40140cca70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f401000c820 tx=0x7f401000cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:20.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.174+0000 7f6c5cdff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c5806a7b0 0x7f6c58154b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.175+0000 7f935659c700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f935806a270 0x7f93580c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.175+0000 7f9355d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f935815ce70 0x7f935806a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.175+0000 7f9356d9d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f93580cdee0 0x7f935806c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.175+0000 7f6c5cdff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c580adf90 0x7f6c580c81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.175+0000 7f6c5cdff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c58160f40 0x7f6c580b2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.175+0000 7f6c5cdff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c580b1d20 con 0x7f6c5806a7b0 2023-11-28T11:21:20.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.175+0000 7f6c5cdff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c580b2000 con 0x7f6c58160f40 2023-11-28T11:21:20.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.175+0000 7f6c5cdff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c580b22e0 con 0x7f6c580adf90 2023-11-28T11:21:20.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.173+0000 7f785b7fe700 1 --2- 172.21.15.92:0/3193811152 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f785c06a270 0x7f785c06a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.173+0000 7f7860f9c700 1 --2- 172.21.15.92:0/3193811152 >> v2:172.21.15.169:3300/0 conn(0x7f785c0cced0 0x7f785c0cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.174+0000 7f7860f9c700 1 -- 172.21.15.92:0/3193811152 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f785c06a270 msgr2=0x7f785c06a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:20.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.174+0000 7f7860f9c700 1 --2- 172.21.15.92:0/3193811152 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f785c06a270 0x7f785c06a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.174+0000 7f7860f9c700 1 -- 172.21.15.92:0/3193811152 >> v1:172.21.15.92:6789/0 conn(0x7f785c160f40 legacy=0x7f785c0ada50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.174+0000 7f7860f9c700 1 -- 172.21.15.92:0/3193811152 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f785c164f30 con 0x7f785c0cced0 2023-11-28T11:21:20.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.175+0000 7f935659c700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f935806a270 0x7f93580c9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.92:57066/0 (socket says 172.21.15.92:57066) 2023-11-28T11:21:20.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.175+0000 7f9356d9d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f93580cdee0 0x7f935806c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.92:34000/0 (socket says 172.21.15.92:34000) 2023-11-28T11:21:20.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.175+0000 7f935659c700 1 -- 172.21.15.92:0/673431313 learned_addr learned my addr 172.21.15.92:0/673431313 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:20.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.175+0000 7f4009ffb700 1 -- 172.21.15.92:0/2008247054 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4010011800 con 0x7f40140b1c60 2023-11-28T11:21:20.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.175+0000 7f4009ffb700 1 -- 172.21.15.92:0/2008247054 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4010011960 con 0x7f40140b1c60 2023-11-28T11:21:20.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.175+0000 7f4009ffb700 1 -- 172.21.15.92:0/2008247054 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4010011d10 con 0x7f40140b1c60 2023-11-28T11:21:20.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.175+0000 7f401b7ff700 1 -- 172.21.15.92:0/2008247054 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40140b88f0 con 0x7f40140b1c60 2023-11-28T11:21:20.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.175+0000 7f401b7ff700 1 -- 172.21.15.92:0/2008247054 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40140d15e0 con 0x7f40140b1c60 2023-11-28T11:21:20.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.175+0000 7f7860f9c700 1 --2- 172.21.15.92:0/3193811152 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f785c0cced0 0x7f785c0cd350 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7854009d90 tx=0x7f785400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b5a037ed7d15a32 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:20.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.176+0000 7f6c56d9d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c58160f40 0x7f6c580b2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.176+0000 7f6c56d9d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c58160f40 0x7f6c580b2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.92:57068/0 (socket says 172.21.15.92:57068) 2023-11-28T11:21:20.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.176+0000 7f6c56d9d700 1 -- 172.21.15.92:0/1083449725 learned_addr learned my addr 172.21.15.92:0/1083449725 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:20.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.176+0000 7f6c5659c700 1 --2- 172.21.15.92:0/1083449725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c5806a7b0 0x7f6c58154b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.176+0000 7f65c27fc700 1 -- 172.21.15.92:0/4037736487 <== osd.1 v2:172.21.15.92:6810/115824 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f65b0074b10 con 0x7f65b0072010 2023-11-28T11:21:20.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.176+0000 7f9356d9d700 1 -- 172.21.15.92:0/673431313 >> v2:172.21.15.169:3300/0 conn(0x7f935806a270 msgr2=0x7f93580c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:20.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.176+0000 7f6c55d9b700 1 --2- 172.21.15.92:0/1083449725 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c580adf90 0x7f6c580c81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.176+0000 7f785affd700 1 -- 172.21.15.92:0/3193811152 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785400e070 con 0x7f785c0cced0 2023-11-28T11:21:20.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.176+0000 7f785affd700 1 -- 172.21.15.92:0/3193811152 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f785400bbd0 con 0x7f785c0cced0 2023-11-28T11:21:20.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.176+0000 7f785affd700 1 -- 172.21.15.92:0/3193811152 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7854012430 con 0x7f785c0cced0 2023-11-28T11:21:20.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.177+0000 7f9356d9d700 1 --2- 172.21.15.92:0/673431313 >> v2:172.21.15.169:3300/0 conn(0x7f935806a270 0x7f93580c9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.177+0000 7f9356d9d700 1 -- 172.21.15.92:0/673431313 >> v2:172.21.15.92:3300/0 conn(0x7f935815ce70 msgr2=0x7f935806a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:20.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.177+0000 7f9356d9d700 1 --2- 172.21.15.92:0/673431313 >> v2:172.21.15.92:3300/0 conn(0x7f935815ce70 0x7f935806a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.177+0000 7f9356d9d700 1 -- 172.21.15.92:0/673431313 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93580cd120 con 0x7f93580cdee0 2023-11-28T11:21:20.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.177+0000 7f401b7ff700 1 -- 172.21.15.92:0/2008247054 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f40140b21b0 con 0x7f40140b1c60 2023-11-28T11:21:20.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.177+0000 7f9355d9b700 1 --2- 172.21.15.92:0/673431313 >> v2:172.21.15.92:3300/0 conn(0x7f935815ce70 0x7f935806a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:21:20.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.177+0000 7f935659c700 1 --2- 172.21.15.92:0/673431313 >> v2:172.21.15.169:3300/0 conn(0x7f935806a270 0x7f93580c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:21:20.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.177+0000 7f9356d9d700 1 --2- 172.21.15.92:0/673431313 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f93580cdee0 0x7f935806c6a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f935806b6d0 tx=0x7f934800b490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a44feed0666acc2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:20.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.178+0000 7f935559a700 1 -- 172.21.15.92:0/673431313 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93480090e0 con 0x7f93580cdee0 2023-11-28T11:21:20.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.178+0000 7f935559a700 1 -- 172.21.15.92:0/673431313 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9348009240 con 0x7f93580cdee0 2023-11-28T11:21:20.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.178+0000 7f935559a700 1 -- 172.21.15.92:0/673431313 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93480095d0 con 0x7f93580cdee0 2023-11-28T11:21:20.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.178+0000 7f6c56d9d700 1 -- 172.21.15.92:0/1083449725 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c580adf90 msgr2=0x7f6c580c81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.178+0000 7f78629ff700 1 -- 172.21.15.92:0/3193811152 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f785c0cced0 msgr2=0x7f785c0cd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.178+0000 7f78629ff700 1 --2- 172.21.15.92:0/3193811152 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f785c0cced0 0x7f785c0cd350 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7854009d90 tx=0x7f785400b330 comp rx=0 tx=0).stop 2023-11-28T11:21:20.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.179+0000 7f4009ffb700 1 -- 172.21.15.92:0/2008247054 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4010021070 con 0x7f40140b1c60 2023-11-28T11:21:20.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.179+0000 7f78629ff700 1 -- 172.21.15.92:0/3193811152 shutdown_connections 2023-11-28T11:21:20.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.179+0000 7f78629ff700 1 --2- 172.21.15.92:0/3193811152 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f785c0cced0 0x7f785c0cd350 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.179+0000 7f78629ff700 1 --2- 172.21.15.92:0/3193811152 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f785c06a270 0x7f785c06a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.179+0000 7f78629ff700 1 -- 172.21.15.92:0/3193811152 >> 172.21.15.92:0/3193811152 conn(0x7f785c0cb490 msgr2=0x7f785c0cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:20.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.179+0000 7f935c9ff700 1 -- 172.21.15.92:0/673431313 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f93580cdee0 msgr2=0x7f935806c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.179+0000 7f935c9ff700 1 --2- 172.21.15.92:0/673431313 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f93580cdee0 0x7f935806c6a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f935806b6d0 tx=0x7f934800b490 comp rx=0 tx=0).stop 2023-11-28T11:21:20.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.178+0000 7f0b96ffd700 1 -- 172.21.15.92:0/1154011601 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0b98019030 con 0x7f0ba0142e80 2023-11-28T11:21:20.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.179+0000 7f935c9ff700 1 -- 172.21.15.92:0/673431313 shutdown_connections 2023-11-28T11:21:20.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.179+0000 7f935c9ff700 1 --2- 172.21.15.92:0/673431313 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f93580cdee0 0x7f935806c6a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.179+0000 7f935c9ff700 1 --2- 172.21.15.92:0/673431313 >> v2:172.21.15.169:3300/0 conn(0x7f935806a270 0x7f93580c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.179+0000 7f935c9ff700 1 --2- 172.21.15.92:0/673431313 >> v2:172.21.15.92:3300/0 conn(0x7f935815ce70 0x7f935806a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.179+0000 7f78629ff700 1 -- 172.21.15.92:0/3193811152 shutdown_connections 2023-11-28T11:21:20.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.179+0000 7f935c9ff700 1 -- 172.21.15.92:0/673431313 >> 172.21.15.92:0/673431313 conn(0x7f93580cb490 msgr2=0x7f93580cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:20.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.179+0000 7f78629ff700 1 -- 172.21.15.92:0/3193811152 wait complete. 2023-11-28T11:21:20.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.180+0000 7f935c9ff700 1 -- 172.21.15.92:0/673431313 shutdown_connections 2023-11-28T11:21:20.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.179+0000 7f6c56d9d700 1 --2- 172.21.15.92:0/1083449725 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6c580adf90 0x7f6c580c81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.179+0000 7f6c56d9d700 1 -- 172.21.15.92:0/1083449725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c5806a7b0 msgr2=0x7f6c58154b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.179+0000 7f6c56d9d700 1 --2- 172.21.15.92:0/1083449725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c5806a7b0 0x7f6c58154b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.179+0000 7f6c56d9d700 1 -- 172.21.15.92:0/1083449725 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c48009550 con 0x7f6c58160f40 2023-11-28T11:21:20.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.179+0000 7f6c5659c700 1 --2- 172.21.15.92:0/1083449725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c5806a7b0 0x7f6c58154b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:21:20.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.180+0000 7f935c9ff700 1 -- 172.21.15.92:0/673431313 wait complete. 2023-11-28T11:21:20.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.180+0000 7f0b96ffd700 1 --2- 172.21.15.92:0/1154011601 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f0b84068340 0x7f0b8406a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.180+0000 7f6c56d9d700 1 --2- 172.21.15.92:0/1083449725 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c58160f40 0x7f6c580b2da0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c001eb0 tx=0x7f6c4c00c730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:20.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.180+0000 7f0b96ffd700 1 -- 172.21.15.92:0/1154011601 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f0b98084e70 con 0x7f0ba0142e80 2023-11-28T11:21:20.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.181+0000 7f6c46ffd700 1 -- 172.21.15.92:0/1083449725 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c0054e0 con 0x7f6c58160f40 2023-11-28T11:21:20.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.181+0000 7f6c46ffd700 1 -- 172.21.15.92:0/1083449725 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c4c003680 con 0x7f6c58160f40 2023-11-28T11:21:20.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.181+0000 7f6c46ffd700 1 -- 172.21.15.92:0/1083449725 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c003a30 con 0x7f6c58160f40 2023-11-28T11:21:20.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.181+0000 7f0b96ffd700 1 --2- 172.21.15.92:0/1154011601 >> [v2:172.21.15.92:6800/115821,v1:172.21.15.92:6801/115821] conn(0x7f0b84072010 0x7f0b84074440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.181+0000 7f0b96ffd700 1 -- 172.21.15.92:0/1154011601 --> [v2:172.21.15.92:6800/115821,v1:172.21.15.92:6801/115821] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0b84074b10 con 0x7f0b84072010 2023-11-28T11:21:20.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.181+0000 7f0b96ffd700 1 -- 172.21.15.92:0/1154011601 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b980851b0 con 0x7f0ba0142e80 2023-11-28T11:21:20.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.181+0000 7f6c5cdff700 1 -- 172.21.15.92:0/1083449725 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c581654f0 con 0x7f6c58160f40 2023-11-28T11:21:20.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.181+0000 7f6c5cdff700 1 -- 172.21.15.92:0/1083449725 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c58165210 con 0x7f6c58160f40 2023-11-28T11:21:20.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.181+0000 7f0ba599a700 1 --2- 172.21.15.92:0/1154011601 >> [v2:172.21.15.92:6800/115821,v1:172.21.15.92:6801/115821] conn(0x7f0b84072010 0x7f0b84074440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.182+0000 7f78629ff700 1 Processor -- start 2023-11-28T11:21:20.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.182+0000 7f0ba599a700 1 --2- 172.21.15.92:0/1154011601 >> [v2:172.21.15.92:6800/115821,v1:172.21.15.92:6801/115821] conn(0x7f0b84072010 0x7f0b84074440 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:20.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.182+0000 7f935c9ff700 1 Processor -- start 2023-11-28T11:21:20.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.183+0000 7f935c9ff700 1 -- start start 2023-11-28T11:21:20.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.182+0000 7f78629ff700 1 -- start start 2023-11-28T11:21:20.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.183+0000 7f0ba619b700 1 --2- 172.21.15.92:0/1154011601 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f0b84068340 0x7f0b8406a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.184+0000 7f0ba619b700 1 --2- 172.21.15.92:0/1154011601 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f0b84068340 0x7f0b8406a7f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c000f80 tx=0x7f0b9c009000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:20.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.184+0000 7f78629ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f785c06a270 0x7f785c1f66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.184+0000 7f935c9ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f935806a270 0x7f93581f9920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.184+0000 7f935c9ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93580cdee0 0x7f935815b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.184+0000 7f935c9ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f935815ce70 0x7f93580af3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.184+0000 7f6c5cdff700 1 -- 172.21.15.92:0/1083449725 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6c58164900 con 0x7f6c58160f40 2023-11-28T11:21:20.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.185+0000 7f935659c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f935806a270 0x7f93581f9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.185+0000 7f9355d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93580cdee0 0x7f935815b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.185+0000 7f935659c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f935806a270 0x7f93581f9920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.92:34020/0 (socket says 172.21.15.92:34020) 2023-11-28T11:21:20.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.185+0000 7f9355d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93580cdee0 0x7f935815b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48820/0 (socket says 172.21.15.92:48820) 2023-11-28T11:21:20.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.185+0000 7f9356d9d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f935815ce70 0x7f93580af3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.185+0000 7f9356d9d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f935815ce70 0x7f93580af3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.92:57084/0 (socket says 172.21.15.92:57084) 2023-11-28T11:21:20.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.185+0000 7f935659c700 1 -- 172.21.15.92:0/1935976780 learned_addr learned my addr 172.21.15.92:0/1935976780 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:20.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.185+0000 7f935c9ff700 1 -- 172.21.15.92:0/1935976780 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93580c75e0 con 0x7f93580cdee0 2023-11-28T11:21:20.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.185+0000 7f935c9ff700 1 -- 172.21.15.92:0/1935976780 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93580c5a50 con 0x7f935815ce70 2023-11-28T11:21:20.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.185+0000 7f935c9ff700 1 -- 172.21.15.92:0/1935976780 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93580c5d30 con 0x7f935806a270 2023-11-28T11:21:20.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.185+0000 7f0b96ffd700 1 -- 172.21.15.92:0/1154011601 <== osd.2 v2:172.21.15.92:6800/115821 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f0b84074b10 con 0x7f0b84072010 2023-11-28T11:21:20.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.186+0000 7f78629ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f785c0cced0 0x7f785c1f9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.186+0000 7f785bfff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f785c06a270 0x7f785c1f66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.186+0000 7f785bfff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f785c06a270 0x7f785c1f66c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.92:34034/0 (socket says 172.21.15.92:34034) 2023-11-28T11:21:20.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.186+0000 7f785bfff700 1 -- 172.21.15.92:0/2727807337 learned_addr learned my addr 172.21.15.92:0/2727807337 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:20.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.186+0000 7f78629ff700 1 --2- 172.21.15.92:0/2727807337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f785c160f40 0x7f785c0b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.186+0000 7f78629ff700 1 -- 172.21.15.92:0/2727807337 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785c1f6c00 con 0x7f785c0cced0 2023-11-28T11:21:20.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.186+0000 7f78629ff700 1 -- 172.21.15.92:0/2727807337 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785c1f9ab0 con 0x7f785c160f40 2023-11-28T11:21:20.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.186+0000 7f78629ff700 1 -- 172.21.15.92:0/2727807337 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f785c0b3390 con 0x7f785c06a270 2023-11-28T11:21:20.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.187+0000 7f7860f9c700 1 --2- 172.21.15.92:0/2727807337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f785c160f40 0x7f785c0b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.187+0000 7f785b7fe700 1 --2- 172.21.15.92:0/2727807337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f785c0cced0 0x7f785c1f9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.187+0000 7f785bfff700 1 -- 172.21.15.92:0/2727807337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f785c160f40 msgr2=0x7f785c0b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.187+0000 7f785bfff700 1 --2- 172.21.15.92:0/2727807337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f785c160f40 0x7f785c0b3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.187+0000 7f9356d9d700 1 -- 172.21.15.92:0/1935976780 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f935806a270 msgr2=0x7f93581f9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.187+0000 7f9356d9d700 1 --2- 172.21.15.92:0/1935976780 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f935806a270 0x7f93581f9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.187+0000 7f9356d9d700 1 -- 172.21.15.92:0/1935976780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93580cdee0 msgr2=0x7f935815b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.187+0000 7f9356d9d700 1 --2- 172.21.15.92:0/1935976780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93580cdee0 0x7f935815b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.187+0000 7f9356d9d700 1 -- 172.21.15.92:0/1935976780 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f934c009550 con 0x7f935815ce70 2023-11-28T11:21:20.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.187+0000 7f785bfff700 1 -- 172.21.15.92:0/2727807337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f785c0cced0 msgr2=0x7f785c1f9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.187+0000 7f785bfff700 1 --2- 172.21.15.92:0/2727807337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f785c0cced0 0x7f785c1f9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.187+0000 7f785bfff700 1 -- 172.21.15.92:0/2727807337 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7854009a70 con 0x7f785c06a270 2023-11-28T11:21:20.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.187+0000 7f7860f9c700 1 --2- 172.21.15.92:0/2727807337 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f785c160f40 0x7f785c0b3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:21:20.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.188+0000 7f9356d9d700 1 --2- 172.21.15.92:0/1935976780 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f935815ce70 0x7f93580af3c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9348016000 tx=0x7f9348004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:20.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.188+0000 7f9346ffd700 1 -- 172.21.15.92:0/1935976780 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9348004450 con 0x7f935815ce70 2023-11-28T11:21:20.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.188+0000 7f9346ffd700 1 -- 172.21.15.92:0/1935976780 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f934801f070 con 0x7f935815ce70 2023-11-28T11:21:20.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.188+0000 7f9346ffd700 1 -- 172.21.15.92:0/1935976780 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93480038e0 con 0x7f935815ce70 2023-11-28T11:21:20.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.188+0000 7f4009ffb700 1 --2- 172.21.15.92:0/2008247054 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3ff80682f0 0x7f3ff806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.188+0000 7f4009ffb700 1 -- 172.21.15.92:0/2008247054 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f4010085f50 con 0x7f40140b1c60 2023-11-28T11:21:20.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.188+0000 7f4009ffb700 1 --2- 172.21.15.92:0/2008247054 >> [v2:172.21.15.92:6824/115825,v1:172.21.15.92:6825/115825] conn(0x7f3ff8071fc0 0x7f3ff80743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.188+0000 7f935c9ff700 1 -- 172.21.15.92:0/1935976780 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9348009d30 con 0x7f935815ce70 2023-11-28T11:21:20.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.188+0000 7f935c9ff700 1 -- 172.21.15.92:0/1935976780 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93580cd6e0 con 0x7f935815ce70 2023-11-28T11:21:20.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.188+0000 7f228d9ff700 1 Processor -- start 2023-11-28T11:21:20.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.188+0000 7f228d9ff700 1 -- start start 2023-11-28T11:21:20.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.189+0000 7f935c9ff700 1 -- 172.21.15.92:0/1935976780 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f93580cd400 con 0x7f935815ce70 2023-11-28T11:21:20.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.188+0000 7f6c46ffd700 1 -- 172.21.15.92:0/1083449725 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6c4c019030 con 0x7f6c58160f40 2023-11-28T11:21:20.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.189+0000 7f6c46ffd700 1 --2- 172.21.15.92:0/1083449725 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6c38068340 0x7f6c3806a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.190+0000 7f6c5659c700 1 --2- 172.21.15.92:0/1083449725 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6c38068340 0x7f6c3806a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.190+0000 7f785bfff700 1 --2- 172.21.15.92:0/2727807337 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f785c06a270 0x7f785c1f66c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f784c00c520 tx=0x7f784c00c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:20.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.190+0000 7f7858ff9700 1 -- 172.21.15.92:0/2727807337 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784c005d70 con 0x7f785c06a270 2023-11-28T11:21:20.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.190+0000 7f7858ff9700 1 -- 172.21.15.92:0/2727807337 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f784c0070d0 con 0x7f785c06a270 2023-11-28T11:21:20.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.190+0000 7f7858ff9700 1 -- 172.21.15.92:0/2727807337 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784c007440 con 0x7f785c06a270 2023-11-28T11:21:20.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.190+0000 7f4018d9a700 1 --2- 172.21.15.92:0/2008247054 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3ff80682f0 0x7f3ff806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.190+0000 7f4009ffb700 1 -- 172.21.15.92:0/2008247054 --> [v2:172.21.15.92:6824/115825,v1:172.21.15.92:6825/115825] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3ff8074ac0 con 0x7f3ff8071fc0 2023-11-28T11:21:20.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.190+0000 7f4009ffb700 1 -- 172.21.15.92:0/2008247054 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4010088570 con 0x7f40140b1c60 2023-11-28T11:21:20.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.191+0000 7f6c46ffd700 1 -- 172.21.15.92:0/1083449725 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6c4c05d580 con 0x7f6c58160f40 2023-11-28T11:21:20.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.191+0000 7f4019d9c700 1 --2- 172.21.15.92:0/2008247054 >> [v2:172.21.15.92:6824/115825,v1:172.21.15.92:6825/115825] conn(0x7f3ff8071fc0 0x7f3ff80743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.191+0000 7f78629ff700 1 -- 172.21.15.92:0/2727807337 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f785c1654f0 con 0x7f785c06a270 2023-11-28T11:21:20.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.191+0000 7f78629ff700 1 -- 172.21.15.92:0/2727807337 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f785c165210 con 0x7f785c06a270 2023-11-28T11:21:20.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.192+0000 7f9346ffd700 1 -- 172.21.15.92:0/1935976780 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f934800f050 con 0x7f935815ce70 2023-11-28T11:21:20.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.192+0000 7f4018d9a700 1 --2- 172.21.15.92:0/2008247054 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3ff80682f0 0x7f3ff806a7a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4014159b90 tx=0x7f4004006be0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:20.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.192+0000 7f4019d9c700 1 --2- 172.21.15.92:0/2008247054 >> [v2:172.21.15.92:6824/115825,v1:172.21.15.92:6825/115825] conn(0x7f3ff8071fc0 0x7f3ff80743f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:20.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.193+0000 7f228d9ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f228806a270 0x7f228806a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.191+0000 7f6c46ffd700 1 --2- 172.21.15.92:0/1083449725 >> [v2:172.21.15.169:6800/116033,v1:172.21.15.169:6801/116033] conn(0x7f6c38072010 0x7f6c38074440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.192+0000 7f6c5659c700 1 --2- 172.21.15.92:0/1083449725 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6c38068340 0x7f6c3806a7f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6c580cca50 tx=0x7f6c50003a80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:20.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.192+0000 7f6c46ffd700 1 -- 172.21.15.92:0/1083449725 --> [v2:172.21.15.169:6800/116033,v1:172.21.15.169:6801/116033] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6c38074b10 con 0x7f6c38072010 2023-11-28T11:21:20.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.194+0000 7f4009ffb700 1 -- 172.21.15.92:0/2008247054 <== osd.3 v2:172.21.15.92:6824/115825 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f3ff8074ac0 con 0x7f3ff8071fc0 2023-11-28T11:21:20.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.192+0000 7f6c46ffd700 1 -- 172.21.15.92:0/1083449725 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c4c05b0c0 con 0x7f6c58160f40 2023-11-28T11:21:20.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.193+0000 7f6c55d9b700 1 --2- 172.21.15.92:0/1083449725 >> [v2:172.21.15.169:6800/116033,v1:172.21.15.169:6801/116033] conn(0x7f6c38072010 0x7f6c38074440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.194+0000 7f228d9ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f228806ac00 con 0x7f2288160e10 2023-11-28T11:21:20.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.194+0000 7f22867fc700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f228806a270 0x7f228806a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.194+0000 7f22867fc700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f228806a270 0x7f228806a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.92:34050/0 (socket says 172.21.15.92:34050) 2023-11-28T11:21:20.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.194+0000 7f22867fc700 1 -- 172.21.15.92:0/3231431826 learned_addr learned my addr 172.21.15.92:0/3231431826 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:20.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.194+0000 7f2286ffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f2288160e10 0x7f22880ad950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47440/0 (socket says 172.21.15.92:47440) 2023-11-28T11:21:20.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.194+0000 7f228d9ff700 1 -- 172.21.15.92:0/3231431826 --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2288165740 con 0x7f22880ccf40 2023-11-28T11:21:20.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.194+0000 7f78629ff700 1 -- 172.21.15.92:0/2727807337 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f783c000fc0 con 0x7f785c06a270 2023-11-28T11:21:20.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.194+0000 7f9346ffd700 1 --2- 172.21.15.92:0/1935976780 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f93380683c0 0x7f933806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.194+0000 7f228d9ff700 1 -- 172.21.15.92:0/3231431826 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22880ae060 con 0x7f228806a270 2023-11-28T11:21:20.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.195+0000 7f9346ffd700 1 -- 172.21.15.92:0/1935976780 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f9348021070 con 0x7f935815ce70 2023-11-28T11:21:20.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.195+0000 7f7858ff9700 1 -- 172.21.15.92:0/2727807337 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f784c019030 con 0x7f785c06a270 2023-11-28T11:21:20.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.195+0000 7f2285ffb700 1 -- 172.21.15.92:0/3231431826 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3304444618 0 0) 0x7f2278001040 con 0x7f2288160e10 2023-11-28T11:21:20.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.195+0000 7f9346ffd700 1 --2- 172.21.15.92:0/1935976780 >> [v2:172.21.15.169:6816/116035,v1:172.21.15.169:6817/116035] conn(0x7f9338072090 0x7f93380744c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.195+0000 7f9346ffd700 1 -- 172.21.15.92:0/1935976780 --> [v2:172.21.15.169:6816/116035,v1:172.21.15.169:6817/116035] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9338074b90 con 0x7f9338072090 2023-11-28T11:21:20.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.195+0000 7f9346ffd700 1 -- 172.21.15.92:0/1935976780 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9348087fd0 con 0x7f935815ce70 2023-11-28T11:21:20.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.195+0000 7f2285ffb700 1 -- 172.21.15.92:0/3231431826 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22700035b0 con 0x7f2288160e10 2023-11-28T11:21:20.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.195+0000 7f935659c700 1 --2- 172.21.15.92:0/1935976780 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f93380683c0 0x7f933806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.196+0000 7f9355d9b700 1 --2- 172.21.15.92:0/1935976780 >> [v2:172.21.15.169:6816/116035,v1:172.21.15.169:6817/116035] conn(0x7f9338072090 0x7f93380744c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.196+0000 7f6c55d9b700 1 --2- 172.21.15.92:0/1083449725 >> [v2:172.21.15.169:6800/116033,v1:172.21.15.169:6801/116033] conn(0x7f6c38072010 0x7f6c38074440 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:20.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.196+0000 7f7858ff9700 1 --2- 172.21.15.92:0/2727807337 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7848068340 0x7f784806a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.196+0000 7f7858ff9700 1 -- 172.21.15.92:0/2727807337 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f784c085200 con 0x7f785c06a270 2023-11-28T11:21:20.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.196+0000 7f22867fc700 1 -- 172.21.15.92:0/3231431826 >> v1:172.21.15.169:6789/0 conn(0x7f22880ccf40 legacy=0x7f22880cd3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.196+0000 7f22867fc700 1 -- 172.21.15.92:0/3231431826 >> v1:172.21.15.92:6789/0 conn(0x7f2288160e10 legacy=0x7f22880ad950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.196+0000 7f22867fc700 1 -- 172.21.15.92:0/3231431826 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2288164bc0 con 0x7f228806a270 2023-11-28T11:21:20.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.196+0000 7f7858ff9700 1 --2- 172.21.15.92:0/2727807337 >> [v2:172.21.15.169:6808/116034,v1:172.21.15.169:6809/116034] conn(0x7f7848072010 0x7f7848074440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.196+0000 7f7858ff9700 1 -- 172.21.15.92:0/2727807337 --> [v2:172.21.15.169:6808/116034,v1:172.21.15.169:6809/116034] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7848074b10 con 0x7f7848072010 2023-11-28T11:21:20.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.196+0000 7f7858ff9700 1 -- 172.21.15.92:0/2727807337 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f784c0877e0 con 0x7f785c06a270 2023-11-28T11:21:20.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.197+0000 7f785b7fe700 1 --2- 172.21.15.92:0/2727807337 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7848068340 0x7f784806a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.196+0000 7f22867fc700 1 --2- 172.21.15.92:0/3231431826 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f228806a270 0x7f228806a6c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f227c009920 tx=0x7f227c009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=434e3dde7e480974 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:20.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.197+0000 7f9355d9b700 1 --2- 172.21.15.92:0/1935976780 >> [v2:172.21.15.169:6816/116035,v1:172.21.15.169:6817/116035] conn(0x7f9338072090 0x7f93380744c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:20.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.197+0000 7f6c46ffd700 1 -- 172.21.15.92:0/1083449725 <== osd.5 v2:172.21.15.169:6800/116033 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f6c38074b10 con 0x7f6c38072010 2023-11-28T11:21:20.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.197+0000 7f2285ffb700 1 -- 172.21.15.92:0/3231431826 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22780016b0 con 0x7f228806a270 2023-11-28T11:21:20.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.198+0000 7f935659c700 1 --2- 172.21.15.92:0/1935976780 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f93380683c0 0x7f933806a870 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f934c006d30 tx=0x7f934c0092c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:20.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.198+0000 7f9346ffd700 1 -- 172.21.15.92:0/1935976780 <== osd.4 v2:172.21.15.169:6816/116035 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f9338074b90 con 0x7f9338072090 2023-11-28T11:21:20.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.197+0000 7f785b7fe700 1 --2- 172.21.15.92:0/2727807337 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7848068340 0x7f784806a7f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f785c0afbc0 tx=0x7f7850008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:20.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.198+0000 7f7860f9c700 1 --2- 172.21.15.92:0/2727807337 >> [v2:172.21.15.169:6808/116034,v1:172.21.15.169:6809/116034] conn(0x7f7848072010 0x7f7848074440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.198+0000 7f7860f9c700 1 --2- 172.21.15.92:0/2727807337 >> [v2:172.21.15.169:6808/116034,v1:172.21.15.169:6809/116034] conn(0x7f7848072010 0x7f7848074440 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:20.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.199+0000 7f7858ff9700 1 -- 172.21.15.92:0/2727807337 <== osd.6 v2:172.21.15.169:6808/116034 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f7848074b10 con 0x7f7848072010 2023-11-28T11:21:20.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.197+0000 7f2285ffb700 1 -- 172.21.15.92:0/3231431826 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f227c004030 con 0x7f228806a270 2023-11-28T11:21:20.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.197+0000 7f2285ffb700 1 -- 172.21.15.92:0/3231431826 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227c0043c0 con 0x7f228806a270 2023-11-28T11:21:20.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.198+0000 7f228d9ff700 1 -- 172.21.15.92:0/3231431826 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f228806a270 msgr2=0x7f228806a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.198+0000 7f228d9ff700 1 --2- 172.21.15.92:0/3231431826 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f228806a270 0x7f228806a6c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f227c009920 tx=0x7f227c009c30 comp rx=0 tx=0).stop 2023-11-28T11:21:20.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.198+0000 7f228d9ff700 1 -- 172.21.15.92:0/3231431826 shutdown_connections 2023-11-28T11:21:20.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.198+0000 7f228d9ff700 1 --2- 172.21.15.92:0/3231431826 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f228806a270 0x7f228806a6c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.198+0000 7f228d9ff700 1 -- 172.21.15.92:0/3231431826 >> 172.21.15.92:0/3231431826 conn(0x7f22880cb490 msgr2=0x7f22880cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:20.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.201+0000 7f228d9ff700 1 -- 172.21.15.92:0/3231431826 shutdown_connections 2023-11-28T11:21:20.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.201+0000 7f228d9ff700 1 -- 172.21.15.92:0/3231431826 wait complete. 2023-11-28T11:21:20.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.203+0000 7f228d9ff700 1 Processor -- start 2023-11-28T11:21:20.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.204+0000 7f228d9ff700 1 -- start start 2023-11-28T11:21:20.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.205+0000 7f228d9ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f228806a270 0x7f2288162aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.205+0000 7f228d9ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22880ccf40 0x7f22881ea620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.205+0000 7f228d9ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2288160e10 0x7f228815f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.205+0000 7f228d9ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2288160360 con 0x7f2288160e10 2023-11-28T11:21:20.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.206+0000 7f228d9ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22881eab60 con 0x7f22880ccf40 2023-11-28T11:21:20.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.206+0000 7f228d9ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f228815f810 con 0x7f228806a270 2023-11-28T11:21:20.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.206+0000 7f22867fc700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22880ccf40 0x7f22881ea620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.206+0000 7f22867fc700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22880ccf40 0x7f22881ea620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.92:57094/0 (socket says 172.21.15.92:57094) 2023-11-28T11:21:20.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.206+0000 7f22867fc700 1 -- 172.21.15.92:0/29191310 learned_addr learned my addr 172.21.15.92:0/29191310 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:20.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.207+0000 7f2286ffd700 1 --2- 172.21.15.92:0/29191310 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f228806a270 0x7f2288162aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.207+0000 7f2286ffd700 1 -- 172.21.15.92:0/29191310 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22880ccf40 msgr2=0x7f22881ea620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.207+0000 7f2286ffd700 1 --2- 172.21.15.92:0/29191310 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22880ccf40 0x7f22881ea620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.207+0000 7f2286ffd700 1 -- 172.21.15.92:0/29191310 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2288160e10 msgr2=0x7f228815f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.207+0000 7f2286ffd700 1 --2- 172.21.15.92:0/29191310 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2288160e10 0x7f228815f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.208+0000 7f2286ffd700 1 -- 172.21.15.92:0/29191310 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f227c009550 con 0x7f228806a270 2023-11-28T11:21:20.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.208+0000 7f4f555ff700 1 -- 172.21.15.92:0/3358853134 --> [v2:172.21.15.92:6808/115823,v1:172.21.15.92:6809/115823] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4f34002d40 con 0x7f4f30071f40 2023-11-28T11:21:20.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.208+0000 7f2286ffd700 1 --2- 172.21.15.92:0/29191310 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f228806a270 0x7f2288162aa0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2278000c00 tx=0x7f227800cae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:20.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.210+0000 7f226ffff700 1 -- 172.21.15.92:0/29191310 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2278004c80 con 0x7f228806a270 2023-11-28T11:21:20.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.210+0000 7f4f47fff700 1 -- 172.21.15.92:0/3358853134 <== osd.0 v2:172.21.15.92:6808/115823 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4f34002d40 con 0x7f4f30071f40 2023-11-28T11:21:20.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.210+0000 7f226ffff700 1 -- 172.21.15.92:0/29191310 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2278004e00 con 0x7f228806a270 2023-11-28T11:21:20.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.210+0000 7f228d9ff700 1 -- 172.21.15.92:0/29191310 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2288165180 con 0x7f228806a270 2023-11-28T11:21:20.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.210+0000 7f226ffff700 1 -- 172.21.15.92:0/29191310 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2278007310 con 0x7f228806a270 2023-11-28T11:21:20.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.211+0000 7f228d9ff700 1 -- 172.21.15.92:0/29191310 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2288164ea0 con 0x7f228806a270 2023-11-28T11:21:20.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.212+0000 7f228d9ff700 1 -- 172.21.15.92:0/29191310 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2288165460 con 0x7f228806a270 2023-11-28T11:21:20.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.213+0000 7f4f555ff700 1 -- 172.21.15.92:0/3358853134 >> [v2:172.21.15.92:6808/115823,v1:172.21.15.92:6809/115823] conn(0x7f4f30071f40 msgr2=0x7f4f30074370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.213+0000 7f4f555ff700 1 --2- 172.21.15.92:0/3358853134 >> [v2:172.21.15.92:6808/115823,v1:172.21.15.92:6809/115823] conn(0x7f4f30071f40 0x7f4f30074370 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.214+0000 7f4f555ff700 1 -- 172.21.15.92:0/3358853134 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f4f300682f0 msgr2=0x7f4f3006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.214+0000 7f4f555ff700 1 --2- 172.21.15.92:0/3358853134 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f4f300682f0 0x7f4f3006a7a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4f40009fd0 tx=0x7f4f40009000 comp rx=0 tx=0).stop 2023-11-28T11:21:20.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.215+0000 7f226ffff700 1 -- 172.21.15.92:0/29191310 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f227801a020 con 0x7f228806a270 2023-11-28T11:21:20.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.216+0000 7f226ffff700 1 --2- 172.21.15.92:0/29191310 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f22740683c0 0x7f227406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.217+0000 7f4f555ff700 1 -- 172.21.15.92:0/3358853134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f50160d30 msgr2=0x7f4f50148470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.217+0000 7f226ffff700 1 -- 172.21.15.92:0/29191310 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2278086350 con 0x7f228806a270 2023-11-28T11:21:20.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.217+0000 7f4f555ff700 1 --2- 172.21.15.92:0/3358853134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f50160d30 0x7f4f50148470 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4f4800e560 tx=0x7f4f4800e920 comp rx=0 tx=0).stop 2023-11-28T11:21:20.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.218+0000 7f4f4f7fe700 1 -- 172.21.15.92:0/3358853134 reap_dead start 2023-11-28T11:21:20.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.217+0000 7f226ffff700 1 --2- 172.21.15.92:0/29191310 >> [v2:172.21.15.169:6821/116044,v1:172.21.15.169:6823/116044] conn(0x7f2274072090 0x7f22740744c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.218+0000 7f4f555ff700 1 -- 172.21.15.92:0/3358853134 shutdown_connections 2023-11-28T11:21:20.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.218+0000 7f4f555ff700 1 -- 172.21.15.92:0/3358853134 >> 172.21.15.92:0/3358853134 conn(0x7f4f500d3ca0 msgr2=0x7f4f5006d970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:20.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.217+0000 7f226ffff700 1 -- 172.21.15.92:0/29191310 --> [v2:172.21.15.169:6821/116044,v1:172.21.15.169:6823/116044] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2274074b90 con 0x7f2274072090 2023-11-28T11:21:20.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.217+0000 7f226ffff700 1 -- 172.21.15.92:0/29191310 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2278086700 con 0x7f228806a270 2023-11-28T11:21:20.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.218+0000 7f4f555ff700 1 -- 172.21.15.92:0/3358853134 shutdown_connections 2023-11-28T11:21:20.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.218+0000 7f4f555ff700 1 -- 172.21.15.92:0/3358853134 wait complete. 2023-11-28T11:21:20.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.218+0000 7f22877fe700 1 --2- 172.21.15.92:0/29191310 >> [v2:172.21.15.169:6821/116044,v1:172.21.15.169:6823/116044] conn(0x7f2274072090 0x7f22740744c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.218+0000 7f22867fc700 1 --2- 172.21.15.92:0/29191310 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f22740683c0 0x7f227406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.218+0000 7f22877fe700 1 --2- 172.21.15.92:0/29191310 >> [v2:172.21.15.169:6821/116044,v1:172.21.15.169:6823/116044] conn(0x7f2274072090 0x7f22740744c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:20.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.219+0000 7f22867fc700 1 --2- 172.21.15.92:0/29191310 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f22740683c0 0x7f227406a870 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2288163800 tx=0x7f227c008000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:20.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.220+0000 7f65d3fff700 1 -- 172.21.15.92:0/4037736487 --> [v2:172.21.15.92:6810/115824,v1:172.21.15.92:6812/115824] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f65b4002d40 con 0x7f65b0072010 2023-11-28T11:21:20.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.220+0000 7f226ffff700 1 -- 172.21.15.92:0/29191310 <== osd.7 v2:172.21.15.169:6821/116044 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f2274074b90 con 0x7f2274072090 2023-11-28T11:21:20.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.221+0000 7f65c27fc700 1 -- 172.21.15.92:0/4037736487 <== osd.1 v2:172.21.15.92:6810/115824 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f65b4002d40 con 0x7f65b0072010 2023-11-28T11:21:20.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.225+0000 7f65d3fff700 1 -- 172.21.15.92:0/4037736487 >> [v2:172.21.15.92:6810/115824,v1:172.21.15.92:6812/115824] conn(0x7f65b0072010 msgr2=0x7f65b0074440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.225+0000 7f65d3fff700 1 --2- 172.21.15.92:0/4037736487 >> [v2:172.21.15.92:6810/115824,v1:172.21.15.92:6812/115824] conn(0x7f65b0072010 0x7f65b0074440 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.225+0000 7f0ba83ff700 1 -- 172.21.15.92:0/1154011601 --> [v2:172.21.15.92:6800/115821,v1:172.21.15.92:6801/115821] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0ba006ac00 con 0x7f0b84072010 2023-11-28T11:21:20.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.225+0000 7f401b7ff700 1 -- 172.21.15.92:0/2008247054 --> [v2:172.21.15.92:6824/115825,v1:172.21.15.92:6825/115825] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f40141f3500 con 0x7f3ff8071fc0 2023-11-28T11:21:20.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.225+0000 7f0b96ffd700 1 -- 172.21.15.92:0/1154011601 <== osd.2 v2:172.21.15.92:6800/115821 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0ba006ac00 con 0x7f0b84072010 2023-11-28T11:21:20.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.225+0000 7f65d3fff700 1 -- 172.21.15.92:0/4037736487 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f65b0068340 msgr2=0x7f65b006a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.225+0000 7f65d3fff700 1 --2- 172.21.15.92:0/4037736487 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f65b0068340 0x7f65b006a7f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f65cc159000 tx=0x7f65c8009040 comp rx=0 tx=0).stop 2023-11-28T11:21:20.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.225+0000 7f65d3fff700 1 -- 172.21.15.92:0/4037736487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65cc1583c0 msgr2=0x7f65cc158860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.225+0000 7f65d3fff700 1 --2- 172.21.15.92:0/4037736487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f65cc1583c0 0x7f65cc158860 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f65bc00c390 tx=0x7f65bc00c6a0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.225+0000 7f65d259c700 1 -- 172.21.15.92:0/4037736487 reap_dead start 2023-11-28T11:21:20.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.226+0000 7f65d3fff700 1 -- 172.21.15.92:0/4037736487 shutdown_connections 2023-11-28T11:21:20.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.226+0000 7f65d3fff700 1 -- 172.21.15.92:0/4037736487 >> 172.21.15.92:0/4037736487 conn(0x7f65cc0cb490 msgr2=0x7f65cc0ab960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:20.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.226+0000 7f65d3fff700 1 -- 172.21.15.92:0/4037736487 shutdown_connections 2023-11-28T11:21:20.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.226+0000 7f65d3fff700 1 -- 172.21.15.92:0/4037736487 wait complete. 2023-11-28T11:21:20.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.227+0000 7f4009ffb700 1 -- 172.21.15.92:0/2008247054 <== osd.3 v2:172.21.15.92:6824/115825 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f40141f3500 con 0x7f3ff8071fc0 2023-11-28T11:21:20.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.227+0000 7f0ba83ff700 1 -- 172.21.15.92:0/1154011601 >> [v2:172.21.15.92:6800/115821,v1:172.21.15.92:6801/115821] conn(0x7f0b84072010 msgr2=0x7f0b84074440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.227+0000 7f0ba83ff700 1 --2- 172.21.15.92:0/1154011601 >> [v2:172.21.15.92:6800/115821,v1:172.21.15.92:6801/115821] conn(0x7f0b84072010 0x7f0b84074440 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.227+0000 7f0ba83ff700 1 -- 172.21.15.92:0/1154011601 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f0b84068340 msgr2=0x7f0b8406a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.227+0000 7f0ba83ff700 1 --2- 172.21.15.92:0/1154011601 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f0b84068340 0x7f0b8406a7f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c000f80 tx=0x7f0b9c009000 comp rx=0 tx=0).stop 2023-11-28T11:21:20.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.227+0000 7f0ba83ff700 1 -- 172.21.15.92:0/1154011601 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ba0142e80 msgr2=0x7f0ba015b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.227+0000 7f0ba83ff700 1 --2- 172.21.15.92:0/1154011601 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0ba0142e80 0x7f0ba015b8a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0b9800c780 tx=0x7f0b9800ca90 comp rx=0 tx=0).stop 2023-11-28T11:21:20.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.228+0000 7f0ba699c700 1 -- 172.21.15.92:0/1154011601 reap_dead start 2023-11-28T11:21:20.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.228+0000 7f0ba83ff700 1 -- 172.21.15.92:0/1154011601 shutdown_connections 2023-11-28T11:21:20.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.228+0000 7f0ba83ff700 1 -- 172.21.15.92:0/1154011601 >> 172.21.15.92:0/1154011601 conn(0x7f0ba00cb490 msgr2=0x7f0ba00ae370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:20.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.227+0000 7f401b7ff700 1 -- 172.21.15.92:0/2008247054 >> [v2:172.21.15.92:6824/115825,v1:172.21.15.92:6825/115825] conn(0x7f3ff8071fc0 msgr2=0x7f3ff80743f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.228+0000 7f401b7ff700 1 --2- 172.21.15.92:0/2008247054 >> [v2:172.21.15.92:6824/115825,v1:172.21.15.92:6825/115825] conn(0x7f3ff8071fc0 0x7f3ff80743f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.228+0000 7f401b7ff700 1 -- 172.21.15.92:0/2008247054 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3ff80682f0 msgr2=0x7f3ff806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.228+0000 7f401b7ff700 1 --2- 172.21.15.92:0/2008247054 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3ff80682f0 0x7f3ff806a7a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4014159b90 tx=0x7f4004006be0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.228+0000 7f401b7ff700 1 -- 172.21.15.92:0/2008247054 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40140b1c60 msgr2=0x7f40140cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.228+0000 7f401b7ff700 1 --2- 172.21.15.92:0/2008247054 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40140b1c60 0x7f40140cca70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f401000c820 tx=0x7f401000cbe0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.228+0000 7f4019d9c700 1 -- 172.21.15.92:0/2008247054 reap_dead start 2023-11-28T11:21:20.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.229+0000 7f401b7ff700 1 -- 172.21.15.92:0/2008247054 shutdown_connections 2023-11-28T11:21:20.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.229+0000 7f401b7ff700 1 -- 172.21.15.92:0/2008247054 >> 172.21.15.92:0/2008247054 conn(0x7f40140719a0 msgr2=0x7f40140b68c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:20.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.230+0000 7f401b7ff700 1 -- 172.21.15.92:0/2008247054 shutdown_connections 2023-11-28T11:21:20.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.231+0000 7f401b7ff700 1 -- 172.21.15.92:0/2008247054 wait complete. 2023-11-28T11:21:20.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.229+0000 7f6c5cdff700 1 -- 172.21.15.92:0/1083449725 --> [v2:172.21.15.169:6800/116033,v1:172.21.15.169:6801/116033] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6c580b2c10 con 0x7f6c38072010 2023-11-28T11:21:20.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.230+0000 7f6c46ffd700 1 -- 172.21.15.92:0/1083449725 <== osd.5 v2:172.21.15.169:6800/116033 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6c580b2c10 con 0x7f6c38072010 2023-11-28T11:21:20.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.231+0000 7f6c5cdff700 1 -- 172.21.15.92:0/1083449725 >> [v2:172.21.15.169:6800/116033,v1:172.21.15.169:6801/116033] conn(0x7f6c38072010 msgr2=0x7f6c38074440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.231+0000 7f6c5cdff700 1 --2- 172.21.15.92:0/1083449725 >> [v2:172.21.15.169:6800/116033,v1:172.21.15.169:6801/116033] conn(0x7f6c38072010 0x7f6c38074440 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.231+0000 7f6c5cdff700 1 -- 172.21.15.92:0/1083449725 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6c38068340 msgr2=0x7f6c3806a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.231+0000 7f6c5cdff700 1 --2- 172.21.15.92:0/1083449725 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6c38068340 0x7f6c3806a7f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6c580cca50 tx=0x7f6c50003a80 comp rx=0 tx=0).stop 2023-11-28T11:21:20.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.231+0000 7f6c5cdff700 1 -- 172.21.15.92:0/1083449725 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c58160f40 msgr2=0x7f6c580b2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.231+0000 7f6c5cdff700 1 --2- 172.21.15.92:0/1083449725 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6c58160f40 0x7f6c580b2da0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c001eb0 tx=0x7f6c4c00c730 comp rx=0 tx=0).stop 2023-11-28T11:21:20.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.231+0000 7f6c56d9d700 1 -- 172.21.15.92:0/1083449725 reap_dead start 2023-11-28T11:21:20.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.231+0000 7f6c5cdff700 1 -- 172.21.15.92:0/1083449725 shutdown_connections 2023-11-28T11:21:20.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.231+0000 7f6c5cdff700 1 -- 172.21.15.92:0/1083449725 >> 172.21.15.92:0/1083449725 conn(0x7f6c580cb490 msgr2=0x7f6c580afab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:20.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.231+0000 7f6c5cdff700 1 -- 172.21.15.92:0/1083449725 shutdown_connections 2023-11-28T11:21:20.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.231+0000 7f6c5cdff700 1 -- 172.21.15.92:0/1083449725 wait complete. 2023-11-28T11:21:20.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.232+0000 7f78629ff700 1 -- 172.21.15.92:0/2727807337 --> [v2:172.21.15.169:6808/116034,v1:172.21.15.169:6809/116034] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f783c002d40 con 0x7f7848072010 2023-11-28T11:21:20.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.232+0000 7f0ba83ff700 1 -- 172.21.15.92:0/1154011601 shutdown_connections 2023-11-28T11:21:20.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.232+0000 7f0ba83ff700 1 -- 172.21.15.92:0/1154011601 wait complete. 2023-11-28T11:21:20.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.233+0000 7f7858ff9700 1 -- 172.21.15.92:0/2727807337 <== osd.6 v2:172.21.15.169:6808/116034 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f783c002d40 con 0x7f7848072010 2023-11-28T11:21:20.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.234+0000 7f78629ff700 1 -- 172.21.15.92:0/2727807337 >> [v2:172.21.15.169:6808/116034,v1:172.21.15.169:6809/116034] conn(0x7f7848072010 msgr2=0x7f7848074440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.234+0000 7f78629ff700 1 --2- 172.21.15.92:0/2727807337 >> [v2:172.21.15.169:6808/116034,v1:172.21.15.169:6809/116034] conn(0x7f7848072010 0x7f7848074440 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.234+0000 7f78629ff700 1 -- 172.21.15.92:0/2727807337 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7848068340 msgr2=0x7f784806a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.234+0000 7f78629ff700 1 --2- 172.21.15.92:0/2727807337 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7848068340 0x7f784806a7f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f785c0afbc0 tx=0x7f7850008040 comp rx=0 tx=0).stop 2023-11-28T11:21:20.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.234+0000 7f78629ff700 1 -- 172.21.15.92:0/2727807337 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f785c06a270 msgr2=0x7f785c1f66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.234+0000 7f78629ff700 1 --2- 172.21.15.92:0/2727807337 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f785c06a270 0x7f785c1f66c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f784c00c520 tx=0x7f784c00c830 comp rx=0 tx=0).stop 2023-11-28T11:21:20.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.234+0000 7f7860f9c700 1 -- 172.21.15.92:0/2727807337 reap_dead start 2023-11-28T11:21:20.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.235+0000 7f78629ff700 1 -- 172.21.15.92:0/2727807337 shutdown_connections 2023-11-28T11:21:20.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.235+0000 7f78629ff700 1 -- 172.21.15.92:0/2727807337 >> 172.21.15.92:0/2727807337 conn(0x7f785c0cb490 msgr2=0x7f785c0cd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:20.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.234+0000 7f935c9ff700 1 -- 172.21.15.92:0/1935976780 --> [v2:172.21.15.169:6816/116035,v1:172.21.15.169:6817/116035] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f93580cd9c0 con 0x7f9338072090 2023-11-28T11:21:20.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.235+0000 7f78629ff700 1 -- 172.21.15.92:0/2727807337 shutdown_connections 2023-11-28T11:21:20.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.235+0000 7f78629ff700 1 -- 172.21.15.92:0/2727807337 wait complete. 2023-11-28T11:21:20.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.236+0000 7f9346ffd700 1 -- 172.21.15.92:0/1935976780 <== osd.4 v2:172.21.15.169:6816/116035 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f93580cd9c0 con 0x7f9338072090 2023-11-28T11:21:20.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.237+0000 7f935c9ff700 1 -- 172.21.15.92:0/1935976780 >> [v2:172.21.15.169:6816/116035,v1:172.21.15.169:6817/116035] conn(0x7f9338072090 msgr2=0x7f93380744c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.237+0000 7f935c9ff700 1 --2- 172.21.15.92:0/1935976780 >> [v2:172.21.15.169:6816/116035,v1:172.21.15.169:6817/116035] conn(0x7f9338072090 0x7f93380744c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.237+0000 7f935c9ff700 1 -- 172.21.15.92:0/1935976780 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f93380683c0 msgr2=0x7f933806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.237+0000 7f935c9ff700 1 --2- 172.21.15.92:0/1935976780 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f93380683c0 0x7f933806a870 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f934c006d30 tx=0x7f934c0092c0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.237+0000 7f935c9ff700 1 -- 172.21.15.92:0/1935976780 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f935815ce70 msgr2=0x7f93580af3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.237+0000 7f935c9ff700 1 --2- 172.21.15.92:0/1935976780 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f935815ce70 0x7f93580af3c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9348016000 tx=0x7f9348004750 comp rx=0 tx=0).stop 2023-11-28T11:21:20.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.238+0000 7f9356d9d700 1 -- 172.21.15.92:0/1935976780 reap_dead start 2023-11-28T11:21:20.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.238+0000 7f935c9ff700 1 -- 172.21.15.92:0/1935976780 shutdown_connections 2023-11-28T11:21:20.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.238+0000 7f935c9ff700 1 -- 172.21.15.92:0/1935976780 >> 172.21.15.92:0/1935976780 conn(0x7f93580cb490 msgr2=0x7f9358070910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:20.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.239+0000 7f935c9ff700 1 -- 172.21.15.92:0/1935976780 shutdown_connections 2023-11-28T11:21:20.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.239+0000 7f935c9ff700 1 -- 172.21.15.92:0/1935976780 wait complete. 2023-11-28T11:21:20.263 INFO:teuthology.orchestra.run.smithi092.stdout:55834574859 2023-11-28T11:21:20.264 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-28T11:21:20.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.257+0000 7f228d9ff700 1 -- 172.21.15.92:0/29191310 --> [v2:172.21.15.169:6821/116044,v1:172.21.15.169:6823/116044] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f228806ac00 con 0x7f2274072090 2023-11-28T11:21:20.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.259+0000 7f226ffff700 1 -- 172.21.15.92:0/29191310 <== osd.7 v2:172.21.15.169:6821/116044 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f228806ac00 con 0x7f2274072090 2023-11-28T11:21:20.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.262+0000 7f228d9ff700 1 -- 172.21.15.92:0/29191310 >> [v2:172.21.15.169:6821/116044,v1:172.21.15.169:6823/116044] conn(0x7f2274072090 msgr2=0x7f22740744c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.262+0000 7f228d9ff700 1 --2- 172.21.15.92:0/29191310 >> [v2:172.21.15.169:6821/116044,v1:172.21.15.169:6823/116044] conn(0x7f2274072090 0x7f22740744c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.262+0000 7f228d9ff700 1 -- 172.21.15.92:0/29191310 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f22740683c0 msgr2=0x7f227406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.262+0000 7f228d9ff700 1 --2- 172.21.15.92:0/29191310 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f22740683c0 0x7f227406a870 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2288163800 tx=0x7f227c008000 comp rx=0 tx=0).stop 2023-11-28T11:21:20.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.262+0000 7f228d9ff700 1 -- 172.21.15.92:0/29191310 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f228806a270 msgr2=0x7f2288162aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.262+0000 7f228d9ff700 1 --2- 172.21.15.92:0/29191310 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f228806a270 0x7f2288162aa0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2278000c00 tx=0x7f227800cae0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.272 INFO:teuthology.orchestra.run.smithi092.stdout:55834574859 2023-11-28T11:21:20.272 INFO:teuthology.orchestra.run.smithi092.stdout:55834574858 2023-11-28T11:21:20.273 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-28T11:21:20.273 INFO:teuthology.orchestra.run.smithi092.stdout:55834574858 2023-11-28T11:21:20.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.263+0000 7f22877fe700 1 -- 172.21.15.92:0/29191310 reap_dead start 2023-11-28T11:21:20.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.266+0000 7f228d9ff700 1 -- 172.21.15.92:0/29191310 shutdown_connections 2023-11-28T11:21:20.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.266+0000 7f228d9ff700 1 -- 172.21.15.92:0/29191310 >> 172.21.15.92:0/29191310 conn(0x7f22880cb490 msgr2=0x7f22880cd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:20.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.267+0000 7f228d9ff700 1 -- 172.21.15.92:0/29191310 shutdown_connections 2023-11-28T11:21:20.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.267+0000 7f228d9ff700 1 -- 172.21.15.92:0/29191310 wait complete. 2023-11-28T11:21:20.275 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-28T11:21:20.275 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-28T11:21:20.276 INFO:teuthology.orchestra.run.smithi092.stdout:55834574857 2023-11-28T11:21:20.276 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-28T11:21:20.277 INFO:teuthology.orchestra.run.smithi092.stdout:55834574859 2023-11-28T11:21:20.277 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-28T11:21:20.278 INFO:teuthology.orchestra.run.smithi092.stdout:55834574858 2023-11-28T11:21:20.279 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-28T11:21:20.290 INFO:teuthology.orchestra.run.smithi092.stdout:55834574859 2023-11-28T11:21:20.290 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-28T11:21:20.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.610+0000 7f016eb9d700 1 Processor -- start 2023-11-28T11:21:20.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.611+0000 7f016eb9d700 1 -- start start 2023-11-28T11:21:20.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.613+0000 7f016eb9d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01600e6e90 0x7f01600060f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.615+0000 7f016eb9d700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f01600e7260 0x7f0160034070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.615+0000 7f016db9b700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01600e6e90 0x7f01600060f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.615+0000 7f016db9b700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01600e6e90 0x7f01600060f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.92:34066/0 (socket says 172.21.15.92:34066) 2023-11-28T11:21:20.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.615+0000 7f016db9b700 1 -- 172.21.15.92:0/1829982501 learned_addr learned my addr 172.21.15.92:0/1829982501 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:20.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.616+0000 7f016d39a700 1 --2- 172.21.15.92:0/1829982501 >> v2:172.21.15.169:3300/0 conn(0x7f01600e7260 0x7f0160034070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.616+0000 7f016eb9d700 1 --2- 172.21.15.92:0/1829982501 >> v2:172.21.15.92:3300/0 conn(0x7f01600367a0 0x7f0160036be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.616+0000 7f016eb9d700 1 -- 172.21.15.92:0/1829982501 --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0160007020 con 0x7f01600367a0 2023-11-28T11:21:20.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.616+0000 7f016eb9d700 1 -- 172.21.15.92:0/1829982501 --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0160037120 con 0x7f01600e7260 2023-11-28T11:21:20.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.617+0000 7f016eb9d700 1 -- 172.21.15.92:0/1829982501 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01600345b0 con 0x7f01600e6e90 2023-11-28T11:21:20.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.618+0000 7f016db9b700 1 -- 172.21.15.92:0/1829982501 >> v2:172.21.15.169:3300/0 conn(0x7f01600e7260 msgr2=0x7f0160034070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:20.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.618+0000 7f016db9b700 1 --2- 172.21.15.92:0/1829982501 >> v2:172.21.15.169:3300/0 conn(0x7f01600e7260 0x7f0160034070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.618+0000 7f016db9b700 1 -- 172.21.15.92:0/1829982501 >> v2:172.21.15.92:3300/0 conn(0x7f01600367a0 msgr2=0x7f0160036be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:20.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.618+0000 7f016db9b700 1 --2- 172.21.15.92:0/1829982501 >> v2:172.21.15.92:3300/0 conn(0x7f01600367a0 0x7f0160036be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.618+0000 7f016db9b700 1 -- 172.21.15.92:0/1829982501 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0160006780 con 0x7f01600e6e90 2023-11-28T11:21:20.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.619+0000 7f016db9b700 1 --2- 172.21.15.92:0/1829982501 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01600e6e90 0x7f01600060f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f01680683c0 tx=0x7f016804edb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=515ee2b913738e8a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:20.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.620+0000 7f016cb99700 1 -- 172.21.15.92:0/1829982501 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0168067430 con 0x7f01600e6e90 2023-11-28T11:21:20.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.620+0000 7f016cb99700 1 -- 172.21.15.92:0/1829982501 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0168067590 con 0x7f01600e6e90 2023-11-28T11:21:20.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.621+0000 7f016eb9d700 1 -- 172.21.15.92:0/1829982501 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01600e6e90 msgr2=0x7f01600060f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.621+0000 7f016eb9d700 1 --2- 172.21.15.92:0/1829982501 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01600e6e90 0x7f01600060f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f01680683c0 tx=0x7f016804edb0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.622+0000 7f016eb9d700 1 -- 172.21.15.92:0/1829982501 shutdown_connections 2023-11-28T11:21:20.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.622+0000 7f016eb9d700 1 --2- 172.21.15.92:0/1829982501 >> v2:172.21.15.92:3300/0 conn(0x7f01600367a0 0x7f0160036be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.622+0000 7f016eb9d700 1 --2- 172.21.15.92:0/1829982501 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01600e6e90 0x7f01600060f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.622+0000 7f016eb9d700 1 --2- 172.21.15.92:0/1829982501 >> v2:172.21.15.169:3300/0 conn(0x7f01600e7260 0x7f0160034070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.622+0000 7f016eb9d700 1 -- 172.21.15.92:0/1829982501 >> 172.21.15.92:0/1829982501 conn(0x7f01600517a0 msgr2=0x7f0160051ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:20.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.622+0000 7f016eb9d700 1 -- 172.21.15.92:0/1829982501 shutdown_connections 2023-11-28T11:21:20.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.622+0000 7f016eb9d700 1 -- 172.21.15.92:0/1829982501 wait complete. 2023-11-28T11:21:20.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.625+0000 7f016eb9d700 1 Processor -- start 2023-11-28T11:21:20.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.625+0000 7f016eb9d700 1 -- start start 2023-11-28T11:21:20.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.628+0000 7f016eb9d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01600367a0 0x7f01600e76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.628+0000 7f016eb9d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01600e6e90 0x7f01600e5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.629+0000 7f016eb9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01600e7260 0x7f016004dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.629+0000 7f016eb9d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016017da40 con 0x7f01600e7260 2023-11-28T11:21:20.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.629+0000 7f016eb9d700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016004bdf0 con 0x7f01600e6e90 2023-11-28T11:21:20.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.629+0000 7f016eb9d700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f016004c0d0 con 0x7f01600367a0 2023-11-28T11:21:20.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.629+0000 7f016d39a700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01600e6e90 0x7f01600e5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.629+0000 7f016d39a700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01600e6e90 0x7f01600e5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.92:57098/0 (socket says 172.21.15.92:57098) 2023-11-28T11:21:20.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.629+0000 7f016d39a700 1 -- 172.21.15.92:0/2174459798 learned_addr learned my addr 172.21.15.92:0/2174459798 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:20.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.629+0000 7f016db9b700 1 --2- 172.21.15.92:0/2174459798 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01600367a0 0x7f01600e76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.629+0000 7f016e39c700 1 --2- 172.21.15.92:0/2174459798 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01600e7260 0x7f016004dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.631+0000 7f016e39c700 1 -- 172.21.15.92:0/2174459798 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01600367a0 msgr2=0x7f01600e76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.631+0000 7f016e39c700 1 --2- 172.21.15.92:0/2174459798 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01600367a0 0x7f01600e76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.631+0000 7f016e39c700 1 -- 172.21.15.92:0/2174459798 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01600e6e90 msgr2=0x7f01600e5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.631+0000 7f016e39c700 1 --2- 172.21.15.92:0/2174459798 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01600e6e90 0x7f01600e5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.631+0000 7f016e39c700 1 -- 172.21.15.92:0/2174459798 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0160006d40 con 0x7f01600e7260 2023-11-28T11:21:20.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.631+0000 7f016e39c700 1 --2- 172.21.15.92:0/2174459798 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01600e7260 0x7f016004dd20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f016400b410 tx=0x7f016400b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:20.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.632+0000 7f015e7fc700 1 -- 172.21.15.92:0/2174459798 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0164011830 con 0x7f01600e7260 2023-11-28T11:21:20.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.632+0000 7f015e7fc700 1 -- 172.21.15.92:0/2174459798 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0164011990 con 0x7f01600e7260 2023-11-28T11:21:20.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.633+0000 7f015e7fc700 1 -- 172.21.15.92:0/2174459798 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0164011d00 con 0x7f01600e7260 2023-11-28T11:21:20.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.634+0000 7f016eb9d700 1 -- 172.21.15.92:0/2174459798 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0160006a60 con 0x7f01600e7260 2023-11-28T11:21:20.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.634+0000 7f016eb9d700 1 -- 172.21.15.92:0/2174459798 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0160052bc0 con 0x7f01600e7260 2023-11-28T11:21:20.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.636+0000 7f016eb9d700 1 -- 172.21.15.92:0/2174459798 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01600054f0 con 0x7f01600e7260 2023-11-28T11:21:20.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.639+0000 7f015e7fc700 1 -- 172.21.15.92:0/2174459798 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0164021070 con 0x7f01600e7260 2023-11-28T11:21:20.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.641+0000 7f015e7fc700 1 --2- 172.21.15.92:0/2174459798 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f014c068340 0x7f014c06a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.641+0000 7f015e7fc700 1 -- 172.21.15.92:0/2174459798 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f01640862d0 con 0x7f01600e7260 2023-11-28T11:21:20.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.643+0000 7f016db9b700 1 --2- 172.21.15.92:0/2174459798 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f014c068340 0x7f014c06a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.644+0000 7f016db9b700 1 --2- 172.21.15.92:0/2174459798 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f014c068340 0x7f014c06a7f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f01600346f0 tx=0x7f0168094040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:20.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.647+0000 7f015e7fc700 1 -- 172.21.15.92:0/2174459798 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0164052810 con 0x7f01600e7260 2023-11-28T11:21:20.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.648+0000 7fa239dff700 1 Processor -- start 2023-11-28T11:21:20.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.649+0000 7fa239dff700 1 -- start start 2023-11-28T11:21:20.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.651+0000 7fa239dff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa234160f40 0x7fa2340ada50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.651+0000 7fa239dff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa23406a980 0x7fa2340cced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.652+0000 7fa239dff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2341657d0 con 0x7fa2340adf90 2023-11-28T11:21:20.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.652+0000 7fa22affd700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fa2340adf90 0x7fa23406a270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.92:60766/0 (socket says 172.21.15.92:60766) 2023-11-28T11:21:20.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.652+0000 7fa22affd700 1 -- 172.21.15.92:0/2149751845 learned_addr learned my addr 172.21.15.92:0/2149751845 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:20.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.652+0000 7fa233fff700 1 --2- 172.21.15.92:0/2149751845 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa23406a980 0x7fa2340cced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.653+0000 7fa239dff700 1 -- 172.21.15.92:0/2149751845 --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa234164c50 con 0x7fa234160f40 2023-11-28T11:21:20.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.653+0000 7fa239dff700 1 -- 172.21.15.92:0/2149751845 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa234164f30 con 0x7fa23406a980 2023-11-28T11:21:20.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.652+0000 7fa2337fe700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa234160f40 0x7fa2340ada50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.653+0000 7f271539d700 1 Processor -- start 2023-11-28T11:21:20.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.653+0000 7fa233fff700 1 -- 172.21.15.92:0/2149751845 >> v2:172.21.15.169:3300/0 conn(0x7fa234160f40 msgr2=0x7fa2340ada50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:20.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.653+0000 7fa233fff700 1 --2- 172.21.15.92:0/2149751845 >> v2:172.21.15.169:3300/0 conn(0x7fa234160f40 0x7fa2340ada50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.653+0000 7fa233fff700 1 -- 172.21.15.92:0/2149751845 >> v1:172.21.15.169:6789/0 conn(0x7fa2340adf90 legacy=0x7fa23406a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.653+0000 7fa233fff700 1 -- 172.21.15.92:0/2149751845 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa234165210 con 0x7fa23406a980 2023-11-28T11:21:20.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.654+0000 7f271539d700 1 -- start start 2023-11-28T11:21:20.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.654+0000 7fa233fff700 1 --2- 172.21.15.92:0/2149751845 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa23406a980 0x7fa2340cced0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa22c009840 tx=0x7fa22c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43d0d8ef79628fe4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:20.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.655+0000 7fa232ffd700 1 -- 172.21.15.92:0/2149751845 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22c011830 con 0x7fa23406a980 2023-11-28T11:21:20.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.655+0000 7fa232ffd700 1 -- 172.21.15.92:0/2149751845 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa22c011990 con 0x7fa23406a980 2023-11-28T11:21:20.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.655+0000 7fa232ffd700 1 -- 172.21.15.92:0/2149751845 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22c011d20 con 0x7fa23406a980 2023-11-28T11:21:20.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.656+0000 7fa239dff700 1 -- 172.21.15.92:0/2149751845 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa23406a980 msgr2=0x7fa2340cced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.656+0000 7fa239dff700 1 --2- 172.21.15.92:0/2149751845 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa23406a980 0x7fa2340cced0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa22c009840 tx=0x7fa22c009b50 comp rx=0 tx=0).stop 2023-11-28T11:21:20.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.656+0000 7fa239dff700 1 -- 172.21.15.92:0/2149751845 shutdown_connections 2023-11-28T11:21:20.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.656+0000 7fa239dff700 1 --2- 172.21.15.92:0/2149751845 >> v2:172.21.15.169:3300/0 conn(0x7fa234160f40 0x7fa2340ada50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.656+0000 7fa239dff700 1 --2- 172.21.15.92:0/2149751845 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa23406a980 0x7fa2340cced0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.657+0000 7fa239dff700 1 -- 172.21.15.92:0/2149751845 >> 172.21.15.92:0/2149751845 conn(0x7fa2340cb490 msgr2=0x7fa2340cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:20.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.657+0000 7fa239dff700 1 -- 172.21.15.92:0/2149751845 shutdown_connections 2023-11-28T11:21:20.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.656+0000 7f271539d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27100d2780 0x7f27100d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.657+0000 7f271539d700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27100b2550 con 0x7f27100d2230 2023-11-28T11:21:20.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.657+0000 7f271539d700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27100d3750 con 0x7f27100d2c70 2023-11-28T11:21:20.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.657+0000 7f271539d700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27100cd330 con 0x7f27100d2780 2023-11-28T11:21:20.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.657+0000 7fa239dff700 1 -- 172.21.15.92:0/2149751845 wait complete. 2023-11-28T11:21:20.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.657+0000 7f270f7fe700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27100d2780 0x7f27100d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.657+0000 7f270f7fe700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27100d2780 0x7f27100d08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.92:34080/0 (socket says 172.21.15.92:34080) 2023-11-28T11:21:20.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.657+0000 7f270f7fe700 1 -- 172.21.15.92:0/3256134700 learned_addr learned my addr 172.21.15.92:0/3256134700 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:20.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.658+0000 7f270effd700 1 -- 172.21.15.92:0/3256134700 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 217925307 0 0) 0x7f27100d3750 con 0x7f27100d2c70 2023-11-28T11:21:20.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.658+0000 7f270effd700 1 -- 172.21.15.92:0/3256134700 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26f80035b0 con 0x7f27100d2c70 2023-11-28T11:21:20.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.658+0000 7f270f7fe700 1 -- 172.21.15.92:0/3256134700 >> v1:172.21.15.169:6789/0 conn(0x7f27100d2c70 legacy=0x7f27100d3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.658+0000 7f270f7fe700 1 -- 172.21.15.92:0/3256134700 >> v1:172.21.15.92:6789/0 conn(0x7f27100d2230 legacy=0x7f27100d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:20.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.658+0000 7f270f7fe700 1 -- 172.21.15.92:0/3256134700 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27100d39f0 con 0x7f27100d2780 2023-11-28T11:21:20.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.661+0000 7fa239dff700 1 Processor -- start 2023-11-28T11:21:20.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.660+0000 7f270f7fe700 1 --2- 172.21.15.92:0/3256134700 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27100d2780 0x7f27100d08b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2704009920 tx=0x7f2704009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=819659da6572d77b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:20.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.660+0000 7f270effd700 1 -- 172.21.15.92:0/3256134700 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2704004230 con 0x7f27100d2780 2023-11-28T11:21:20.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.660+0000 7f270effd700 1 -- 172.21.15.92:0/3256134700 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2704004390 con 0x7f27100d2780 2023-11-28T11:21:20.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.660+0000 7f270effd700 1 -- 172.21.15.92:0/3256134700 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2704004230 con 0x7f27100d2780 2023-11-28T11:21:20.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.661+0000 7fa239dff700 1 -- start start 2023-11-28T11:21:20.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.662+0000 7f271539d700 1 -- 172.21.15.92:0/3256134700 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27100d2780 msgr2=0x7f27100d08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.662+0000 7f271539d700 1 --2- 172.21.15.92:0/3256134700 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27100d2780 0x7f27100d08b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2704009920 tx=0x7f2704009c30 comp rx=0 tx=0).stop 2023-11-28T11:21:20.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.662+0000 7f271539d700 1 -- 172.21.15.92:0/3256134700 shutdown_connections 2023-11-28T11:21:20.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.662+0000 7f271539d700 1 --2- 172.21.15.92:0/3256134700 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27100d2780 0x7f27100d08b0 secure :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2704009920 tx=0x7f2704009c30 comp rx=0 tx=0).stop 2023-11-28T11:21:20.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.662+0000 7f271539d700 1 -- 172.21.15.92:0/3256134700 >> 172.21.15.92:0/3256134700 conn(0x7f27100719a0 msgr2=0x7f2710071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:20.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.663+0000 7fa239dff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa23406a980 0x7fa23415d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.663+0000 7fa239dff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2340adf90 0x7fa2340c6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.664+0000 7fa239dff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa234160f40 0x7fa23415a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.664+0000 7fafda39d700 1 Processor -- start 2023-11-28T11:21:20.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.664+0000 7fa239dff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2341fa7e0 con 0x7fa2340adf90 2023-11-28T11:21:20.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.664+0000 7fa239dff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2341f9e40 con 0x7fa23406a980 2023-11-28T11:21:20.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.664+0000 7fa239dff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2341fa120 con 0x7fa234160f40 2023-11-28T11:21:20.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.664+0000 7fa2337fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa23406a980 0x7fa23415d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.664+0000 7f7b389fe700 1 Processor -- start 2023-11-28T11:21:20.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.665+0000 7f7b389fe700 1 -- start start 2023-11-28T11:21:20.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.665+0000 7fa233fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa234160f40 0x7fa23415a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.665+0000 7fa22affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2340adf90 0x7fa2340c6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.665+0000 7fa22affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2340adf90 0x7fa2340c6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48870/0 (socket says 172.21.15.92:48870) 2023-11-28T11:21:20.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.665+0000 7fa22affd700 1 -- 172.21.15.92:0/1109882555 learned_addr learned my addr 172.21.15.92:0/1109882555 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:20.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.664+0000 7fafda39d700 1 -- start start 2023-11-28T11:21:20.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.663+0000 7f271539d700 1 -- 172.21.15.92:0/3256134700 shutdown_connections 2023-11-28T11:21:20.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.663+0000 7f271539d700 1 -- 172.21.15.92:0/3256134700 wait complete. 2023-11-28T11:21:20.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.666+0000 7f271539d700 1 Processor -- start 2023-11-28T11:21:20.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.666+0000 7f271539d700 1 -- start start 2023-11-28T11:21:20.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.666+0000 7fa22affd700 1 -- 172.21.15.92:0/1109882555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa234160f40 msgr2=0x7fa23415a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.666+0000 7fa22affd700 1 --2- 172.21.15.92:0/1109882555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa234160f40 0x7fa23415a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.666+0000 7fa22affd700 1 -- 172.21.15.92:0/1109882555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa23406a980 msgr2=0x7fa23415d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.666+0000 7fa22affd700 1 --2- 172.21.15.92:0/1109882555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa23406a980 0x7fa23415d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.666+0000 7fa22affd700 1 -- 172.21.15.92:0/1109882555 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa22c0094a0 con 0x7fa2340adf90 2023-11-28T11:21:20.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.667+0000 7fafd939b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fafcc0e6d00 0x7fafcc04f1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47458/0 (socket says 172.21.15.92:47458) 2023-11-28T11:21:20.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.667+0000 7fa22affd700 1 --2- 172.21.15.92:0/1109882555 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2340adf90 0x7fa2340c6340 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa21c00c780 tx=0x7fa21c00ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:20.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.667+0000 7f7b389fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7b340d2c70 0x7f7b340d3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.667+0000 7f7b389fe700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b340d3580 0x7f7b34160080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.667+0000 7f7b389fe700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b340d18c0 con 0x7f7b340d2780 2023-11-28T11:21:20.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.667+0000 7f7b389fe700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b340b2550 con 0x7f7b340d2c70 2023-11-28T11:21:20.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.667+0000 7f7b389fe700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b340cd330 con 0x7f7b340d3580 2023-11-28T11:21:20.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.667+0000 7fa230ff9700 1 -- 172.21.15.92:0/1109882555 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21c005b40 con 0x7fa2340adf90 2023-11-28T11:21:20.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.667+0000 7fafd939b700 1 -- 172.21.15.92:0/2198416530 learned_addr learned my addr 172.21.15.92:0/2198416530 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:20.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.667+0000 7fa239dff700 1 -- 172.21.15.92:0/1109882555 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa234164900 con 0x7fa2340adf90 2023-11-28T11:21:20.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.668+0000 7fa239dff700 1 -- 172.21.15.92:0/1109882555 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2341654f0 con 0x7fa2340adf90 2023-11-28T11:21:20.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.669+0000 7fa230ff9700 1 -- 172.21.15.92:0/1109882555 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa21c005ca0 con 0x7fa2340adf90 2023-11-28T11:21:20.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.669+0000 7fa230ff9700 1 -- 172.21.15.92:0/1109882555 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21c005b40 con 0x7fa2340adf90 2023-11-28T11:21:20.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.668+0000 7f271539d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27100d2230 0x7f27100cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.669+0000 7f271539d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27100d2c70 0x7f271015e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.669+0000 7f271539d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27101fe8b0 0x7f27100b2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.669+0000 7f270ffff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27100d2230 0x7f27100cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.669+0000 7f270f7fe700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27100d2c70 0x7f271015e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.669+0000 7f270f7fe700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27100d2c70 0x7f271015e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.92:34104/0 (socket says 172.21.15.92:34104) 2023-11-28T11:21:20.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.669+0000 7f270f7fe700 1 -- 172.21.15.92:0/1652260701 learned_addr learned my addr 172.21.15.92:0/1652260701 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:20.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.669+0000 7f271539d700 1 -- 172.21.15.92:0/1652260701 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f271015f580 con 0x7f27101fe8b0 2023-11-28T11:21:20.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.670+0000 7f271539d700 1 -- 172.21.15.92:0/1652260701 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27100cc2c0 con 0x7f27100d2230 2023-11-28T11:21:20.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.670+0000 7f271539d700 1 -- 172.21.15.92:0/1652260701 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27100cb6b0 con 0x7f27100d2c70 2023-11-28T11:21:20.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.670+0000 7f2714b9c700 1 --2- 172.21.15.92:0/1652260701 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27101fe8b0 0x7f27100b2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.670+0000 7fe9acbff700 1 Processor -- start 2023-11-28T11:21:20.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.668+0000 7fafda39d700 1 --2- 172.21.15.92:0/2198416530 >> v2:172.21.15.169:3300/0 conn(0x7fafcc051150 0x7fafcc036610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.668+0000 7fafda39d700 1 --2- 172.21.15.92:0/2198416530 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fafcc036b50 0x7fafcc052c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.668+0000 7fafd9b9c700 1 --2- 172.21.15.92:0/2198416530 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fafcc036b50 0x7fafcc052c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.668+0000 7fafd8b9a700 1 --2- 172.21.15.92:0/2198416530 >> v2:172.21.15.169:3300/0 conn(0x7fafcc051150 0x7fafcc036610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.669+0000 7fafda39d700 1 -- 172.21.15.92:0/2198416530 --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafcc036f90 con 0x7fafcc0e6d00 2023-11-28T11:21:20.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.669+0000 7fafda39d700 1 -- 172.21.15.92:0/2198416530 --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafcc048550 con 0x7fafcc051150 2023-11-28T11:21:20.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.669+0000 7fafda39d700 1 -- 172.21.15.92:0/2198416530 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafcc04f8f0 con 0x7fafcc036b50 2023-11-28T11:21:20.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.669+0000 7fafd9b9c700 1 -- 172.21.15.92:0/2198416530 >> v2:172.21.15.169:3300/0 conn(0x7fafcc051150 msgr2=0x7fafcc036610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:20.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.669+0000 7fafd9b9c700 1 --2- 172.21.15.92:0/2198416530 >> v2:172.21.15.169:3300/0 conn(0x7fafcc051150 0x7fafcc036610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.669+0000 7fafd9b9c700 1 -- 172.21.15.92:0/2198416530 >> v1:172.21.15.92:6789/0 conn(0x7fafcc0e6d00 legacy=0x7fafcc04f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:20.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.670+0000 7fafd9b9c700 1 -- 172.21.15.92:0/2198416530 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafcc033920 con 0x7fafcc036b50 2023-11-28T11:21:20.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.670+0000 7fafd9b9c700 1 --2- 172.21.15.92:0/2198416530 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fafcc036b50 0x7fafcc052c80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fafd4000eb0 tx=0x7fafd404ee10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe02372a0ee28879 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:20.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.670+0000 7f270ffff700 1 -- 172.21.15.92:0/1652260701 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27100d2c70 msgr2=0x7f271015e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.668+0000 7f7b3359e700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f7b340d2780 0x7f7b341605c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47464/0 (socket says 172.21.15.92:47464) 2023-11-28T11:21:20.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.670+0000 7f7b3359e700 1 -- 172.21.15.92:0/891577333 learned_addr learned my addr 172.21.15.92:0/891577333 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:20.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.668+0000 7f7b3259c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b340d3580 0x7f7b34160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.669+0000 7f7b32d9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7b340d2c70 0x7f7b340d3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.671+0000 7fafcbfff700 1 -- 172.21.15.92:0/2198416530 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd4067580 con 0x7fafcc036b50 2023-11-28T11:21:20.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.670+0000 7f270ffff700 1 --2- 172.21.15.92:0/1652260701 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27100d2c70 0x7f271015e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.670+0000 7f270ffff700 1 -- 172.21.15.92:0/1652260701 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27101fe8b0 msgr2=0x7f27100b2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.670+0000 7f270ffff700 1 --2- 172.21.15.92:0/1652260701 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27101fe8b0 0x7f27100b2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.670+0000 7f270ffff700 1 -- 172.21.15.92:0/1652260701 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2704009550 con 0x7f27100d2230 2023-11-28T11:21:20.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.672+0000 7fafcbfff700 1 -- 172.21.15.92:0/2198416530 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafd40676e0 con 0x7fafcc036b50 2023-11-28T11:21:20.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.670+0000 7fa239dff700 1 -- 172.21.15.92:0/1109882555 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2340cd410 con 0x7fa2340adf90 2023-11-28T11:21:20.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.672+0000 7fa230ff9700 1 -- 172.21.15.92:0/1109882555 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa21c019030 con 0x7fa2340adf90 2023-11-28T11:21:20.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.672+0000 7fafcbfff700 1 -- 172.21.15.92:0/2198416530 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd4072ef0 con 0x7fafcc036b50 2023-11-28T11:21:20.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.673+0000 7fafda39d700 1 -- 172.21.15.92:0/2198416530 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fafcc036b50 msgr2=0x7fafcc052c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.673+0000 7fafda39d700 1 --2- 172.21.15.92:0/2198416530 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fafcc036b50 0x7fafcc052c80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fafd4000eb0 tx=0x7fafd404ee10 comp rx=0 tx=0).stop 2023-11-28T11:21:20.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.672+0000 7f2714b9c700 1 --2- 172.21.15.92:0/1652260701 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27101fe8b0 0x7f27100b2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:21:20.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.673+0000 7f270ffff700 1 --2- 172.21.15.92:0/1652260701 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27100d2230 0x7f27100cb0d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f270000c3f0 tx=0x7f270000c700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:20.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.671+0000 7fe9acbff700 1 -- start start 2023-11-28T11:21:20.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.673+0000 7fe9acbff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9a80cfd40 0x7fe9a81609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.673+0000 7fe9acbff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe9a806c290 0x7fe9a80d10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.673+0000 7fe9acbff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9a80d0bc0 con 0x7fe9a815c400 2023-11-28T11:21:20.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.673+0000 7fe9acbff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9a806c830 con 0x7fe9a806c290 2023-11-28T11:21:20.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.673+0000 7fe9acbff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a80cd410 con 0x7fe9a80cfd40 2023-11-28T11:21:20.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.673+0000 7fe9a659c700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fe9a815c400 0x7fe9a80cf630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.92:60790/0 (socket says 172.21.15.92:60790) 2023-11-28T11:21:20.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.673+0000 7fe9a659c700 1 -- 172.21.15.92:0/4050641657 learned_addr learned my addr 172.21.15.92:0/4050641657 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:20.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.673+0000 7fe9a5d9b700 1 --2- 172.21.15.92:0/4050641657 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9a80cfd40 0x7fe9a81609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.674+0000 7f270cff9700 1 -- 172.21.15.92:0/1652260701 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27000052d0 con 0x7f27100d2230 2023-11-28T11:21:20.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.673+0000 7f7b3259c700 1 -- 172.21.15.92:0/891577333 >> v2:172.21.15.92:3300/0 conn(0x7f7b340d2c70 msgr2=0x7f7b340d3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:20.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.673+0000 7f7b3259c700 1 --2- 172.21.15.92:0/891577333 >> v2:172.21.15.92:3300/0 conn(0x7f7b340d2c70 0x7f7b340d3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.673+0000 7f7b3259c700 1 -- 172.21.15.92:0/891577333 >> v1:172.21.15.92:6789/0 conn(0x7f7b340d2780 legacy=0x7f7b341605c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:20.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.673+0000 7f7b3259c700 1 -- 172.21.15.92:0/891577333 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b340d39f0 con 0x7f7b340d3580 2023-11-28T11:21:20.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.674+0000 7f270cff9700 1 -- 172.21.15.92:0/1652260701 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2700005450 con 0x7f27100d2230 2023-11-28T11:21:20.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.674+0000 7f270cff9700 1 -- 172.21.15.92:0/1652260701 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27000038c0 con 0x7f27100d2230 2023-11-28T11:21:20.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.674+0000 7f271539d700 1 -- 172.21.15.92:0/1652260701 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27100cfa40 con 0x7f27100d2230 2023-11-28T11:21:20.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.674+0000 7fe9a559a700 1 -- 172.21.15.92:0/4050641657 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1282541825 0 0) 0x7fe9a80d0bc0 con 0x7fe9a815c400 2023-11-28T11:21:20.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.674+0000 7fe9a559a700 1 -- 172.21.15.92:0/4050641657 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9900035b0 con 0x7fe9a815c400 2023-11-28T11:21:20.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.674+0000 7f271539d700 1 -- 172.21.15.92:0/1652260701 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27100cd5f0 con 0x7f27100d2230 2023-11-28T11:21:20.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.674+0000 7fe9a6d9d700 1 --2- 172.21.15.92:0/4050641657 >> v2:172.21.15.169:3300/0 conn(0x7fe9a806c290 0x7fe9a80d10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.675+0000 7f7b3259c700 1 --2- 172.21.15.92:0/891577333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b340d3580 0x7f7b34160080 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7b20009840 tx=0x7f7b20009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18ec11e50165b669 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:20.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.675+0000 7f7b31d9b700 1 -- 172.21.15.92:0/891577333 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b20011830 con 0x7f7b340d3580 2023-11-28T11:21:20.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.675+0000 7fe9a6d9d700 1 -- 172.21.15.92:0/4050641657 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9a80cfd40 msgr2=0x7fe9a81609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:20.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.675+0000 7fe9a6d9d700 1 --2- 172.21.15.92:0/4050641657 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9a80cfd40 0x7fe9a81609c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.675+0000 7fe9a6d9d700 1 -- 172.21.15.92:0/4050641657 >> v1:172.21.15.169:6789/0 conn(0x7fe9a815c400 legacy=0x7fe9a80cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.675+0000 7fe9a6d9d700 1 -- 172.21.15.92:0/4050641657 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a80cd6f0 con 0x7fe9a806c290 2023-11-28T11:21:20.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.675+0000 7fa230ff9700 1 --2- 172.21.15.92:0/1109882555 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa214068350 0x7fa21406a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.675+0000 7fafda39d700 1 -- 172.21.15.92:0/2198416530 shutdown_connections 2023-11-28T11:21:20.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.675+0000 7f7b31d9b700 1 -- 172.21.15.92:0/891577333 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b20011990 con 0x7f7b340d3580 2023-11-28T11:21:20.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.675+0000 7f7b31d9b700 1 -- 172.21.15.92:0/891577333 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b20011d20 con 0x7f7b340d3580 2023-11-28T11:21:20.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.675+0000 7fafda39d700 1 --2- 172.21.15.92:0/2198416530 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fafcc036b50 0x7fafcc052c80 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.676+0000 7fafda39d700 1 --2- 172.21.15.92:0/2198416530 >> v2:172.21.15.169:3300/0 conn(0x7fafcc051150 0x7fafcc036610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.676+0000 7fafda39d700 1 -- 172.21.15.92:0/2198416530 >> 172.21.15.92:0/2198416530 conn(0x7fafcc051610 msgr2=0x7fafcc0519e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:20.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.676+0000 7fa230ff9700 1 -- 172.21.15.92:0/1109882555 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa21c084dc0 con 0x7fa2340adf90 2023-11-28T11:21:20.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.676+0000 7fe9a6d9d700 1 --2- 172.21.15.92:0/4050641657 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9a806c290 0x7fe9a80d10a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe9a00025c0 tx=0x7fe9a000c150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c81ba737fea54f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:20.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.676+0000 7fa2337fe700 1 --2- 172.21.15.92:0/1109882555 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa214068350 0x7fa21406a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.676+0000 7fd17fdff700 1 Processor -- start 2023-11-28T11:21:20.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.677+0000 7fe9a559a700 1 -- 172.21.15.92:0/4050641657 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe998001670 con 0x7fe9a806c290 2023-11-28T11:21:20.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.677+0000 7fd17fdff700 1 -- start start 2023-11-28T11:21:20.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.677+0000 7fe9a559a700 1 -- 172.21.15.92:0/4050641657 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a000c960 con 0x7fe9a806c290 2023-11-28T11:21:20.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.677+0000 7fa2337fe700 1 --2- 172.21.15.92:0/1109882555 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa214068350 0x7fa21406a800 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa224011340 tx=0x7fa22400b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:20.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.678+0000 7fafda39d700 1 -- 172.21.15.92:0/2198416530 shutdown_connections 2023-11-28T11:21:20.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.676+0000 7f7b389fe700 1 -- 172.21.15.92:0/891577333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b340d3580 msgr2=0x7f7b34160080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.676+0000 7f7b389fe700 1 --2- 172.21.15.92:0/891577333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b340d3580 0x7f7b34160080 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7b20009840 tx=0x7f7b20009b50 comp rx=0 tx=0).stop 2023-11-28T11:21:20.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.678+0000 7f7b389fe700 1 -- 172.21.15.92:0/891577333 shutdown_connections 2023-11-28T11:21:20.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.678+0000 7f7b389fe700 1 --2- 172.21.15.92:0/891577333 >> v2:172.21.15.92:3300/0 conn(0x7f7b340d2c70 0x7f7b340d3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.678+0000 7f7b389fe700 1 --2- 172.21.15.92:0/891577333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b340d3580 0x7f7b34160080 secure :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7b20009840 tx=0x7f7b20009b50 comp rx=0 tx=0).stop 2023-11-28T11:21:20.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.678+0000 7f7b389fe700 1 -- 172.21.15.92:0/891577333 >> 172.21.15.92:0/891577333 conn(0x7f7b340719a0 msgr2=0x7f7b34071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:20.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.678+0000 7fafda39d700 1 -- 172.21.15.92:0/2198416530 wait complete. 2023-11-28T11:21:20.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.678+0000 7f271539d700 1 -- 172.21.15.92:0/1652260701 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27100cff60 con 0x7f27100d2230 2023-11-28T11:21:20.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.678+0000 7f7b389fe700 1 -- 172.21.15.92:0/891577333 shutdown_connections 2023-11-28T11:21:20.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.679+0000 7f7b389fe700 1 -- 172.21.15.92:0/891577333 wait complete. 2023-11-28T11:21:20.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.678+0000 7f270cff9700 1 -- 172.21.15.92:0/1652260701 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2700019030 con 0x7f27100d2230 2023-11-28T11:21:20.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.677+0000 7fe9a559a700 1 -- 172.21.15.92:0/4050641657 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a000cd10 con 0x7fe9a806c290 2023-11-28T11:21:20.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.678+0000 7fe9acbff700 1 -- 172.21.15.92:0/4050641657 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9a806c290 msgr2=0x7fe9a80d10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.678+0000 7fe9acbff700 1 --2- 172.21.15.92:0/4050641657 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9a806c290 0x7fe9a80d10a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe9a00025c0 tx=0x7fe9a000c150 comp rx=0 tx=0).stop 2023-11-28T11:21:20.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.679+0000 7fd17fdff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd1780c9ed0 0x7fd1780cdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.679+0000 7fd17fdff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1780ccb60 con 0x7fd1780cbf70 2023-11-28T11:21:20.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.679+0000 7fd17fdff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1780cce40 con 0x7fd1780ce420 2023-11-28T11:21:20.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.679+0000 7fd17fdff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1780cd120 con 0x7fd1780c9ed0 2023-11-28T11:21:20.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.680+0000 7fd17db9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd1780cbf70 0x7fd17815ce70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47474/0 (socket says 172.21.15.92:47474) 2023-11-28T11:21:20.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.680+0000 7fd17db9b700 1 -- 172.21.15.92:0/1576264106 learned_addr learned my addr 172.21.15.92:0/1576264106 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:20.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.681+0000 7fd17d39a700 1 --2- 172.21.15.92:0/1576264106 >> v2:172.21.15.92:3300/0 conn(0x7fd1780c9ed0 0x7fd1780cdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.681+0000 7fafda39d700 1 Processor -- start 2023-11-28T11:21:20.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.681+0000 7f7b389fe700 1 Processor -- start 2023-11-28T11:21:20.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.681+0000 7f270cff9700 1 --2- 172.21.15.92:0/1652260701 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f26fc0682f0 0x7f26fc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.682+0000 7fa230ff9700 1 -- 172.21.15.92:0/1109882555 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa21c04b5e0 con 0x7fa2340adf90 2023-11-28T11:21:20.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.682+0000 7f7b389fe700 1 -- start start 2023-11-28T11:21:20.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.682+0000 7fafda39d700 1 -- start start 2023-11-28T11:21:20.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.682+0000 7fd17d39a700 1 -- 172.21.15.92:0/1576264106 >> v1:172.21.15.169:6789/0 conn(0x7fd1780ce420 legacy=0x7fd17806a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.682+0000 7fd17d39a700 1 -- 172.21.15.92:0/1576264106 >> v1:172.21.15.92:6789/0 conn(0x7fd1780cbf70 legacy=0x7fd17815ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.682+0000 7fd17d39a700 1 -- 172.21.15.92:0/1576264106 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1780cd400 con 0x7fd1780c9ed0 2023-11-28T11:21:20.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.682+0000 7fe9acbff700 1 -- 172.21.15.92:0/4050641657 shutdown_connections 2023-11-28T11:21:20.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.683+0000 7fd17d39a700 1 --2- 172.21.15.92:0/1576264106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1780c9ed0 0x7fd1780cdee0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd168009840 tx=0x7fd168009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=395cc2b68c746a64 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:20.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.682+0000 7fe9acbff700 1 --2- 172.21.15.92:0/4050641657 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9a806c290 0x7fe9a80d10a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.682+0000 7fe9acbff700 1 --2- 172.21.15.92:0/4050641657 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9a80cfd40 0x7fe9a81609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.683+0000 7fc7d5fff700 1 Processor -- start 2023-11-28T11:21:20.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.683+0000 7fd17cb99700 1 -- 172.21.15.92:0/1576264106 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd168004230 con 0x7fd1780c9ed0 2023-11-28T11:21:20.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.683+0000 7fd17cb99700 1 -- 172.21.15.92:0/1576264106 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd168004390 con 0x7fd1780c9ed0 2023-11-28T11:21:20.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.683+0000 7fd17cb99700 1 -- 172.21.15.92:0/1576264106 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1680036a0 con 0x7fd1780c9ed0 2023-11-28T11:21:20.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.682+0000 7f270cff9700 1 -- 172.21.15.92:0/1652260701 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f27000852c0 con 0x7f27100d2230 2023-11-28T11:21:20.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.683+0000 7f270f7fe700 1 --2- 172.21.15.92:0/1652260701 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f26fc0682f0 0x7f26fc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.682+0000 7fe9acbff700 1 -- 172.21.15.92:0/4050641657 >> 172.21.15.92:0/4050641657 conn(0x7fe9a80719a0 msgr2=0x7fe9a8071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:20.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.684+0000 7fafda39d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fafcc036b50 0x7fafcc0e90d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.683+0000 7fe9acbff700 1 -- 172.21.15.92:0/4050641657 shutdown_connections 2023-11-28T11:21:20.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.683+0000 7fe9acbff700 1 -- 172.21.15.92:0/4050641657 wait complete. 2023-11-28T11:21:20.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.684+0000 7fc7d5fff700 1 -- start start 2023-11-28T11:21:20.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.684+0000 7f7b389fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b340d2780 0x7f7b340c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.684+0000 7f7b389fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b340d2c70 0x7f7b3415a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.684+0000 7fafd939b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fafcc036b50 0x7fafcc0e90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.685+0000 7fd17fdff700 1 -- 172.21.15.92:0/1576264106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1780c9ed0 msgr2=0x7fd1780cdee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.685+0000 7fd17fdff700 1 --2- 172.21.15.92:0/1576264106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1780c9ed0 0x7fd1780cdee0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd168009840 tx=0x7fd168009b50 comp rx=0 tx=0).stop 2023-11-28T11:21:20.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.685+0000 7f270f7fe700 1 --2- 172.21.15.92:0/1652260701 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f26fc0682f0 0x7f26fc06a7a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2710202670 tx=0x7f2704008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:20.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.685+0000 7fafd939b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fafcc036b50 0x7fafcc0e90d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48910/0 (socket says 172.21.15.92:48910) 2023-11-28T11:21:20.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.685+0000 7f270cff9700 1 -- 172.21.15.92:0/1652260701 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2700051800 con 0x7f27100d2230 2023-11-28T11:21:20.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.684+0000 7f7b389fe700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b340cb5c0 0x7f7b340cc420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.684+0000 7f7b389fe700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b340ca2f0 con 0x7f7b340d2c70 2023-11-28T11:21:20.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.684+0000 7f7b389fe700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b340ca520 con 0x7f7b340d2780 2023-11-28T11:21:20.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.684+0000 7f7b389fe700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b340ca800 con 0x7f7b340cb5c0 2023-11-28T11:21:20.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.685+0000 7fd17fdff700 1 -- 172.21.15.92:0/1576264106 shutdown_connections 2023-11-28T11:21:20.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.685+0000 7fd17fdff700 1 --2- 172.21.15.92:0/1576264106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1780c9ed0 0x7fd1780cdee0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.685+0000 7fd17fdff700 1 -- 172.21.15.92:0/1576264106 >> 172.21.15.92:0/1576264106 conn(0x7fd1780cb490 msgr2=0x7fd1780cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:20.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.686+0000 7fd17fdff700 1 -- 172.21.15.92:0/1576264106 shutdown_connections 2023-11-28T11:21:20.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.686+0000 7fafda39d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fafcc051150 0x7fafcc0e70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.686+0000 7fafda39d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafcc0e6d00 0x7fafcc0e7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.686+0000 7fafda39d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafcc0e7b50 con 0x7fafcc036b50 2023-11-28T11:21:20.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.686+0000 7fafda39d700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafcc0e7d90 con 0x7fafcc0e6d00 2023-11-28T11:21:20.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.686+0000 7fafda39d700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafcc0e8070 con 0x7fafcc051150 2023-11-28T11:21:20.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.686+0000 7fafd939b700 1 -- 172.21.15.92:0/4157324327 learned_addr learned my addr 172.21.15.92:0/4157324327 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:20.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.686+0000 7fd17fdff700 1 -- 172.21.15.92:0/1576264106 wait complete. 2023-11-28T11:21:20.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.686+0000 7fafd8b9a700 1 --2- 172.21.15.92:0/4157324327 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fafcc051150 0x7fafcc0e70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.686+0000 7fafd939b700 1 -- 172.21.15.92:0/4157324327 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fafcc051150 msgr2=0x7fafcc0e70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.686+0000 7fafd939b700 1 --2- 172.21.15.92:0/4157324327 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fafcc051150 0x7fafcc0e70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.687+0000 7fc7d5fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7d00b1c60 0x7fc7d00cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.687+0000 7fc7d5fff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7d00d1950 con 0x7fc7d00d2df0 2023-11-28T11:21:20.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.687+0000 7fc7d5fff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7d00b2500 con 0x7fc7d00b4fc0 2023-11-28T11:21:20.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.687+0000 7fc7d5fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d00d3d30 con 0x7fc7d00b1c60 2023-11-28T11:21:20.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.686+0000 7fafd939b700 1 -- 172.21.15.92:0/4157324327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafcc0e6d00 msgr2=0x7fafcc0e7610 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T11:21:20.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.687+0000 7f7b3359e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b340cb5c0 0x7f7b340cc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.687+0000 7f7b3359e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b340cb5c0 0x7f7b340cc420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.92:34120/0 (socket says 172.21.15.92:34120) 2023-11-28T11:21:20.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.687+0000 7f7b3359e700 1 -- 172.21.15.92:0/2545255219 learned_addr learned my addr 172.21.15.92:0/2545255219 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:20.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.686+0000 7fafd939b700 1 --2- 172.21.15.92:0/4157324327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafcc0e6d00 0x7fafcc0e7610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.687+0000 7f7b3259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b340d2c70 0x7f7b3415a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.687+0000 7fafd939b700 1 -- 172.21.15.92:0/4157324327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafd404ec80 con 0x7fafcc036b50 2023-11-28T11:21:20.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.687+0000 7fe9acbff700 1 Processor -- start 2023-11-28T11:21:20.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.687+0000 7fafd939b700 1 --2- 172.21.15.92:0/4157324327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fafcc036b50 0x7fafcc0e90d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fafc400c500 tx=0x7fafc400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:20.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.687+0000 7fe9acbff700 1 -- start start 2023-11-28T11:21:20.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.688+0000 7fc7cf7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc7d00d2df0 0x7fc7d00d31c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47478/0 (socket says 172.21.15.92:47478) 2023-11-28T11:21:20.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.688+0000 7fc7cf7fe700 1 -- 172.21.15.92:0/4175939789 learned_addr learned my addr 172.21.15.92:0/4175939789 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:20.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.688+0000 7fafc9ffb700 1 -- 172.21.15.92:0/4157324327 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc4005d70 con 0x7fafcc036b50 2023-11-28T11:21:20.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.688+0000 7fafc9ffb700 1 -- 172.21.15.92:0/4157324327 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafc40070d0 con 0x7fafcc036b50 2023-11-28T11:21:20.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.688+0000 7fafc9ffb700 1 -- 172.21.15.92:0/4157324327 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc4007480 con 0x7fafcc036b50 2023-11-28T11:21:20.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.688+0000 7fafda39d700 1 -- 172.21.15.92:0/4157324327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafcc033ee0 con 0x7fafcc036b50 2023-11-28T11:21:20.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.688+0000 7f7b32d9d700 1 --2- 172.21.15.92:0/2545255219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b340d2780 0x7f7b340c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.688+0000 7f7b3359e700 1 -- 172.21.15.92:0/2545255219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b340d2780 msgr2=0x7f7b340c9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.688+0000 7f7b3359e700 1 --2- 172.21.15.92:0/2545255219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b340d2780 0x7f7b340c9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.688+0000 7f7b3359e700 1 -- 172.21.15.92:0/2545255219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b340d2c70 msgr2=0x7f7b3415a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.688+0000 7fc7cffff700 1 --2- 172.21.15.92:0/4175939789 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7d00b1c60 0x7fc7d00cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.688+0000 7f7b3359e700 1 --2- 172.21.15.92:0/2545255219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b340d2c70 0x7f7b3415a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.688+0000 7f7b3359e700 1 -- 172.21.15.92:0/2545255219 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b200094a0 con 0x7f7b340cb5c0 2023-11-28T11:21:20.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.688+0000 7fafda39d700 1 -- 172.21.15.92:0/4157324327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafcc033c00 con 0x7fafcc036b50 2023-11-28T11:21:20.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.689+0000 7fe9acbff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9a806c290 0x7fe9a80b2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.689+0000 7fe9acbff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9a80cfd40 0x7fe9a8157150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.689+0000 7fc7ce7fc700 1 -- 172.21.15.92:0/4175939789 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2396067065 0 0) 0x7fc7d00b2500 con 0x7fc7d00b4fc0 2023-11-28T11:21:20.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.689+0000 7fc7cffff700 1 -- 172.21.15.92:0/4175939789 >> v1:172.21.15.169:6789/0 conn(0x7fc7d00b4fc0 legacy=0x7fc7d00cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.689+0000 7fc7cffff700 1 -- 172.21.15.92:0/4175939789 >> v1:172.21.15.92:6789/0 conn(0x7fc7d00d2df0 legacy=0x7fc7d00d31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:20.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.689+0000 7fc7cffff700 1 -- 172.21.15.92:0/4175939789 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7d00d3a50 con 0x7fc7d00b1c60 2023-11-28T11:21:20.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.689+0000 7fe9acbff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9a815c400 0x7fe9a8157690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.689+0000 7fe9acbff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a80b47d0 con 0x7fe9a80cfd40 2023-11-28T11:21:20.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.689+0000 7fe9acbff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a80b4ab0 con 0x7fe9a815c400 2023-11-28T11:21:20.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.689+0000 7fe9acbff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a80b4d90 con 0x7fe9a806c290 2023-11-28T11:21:20.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.689+0000 7fe9a659c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9a806c290 0x7fe9a80b2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.689+0000 7fd17fdff700 1 Processor -- start 2023-11-28T11:21:20.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.689+0000 7fc7cffff700 1 --2- 172.21.15.92:0/4175939789 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7d00b1c60 0x7fc7d00cd620 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4009870 tx=0x7fc7c4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ee519116a10c99 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:20.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.689+0000 7fe9a659c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9a806c290 0x7fe9a80b2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.92:34140/0 (socket says 172.21.15.92:34140) 2023-11-28T11:21:20.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.689+0000 7fe9a659c700 1 -- 172.21.15.92:0/2672378293 learned_addr learned my addr 172.21.15.92:0/2672378293 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:20.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.690+0000 7fe9a6d9d700 1 --2- 172.21.15.92:0/2672378293 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9a815c400 0x7fe9a8157690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.690+0000 7fc7ce7fc700 1 -- 172.21.15.92:0/4175939789 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c4004230 con 0x7fc7d00b1c60 2023-11-28T11:21:20.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.689+0000 7f7b3259c700 1 --2- 172.21.15.92:0/2545255219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b340d2c70 0x7f7b3415a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:21:20.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.690+0000 7f7b3359e700 1 --2- 172.21.15.92:0/2545255219 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b340cb5c0 0x7f7b340cc420 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c00cbc0 tx=0x7f7b2c00cf80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:20.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.689+0000 7fd17fdff700 1 -- start start 2023-11-28T11:21:20.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.690+0000 7fc7ce7fc700 1 -- 172.21.15.92:0/4175939789 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c4004390 con 0x7fc7d00b1c60 2023-11-28T11:21:20.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.690+0000 7fe9a659c700 1 -- 172.21.15.92:0/2672378293 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9a815c400 msgr2=0x7fe9a8157690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.690+0000 7fe9a659c700 1 --2- 172.21.15.92:0/2672378293 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9a815c400 0x7fe9a8157690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.690+0000 7fe9a659c700 1 -- 172.21.15.92:0/2672378293 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9a80cfd40 msgr2=0x7fe9a8157150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.690+0000 7fe9a659c700 1 --2- 172.21.15.92:0/2672378293 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9a80cfd40 0x7fe9a8157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.690+0000 7fe9a659c700 1 -- 172.21.15.92:0/2672378293 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a0007940 con 0x7fe9a806c290 2023-11-28T11:21:20.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.690+0000 7fc7ce7fc700 1 -- 172.21.15.92:0/4175939789 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c40036a0 con 0x7fc7d00b1c60 2023-11-28T11:21:20.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.691+0000 7fc7d5fff700 1 -- 172.21.15.92:0/4175939789 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7d00b1c60 msgr2=0x7fc7d00cd620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.691+0000 7fc7d5fff700 1 --2- 172.21.15.92:0/4175939789 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7d00b1c60 0x7fc7d00cd620 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4009870 tx=0x7fc7c4009b80 comp rx=0 tx=0).stop 2023-11-28T11:21:20.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.691+0000 7fc7d5fff700 1 -- 172.21.15.92:0/4175939789 shutdown_connections 2023-11-28T11:21:20.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.691+0000 7fc7d5fff700 1 --2- 172.21.15.92:0/4175939789 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7d00b1c60 0x7fc7d00cd620 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.691+0000 7fc7d5fff700 1 -- 172.21.15.92:0/4175939789 >> 172.21.15.92:0/4175939789 conn(0x7fc7d00719a0 msgr2=0x7fc7d0071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:20.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.690+0000 7fafda39d700 1 -- 172.21.15.92:0/4157324327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafcc0341c0 con 0x7fafcc036b50 2023-11-28T11:21:20.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.692+0000 7fc7d5fff700 1 -- 172.21.15.92:0/4175939789 shutdown_connections 2023-11-28T11:21:20.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.690+0000 7f7b2b7fe700 1 -- 172.21.15.92:0/2545255219 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c00ab50 con 0x7f7b340cb5c0 2023-11-28T11:21:20.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.690+0000 7f7b2b7fe700 1 -- 172.21.15.92:0/2545255219 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b2c00acb0 con 0x7f7b340cb5c0 2023-11-28T11:21:20.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.690+0000 7f7b2b7fe700 1 -- 172.21.15.92:0/2545255219 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c012600 con 0x7f7b340cb5c0 2023-11-28T11:21:20.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.690+0000 7f7b389fe700 1 -- 172.21.15.92:0/2545255219 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b340cfa40 con 0x7f7b340cb5c0 2023-11-28T11:21:20.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.690+0000 7f7b389fe700 1 -- 172.21.15.92:0/2545255219 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b340cd5f0 con 0x7f7b340cb5c0 2023-11-28T11:21:20.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.693+0000 7fc7d5fff700 1 -- 172.21.15.92:0/4175939789 wait complete. 2023-11-28T11:21:20.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.692+0000 7f7b389fe700 1 -- 172.21.15.92:0/2545255219 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b100052f0 con 0x7f7b340cb5c0 2023-11-28T11:21:20.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.693+0000 7fd17fdff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1780c9ed0 0x7fd178159090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.693+0000 7fd17fdff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1780cbf70 0x7fd178145b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.693+0000 7fd17fdff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1780ce420 0x7fd178146050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.694+0000 7fe9a659c700 1 --2- 172.21.15.92:0/2672378293 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9a806c290 0x7fe9a80b2630 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe9980011e0 tx=0x7fe99800ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:20.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.694+0000 7fd17fdff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd178159fd0 con 0x7fd1780cbf70 2023-11-28T11:21:20.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.694+0000 7fd17fdff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1780c5bf0 con 0x7fd1780c9ed0 2023-11-28T11:21:20.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.694+0000 7fd17fdff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1780c5ed0 con 0x7fd1780ce420 2023-11-28T11:21:20.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.694+0000 7fd17db9b700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1780c9ed0 0x7fd178159090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.694+0000 7fd17db9b700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1780c9ed0 0x7fd178159090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.92:57178/0 (socket says 172.21.15.92:57178) 2023-11-28T11:21:20.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.694+0000 7fd17e39c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1780ce420 0x7fd178146050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.694+0000 7fd17e39c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1780ce420 0x7fd178146050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.92:34146/0 (socket says 172.21.15.92:34146) 2023-11-28T11:21:20.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.694+0000 7fd17e39c700 1 -- 172.21.15.92:0/2887326508 learned_addr learned my addr 172.21.15.92:0/2887326508 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:20.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.694+0000 7fd17d39a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1780cbf70 0x7fd178145b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.695+0000 7fd17e39c700 1 -- 172.21.15.92:0/2887326508 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1780c9ed0 msgr2=0x7fd178159090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.695+0000 7fd17e39c700 1 --2- 172.21.15.92:0/2887326508 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1780c9ed0 0x7fd178159090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.695+0000 7fd17e39c700 1 -- 172.21.15.92:0/2887326508 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1780cbf70 msgr2=0x7fd178145b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.695+0000 7fd17e39c700 1 --2- 172.21.15.92:0/2887326508 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1780cbf70 0x7fd178145b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.695+0000 7fd17e39c700 1 -- 172.21.15.92:0/2887326508 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1680094a0 con 0x7fd1780ce420 2023-11-28T11:21:20.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.696+0000 7fd17e39c700 1 --2- 172.21.15.92:0/2887326508 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1780ce420 0x7fd178146050 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd17000cb10 tx=0x7fd17000ced0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:20.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.694+0000 7fafc9ffb700 1 -- 172.21.15.92:0/4157324327 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fafc4019030 con 0x7fafcc036b50 2023-11-28T11:21:20.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.697+0000 7fafc9ffb700 1 --2- 172.21.15.92:0/4157324327 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fafb8068340 0x7fafb806a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.697+0000 7fafc9ffb700 1 -- 172.21.15.92:0/4157324327 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fafc4085870 con 0x7fafcc036b50 2023-11-28T11:21:20.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.695+0000 7fc7d5fff700 1 Processor -- start 2023-11-28T11:21:20.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.698+0000 7fafc9ffb700 1 -- 172.21.15.92:0/4157324327 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fafc4051d70 con 0x7fafcc036b50 2023-11-28T11:21:20.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.698+0000 7f7b2b7fe700 1 -- 172.21.15.92:0/2545255219 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7b2c019030 con 0x7f7b340cb5c0 2023-11-28T11:21:20.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.699+0000 7fafd8b9a700 1 --2- 172.21.15.92:0/4157324327 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fafb8068340 0x7fafb806a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.700+0000 7f7b2b7fe700 1 --2- 172.21.15.92:0/2545255219 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7b1c0683c0 0x7f7b1c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.700+0000 7fe996ffd700 1 -- 172.21.15.92:0/2672378293 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe998004c80 con 0x7fe9a806c290 2023-11-28T11:21:20.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.700+0000 7fe996ffd700 1 -- 172.21.15.92:0/2672378293 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe998004de0 con 0x7fe9a806c290 2023-11-28T11:21:20.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.700+0000 7fe996ffd700 1 -- 172.21.15.92:0/2672378293 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9980072f0 con 0x7fe9a806c290 2023-11-28T11:21:20.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.698+0000 7fc7d5fff700 1 -- start start 2023-11-28T11:21:20.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.699+0000 7fc7d5fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7d00b1c60 0x7fc7d006f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.699+0000 7fc7d5fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7d00b4fc0 0x7fc7d00cb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.699+0000 7fc7d5fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7d00d2df0 0x7fc7d00ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.699+0000 7fc7d5fff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d00c9f30 con 0x7fc7d00b4fc0 2023-11-28T11:21:20.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.699+0000 7fc7d5fff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d00cba90 con 0x7fc7d00b1c60 2023-11-28T11:21:20.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.699+0000 7fc7d5fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d00cbd70 con 0x7fc7d00d2df0 2023-11-28T11:21:20.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.700+0000 7fc7cf7fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7d00b1c60 0x7fc7d006f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.700+0000 7fc7cf7fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7d00b1c60 0x7fc7d006f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.92:57188/0 (socket says 172.21.15.92:57188) 2023-11-28T11:21:20.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.700+0000 7fc7cf7fe700 1 -- 172.21.15.92:0/1528971396 learned_addr learned my addr 172.21.15.92:0/1528971396 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:20.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.700+0000 7fc7ceffd700 1 --2- 172.21.15.92:0/1528971396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7d00b4fc0 0x7fc7d00cb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.700+0000 7fc7cffff700 1 --2- 172.21.15.92:0/1528971396 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7d00d2df0 0x7fc7d00ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.700+0000 7fc7cf7fe700 1 -- 172.21.15.92:0/1528971396 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7d00d2df0 msgr2=0x7fc7d00ccbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.700+0000 7fc7cf7fe700 1 --2- 172.21.15.92:0/1528971396 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7d00d2df0 0x7fc7d00ccbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.700+0000 7fc7cf7fe700 1 -- 172.21.15.92:0/1528971396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7d00b4fc0 msgr2=0x7fc7d00cb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:20.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.701+0000 7fc7cf7fe700 1 --2- 172.21.15.92:0/1528971396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7d00b4fc0 0x7fc7d00cb3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:20.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.701+0000 7fc7cf7fe700 1 -- 172.21.15.92:0/1528971396 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c40094a0 con 0x7fc7d00b1c60 2023-11-28T11:21:20.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.700+0000 7fd16e7fc700 1 -- 172.21.15.92:0/2887326508 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd170003c00 con 0x7fd1780ce420 2023-11-28T11:21:20.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.701+0000 7fc7cf7fe700 1 --2- 172.21.15.92:0/1528971396 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7d00b1c60 0x7fc7d006f5b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc7c800cc10 tx=0x7fc7c800cf20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:20.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.701+0000 7f7b32d9d700 1 --2- 172.21.15.92:0/2545255219 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7b1c0683c0 0x7f7b1c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.701+0000 7fc7b7fff700 1 -- 172.21.15.92:0/1528971396 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c800ab50 con 0x7fc7d00b1c60 2023-11-28T11:21:20.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.701+0000 7fc7d5fff700 1 -- 172.21.15.92:0/1528971396 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7d00b88f0 con 0x7fc7d00b1c60 2023-11-28T11:21:20.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.701+0000 7fc7b7fff700 1 -- 172.21.15.92:0/1528971396 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c800acd0 con 0x7fc7d00b1c60 2023-11-28T11:21:20.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.701+0000 7fc7b7fff700 1 -- 172.21.15.92:0/1528971396 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c8012640 con 0x7fc7d00b1c60 2023-11-28T11:21:20.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.702+0000 7fc7d5fff700 1 -- 172.21.15.92:0/1528971396 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7d00d15e0 con 0x7fc7d00b1c60 2023-11-28T11:21:20.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.700+0000 7fd17fdff700 1 -- 172.21.15.92:0/2887326508 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1780cd9c0 con 0x7fd1780ce420 2023-11-28T11:21:20.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.700+0000 7fd16e7fc700 1 -- 172.21.15.92:0/2887326508 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd170003d60 con 0x7fd1780ce420 2023-11-28T11:21:20.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.702+0000 7fd17fdff700 1 -- 172.21.15.92:0/2887326508 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1780cd6e0 con 0x7fd1780ce420 2023-11-28T11:21:20.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.703+0000 7f7b2b7fe700 1 -- 172.21.15.92:0/2545255219 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7b2c086390 con 0x7f7b340cb5c0 2023-11-28T11:21:20.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.703+0000 7fd16e7fc700 1 -- 172.21.15.92:0/2887326508 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17000ab00 con 0x7fd1780ce420 2023-11-28T11:21:20.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.703+0000 7f7b2b7fe700 1 -- 172.21.15.92:0/2545255219 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7b2c0528e0 con 0x7f7b340cb5c0 2023-11-28T11:21:20.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.704+0000 7f7b32d9d700 1 --2- 172.21.15.92:0/2545255219 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7b1c0683c0 0x7f7b1c06a870 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7b24009ab0 tx=0x7f7b24008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:20.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.704+0000 7fe9acbff700 1 -- 172.21.15.92:0/2672378293 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9a80cd9d0 con 0x7fe9a806c290 2023-11-28T11:21:20.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.704+0000 7fe9acbff700 1 -- 172.21.15.92:0/2672378293 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9a815c7d0 con 0x7fe9a806c290 2023-11-28T11:21:20.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.705+0000 7fc7b7fff700 1 -- 172.21.15.92:0/1528971396 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc7c8019030 con 0x7fc7d00b1c60 2023-11-28T11:21:20.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.705+0000 7fc7d5fff700 1 -- 172.21.15.92:0/1528971396 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7d00b21b0 con 0x7fc7d00b1c60 2023-11-28T11:21:20.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.706+0000 7fd16e7fc700 1 -- 172.21.15.92:0/2887326508 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd170010050 con 0x7fd1780ce420 2023-11-28T11:21:20.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.706+0000 7fafd8b9a700 1 --2- 172.21.15.92:0/4157324327 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fafb8068340 0x7fafb806a7f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fafcc033850 tx=0x7fafd0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:20.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.706+0000 7fc7b7fff700 1 --2- 172.21.15.92:0/1528971396 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc7bc0682f0 0x7fc7bc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.707+0000 7fd16e7fc700 1 --2- 172.21.15.92:0/2887326508 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd15c0682f0 0x7fd15c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.708+0000 7fd16e7fc700 1 -- 172.21.15.92:0/2887326508 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd17005d8b0 con 0x7fd1780ce420 2023-11-28T11:21:20.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.708+0000 7fd17fdff700 1 -- 172.21.15.92:0/2887326508 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd17815c110 con 0x7fd1780ce420 2023-11-28T11:21:20.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.707+0000 7fe996ffd700 1 -- 172.21.15.92:0/2672378293 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe99801a020 con 0x7fe9a806c290 2023-11-28T11:21:20.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.707+0000 7fe9acbff700 1 -- 172.21.15.92:0/2672378293 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9a80cdb60 con 0x7fe9a806c290 2023-11-28T11:21:20.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.708+0000 7fc7b7fff700 1 -- 172.21.15.92:0/1528971396 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fc7c8085b50 con 0x7fc7d00b1c60 2023-11-28T11:21:20.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.708+0000 7fc7ceffd700 1 --2- 172.21.15.92:0/1528971396 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc7bc0682f0 0x7fc7bc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.708+0000 7fe996ffd700 1 --2- 172.21.15.92:0/2672378293 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe988068340 0x7fe98806a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:20.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.708+0000 7fe996ffd700 1 -- 172.21.15.92:0/2672378293 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe998085e00 con 0x7fe9a806c290 2023-11-28T11:21:20.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.709+0000 7fd17db9b700 1 --2- 172.21.15.92:0/2887326508 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd15c0682f0 0x7fd15c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.709+0000 7fe9a5d9b700 1 --2- 172.21.15.92:0/2672378293 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe988068340 0x7fe98806a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:20.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.710+0000 7fc7ceffd700 1 --2- 172.21.15.92:0/1528971396 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc7bc0682f0 0x7fc7bc06a7a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc7d00c9430 tx=0x7fc7c0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:20.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.710+0000 7fe9a5d9b700 1 --2- 172.21.15.92:0/2672378293 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe988068340 0x7fe98806a7f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe99c006b20 tx=0x7fe99c006ab0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:20.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.710+0000 7fc7b7fff700 1 -- 172.21.15.92:0/1528971396 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc7c80520c0 con 0x7fc7d00b1c60 2023-11-28T11:21:20.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.713+0000 7fd17db9b700 1 --2- 172.21.15.92:0/2887326508 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd15c0682f0 0x7fd15c06a7a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd174007c20 tx=0x7fd174007d70 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:20.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.713+0000 7fe996ffd700 1 -- 172.21.15.92:0/2672378293 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe998052270 con 0x7fe9a806c290 2023-11-28T11:21:20.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:20.718+0000 7fd16e7fc700 1 -- 172.21.15.92:0/2887326508 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd1700522f0 con 0x7fd1780ce420 2023-11-28T11:21:21.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.083+0000 7f016eb9d700 1 -- 172.21.15.92:0/2174459798 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f01600076a0 con 0x7f01600e7260 2023-11-28T11:21:21.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.084+0000 7f015e7fc700 1 -- 172.21.15.92:0/2174459798 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0164011e60 con 0x7f01600e7260 2023-11-28T11:21:21.086 INFO:teuthology.orchestra.run.smithi092.stdout:55834574858 2023-11-28T11:21:21.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.088+0000 7f016eb9d700 1 -- 172.21.15.92:0/2174459798 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f014c068340 msgr2=0x7f014c06a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:21.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.088+0000 7f016eb9d700 1 --2- 172.21.15.92:0/2174459798 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f014c068340 0x7f014c06a7f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f01600346f0 tx=0x7f0168094040 comp rx=0 tx=0).stop 2023-11-28T11:21:21.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.089+0000 7f016eb9d700 1 -- 172.21.15.92:0/2174459798 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01600e7260 msgr2=0x7f016004dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:21.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.089+0000 7f016eb9d700 1 --2- 172.21.15.92:0/2174459798 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01600e7260 0x7f016004dd20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f016400b410 tx=0x7f016400b720 comp rx=0 tx=0).stop 2023-11-28T11:21:21.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.089+0000 7f016eb9d700 1 -- 172.21.15.92:0/2174459798 shutdown_connections 2023-11-28T11:21:21.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.089+0000 7f016eb9d700 1 --2- 172.21.15.92:0/2174459798 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f014c068340 0x7f014c06a7f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:21.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.089+0000 7f016eb9d700 1 --2- 172.21.15.92:0/2174459798 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01600e7260 0x7f016004dd20 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:21.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.089+0000 7f016eb9d700 1 --2- 172.21.15.92:0/2174459798 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01600367a0 0x7f01600e76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:21.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.089+0000 7f016eb9d700 1 --2- 172.21.15.92:0/2174459798 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01600e6e90 0x7f01600e5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:21.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.089+0000 7f016eb9d700 1 -- 172.21.15.92:0/2174459798 >> 172.21.15.92:0/2174459798 conn(0x7f01600517a0 msgr2=0x7f0160037cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:21.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.090+0000 7f016eb9d700 1 -- 172.21.15.92:0/2174459798 shutdown_connections 2023-11-28T11:21:21.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.090+0000 7f016eb9d700 1 -- 172.21.15.92:0/2174459798 wait complete. 2023-11-28T11:21:21.123 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.0 2023-11-28T11:21:21.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.127+0000 7fa239dff700 1 -- 172.21.15.92:0/1109882555 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa234067170 con 0x7fa2340adf90 2023-11-28T11:21:21.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.127+0000 7fa230ff9700 1 -- 172.21.15.92:0/1109882555 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa21c017020 con 0x7fa2340adf90 2023-11-28T11:21:21.129 INFO:teuthology.orchestra.run.smithi092.stdout:55834574858 2023-11-28T11:21:21.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.130+0000 7fa239dff700 1 -- 172.21.15.92:0/1109882555 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa214068350 msgr2=0x7fa21406a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:21.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.130+0000 7fa239dff700 1 --2- 172.21.15.92:0/1109882555 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa214068350 0x7fa21406a800 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa224011340 tx=0x7fa22400b040 comp rx=0 tx=0).stop 2023-11-28T11:21:21.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.130+0000 7fa239dff700 1 -- 172.21.15.92:0/1109882555 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2340adf90 msgr2=0x7fa2340c6340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:21.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.130+0000 7fa239dff700 1 --2- 172.21.15.92:0/1109882555 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2340adf90 0x7fa2340c6340 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa21c00c780 tx=0x7fa21c00ca90 comp rx=0 tx=0).stop 2023-11-28T11:21:21.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.130+0000 7fa239dff700 1 -- 172.21.15.92:0/1109882555 shutdown_connections 2023-11-28T11:21:21.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.130+0000 7fa239dff700 1 --2- 172.21.15.92:0/1109882555 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa234160f40 0x7fa23415a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:21.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.130+0000 7fa239dff700 1 --2- 172.21.15.92:0/1109882555 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa23406a980 0x7fa23415d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:21.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.130+0000 7fa239dff700 1 --2- 172.21.15.92:0/1109882555 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa214068350 0x7fa21406a800 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:21.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.130+0000 7fa239dff700 1 --2- 172.21.15.92:0/1109882555 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2340adf90 0x7fa2340c6340 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:21.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.131+0000 7fa239dff700 1 -- 172.21.15.92:0/1109882555 >> 172.21.15.92:0/1109882555 conn(0x7fa2340cb490 msgr2=0x7fa23406c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:21.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.131+0000 7fa239dff700 1 -- 172.21.15.92:0/1109882555 shutdown_connections 2023-11-28T11:21:21.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.131+0000 7fa239dff700 1 -- 172.21.15.92:0/1109882555 wait complete. 2023-11-28T11:21:21.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.132+0000 7f271539d700 1 -- 172.21.15.92:0/1652260701 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f271004e620 con 0x7f27100d2230 2023-11-28T11:21:21.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.133+0000 7f270cff9700 1 -- 172.21.15.92:0/1652260701 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2700054e20 con 0x7f27100d2230 2023-11-28T11:21:21.136 INFO:teuthology.orchestra.run.smithi092.stdout:55834574857 2023-11-28T11:21:21.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.137+0000 7f271539d700 1 -- 172.21.15.92:0/1652260701 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f26fc0682f0 msgr2=0x7f26fc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:21.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.137+0000 7f271539d700 1 --2- 172.21.15.92:0/1652260701 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f26fc0682f0 0x7f26fc06a7a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2710202670 tx=0x7f2704008030 comp rx=0 tx=0).stop 2023-11-28T11:21:21.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.137+0000 7f271539d700 1 -- 172.21.15.92:0/1652260701 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27100d2230 msgr2=0x7f27100cb0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:21.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.137+0000 7f271539d700 1 --2- 172.21.15.92:0/1652260701 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27100d2230 0x7f27100cb0d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f270000c3f0 tx=0x7f270000c700 comp rx=0 tx=0).stop 2023-11-28T11:21:21.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.137+0000 7f271539d700 1 -- 172.21.15.92:0/1652260701 shutdown_connections 2023-11-28T11:21:21.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.137+0000 7f271539d700 1 --2- 172.21.15.92:0/1652260701 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f26fc0682f0 0x7f26fc06a7a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:21.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.137+0000 7f271539d700 1 --2- 172.21.15.92:0/1652260701 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27101fe8b0 0x7f27100b2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:21.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.137+0000 7f271539d700 1 --2- 172.21.15.92:0/1652260701 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f27100d2230 0x7f27100cb0d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:21.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.137+0000 7f271539d700 1 --2- 172.21.15.92:0/1652260701 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27100d2c70 0x7f271015e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:21.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.137+0000 7f271539d700 1 -- 172.21.15.92:0/1652260701 >> 172.21.15.92:0/1652260701 conn(0x7f27100719a0 msgr2=0x7f27100b3f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:21.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.138+0000 7f271539d700 1 -- 172.21.15.92:0/1652260701 shutdown_connections 2023-11-28T11:21:21.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.138+0000 7f271539d700 1 -- 172.21.15.92:0/1652260701 wait complete. 2023-11-28T11:21:21.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.142+0000 7fafda39d700 1 -- 172.21.15.92:0/4157324327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fafcc0053d0 con 0x7fafcc036b50 2023-11-28T11:21:21.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.143+0000 7fafc9ffb700 1 -- 172.21.15.92:0/4157324327 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fafc4017020 con 0x7fafcc036b50 2023-11-28T11:21:21.145 INFO:teuthology.orchestra.run.smithi092.stdout:55834574856 2023-11-28T11:21:21.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.143+0000 7f7b389fe700 1 -- 172.21.15.92:0/2545255219 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f7b10005870 con 0x7f7b340cb5c0 2023-11-28T11:21:21.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.145+0000 7f7b2b7fe700 1 -- 172.21.15.92:0/2545255219 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7b2c021020 con 0x7f7b340cb5c0 2023-11-28T11:21:21.147 INFO:teuthology.orchestra.run.smithi092.stdout:55834574858 2023-11-28T11:21:21.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.146+0000 7fafda39d700 1 -- 172.21.15.92:0/4157324327 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fafb8068340 msgr2=0x7fafb806a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:21.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.146+0000 7fafda39d700 1 --2- 172.21.15.92:0/4157324327 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fafb8068340 0x7fafb806a7f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fafcc033850 tx=0x7fafd0008040 comp rx=0 tx=0).stop 2023-11-28T11:21:21.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.146+0000 7fafda39d700 1 -- 172.21.15.92:0/4157324327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fafcc036b50 msgr2=0x7fafcc0e90d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:21.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.146+0000 7fafda39d700 1 --2- 172.21.15.92:0/4157324327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fafcc036b50 0x7fafcc0e90d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fafc400c500 tx=0x7fafc400c8c0 comp rx=0 tx=0).stop 2023-11-28T11:21:21.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.147+0000 7fafda39d700 1 -- 172.21.15.92:0/4157324327 shutdown_connections 2023-11-28T11:21:21.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.147+0000 7fafda39d700 1 --2- 172.21.15.92:0/4157324327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fafcc0e6d00 0x7fafcc0e7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:21.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.147+0000 7fafda39d700 1 --2- 172.21.15.92:0/4157324327 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fafb8068340 0x7fafb806a7f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:21.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.147+0000 7fafda39d700 1 --2- 172.21.15.92:0/4157324327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fafcc036b50 0x7fafcc0e90d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:21.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.147+0000 7fafda39d700 1 --2- 172.21.15.92:0/4157324327 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fafcc051150 0x7fafcc0e70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:21.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.147+0000 7fafda39d700 1 -- 172.21.15.92:0/4157324327 >> 172.21.15.92:0/4157324327 conn(0x7fafcc051610 msgr2=0x7fafcc034380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:21.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.147+0000 7fafda39d700 1 -- 172.21.15.92:0/4157324327 shutdown_connections 2023-11-28T11:21:21.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.148+0000 7fafda39d700 1 -- 172.21.15.92:0/4157324327 wait complete. 2023-11-28T11:21:21.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.149+0000 7f7b389fe700 1 -- 172.21.15.92:0/2545255219 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7b1c0683c0 msgr2=0x7f7b1c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:21.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.149+0000 7f7b389fe700 1 --2- 172.21.15.92:0/2545255219 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7b1c0683c0 0x7f7b1c06a870 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7b24009ab0 tx=0x7f7b24008040 comp rx=0 tx=0).stop 2023-11-28T11:21:21.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.149+0000 7f7b389fe700 1 -- 172.21.15.92:0/2545255219 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b340cb5c0 msgr2=0x7f7b340cc420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:21.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.149+0000 7f7b389fe700 1 --2- 172.21.15.92:0/2545255219 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b340cb5c0 0x7f7b340cc420 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c00cbc0 tx=0x7f7b2c00cf80 comp rx=0 tx=0).stop 2023-11-28T11:21:21.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.150+0000 7f7b389fe700 1 -- 172.21.15.92:0/2545255219 shutdown_connections 2023-11-28T11:21:21.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.150+0000 7f7b389fe700 1 --2- 172.21.15.92:0/2545255219 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b340cb5c0 0x7f7b340cc420 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:21.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.150+0000 7f7b389fe700 1 --2- 172.21.15.92:0/2545255219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b340d2780 0x7f7b340c9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:21.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.150+0000 7f7b389fe700 1 --2- 172.21.15.92:0/2545255219 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7b1c0683c0 0x7f7b1c06a870 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:21.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.150+0000 7f7b389fe700 1 --2- 172.21.15.92:0/2545255219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b340d2c70 0x7f7b3415a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:21.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.150+0000 7f7b389fe700 1 -- 172.21.15.92:0/2545255219 >> 172.21.15.92:0/2545255219 conn(0x7f7b340719a0 msgr2=0x7f7b340b2930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:21.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.150+0000 7f7b389fe700 1 -- 172.21.15.92:0/2545255219 shutdown_connections 2023-11-28T11:21:21.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.150+0000 7f7b389fe700 1 -- 172.21.15.92:0/2545255219 wait complete. 2023-11-28T11:21:21.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.155+0000 7fc7d5fff700 1 -- 172.21.15.92:0/1528971396 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fc7d0067170 con 0x7fc7d00b1c60 2023-11-28T11:21:21.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.155+0000 7fc7b7fff700 1 -- 172.21.15.92:0/1528971396 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc7c8017090 con 0x7fc7d00b1c60 2023-11-28T11:21:21.157 INFO:teuthology.orchestra.run.smithi092.stdout:55834574858 2023-11-28T11:21:21.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.157+0000 7fd17fdff700 1 -- 172.21.15.92:0/2887326508 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fd17816cd70 con 0x7fd1780ce420 2023-11-28T11:21:21.160 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.1 2023-11-28T11:21:21.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.158+0000 7fd16e7fc700 1 -- 172.21.15.92:0/2887326508 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd17816cd70 con 0x7fd1780ce420 2023-11-28T11:21:21.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.159+0000 7fc7d5fff700 1 -- 172.21.15.92:0/1528971396 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc7bc0682f0 msgr2=0x7fc7bc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:21.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.159+0000 7fc7d5fff700 1 --2- 172.21.15.92:0/1528971396 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc7bc0682f0 0x7fc7bc06a7a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc7d00c9430 tx=0x7fc7c0009040 comp rx=0 tx=0).stop 2023-11-28T11:21:21.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.159+0000 7fc7d5fff700 1 -- 172.21.15.92:0/1528971396 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7d00b1c60 msgr2=0x7fc7d006f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:21.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.159+0000 7fc7d5fff700 1 --2- 172.21.15.92:0/1528971396 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7d00b1c60 0x7fc7d006f5b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc7c800cc10 tx=0x7fc7c800cf20 comp rx=0 tx=0).stop 2023-11-28T11:21:21.161 INFO:teuthology.orchestra.run.smithi092.stdout:55834574857 2023-11-28T11:21:21.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.159+0000 7fc7d5fff700 1 -- 172.21.15.92:0/1528971396 shutdown_connections 2023-11-28T11:21:21.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.159+0000 7fc7d5fff700 1 --2- 172.21.15.92:0/1528971396 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7d00d2df0 0x7fc7d00ccbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:21.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.159+0000 7fc7d5fff700 1 --2- 172.21.15.92:0/1528971396 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7d00b1c60 0x7fc7d006f5b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:21.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.159+0000 7fc7d5fff700 1 --2- 172.21.15.92:0/1528971396 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc7bc0682f0 0x7fc7bc06a7a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:21.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.159+0000 7fc7d5fff700 1 --2- 172.21.15.92:0/1528971396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7d00b4fc0 0x7fc7d00cb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:21.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.159+0000 7fc7d5fff700 1 -- 172.21.15.92:0/1528971396 >> 172.21.15.92:0/1528971396 conn(0x7fc7d00719a0 msgr2=0x7fc7d00d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:21.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.160+0000 7fc7d5fff700 1 -- 172.21.15.92:0/1528971396 shutdown_connections 2023-11-28T11:21:21.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.160+0000 7fc7d5fff700 1 -- 172.21.15.92:0/1528971396 wait complete. 2023-11-28T11:21:21.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.162+0000 7fd17fdff700 1 -- 172.21.15.92:0/2887326508 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd15c0682f0 msgr2=0x7fd15c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:21.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.162+0000 7fd17fdff700 1 --2- 172.21.15.92:0/2887326508 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd15c0682f0 0x7fd15c06a7a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd174007c20 tx=0x7fd174007d70 comp rx=0 tx=0).stop 2023-11-28T11:21:21.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.162+0000 7fd17fdff700 1 -- 172.21.15.92:0/2887326508 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1780ce420 msgr2=0x7fd178146050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:21.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.162+0000 7fd17fdff700 1 --2- 172.21.15.92:0/2887326508 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1780ce420 0x7fd178146050 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd17000cb10 tx=0x7fd17000ced0 comp rx=0 tx=0).stop 2023-11-28T11:21:21.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.163+0000 7fd17fdff700 1 -- 172.21.15.92:0/2887326508 shutdown_connections 2023-11-28T11:21:21.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.163+0000 7fd17fdff700 1 --2- 172.21.15.92:0/2887326508 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1780ce420 0x7fd178146050 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:21.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.163+0000 7fd17fdff700 1 --2- 172.21.15.92:0/2887326508 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd1780c9ed0 0x7fd178159090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:21.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.163+0000 7fd17fdff700 1 --2- 172.21.15.92:0/2887326508 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd15c0682f0 0x7fd15c06a7a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:21.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.163+0000 7fd17fdff700 1 --2- 172.21.15.92:0/2887326508 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1780cbf70 0x7fd178145b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:21.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.163+0000 7fd17fdff700 1 -- 172.21.15.92:0/2887326508 >> 172.21.15.92:0/2887326508 conn(0x7fd1780cb490 msgr2=0x7fd17806a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:21.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.164+0000 7fd17fdff700 1 -- 172.21.15.92:0/2887326508 shutdown_connections 2023-11-28T11:21:21.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.164+0000 7fd17fdff700 1 -- 172.21.15.92:0/2887326508 wait complete. 2023-11-28T11:21:21.169 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.3 2023-11-28T11:21:21.178 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.2 2023-11-28T11:21:21.181 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.6 2023-11-28T11:21:21.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.181+0000 7fe9acbff700 1 -- 172.21.15.92:0/2672378293 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fe9a8142db0 con 0x7fe9a806c290 2023-11-28T11:21:21.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.182+0000 7fe996ffd700 1 -- 172.21.15.92:0/2672378293 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe998055890 con 0x7fe9a806c290 2023-11-28T11:21:21.184 INFO:teuthology.orchestra.run.smithi092.stdout:55834574857 2023-11-28T11:21:21.185 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.7 2023-11-28T11:21:21.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.184+0000 7fe9acbff700 1 -- 172.21.15.92:0/2672378293 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe988068340 msgr2=0x7fe98806a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:21.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.184+0000 7fe9acbff700 1 --2- 172.21.15.92:0/2672378293 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe988068340 0x7fe98806a7f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe99c006b20 tx=0x7fe99c006ab0 comp rx=0 tx=0).stop 2023-11-28T11:21:21.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.184+0000 7fe9acbff700 1 -- 172.21.15.92:0/2672378293 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9a806c290 msgr2=0x7fe9a80b2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:21.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.184+0000 7fe9acbff700 1 --2- 172.21.15.92:0/2672378293 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9a806c290 0x7fe9a80b2630 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe9980011e0 tx=0x7fe99800ca30 comp rx=0 tx=0).stop 2023-11-28T11:21:21.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.184+0000 7fe9acbff700 1 -- 172.21.15.92:0/2672378293 shutdown_connections 2023-11-28T11:21:21.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.184+0000 7fe9acbff700 1 --2- 172.21.15.92:0/2672378293 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe9a815c400 0x7fe9a8157690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:21.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.184+0000 7fe9acbff700 1 --2- 172.21.15.92:0/2672378293 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe9a806c290 0x7fe9a80b2630 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:21.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.184+0000 7fe9acbff700 1 --2- 172.21.15.92:0/2672378293 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe988068340 0x7fe98806a7f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:21.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.184+0000 7fe9acbff700 1 --2- 172.21.15.92:0/2672378293 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9a80cfd40 0x7fe9a8157150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:21.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.184+0000 7fe9acbff700 1 -- 172.21.15.92:0/2672378293 >> 172.21.15.92:0/2672378293 conn(0x7fe9a80719a0 msgr2=0x7fe9a80d2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:21.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.184+0000 7fe9acbff700 1 -- 172.21.15.92:0/2672378293 shutdown_connections 2023-11-28T11:21:21.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:21.184+0000 7fe9acbff700 1 -- 172.21.15.92:0/2672378293 wait complete. 2023-11-28T11:21:21.193 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.4 2023-11-28T11:21:21.205 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.5 2023-11-28T11:21:22.124 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-28T11:21:22.160 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-28T11:21:22.169 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-28T11:21:22.179 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-28T11:21:22.181 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-28T11:21:22.186 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-28T11:21:22.193 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-28T11:21:22.206 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-28T11:21:22.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.410+0000 7f7a0f3ff700 1 Processor -- start 2023-11-28T11:21:22.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.410+0000 7f7a0f3ff700 1 -- start start 2023-11-28T11:21:22.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.413+0000 7f7a0f3ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a000513f0 0x7f7a00038900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:22.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.413+0000 7f7a0f3ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7a00038e40 0x7f7a000e7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:22.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.413+0000 7f7a0f3ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a00051840 con 0x7f7a00054970 2023-11-28T11:21:22.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.413+0000 7f7a0f3ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a00039290 con 0x7f7a00038e40 2023-11-28T11:21:22.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.413+0000 7f7a0f3ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a000ea8f0 con 0x7f7a000513f0 2023-11-28T11:21:22.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.414+0000 7f7a0d99c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7a00038e40 0x7f7a000e7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:22.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.414+0000 7f7a0c99a700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a000513f0 0x7f7a00038900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:22.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.415+0000 7f7a0c99a700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a000513f0 0x7f7a00038900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.92:34156/0 (socket says 172.21.15.92:34156) 2023-11-28T11:21:22.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.415+0000 7f7a0c99a700 1 -- 172.21.15.92:0/2679406423 learned_addr learned my addr 172.21.15.92:0/2679406423 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:22.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.415+0000 7f7a0c99a700 1 -- 172.21.15.92:0/2679406423 >> v2:172.21.15.92:3300/0 conn(0x7f7a00038e40 msgr2=0x7f7a000e7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:22.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.416+0000 7f7a0c99a700 1 --2- 172.21.15.92:0/2679406423 >> v2:172.21.15.92:3300/0 conn(0x7f7a00038e40 0x7f7a000e7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:22.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.416+0000 7f79fffff700 1 -- 172.21.15.92:0/2679406423 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2806765380 0 0) 0x7f7a04001110 con 0x7f7a00054970 2023-11-28T11:21:22.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.416+0000 7f7a0c99a700 1 -- 172.21.15.92:0/2679406423 >> v1:172.21.15.169:6789/0 conn(0x7f7a00054970 legacy=0x7f7a00054d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:22.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.416+0000 7f7a0c99a700 1 -- 172.21.15.92:0/2679406423 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a000eabd0 con 0x7f7a000513f0 2023-11-28T11:21:22.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.416+0000 7f7a0d99c700 1 --2- 172.21.15.92:0/2679406423 >> v2:172.21.15.92:3300/0 conn(0x7f7a00038e40 0x7f7a000e7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:21:22.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.417+0000 7f7a0c99a700 1 --2- 172.21.15.92:0/2679406423 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a000513f0 0x7f7a00038900 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f79f80098f0 tx=0x7f79f8009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f75092b638619c13 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:22.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.417+0000 7f79fffff700 1 -- 172.21.15.92:0/2679406423 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f800b8a0 con 0x7f7a000513f0 2023-11-28T11:21:22.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.417+0000 7f79fffff700 1 -- 172.21.15.92:0/2679406423 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f800ba00 con 0x7f7a000513f0 2023-11-28T11:21:22.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.418+0000 7f79fffff700 1 -- 172.21.15.92:0/2679406423 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f800bd90 con 0x7f7a000513f0 2023-11-28T11:21:22.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.419+0000 7f7a0f3ff700 1 -- 172.21.15.92:0/2679406423 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a000513f0 msgr2=0x7f7a00038900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:22.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.419+0000 7f7a0f3ff700 1 --2- 172.21.15.92:0/2679406423 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a000513f0 0x7f7a00038900 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f79f80098f0 tx=0x7f79f8009c00 comp rx=0 tx=0).stop 2023-11-28T11:21:22.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.419+0000 7f7a0f3ff700 1 -- 172.21.15.92:0/2679406423 shutdown_connections 2023-11-28T11:21:22.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.419+0000 7f7a0f3ff700 1 --2- 172.21.15.92:0/2679406423 >> v2:172.21.15.92:3300/0 conn(0x7f7a00038e40 0x7f7a000e7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:22.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.419+0000 7f7a0f3ff700 1 --2- 172.21.15.92:0/2679406423 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a000513f0 0x7f7a00038900 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:22.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.419+0000 7f7a0f3ff700 1 -- 172.21.15.92:0/2679406423 >> 172.21.15.92:0/2679406423 conn(0x7f7a0000cf40 msgr2=0x7f7a00053820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:22.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.420+0000 7f7a0f3ff700 1 -- 172.21.15.92:0/2679406423 shutdown_connections 2023-11-28T11:21:22.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.420+0000 7f7a0f3ff700 1 -- 172.21.15.92:0/2679406423 wait complete. 2023-11-28T11:21:22.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.424+0000 7f7a0f3ff700 1 Processor -- start 2023-11-28T11:21:22.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.424+0000 7f7a0f3ff700 1 -- start start 2023-11-28T11:21:22.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.426+0000 7f7a0f3ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a00038e40 0x7f7a000e6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:22.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.427+0000 7f7a0f3ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a000513f0 0x7f7a0004b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:22.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.427+0000 7f7a0f3ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a00054970 0x7f7a0004be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:22.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.427+0000 7f7a0c99a700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a000513f0 0x7f7a0004b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:22.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.427+0000 7f7a0c99a700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a000513f0 0x7f7a0004b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.92:57194/0 (socket says 172.21.15.92:57194) 2023-11-28T11:21:22.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.428+0000 7f7a0c99a700 1 -- 172.21.15.92:0/3909703649 learned_addr learned my addr 172.21.15.92:0/3909703649 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:22.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.428+0000 7f7a0f3ff700 1 -- 172.21.15.92:0/3909703649 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a000e95b0 con 0x7f7a00038e40 2023-11-28T11:21:22.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.428+0000 7f7a0f3ff700 1 -- 172.21.15.92:0/3909703649 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0004c490 con 0x7f7a000513f0 2023-11-28T11:21:22.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.428+0000 7f7a0f3ff700 1 -- 172.21.15.92:0/3909703649 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0004c770 con 0x7f7a00054970 2023-11-28T11:21:22.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.428+0000 7f7a0d99c700 1 --2- 172.21.15.92:0/3909703649 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a00054970 0x7f7a0004be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:22.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.429+0000 7f7a0c99a700 1 -- 172.21.15.92:0/3909703649 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a00054970 msgr2=0x7f7a0004be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:22.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.429+0000 7f7a0c99a700 1 --2- 172.21.15.92:0/3909703649 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a00054970 0x7f7a0004be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:22.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.429+0000 7f7a0c99a700 1 -- 172.21.15.92:0/3909703649 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a00038e40 msgr2=0x7f7a000e6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:22.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.429+0000 7f7a0d19b700 1 --2- 172.21.15.92:0/3909703649 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a00038e40 0x7f7a000e6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:22.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.429+0000 7f7a0c99a700 1 --2- 172.21.15.92:0/3909703649 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a00038e40 0x7f7a000e6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:22.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.429+0000 7f7a0c99a700 1 -- 172.21.15.92:0/3909703649 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f8009550 con 0x7f7a000513f0 2023-11-28T11:21:22.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.429+0000 7f7a0d19b700 1 --2- 172.21.15.92:0/3909703649 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a00038e40 0x7f7a000e6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:21:22.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.430+0000 7f7a0c99a700 1 --2- 172.21.15.92:0/3909703649 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a000513f0 0x7f7a0004b920 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f79f8009840 tx=0x7f79f8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:22.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.431+0000 7f79fdffb700 1 -- 172.21.15.92:0/3909703649 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f80038a0 con 0x7f7a000513f0 2023-11-28T11:21:22.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.431+0000 7f79fdffb700 1 -- 172.21.15.92:0/3909703649 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f801f070 con 0x7f7a000513f0 2023-11-28T11:21:22.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.431+0000 7f7a0f3ff700 1 -- 172.21.15.92:0/3909703649 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a000eb190 con 0x7f7a000513f0 2023-11-28T11:21:22.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.431+0000 7f7a0f3ff700 1 -- 172.21.15.92:0/3909703649 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a000eaeb0 con 0x7f7a000513f0 2023-11-28T11:21:22.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.431+0000 7f79fdffb700 1 -- 172.21.15.92:0/3909703649 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f80038a0 con 0x7f7a000513f0 2023-11-28T11:21:22.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.433+0000 7f7a0f3ff700 1 -- 172.21.15.92:0/3909703649 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79e80052f0 con 0x7f7a000513f0 2023-11-28T11:21:22.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.436+0000 7f79fdffb700 1 -- 172.21.15.92:0/3909703649 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f79f802a020 con 0x7f7a000513f0 2023-11-28T11:21:22.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.437+0000 7f79fdffb700 1 --2- 172.21.15.92:0/3909703649 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f79ec068350 0x7f79ec06a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:22.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.438+0000 7f79fdffb700 1 -- 172.21.15.92:0/3909703649 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f79f80878b0 con 0x7f7a000513f0 2023-11-28T11:21:22.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.439+0000 7f7a0d19b700 1 --2- 172.21.15.92:0/3909703649 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f79ec068350 0x7f79ec06a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:22.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.439+0000 7f79fdffb700 1 -- 172.21.15.92:0/3909703649 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f79f8053db0 con 0x7f7a000513f0 2023-11-28T11:21:22.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.442+0000 7f7a0d19b700 1 --2- 172.21.15.92:0/3909703649 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f79ec068350 0x7f79ec06a800 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7a04007fd0 tx=0x7f7a04009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:22.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.493+0000 7ffac53ff700 1 Processor -- start 2023-11-28T11:21:22.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.493+0000 7ffac53ff700 1 -- start start 2023-11-28T11:21:22.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.495+0000 7ffac53ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffac00cbf70 0x7ffac00c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:22.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.495+0000 7ffac53ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ffac00cdee0 0x7ffac006a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:22.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.495+0000 7ffac53ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffac00cce40 con 0x7ffac015bea0 2023-11-28T11:21:22.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.495+0000 7ffac53ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffac00cd120 con 0x7ffac00cdee0 2023-11-28T11:21:22.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.495+0000 7ffac53ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffac00cd400 con 0x7ffac00cbf70 2023-11-28T11:21:22.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.496+0000 7ffabf7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ffac00cdee0 0x7ffac006a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:22.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.496+0000 7ffabf7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ffac00cdee0 0x7ffac006a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48996/0 (socket says 172.21.15.92:48996) 2023-11-28T11:21:22.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.496+0000 7ffabf7fe700 1 -- 172.21.15.92:0/3273169549 learned_addr learned my addr 172.21.15.92:0/3273169549 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:22.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.497+0000 7ffabf7fe700 1 -- 172.21.15.92:0/3273169549 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffac00cbf70 msgr2=0x7ffac00c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:22.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.497+0000 7ffabf7fe700 1 --2- 172.21.15.92:0/3273169549 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffac00cbf70 0x7ffac00c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:22.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.497+0000 7ffabf7fe700 1 -- 172.21.15.92:0/3273169549 >> v1:172.21.15.92:6789/0 conn(0x7ffac015bea0 legacy=0x7ffac015ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:22.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.497+0000 7ffabf7fe700 1 -- 172.21.15.92:0/3273169549 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac00cd6e0 con 0x7ffac00cdee0 2023-11-28T11:21:22.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.498+0000 7ffabf7fe700 1 --2- 172.21.15.92:0/3273169549 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffac00cdee0 0x7ffac006a270 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ffaa80098f0 tx=0x7ffaa8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5936dab0dbc7bcd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:22.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.499+0000 7ffabdffb700 1 -- 172.21.15.92:0/3273169549 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa8004230 con 0x7ffac00cdee0 2023-11-28T11:21:22.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.499+0000 7ffabdffb700 1 -- 172.21.15.92:0/3273169549 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaa8004390 con 0x7ffac00cdee0 2023-11-28T11:21:22.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.499+0000 7ffabdffb700 1 -- 172.21.15.92:0/3273169549 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa80036a0 con 0x7ffac00cdee0 2023-11-28T11:21:22.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.500+0000 7ffac53ff700 1 -- 172.21.15.92:0/3273169549 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffac00cdee0 msgr2=0x7ffac006a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:22.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.500+0000 7ffac53ff700 1 --2- 172.21.15.92:0/3273169549 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffac00cdee0 0x7ffac006a270 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ffaa80098f0 tx=0x7ffaa8009c00 comp rx=0 tx=0).stop 2023-11-28T11:21:22.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.500+0000 7ffac53ff700 1 -- 172.21.15.92:0/3273169549 shutdown_connections 2023-11-28T11:21:22.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.500+0000 7ffac53ff700 1 --2- 172.21.15.92:0/3273169549 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffac00cbf70 0x7ffac00c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:22.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.500+0000 7ffac53ff700 1 --2- 172.21.15.92:0/3273169549 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffac00cdee0 0x7ffac006a270 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:22.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.500+0000 7ffac53ff700 1 -- 172.21.15.92:0/3273169549 >> 172.21.15.92:0/3273169549 conn(0x7ffac00cb490 msgr2=0x7ffac00cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:22.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.501+0000 7ffac53ff700 1 -- 172.21.15.92:0/3273169549 shutdown_connections 2023-11-28T11:21:22.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.501+0000 7ffac53ff700 1 -- 172.21.15.92:0/3273169549 wait complete. 2023-11-28T11:21:22.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.501+0000 7f61c0fff700 1 Processor -- start 2023-11-28T11:21:22.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.501+0000 7f61c0fff700 1 -- start start 2023-11-28T11:21:22.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.505+0000 7ffac53ff700 1 Processor -- start 2023-11-28T11:21:22.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.505+0000 7ffac53ff700 1 -- start start 2023-11-28T11:21:22.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.506+0000 7ffac53ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffac00cbf70 0x7ffac0169da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:22.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.506+0000 7ffac53ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffac00cdee0 0x7ffac0144ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:22.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.506+0000 7ffac53ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffac015bea0 0x7ffac0145000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:22.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.506+0000 7f61c0fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61bc0d1f20 0x7f61bc0d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:22.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.506+0000 7f61c0fff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61bc15b4e0 con 0x7f61bc160fd0 2023-11-28T11:21:22.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.506+0000 7f61c0fff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61bc0d2970 con 0x7f61bc0cf8b0 2023-11-28T11:21:22.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.506+0000 7f61c0fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61bc06c1b0 con 0x7f61bc0d1f20 2023-11-28T11:21:22.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.507+0000 7f61b9d9b700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61bc0d1f20 0x7f61bc0d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:22.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.507+0000 7f61b9d9b700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61bc0d1f20 0x7f61bc0d2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.92:34190/0 (socket says 172.21.15.92:34190) 2023-11-28T11:21:22.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.507+0000 7f61b9d9b700 1 -- 172.21.15.92:0/830976310 learned_addr learned my addr 172.21.15.92:0/830976310 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:22.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.508+0000 7f61b9d9b700 1 -- 172.21.15.92:0/830976310 >> v1:172.21.15.169:6789/0 conn(0x7f61bc0cf8b0 legacy=0x7f61bc0cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:22.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.508+0000 7f61b9d9b700 1 -- 172.21.15.92:0/830976310 >> v1:172.21.15.92:6789/0 conn(0x7f61bc160fd0 legacy=0x7f61bc161440 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T11:21:22.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.508+0000 7f61b9d9b700 1 -- 172.21.15.92:0/830976310 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61bc06baf0 con 0x7f61bc0d1f20 2023-11-28T11:21:22.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.509+0000 7ffabeffd700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffac00cbf70 0x7ffac0169da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:22.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.509+0000 7f61b9d9b700 1 --2- 172.21.15.92:0/830976310 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61bc0d1f20 0x7f61bc0d2370 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f61b4009e80 tx=0x7f61b400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4066ff865288b764 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:22.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.509+0000 7ffabeffd700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffac00cbf70 0x7ffac0169da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.92:34196/0 (socket says 172.21.15.92:34196) 2023-11-28T11:21:22.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.509+0000 7ffabf7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffac015bea0 0x7ffac0145000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:22.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.509+0000 7ffabf7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffac015bea0 0x7ffac0145000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49006/0 (socket says 172.21.15.92:49006) 2023-11-28T11:21:22.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.509+0000 7f61b959a700 1 -- 172.21.15.92:0/830976310 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b400e070 con 0x7f61bc0d1f20 2023-11-28T11:21:22.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.509+0000 7f61b959a700 1 -- 172.21.15.92:0/830976310 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b4003a20 con 0x7f61bc0d1f20 2023-11-28T11:21:22.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.509+0000 7f61b959a700 1 -- 172.21.15.92:0/830976310 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b4003dd0 con 0x7f61bc0d1f20 2023-11-28T11:21:22.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.509+0000 7ffabe7fc700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffac00cdee0 0x7ffac0144ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:22.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.509+0000 7ffabe7fc700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffac00cdee0 0x7ffac0144ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.92:57202/0 (socket says 172.21.15.92:57202) 2023-11-28T11:21:22.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.509+0000 7ffabf7fe700 1 -- 172.21.15.92:0/3540546219 learned_addr learned my addr 172.21.15.92:0/3540546219 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:22.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.510+0000 7ffac53ff700 1 -- 172.21.15.92:0/3540546219 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac00c79a0 con 0x7ffac015bea0 2023-11-28T11:21:22.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.510+0000 7ffac53ff700 1 -- 172.21.15.92:0/3540546219 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac016a2e0 con 0x7ffac00cdee0 2023-11-28T11:21:22.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.510+0000 7ffac53ff700 1 -- 172.21.15.92:0/3540546219 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffac00c7e30 con 0x7ffac00cbf70 2023-11-28T11:21:22.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.510+0000 7f61c0fff700 1 -- 172.21.15.92:0/830976310 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61bc0d1f20 msgr2=0x7f61bc0d2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:22.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.510+0000 7f61c0fff700 1 --2- 172.21.15.92:0/830976310 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61bc0d1f20 0x7f61bc0d2370 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f61b4009e80 tx=0x7f61b400b330 comp rx=0 tx=0).stop 2023-11-28T11:21:22.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.511+0000 7f61c0fff700 1 -- 172.21.15.92:0/830976310 shutdown_connections 2023-11-28T11:21:22.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.511+0000 7f61c0fff700 1 --2- 172.21.15.92:0/830976310 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61bc0d1f20 0x7f61bc0d2370 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:22.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.511+0000 7f61c0fff700 1 -- 172.21.15.92:0/830976310 >> 172.21.15.92:0/830976310 conn(0x7f61bc0719a0 msgr2=0x7f61bc071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:22.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.511+0000 7f61c0fff700 1 -- 172.21.15.92:0/830976310 shutdown_connections 2023-11-28T11:21:22.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.513+0000 7ffabe7fc700 1 -- 172.21.15.92:0/3540546219 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffac00cbf70 msgr2=0x7ffac0169da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:22.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.513+0000 7ffabe7fc700 1 --2- 172.21.15.92:0/3540546219 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffac00cbf70 0x7ffac0169da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:22.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.513+0000 7ffabe7fc700 1 -- 172.21.15.92:0/3540546219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffac015bea0 msgr2=0x7ffac0145000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:22.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.513+0000 7ffabe7fc700 1 --2- 172.21.15.92:0/3540546219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffac015bea0 0x7ffac0145000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:22.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.513+0000 7ffabe7fc700 1 -- 172.21.15.92:0/3540546219 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa8009550 con 0x7ffac00cdee0 2023-11-28T11:21:22.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.513+0000 7ffabf7fe700 1 --2- 172.21.15.92:0/3540546219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffac015bea0 0x7ffac0145000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:21:22.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.511+0000 7f61c0fff700 1 -- 172.21.15.92:0/830976310 wait complete. 2023-11-28T11:21:22.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.514+0000 7ffabe7fc700 1 --2- 172.21.15.92:0/3540546219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffac00cdee0 0x7ffac0144ac0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ffab000bd50 tx=0x7ffab000bd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:22.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.515+0000 7ffab77fe700 1 -- 172.21.15.92:0/3540546219 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab000cbb0 con 0x7ffac00cdee0 2023-11-28T11:21:22.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.515+0000 7ffab77fe700 1 -- 172.21.15.92:0/3540546219 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffab000cd10 con 0x7ffac00cdee0 2023-11-28T11:21:22.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.515+0000 7f61c0fff700 1 Processor -- start 2023-11-28T11:21:22.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.515+0000 7ffac53ff700 1 -- 172.21.15.92:0/3540546219 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffac00ce360 con 0x7ffac00cdee0 2023-11-28T11:21:22.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.515+0000 7ffac53ff700 1 -- 172.21.15.92:0/3540546219 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffac00cd9c0 con 0x7ffac00cdee0 2023-11-28T11:21:22.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.516+0000 7ffab77fe700 1 -- 172.21.15.92:0/3540546219 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab0014680 con 0x7ffac00cdee0 2023-11-28T11:21:22.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.516+0000 7f61c0fff700 1 -- start start 2023-11-28T11:21:22.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.517+0000 7f61c0fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61bc0cf8b0 0x7f61bc0cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:22.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.517+0000 7f61c0fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61bc0d1f20 0x7f61bc0cc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:22.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.517+0000 7f61c0fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61bc160fd0 0x7f61bc0b5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:22.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.517+0000 7f61c0fff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61bc2023b0 con 0x7f61bc0d1f20 2023-11-28T11:21:22.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.517+0000 7f61c0fff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61bc0c80d0 con 0x7f61bc0cf8b0 2023-11-28T11:21:22.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.517+0000 7f61c0fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61bc0b6ba0 con 0x7f61bc160fd0 2023-11-28T11:21:22.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.519+0000 7f61ba59c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61bc0cf8b0 0x7f61bc0cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:22.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.519+0000 7f61ba59c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61bc0cf8b0 0x7f61bc0cc0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.92:57210/0 (socket says 172.21.15.92:57210) 2023-11-28T11:21:22.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.519+0000 7f61b9d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61bc0d1f20 0x7f61bc0cc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:22.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.519+0000 7f61bad9d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61bc160fd0 0x7f61bc0b5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:22.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.519+0000 7f61bad9d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61bc160fd0 0x7f61bc0b5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.92:34204/0 (socket says 172.21.15.92:34204) 2023-11-28T11:21:22.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.519+0000 7f61bad9d700 1 -- 172.21.15.92:0/3647252921 learned_addr learned my addr 172.21.15.92:0/3647252921 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:22.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.518+0000 7ffac53ff700 1 -- 172.21.15.92:0/3540546219 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaa40052f0 con 0x7ffac00cdee0 2023-11-28T11:21:22.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.520+0000 7f61bad9d700 1 -- 172.21.15.92:0/3647252921 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61bc0cf8b0 msgr2=0x7f61bc0cc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:22.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.520+0000 7f61bad9d700 1 --2- 172.21.15.92:0/3647252921 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61bc0cf8b0 0x7f61bc0cc0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:22.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.520+0000 7f61bad9d700 1 -- 172.21.15.92:0/3647252921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61bc0d1f20 msgr2=0x7f61bc0cc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:22.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.520+0000 7f61bad9d700 1 --2- 172.21.15.92:0/3647252921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61bc0d1f20 0x7f61bc0cc5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:22.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.520+0000 7f61bad9d700 1 -- 172.21.15.92:0/3647252921 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b4009b30 con 0x7f61bc160fd0 2023-11-28T11:21:22.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.521+0000 7f61b9d9b700 1 --2- 172.21.15.92:0/3647252921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61bc0d1f20 0x7f61bc0cc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:21:22.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.521+0000 7f61bad9d700 1 --2- 172.21.15.92:0/3647252921 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61bc160fd0 0x7f61bc0b5ee0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f61ac00b440 tx=0x7f61ac00b750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:22.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.522+0000 7f61aaffd700 1 -- 172.21.15.92:0/3647252921 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ac011830 con 0x7f61bc160fd0 2023-11-28T11:21:22.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.522+0000 7f61aaffd700 1 -- 172.21.15.92:0/3647252921 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61ac011990 con 0x7f61bc160fd0 2023-11-28T11:21:22.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.522+0000 7f61aaffd700 1 -- 172.21.15.92:0/3647252921 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ac011d40 con 0x7f61bc160fd0 2023-11-28T11:21:22.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.524+0000 7ffab77fe700 1 -- 172.21.15.92:0/3540546219 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ffab001b030 con 0x7ffac00cdee0 2023-11-28T11:21:22.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.524+0000 7f61c0fff700 1 -- 172.21.15.92:0/3647252921 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61bc0b5270 con 0x7f61bc160fd0 2023-11-28T11:21:22.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.525+0000 7f61c0fff700 1 -- 172.21.15.92:0/3647252921 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61bc06bdd0 con 0x7f61bc160fd0 2023-11-28T11:21:22.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.528+0000 7f61c0fff700 1 -- 172.21.15.92:0/3647252921 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61bc0b5400 con 0x7f61bc160fd0 2023-11-28T11:21:22.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.529+0000 7ffab77fe700 1 --2- 172.21.15.92:0/3540546219 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ffaa00683c0 0x7ffaa006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:22.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.529+0000 7ffabeffd700 1 --2- 172.21.15.92:0/3540546219 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ffaa00683c0 0x7ffaa006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:22.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.530+0000 7ffab77fe700 1 -- 172.21.15.92:0/3540546219 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ffab0085d20 con 0x7ffac00cdee0 2023-11-28T11:21:22.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.530+0000 7ffab77fe700 1 -- 172.21.15.92:0/3540546219 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ffab008a310 con 0x7ffac00cdee0 2023-11-28T11:21:22.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.532+0000 7ffabeffd700 1 --2- 172.21.15.92:0/3540546219 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ffaa00683c0 0x7ffaa006a870 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ffac00c8a50 tx=0x7ffab8009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:22.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.532+0000 7f61aaffd700 1 -- 172.21.15.92:0/3647252921 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f61ac00f050 con 0x7f61bc160fd0 2023-11-28T11:21:22.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.534+0000 7f61aaffd700 1 --2- 172.21.15.92:0/3647252921 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f619c0683c0 0x7f619c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:22.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.534+0000 7f61aaffd700 1 -- 172.21.15.92:0/3647252921 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f61ac0859d0 con 0x7f61bc160fd0 2023-11-28T11:21:22.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.534+0000 7f61ba59c700 1 --2- 172.21.15.92:0/3647252921 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f619c0683c0 0x7f619c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:22.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.536+0000 7f61ba59c700 1 --2- 172.21.15.92:0/3647252921 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f619c0683c0 0x7f619c06a870 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f61b0000f60 tx=0x7f61b0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:22.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.539+0000 7f61aaffd700 1 -- 172.21.15.92:0/3647252921 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f61ac051ed0 con 0x7f61bc160fd0 2023-11-28T11:21:22.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.582+0000 7f84329ff700 1 Processor -- start 2023-11-28T11:21:22.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.582+0000 7f84329ff700 1 -- start start 2023-11-28T11:21:22.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.584+0000 7f631959d700 1 Processor -- start 2023-11-28T11:21:22.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.584+0000 7f84329ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f842c06a270 0x7f842c0c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:22.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.585+0000 7f84329ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f842c15ce70 0x7f842c06a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:22.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.585+0000 7f84329ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f842c06ab80 con 0x7f842c0cdee0 2023-11-28T11:21:22.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.585+0000 7f84329ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f842c0ccb60 con 0x7f842c15ce70 2023-11-28T11:21:22.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.585+0000 7f84329ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f842c0cce40 con 0x7f842c06a270 2023-11-28T11:21:22.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.585+0000 7f842bfff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f842c06a270 0x7f842c0c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:22.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.585+0000 7f842bfff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f842c06a270 0x7f842c0c9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.92:57216/0 (socket says 172.21.15.92:57216) 2023-11-28T11:21:22.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.585+0000 7f842bfff700 1 -- 172.21.15.92:0/4139269686 learned_addr learned my addr 172.21.15.92:0/4139269686 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:22.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.584+0000 7f631959d700 1 -- start start 2023-11-28T11:21:22.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.586+0000 7f631959d700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f630c051150 0x7f630c036610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:22.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.586+0000 7f631959d700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f630c04f8f0 con 0x7f630c036b50 2023-11-28T11:21:22.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.586+0000 7f631959d700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f630c036f50 con 0x7f630c0e6d00 2023-11-28T11:21:22.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.586+0000 7f631959d700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f630c048550 con 0x7f630c051150 2023-11-28T11:21:22.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.587+0000 7f6313fff700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f630c0e6d00 0x7f630c04f1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.92:60828/0 (socket says 172.21.15.92:60828) 2023-11-28T11:21:22.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.587+0000 7f6313fff700 1 -- 172.21.15.92:0/3751699527 learned_addr learned my addr 172.21.15.92:0/3751699527 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:22.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.587+0000 7f63137fe700 1 --2- 172.21.15.92:0/3751699527 >> v2:172.21.15.169:3300/0 conn(0x7f630c051150 0x7f630c036610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:22.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.587+0000 7f63137fe700 1 -- 172.21.15.92:0/3751699527 >> v1:172.21.15.169:6789/0 conn(0x7f630c0e6d00 legacy=0x7f630c04f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:22.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.587+0000 7f63137fe700 1 -- 172.21.15.92:0/3751699527 >> v1:172.21.15.92:6789/0 conn(0x7f630c036b50 legacy=0x7f630c052c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:22.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.587+0000 7f63137fe700 1 -- 172.21.15.92:0/3751699527 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f630c033920 con 0x7f630c051150 2023-11-28T11:21:22.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.586+0000 7f842b7fe700 1 --2- 172.21.15.92:0/4139269686 >> v2:172.21.15.92:3300/0 conn(0x7f842c15ce70 0x7f842c06a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:22.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.587+0000 7f842bfff700 1 -- 172.21.15.92:0/4139269686 >> v2:172.21.15.92:3300/0 conn(0x7f842c15ce70 msgr2=0x7f842c06a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:22.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.587+0000 7f842bfff700 1 --2- 172.21.15.92:0/4139269686 >> v2:172.21.15.92:3300/0 conn(0x7f842c15ce70 0x7f842c06a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:22.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.587+0000 7f842bfff700 1 -- 172.21.15.92:0/4139269686 >> v1:172.21.15.92:6789/0 conn(0x7f842c0cdee0 legacy=0x7f842c06c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:22.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.587+0000 7f842bfff700 1 -- 172.21.15.92:0/4139269686 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f842c0cd120 con 0x7f842c06a270 2023-11-28T11:21:22.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.587+0000 7f842b7fe700 1 --2- 172.21.15.92:0/4139269686 >> v2:172.21.15.92:3300/0 conn(0x7f842c15ce70 0x7f842c06a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:21:22.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.588+0000 7f842bfff700 1 --2- 172.21.15.92:0/4139269686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f842c06a270 0x7f842c0c9ed0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f842c06b6d0 tx=0x7f841c009a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad6867020ce68ba4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:22.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.588+0000 7f63137fe700 1 --2- 172.21.15.92:0/3751699527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f630c051150 0x7f630c036610 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6304009840 tx=0x7f6304009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e89659802b57d18 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:22.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.589+0000 7f6312ffd700 1 -- 172.21.15.92:0/3751699527 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6304004230 con 0x7f630c051150 2023-11-28T11:21:22.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.589+0000 7f842affd700 1 -- 172.21.15.92:0/4139269686 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841c013c90 con 0x7f842c06a270 2023-11-28T11:21:22.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.589+0000 7f6312ffd700 1 -- 172.21.15.92:0/3751699527 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6304004390 con 0x7f630c051150 2023-11-28T11:21:22.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.589+0000 7f842affd700 1 -- 172.21.15.92:0/4139269686 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f841c013df0 con 0x7f842c06a270 2023-11-28T11:21:22.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.589+0000 7f6312ffd700 1 -- 172.21.15.92:0/3751699527 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63040036a0 con 0x7f630c051150 2023-11-28T11:21:22.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.589+0000 7f842affd700 1 -- 172.21.15.92:0/4139269686 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841c013c90 con 0x7f842c06a270 2023-11-28T11:21:22.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.590+0000 7f84329ff700 1 -- 172.21.15.92:0/4139269686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f842c06a270 msgr2=0x7f842c0c9ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:22.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.591+0000 7f84329ff700 1 --2- 172.21.15.92:0/4139269686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f842c06a270 0x7f842c0c9ed0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f842c06b6d0 tx=0x7f841c009a40 comp rx=0 tx=0).stop 2023-11-28T11:21:22.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.591+0000 7f631959d700 1 -- 172.21.15.92:0/3751699527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f630c051150 msgr2=0x7f630c036610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:22.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.591+0000 7f631959d700 1 --2- 172.21.15.92:0/3751699527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f630c051150 0x7f630c036610 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6304009840 tx=0x7f6304009b50 comp rx=0 tx=0).stop 2023-11-28T11:21:22.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.591+0000 7f84329ff700 1 -- 172.21.15.92:0/4139269686 shutdown_connections 2023-11-28T11:21:22.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.591+0000 7f84329ff700 1 --2- 172.21.15.92:0/4139269686 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f842c06a270 0x7f842c0c9ed0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:22.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.591+0000 7f84329ff700 1 --2- 172.21.15.92:0/4139269686 >> v2:172.21.15.92:3300/0 conn(0x7f842c15ce70 0x7f842c06a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:22.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.591+0000 7f84329ff700 1 -- 172.21.15.92:0/4139269686 >> 172.21.15.92:0/4139269686 conn(0x7f842c0cb490 msgr2=0x7f842c0cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:22.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.591+0000 7f631959d700 1 -- 172.21.15.92:0/3751699527 shutdown_connections 2023-11-28T11:21:22.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.591+0000 7f631959d700 1 --2- 172.21.15.92:0/3751699527 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f630c051150 0x7f630c036610 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:22.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.592+0000 7f631959d700 1 -- 172.21.15.92:0/3751699527 >> 172.21.15.92:0/3751699527 conn(0x7f630c051610 msgr2=0x7f630c0519e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:22.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.592+0000 7f631959d700 1 -- 172.21.15.92:0/3751699527 shutdown_connections 2023-11-28T11:21:22.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.592+0000 7f84329ff700 1 -- 172.21.15.92:0/4139269686 shutdown_connections 2023-11-28T11:21:22.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.592+0000 7f84329ff700 1 -- 172.21.15.92:0/4139269686 wait complete. 2023-11-28T11:21:22.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.593+0000 7f631959d700 1 -- 172.21.15.92:0/3751699527 wait complete. 2023-11-28T11:21:22.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.595+0000 7f631959d700 1 Processor -- start 2023-11-28T11:21:22.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.596+0000 7f631959d700 1 -- start start 2023-11-28T11:21:22.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.596+0000 7f84329ff700 1 Processor -- start 2023-11-28T11:21:22.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.596+0000 7f84329ff700 1 -- start start 2023-11-28T11:21:22.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.596+0000 7fa70359e700 1 Processor -- start 2023-11-28T11:21:22.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.597+0000 7fa70359e700 1 -- start start 2023-11-28T11:21:22.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.597+0000 7f631959d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f630c036b50 0x7f630c0e8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:22.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.597+0000 7f631959d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f630c051150 0x7f630c0dbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:22.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.597+0000 7f631959d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f630c0e6d00 0x7f630c0e9650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:22.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.597+0000 7f631959d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f630c0e0440 con 0x7f630c051150 2023-11-28T11:21:22.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.597+0000 7f631959d700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f630c0e7ce0 con 0x7f630c0e6d00 2023-11-28T11:21:22.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.597+0000 7f631959d700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f630c0e70d0 con 0x7f630c036b50 2023-11-28T11:21:22.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.598+0000 7f84329ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f842c06a270 0x7f842c0ae5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:22.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.598+0000 7fa70359e700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa6fc038410 0x7fa6fc006d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:22.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.598+0000 7fa70359e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa6fc0558e0 0x7fa6fc03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:22.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.599+0000 7fa70359e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6fc03b1f0 0x7fa6fc0e6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:22.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.599+0000 7fa70359e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6fc03c080 con 0x7fa6fc0558e0 2023-11-28T11:21:22.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.599+0000 7fa70359e700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6fc03b690 con 0x7fa6fc038410 2023-11-28T11:21:22.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.599+0000 7fa70359e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc007a10 con 0x7fa6fc03b1f0 2023-11-28T11:21:22.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.599+0000 7f84329ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f842c0cdee0 0x7f842c0c2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:22.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.599+0000 7f842bfff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f842c06a270 0x7f842c0ae5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:22.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.599+0000 7f842bfff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f842c06a270 0x7f842c0ae5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.92:57254/0 (socket says 172.21.15.92:57254) 2023-11-28T11:21:22.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.599+0000 7f842bfff700 1 -- 172.21.15.92:0/3071987121 learned_addr learned my addr 172.21.15.92:0/3071987121 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:22.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.599+0000 7f84329ff700 1 --2- 172.21.15.92:0/3071987121 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f842c15ce70 0x7f842c0b1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:22.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.599+0000 7fa702d9d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6fc03b1f0 0x7fa6fc0e6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:22.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.600+0000 7fa70259c700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa6fc038410 0x7fa6fc006d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:22.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.600+0000 7fa702d9d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6fc03b1f0 0x7fa6fc0e6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.92:34212/0 (socket says 172.21.15.92:34212) 2023-11-28T11:21:22.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.600+0000 7fa702d9d700 1 -- 172.21.15.92:0/621217146 learned_addr learned my addr 172.21.15.92:0/621217146 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:22.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.600+0000 7fa701d9b700 1 --2- 172.21.15.92:0/621217146 >> v2:172.21.15.92:3300/0 conn(0x7fa6fc0558e0 0x7fa6fc03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:22.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.600+0000 7fa702d9d700 1 -- 172.21.15.92:0/621217146 >> v2:172.21.15.169:3300/0 conn(0x7fa6fc038410 msgr2=0x7fa6fc006d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:22.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.599+0000 7f6318d9c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f630c0e6d00 0x7f630c0e9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:22.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.600+0000 7f63137fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f630c051150 0x7f630c0dbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:22.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.601+0000 7f6318d9c700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f630c0e6d00 0x7f630c0e9650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.92:57224/0 (socket says 172.21.15.92:57224) 2023-11-28T11:21:22.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.601+0000 7f6318d9c700 1 -- 172.21.15.92:0/3984371319 learned_addr learned my addr 172.21.15.92:0/3984371319 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:22.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.600+0000 7fa702d9d700 1 --2- 172.21.15.92:0/621217146 >> v2:172.21.15.169:3300/0 conn(0x7fa6fc038410 0x7fa6fc006d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:22.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.601+0000 7f6313fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f630c036b50 0x7f630c0e8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:22.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.600+0000 7f842b7fe700 1 --2- 172.21.15.92:0/3071987121 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f842c0cdee0 0x7f842c0c2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:22.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.600+0000 7f8430f9c700 1 --2- 172.21.15.92:0/3071987121 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f842c15ce70 0x7f842c0b1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:22.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.601+0000 7f84329ff700 1 -- 172.21.15.92:0/3071987121 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842c0c7120 con 0x7f842c15ce70 2023-11-28T11:21:22.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.601+0000 7f84329ff700 1 -- 172.21.15.92:0/3071987121 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842c0c7400 con 0x7f842c06a270 2023-11-28T11:21:22.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.601+0000 7f84329ff700 1 -- 172.21.15.92:0/3071987121 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f842c0c76e0 con 0x7f842c0cdee0 2023-11-28T11:21:22.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.600+0000 7fa702d9d700 1 -- 172.21.15.92:0/621217146 >> v2:172.21.15.92:3300/0 conn(0x7fa6fc0558e0 msgr2=0x7fa6fc03acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:22.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.601+0000 7f8430f9c700 1 -- 172.21.15.92:0/3071987121 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f842c0cdee0 msgr2=0x7f842c0c2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:22.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.602+0000 7f6318d9c700 1 -- 172.21.15.92:0/3984371319 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f630c036b50 msgr2=0x7f630c0e8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:22.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.602+0000 7f6318d9c700 1 --2- 172.21.15.92:0/3984371319 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f630c036b50 0x7f630c0e8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:22.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.602+0000 7f6318d9c700 1 -- 172.21.15.92:0/3984371319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f630c051150 msgr2=0x7f630c0dbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:22.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.602+0000 7f6318d9c700 1 --2- 172.21.15.92:0/3984371319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f630c051150 0x7f630c0dbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:22.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.602+0000 7fa702d9d700 1 --2- 172.21.15.92:0/621217146 >> v2:172.21.15.92:3300/0 conn(0x7fa6fc0558e0 0x7fa6fc03acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:22.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.602+0000 7fa702d9d700 1 -- 172.21.15.92:0/621217146 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6fc053880 con 0x7fa6fc03b1f0 2023-11-28T11:21:22.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.602+0000 7fa701d9b700 1 --2- 172.21.15.92:0/621217146 >> v2:172.21.15.92:3300/0 conn(0x7fa6fc0558e0 0x7fa6fc03acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:21:22.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.603+0000 7fa702d9d700 1 --2- 172.21.15.92:0/621217146 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6fc03b1f0 0x7fa6fc0e6560 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa7040683c0 tx=0x7fa70404edb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=651b46c5af1b358b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:22.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.602+0000 7f8430f9c700 1 --2- 172.21.15.92:0/3071987121 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f842c0cdee0 0x7f842c0c2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:22.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.602+0000 7f8430f9c700 1 -- 172.21.15.92:0/3071987121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f842c06a270 msgr2=0x7f842c0ae5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:22.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.603+0000 7f6318d9c700 1 -- 172.21.15.92:0/3984371319 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63040094a0 con 0x7f630c0e6d00 2023-11-28T11:21:22.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.603+0000 7f63137fe700 1 --2- 172.21.15.92:0/3984371319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f630c051150 0x7f630c0dbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:21:22.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.602+0000 7f8430f9c700 1 --2- 172.21.15.92:0/3071987121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f842c06a270 0x7f842c0ae5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:22.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.602+0000 7f8430f9c700 1 -- 172.21.15.92:0/3071987121 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f841c009550 con 0x7f842c15ce70 2023-11-28T11:21:22.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.603+0000 7fa70159a700 1 -- 172.21.15.92:0/621217146 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa704067430 con 0x7fa6fc03b1f0 2023-11-28T11:21:22.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.602+0000 7f842b7fe700 1 --2- 172.21.15.92:0/3071987121 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f842c0cdee0 0x7f842c0c2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:21:22.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.603+0000 7f8430f9c700 1 --2- 172.21.15.92:0/3071987121 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f842c15ce70 0x7f842c0b1840 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f842400cb90 tx=0x7f842400cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:22.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.604+0000 7f8428ff9700 1 -- 172.21.15.92:0/3071987121 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8424003c00 con 0x7f842c15ce70 2023-11-28T11:21:22.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.604+0000 7fa70159a700 1 -- 172.21.15.92:0/621217146 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa704067590 con 0x7fa6fc03b1f0 2023-11-28T11:21:22.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.604+0000 7f6318d9c700 1 --2- 172.21.15.92:0/3984371319 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f630c0e6d00 0x7f630c0e9650 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f63140683c0 tx=0x7f63140677b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:22.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.604+0000 7f84329ff700 1 -- 172.21.15.92:0/3071987121 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f842c0cd6e0 con 0x7f842c15ce70 2023-11-28T11:21:22.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.604+0000 7f8428ff9700 1 -- 172.21.15.92:0/3071987121 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8424003d60 con 0x7f842c15ce70 2023-11-28T11:21:22.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.604+0000 7f6310ff9700 1 -- 172.21.15.92:0/3984371319 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6314072150 con 0x7f630c0e6d00 2023-11-28T11:21:22.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.604+0000 7f84329ff700 1 -- 172.21.15.92:0/3071987121 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f842c0cd400 con 0x7f842c15ce70 2023-11-28T11:21:22.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.604+0000 7f8428ff9700 1 -- 172.21.15.92:0/3071987121 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842400ab50 con 0x7f842c15ce70 2023-11-28T11:21:22.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.604+0000 7f6310ff9700 1 -- 172.21.15.92:0/3984371319 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63140722b0 con 0x7f630c0e6d00 2023-11-28T11:21:22.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.604+0000 7f6310ff9700 1 -- 172.21.15.92:0/3984371319 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6314072620 con 0x7f630c0e6d00 2023-11-28T11:21:22.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.605+0000 7fa70159a700 1 -- 172.21.15.92:0/621217146 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa704072ef0 con 0x7fa6fc03b1f0 2023-11-28T11:21:22.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.606+0000 7f631959d700 1 -- 172.21.15.92:0/3984371319 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f630c033ee0 con 0x7f630c0e6d00 2023-11-28T11:21:22.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.605+0000 7f7e2d5ff700 1 Processor -- start 2023-11-28T11:21:22.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.605+0000 7f7e2d5ff700 1 -- start start 2023-11-28T11:21:22.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.606+0000 7f631959d700 1 -- 172.21.15.92:0/3984371319 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f630c033c00 con 0x7f630c0e6d00 2023-11-28T11:21:22.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.606+0000 7fa70359e700 1 -- 172.21.15.92:0/621217146 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6fc03b1f0 msgr2=0x7fa6fc0e6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:22.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.606+0000 7fa70359e700 1 --2- 172.21.15.92:0/621217146 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6fc03b1f0 0x7fa6fc0e6560 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa7040683c0 tx=0x7fa70404edb0 comp rx=0 tx=0).stop 2023-11-28T11:21:22.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.606+0000 7fa70359e700 1 -- 172.21.15.92:0/621217146 shutdown_connections 2023-11-28T11:21:22.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.606+0000 7fa70359e700 1 --2- 172.21.15.92:0/621217146 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6fc03b1f0 0x7fa6fc0e6560 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:22.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.606+0000 7fa70359e700 1 --2- 172.21.15.92:0/621217146 >> v2:172.21.15.169:3300/0 conn(0x7fa6fc038410 0x7fa6fc006d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:22.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.606+0000 7fa70359e700 1 --2- 172.21.15.92:0/621217146 >> v2:172.21.15.92:3300/0 conn(0x7fa6fc0558e0 0x7fa6fc03acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:22.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.606+0000 7fa70359e700 1 -- 172.21.15.92:0/621217146 >> 172.21.15.92:0/621217146 conn(0x7fa6fc055cb0 msgr2=0x7fa6fc056080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:22.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.606+0000 7f3e58bff700 1 Processor -- start 2023-11-28T11:21:22.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.606+0000 7f3e58bff700 1 -- start start 2023-11-28T11:21:22.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.608+0000 7fa70359e700 1 -- 172.21.15.92:0/621217146 shutdown_connections 2023-11-28T11:21:22.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.608+0000 7fa70359e700 1 -- 172.21.15.92:0/621217146 wait complete. 2023-11-28T11:21:22.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.608+0000 7f84329ff700 1 -- 172.21.15.92:0/3071987121 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f840c0052f0 con 0x7f842c15ce70 2023-11-28T11:21:22.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.609+0000 7f8428ff9700 1 -- 172.21.15.92:0/3071987121 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8424010050 con 0x7f842c15ce70 2023-11-28T11:21:22.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.610+0000 7f8428ff9700 1 --2- 172.21.15.92:0/3071987121 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f84180682f0 0x7f841806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:22.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.610+0000 7f6310ff9700 1 -- 172.21.15.92:0/3984371319 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6314072780 con 0x7f630c0e6d00 2023-11-28T11:21:22.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.610+0000 7f631959d700 1 -- 172.21.15.92:0/3984371319 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f630c0341c0 con 0x7f630c0e6d00 2023-11-28T11:21:22.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.611+0000 7f6310ff9700 1 --2- 172.21.15.92:0/3984371319 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6300068340 0x7f630006a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:22.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.611+0000 7f842bfff700 1 --2- 172.21.15.92:0/3071987121 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f84180682f0 0x7f841806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:22.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.611+0000 7fa70359e700 1 Processor -- start 2023-11-28T11:21:22.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.611+0000 7f8428ff9700 1 -- 172.21.15.92:0/3071987121 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f842405d8b0 con 0x7f842c15ce70 2023-11-28T11:21:22.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.612+0000 7f842bfff700 1 --2- 172.21.15.92:0/3071987121 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f84180682f0 0x7f841806a7a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f841c009300 tx=0x7f841c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:22.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.607+0000 7f7e2d5ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e28160b40 0x7f7e280cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:22.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.607+0000 7f7e2d5ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7e28164cb0 0x7f7e280d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:22.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.607+0000 7f7e2d5ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e280cfe40 con 0x7f7e280cf900 2023-11-28T11:21:22.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.607+0000 7f7e2d5ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e280d0120 con 0x7f7e28164cb0 2023-11-28T11:21:22.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.607+0000 7f7e2d5ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e280d2110 con 0x7f7e28160b40 2023-11-28T11:21:22.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.608+0000 7f7e26ffd700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e28160b40 0x7f7e280cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:22.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.608+0000 7f7e26ffd700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e28160b40 0x7f7e280cf3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.92:34234/0 (socket says 172.21.15.92:34234) 2023-11-28T11:21:22.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.608+0000 7f7e26ffd700 1 -- 172.21.15.92:0/3903966575 learned_addr learned my addr 172.21.15.92:0/3903966575 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:22.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.608+0000 7f7e277fe700 1 --2- 172.21.15.92:0/3903966575 >> v2:172.21.15.92:3300/0 conn(0x7f7e28164cb0 0x7f7e280d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:22.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.608+0000 7f7e26ffd700 1 -- 172.21.15.92:0/3903966575 >> v2:172.21.15.92:3300/0 conn(0x7f7e28164cb0 msgr2=0x7f7e280d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:22.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.608+0000 7f7e26ffd700 1 --2- 172.21.15.92:0/3903966575 >> v2:172.21.15.92:3300/0 conn(0x7f7e28164cb0 0x7f7e280d2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:22.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.608+0000 7f7e26ffd700 1 -- 172.21.15.92:0/3903966575 >> v1:172.21.15.169:6789/0 conn(0x7f7e280cf900 legacy=0x7f7e281645a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:22.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.609+0000 7f7e26ffd700 1 -- 172.21.15.92:0/3903966575 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e280d23f0 con 0x7f7e28160b40 2023-11-28T11:21:22.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.611+0000 7f7e26ffd700 1 --2- 172.21.15.92:0/3903966575 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e28160b40 0x7f7e280cf3c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7e20009e50 tx=0x7f7e2000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c02cf3da0dca60a1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:22.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.611+0000 7f7e25ffb700 1 -- 172.21.15.92:0/3903966575 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2000e070 con 0x7f7e28160b40 2023-11-28T11:21:22.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.611+0000 7f7e25ffb700 1 -- 172.21.15.92:0/3903966575 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e20003a20 con 0x7f7e28160b40 2023-11-28T11:21:22.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.611+0000 7f7e25ffb700 1 -- 172.21.15.92:0/3903966575 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e20003dd0 con 0x7f7e28160b40 2023-11-28T11:21:22.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.612+0000 7f6310ff9700 1 -- 172.21.15.92:0/3984371319 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f63140d6c00 con 0x7f630c0e6d00 2023-11-28T11:21:22.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.612+0000 7f7e2d5ff700 1 -- 172.21.15.92:0/3903966575 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e28160b40 msgr2=0x7f7e280cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:22.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.612+0000 7f7e2d5ff700 1 --2- 172.21.15.92:0/3903966575 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e28160b40 0x7f7e280cf3c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7e20009e50 tx=0x7f7e2000b330 comp rx=0 tx=0).stop 2023-11-28T11:21:22.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.612+0000 7f3e58bff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3e541645a0 0x7f3e54164a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:22.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.612+0000 7f7e2d5ff700 1 -- 172.21.15.92:0/3903966575 shutdown_connections 2023-11-28T11:21:22.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.612+0000 7f7e2d5ff700 1 --2- 172.21.15.92:0/3903966575 >> v2:172.21.15.92:3300/0 conn(0x7f7e28164cb0 0x7f7e280d2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:22.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.612+0000 7f7e2d5ff700 1 --2- 172.21.15.92:0/3903966575 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e28160b40 0x7f7e280cf3c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:22.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.613+0000 7f7e2d5ff700 1 -- 172.21.15.92:0/3903966575 >> 172.21.15.92:0/3903966575 conn(0x7f7e280719a0 msgr2=0x7f7e28071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:22.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.612+0000 7f3e58bff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3e540d2bf0 0x7f3e540d3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:22.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.613+0000 7f7e2d5ff700 1 -- 172.21.15.92:0/3903966575 shutdown_connections 2023-11-28T11:21:22.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.613+0000 7f7e2d5ff700 1 -- 172.21.15.92:0/3903966575 wait complete. 2023-11-28T11:21:22.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.612+0000 7f3e58bff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e540cfe40 con 0x7f3e54160b40 2023-11-28T11:21:22.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.612+0000 7f3e58bff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e54164f60 con 0x7f3e541645a0 2023-11-28T11:21:22.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.613+0000 7fa70359e700 1 -- start start 2023-11-28T11:21:22.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.612+0000 7f3e58bff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e540d0120 con 0x7f3e540d2bf0 2023-11-28T11:21:22.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.613+0000 7f3e52d9d700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3e540d2bf0 0x7f3e540d3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:22.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.613+0000 7f3e52d9d700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3e540d2bf0 0x7f3e540d3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.92:57264/0 (socket says 172.21.15.92:57264) 2023-11-28T11:21:22.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.613+0000 7f3e52d9d700 1 -- 172.21.15.92:0/2008621617 learned_addr learned my addr 172.21.15.92:0/2008621617 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:22.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.613+0000 7f3e52d9d700 1 -- 172.21.15.92:0/2008621617 >> v2:172.21.15.92:3300/0 conn(0x7f3e541645a0 msgr2=0x7f3e54164a20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T11:21:22.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.613+0000 7f3e52d9d700 1 --2- 172.21.15.92:0/2008621617 >> v2:172.21.15.92:3300/0 conn(0x7f3e541645a0 0x7f3e54164a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:22.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.613+0000 7f3e52d9d700 1 -- 172.21.15.92:0/2008621617 >> v1:172.21.15.169:6789/0 conn(0x7f3e54160b40 legacy=0x7f3e540cf3c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T11:21:22.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.613+0000 7f3e52d9d700 1 -- 172.21.15.92:0/2008621617 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e540d2110 con 0x7f3e540d2bf0 2023-11-28T11:21:22.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.614+0000 7f3e52d9d700 1 --2- 172.21.15.92:0/2008621617 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e540d2bf0 0x7f3e540d3010 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3e540cd440 tx=0x7f3e4c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a16e8f343d553e5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:22.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.614+0000 7f6313fff700 1 --2- 172.21.15.92:0/3984371319 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6300068340 0x7f630006a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:22.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.615+0000 7fa70359e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6fc038410 0x7fa6fc0e0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:22.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.615+0000 7fa70359e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6fc03b1f0 0x7fa6fc0e40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:22.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.615+0000 7fa70359e700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6fc0558e0 0x7fa6fc0e24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:22.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.615+0000 7fa70359e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc0e3010 con 0x7fa6fc03b1f0 2023-11-28T11:21:22.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.615+0000 7fa70359e700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc0e32f0 con 0x7fa6fc0558e0 2023-11-28T11:21:22.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.615+0000 7fa70359e700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc0e35d0 con 0x7fa6fc038410 2023-11-28T11:21:22.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.615+0000 7fa70259c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6fc038410 0x7fa6fc0e0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:22.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.615+0000 7fa70259c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6fc038410 0x7fa6fc0e0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.92:34236/0 (socket says 172.21.15.92:34236) 2023-11-28T11:21:22.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.615+0000 7fa70259c700 1 -- 172.21.15.92:0/2529595056 learned_addr learned my addr 172.21.15.92:0/2529595056 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:22.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.615+0000 7f7e2d5ff700 1 Processor -- start 2023-11-28T11:21:22.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.615+0000 7f6313fff700 1 --2- 172.21.15.92:0/3984371319 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6300068340 0x7f630006a7f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6308005910 tx=0x7f6308009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:22.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.616+0000 7fa702d9d700 1 --2- 172.21.15.92:0/2529595056 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6fc0558e0 0x7fa6fc0e24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:22.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.616+0000 7fa701d9b700 1 --2- 172.21.15.92:0/2529595056 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6fc03b1f0 0x7fa6fc0e40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:22.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.616+0000 7f7e2d5ff700 1 -- start start 2023-11-28T11:21:22.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.616+0000 7fa70259c700 1 -- 172.21.15.92:0/2529595056 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6fc0558e0 msgr2=0x7fa6fc0e24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:22.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.616+0000 7fa70259c700 1 --2- 172.21.15.92:0/2529595056 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6fc0558e0 0x7fa6fc0e24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:22.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.616+0000 7fa70259c700 1 -- 172.21.15.92:0/2529595056 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6fc03b1f0 msgr2=0x7fa6fc0e40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:22.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.616+0000 7fa70259c700 1 --2- 172.21.15.92:0/2529595056 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6fc03b1f0 0x7fa6fc0e40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:22.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.616+0000 7fa70259c700 1 -- 172.21.15.92:0/2529595056 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6fc054090 con 0x7fa6fc038410 2023-11-28T11:21:22.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.616+0000 7fa702d9d700 1 --2- 172.21.15.92:0/2529595056 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6fc0558e0 0x7fa6fc0e24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:21:22.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.616+0000 7fa70259c700 1 --2- 172.21.15.92:0/2529595056 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6fc038410 0x7fa6fc0e0510 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa6f800e930 tx=0x7fa6f800ec40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:22.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.617+0000 7fa6f2ffd700 1 -- 172.21.15.92:0/2529595056 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f800ee80 con 0x7fa6fc038410 2023-11-28T11:21:22.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.617+0000 7f6310ff9700 1 -- 172.21.15.92:0/3984371319 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f63140cb6c0 con 0x7f630c0e6d00 2023-11-28T11:21:22.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.617+0000 7fa6f2ffd700 1 -- 172.21.15.92:0/2529595056 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6f800c850 con 0x7fa6fc038410 2023-11-28T11:21:22.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.617+0000 7fa6f2ffd700 1 -- 172.21.15.92:0/2529595056 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f800cbe0 con 0x7fa6fc038410 2023-11-28T11:21:22.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.617+0000 7fa70359e700 1 -- 172.21.15.92:0/2529595056 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6fc007730 con 0x7fa6fc038410 2023-11-28T11:21:22.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.617+0000 7fa70359e700 1 -- 172.21.15.92:0/2529595056 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6fc0072a0 con 0x7fa6fc038410 2023-11-28T11:21:22.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.618+0000 7f3e51d9b700 1 -- 172.21.15.92:0/2008621617 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c00e070 con 0x7f3e540d2bf0 2023-11-28T11:21:22.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.618+0000 7f3e51d9b700 1 -- 172.21.15.92:0/2008621617 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e4c003a20 con 0x7f3e540d2bf0 2023-11-28T11:21:22.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.618+0000 7f3e51d9b700 1 -- 172.21.15.92:0/2008621617 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c003dd0 con 0x7f3e540d2bf0 2023-11-28T11:21:22.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.619+0000 7fa70359e700 1 -- 172.21.15.92:0/2529595056 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6fc008220 con 0x7fa6fc038410 2023-11-28T11:21:22.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.619+0000 7f3e58bff700 1 -- 172.21.15.92:0/2008621617 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e540d2bf0 msgr2=0x7f3e540d3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:22.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.619+0000 7f3e58bff700 1 --2- 172.21.15.92:0/2008621617 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e540d2bf0 0x7f3e540d3010 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3e540cd440 tx=0x7f3e4c00b330 comp rx=0 tx=0).stop 2023-11-28T11:21:22.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.619+0000 7f3e58bff700 1 -- 172.21.15.92:0/2008621617 shutdown_connections 2023-11-28T11:21:22.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.619+0000 7f3e58bff700 1 --2- 172.21.15.92:0/2008621617 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e540d2bf0 0x7f3e540d3010 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:22.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.620+0000 7f3e58bff700 1 --2- 172.21.15.92:0/2008621617 >> v2:172.21.15.92:3300/0 conn(0x7f3e541645a0 0x7f3e54164a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:22.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.620+0000 7f3e58bff700 1 -- 172.21.15.92:0/2008621617 >> 172.21.15.92:0/2008621617 conn(0x7f3e540719a0 msgr2=0x7f3e54071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:22.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.620+0000 7f3e58bff700 1 -- 172.21.15.92:0/2008621617 shutdown_connections 2023-11-28T11:21:22.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.619+0000 7f7e2d5ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e280cf900 0x7f7e2815b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:22.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.620+0000 7f7e2d5ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e28160b40 0x7f7e281ea3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:22.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.621+0000 7f7e2d5ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e28164cb0 0x7f7e280cba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:22.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.621+0000 7f7e2d5ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e281ead50 con 0x7f7e280cf900 2023-11-28T11:21:22.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.621+0000 7f7e2d5ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e281eb030 con 0x7f7e28160b40 2023-11-28T11:21:22.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.621+0000 7f7e2d5ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e281eb310 con 0x7f7e28164cb0 2023-11-28T11:21:22.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.621+0000 7f7e267fc700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e28160b40 0x7f7e281ea3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:22.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.621+0000 7f7e277fe700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e28164cb0 0x7f7e280cba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:22.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.621+0000 7f7e267fc700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e28160b40 0x7f7e281ea3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.92:57282/0 (socket says 172.21.15.92:57282) 2023-11-28T11:21:22.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.621+0000 7f7e277fe700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e28164cb0 0x7f7e280cba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.92:34252/0 (socket says 172.21.15.92:34252) 2023-11-28T11:21:22.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.621+0000 7f7e267fc700 1 -- 172.21.15.92:0/2642059246 learned_addr learned my addr 172.21.15.92:0/2642059246 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:22.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.620+0000 7f3e58bff700 1 -- 172.21.15.92:0/2008621617 wait complete. 2023-11-28T11:21:22.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.622+0000 7f7e26ffd700 1 --2- 172.21.15.92:0/2642059246 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e280cf900 0x7f7e2815b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:22.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.622+0000 7f7e267fc700 1 -- 172.21.15.92:0/2642059246 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e28164cb0 msgr2=0x7f7e280cba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:22.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.622+0000 7f7e267fc700 1 --2- 172.21.15.92:0/2642059246 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e28164cb0 0x7f7e280cba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:22.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.622+0000 7f7e267fc700 1 -- 172.21.15.92:0/2642059246 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e280cf900 msgr2=0x7f7e2815b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:22.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.622+0000 7f8428ff9700 1 -- 172.21.15.92:0/3071987121 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f842408a060 con 0x7f842c15ce70 2023-11-28T11:21:22.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.622+0000 7f7e267fc700 1 --2- 172.21.15.92:0/2642059246 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e280cf900 0x7f7e2815b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:22.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.622+0000 7f7e267fc700 1 -- 172.21.15.92:0/2642059246 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e20009b30 con 0x7f7e28160b40 2023-11-28T11:21:22.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.622+0000 7f7e277fe700 1 --2- 172.21.15.92:0/2642059246 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e28164cb0 0x7f7e280cba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:21:22.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.622+0000 7f7e26ffd700 1 --2- 172.21.15.92:0/2642059246 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e280cf900 0x7f7e2815b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:21:22.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.622+0000 7f7e267fc700 1 --2- 172.21.15.92:0/2642059246 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e28160b40 0x7f7e281ea3e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7e1800c4a0 tx=0x7f7e1800c7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:22.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.623+0000 7fa6f2ffd700 1 -- 172.21.15.92:0/2529595056 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa6f8019030 con 0x7fa6fc038410 2023-11-28T11:21:22.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.623+0000 7f7e1f7fe700 1 -- 172.21.15.92:0/2642059246 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e18011790 con 0x7f7e28160b40 2023-11-28T11:21:22.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.623+0000 7f3e58bff700 1 Processor -- start 2023-11-28T11:21:22.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.623+0000 7f3e58bff700 1 -- start start 2023-11-28T11:21:22.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.623+0000 7f7e2d5ff700 1 -- 172.21.15.92:0/2642059246 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e280b5c90 con 0x7f7e28160b40 2023-11-28T11:21:22.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.624+0000 7f7e2d5ff700 1 -- 172.21.15.92:0/2642059246 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e280d26d0 con 0x7f7e28160b40 2023-11-28T11:21:22.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.624+0000 7f7e1f7fe700 1 -- 172.21.15.92:0/2642059246 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e180118f0 con 0x7f7e28160b40 2023-11-28T11:21:22.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.624+0000 7f7e1f7fe700 1 -- 172.21.15.92:0/2642059246 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e18011790 con 0x7f7e28160b40 2023-11-28T11:21:22.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.625+0000 7fa6f2ffd700 1 --2- 172.21.15.92:0/2529595056 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa6e4068340 0x7fa6e406a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:22.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.625+0000 7fa6f2ffd700 1 -- 172.21.15.92:0/2529595056 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa6f8085d90 con 0x7fa6fc038410 2023-11-28T11:21:22.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.626+0000 7fa701d9b700 1 --2- 172.21.15.92:0/2529595056 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa6e4068340 0x7fa6e406a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:22.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.624+0000 7f3e58bff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e540d2bf0 0x7f3e540b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:22.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.625+0000 7f3e58bff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e54160b40 0x7f3e540c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:22.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.625+0000 7f3e58bff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e541645a0 0x7f3e54151470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:22.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.625+0000 7f3e58bff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e540c87c0 con 0x7f3e541645a0 2023-11-28T11:21:22.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.625+0000 7f3e58bff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e541519b0 con 0x7f3e540d2bf0 2023-11-28T11:21:22.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.625+0000 7f3e58bff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e540c7bb0 con 0x7f3e54160b40 2023-11-28T11:21:22.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.625+0000 7f3e4bfff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e54160b40 0x7f3e540c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:22.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.625+0000 7f3e4bfff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e54160b40 0x7f3e540c8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.92:34258/0 (socket says 172.21.15.92:34258) 2023-11-28T11:21:22.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.625+0000 7f3e4bfff700 1 -- 172.21.15.92:0/829393197 learned_addr learned my addr 172.21.15.92:0/829393197 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:22.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.626+0000 7f3e5259c700 1 --2- 172.21.15.92:0/829393197 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e540d2bf0 0x7f3e540b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:22.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.626+0000 7f3e52d9d700 1 --2- 172.21.15.92:0/829393197 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e541645a0 0x7f3e54151470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:22.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.626+0000 7f7e2d5ff700 1 -- 172.21.15.92:0/2642059246 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e0c0052f0 con 0x7f7e28160b40 2023-11-28T11:21:22.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.627+0000 7f3e52d9d700 1 -- 172.21.15.92:0/829393197 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e54160b40 msgr2=0x7f3e540c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:22.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.627+0000 7f3e52d9d700 1 --2- 172.21.15.92:0/829393197 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e54160b40 0x7f3e540c8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:22.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.627+0000 7f3e52d9d700 1 -- 172.21.15.92:0/829393197 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e540d2bf0 msgr2=0x7f3e540b89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:22.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.627+0000 7f3e52d9d700 1 --2- 172.21.15.92:0/829393197 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e540d2bf0 0x7f3e540b89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:22.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.627+0000 7f3e52d9d700 1 -- 172.21.15.92:0/829393197 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e4c009b30 con 0x7f3e541645a0 2023-11-28T11:21:22.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.627+0000 7f3e52d9d700 1 --2- 172.21.15.92:0/829393197 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e541645a0 0x7f3e54151470 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c014000 tx=0x7f3e4c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:22.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.628+0000 7f3e4affd700 1 -- 172.21.15.92:0/829393197 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c004440 con 0x7f3e541645a0 2023-11-28T11:21:22.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.628+0000 7fa701d9b700 1 --2- 172.21.15.92:0/2529595056 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa6e4068340 0x7fa6e406a7f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc04e400 tx=0x7fa6f4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:22.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.629+0000 7f3e4affd700 1 -- 172.21.15.92:0/829393197 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e4c0092e0 con 0x7f3e541645a0 2023-11-28T11:21:22.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.629+0000 7f3e4affd700 1 -- 172.21.15.92:0/829393197 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c004800 con 0x7f3e541645a0 2023-11-28T11:21:22.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.629+0000 7f3e58bff700 1 -- 172.21.15.92:0/829393197 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e540d26d0 con 0x7f3e541645a0 2023-11-28T11:21:22.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.629+0000 7f7e1f7fe700 1 -- 172.21.15.92:0/2642059246 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7e18010050 con 0x7f7e28160b40 2023-11-28T11:21:22.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.629+0000 7f3e58bff700 1 -- 172.21.15.92:0/829393197 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e540d23f0 con 0x7f3e541645a0 2023-11-28T11:21:22.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.630+0000 7f3e58bff700 1 -- 172.21.15.92:0/829393197 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e540b5c90 con 0x7f3e541645a0 2023-11-28T11:21:22.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.631+0000 7f7e1f7fe700 1 --2- 172.21.15.92:0/2642059246 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7e08068350 0x7f7e0806a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:22.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.631+0000 7fa6f2ffd700 1 -- 172.21.15.92:0/2529595056 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa6f80523a0 con 0x7fa6fc038410 2023-11-28T11:21:22.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.631+0000 7f7e1f7fe700 1 -- 172.21.15.92:0/2642059246 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7e18085620 con 0x7f7e28160b40 2023-11-28T11:21:22.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.632+0000 7f7e26ffd700 1 --2- 172.21.15.92:0/2642059246 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7e08068350 0x7f7e0806a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:22.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.633+0000 7f7e26ffd700 1 --2- 172.21.15.92:0/2642059246 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7e08068350 0x7f7e0806a800 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7e281f9fb0 tx=0x7f7e20009000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:22.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.634+0000 7f7e1f7fe700 1 -- 172.21.15.92:0/2642059246 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7e18051aa0 con 0x7f7e28160b40 2023-11-28T11:21:22.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.634+0000 7f3e4affd700 1 -- 172.21.15.92:0/829393197 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3e4c034030 con 0x7f3e541645a0 2023-11-28T11:21:22.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.635+0000 7f3e4affd700 1 --2- 172.21.15.92:0/829393197 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3e380682f0 0x7f3e3806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:22.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.635+0000 7f3e4affd700 1 -- 172.21.15.92:0/829393197 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3e4c0985a0 con 0x7f3e541645a0 2023-11-28T11:21:22.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.636+0000 7f3e5259c700 1 --2- 172.21.15.92:0/829393197 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3e380682f0 0x7f3e3806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:22.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.638+0000 7f3e5259c700 1 --2- 172.21.15.92:0/829393197 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3e380682f0 0x7f3e3806a7a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3e44006fd0 tx=0x7f3e44009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:22.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.644+0000 7f3e4affd700 1 -- 172.21.15.92:0/829393197 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3e4c064b10 con 0x7f3e541645a0 2023-11-28T11:21:22.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.866+0000 7f7a0f3ff700 1 -- 172.21.15.92:0/3909703649 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f79e8005870 con 0x7f7a000513f0 2023-11-28T11:21:22.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.867+0000 7f79fdffb700 1 -- 172.21.15.92:0/3909703649 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f79f80573d0 con 0x7f7a000513f0 2023-11-28T11:21:22.872 INFO:teuthology.orchestra.run.smithi092.stdout:55834574859 2023-11-28T11:21:22.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.871+0000 7f7a0f3ff700 1 -- 172.21.15.92:0/3909703649 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f79ec068350 msgr2=0x7f79ec06a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:22.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.871+0000 7f7a0f3ff700 1 --2- 172.21.15.92:0/3909703649 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f79ec068350 0x7f79ec06a800 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7a04007fd0 tx=0x7f7a04009040 comp rx=0 tx=0).stop 2023-11-28T11:21:22.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.871+0000 7f7a0f3ff700 1 -- 172.21.15.92:0/3909703649 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a000513f0 msgr2=0x7f7a0004b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:22.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.871+0000 7f7a0f3ff700 1 --2- 172.21.15.92:0/3909703649 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a000513f0 0x7f7a0004b920 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f79f8009840 tx=0x7f79f8004750 comp rx=0 tx=0).stop 2023-11-28T11:21:22.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.872+0000 7f7a0f3ff700 1 -- 172.21.15.92:0/3909703649 shutdown_connections 2023-11-28T11:21:22.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.872+0000 7f7a0f3ff700 1 --2- 172.21.15.92:0/3909703649 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7a00054970 0x7f7a0004be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:22.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.872+0000 7f7a0f3ff700 1 --2- 172.21.15.92:0/3909703649 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f79ec068350 0x7f79ec06a800 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:22.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.872+0000 7f7a0f3ff700 1 --2- 172.21.15.92:0/3909703649 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a00038e40 0x7f7a000e6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:22.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.872+0000 7f7a0f3ff700 1 --2- 172.21.15.92:0/3909703649 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7a000513f0 0x7f7a0004b920 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:22.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.872+0000 7f7a0f3ff700 1 -- 172.21.15.92:0/3909703649 >> 172.21.15.92:0/3909703649 conn(0x7f7a0000cf40 msgr2=0x7f7a00009450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:22.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.872+0000 7f7a0f3ff700 1 -- 172.21.15.92:0/3909703649 shutdown_connections 2023-11-28T11:21:22.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.873+0000 7f7a0f3ff700 1 -- 172.21.15.92:0/3909703649 wait complete. 2023-11-28T11:21:22.907 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.0 2023-11-28T11:21:22.907 DEBUG:teuthology.parallel:result is None 2023-11-28T11:21:22.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.963+0000 7f61c0fff700 1 -- 172.21.15.92:0/3647252921 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f61bc0c8240 con 0x7f61bc160fd0 2023-11-28T11:21:22.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.964+0000 7f61aaffd700 1 -- 172.21.15.92:0/3647252921 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f61ac0554f0 con 0x7f61bc160fd0 2023-11-28T11:21:22.966 INFO:teuthology.orchestra.run.smithi092.stdout:55834574859 2023-11-28T11:21:22.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.968+0000 7f61c0fff700 1 -- 172.21.15.92:0/3647252921 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f619c0683c0 msgr2=0x7f619c06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:22.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.968+0000 7f61c0fff700 1 --2- 172.21.15.92:0/3647252921 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f619c0683c0 0x7f619c06a870 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f61b0000f60 tx=0x7f61b0009040 comp rx=0 tx=0).stop 2023-11-28T11:21:22.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.968+0000 7f61c0fff700 1 -- 172.21.15.92:0/3647252921 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61bc160fd0 msgr2=0x7f61bc0b5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:22.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.968+0000 7f61c0fff700 1 --2- 172.21.15.92:0/3647252921 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61bc160fd0 0x7f61bc0b5ee0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f61ac00b440 tx=0x7f61ac00b750 comp rx=0 tx=0).stop 2023-11-28T11:21:22.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.968+0000 7f61c0fff700 1 -- 172.21.15.92:0/3647252921 shutdown_connections 2023-11-28T11:21:22.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.968+0000 7f61c0fff700 1 --2- 172.21.15.92:0/3647252921 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f61bc160fd0 0x7f61bc0b5ee0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:22.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.968+0000 7f61c0fff700 1 --2- 172.21.15.92:0/3647252921 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f61bc0cf8b0 0x7f61bc0cc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:22.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.968+0000 7f61c0fff700 1 --2- 172.21.15.92:0/3647252921 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f619c0683c0 0x7f619c06a870 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:22.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.968+0000 7f61c0fff700 1 --2- 172.21.15.92:0/3647252921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61bc0d1f20 0x7f61bc0cc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:22.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.968+0000 7f61c0fff700 1 -- 172.21.15.92:0/3647252921 >> 172.21.15.92:0/3647252921 conn(0x7f61bc0719a0 msgr2=0x7f61bc15ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:22.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.968+0000 7f61c0fff700 1 -- 172.21.15.92:0/3647252921 shutdown_connections 2023-11-28T11:21:22.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.968+0000 7f61c0fff700 1 -- 172.21.15.92:0/3647252921 wait complete. 2023-11-28T11:21:22.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.980+0000 7ffac53ff700 1 -- 172.21.15.92:0/3540546219 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ffaa40055a0 con 0x7ffac00cdee0 2023-11-28T11:21:22.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.981+0000 7ffab77fe700 1 -- 172.21.15.92:0/3540546219 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ffab0052220 con 0x7ffac00cdee0 2023-11-28T11:21:22.983 INFO:teuthology.orchestra.run.smithi092.stdout:55834574858 2023-11-28T11:21:22.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.981+0000 7fa70359e700 1 -- 172.21.15.92:0/2529595056 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fa6fc005940 con 0x7fa6fc038410 2023-11-28T11:21:22.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.982+0000 7fa6f2ffd700 1 -- 172.21.15.92:0/2529595056 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa6f80559c0 con 0x7fa6fc038410 2023-11-28T11:21:22.984 INFO:teuthology.orchestra.run.smithi092.stdout:55834574858 2023-11-28T11:21:22.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.985+0000 7ffac53ff700 1 -- 172.21.15.92:0/3540546219 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ffaa00683c0 msgr2=0x7ffaa006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:22.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.985+0000 7ffac53ff700 1 --2- 172.21.15.92:0/3540546219 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ffaa00683c0 0x7ffaa006a870 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ffac00c8a50 tx=0x7ffab8009190 comp rx=0 tx=0).stop 2023-11-28T11:21:22.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.985+0000 7ffac53ff700 1 -- 172.21.15.92:0/3540546219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffac00cdee0 msgr2=0x7ffac0144ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:22.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.985+0000 7ffac53ff700 1 --2- 172.21.15.92:0/3540546219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffac00cdee0 0x7ffac0144ac0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ffab000bd50 tx=0x7ffab000bd80 comp rx=0 tx=0).stop 2023-11-28T11:21:22.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.985+0000 7fa70359e700 1 -- 172.21.15.92:0/2529595056 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa6e4068340 msgr2=0x7fa6e406a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:22.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.985+0000 7fa70359e700 1 --2- 172.21.15.92:0/2529595056 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa6e4068340 0x7fa6e406a7f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc04e400 tx=0x7fa6f4008040 comp rx=0 tx=0).stop 2023-11-28T11:21:22.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.985+0000 7fa70359e700 1 -- 172.21.15.92:0/2529595056 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6fc038410 msgr2=0x7fa6fc0e0510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:22.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.985+0000 7fa70359e700 1 --2- 172.21.15.92:0/2529595056 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6fc038410 0x7fa6fc0e0510 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa6f800e930 tx=0x7fa6f800ec40 comp rx=0 tx=0).stop 2023-11-28T11:21:22.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.986+0000 7fa70359e700 1 -- 172.21.15.92:0/2529595056 shutdown_connections 2023-11-28T11:21:22.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.986+0000 7fa70359e700 1 --2- 172.21.15.92:0/2529595056 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6fc0558e0 0x7fa6fc0e24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:22.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.986+0000 7fa70359e700 1 --2- 172.21.15.92:0/2529595056 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6fc038410 0x7fa6fc0e0510 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:22.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.986+0000 7fa70359e700 1 --2- 172.21.15.92:0/2529595056 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa6e4068340 0x7fa6e406a7f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:22.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.986+0000 7fa70359e700 1 --2- 172.21.15.92:0/2529595056 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6fc03b1f0 0x7fa6fc0e40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:22.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.986+0000 7fa70359e700 1 -- 172.21.15.92:0/2529595056 >> 172.21.15.92:0/2529595056 conn(0x7fa6fc055cb0 msgr2=0x7fa6fc005d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:22.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.986+0000 7ffac53ff700 1 -- 172.21.15.92:0/3540546219 shutdown_connections 2023-11-28T11:21:22.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.986+0000 7ffac53ff700 1 --2- 172.21.15.92:0/3540546219 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ffaa00683c0 0x7ffaa006a870 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:22.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.986+0000 7ffac53ff700 1 --2- 172.21.15.92:0/3540546219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffac015bea0 0x7ffac0145000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:22.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.986+0000 7ffac53ff700 1 --2- 172.21.15.92:0/3540546219 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffac00cbf70 0x7ffac0169da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:22.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.986+0000 7ffac53ff700 1 --2- 172.21.15.92:0/3540546219 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffac00cdee0 0x7ffac0144ac0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:22.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.986+0000 7ffac53ff700 1 -- 172.21.15.92:0/3540546219 >> 172.21.15.92:0/3540546219 conn(0x7ffac00cb490 msgr2=0x7ffac00c9ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:22.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.986+0000 7fa70359e700 1 -- 172.21.15.92:0/2529595056 shutdown_connections 2023-11-28T11:21:22.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.986+0000 7ffac53ff700 1 -- 172.21.15.92:0/3540546219 shutdown_connections 2023-11-28T11:21:22.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.986+0000 7ffac53ff700 1 -- 172.21.15.92:0/3540546219 wait complete. 2023-11-28T11:21:22.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:22.986+0000 7fa70359e700 1 -- 172.21.15.92:0/2529595056 wait complete. 2023-11-28T11:21:23.001 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.1 2023-11-28T11:21:23.001 DEBUG:teuthology.parallel:result is None 2023-11-28T11:21:23.010 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.5 2023-11-28T11:21:23.011 DEBUG:teuthology.parallel:result is None 2023-11-28T11:21:23.020 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.3 2023-11-28T11:21:23.020 DEBUG:teuthology.parallel:result is None 2023-11-28T11:21:23.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.036+0000 7f3e58bff700 1 -- 172.21.15.92:0/829393197 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f3e54067170 con 0x7f3e541645a0 2023-11-28T11:21:23.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.037+0000 7f3e4affd700 1 -- 172.21.15.92:0/829393197 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3e4c017030 con 0x7f3e541645a0 2023-11-28T11:21:23.039 INFO:teuthology.orchestra.run.smithi092.stdout:55834574859 2023-11-28T11:21:23.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.039+0000 7f3e58bff700 1 -- 172.21.15.92:0/829393197 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3e380682f0 msgr2=0x7f3e3806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:23.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.039+0000 7f3e58bff700 1 --2- 172.21.15.92:0/829393197 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3e380682f0 0x7f3e3806a7a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3e44006fd0 tx=0x7f3e44009190 comp rx=0 tx=0).stop 2023-11-28T11:21:23.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.039+0000 7f3e58bff700 1 -- 172.21.15.92:0/829393197 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e541645a0 msgr2=0x7f3e54151470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:23.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.039+0000 7f3e58bff700 1 --2- 172.21.15.92:0/829393197 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e541645a0 0x7f3e54151470 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c014000 tx=0x7f3e4c004ad0 comp rx=0 tx=0).stop 2023-11-28T11:21:23.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.039+0000 7f3e58bff700 1 -- 172.21.15.92:0/829393197 shutdown_connections 2023-11-28T11:21:23.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.039+0000 7f3e58bff700 1 --2- 172.21.15.92:0/829393197 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3e380682f0 0x7f3e3806a7a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:23.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.039+0000 7f3e58bff700 1 --2- 172.21.15.92:0/829393197 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e541645a0 0x7f3e54151470 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:23.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.039+0000 7f3e58bff700 1 --2- 172.21.15.92:0/829393197 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e540d2bf0 0x7f3e540b89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:23.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.039+0000 7f3e58bff700 1 --2- 172.21.15.92:0/829393197 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e54160b40 0x7f3e540c8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:23.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.039+0000 7f3e58bff700 1 -- 172.21.15.92:0/829393197 >> 172.21.15.92:0/829393197 conn(0x7f3e540719a0 msgr2=0x7f3e540cf6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:23.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.039+0000 7f3e58bff700 1 -- 172.21.15.92:0/829393197 shutdown_connections 2023-11-28T11:21:23.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.039+0000 7f3e58bff700 1 -- 172.21.15.92:0/829393197 wait complete. 2023-11-28T11:21:23.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.042+0000 7f84329ff700 1 -- 172.21.15.92:0/3071987121 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f840c0055a0 con 0x7f842c15ce70 2023-11-28T11:21:23.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.043+0000 7f8428ff9700 1 -- 172.21.15.92:0/3071987121 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f842408a020 con 0x7f842c15ce70 2023-11-28T11:21:23.044 INFO:teuthology.orchestra.run.smithi092.stdout:55834574859 2023-11-28T11:21:23.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.045+0000 7f84329ff700 1 -- 172.21.15.92:0/3071987121 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f84180682f0 msgr2=0x7f841806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:23.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.045+0000 7f84329ff700 1 --2- 172.21.15.92:0/3071987121 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f84180682f0 0x7f841806a7a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f841c009300 tx=0x7f841c008030 comp rx=0 tx=0).stop 2023-11-28T11:21:23.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.045+0000 7f84329ff700 1 -- 172.21.15.92:0/3071987121 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f842c15ce70 msgr2=0x7f842c0b1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:23.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.045+0000 7f84329ff700 1 --2- 172.21.15.92:0/3071987121 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f842c15ce70 0x7f842c0b1840 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f842400cb90 tx=0x7f842400cf50 comp rx=0 tx=0).stop 2023-11-28T11:21:23.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.045+0000 7f84329ff700 1 -- 172.21.15.92:0/3071987121 shutdown_connections 2023-11-28T11:21:23.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.045+0000 7f84329ff700 1 --2- 172.21.15.92:0/3071987121 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f84180682f0 0x7f841806a7a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:23.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.045+0000 7f84329ff700 1 --2- 172.21.15.92:0/3071987121 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f842c15ce70 0x7f842c0b1840 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:23.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.045+0000 7f84329ff700 1 --2- 172.21.15.92:0/3071987121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f842c06a270 0x7f842c0ae5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:23.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.045+0000 7f84329ff700 1 --2- 172.21.15.92:0/3071987121 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f842c0cdee0 0x7f842c0c2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:23.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.045+0000 7f84329ff700 1 -- 172.21.15.92:0/3071987121 >> 172.21.15.92:0/3071987121 conn(0x7f842c0cb490 msgr2=0x7f842c0cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:23.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.045+0000 7f84329ff700 1 -- 172.21.15.92:0/3071987121 shutdown_connections 2023-11-28T11:21:23.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.045+0000 7f84329ff700 1 -- 172.21.15.92:0/3071987121 wait complete. 2023-11-28T11:21:23.066 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.7 2023-11-28T11:21:23.066 DEBUG:teuthology.parallel:result is None 2023-11-28T11:21:23.067 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.6 2023-11-28T11:21:23.067 DEBUG:teuthology.parallel:result is None 2023-11-28T11:21:23.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.067+0000 7f631959d700 1 -- 172.21.15.92:0/3984371319 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f630c0e7e20 con 0x7f630c0e6d00 2023-11-28T11:21:23.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.067+0000 7f6310ff9700 1 -- 172.21.15.92:0/3984371319 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f63140cb860 con 0x7f630c0e6d00 2023-11-28T11:21:23.069 INFO:teuthology.orchestra.run.smithi092.stdout:55834574857 2023-11-28T11:21:23.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.071+0000 7f631959d700 1 -- 172.21.15.92:0/3984371319 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6300068340 msgr2=0x7f630006a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:23.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.071+0000 7f631959d700 1 --2- 172.21.15.92:0/3984371319 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6300068340 0x7f630006a7f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6308005910 tx=0x7f6308009040 comp rx=0 tx=0).stop 2023-11-28T11:21:23.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.071+0000 7f631959d700 1 -- 172.21.15.92:0/3984371319 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f630c0e6d00 msgr2=0x7f630c0e9650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:23.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.071+0000 7f631959d700 1 --2- 172.21.15.92:0/3984371319 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f630c0e6d00 0x7f630c0e9650 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f63140683c0 tx=0x7f63140677b0 comp rx=0 tx=0).stop 2023-11-28T11:21:23.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.071+0000 7f631959d700 1 -- 172.21.15.92:0/3984371319 shutdown_connections 2023-11-28T11:21:23.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.071+0000 7f631959d700 1 --2- 172.21.15.92:0/3984371319 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f630c0e6d00 0x7f630c0e9650 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:23.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.071+0000 7f631959d700 1 --2- 172.21.15.92:0/3984371319 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f630c036b50 0x7f630c0e8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:23.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.071+0000 7f631959d700 1 --2- 172.21.15.92:0/3984371319 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6300068340 0x7f630006a7f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:23.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.071+0000 7f631959d700 1 --2- 172.21.15.92:0/3984371319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f630c051150 0x7f630c0dbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:23.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.071+0000 7f631959d700 1 -- 172.21.15.92:0/3984371319 >> 172.21.15.92:0/3984371319 conn(0x7f630c051610 msgr2=0x7f630c035560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:23.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.071+0000 7f631959d700 1 -- 172.21.15.92:0/3984371319 shutdown_connections 2023-11-28T11:21:23.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.071+0000 7f631959d700 1 -- 172.21.15.92:0/3984371319 wait complete. 2023-11-28T11:21:23.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.078+0000 7f7e2d5ff700 1 -- 172.21.15.92:0/2642059246 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f7e0c0055a0 con 0x7f7e28160b40 2023-11-28T11:21:23.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.078+0000 7f7e1f7fe700 1 -- 172.21.15.92:0/2642059246 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7e180550c0 con 0x7f7e28160b40 2023-11-28T11:21:23.080 INFO:teuthology.orchestra.run.smithi092.stdout:55834574858 2023-11-28T11:21:23.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.081+0000 7f7e2d5ff700 1 -- 172.21.15.92:0/2642059246 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7e08068350 msgr2=0x7f7e0806a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:23.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.081+0000 7f7e2d5ff700 1 --2- 172.21.15.92:0/2642059246 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7e08068350 0x7f7e0806a800 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7e281f9fb0 tx=0x7f7e20009000 comp rx=0 tx=0).stop 2023-11-28T11:21:23.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.081+0000 7f7e2d5ff700 1 -- 172.21.15.92:0/2642059246 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e28160b40 msgr2=0x7f7e281ea3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:23.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.081+0000 7f7e2d5ff700 1 --2- 172.21.15.92:0/2642059246 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e28160b40 0x7f7e281ea3e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7e1800c4a0 tx=0x7f7e1800c7b0 comp rx=0 tx=0).stop 2023-11-28T11:21:23.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.081+0000 7f7e2d5ff700 1 -- 172.21.15.92:0/2642059246 shutdown_connections 2023-11-28T11:21:23.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.081+0000 7f7e2d5ff700 1 --2- 172.21.15.92:0/2642059246 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e28164cb0 0x7f7e280cba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:23.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.081+0000 7f7e2d5ff700 1 --2- 172.21.15.92:0/2642059246 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7e08068350 0x7f7e0806a800 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:23.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.081+0000 7f7e2d5ff700 1 --2- 172.21.15.92:0/2642059246 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e280cf900 0x7f7e2815b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:23.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.081+0000 7f7e2d5ff700 1 --2- 172.21.15.92:0/2642059246 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e28160b40 0x7f7e281ea3e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:23.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.081+0000 7f7e2d5ff700 1 -- 172.21.15.92:0/2642059246 >> 172.21.15.92:0/2642059246 conn(0x7f7e280719a0 msgr2=0x7f7e280b28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:23.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.081+0000 7f7e2d5ff700 1 -- 172.21.15.92:0/2642059246 shutdown_connections 2023-11-28T11:21:23.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.081+0000 7f7e2d5ff700 1 -- 172.21.15.92:0/2642059246 wait complete. 2023-11-28T11:21:23.093 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.2 2023-11-28T11:21:23.093 DEBUG:teuthology.parallel:result is None 2023-11-28T11:21:23.101 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.4 2023-11-28T11:21:23.101 DEBUG:teuthology.parallel:result is None 2023-11-28T11:21:23.101 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-28T11:21:23.102 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:21:23.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.337+0000 7fad255ff700 1 Processor -- start 2023-11-28T11:21:23.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.337+0000 7fad255ff700 1 -- start start 2023-11-28T11:21:23.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.338+0000 7fad255ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad200d2df0 0x7fad200cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:23.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.338+0000 7fad255ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad200d1950 con 0x7fad200b1c60 2023-11-28T11:21:23.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.338+0000 7fad255ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad200b2500 con 0x7fad200b4fc0 2023-11-28T11:21:23.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.338+0000 7fad255ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad200d3d30 con 0x7fad200d2df0 2023-11-28T11:21:23.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.338+0000 7fad1f7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fad200b1c60 0x7fad200cf010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47500/0 (socket says 172.21.15.92:47500) 2023-11-28T11:21:23.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.339+0000 7fad1f7fe700 1 -- 172.21.15.92:0/2640397628 learned_addr learned my addr 172.21.15.92:0/2640397628 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:23.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.339+0000 7fad1effd700 1 --2- 172.21.15.92:0/2640397628 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad200d2df0 0x7fad200cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:23.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.339+0000 7fad1dffb700 1 -- 172.21.15.92:0/2640397628 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2057483731 0 0) 0x7fad200d1950 con 0x7fad200b1c60 2023-11-28T11:21:23.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.339+0000 7fad1dffb700 1 -- 172.21.15.92:0/2640397628 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad080035b0 con 0x7fad200b1c60 2023-11-28T11:21:23.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.339+0000 7fad1dffb700 1 -- 172.21.15.92:0/2640397628 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 302474224 0 0) 0x7fad080035b0 con 0x7fad200b1c60 2023-11-28T11:21:23.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.339+0000 7fad1dffb700 1 -- 172.21.15.92:0/2640397628 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad200d2df0 msgr2=0x7fad200cd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:23.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.339+0000 7fad1dffb700 1 --2- 172.21.15.92:0/2640397628 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad200d2df0 0x7fad200cd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:23.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.339+0000 7fad1dffb700 1 -- 172.21.15.92:0/2640397628 >> v1:172.21.15.169:6789/0 conn(0x7fad200b4fc0 legacy=0x7fad200d31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:23.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.339+0000 7fad1dffb700 1 -- 172.21.15.92:0/2640397628 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad200d3a50 con 0x7fad200b1c60 2023-11-28T11:21:23.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.339+0000 7fad1dffb700 1 -- 172.21.15.92:0/2640397628 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532161587 0 0) 0x7fad180019f0 con 0x7fad200b1c60 2023-11-28T11:21:23.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.339+0000 7fad1dffb700 1 -- 172.21.15.92:0/2640397628 >> v1:172.21.15.92:6789/0 conn(0x7fad200b1c60 legacy=0x7fad200cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:23.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.339+0000 7fad1dffb700 1 --2- 172.21.15.92:0/2640397628 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad08003a30 0x7fad08003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:23.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.339+0000 7fad1dffb700 1 --2- 172.21.15.92:0/2640397628 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad080043d0 0x7fad08004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:23.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.339+0000 7fad1dffb700 1 --2- 172.21.15.92:0/2640397628 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad200d2df0 0x7fad080050f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:23.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.339+0000 7fad1dffb700 1 -- 172.21.15.92:0/2640397628 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad200d3d30 con 0x7fad08003a30 2023-11-28T11:21:23.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.339+0000 7fad1dffb700 1 -- 172.21.15.92:0/2640397628 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0800ba10 con 0x7fad080043d0 2023-11-28T11:21:23.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.339+0000 7fad1dffb700 1 -- 172.21.15.92:0/2640397628 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad0800bba0 con 0x7fad200d2df0 2023-11-28T11:21:23.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.340+0000 7fad1effd700 1 --2- 172.21.15.92:0/2640397628 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad08003a30 0x7fad08003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:23.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.340+0000 7fad1e7fc700 1 --2- 172.21.15.92:0/2640397628 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad080043d0 0x7fad08004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:23.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.340+0000 7fad1f7fe700 1 --2- 172.21.15.92:0/2640397628 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad200d2df0 0x7fad080050f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:23.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.340+0000 7fad1effd700 1 -- 172.21.15.92:0/2640397628 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad200d2df0 msgr2=0x7fad080050f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:23.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.340+0000 7fad1effd700 1 --2- 172.21.15.92:0/2640397628 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad200d2df0 0x7fad080050f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:23.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.340+0000 7fad1effd700 1 -- 172.21.15.92:0/2640397628 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad080043d0 msgr2=0x7fad08004890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:23.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.340+0000 7fad1effd700 1 --2- 172.21.15.92:0/2640397628 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad080043d0 0x7fad08004890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:23.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.340+0000 7fad1effd700 1 -- 172.21.15.92:0/2640397628 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad080035b0 con 0x7fad08003a30 2023-11-28T11:21:23.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.340+0000 7fad1f7fe700 1 --2- 172.21.15.92:0/2640397628 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad200d2df0 0x7fad080050f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:21:23.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.340+0000 7fad1effd700 1 --2- 172.21.15.92:0/2640397628 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad08003a30 0x7fad08003e00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fad10009b60 tx=0x7fad08005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81b6f3121d46c9cd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:23.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.340+0000 7fad1dffb700 1 -- 172.21.15.92:0/2640397628 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1000e6c0 con 0x7fad08003a30 2023-11-28T11:21:23.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.340+0000 7fad1dffb700 1 -- 172.21.15.92:0/2640397628 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad1000e820 con 0x7fad08003a30 2023-11-28T11:21:23.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.340+0000 7fad1dffb700 1 -- 172.21.15.92:0/2640397628 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1000eb90 con 0x7fad08003a30 2023-11-28T11:21:23.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.341+0000 7fad255ff700 1 -- 172.21.15.92:0/2640397628 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad08003a30 msgr2=0x7fad08003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:23.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.341+0000 7fad255ff700 1 --2- 172.21.15.92:0/2640397628 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad08003a30 0x7fad08003e00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fad10009b60 tx=0x7fad08005880 comp rx=0 tx=0).stop 2023-11-28T11:21:23.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.341+0000 7fad1f7fe700 1 -- 172.21.15.92:0/2640397628 reap_dead start 2023-11-28T11:21:23.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.341+0000 7fad255ff700 1 -- 172.21.15.92:0/2640397628 shutdown_connections 2023-11-28T11:21:23.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.341+0000 7fad255ff700 1 -- 172.21.15.92:0/2640397628 >> 172.21.15.92:0/2640397628 conn(0x7fad200719a0 msgr2=0x7fad20071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:23.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.341+0000 7fad255ff700 1 -- 172.21.15.92:0/2640397628 shutdown_connections 2023-11-28T11:21:23.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.341+0000 7fad255ff700 1 -- 172.21.15.92:0/2640397628 wait complete. 2023-11-28T11:21:23.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.343+0000 7fad255ff700 1 Processor -- start 2023-11-28T11:21:23.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.344+0000 7fad255ff700 1 -- start start 2023-11-28T11:21:23.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.345+0000 7fad255ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad08003a30 0x7fad201595e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:23.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.345+0000 7fad255ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad201f2190 0x7fad201563c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:23.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.345+0000 7fad255ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad200b3ff0 0x7fad200b4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:23.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.345+0000 7fad255ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad200b49a0 con 0x7fad200b3ff0 2023-11-28T11:21:23.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.345+0000 7fad255ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad200cb320 con 0x7fad201f2190 2023-11-28T11:21:23.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.345+0000 7fad255ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad200caff0 con 0x7fad08003a30 2023-11-28T11:21:23.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.345+0000 7fad1f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad200b3ff0 0x7fad200b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:23.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.345+0000 7fad1f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad200b3ff0 0x7fad200b4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49084/0 (socket says 172.21.15.92:49084) 2023-11-28T11:21:23.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.345+0000 7fad1f7fe700 1 -- 172.21.15.92:0/2316629899 learned_addr learned my addr 172.21.15.92:0/2316629899 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:23.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.346+0000 7fad1e7fc700 1 --2- 172.21.15.92:0/2316629899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad201f2190 0x7fad201563c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:23.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.346+0000 7fad1effd700 1 --2- 172.21.15.92:0/2316629899 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad08003a30 0x7fad201595e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:23.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.346+0000 7fad1f7fe700 1 -- 172.21.15.92:0/2316629899 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad08003a30 msgr2=0x7fad201595e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:23.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.346+0000 7fad1f7fe700 1 --2- 172.21.15.92:0/2316629899 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad08003a30 0x7fad201595e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:23.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.346+0000 7fad1f7fe700 1 -- 172.21.15.92:0/2316629899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad201f2190 msgr2=0x7fad201563c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:23.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.346+0000 7fad1f7fe700 1 --2- 172.21.15.92:0/2316629899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad201f2190 0x7fad201563c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:23.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.346+0000 7fad1f7fe700 1 -- 172.21.15.92:0/2316629899 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad200b88f0 con 0x7fad200b3ff0 2023-11-28T11:21:23.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.346+0000 7fad1effd700 1 --2- 172.21.15.92:0/2316629899 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad08003a30 0x7fad201595e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:21:23.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.346+0000 7fad1f7fe700 1 --2- 172.21.15.92:0/2316629899 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad200b3ff0 0x7fad200b4460 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fad2008c150 tx=0x7fad1800e440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:23.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.346+0000 7fad077fe700 1 -- 172.21.15.92:0/2316629899 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1800ecd0 con 0x7fad200b3ff0 2023-11-28T11:21:23.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.346+0000 7fad077fe700 1 -- 172.21.15.92:0/2316629899 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad1800ee30 con 0x7fad200b3ff0 2023-11-28T11:21:23.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.346+0000 7fad255ff700 1 -- 172.21.15.92:0/2316629899 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad200d15e0 con 0x7fad200b3ff0 2023-11-28T11:21:23.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.346+0000 7fad077fe700 1 -- 172.21.15.92:0/2316629899 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad18013620 con 0x7fad200b3ff0 2023-11-28T11:21:23.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.346+0000 7fad255ff700 1 -- 172.21.15.92:0/2316629899 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad200b21b0 con 0x7fad200b3ff0 2023-11-28T11:21:23.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.347+0000 7fad255ff700 1 -- 172.21.15.92:0/2316629899 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad20067170 con 0x7fad200b3ff0 2023-11-28T11:21:23.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.348+0000 7fad077fe700 1 -- 172.21.15.92:0/2316629899 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fad18013780 con 0x7fad200b3ff0 2023-11-28T11:21:23.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.349+0000 7fad077fe700 1 --2- 172.21.15.92:0/2316629899 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fad0c0682f0 0x7fad0c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:23.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.349+0000 7fad077fe700 1 -- 172.21.15.92:0/2316629899 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fad180867b0 con 0x7fad200b3ff0 2023-11-28T11:21:23.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.349+0000 7fad1effd700 1 --2- 172.21.15.92:0/2316629899 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fad0c0682f0 0x7fad0c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:23.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.350+0000 7fad1effd700 1 --2- 172.21.15.92:0/2316629899 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fad0c0682f0 0x7fad0c06a7a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fad100090b0 tx=0x7fad10008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:23.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.353+0000 7fad077fe700 1 -- 172.21.15.92:0/2316629899 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fad18052bd0 con 0x7fad200b3ff0 2023-11-28T11:21:23.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.541+0000 7fad255ff700 1 -- 172.21.15.92:0/2316629899 --> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fad200cb460 con 0x7fad0c0682f0 2023-11-28T11:21:23.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.552+0000 7fad077fe700 1 -- 172.21.15.92:0/2316629899 <== mgr.4103 v2:172.21.15.92:6832/115482 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367352 (secure 0 0 0) 0x7fad200cb460 con 0x7fad0c0682f0 2023-11-28T11:21:23.562 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:21:23.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.557+0000 7fad255ff700 1 -- 172.21.15.92:0/2316629899 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fad0c0682f0 msgr2=0x7fad0c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:23.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.557+0000 7fad255ff700 1 --2- 172.21.15.92:0/2316629899 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fad0c0682f0 0x7fad0c06a7a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fad100090b0 tx=0x7fad10008030 comp rx=0 tx=0).stop 2023-11-28T11:21:23.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.557+0000 7fad255ff700 1 -- 172.21.15.92:0/2316629899 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad200b3ff0 msgr2=0x7fad200b4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:23.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.557+0000 7fad255ff700 1 --2- 172.21.15.92:0/2316629899 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad200b3ff0 0x7fad200b4460 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fad2008c150 tx=0x7fad1800e440 comp rx=0 tx=0).stop 2023-11-28T11:21:23.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.557+0000 7fad255ff700 1 -- 172.21.15.92:0/2316629899 shutdown_connections 2023-11-28T11:21:23.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.557+0000 7fad255ff700 1 --2- 172.21.15.92:0/2316629899 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fad0c0682f0 0x7fad0c06a7a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:23.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.557+0000 7fad255ff700 1 --2- 172.21.15.92:0/2316629899 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad200b3ff0 0x7fad200b4460 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:23.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.557+0000 7fad255ff700 1 --2- 172.21.15.92:0/2316629899 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad08003a30 0x7fad201595e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:23.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.557+0000 7fad255ff700 1 --2- 172.21.15.92:0/2316629899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad201f2190 0x7fad201563c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:23.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.557+0000 7fad255ff700 1 -- 172.21.15.92:0/2316629899 >> 172.21.15.92:0/2316629899 conn(0x7fad200719a0 msgr2=0x7fad200b7310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:23.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.557+0000 7fad255ff700 1 -- 172.21.15.92:0/2316629899 shutdown_connections 2023-11-28T11:21:23.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.557+0000 7fad255ff700 1 -- 172.21.15.92:0/2316629899 wait complete. 2023-11-28T11:21:23.569 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-11-28T11:21:23.577 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2023-11-28T11:21:21.866009+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594896,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":67492,"kb_used_data":3492,"kb_used_omap":0,"kb_used_meta":63936,"kb_avail":754907228,"statfs":{"total":773094113280,"available":773025001472,"internally_reserved":0,"allocated":3575808,"data_stored":2148378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65470464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1749,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002529"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.282064+0000","last_change":"2023-11-28T11:20:56.269261+0000","last_active":"2023-11-28T11:21:00.282064+0000","last_peered":"2023-11-28T11:21:00.282064+0000","last_clean":"2023-11-28T11:21:00.282064+0000","last_became_active":"2023-11-28T11:20:56.268959+0000","last_became_peered":"2023-11-28T11:20:56.268959+0000","last_unstale":"2023-11-28T11:21:00.282064+0000","last_undegraded":"2023-11-28T11:21:00.282064+0000","last_fullsized":"2023-11-28T11:21:00.282064+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:39:28.472955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.273172+0000","last_change":"2023-11-28T11:20:56.258503+0000","last_active":"2023-11-28T11:21:00.273172+0000","last_peered":"2023-11-28T11:21:00.273172+0000","last_clean":"2023-11-28T11:21:00.273172+0000","last_became_active":"2023-11-28T11:20:56.258101+0000","last_became_peered":"2023-11-28T11:20:56.258101+0000","last_unstale":"2023-11-28T11:21:00.273172+0000","last_undegraded":"2023-11-28T11:21:00.273172+0000","last_fullsized":"2023-11-28T11:21:00.273172+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:14:34.876987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.274259+0000","last_change":"2023-11-28T11:20:56.269696+0000","last_active":"2023-11-28T11:21:00.274259+0000","last_peered":"2023-11-28T11:21:00.274259+0000","last_clean":"2023-11-28T11:21:00.274259+0000","last_became_active":"2023-11-28T11:20:56.264650+0000","last_became_peered":"2023-11-28T11:20:56.264650+0000","last_unstale":"2023-11-28T11:21:00.274259+0000","last_undegraded":"2023-11-28T11:21:00.274259+0000","last_fullsized":"2023-11-28T11:21:00.274259+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:25:52.242410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.286447+0000","last_change":"2023-11-28T11:20:56.269817+0000","last_active":"2023-11-28T11:21:00.286447+0000","last_peered":"2023-11-28T11:21:00.286447+0000","last_clean":"2023-11-28T11:21:00.286447+0000","last_became_active":"2023-11-28T11:20:56.269514+0000","last_became_peered":"2023-11-28T11:20:56.269514+0000","last_unstale":"2023-11-28T11:21:00.286447+0000","last_undegraded":"2023-11-28T11:21:00.286447+0000","last_fullsized":"2023-11-28T11:21:00.286447+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:09:41.753917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.289921+0000","last_change":"2023-11-28T11:20:56.274319+0000","last_active":"2023-11-28T11:21:00.289921+0000","last_peered":"2023-11-28T11:21:00.289921+0000","last_clean":"2023-11-28T11:21:00.289921+0000","last_became_active":"2023-11-28T11:20:56.273667+0000","last_became_peered":"2023-11-28T11:20:56.273667+0000","last_unstale":"2023-11-28T11:21:00.289921+0000","last_undegraded":"2023-11-28T11:21:00.289921+0000","last_fullsized":"2023-11-28T11:21:00.289921+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:31:26.827850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.274908+0000","last_change":"2023-11-28T11:20:56.263963+0000","last_active":"2023-11-28T11:21:00.274908+0000","last_peered":"2023-11-28T11:21:00.274908+0000","last_clean":"2023-11-28T11:21:00.274908+0000","last_became_active":"2023-11-28T11:20:56.260761+0000","last_became_peered":"2023-11-28T11:20:56.260761+0000","last_unstale":"2023-11-28T11:21:00.274908+0000","last_undegraded":"2023-11-28T11:21:00.274908+0000","last_fullsized":"2023-11-28T11:21:00.274908+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:13:46.683000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.288334+0000","last_change":"2023-11-28T11:20:56.261054+0000","last_active":"2023-11-28T11:21:00.288334+0000","last_peered":"2023-11-28T11:21:00.288334+0000","last_clean":"2023-11-28T11:21:00.288334+0000","last_became_active":"2023-11-28T11:20:56.260774+0000","last_became_peered":"2023-11-28T11:20:56.260774+0000","last_unstale":"2023-11-28T11:21:00.288334+0000","last_undegraded":"2023-11-28T11:21:00.288334+0000","last_fullsized":"2023-11-28T11:21:00.288334+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:08.079216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.273485+0000","last_change":"2023-11-28T11:20:56.252717+0000","last_active":"2023-11-28T11:21:00.273485+0000","last_peered":"2023-11-28T11:21:00.273485+0000","last_clean":"2023-11-28T11:21:00.273485+0000","last_became_active":"2023-11-28T11:20:56.252066+0000","last_became_peered":"2023-11-28T11:20:56.252066+0000","last_unstale":"2023-11-28T11:21:00.273485+0000","last_undegraded":"2023-11-28T11:21:00.273485+0000","last_fullsized":"2023-11-28T11:21:00.273485+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:42:57.864956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.291971+0000","last_change":"2023-11-28T11:20:56.273810+0000","last_active":"2023-11-28T11:21:00.291971+0000","last_peered":"2023-11-28T11:21:00.291971+0000","last_clean":"2023-11-28T11:21:00.291971+0000","last_became_active":"2023-11-28T11:20:56.273477+0000","last_became_peered":"2023-11-28T11:20:56.273477+0000","last_unstale":"2023-11-28T11:21:00.291971+0000","last_undegraded":"2023-11-28T11:21:00.291971+0000","last_fullsized":"2023-11-28T11:21:00.291971+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:50:45.228070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.277322+0000","last_change":"2023-11-28T11:20:56.256667+0000","last_active":"2023-11-28T11:21:00.277322+0000","last_peered":"2023-11-28T11:21:00.277322+0000","last_clean":"2023-11-28T11:21:00.277322+0000","last_became_active":"2023-11-28T11:20:56.256387+0000","last_became_peered":"2023-11-28T11:20:56.256387+0000","last_unstale":"2023-11-28T11:21:00.277322+0000","last_undegraded":"2023-11-28T11:21:00.277322+0000","last_fullsized":"2023-11-28T11:21:00.277322+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:24:07.384730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.274154+0000","last_change":"2023-11-28T11:20:56.281728+0000","last_active":"2023-11-28T11:21:00.274154+0000","last_peered":"2023-11-28T11:21:00.274154+0000","last_clean":"2023-11-28T11:21:00.274154+0000","last_became_active":"2023-11-28T11:20:56.281437+0000","last_became_peered":"2023-11-28T11:20:56.281437+0000","last_unstale":"2023-11-28T11:21:00.274154+0000","last_undegraded":"2023-11-28T11:21:00.274154+0000","last_fullsized":"2023-11-28T11:21:00.274154+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:01:57.936285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.280518+0000","last_change":"2023-11-28T11:20:56.256016+0000","last_active":"2023-11-28T11:21:00.280518+0000","last_peered":"2023-11-28T11:21:00.280518+0000","last_clean":"2023-11-28T11:21:00.280518+0000","last_became_active":"2023-11-28T11:20:56.255480+0000","last_became_peered":"2023-11-28T11:20:56.255480+0000","last_unstale":"2023-11-28T11:21:00.280518+0000","last_undegraded":"2023-11-28T11:21:00.280518+0000","last_fullsized":"2023-11-28T11:21:00.280518+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:45:09.586958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.280455+0000","last_change":"2023-11-28T11:20:56.265497+0000","last_active":"2023-11-28T11:21:00.280455+0000","last_peered":"2023-11-28T11:21:00.280455+0000","last_clean":"2023-11-28T11:21:00.280455+0000","last_became_active":"2023-11-28T11:20:56.265236+0000","last_became_peered":"2023-11-28T11:20:56.265236+0000","last_unstale":"2023-11-28T11:21:00.280455+0000","last_undegraded":"2023-11-28T11:21:00.280455+0000","last_fullsized":"2023-11-28T11:21:00.280455+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:47:39.108512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.274722+0000","last_change":"2023-11-28T11:20:56.262165+0000","last_active":"2023-11-28T11:21:00.274722+0000","last_peered":"2023-11-28T11:21:00.274722+0000","last_clean":"2023-11-28T11:21:00.274722+0000","last_became_active":"2023-11-28T11:20:56.261947+0000","last_became_peered":"2023-11-28T11:20:56.261947+0000","last_unstale":"2023-11-28T11:21:00.274722+0000","last_undegraded":"2023-11-28T11:21:00.274722+0000","last_fullsized":"2023-11-28T11:21:00.274722+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:45:14.478416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.288123+0000","last_change":"2023-11-28T11:20:56.264861+0000","last_active":"2023-11-28T11:21:00.288123+0000","last_peered":"2023-11-28T11:21:00.288123+0000","last_clean":"2023-11-28T11:21:00.288123+0000","last_became_active":"2023-11-28T11:20:56.264538+0000","last_became_peered":"2023-11-28T11:20:56.264538+0000","last_unstale":"2023-11-28T11:21:00.288123+0000","last_undegraded":"2023-11-28T11:21:00.288123+0000","last_fullsized":"2023-11-28T11:21:00.288123+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:40:42.059656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.273219+0000","last_change":"2023-11-28T11:20:56.258923+0000","last_active":"2023-11-28T11:21:00.273219+0000","last_peered":"2023-11-28T11:21:00.273219+0000","last_clean":"2023-11-28T11:21:00.273219+0000","last_became_active":"2023-11-28T11:20:56.258633+0000","last_became_peered":"2023-11-28T11:20:56.258633+0000","last_unstale":"2023-11-28T11:21:00.273219+0000","last_undegraded":"2023-11-28T11:21:00.273219+0000","last_fullsized":"2023-11-28T11:21:00.273219+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:26:32.343329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.292050+0000","last_change":"2023-11-28T11:20:56.271537+0000","last_active":"2023-11-28T11:21:00.292050+0000","last_peered":"2023-11-28T11:21:00.292050+0000","last_clean":"2023-11-28T11:21:00.292050+0000","last_became_active":"2023-11-28T11:20:56.270556+0000","last_became_peered":"2023-11-28T11:20:56.270556+0000","last_unstale":"2023-11-28T11:21:00.292050+0000","last_undegraded":"2023-11-28T11:21:00.292050+0000","last_fullsized":"2023-11-28T11:21:00.292050+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:11:36.091056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.274876+0000","last_change":"2023-11-28T11:20:56.274810+0000","last_active":"2023-11-28T11:21:00.274876+0000","last_peered":"2023-11-28T11:21:00.274876+0000","last_clean":"2023-11-28T11:21:00.274876+0000","last_became_active":"2023-11-28T11:20:56.274523+0000","last_became_peered":"2023-11-28T11:20:56.274523+0000","last_unstale":"2023-11-28T11:21:00.274876+0000","last_undegraded":"2023-11-28T11:21:00.274876+0000","last_fullsized":"2023-11-28T11:21:00.274876+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:42:57.864956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.281099+0000","last_change":"2023-11-28T11:20:56.258601+0000","last_active":"2023-11-28T11:21:00.281099+0000","last_peered":"2023-11-28T11:21:00.281099+0000","last_clean":"2023-11-28T11:21:00.281099+0000","last_became_active":"2023-11-28T11:20:56.258282+0000","last_became_peered":"2023-11-28T11:20:56.258282+0000","last_unstale":"2023-11-28T11:21:00.281099+0000","last_undegraded":"2023-11-28T11:21:00.281099+0000","last_fullsized":"2023-11-28T11:21:00.281099+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:58:47.731976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.275898+0000","last_change":"2023-11-28T11:20:56.263948+0000","last_active":"2023-11-28T11:21:00.275898+0000","last_peered":"2023-11-28T11:21:00.275898+0000","last_clean":"2023-11-28T11:21:00.275898+0000","last_became_active":"2023-11-28T11:20:56.263627+0000","last_became_peered":"2023-11-28T11:20:56.263627+0000","last_unstale":"2023-11-28T11:21:00.275898+0000","last_undegraded":"2023-11-28T11:21:00.275898+0000","last_fullsized":"2023-11-28T11:21:00.275898+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:08.079216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.300168+0000","last_change":"2023-11-28T11:20:56.274116+0000","last_active":"2023-11-28T11:21:00.300168+0000","last_peered":"2023-11-28T11:21:00.300168+0000","last_clean":"2023-11-28T11:21:00.300168+0000","last_became_active":"2023-11-28T11:20:56.273403+0000","last_became_peered":"2023-11-28T11:20:56.273403+0000","last_unstale":"2023-11-28T11:21:00.300168+0000","last_undegraded":"2023-11-28T11:21:00.300168+0000","last_fullsized":"2023-11-28T11:21:00.300168+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:45:14.478416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.291080+0000","last_change":"2023-11-28T11:20:56.259411+0000","last_active":"2023-11-28T11:21:00.291080+0000","last_peered":"2023-11-28T11:21:00.291080+0000","last_clean":"2023-11-28T11:21:00.291080+0000","last_became_active":"2023-11-28T11:20:56.259189+0000","last_became_peered":"2023-11-28T11:20:56.259189+0000","last_unstale":"2023-11-28T11:21:00.291080+0000","last_undegraded":"2023-11-28T11:21:00.291080+0000","last_fullsized":"2023-11-28T11:21:00.291080+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:24:07.384730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.270889+0000","last_change":"2023-11-28T11:20:56.257033+0000","last_active":"2023-11-28T11:21:00.270889+0000","last_peered":"2023-11-28T11:21:00.270889+0000","last_clean":"2023-11-28T11:21:00.270889+0000","last_became_active":"2023-11-28T11:20:56.256549+0000","last_became_peered":"2023-11-28T11:20:56.256549+0000","last_unstale":"2023-11-28T11:21:00.270889+0000","last_undegraded":"2023-11-28T11:21:00.270889+0000","last_fullsized":"2023-11-28T11:21:00.270889+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:24:07.384730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.286598+0000","last_change":"2023-11-28T11:20:56.270979+0000","last_active":"2023-11-28T11:21:00.286598+0000","last_peered":"2023-11-28T11:21:00.286598+0000","last_clean":"2023-11-28T11:21:00.286598+0000","last_became_active":"2023-11-28T11:20:56.270710+0000","last_became_peered":"2023-11-28T11:20:56.270710+0000","last_unstale":"2023-11-28T11:21:00.286598+0000","last_undegraded":"2023-11-28T11:21:00.286598+0000","last_fullsized":"2023-11-28T11:21:00.286598+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:42:57.864956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.290274+0000","last_change":"2023-11-28T11:20:56.262561+0000","last_active":"2023-11-28T11:21:00.290274+0000","last_peered":"2023-11-28T11:21:00.290274+0000","last_clean":"2023-11-28T11:21:00.290274+0000","last_became_active":"2023-11-28T11:20:56.262251+0000","last_became_peered":"2023-11-28T11:20:56.262251+0000","last_unstale":"2023-11-28T11:21:00.290274+0000","last_undegraded":"2023-11-28T11:21:00.290274+0000","last_fullsized":"2023-11-28T11:21:00.290274+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:11:36.091056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.292352+0000","last_change":"2023-11-28T11:20:56.261421+0000","last_active":"2023-11-28T11:21:00.292352+0000","last_peered":"2023-11-28T11:21:00.292352+0000","last_clean":"2023-11-28T11:21:00.292352+0000","last_became_active":"2023-11-28T11:20:56.259655+0000","last_became_peered":"2023-11-28T11:20:56.259655+0000","last_unstale":"2023-11-28T11:21:00.292352+0000","last_undegraded":"2023-11-28T11:21:00.292352+0000","last_fullsized":"2023-11-28T11:21:00.292352+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:01:57.936285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.273393+0000","last_change":"2023-11-28T11:20:56.258358+0000","last_active":"2023-11-28T11:21:00.273393+0000","last_peered":"2023-11-28T11:21:00.273393+0000","last_clean":"2023-11-28T11:21:00.273393+0000","last_became_active":"2023-11-28T11:20:56.257895+0000","last_became_peered":"2023-11-28T11:20:56.257895+0000","last_unstale":"2023-11-28T11:21:00.273393+0000","last_undegraded":"2023-11-28T11:21:00.273393+0000","last_fullsized":"2023-11-28T11:21:00.273393+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:08.079216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.777639+0000","last_change":"2023-11-28T11:21:00.306498+0000","last_active":"2023-11-28T11:21:00.777639+0000","last_peered":"2023-11-28T11:21:00.777639+0000","last_clean":"2023-11-28T11:21:00.777639+0000","last_became_active":"2023-11-28T11:21:00.305632+0000","last_became_peered":"2023-11-28T11:21:00.305632+0000","last_unstale":"2023-11-28T11:21:00.777639+0000","last_undegraded":"2023-11-28T11:21:00.777639+0000","last_fullsized":"2023-11-28T11:21:00.777639+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:08:42.028668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.274765+0000","last_change":"2023-11-28T11:20:55.761598+0000","last_active":"2023-11-28T11:21:00.274765+0000","last_peered":"2023-11-28T11:21:00.274765+0000","last_clean":"2023-11-28T11:21:00.274765+0000","last_became_active":"2023-11-28T11:20:55.761257+0000","last_became_peered":"2023-11-28T11:20:55.761257+0000","last_unstale":"2023-11-28T11:21:00.274765+0000","last_undegraded":"2023-11-28T11:21:00.274765+0000","last_fullsized":"2023-11-28T11:21:00.274765+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:13:32.428410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.271441+0000","last_change":"2023-11-28T11:20:56.269300+0000","last_active":"2023-11-28T11:21:00.271441+0000","last_peered":"2023-11-28T11:21:00.271441+0000","last_clean":"2023-11-28T11:21:00.271441+0000","last_became_active":"2023-11-28T11:20:56.269068+0000","last_became_peered":"2023-11-28T11:20:56.269068+0000","last_unstale":"2023-11-28T11:21:00.271441+0000","last_undegraded":"2023-11-28T11:21:00.271441+0000","last_fullsized":"2023-11-28T11:21:00.271441+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:42:57.864956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.779188+0000","last_change":"2023-11-28T11:21:00.303150+0000","last_active":"2023-11-28T11:21:00.779188+0000","last_peered":"2023-11-28T11:21:00.779188+0000","last_clean":"2023-11-28T11:21:00.779188+0000","last_became_active":"2023-11-28T11:21:00.302669+0000","last_became_peered":"2023-11-28T11:21:00.302669+0000","last_unstale":"2023-11-28T11:21:00.779188+0000","last_undegraded":"2023-11-28T11:21:00.779188+0000","last_fullsized":"2023-11-28T11:21:00.779188+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:26:35.867941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.284078+0000","last_change":"2023-11-28T11:20:55.769854+0000","last_active":"2023-11-28T11:21:00.284078+0000","last_peered":"2023-11-28T11:21:00.284078+0000","last_clean":"2023-11-28T11:21:00.284078+0000","last_became_active":"2023-11-28T11:20:55.769322+0000","last_became_peered":"2023-11-28T11:20:55.769322+0000","last_unstale":"2023-11-28T11:21:00.284078+0000","last_undegraded":"2023-11-28T11:21:00.284078+0000","last_fullsized":"2023-11-28T11:21:00.284078+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:37:05.627049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.291807+0000","last_change":"2023-11-28T11:20:56.265137+0000","last_active":"2023-11-28T11:21:00.291807+0000","last_peered":"2023-11-28T11:21:00.291807+0000","last_clean":"2023-11-28T11:21:00.291807+0000","last_became_active":"2023-11-28T11:20:56.264228+0000","last_became_peered":"2023-11-28T11:20:56.264228+0000","last_unstale":"2023-11-28T11:21:00.291807+0000","last_undegraded":"2023-11-28T11:21:00.291807+0000","last_fullsized":"2023-11-28T11:21:00.291807+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:42:57.864956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.884947+0000","last_change":"2023-11-28T11:21:00.314686+0000","last_active":"2023-11-28T11:21:01.884947+0000","last_peered":"2023-11-28T11:21:01.884947+0000","last_clean":"2023-11-28T11:21:01.884947+0000","last_became_active":"2023-11-28T11:21:00.314281+0000","last_became_peered":"2023-11-28T11:21:00.314281+0000","last_unstale":"2023-11-28T11:21:01.884947+0000","last_undegraded":"2023-11-28T11:21:01.884947+0000","last_fullsized":"2023-11-28T11:21:01.884947+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:29:41.143865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.271248+0000","last_change":"2023-11-28T11:20:55.768237+0000","last_active":"2023-11-28T11:21:00.271248+0000","last_peered":"2023-11-28T11:21:00.271248+0000","last_clean":"2023-11-28T11:21:00.271248+0000","last_became_active":"2023-11-28T11:20:55.767948+0000","last_became_peered":"2023-11-28T11:20:55.767948+0000","last_unstale":"2023-11-28T11:21:00.271248+0000","last_undegraded":"2023-11-28T11:21:00.271248+0000","last_fullsized":"2023-11-28T11:21:00.271248+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:23:36.801918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.286831+0000","last_change":"2023-11-28T11:20:56.260663+0000","last_active":"2023-11-28T11:21:00.286831+0000","last_peered":"2023-11-28T11:21:00.286831+0000","last_clean":"2023-11-28T11:21:00.286831+0000","last_became_active":"2023-11-28T11:20:56.259825+0000","last_became_peered":"2023-11-28T11:20:56.259825+0000","last_unstale":"2023-11-28T11:21:00.286831+0000","last_undegraded":"2023-11-28T11:21:00.286831+0000","last_fullsized":"2023-11-28T11:21:00.286831+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:45:09.586958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.778585+0000","last_change":"2023-11-28T11:21:00.303720+0000","last_active":"2023-11-28T11:21:00.778585+0000","last_peered":"2023-11-28T11:21:00.778585+0000","last_clean":"2023-11-28T11:21:00.778585+0000","last_became_active":"2023-11-28T11:21:00.302585+0000","last_became_peered":"2023-11-28T11:21:00.302585+0000","last_unstale":"2023-11-28T11:21:00.778585+0000","last_undegraded":"2023-11-28T11:21:00.778585+0000","last_fullsized":"2023-11-28T11:21:00.778585+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:28:33.645294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.292130+0000","last_change":"2023-11-28T11:20:55.764526+0000","last_active":"2023-11-28T11:21:00.292130+0000","last_peered":"2023-11-28T11:21:00.292130+0000","last_clean":"2023-11-28T11:21:00.292130+0000","last_became_active":"2023-11-28T11:20:55.764278+0000","last_became_peered":"2023-11-28T11:20:55.764278+0000","last_unstale":"2023-11-28T11:21:00.292130+0000","last_undegraded":"2023-11-28T11:21:00.292130+0000","last_fullsized":"2023-11-28T11:21:00.292130+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:23:36.801918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.271155+0000","last_change":"2023-11-28T11:20:56.265473+0000","last_active":"2023-11-28T11:21:00.271155+0000","last_peered":"2023-11-28T11:21:00.271155+0000","last_clean":"2023-11-28T11:21:00.271155+0000","last_became_active":"2023-11-28T11:20:56.262049+0000","last_became_peered":"2023-11-28T11:20:56.262049+0000","last_unstale":"2023-11-28T11:21:00.271155+0000","last_undegraded":"2023-11-28T11:21:00.271155+0000","last_fullsized":"2023-11-28T11:21:00.271155+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:08.079216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.806510+0000","last_change":"2023-11-28T11:21:00.306448+0000","last_active":"2023-11-28T11:21:01.806510+0000","last_peered":"2023-11-28T11:21:01.806510+0000","last_clean":"2023-11-28T11:21:01.806510+0000","last_became_active":"2023-11-28T11:21:00.305892+0000","last_became_peered":"2023-11-28T11:21:00.305892+0000","last_unstale":"2023-11-28T11:21:01.806510+0000","last_undegraded":"2023-11-28T11:21:01.806510+0000","last_fullsized":"2023-11-28T11:21:01.806510+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:41.700110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.277223+0000","last_change":"2023-11-28T11:20:55.764162+0000","last_active":"2023-11-28T11:21:00.277223+0000","last_peered":"2023-11-28T11:21:00.277223+0000","last_clean":"2023-11-28T11:21:00.277223+0000","last_became_active":"2023-11-28T11:20:55.760490+0000","last_became_peered":"2023-11-28T11:20:55.760490+0000","last_unstale":"2023-11-28T11:21:00.277223+0000","last_undegraded":"2023-11-28T11:21:00.277223+0000","last_fullsized":"2023-11-28T11:21:00.277223+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:39:27.467031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.271241+0000","last_change":"2023-11-28T11:20:56.269435+0000","last_active":"2023-11-28T11:21:00.271241+0000","last_peered":"2023-11-28T11:21:00.271241+0000","last_clean":"2023-11-28T11:21:00.271241+0000","last_became_active":"2023-11-28T11:20:56.269179+0000","last_became_peered":"2023-11-28T11:20:56.269179+0000","last_unstale":"2023-11-28T11:21:00.271241+0000","last_undegraded":"2023-11-28T11:21:00.271241+0000","last_fullsized":"2023-11-28T11:21:00.271241+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:50:45.228070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.844101+0000","last_change":"2023-11-28T11:21:00.292120+0000","last_active":"2023-11-28T11:21:01.844101+0000","last_peered":"2023-11-28T11:21:01.844101+0000","last_clean":"2023-11-28T11:21:01.844101+0000","last_became_active":"2023-11-28T11:21:00.289430+0000","last_became_peered":"2023-11-28T11:21:00.289430+0000","last_unstale":"2023-11-28T11:21:01.844101+0000","last_undegraded":"2023-11-28T11:21:01.844101+0000","last_fullsized":"2023-11-28T11:21:01.844101+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:41.700110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.298184+0000","last_change":"2023-11-28T11:20:55.771668+0000","last_active":"2023-11-28T11:21:00.298184+0000","last_peered":"2023-11-28T11:21:00.298184+0000","last_clean":"2023-11-28T11:21:00.298184+0000","last_became_active":"2023-11-28T11:20:55.771400+0000","last_became_peered":"2023-11-28T11:20:55.771400+0000","last_unstale":"2023-11-28T11:21:00.298184+0000","last_undegraded":"2023-11-28T11:21:00.298184+0000","last_fullsized":"2023-11-28T11:21:00.298184+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:27:11.291583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.290416+0000","last_change":"2023-11-28T11:20:56.263283+0000","last_active":"2023-11-28T11:21:00.290416+0000","last_peered":"2023-11-28T11:21:00.290416+0000","last_clean":"2023-11-28T11:21:00.290416+0000","last_became_active":"2023-11-28T11:20:56.262787+0000","last_became_peered":"2023-11-28T11:20:56.262787+0000","last_unstale":"2023-11-28T11:21:00.290416+0000","last_undegraded":"2023-11-28T11:21:00.290416+0000","last_fullsized":"2023-11-28T11:21:00.290416+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:40:42.059656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.883480+0000","last_change":"2023-11-28T11:21:00.297947+0000","last_active":"2023-11-28T11:21:01.883480+0000","last_peered":"2023-11-28T11:21:01.883480+0000","last_clean":"2023-11-28T11:21:01.883480+0000","last_became_active":"2023-11-28T11:21:00.297320+0000","last_became_peered":"2023-11-28T11:21:00.297320+0000","last_unstale":"2023-11-28T11:21:01.883480+0000","last_undegraded":"2023-11-28T11:21:01.883480+0000","last_fullsized":"2023-11-28T11:21:01.883480+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:56:39.135021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"27'10","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:13.462618+0000","last_change":"2023-11-28T11:20:55.761529+0000","last_active":"2023-11-28T11:21:13.462618+0000","last_peered":"2023-11-28T11:21:13.462618+0000","last_clean":"2023-11-28T11:21:13.462618+0000","last_became_active":"2023-11-28T11:20:55.759570+0000","last_became_peered":"2023-11-28T11:20:55.759570+0000","last_unstale":"2023-11-28T11:21:13.462618+0000","last_undegraded":"2023-11-28T11:21:13.462618+0000","last_fullsized":"2023-11-28T11:21:13.462618+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:54:56.683286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3746,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.272939+0000","last_change":"2023-11-28T11:20:56.280987+0000","last_active":"2023-11-28T11:21:00.272939+0000","last_peered":"2023-11-28T11:21:00.272939+0000","last_clean":"2023-11-28T11:21:00.272939+0000","last_became_active":"2023-11-28T11:20:56.279820+0000","last_became_peered":"2023-11-28T11:20:56.279820+0000","last_unstale":"2023-11-28T11:21:00.272939+0000","last_undegraded":"2023-11-28T11:21:00.272939+0000","last_fullsized":"2023-11-28T11:21:00.272939+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:50:45.228070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.383399+0000","last_change":"2023-11-28T11:21:00.294226+0000","last_active":"2023-11-28T11:21:01.383399+0000","last_peered":"2023-11-28T11:21:01.383399+0000","last_clean":"2023-11-28T11:21:01.383399+0000","last_became_active":"2023-11-28T11:21:00.293015+0000","last_became_peered":"2023-11-28T11:21:00.293015+0000","last_unstale":"2023-11-28T11:21:01.383399+0000","last_undegraded":"2023-11-28T11:21:01.383399+0000","last_fullsized":"2023-11-28T11:21:01.383399+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:35:00.472959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.298041+0000","last_change":"2023-11-28T11:20:55.766180+0000","last_active":"2023-11-28T11:21:00.298041+0000","last_peered":"2023-11-28T11:21:00.298041+0000","last_clean":"2023-11-28T11:21:00.298041+0000","last_became_active":"2023-11-28T11:20:55.765845+0000","last_became_peered":"2023-11-28T11:20:55.765845+0000","last_unstale":"2023-11-28T11:21:00.298041+0000","last_undegraded":"2023-11-28T11:21:00.298041+0000","last_fullsized":"2023-11-28T11:21:00.298041+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:39:27.467031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.271449+0000","last_change":"2023-11-28T11:20:56.268499+0000","last_active":"2023-11-28T11:21:00.271449+0000","last_peered":"2023-11-28T11:21:00.271449+0000","last_clean":"2023-11-28T11:21:00.271449+0000","last_became_active":"2023-11-28T11:20:56.267802+0000","last_became_peered":"2023-11-28T11:20:56.267802+0000","last_unstale":"2023-11-28T11:21:00.271449+0000","last_undegraded":"2023-11-28T11:21:00.271449+0000","last_fullsized":"2023-11-28T11:21:00.271449+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:11:36.091056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.844455+0000","last_change":"2023-11-28T11:21:00.302400+0000","last_active":"2023-11-28T11:21:01.844455+0000","last_peered":"2023-11-28T11:21:01.844455+0000","last_clean":"2023-11-28T11:21:01.844455+0000","last_became_active":"2023-11-28T11:21:00.301539+0000","last_became_peered":"2023-11-28T11:21:00.301539+0000","last_unstale":"2023-11-28T11:21:01.844455+0000","last_undegraded":"2023-11-28T11:21:01.844455+0000","last_fullsized":"2023-11-28T11:21:01.844455+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:53:42.920160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.290955+0000","last_change":"2023-11-28T11:20:55.766030+0000","last_active":"2023-11-28T11:21:00.290955+0000","last_peered":"2023-11-28T11:21:00.290955+0000","last_clean":"2023-11-28T11:21:00.290955+0000","last_became_active":"2023-11-28T11:20:55.765787+0000","last_became_peered":"2023-11-28T11:20:55.765787+0000","last_unstale":"2023-11-28T11:21:00.290955+0000","last_undegraded":"2023-11-28T11:21:00.290955+0000","last_fullsized":"2023-11-28T11:21:00.290955+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:58:46.726052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.273291+0000","last_change":"2023-11-28T11:20:56.258188+0000","last_active":"2023-11-28T11:21:00.273291+0000","last_peered":"2023-11-28T11:21:00.273291+0000","last_clean":"2023-11-28T11:21:00.273291+0000","last_became_active":"2023-11-28T11:20:56.257558+0000","last_became_peered":"2023-11-28T11:20:56.257558+0000","last_unstale":"2023-11-28T11:21:00.273291+0000","last_undegraded":"2023-11-28T11:21:00.273291+0000","last_fullsized":"2023-11-28T11:21:00.273291+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:11:36.091056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.844327+0000","last_change":"2023-11-28T11:21:00.307457+0000","last_active":"2023-11-28T11:21:01.844327+0000","last_peered":"2023-11-28T11:21:01.844327+0000","last_clean":"2023-11-28T11:21:01.844327+0000","last_became_active":"2023-11-28T11:21:00.307044+0000","last_became_peered":"2023-11-28T11:21:00.307044+0000","last_unstale":"2023-11-28T11:21:01.844327+0000","last_undegraded":"2023-11-28T11:21:01.844327+0000","last_fullsized":"2023-11-28T11:21:01.844327+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:35:00.472959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.288230+0000","last_change":"2023-11-28T11:20:55.765803+0000","last_active":"2023-11-28T11:21:00.288230+0000","last_peered":"2023-11-28T11:21:00.288230+0000","last_clean":"2023-11-28T11:21:00.288230+0000","last_became_active":"2023-11-28T11:20:55.765501+0000","last_became_peered":"2023-11-28T11:20:55.765501+0000","last_unstale":"2023-11-28T11:21:00.288230+0000","last_undegraded":"2023-11-28T11:21:00.288230+0000","last_fullsized":"2023-11-28T11:21:00.288230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:09:40.747993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.291211+0000","last_change":"2023-11-28T11:20:56.268222+0000","last_active":"2023-11-28T11:21:00.291211+0000","last_peered":"2023-11-28T11:21:00.291211+0000","last_clean":"2023-11-28T11:21:00.291211+0000","last_became_active":"2023-11-28T11:20:56.267973+0000","last_became_peered":"2023-11-28T11:20:56.267973+0000","last_unstale":"2023-11-28T11:21:00.291211+0000","last_undegraded":"2023-11-28T11:21:00.291211+0000","last_fullsized":"2023-11-28T11:21:00.291211+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:40:42.059656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.383115+0000","last_change":"2023-11-28T11:21:00.291222+0000","last_active":"2023-11-28T11:21:01.383115+0000","last_peered":"2023-11-28T11:21:01.383115+0000","last_clean":"2023-11-28T11:21:01.383115+0000","last_became_active":"2023-11-28T11:21:00.290768+0000","last_became_peered":"2023-11-28T11:21:00.290768+0000","last_unstale":"2023-11-28T11:21:01.383115+0000","last_undegraded":"2023-11-28T11:21:01.383115+0000","last_fullsized":"2023-11-28T11:21:01.383115+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:56:39.135021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.286149+0000","last_change":"2023-11-28T11:20:55.766427+0000","last_active":"2023-11-28T11:21:00.286149+0000","last_peered":"2023-11-28T11:21:00.286149+0000","last_clean":"2023-11-28T11:21:00.286149+0000","last_became_active":"2023-11-28T11:20:55.764723+0000","last_became_peered":"2023-11-28T11:20:55.764723+0000","last_unstale":"2023-11-28T11:21:00.286149+0000","last_undegraded":"2023-11-28T11:21:00.286149+0000","last_fullsized":"2023-11-28T11:21:00.286149+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:54:56.683286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.272205+0000","last_change":"2023-11-28T11:20:56.274667+0000","last_active":"2023-11-28T11:21:00.272205+0000","last_peered":"2023-11-28T11:21:00.272205+0000","last_clean":"2023-11-28T11:21:00.272205+0000","last_became_active":"2023-11-28T11:20:56.274325+0000","last_became_peered":"2023-11-28T11:20:56.274325+0000","last_unstale":"2023-11-28T11:21:00.272205+0000","last_undegraded":"2023-11-28T11:21:00.272205+0000","last_fullsized":"2023-11-28T11:21:00.272205+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:33:54.639487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.807008+0000","last_change":"2023-11-28T11:21:00.301042+0000","last_active":"2023-11-28T11:21:01.807008+0000","last_peered":"2023-11-28T11:21:01.807008+0000","last_clean":"2023-11-28T11:21:01.807008+0000","last_became_active":"2023-11-28T11:21:00.300581+0000","last_became_peered":"2023-11-28T11:21:00.300581+0000","last_unstale":"2023-11-28T11:21:01.807008+0000","last_undegraded":"2023-11-28T11:21:01.807008+0000","last_fullsized":"2023-11-28T11:21:01.807008+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:21:12.545633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.298437+0000","last_change":"2023-11-28T11:20:55.774510+0000","last_active":"2023-11-28T11:21:00.298437+0000","last_peered":"2023-11-28T11:21:00.298437+0000","last_clean":"2023-11-28T11:21:00.298437+0000","last_became_active":"2023-11-28T11:20:55.774211+0000","last_became_peered":"2023-11-28T11:20:55.774211+0000","last_unstale":"2023-11-28T11:21:00.298437+0000","last_undegraded":"2023-11-28T11:21:00.298437+0000","last_fullsized":"2023-11-28T11:21:00.298437+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:47:23.879025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.273372+0000","last_change":"2023-11-28T11:20:56.263432+0000","last_active":"2023-11-28T11:21:00.273372+0000","last_peered":"2023-11-28T11:21:00.273372+0000","last_clean":"2023-11-28T11:21:00.273372+0000","last_became_active":"2023-11-28T11:20:56.263125+0000","last_became_peered":"2023-11-28T11:20:56.263125+0000","last_unstale":"2023-11-28T11:21:00.273372+0000","last_undegraded":"2023-11-28T11:21:00.273372+0000","last_fullsized":"2023-11-28T11:21:00.273372+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:40:42.059656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.774944+0000","last_change":"2023-11-28T11:21:00.291421+0000","last_active":"2023-11-28T11:21:00.774944+0000","last_peered":"2023-11-28T11:21:00.774944+0000","last_clean":"2023-11-28T11:21:00.774944+0000","last_became_active":"2023-11-28T11:21:00.291077+0000","last_became_peered":"2023-11-28T11:21:00.291077+0000","last_unstale":"2023-11-28T11:21:00.774944+0000","last_undegraded":"2023-11-28T11:21:00.774944+0000","last_fullsized":"2023-11-28T11:21:00.774944+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:26:35.867941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.298743+0000","last_change":"2023-11-28T11:20:55.774653+0000","last_active":"2023-11-28T11:21:00.298743+0000","last_peered":"2023-11-28T11:21:00.298743+0000","last_clean":"2023-11-28T11:21:00.298743+0000","last_became_active":"2023-11-28T11:20:55.774352+0000","last_became_peered":"2023-11-28T11:20:55.774352+0000","last_unstale":"2023-11-28T11:21:00.298743+0000","last_undegraded":"2023-11-28T11:21:00.298743+0000","last_fullsized":"2023-11-28T11:21:00.298743+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:33:55.577608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.274658+0000","last_change":"2023-11-28T11:20:56.266409+0000","last_active":"2023-11-28T11:21:00.274658+0000","last_peered":"2023-11-28T11:21:00.274658+0000","last_clean":"2023-11-28T11:21:00.274658+0000","last_became_active":"2023-11-28T11:20:56.266091+0000","last_became_peered":"2023-11-28T11:20:56.266091+0000","last_unstale":"2023-11-28T11:21:00.274658+0000","last_undegraded":"2023-11-28T11:21:00.274658+0000","last_fullsized":"2023-11-28T11:21:00.274658+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:28:30.120683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.775175+0000","last_change":"2023-11-28T11:21:00.311439+0000","last_active":"2023-11-28T11:21:00.775175+0000","last_peered":"2023-11-28T11:21:00.775175+0000","last_clean":"2023-11-28T11:21:00.775175+0000","last_became_active":"2023-11-28T11:21:00.308732+0000","last_became_peered":"2023-11-28T11:21:00.308732+0000","last_unstale":"2023-11-28T11:21:00.775175+0000","last_undegraded":"2023-11-28T11:21:00.775175+0000","last_fullsized":"2023-11-28T11:21:00.775175+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:53:19.245981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.298398+0000","last_change":"2023-11-28T11:20:55.771105+0000","last_active":"2023-11-28T11:21:00.298398+0000","last_peered":"2023-11-28T11:21:00.298398+0000","last_clean":"2023-11-28T11:21:00.298398+0000","last_became_active":"2023-11-28T11:20:55.770583+0000","last_became_peered":"2023-11-28T11:20:55.770583+0000","last_unstale":"2023-11-28T11:21:00.298398+0000","last_undegraded":"2023-11-28T11:21:00.298398+0000","last_fullsized":"2023-11-28T11:21:00.298398+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:06:01.854141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.286992+0000","last_change":"2023-11-28T11:20:56.263719+0000","last_active":"2023-11-28T11:21:00.286992+0000","last_peered":"2023-11-28T11:21:00.286992+0000","last_clean":"2023-11-28T11:21:00.286992+0000","last_became_active":"2023-11-28T11:20:56.263444+0000","last_became_peered":"2023-11-28T11:20:56.263444+0000","last_unstale":"2023-11-28T11:21:00.286992+0000","last_undegraded":"2023-11-28T11:21:00.286992+0000","last_fullsized":"2023-11-28T11:21:00.286992+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:33:56.583532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.775522+0000","last_change":"2023-11-28T11:21:00.294534+0000","last_active":"2023-11-28T11:21:00.775522+0000","last_peered":"2023-11-28T11:21:00.775522+0000","last_clean":"2023-11-28T11:21:00.775522+0000","last_became_active":"2023-11-28T11:21:00.294055+0000","last_became_peered":"2023-11-28T11:21:00.294055+0000","last_unstale":"2023-11-28T11:21:00.775522+0000","last_undegraded":"2023-11-28T11:21:00.775522+0000","last_fullsized":"2023-11-28T11:21:00.775522+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:21:24.577104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.275166+0000","last_change":"2023-11-28T11:20:55.763501+0000","last_active":"2023-11-28T11:21:00.275166+0000","last_peered":"2023-11-28T11:21:00.275166+0000","last_clean":"2023-11-28T11:21:00.275166+0000","last_became_active":"2023-11-28T11:20:55.763188+0000","last_became_peered":"2023-11-28T11:20:55.763188+0000","last_unstale":"2023-11-28T11:21:00.275166+0000","last_undegraded":"2023-11-28T11:21:00.275166+0000","last_fullsized":"2023-11-28T11:21:00.275166+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:58:46.726052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.290571+0000","last_change":"2023-11-28T11:20:56.260904+0000","last_active":"2023-11-28T11:21:00.290571+0000","last_peered":"2023-11-28T11:21:00.290571+0000","last_clean":"2023-11-28T11:21:00.290571+0000","last_became_active":"2023-11-28T11:20:56.260584+0000","last_became_peered":"2023-11-28T11:20:56.260584+0000","last_unstale":"2023-11-28T11:21:00.290571+0000","last_undegraded":"2023-11-28T11:21:00.290571+0000","last_fullsized":"2023-11-28T11:21:00.290571+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:08.079216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.775218+0000","last_change":"2023-11-28T11:21:00.299500+0000","last_active":"2023-11-28T11:21:00.775218+0000","last_peered":"2023-11-28T11:21:00.775218+0000","last_clean":"2023-11-28T11:21:00.775218+0000","last_became_active":"2023-11-28T11:21:00.299065+0000","last_became_peered":"2023-11-28T11:21:00.299065+0000","last_unstale":"2023-11-28T11:21:00.775218+0000","last_undegraded":"2023-11-28T11:21:00.775218+0000","last_fullsized":"2023-11-28T11:21:00.775218+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:59:57.725780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.271583+0000","last_change":"2023-11-28T11:20:55.767855+0000","last_active":"2023-11-28T11:21:00.271583+0000","last_peered":"2023-11-28T11:21:00.271583+0000","last_clean":"2023-11-28T11:21:00.271583+0000","last_became_active":"2023-11-28T11:20:55.767418+0000","last_became_peered":"2023-11-28T11:20:55.767418+0000","last_unstale":"2023-11-28T11:21:00.271583+0000","last_undegraded":"2023-11-28T11:21:00.271583+0000","last_fullsized":"2023-11-28T11:21:00.271583+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:06:11.286885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.275680+0000","last_change":"2023-11-28T11:20:56.282762+0000","last_active":"2023-11-28T11:21:00.275680+0000","last_peered":"2023-11-28T11:21:00.275680+0000","last_clean":"2023-11-28T11:21:00.275680+0000","last_became_active":"2023-11-28T11:20:56.280729+0000","last_became_peered":"2023-11-28T11:20:56.280729+0000","last_unstale":"2023-11-28T11:21:00.275680+0000","last_undegraded":"2023-11-28T11:21:00.275680+0000","last_fullsized":"2023-11-28T11:21:00.275680+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:53:15.721370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.883819+0000","last_change":"2023-11-28T11:21:00.305663+0000","last_active":"2023-11-28T11:21:01.883819+0000","last_peered":"2023-11-28T11:21:01.883819+0000","last_clean":"2023-11-28T11:21:01.883819+0000","last_became_active":"2023-11-28T11:21:00.305308+0000","last_became_peered":"2023-11-28T11:21:00.305308+0000","last_unstale":"2023-11-28T11:21:01.883819+0000","last_undegraded":"2023-11-28T11:21:01.883819+0000","last_fullsized":"2023-11-28T11:21:01.883819+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:05:27.287399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.287091+0000","last_change":"2023-11-28T11:20:55.774271+0000","last_active":"2023-11-28T11:21:00.287091+0000","last_peered":"2023-11-28T11:21:00.287091+0000","last_clean":"2023-11-28T11:21:00.287091+0000","last_became_active":"2023-11-28T11:20:55.773041+0000","last_became_peered":"2023-11-28T11:20:55.773041+0000","last_unstale":"2023-11-28T11:21:00.287091+0000","last_undegraded":"2023-11-28T11:21:00.287091+0000","last_fullsized":"2023-11-28T11:21:00.287091+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:06:11.286885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.273977+0000","last_change":"2023-11-28T11:20:56.269860+0000","last_active":"2023-11-28T11:21:00.273977+0000","last_peered":"2023-11-28T11:21:00.273977+0000","last_clean":"2023-11-28T11:21:00.273977+0000","last_became_active":"2023-11-28T11:20:56.269355+0000","last_became_peered":"2023-11-28T11:20:56.269355+0000","last_unstale":"2023-11-28T11:21:00.273977+0000","last_undegraded":"2023-11-28T11:21:00.273977+0000","last_fullsized":"2023-11-28T11:21:00.273977+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:24:07.384730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.383366+0000","last_change":"2023-11-28T11:21:00.297532+0000","last_active":"2023-11-28T11:21:01.383366+0000","last_peered":"2023-11-28T11:21:01.383366+0000","last_clean":"2023-11-28T11:21:01.383366+0000","last_became_active":"2023-11-28T11:21:00.295523+0000","last_became_peered":"2023-11-28T11:21:00.295523+0000","last_unstale":"2023-11-28T11:21:01.383366+0000","last_undegraded":"2023-11-28T11:21:01.383366+0000","last_fullsized":"2023-11-28T11:21:01.383366+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:41.700110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.274453+0000","last_change":"2023-11-28T11:20:55.764095+0000","last_active":"2023-11-28T11:21:00.274453+0000","last_peered":"2023-11-28T11:21:00.274453+0000","last_clean":"2023-11-28T11:21:00.274453+0000","last_became_active":"2023-11-28T11:20:55.762342+0000","last_became_peered":"2023-11-28T11:20:55.762342+0000","last_unstale":"2023-11-28T11:21:00.274453+0000","last_undegraded":"2023-11-28T11:21:00.274453+0000","last_fullsized":"2023-11-28T11:21:00.274453+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:25:25.914486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.274835+0000","last_change":"2023-11-28T11:20:56.268448+0000","last_active":"2023-11-28T11:21:00.274835+0000","last_peered":"2023-11-28T11:21:00.274835+0000","last_clean":"2023-11-28T11:21:00.274835+0000","last_became_active":"2023-11-28T11:20:56.268150+0000","last_became_peered":"2023-11-28T11:20:56.268150+0000","last_unstale":"2023-11-28T11:21:00.274835+0000","last_undegraded":"2023-11-28T11:21:00.274835+0000","last_fullsized":"2023-11-28T11:21:00.274835+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:21:21.052493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.777908+0000","last_change":"2023-11-28T11:21:00.306271+0000","last_active":"2023-11-28T11:21:00.777908+0000","last_peered":"2023-11-28T11:21:00.777908+0000","last_clean":"2023-11-28T11:21:00.777908+0000","last_became_active":"2023-11-28T11:21:00.305835+0000","last_became_peered":"2023-11-28T11:21:00.305835+0000","last_unstale":"2023-11-28T11:21:00.777908+0000","last_undegraded":"2023-11-28T11:21:00.777908+0000","last_fullsized":"2023-11-28T11:21:00.777908+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:04:07.374394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.270977+0000","last_change":"2023-11-28T11:20:55.762830+0000","last_active":"2023-11-28T11:21:00.270977+0000","last_peered":"2023-11-28T11:21:00.270977+0000","last_clean":"2023-11-28T11:21:00.270977+0000","last_became_active":"2023-11-28T11:20:55.760841+0000","last_became_peered":"2023-11-28T11:20:55.760841+0000","last_unstale":"2023-11-28T11:21:00.270977+0000","last_undegraded":"2023-11-28T11:21:00.270977+0000","last_fullsized":"2023-11-28T11:21:00.270977+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:54:56.683286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.290757+0000","last_change":"2023-11-28T11:20:56.271356+0000","last_active":"2023-11-28T11:21:00.290757+0000","last_peered":"2023-11-28T11:21:00.290757+0000","last_clean":"2023-11-28T11:21:00.290757+0000","last_became_active":"2023-11-28T11:20:56.271110+0000","last_became_peered":"2023-11-28T11:20:56.271110+0000","last_unstale":"2023-11-28T11:21:00.290757+0000","last_undegraded":"2023-11-28T11:21:00.290757+0000","last_fullsized":"2023-11-28T11:21:00.290757+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:08.079216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.884172+0000","last_change":"2023-11-28T11:21:00.295227+0000","last_active":"2023-11-28T11:21:01.884172+0000","last_peered":"2023-11-28T11:21:01.884172+0000","last_clean":"2023-11-28T11:21:01.884172+0000","last_became_active":"2023-11-28T11:21:00.293463+0000","last_became_peered":"2023-11-28T11:21:00.293463+0000","last_unstale":"2023-11-28T11:21:01.884172+0000","last_undegraded":"2023-11-28T11:21:01.884172+0000","last_fullsized":"2023-11-28T11:21:01.884172+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:41.700110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.288397+0000","last_change":"2023-11-28T11:20:51.020016+0000","last_active":"2023-11-28T11:21:00.288397+0000","last_peered":"2023-11-28T11:21:00.288397+0000","last_clean":"2023-11-28T11:21:00.288397+0000","last_became_active":"2023-11-28T11:20:51.019492+0000","last_became_peered":"2023-11-28T11:20:51.019492+0000","last_unstale":"2023-11-28T11:21:00.288397+0000","last_undegraded":"2023-11-28T11:21:00.288397+0000","last_fullsized":"2023-11-28T11:21:00.288397+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:27.884538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.273192+0000","last_change":"2023-11-28T11:20:55.770921+0000","last_active":"2023-11-28T11:21:00.273192+0000","last_peered":"2023-11-28T11:21:00.273192+0000","last_clean":"2023-11-28T11:21:00.273192+0000","last_became_active":"2023-11-28T11:20:55.770693+0000","last_became_peered":"2023-11-28T11:20:55.770693+0000","last_unstale":"2023-11-28T11:21:00.273192+0000","last_undegraded":"2023-11-28T11:21:00.273192+0000","last_fullsized":"2023-11-28T11:21:00.273192+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:37:05.627049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.285261+0000","last_change":"2023-11-28T11:20:56.257635+0000","last_active":"2023-11-28T11:21:00.285261+0000","last_peered":"2023-11-28T11:21:00.285261+0000","last_clean":"2023-11-28T11:21:00.285261+0000","last_became_active":"2023-11-28T11:20:56.255047+0000","last_became_peered":"2023-11-28T11:20:56.255047+0000","last_unstale":"2023-11-28T11:21:00.285261+0000","last_undegraded":"2023-11-28T11:21:00.285261+0000","last_fullsized":"2023-11-28T11:21:00.285261+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:27:12.297507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.884415+0000","last_change":"2023-11-28T11:21:00.311287+0000","last_active":"2023-11-28T11:21:01.884415+0000","last_peered":"2023-11-28T11:21:01.884415+0000","last_clean":"2023-11-28T11:21:01.884415+0000","last_became_active":"2023-11-28T11:21:00.305808+0000","last_became_peered":"2023-11-28T11:21:00.305808+0000","last_unstale":"2023-11-28T11:21:01.884415+0000","last_undegraded":"2023-11-28T11:21:01.884415+0000","last_fullsized":"2023-11-28T11:21:01.884415+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:56:39.135021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.276695+0000","last_change":"2023-11-28T11:20:50.521908+0000","last_active":"2023-11-28T11:21:00.276695+0000","last_peered":"2023-11-28T11:21:00.276695+0000","last_clean":"2023-11-28T11:21:00.276695+0000","last_became_active":"2023-11-28T11:20:50.521059+0000","last_became_peered":"2023-11-28T11:20:50.521059+0000","last_unstale":"2023-11-28T11:21:00.276695+0000","last_undegraded":"2023-11-28T11:21:00.276695+0000","last_fullsized":"2023-11-28T11:21:00.276695+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:05.302498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.273078+0000","last_change":"2023-11-28T11:20:55.767339+0000","last_active":"2023-11-28T11:21:00.273078+0000","last_peered":"2023-11-28T11:21:00.273078+0000","last_clean":"2023-11-28T11:21:00.273078+0000","last_became_active":"2023-11-28T11:20:55.766065+0000","last_became_peered":"2023-11-28T11:20:55.766065+0000","last_unstale":"2023-11-28T11:21:00.273078+0000","last_undegraded":"2023-11-28T11:21:00.273078+0000","last_fullsized":"2023-11-28T11:21:00.273078+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:06:11.286885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.289236+0000","last_change":"2023-11-28T11:20:56.261611+0000","last_active":"2023-11-28T11:21:00.289236+0000","last_peered":"2023-11-28T11:21:00.289236+0000","last_clean":"2023-11-28T11:21:00.289236+0000","last_became_active":"2023-11-28T11:20:56.260557+0000","last_became_peered":"2023-11-28T11:20:56.260557+0000","last_unstale":"2023-11-28T11:21:00.289236+0000","last_undegraded":"2023-11-28T11:21:00.289236+0000","last_fullsized":"2023-11-28T11:21:00.289236+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:14:34.876987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.775674+0000","last_change":"2023-11-28T11:21:00.301087+0000","last_active":"2023-11-28T11:21:00.775674+0000","last_peered":"2023-11-28T11:21:00.775674+0000","last_clean":"2023-11-28T11:21:00.775674+0000","last_became_active":"2023-11-28T11:21:00.300674+0000","last_became_peered":"2023-11-28T11:21:00.300674+0000","last_unstale":"2023-11-28T11:21:00.775674+0000","last_undegraded":"2023-11-28T11:21:00.775674+0000","last_fullsized":"2023-11-28T11:21:00.775674+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:13:50.207611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.282488+0000","last_change":"2023-11-28T11:20:50.619004+0000","last_active":"2023-11-28T11:21:00.282488+0000","last_peered":"2023-11-28T11:21:00.282488+0000","last_clean":"2023-11-28T11:21:00.282488+0000","last_became_active":"2023-11-28T11:20:50.618315+0000","last_became_peered":"2023-11-28T11:20:50.618315+0000","last_unstale":"2023-11-28T11:21:00.282488+0000","last_undegraded":"2023-11-28T11:21:00.282488+0000","last_fullsized":"2023-11-28T11:21:00.282488+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:27.884538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.283998+0000","last_change":"2023-11-28T11:20:55.758384+0000","last_active":"2023-11-28T11:21:00.283998+0000","last_peered":"2023-11-28T11:21:00.283998+0000","last_clean":"2023-11-28T11:21:00.283998+0000","last_became_active":"2023-11-28T11:20:55.758125+0000","last_became_peered":"2023-11-28T11:20:55.758125+0000","last_unstale":"2023-11-28T11:21:00.283998+0000","last_undegraded":"2023-11-28T11:21:00.283998+0000","last_fullsized":"2023-11-28T11:21:00.283998+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:47:38.102588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.290667+0000","last_change":"2023-11-28T11:20:56.253940+0000","last_active":"2023-11-28T11:21:00.290667+0000","last_peered":"2023-11-28T11:21:00.290667+0000","last_clean":"2023-11-28T11:21:00.290667+0000","last_became_active":"2023-11-28T11:20:56.253635+0000","last_became_peered":"2023-11-28T11:20:56.253635+0000","last_unstale":"2023-11-28T11:21:00.290667+0000","last_undegraded":"2023-11-28T11:21:00.290667+0000","last_fullsized":"2023-11-28T11:21:00.290667+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:00:17.958443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.806956+0000","last_change":"2023-11-28T11:21:00.311172+0000","last_active":"2023-11-28T11:21:01.806956+0000","last_peered":"2023-11-28T11:21:01.806956+0000","last_clean":"2023-11-28T11:21:01.806956+0000","last_became_active":"2023-11-28T11:21:00.310707+0000","last_became_peered":"2023-11-28T11:21:00.310707+0000","last_unstale":"2023-11-28T11:21:01.806956+0000","last_undegraded":"2023-11-28T11:21:01.806956+0000","last_fullsized":"2023-11-28T11:21:01.806956+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:41:46.264594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.273954+0000","last_change":"2023-11-28T11:20:50.522582+0000","last_active":"2023-11-28T11:21:00.273954+0000","last_peered":"2023-11-28T11:21:00.273954+0000","last_clean":"2023-11-28T11:21:00.273954+0000","last_became_active":"2023-11-28T11:20:50.522157+0000","last_became_peered":"2023-11-28T11:20:50.522157+0000","last_unstale":"2023-11-28T11:21:00.273954+0000","last_undegraded":"2023-11-28T11:21:00.273954+0000","last_fullsized":"2023-11-28T11:21:00.273954+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:27.884538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.286501+0000","last_change":"2023-11-28T11:20:43.294845+0000","last_active":"2023-11-28T11:21:00.286501+0000","last_peered":"2023-11-28T11:21:00.286501+0000","last_clean":"2023-11-28T11:21:00.286501+0000","last_became_active":"2023-11-28T11:20:43.294483+0000","last_became_peered":"2023-11-28T11:20:43.294483+0000","last_unstale":"2023-11-28T11:21:00.286501+0000","last_undegraded":"2023-11-28T11:21:00.286501+0000","last_fullsized":"2023-11-28T11:21:00.286501+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:42.270512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:42.270512+0000","last_clean_scrub_stamp":"2023-11-28T11:20:42.270512+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:17:03.376680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.270966+0000","last_change":"2023-11-28T11:20:55.767420+0000","last_active":"2023-11-28T11:21:00.270966+0000","last_peered":"2023-11-28T11:21:00.270966+0000","last_clean":"2023-11-28T11:21:00.270966+0000","last_became_active":"2023-11-28T11:20:55.766809+0000","last_became_peered":"2023-11-28T11:20:55.766809+0000","last_unstale":"2023-11-28T11:21:00.270966+0000","last_undegraded":"2023-11-28T11:21:00.270966+0000","last_fullsized":"2023-11-28T11:21:00.270966+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:37:05.627049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.286972+0000","last_change":"2023-11-28T11:20:56.276228+0000","last_active":"2023-11-28T11:21:00.286972+0000","last_peered":"2023-11-28T11:21:00.286972+0000","last_clean":"2023-11-28T11:21:00.286972+0000","last_became_active":"2023-11-28T11:20:56.275905+0000","last_became_peered":"2023-11-28T11:20:56.275905+0000","last_unstale":"2023-11-28T11:21:00.286972+0000","last_undegraded":"2023-11-28T11:21:00.286972+0000","last_fullsized":"2023-11-28T11:21:00.286972+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:47:24.884949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.383236+0000","last_change":"2023-11-28T11:21:00.305959+0000","last_active":"2023-11-28T11:21:01.383236+0000","last_peered":"2023-11-28T11:21:01.383236+0000","last_clean":"2023-11-28T11:21:01.383236+0000","last_became_active":"2023-11-28T11:21:00.305530+0000","last_became_peered":"2023-11-28T11:21:00.305530+0000","last_unstale":"2023-11-28T11:21:01.383236+0000","last_undegraded":"2023-11-28T11:21:01.383236+0000","last_fullsized":"2023-11-28T11:21:01.383236+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:22:04.224877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.286683+0000","last_change":"2023-11-28T11:20:50.526865+0000","last_active":"2023-11-28T11:21:00.286683+0000","last_peered":"2023-11-28T11:21:00.286683+0000","last_clean":"2023-11-28T11:21:00.286683+0000","last_became_active":"2023-11-28T11:20:50.526576+0000","last_became_peered":"2023-11-28T11:20:50.526576+0000","last_unstale":"2023-11-28T11:21:00.286683+0000","last_undegraded":"2023-11-28T11:21:00.286683+0000","last_fullsized":"2023-11-28T11:21:00.286683+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:09:18.460112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.289849+0000","last_change":"2023-11-28T11:20:55.772865+0000","last_active":"2023-11-28T11:21:00.289849+0000","last_peered":"2023-11-28T11:21:00.289849+0000","last_clean":"2023-11-28T11:21:00.289849+0000","last_became_active":"2023-11-28T11:20:55.772559+0000","last_became_peered":"2023-11-28T11:20:55.772559+0000","last_unstale":"2023-11-28T11:21:00.289849+0000","last_undegraded":"2023-11-28T11:21:00.289849+0000","last_fullsized":"2023-11-28T11:21:00.289849+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:37:05.627049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.290171+0000","last_change":"2023-11-28T11:20:56.268570+0000","last_active":"2023-11-28T11:21:00.290171+0000","last_peered":"2023-11-28T11:21:00.290171+0000","last_clean":"2023-11-28T11:21:00.290171+0000","last_became_active":"2023-11-28T11:20:56.264598+0000","last_became_peered":"2023-11-28T11:20:56.264598+0000","last_unstale":"2023-11-28T11:21:00.290171+0000","last_undegraded":"2023-11-28T11:21:00.290171+0000","last_fullsized":"2023-11-28T11:21:00.290171+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:14:34.876987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.778849+0000","last_change":"2023-11-28T11:21:00.311861+0000","last_active":"2023-11-28T11:21:00.778849+0000","last_peered":"2023-11-28T11:21:00.778849+0000","last_clean":"2023-11-28T11:21:00.778849+0000","last_became_active":"2023-11-28T11:21:00.311394+0000","last_became_peered":"2023-11-28T11:21:00.311394+0000","last_unstale":"2023-11-28T11:21:00.778849+0000","last_undegraded":"2023-11-28T11:21:00.778849+0000","last_fullsized":"2023-11-28T11:21:00.778849+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:25:55.767021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.298282+0000","last_change":"2023-11-28T11:20:51.020043+0000","last_active":"2023-11-28T11:21:00.298282+0000","last_peered":"2023-11-28T11:21:00.298282+0000","last_clean":"2023-11-28T11:21:00.298282+0000","last_became_active":"2023-11-28T11:20:51.019702+0000","last_became_peered":"2023-11-28T11:20:51.019702+0000","last_unstale":"2023-11-28T11:21:00.298282+0000","last_undegraded":"2023-11-28T11:21:00.298282+0000","last_fullsized":"2023-11-28T11:21:00.298282+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:26:20.246873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.288664+0000","last_change":"2023-11-28T11:20:55.760221+0000","last_active":"2023-11-28T11:21:00.288664+0000","last_peered":"2023-11-28T11:21:00.288664+0000","last_clean":"2023-11-28T11:21:00.288664+0000","last_became_active":"2023-11-28T11:20:55.759960+0000","last_became_peered":"2023-11-28T11:20:55.759960+0000","last_unstale":"2023-11-28T11:21:00.288664+0000","last_undegraded":"2023-11-28T11:21:00.288664+0000","last_fullsized":"2023-11-28T11:21:00.288664+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:54:56.683286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.290072+0000","last_change":"2023-11-28T11:20:56.251820+0000","last_active":"2023-11-28T11:21:00.290072+0000","last_peered":"2023-11-28T11:21:00.290072+0000","last_clean":"2023-11-28T11:21:00.290072+0000","last_became_active":"2023-11-28T11:20:56.251550+0000","last_became_peered":"2023-11-28T11:20:56.251550+0000","last_unstale":"2023-11-28T11:21:00.290072+0000","last_undegraded":"2023-11-28T11:21:00.290072+0000","last_fullsized":"2023-11-28T11:21:00.290072+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:45:14.478416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.777012+0000","last_change":"2023-11-28T11:21:00.298541+0000","last_active":"2023-11-28T11:21:00.777012+0000","last_peered":"2023-11-28T11:21:00.777012+0000","last_clean":"2023-11-28T11:21:00.777012+0000","last_became_active":"2023-11-28T11:21:00.298160+0000","last_became_peered":"2023-11-28T11:21:00.298160+0000","last_unstale":"2023-11-28T11:21:00.777012+0000","last_undegraded":"2023-11-28T11:21:00.777012+0000","last_fullsized":"2023-11-28T11:21:00.777012+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:16:48.936607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.272276+0000","last_change":"2023-11-28T11:20:55.766385+0000","last_active":"2023-11-28T11:21:00.272276+0000","last_peered":"2023-11-28T11:21:00.272276+0000","last_clean":"2023-11-28T11:21:00.272276+0000","last_became_active":"2023-11-28T11:20:55.766148+0000","last_became_peered":"2023-11-28T11:20:55.766148+0000","last_unstale":"2023-11-28T11:21:00.272276+0000","last_undegraded":"2023-11-28T11:21:00.272276+0000","last_fullsized":"2023-11-28T11:21:00.272276+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:33:02.765966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.288602+0000","last_change":"2023-11-28T11:20:50.524251+0000","last_active":"2023-11-28T11:21:00.288602+0000","last_peered":"2023-11-28T11:21:00.288602+0000","last_clean":"2023-11-28T11:21:00.288602+0000","last_became_active":"2023-11-28T11:20:50.524012+0000","last_became_peered":"2023-11-28T11:20:50.524012+0000","last_unstale":"2023-11-28T11:21:00.288602+0000","last_undegraded":"2023-11-28T11:21:00.288602+0000","last_fullsized":"2023-11-28T11:21:00.288602+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:05.302498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.270885+0000","last_change":"2023-11-28T11:20:56.266167+0000","last_active":"2023-11-28T11:21:00.270885+0000","last_peered":"2023-11-28T11:21:00.270885+0000","last_clean":"2023-11-28T11:21:00.270885+0000","last_became_active":"2023-11-28T11:20:56.264890+0000","last_became_peered":"2023-11-28T11:20:56.264890+0000","last_unstale":"2023-11-28T11:21:00.270885+0000","last_undegraded":"2023-11-28T11:21:00.270885+0000","last_fullsized":"2023-11-28T11:21:00.270885+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:31:26.827850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.843672+0000","last_change":"2023-11-28T11:21:00.295193+0000","last_active":"2023-11-28T11:21:01.843672+0000","last_peered":"2023-11-28T11:21:01.843672+0000","last_clean":"2023-11-28T11:21:01.843672+0000","last_became_active":"2023-11-28T11:21:00.294668+0000","last_became_peered":"2023-11-28T11:21:00.294668+0000","last_unstale":"2023-11-28T11:21:01.843672+0000","last_undegraded":"2023-11-28T11:21:01.843672+0000","last_fullsized":"2023-11-28T11:21:01.843672+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:53:58.267145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.287437+0000","last_change":"2023-11-28T11:20:50.524838+0000","last_active":"2023-11-28T11:21:00.287437+0000","last_peered":"2023-11-28T11:21:00.287437+0000","last_clean":"2023-11-28T11:21:00.287437+0000","last_became_active":"2023-11-28T11:20:50.524402+0000","last_became_peered":"2023-11-28T11:20:50.524402+0000","last_unstale":"2023-11-28T11:21:00.287437+0000","last_undegraded":"2023-11-28T11:21:00.287437+0000","last_fullsized":"2023-11-28T11:21:00.287437+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:22.699447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"26'1","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.725299+0000","last_change":"2023-11-28T11:20:55.765528+0000","last_active":"2023-11-28T11:21:00.725299+0000","last_peered":"2023-11-28T11:21:00.725299+0000","last_clean":"2023-11-28T11:21:00.725299+0000","last_became_active":"2023-11-28T11:20:55.765244+0000","last_became_peered":"2023-11-28T11:20:55.765244+0000","last_unstale":"2023-11-28T11:21:00.725299+0000","last_undegraded":"2023-11-28T11:21:00.725299+0000","last_fullsized":"2023-11-28T11:21:00.725299+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:47:38.102588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.274546+0000","last_change":"2023-11-28T11:20:56.275241+0000","last_active":"2023-11-28T11:21:00.274546+0000","last_peered":"2023-11-28T11:21:00.274546+0000","last_clean":"2023-11-28T11:21:00.274546+0000","last_became_active":"2023-11-28T11:20:56.271190+0000","last_became_peered":"2023-11-28T11:20:56.271190+0000","last_unstale":"2023-11-28T11:21:00.274546+0000","last_undegraded":"2023-11-28T11:21:00.274546+0000","last_fullsized":"2023-11-28T11:21:00.274546+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:45:14.478416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.883997+0000","last_change":"2023-11-28T11:21:00.297029+0000","last_active":"2023-11-28T11:21:01.883997+0000","last_peered":"2023-11-28T11:21:01.883997+0000","last_clean":"2023-11-28T11:21:01.883997+0000","last_became_active":"2023-11-28T11:21:00.296497+0000","last_became_peered":"2023-11-28T11:21:00.296497+0000","last_unstale":"2023-11-28T11:21:01.883997+0000","last_undegraded":"2023-11-28T11:21:01.883997+0000","last_fullsized":"2023-11-28T11:21:01.883997+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:41.700110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.290180+0000","last_change":"2023-11-28T11:20:55.773378+0000","last_active":"2023-11-28T11:21:00.290180+0000","last_peered":"2023-11-28T11:21:00.290180+0000","last_clean":"2023-11-28T11:21:00.290180+0000","last_became_active":"2023-11-28T11:20:55.773095+0000","last_became_peered":"2023-11-28T11:20:55.773095+0000","last_unstale":"2023-11-28T11:21:00.290180+0000","last_undegraded":"2023-11-28T11:21:00.290180+0000","last_fullsized":"2023-11-28T11:21:00.290180+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:54:56.683286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.280606+0000","last_change":"2023-11-28T11:20:56.258626+0000","last_active":"2023-11-28T11:21:00.280606+0000","last_peered":"2023-11-28T11:21:00.280606+0000","last_clean":"2023-11-28T11:21:00.280606+0000","last_became_active":"2023-11-28T11:20:56.258350+0000","last_became_peered":"2023-11-28T11:20:56.258350+0000","last_unstale":"2023-11-28T11:21:00.280606+0000","last_undegraded":"2023-11-28T11:21:00.280606+0000","last_fullsized":"2023-11-28T11:21:00.280606+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:33:03.771889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.806818+0000","last_change":"2023-11-28T11:21:00.306341+0000","last_active":"2023-11-28T11:21:01.806818+0000","last_peered":"2023-11-28T11:21:01.806818+0000","last_clean":"2023-11-28T11:21:01.806818+0000","last_became_active":"2023-11-28T11:21:00.304545+0000","last_became_peered":"2023-11-28T11:21:00.304545+0000","last_unstale":"2023-11-28T11:21:01.806818+0000","last_undegraded":"2023-11-28T11:21:01.806818+0000","last_fullsized":"2023-11-28T11:21:01.806818+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:53:58.267145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.275266+0000","last_change":"2023-11-28T11:20:55.766530+0000","last_active":"2023-11-28T11:21:00.275266+0000","last_peered":"2023-11-28T11:21:00.275266+0000","last_clean":"2023-11-28T11:21:00.275266+0000","last_became_active":"2023-11-28T11:20:55.766205+0000","last_became_peered":"2023-11-28T11:20:55.766205+0000","last_unstale":"2023-11-28T11:21:00.275266+0000","last_undegraded":"2023-11-28T11:21:00.275266+0000","last_fullsized":"2023-11-28T11:21:00.275266+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:33:02.765966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.273248+0000","last_change":"2023-11-28T11:20:56.279673+0000","last_active":"2023-11-28T11:21:00.273248+0000","last_peered":"2023-11-28T11:21:00.273248+0000","last_clean":"2023-11-28T11:21:00.273248+0000","last_became_active":"2023-11-28T11:20:56.276338+0000","last_became_peered":"2023-11-28T11:20:56.276338+0000","last_unstale":"2023-11-28T11:21:00.273248+0000","last_undegraded":"2023-11-28T11:21:00.273248+0000","last_fullsized":"2023-11-28T11:21:00.273248+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:59:54.201169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.382912+0000","last_change":"2023-11-28T11:21:00.308516+0000","last_active":"2023-11-28T11:21:01.382912+0000","last_peered":"2023-11-28T11:21:01.382912+0000","last_clean":"2023-11-28T11:21:01.382912+0000","last_became_active":"2023-11-28T11:21:00.307909+0000","last_became_peered":"2023-11-28T11:21:00.307909+0000","last_unstale":"2023-11-28T11:21:01.382912+0000","last_undegraded":"2023-11-28T11:21:01.382912+0000","last_fullsized":"2023-11-28T11:21:01.382912+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:53:42.920160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.273087+0000","last_change":"2023-11-28T11:20:55.772607+0000","last_active":"2023-11-28T11:21:00.273087+0000","last_peered":"2023-11-28T11:21:00.273087+0000","last_clean":"2023-11-28T11:21:00.273087+0000","last_became_active":"2023-11-28T11:20:55.772335+0000","last_became_peered":"2023-11-28T11:20:55.772335+0000","last_unstale":"2023-11-28T11:21:00.273087+0000","last_undegraded":"2023-11-28T11:21:00.273087+0000","last_fullsized":"2023-11-28T11:21:00.273087+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:23:36.801918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.274556+0000","last_change":"2023-11-28T11:20:56.279846+0000","last_active":"2023-11-28T11:21:00.274556+0000","last_peered":"2023-11-28T11:21:00.274556+0000","last_clean":"2023-11-28T11:21:00.274556+0000","last_became_active":"2023-11-28T11:20:56.276529+0000","last_became_peered":"2023-11-28T11:20:56.276529+0000","last_unstale":"2023-11-28T11:21:00.274556+0000","last_undegraded":"2023-11-28T11:21:00.274556+0000","last_fullsized":"2023-11-28T11:21:00.274556+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:29:57.585501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.776665+0000","last_change":"2023-11-28T11:21:00.294235+0000","last_active":"2023-11-28T11:21:00.776665+0000","last_peered":"2023-11-28T11:21:00.776665+0000","last_clean":"2023-11-28T11:21:00.776665+0000","last_became_active":"2023-11-28T11:21:00.293813+0000","last_became_peered":"2023-11-28T11:21:00.293813+0000","last_unstale":"2023-11-28T11:21:00.776665+0000","last_undegraded":"2023-11-28T11:21:00.776665+0000","last_fullsized":"2023-11-28T11:21:00.776665+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:44:23.280744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.271358+0000","last_change":"2023-11-28T11:20:55.767813+0000","last_active":"2023-11-28T11:21:00.271358+0000","last_peered":"2023-11-28T11:21:00.271358+0000","last_clean":"2023-11-28T11:21:00.271358+0000","last_became_active":"2023-11-28T11:20:55.765782+0000","last_became_peered":"2023-11-28T11:20:55.765782+0000","last_unstale":"2023-11-28T11:21:00.271358+0000","last_undegraded":"2023-11-28T11:21:00.271358+0000","last_fullsized":"2023-11-28T11:21:00.271358+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:47:38.102588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.271000+0000","last_change":"2023-11-28T11:20:56.265782+0000","last_active":"2023-11-28T11:21:00.271000+0000","last_peered":"2023-11-28T11:21:00.271000+0000","last_clean":"2023-11-28T11:21:00.271000+0000","last_became_active":"2023-11-28T11:20:56.265102+0000","last_became_peered":"2023-11-28T11:20:56.265102+0000","last_unstale":"2023-11-28T11:21:00.271000+0000","last_undegraded":"2023-11-28T11:21:00.271000+0000","last_fullsized":"2023-11-28T11:21:00.271000+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:45:14.478416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.883772+0000","last_change":"2023-11-28T11:21:00.297751+0000","last_active":"2023-11-28T11:21:01.883772+0000","last_peered":"2023-11-28T11:21:01.883772+0000","last_clean":"2023-11-28T11:21:01.883772+0000","last_became_active":"2023-11-28T11:21:00.297281+0000","last_became_peered":"2023-11-28T11:21:00.297281+0000","last_unstale":"2023-11-28T11:21:01.883772+0000","last_undegraded":"2023-11-28T11:21:01.883772+0000","last_fullsized":"2023-11-28T11:21:01.883772+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:42:41.996906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.292230+0000","last_change":"2023-11-28T11:20:55.764439+0000","last_active":"2023-11-28T11:21:00.292230+0000","last_peered":"2023-11-28T11:21:00.292230+0000","last_clean":"2023-11-28T11:21:00.292230+0000","last_became_active":"2023-11-28T11:20:55.764165+0000","last_became_peered":"2023-11-28T11:20:55.764165+0000","last_unstale":"2023-11-28T11:21:00.292230+0000","last_undegraded":"2023-11-28T11:21:00.292230+0000","last_fullsized":"2023-11-28T11:21:00.292230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:33:02.765966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.273131+0000","last_change":"2023-11-28T11:20:56.265074+0000","last_active":"2023-11-28T11:21:00.273131+0000","last_peered":"2023-11-28T11:21:00.273131+0000","last_clean":"2023-11-28T11:21:00.273131+0000","last_became_active":"2023-11-28T11:20:56.264495+0000","last_became_peered":"2023-11-28T11:20:56.264495+0000","last_unstale":"2023-11-28T11:21:00.273131+0000","last_undegraded":"2023-11-28T11:21:00.273131+0000","last_fullsized":"2023-11-28T11:21:00.273131+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:31:26.827850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.286246+0000","last_change":"2023-11-28T11:20:56.260972+0000","last_active":"2023-11-28T11:21:00.286246+0000","last_peered":"2023-11-28T11:21:00.286246+0000","last_clean":"2023-11-28T11:21:00.286246+0000","last_became_active":"2023-11-28T11:20:56.259491+0000","last_became_peered":"2023-11-28T11:20:56.259491+0000","last_unstale":"2023-11-28T11:21:00.286246+0000","last_undegraded":"2023-11-28T11:21:00.286246+0000","last_fullsized":"2023-11-28T11:21:00.286246+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:06:02.860064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.271158+0000","last_change":"2023-11-28T11:20:56.263917+0000","last_active":"2023-11-28T11:21:00.271158+0000","last_peered":"2023-11-28T11:21:00.271158+0000","last_clean":"2023-11-28T11:21:00.271158+0000","last_became_active":"2023-11-28T11:20:56.261946+0000","last_became_peered":"2023-11-28T11:20:56.261946+0000","last_unstale":"2023-11-28T11:21:00.271158+0000","last_undegraded":"2023-11-28T11:21:00.271158+0000","last_fullsized":"2023-11-28T11:21:00.271158+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:40:42.059656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.274852+0000","last_change":"2023-11-28T11:20:56.272089+0000","last_active":"2023-11-28T11:21:00.274852+0000","last_peered":"2023-11-28T11:21:00.274852+0000","last_clean":"2023-11-28T11:21:00.274852+0000","last_became_active":"2023-11-28T11:20:56.271704+0000","last_became_peered":"2023-11-28T11:20:56.271704+0000","last_unstale":"2023-11-28T11:21:00.274852+0000","last_undegraded":"2023-11-28T11:21:00.274852+0000","last_fullsized":"2023-11-28T11:21:00.274852+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:33:54.639487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.270968+0000","last_change":"2023-11-28T11:20:56.270543+0000","last_active":"2023-11-28T11:21:00.270968+0000","last_peered":"2023-11-28T11:21:00.270968+0000","last_clean":"2023-11-28T11:21:00.270968+0000","last_became_active":"2023-11-28T11:20:56.270310+0000","last_became_peered":"2023-11-28T11:20:56.270310+0000","last_unstale":"2023-11-28T11:21:00.270968+0000","last_undegraded":"2023-11-28T11:21:00.270968+0000","last_fullsized":"2023-11-28T11:21:00.270968+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:21:21.052493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.288589+0000","last_change":"2023-11-28T11:20:56.262145+0000","last_active":"2023-11-28T11:21:00.288589+0000","last_peered":"2023-11-28T11:21:00.288589+0000","last_clean":"2023-11-28T11:21:00.288589+0000","last_became_active":"2023-11-28T11:20:56.260881+0000","last_became_peered":"2023-11-28T11:20:56.260881+0000","last_unstale":"2023-11-28T11:21:00.288589+0000","last_undegraded":"2023-11-28T11:21:00.288589+0000","last_fullsized":"2023-11-28T11:21:00.288589+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:32:52.386018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4528,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9280,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362560,"statfs":{"total":96636764160,"available":96627261440,"internally_reserved":0,"allocated":909312,"data_stored":713609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8100,"kb_used_data":284,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363740,"statfs":{"total":96636764160,"available":96628469760,"internally_reserved":0,"allocated":290816,"data_stored":119901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81200000000000006}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88800000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8164,"kb_used_data":284,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363676,"statfs":{"total":96636764160,"available":96628404224,"internally_reserved":0,"allocated":290816,"data_stored":119901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9260,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362580,"statfs":{"total":96636764160,"available":96627281920,"internally_reserved":0,"allocated":888832,"data_stored":710835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96799999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.681}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.6160000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8100,"kb_used_data":284,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363740,"statfs":{"total":96636764160,"available":96628469760,"internally_reserved":0,"allocated":290816,"data_stored":119901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8168,"kb_used_data":288,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363672,"statfs":{"total":96636764160,"available":96628400128,"internally_reserved":0,"allocated":294912,"data_stored":119991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95999999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8248,"kb_used_data":304,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363592,"statfs":{"total":96636764160,"available":96628318208,"internally_reserved":0,"allocated":311296,"data_stored":123703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8172,"kb_used_data":292,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363668,"statfs":{"total":96636764160,"available":96628396032,"internally_reserved":0,"allocated":299008,"data_stored":120537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.877}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82499999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94899999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:21:23.588 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:21:23.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.812+0000 7f8e0bbff700 1 Processor -- start 2023-11-28T11:21:23.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.813+0000 7f8e0bbff700 1 -- start start 2023-11-28T11:21:23.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.814+0000 7f8e0bbff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e040cd460 0x7f8e0406c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:23.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.814+0000 7f8e0bbff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8e040d0d60 0x7f8e040d11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:23.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.814+0000 7f8e0bbff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e040b88e0 con 0x7f8e040b8510 2023-11-28T11:21:23.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.814+0000 7f8e0bbff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e040cdbd0 con 0x7f8e040d0d60 2023-11-28T11:21:23.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.814+0000 7f8e0bbff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e040d02d0 con 0x7f8e040cd460 2023-11-28T11:21:23.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.814+0000 7f8e0919a700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e040cd460 0x7f8e0406c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:23.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.814+0000 7f8e0a19c700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8e040d0d60 0x7f8e040d11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:23.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.814+0000 7f8e0919a700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e040cd460 0x7f8e0406c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.92:34290/0 (socket says 172.21.15.92:34290) 2023-11-28T11:21:23.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.814+0000 7f8e0919a700 1 -- 172.21.15.92:0/1266017074 learned_addr learned my addr 172.21.15.92:0/1266017074 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:23.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.814+0000 7f8e0999b700 1 --1- 172.21.15.92:0/1266017074 >> v1:172.21.15.169:6789/0 conn(0x7f8e040b8510 0x7f8e0406b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.92:60850/0 (socket says 172.21.15.92:60850) 2023-11-28T11:21:23.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.815+0000 7f8e0919a700 1 -- 172.21.15.92:0/1266017074 >> v2:172.21.15.169:3300/0 conn(0x7f8e040d0d60 msgr2=0x7f8e040d11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:23.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.815+0000 7f8e0919a700 1 --2- 172.21.15.92:0/1266017074 >> v2:172.21.15.169:3300/0 conn(0x7f8e040d0d60 0x7f8e040d11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:23.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.815+0000 7f8e0919a700 1 -- 172.21.15.92:0/1266017074 >> v1:172.21.15.169:6789/0 conn(0x7f8e040b8510 legacy=0x7f8e0406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:23.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.815+0000 7f8e0919a700 1 -- 172.21.15.92:0/1266017074 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e040cd8f0 con 0x7f8e040cd460 2023-11-28T11:21:23.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.815+0000 7f8e0a19c700 1 --2- 172.21.15.92:0/1266017074 >> v2:172.21.15.169:3300/0 conn(0x7f8e040d0d60 0x7f8e040d11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:21:23.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.815+0000 7f8e0919a700 1 --2- 172.21.15.92:0/1266017074 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e040cd460 0x7f8e0406c080 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8df40098f0 tx=0x7f8df4009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e69271871670a4ab server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:23.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.815+0000 7f8e08999700 1 -- 172.21.15.92:0/1266017074 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df4011830 con 0x7f8e040cd460 2023-11-28T11:21:23.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.815+0000 7f8e08999700 1 -- 172.21.15.92:0/1266017074 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8df4011990 con 0x7f8e040cd460 2023-11-28T11:21:23.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.815+0000 7f8e08999700 1 -- 172.21.15.92:0/1266017074 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df4011d20 con 0x7f8e040cd460 2023-11-28T11:21:23.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.816+0000 7f8e0bbff700 1 -- 172.21.15.92:0/1266017074 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e040cd460 msgr2=0x7f8e0406c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:23.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.816+0000 7f8e0bbff700 1 --2- 172.21.15.92:0/1266017074 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e040cd460 0x7f8e0406c080 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8df40098f0 tx=0x7f8df4009c00 comp rx=0 tx=0).stop 2023-11-28T11:21:23.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.816+0000 7f8e0bbff700 1 -- 172.21.15.92:0/1266017074 shutdown_connections 2023-11-28T11:21:23.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.816+0000 7f8e0bbff700 1 --2- 172.21.15.92:0/1266017074 >> v2:172.21.15.169:3300/0 conn(0x7f8e040d0d60 0x7f8e040d11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:23.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.816+0000 7f8e0bbff700 1 --2- 172.21.15.92:0/1266017074 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e040cd460 0x7f8e0406c080 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:23.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.816+0000 7f8e0bbff700 1 -- 172.21.15.92:0/1266017074 >> 172.21.15.92:0/1266017074 conn(0x7f8e040719a0 msgr2=0x7f8e04071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:23.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.816+0000 7f8e0bbff700 1 -- 172.21.15.92:0/1266017074 shutdown_connections 2023-11-28T11:21:23.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.816+0000 7f8e0bbff700 1 -- 172.21.15.92:0/1266017074 wait complete. 2023-11-28T11:21:23.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.817+0000 7f8e0bbff700 1 Processor -- start 2023-11-28T11:21:23.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.818+0000 7f8e0bbff700 1 -- start start 2023-11-28T11:21:23.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.818+0000 7f8e0bbff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e040b8510 0x7f8e041e29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:23.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.818+0000 7f8e0bbff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e040cd460 0x7f8e041f5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:23.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.819+0000 7f8e0bbff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e040d0d60 0x7f8e041f1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:23.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.819+0000 7f8e0bbff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e040cc030 con 0x7f8e040b8510 2023-11-28T11:21:23.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.819+0000 7f8e0bbff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e040cbd00 con 0x7f8e040d0d60 2023-11-28T11:21:23.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.819+0000 7f8e0bbff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e041f5ad0 con 0x7f8e040cd460 2023-11-28T11:21:23.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.819+0000 7f8e0999b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e040b8510 0x7f8e041e29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:23.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.819+0000 7f8e0919a700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e040cd460 0x7f8e041f5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:23.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.819+0000 7f8e0999b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e040b8510 0x7f8e041e29a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49086/0 (socket says 172.21.15.92:49086) 2023-11-28T11:21:23.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.819+0000 7f8e0999b700 1 -- 172.21.15.92:0/2374351828 learned_addr learned my addr 172.21.15.92:0/2374351828 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:23.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.819+0000 7f8e0a19c700 1 --2- 172.21.15.92:0/2374351828 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e040d0d60 0x7f8e041f1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:23.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.819+0000 7f8e0919a700 1 -- 172.21.15.92:0/2374351828 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e040d0d60 msgr2=0x7f8e041f1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:23.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.819+0000 7f8e0919a700 1 --2- 172.21.15.92:0/2374351828 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e040d0d60 0x7f8e041f1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:23.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.819+0000 7f8e0919a700 1 -- 172.21.15.92:0/2374351828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e040b8510 msgr2=0x7f8e041e29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:23.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.819+0000 7f8e0919a700 1 --2- 172.21.15.92:0/2374351828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e040b8510 0x7f8e041e29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:23.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.819+0000 7f8e0919a700 1 -- 172.21.15.92:0/2374351828 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dfc009a70 con 0x7f8e040cd460 2023-11-28T11:21:23.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.819+0000 7f8e0999b700 1 --2- 172.21.15.92:0/2374351828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e040b8510 0x7f8e041e29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:21:23.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.820+0000 7f8e0919a700 1 --2- 172.21.15.92:0/2374351828 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e040cd460 0x7f8e041f5280 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8df40098f0 tx=0x7f8df4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:23.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.820+0000 7f8dfa7fc700 1 -- 172.21.15.92:0/2374351828 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df4004340 con 0x7f8e040cd460 2023-11-28T11:21:23.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.820+0000 7f8dfa7fc700 1 -- 172.21.15.92:0/2374351828 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8df401f070 con 0x7f8e040cd460 2023-11-28T11:21:23.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.820+0000 7f8dfa7fc700 1 -- 172.21.15.92:0/2374351828 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df40036a0 con 0x7f8e040cd460 2023-11-28T11:21:23.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.820+0000 7f8e0bbff700 1 -- 172.21.15.92:0/2374351828 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8df4009550 con 0x7f8e040cd460 2023-11-28T11:21:23.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.820+0000 7f8e0bbff700 1 -- 172.21.15.92:0/2374351828 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e040cfff0 con 0x7f8e040cd460 2023-11-28T11:21:23.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.821+0000 7f8e0bbff700 1 -- 172.21.15.92:0/2374351828 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e040cfd10 con 0x7f8e040cd460 2023-11-28T11:21:23.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.822+0000 7f8dfa7fc700 1 -- 172.21.15.92:0/2374351828 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8df402a020 con 0x7f8e040cd460 2023-11-28T11:21:23.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.823+0000 7f8dfa7fc700 1 --2- 172.21.15.92:0/2374351828 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8de80682f0 0x7f8de806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:23.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.823+0000 7f8dfa7fc700 1 -- 172.21.15.92:0/2374351828 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f8df4087e10 con 0x7f8e040cd460 2023-11-28T11:21:23.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.823+0000 7f8e0999b700 1 --2- 172.21.15.92:0/2374351828 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8de80682f0 0x7f8de806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:23.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.824+0000 7f8e0999b700 1 --2- 172.21.15.92:0/2374351828 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8de80682f0 0x7f8de806a7a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8e00000f60 tx=0x7f8e00007f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:23.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:23.827+0000 7f8dfa7fc700 1 -- 172.21.15.92:0/2374351828 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8df4054310 con 0x7f8e040cd460 2023-11-28T11:21:24.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.011+0000 7f8e0bbff700 1 -- 172.21.15.92:0/2374351828 --> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8e04067880 con 0x7f8de80682f0 2023-11-28T11:21:24.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.022+0000 7f8dfa7fc700 1 -- 172.21.15.92:0/2374351828 <== mgr.4103 v2:172.21.15.92:6832/115482 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367330 (secure 0 0 0) 0x7f8e04067880 con 0x7f8de80682f0 2023-11-28T11:21:24.029 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:21:24.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.026+0000 7f8e0bbff700 1 -- 172.21.15.92:0/2374351828 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8de80682f0 msgr2=0x7f8de806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:24.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.026+0000 7f8e0bbff700 1 --2- 172.21.15.92:0/2374351828 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8de80682f0 0x7f8de806a7a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8e00000f60 tx=0x7f8e00007f90 comp rx=0 tx=0).stop 2023-11-28T11:21:24.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.026+0000 7f8e0bbff700 1 -- 172.21.15.92:0/2374351828 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e040cd460 msgr2=0x7f8e041f5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:24.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.026+0000 7f8e0bbff700 1 --2- 172.21.15.92:0/2374351828 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e040cd460 0x7f8e041f5280 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8df40098f0 tx=0x7f8df4004750 comp rx=0 tx=0).stop 2023-11-28T11:21:24.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.026+0000 7f8e0bbff700 1 -- 172.21.15.92:0/2374351828 shutdown_connections 2023-11-28T11:21:24.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.026+0000 7f8e0bbff700 1 --2- 172.21.15.92:0/2374351828 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8e040d0d60 0x7f8e041f1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:24.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.026+0000 7f8e0bbff700 1 --2- 172.21.15.92:0/2374351828 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8de80682f0 0x7f8de806a7a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:24.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.026+0000 7f8e0bbff700 1 --2- 172.21.15.92:0/2374351828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e040b8510 0x7f8e041e29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:24.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.026+0000 7f8e0bbff700 1 --2- 172.21.15.92:0/2374351828 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8e040cd460 0x7f8e041f5280 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:24.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.026+0000 7f8e0bbff700 1 -- 172.21.15.92:0/2374351828 >> 172.21.15.92:0/2374351828 conn(0x7f8e040719a0 msgr2=0x7f8e040d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:24.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.028+0000 7f8e0bbff700 1 -- 172.21.15.92:0/2374351828 shutdown_connections 2023-11-28T11:21:24.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.028+0000 7f8e0bbff700 1 -- 172.21.15.92:0/2374351828 wait complete. 2023-11-28T11:21:24.034 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-11-28T11:21:24.047 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":41,"stamp":"2023-11-28T11:21:23.866578+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594896,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":67492,"kb_used_data":3492,"kb_used_omap":0,"kb_used_meta":63936,"kb_avail":754907228,"statfs":{"total":773094113280,"available":773025001472,"internally_reserved":0,"allocated":3575808,"data_stored":2148378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65470464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1749,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002806"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.282064+0000","last_change":"2023-11-28T11:20:56.269261+0000","last_active":"2023-11-28T11:21:00.282064+0000","last_peered":"2023-11-28T11:21:00.282064+0000","last_clean":"2023-11-28T11:21:00.282064+0000","last_became_active":"2023-11-28T11:20:56.268959+0000","last_became_peered":"2023-11-28T11:20:56.268959+0000","last_unstale":"2023-11-28T11:21:00.282064+0000","last_undegraded":"2023-11-28T11:21:00.282064+0000","last_fullsized":"2023-11-28T11:21:00.282064+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:39:28.472955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.273172+0000","last_change":"2023-11-28T11:20:56.258503+0000","last_active":"2023-11-28T11:21:00.273172+0000","last_peered":"2023-11-28T11:21:00.273172+0000","last_clean":"2023-11-28T11:21:00.273172+0000","last_became_active":"2023-11-28T11:20:56.258101+0000","last_became_peered":"2023-11-28T11:20:56.258101+0000","last_unstale":"2023-11-28T11:21:00.273172+0000","last_undegraded":"2023-11-28T11:21:00.273172+0000","last_fullsized":"2023-11-28T11:21:00.273172+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:14:34.876987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.274259+0000","last_change":"2023-11-28T11:20:56.269696+0000","last_active":"2023-11-28T11:21:00.274259+0000","last_peered":"2023-11-28T11:21:00.274259+0000","last_clean":"2023-11-28T11:21:00.274259+0000","last_became_active":"2023-11-28T11:20:56.264650+0000","last_became_peered":"2023-11-28T11:20:56.264650+0000","last_unstale":"2023-11-28T11:21:00.274259+0000","last_undegraded":"2023-11-28T11:21:00.274259+0000","last_fullsized":"2023-11-28T11:21:00.274259+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:25:52.242410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.286447+0000","last_change":"2023-11-28T11:20:56.269817+0000","last_active":"2023-11-28T11:21:00.286447+0000","last_peered":"2023-11-28T11:21:00.286447+0000","last_clean":"2023-11-28T11:21:00.286447+0000","last_became_active":"2023-11-28T11:20:56.269514+0000","last_became_peered":"2023-11-28T11:20:56.269514+0000","last_unstale":"2023-11-28T11:21:00.286447+0000","last_undegraded":"2023-11-28T11:21:00.286447+0000","last_fullsized":"2023-11-28T11:21:00.286447+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:09:41.753917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.289921+0000","last_change":"2023-11-28T11:20:56.274319+0000","last_active":"2023-11-28T11:21:00.289921+0000","last_peered":"2023-11-28T11:21:00.289921+0000","last_clean":"2023-11-28T11:21:00.289921+0000","last_became_active":"2023-11-28T11:20:56.273667+0000","last_became_peered":"2023-11-28T11:20:56.273667+0000","last_unstale":"2023-11-28T11:21:00.289921+0000","last_undegraded":"2023-11-28T11:21:00.289921+0000","last_fullsized":"2023-11-28T11:21:00.289921+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:31:26.827850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.274908+0000","last_change":"2023-11-28T11:20:56.263963+0000","last_active":"2023-11-28T11:21:00.274908+0000","last_peered":"2023-11-28T11:21:00.274908+0000","last_clean":"2023-11-28T11:21:00.274908+0000","last_became_active":"2023-11-28T11:20:56.260761+0000","last_became_peered":"2023-11-28T11:20:56.260761+0000","last_unstale":"2023-11-28T11:21:00.274908+0000","last_undegraded":"2023-11-28T11:21:00.274908+0000","last_fullsized":"2023-11-28T11:21:00.274908+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:13:46.683000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.288334+0000","last_change":"2023-11-28T11:20:56.261054+0000","last_active":"2023-11-28T11:21:00.288334+0000","last_peered":"2023-11-28T11:21:00.288334+0000","last_clean":"2023-11-28T11:21:00.288334+0000","last_became_active":"2023-11-28T11:20:56.260774+0000","last_became_peered":"2023-11-28T11:20:56.260774+0000","last_unstale":"2023-11-28T11:21:00.288334+0000","last_undegraded":"2023-11-28T11:21:00.288334+0000","last_fullsized":"2023-11-28T11:21:00.288334+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:08.079216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.273485+0000","last_change":"2023-11-28T11:20:56.252717+0000","last_active":"2023-11-28T11:21:00.273485+0000","last_peered":"2023-11-28T11:21:00.273485+0000","last_clean":"2023-11-28T11:21:00.273485+0000","last_became_active":"2023-11-28T11:20:56.252066+0000","last_became_peered":"2023-11-28T11:20:56.252066+0000","last_unstale":"2023-11-28T11:21:00.273485+0000","last_undegraded":"2023-11-28T11:21:00.273485+0000","last_fullsized":"2023-11-28T11:21:00.273485+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:42:57.864956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.291971+0000","last_change":"2023-11-28T11:20:56.273810+0000","last_active":"2023-11-28T11:21:00.291971+0000","last_peered":"2023-11-28T11:21:00.291971+0000","last_clean":"2023-11-28T11:21:00.291971+0000","last_became_active":"2023-11-28T11:20:56.273477+0000","last_became_peered":"2023-11-28T11:20:56.273477+0000","last_unstale":"2023-11-28T11:21:00.291971+0000","last_undegraded":"2023-11-28T11:21:00.291971+0000","last_fullsized":"2023-11-28T11:21:00.291971+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:50:45.228070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.277322+0000","last_change":"2023-11-28T11:20:56.256667+0000","last_active":"2023-11-28T11:21:00.277322+0000","last_peered":"2023-11-28T11:21:00.277322+0000","last_clean":"2023-11-28T11:21:00.277322+0000","last_became_active":"2023-11-28T11:20:56.256387+0000","last_became_peered":"2023-11-28T11:20:56.256387+0000","last_unstale":"2023-11-28T11:21:00.277322+0000","last_undegraded":"2023-11-28T11:21:00.277322+0000","last_fullsized":"2023-11-28T11:21:00.277322+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:24:07.384730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.274154+0000","last_change":"2023-11-28T11:20:56.281728+0000","last_active":"2023-11-28T11:21:00.274154+0000","last_peered":"2023-11-28T11:21:00.274154+0000","last_clean":"2023-11-28T11:21:00.274154+0000","last_became_active":"2023-11-28T11:20:56.281437+0000","last_became_peered":"2023-11-28T11:20:56.281437+0000","last_unstale":"2023-11-28T11:21:00.274154+0000","last_undegraded":"2023-11-28T11:21:00.274154+0000","last_fullsized":"2023-11-28T11:21:00.274154+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:01:57.936285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.280518+0000","last_change":"2023-11-28T11:20:56.256016+0000","last_active":"2023-11-28T11:21:00.280518+0000","last_peered":"2023-11-28T11:21:00.280518+0000","last_clean":"2023-11-28T11:21:00.280518+0000","last_became_active":"2023-11-28T11:20:56.255480+0000","last_became_peered":"2023-11-28T11:20:56.255480+0000","last_unstale":"2023-11-28T11:21:00.280518+0000","last_undegraded":"2023-11-28T11:21:00.280518+0000","last_fullsized":"2023-11-28T11:21:00.280518+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:45:09.586958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.280455+0000","last_change":"2023-11-28T11:20:56.265497+0000","last_active":"2023-11-28T11:21:00.280455+0000","last_peered":"2023-11-28T11:21:00.280455+0000","last_clean":"2023-11-28T11:21:00.280455+0000","last_became_active":"2023-11-28T11:20:56.265236+0000","last_became_peered":"2023-11-28T11:20:56.265236+0000","last_unstale":"2023-11-28T11:21:00.280455+0000","last_undegraded":"2023-11-28T11:21:00.280455+0000","last_fullsized":"2023-11-28T11:21:00.280455+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:47:39.108512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.274722+0000","last_change":"2023-11-28T11:20:56.262165+0000","last_active":"2023-11-28T11:21:00.274722+0000","last_peered":"2023-11-28T11:21:00.274722+0000","last_clean":"2023-11-28T11:21:00.274722+0000","last_became_active":"2023-11-28T11:20:56.261947+0000","last_became_peered":"2023-11-28T11:20:56.261947+0000","last_unstale":"2023-11-28T11:21:00.274722+0000","last_undegraded":"2023-11-28T11:21:00.274722+0000","last_fullsized":"2023-11-28T11:21:00.274722+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:45:14.478416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.288123+0000","last_change":"2023-11-28T11:20:56.264861+0000","last_active":"2023-11-28T11:21:00.288123+0000","last_peered":"2023-11-28T11:21:00.288123+0000","last_clean":"2023-11-28T11:21:00.288123+0000","last_became_active":"2023-11-28T11:20:56.264538+0000","last_became_peered":"2023-11-28T11:20:56.264538+0000","last_unstale":"2023-11-28T11:21:00.288123+0000","last_undegraded":"2023-11-28T11:21:00.288123+0000","last_fullsized":"2023-11-28T11:21:00.288123+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:40:42.059656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.273219+0000","last_change":"2023-11-28T11:20:56.258923+0000","last_active":"2023-11-28T11:21:00.273219+0000","last_peered":"2023-11-28T11:21:00.273219+0000","last_clean":"2023-11-28T11:21:00.273219+0000","last_became_active":"2023-11-28T11:20:56.258633+0000","last_became_peered":"2023-11-28T11:20:56.258633+0000","last_unstale":"2023-11-28T11:21:00.273219+0000","last_undegraded":"2023-11-28T11:21:00.273219+0000","last_fullsized":"2023-11-28T11:21:00.273219+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:26:32.343329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.292050+0000","last_change":"2023-11-28T11:20:56.271537+0000","last_active":"2023-11-28T11:21:00.292050+0000","last_peered":"2023-11-28T11:21:00.292050+0000","last_clean":"2023-11-28T11:21:00.292050+0000","last_became_active":"2023-11-28T11:20:56.270556+0000","last_became_peered":"2023-11-28T11:20:56.270556+0000","last_unstale":"2023-11-28T11:21:00.292050+0000","last_undegraded":"2023-11-28T11:21:00.292050+0000","last_fullsized":"2023-11-28T11:21:00.292050+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:11:36.091056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.274876+0000","last_change":"2023-11-28T11:20:56.274810+0000","last_active":"2023-11-28T11:21:00.274876+0000","last_peered":"2023-11-28T11:21:00.274876+0000","last_clean":"2023-11-28T11:21:00.274876+0000","last_became_active":"2023-11-28T11:20:56.274523+0000","last_became_peered":"2023-11-28T11:20:56.274523+0000","last_unstale":"2023-11-28T11:21:00.274876+0000","last_undegraded":"2023-11-28T11:21:00.274876+0000","last_fullsized":"2023-11-28T11:21:00.274876+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:42:57.864956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.281099+0000","last_change":"2023-11-28T11:20:56.258601+0000","last_active":"2023-11-28T11:21:00.281099+0000","last_peered":"2023-11-28T11:21:00.281099+0000","last_clean":"2023-11-28T11:21:00.281099+0000","last_became_active":"2023-11-28T11:20:56.258282+0000","last_became_peered":"2023-11-28T11:20:56.258282+0000","last_unstale":"2023-11-28T11:21:00.281099+0000","last_undegraded":"2023-11-28T11:21:00.281099+0000","last_fullsized":"2023-11-28T11:21:00.281099+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:58:47.731976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.275898+0000","last_change":"2023-11-28T11:20:56.263948+0000","last_active":"2023-11-28T11:21:00.275898+0000","last_peered":"2023-11-28T11:21:00.275898+0000","last_clean":"2023-11-28T11:21:00.275898+0000","last_became_active":"2023-11-28T11:20:56.263627+0000","last_became_peered":"2023-11-28T11:20:56.263627+0000","last_unstale":"2023-11-28T11:21:00.275898+0000","last_undegraded":"2023-11-28T11:21:00.275898+0000","last_fullsized":"2023-11-28T11:21:00.275898+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:08.079216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.300168+0000","last_change":"2023-11-28T11:20:56.274116+0000","last_active":"2023-11-28T11:21:00.300168+0000","last_peered":"2023-11-28T11:21:00.300168+0000","last_clean":"2023-11-28T11:21:00.300168+0000","last_became_active":"2023-11-28T11:20:56.273403+0000","last_became_peered":"2023-11-28T11:20:56.273403+0000","last_unstale":"2023-11-28T11:21:00.300168+0000","last_undegraded":"2023-11-28T11:21:00.300168+0000","last_fullsized":"2023-11-28T11:21:00.300168+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:45:14.478416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.291080+0000","last_change":"2023-11-28T11:20:56.259411+0000","last_active":"2023-11-28T11:21:00.291080+0000","last_peered":"2023-11-28T11:21:00.291080+0000","last_clean":"2023-11-28T11:21:00.291080+0000","last_became_active":"2023-11-28T11:20:56.259189+0000","last_became_peered":"2023-11-28T11:20:56.259189+0000","last_unstale":"2023-11-28T11:21:00.291080+0000","last_undegraded":"2023-11-28T11:21:00.291080+0000","last_fullsized":"2023-11-28T11:21:00.291080+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:24:07.384730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.270889+0000","last_change":"2023-11-28T11:20:56.257033+0000","last_active":"2023-11-28T11:21:00.270889+0000","last_peered":"2023-11-28T11:21:00.270889+0000","last_clean":"2023-11-28T11:21:00.270889+0000","last_became_active":"2023-11-28T11:20:56.256549+0000","last_became_peered":"2023-11-28T11:20:56.256549+0000","last_unstale":"2023-11-28T11:21:00.270889+0000","last_undegraded":"2023-11-28T11:21:00.270889+0000","last_fullsized":"2023-11-28T11:21:00.270889+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:24:07.384730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.286598+0000","last_change":"2023-11-28T11:20:56.270979+0000","last_active":"2023-11-28T11:21:00.286598+0000","last_peered":"2023-11-28T11:21:00.286598+0000","last_clean":"2023-11-28T11:21:00.286598+0000","last_became_active":"2023-11-28T11:20:56.270710+0000","last_became_peered":"2023-11-28T11:20:56.270710+0000","last_unstale":"2023-11-28T11:21:00.286598+0000","last_undegraded":"2023-11-28T11:21:00.286598+0000","last_fullsized":"2023-11-28T11:21:00.286598+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:42:57.864956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.290274+0000","last_change":"2023-11-28T11:20:56.262561+0000","last_active":"2023-11-28T11:21:00.290274+0000","last_peered":"2023-11-28T11:21:00.290274+0000","last_clean":"2023-11-28T11:21:00.290274+0000","last_became_active":"2023-11-28T11:20:56.262251+0000","last_became_peered":"2023-11-28T11:20:56.262251+0000","last_unstale":"2023-11-28T11:21:00.290274+0000","last_undegraded":"2023-11-28T11:21:00.290274+0000","last_fullsized":"2023-11-28T11:21:00.290274+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:11:36.091056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.292352+0000","last_change":"2023-11-28T11:20:56.261421+0000","last_active":"2023-11-28T11:21:00.292352+0000","last_peered":"2023-11-28T11:21:00.292352+0000","last_clean":"2023-11-28T11:21:00.292352+0000","last_became_active":"2023-11-28T11:20:56.259655+0000","last_became_peered":"2023-11-28T11:20:56.259655+0000","last_unstale":"2023-11-28T11:21:00.292352+0000","last_undegraded":"2023-11-28T11:21:00.292352+0000","last_fullsized":"2023-11-28T11:21:00.292352+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:01:57.936285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.273393+0000","last_change":"2023-11-28T11:20:56.258358+0000","last_active":"2023-11-28T11:21:00.273393+0000","last_peered":"2023-11-28T11:21:00.273393+0000","last_clean":"2023-11-28T11:21:00.273393+0000","last_became_active":"2023-11-28T11:20:56.257895+0000","last_became_peered":"2023-11-28T11:20:56.257895+0000","last_unstale":"2023-11-28T11:21:00.273393+0000","last_undegraded":"2023-11-28T11:21:00.273393+0000","last_fullsized":"2023-11-28T11:21:00.273393+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:08.079216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.777639+0000","last_change":"2023-11-28T11:21:00.306498+0000","last_active":"2023-11-28T11:21:00.777639+0000","last_peered":"2023-11-28T11:21:00.777639+0000","last_clean":"2023-11-28T11:21:00.777639+0000","last_became_active":"2023-11-28T11:21:00.305632+0000","last_became_peered":"2023-11-28T11:21:00.305632+0000","last_unstale":"2023-11-28T11:21:00.777639+0000","last_undegraded":"2023-11-28T11:21:00.777639+0000","last_fullsized":"2023-11-28T11:21:00.777639+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:08:42.028668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.274765+0000","last_change":"2023-11-28T11:20:55.761598+0000","last_active":"2023-11-28T11:21:00.274765+0000","last_peered":"2023-11-28T11:21:00.274765+0000","last_clean":"2023-11-28T11:21:00.274765+0000","last_became_active":"2023-11-28T11:20:55.761257+0000","last_became_peered":"2023-11-28T11:20:55.761257+0000","last_unstale":"2023-11-28T11:21:00.274765+0000","last_undegraded":"2023-11-28T11:21:00.274765+0000","last_fullsized":"2023-11-28T11:21:00.274765+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:13:32.428410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.271441+0000","last_change":"2023-11-28T11:20:56.269300+0000","last_active":"2023-11-28T11:21:00.271441+0000","last_peered":"2023-11-28T11:21:00.271441+0000","last_clean":"2023-11-28T11:21:00.271441+0000","last_became_active":"2023-11-28T11:20:56.269068+0000","last_became_peered":"2023-11-28T11:20:56.269068+0000","last_unstale":"2023-11-28T11:21:00.271441+0000","last_undegraded":"2023-11-28T11:21:00.271441+0000","last_fullsized":"2023-11-28T11:21:00.271441+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:42:57.864956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.779188+0000","last_change":"2023-11-28T11:21:00.303150+0000","last_active":"2023-11-28T11:21:00.779188+0000","last_peered":"2023-11-28T11:21:00.779188+0000","last_clean":"2023-11-28T11:21:00.779188+0000","last_became_active":"2023-11-28T11:21:00.302669+0000","last_became_peered":"2023-11-28T11:21:00.302669+0000","last_unstale":"2023-11-28T11:21:00.779188+0000","last_undegraded":"2023-11-28T11:21:00.779188+0000","last_fullsized":"2023-11-28T11:21:00.779188+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:26:35.867941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.284078+0000","last_change":"2023-11-28T11:20:55.769854+0000","last_active":"2023-11-28T11:21:00.284078+0000","last_peered":"2023-11-28T11:21:00.284078+0000","last_clean":"2023-11-28T11:21:00.284078+0000","last_became_active":"2023-11-28T11:20:55.769322+0000","last_became_peered":"2023-11-28T11:20:55.769322+0000","last_unstale":"2023-11-28T11:21:00.284078+0000","last_undegraded":"2023-11-28T11:21:00.284078+0000","last_fullsized":"2023-11-28T11:21:00.284078+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:37:05.627049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.291807+0000","last_change":"2023-11-28T11:20:56.265137+0000","last_active":"2023-11-28T11:21:00.291807+0000","last_peered":"2023-11-28T11:21:00.291807+0000","last_clean":"2023-11-28T11:21:00.291807+0000","last_became_active":"2023-11-28T11:20:56.264228+0000","last_became_peered":"2023-11-28T11:20:56.264228+0000","last_unstale":"2023-11-28T11:21:00.291807+0000","last_undegraded":"2023-11-28T11:21:00.291807+0000","last_fullsized":"2023-11-28T11:21:00.291807+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:42:57.864956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.884947+0000","last_change":"2023-11-28T11:21:00.314686+0000","last_active":"2023-11-28T11:21:01.884947+0000","last_peered":"2023-11-28T11:21:01.884947+0000","last_clean":"2023-11-28T11:21:01.884947+0000","last_became_active":"2023-11-28T11:21:00.314281+0000","last_became_peered":"2023-11-28T11:21:00.314281+0000","last_unstale":"2023-11-28T11:21:01.884947+0000","last_undegraded":"2023-11-28T11:21:01.884947+0000","last_fullsized":"2023-11-28T11:21:01.884947+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:29:41.143865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.271248+0000","last_change":"2023-11-28T11:20:55.768237+0000","last_active":"2023-11-28T11:21:00.271248+0000","last_peered":"2023-11-28T11:21:00.271248+0000","last_clean":"2023-11-28T11:21:00.271248+0000","last_became_active":"2023-11-28T11:20:55.767948+0000","last_became_peered":"2023-11-28T11:20:55.767948+0000","last_unstale":"2023-11-28T11:21:00.271248+0000","last_undegraded":"2023-11-28T11:21:00.271248+0000","last_fullsized":"2023-11-28T11:21:00.271248+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:23:36.801918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.286831+0000","last_change":"2023-11-28T11:20:56.260663+0000","last_active":"2023-11-28T11:21:00.286831+0000","last_peered":"2023-11-28T11:21:00.286831+0000","last_clean":"2023-11-28T11:21:00.286831+0000","last_became_active":"2023-11-28T11:20:56.259825+0000","last_became_peered":"2023-11-28T11:20:56.259825+0000","last_unstale":"2023-11-28T11:21:00.286831+0000","last_undegraded":"2023-11-28T11:21:00.286831+0000","last_fullsized":"2023-11-28T11:21:00.286831+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:45:09.586958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.778585+0000","last_change":"2023-11-28T11:21:00.303720+0000","last_active":"2023-11-28T11:21:00.778585+0000","last_peered":"2023-11-28T11:21:00.778585+0000","last_clean":"2023-11-28T11:21:00.778585+0000","last_became_active":"2023-11-28T11:21:00.302585+0000","last_became_peered":"2023-11-28T11:21:00.302585+0000","last_unstale":"2023-11-28T11:21:00.778585+0000","last_undegraded":"2023-11-28T11:21:00.778585+0000","last_fullsized":"2023-11-28T11:21:00.778585+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:28:33.645294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.292130+0000","last_change":"2023-11-28T11:20:55.764526+0000","last_active":"2023-11-28T11:21:00.292130+0000","last_peered":"2023-11-28T11:21:00.292130+0000","last_clean":"2023-11-28T11:21:00.292130+0000","last_became_active":"2023-11-28T11:20:55.764278+0000","last_became_peered":"2023-11-28T11:20:55.764278+0000","last_unstale":"2023-11-28T11:21:00.292130+0000","last_undegraded":"2023-11-28T11:21:00.292130+0000","last_fullsized":"2023-11-28T11:21:00.292130+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:23:36.801918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.271155+0000","last_change":"2023-11-28T11:20:56.265473+0000","last_active":"2023-11-28T11:21:00.271155+0000","last_peered":"2023-11-28T11:21:00.271155+0000","last_clean":"2023-11-28T11:21:00.271155+0000","last_became_active":"2023-11-28T11:20:56.262049+0000","last_became_peered":"2023-11-28T11:20:56.262049+0000","last_unstale":"2023-11-28T11:21:00.271155+0000","last_undegraded":"2023-11-28T11:21:00.271155+0000","last_fullsized":"2023-11-28T11:21:00.271155+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:08.079216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.806510+0000","last_change":"2023-11-28T11:21:00.306448+0000","last_active":"2023-11-28T11:21:01.806510+0000","last_peered":"2023-11-28T11:21:01.806510+0000","last_clean":"2023-11-28T11:21:01.806510+0000","last_became_active":"2023-11-28T11:21:00.305892+0000","last_became_peered":"2023-11-28T11:21:00.305892+0000","last_unstale":"2023-11-28T11:21:01.806510+0000","last_undegraded":"2023-11-28T11:21:01.806510+0000","last_fullsized":"2023-11-28T11:21:01.806510+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:41.700110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.277223+0000","last_change":"2023-11-28T11:20:55.764162+0000","last_active":"2023-11-28T11:21:00.277223+0000","last_peered":"2023-11-28T11:21:00.277223+0000","last_clean":"2023-11-28T11:21:00.277223+0000","last_became_active":"2023-11-28T11:20:55.760490+0000","last_became_peered":"2023-11-28T11:20:55.760490+0000","last_unstale":"2023-11-28T11:21:00.277223+0000","last_undegraded":"2023-11-28T11:21:00.277223+0000","last_fullsized":"2023-11-28T11:21:00.277223+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:39:27.467031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.271241+0000","last_change":"2023-11-28T11:20:56.269435+0000","last_active":"2023-11-28T11:21:00.271241+0000","last_peered":"2023-11-28T11:21:00.271241+0000","last_clean":"2023-11-28T11:21:00.271241+0000","last_became_active":"2023-11-28T11:20:56.269179+0000","last_became_peered":"2023-11-28T11:20:56.269179+0000","last_unstale":"2023-11-28T11:21:00.271241+0000","last_undegraded":"2023-11-28T11:21:00.271241+0000","last_fullsized":"2023-11-28T11:21:00.271241+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:50:45.228070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.844101+0000","last_change":"2023-11-28T11:21:00.292120+0000","last_active":"2023-11-28T11:21:01.844101+0000","last_peered":"2023-11-28T11:21:01.844101+0000","last_clean":"2023-11-28T11:21:01.844101+0000","last_became_active":"2023-11-28T11:21:00.289430+0000","last_became_peered":"2023-11-28T11:21:00.289430+0000","last_unstale":"2023-11-28T11:21:01.844101+0000","last_undegraded":"2023-11-28T11:21:01.844101+0000","last_fullsized":"2023-11-28T11:21:01.844101+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:41.700110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.298184+0000","last_change":"2023-11-28T11:20:55.771668+0000","last_active":"2023-11-28T11:21:00.298184+0000","last_peered":"2023-11-28T11:21:00.298184+0000","last_clean":"2023-11-28T11:21:00.298184+0000","last_became_active":"2023-11-28T11:20:55.771400+0000","last_became_peered":"2023-11-28T11:20:55.771400+0000","last_unstale":"2023-11-28T11:21:00.298184+0000","last_undegraded":"2023-11-28T11:21:00.298184+0000","last_fullsized":"2023-11-28T11:21:00.298184+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:27:11.291583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.290416+0000","last_change":"2023-11-28T11:20:56.263283+0000","last_active":"2023-11-28T11:21:00.290416+0000","last_peered":"2023-11-28T11:21:00.290416+0000","last_clean":"2023-11-28T11:21:00.290416+0000","last_became_active":"2023-11-28T11:20:56.262787+0000","last_became_peered":"2023-11-28T11:20:56.262787+0000","last_unstale":"2023-11-28T11:21:00.290416+0000","last_undegraded":"2023-11-28T11:21:00.290416+0000","last_fullsized":"2023-11-28T11:21:00.290416+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:40:42.059656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.883480+0000","last_change":"2023-11-28T11:21:00.297947+0000","last_active":"2023-11-28T11:21:01.883480+0000","last_peered":"2023-11-28T11:21:01.883480+0000","last_clean":"2023-11-28T11:21:01.883480+0000","last_became_active":"2023-11-28T11:21:00.297320+0000","last_became_peered":"2023-11-28T11:21:00.297320+0000","last_unstale":"2023-11-28T11:21:01.883480+0000","last_undegraded":"2023-11-28T11:21:01.883480+0000","last_fullsized":"2023-11-28T11:21:01.883480+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:56:39.135021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"27'10","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:13.462618+0000","last_change":"2023-11-28T11:20:55.761529+0000","last_active":"2023-11-28T11:21:13.462618+0000","last_peered":"2023-11-28T11:21:13.462618+0000","last_clean":"2023-11-28T11:21:13.462618+0000","last_became_active":"2023-11-28T11:20:55.759570+0000","last_became_peered":"2023-11-28T11:20:55.759570+0000","last_unstale":"2023-11-28T11:21:13.462618+0000","last_undegraded":"2023-11-28T11:21:13.462618+0000","last_fullsized":"2023-11-28T11:21:13.462618+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:54:56.683286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3746,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.272939+0000","last_change":"2023-11-28T11:20:56.280987+0000","last_active":"2023-11-28T11:21:00.272939+0000","last_peered":"2023-11-28T11:21:00.272939+0000","last_clean":"2023-11-28T11:21:00.272939+0000","last_became_active":"2023-11-28T11:20:56.279820+0000","last_became_peered":"2023-11-28T11:20:56.279820+0000","last_unstale":"2023-11-28T11:21:00.272939+0000","last_undegraded":"2023-11-28T11:21:00.272939+0000","last_fullsized":"2023-11-28T11:21:00.272939+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:50:45.228070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.383399+0000","last_change":"2023-11-28T11:21:00.294226+0000","last_active":"2023-11-28T11:21:01.383399+0000","last_peered":"2023-11-28T11:21:01.383399+0000","last_clean":"2023-11-28T11:21:01.383399+0000","last_became_active":"2023-11-28T11:21:00.293015+0000","last_became_peered":"2023-11-28T11:21:00.293015+0000","last_unstale":"2023-11-28T11:21:01.383399+0000","last_undegraded":"2023-11-28T11:21:01.383399+0000","last_fullsized":"2023-11-28T11:21:01.383399+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:35:00.472959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.298041+0000","last_change":"2023-11-28T11:20:55.766180+0000","last_active":"2023-11-28T11:21:00.298041+0000","last_peered":"2023-11-28T11:21:00.298041+0000","last_clean":"2023-11-28T11:21:00.298041+0000","last_became_active":"2023-11-28T11:20:55.765845+0000","last_became_peered":"2023-11-28T11:20:55.765845+0000","last_unstale":"2023-11-28T11:21:00.298041+0000","last_undegraded":"2023-11-28T11:21:00.298041+0000","last_fullsized":"2023-11-28T11:21:00.298041+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:39:27.467031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.271449+0000","last_change":"2023-11-28T11:20:56.268499+0000","last_active":"2023-11-28T11:21:00.271449+0000","last_peered":"2023-11-28T11:21:00.271449+0000","last_clean":"2023-11-28T11:21:00.271449+0000","last_became_active":"2023-11-28T11:20:56.267802+0000","last_became_peered":"2023-11-28T11:20:56.267802+0000","last_unstale":"2023-11-28T11:21:00.271449+0000","last_undegraded":"2023-11-28T11:21:00.271449+0000","last_fullsized":"2023-11-28T11:21:00.271449+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:11:36.091056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.844455+0000","last_change":"2023-11-28T11:21:00.302400+0000","last_active":"2023-11-28T11:21:01.844455+0000","last_peered":"2023-11-28T11:21:01.844455+0000","last_clean":"2023-11-28T11:21:01.844455+0000","last_became_active":"2023-11-28T11:21:00.301539+0000","last_became_peered":"2023-11-28T11:21:00.301539+0000","last_unstale":"2023-11-28T11:21:01.844455+0000","last_undegraded":"2023-11-28T11:21:01.844455+0000","last_fullsized":"2023-11-28T11:21:01.844455+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:53:42.920160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.290955+0000","last_change":"2023-11-28T11:20:55.766030+0000","last_active":"2023-11-28T11:21:00.290955+0000","last_peered":"2023-11-28T11:21:00.290955+0000","last_clean":"2023-11-28T11:21:00.290955+0000","last_became_active":"2023-11-28T11:20:55.765787+0000","last_became_peered":"2023-11-28T11:20:55.765787+0000","last_unstale":"2023-11-28T11:21:00.290955+0000","last_undegraded":"2023-11-28T11:21:00.290955+0000","last_fullsized":"2023-11-28T11:21:00.290955+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:58:46.726052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.273291+0000","last_change":"2023-11-28T11:20:56.258188+0000","last_active":"2023-11-28T11:21:00.273291+0000","last_peered":"2023-11-28T11:21:00.273291+0000","last_clean":"2023-11-28T11:21:00.273291+0000","last_became_active":"2023-11-28T11:20:56.257558+0000","last_became_peered":"2023-11-28T11:20:56.257558+0000","last_unstale":"2023-11-28T11:21:00.273291+0000","last_undegraded":"2023-11-28T11:21:00.273291+0000","last_fullsized":"2023-11-28T11:21:00.273291+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:11:36.091056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.844327+0000","last_change":"2023-11-28T11:21:00.307457+0000","last_active":"2023-11-28T11:21:01.844327+0000","last_peered":"2023-11-28T11:21:01.844327+0000","last_clean":"2023-11-28T11:21:01.844327+0000","last_became_active":"2023-11-28T11:21:00.307044+0000","last_became_peered":"2023-11-28T11:21:00.307044+0000","last_unstale":"2023-11-28T11:21:01.844327+0000","last_undegraded":"2023-11-28T11:21:01.844327+0000","last_fullsized":"2023-11-28T11:21:01.844327+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:35:00.472959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.288230+0000","last_change":"2023-11-28T11:20:55.765803+0000","last_active":"2023-11-28T11:21:00.288230+0000","last_peered":"2023-11-28T11:21:00.288230+0000","last_clean":"2023-11-28T11:21:00.288230+0000","last_became_active":"2023-11-28T11:20:55.765501+0000","last_became_peered":"2023-11-28T11:20:55.765501+0000","last_unstale":"2023-11-28T11:21:00.288230+0000","last_undegraded":"2023-11-28T11:21:00.288230+0000","last_fullsized":"2023-11-28T11:21:00.288230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:09:40.747993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.291211+0000","last_change":"2023-11-28T11:20:56.268222+0000","last_active":"2023-11-28T11:21:00.291211+0000","last_peered":"2023-11-28T11:21:00.291211+0000","last_clean":"2023-11-28T11:21:00.291211+0000","last_became_active":"2023-11-28T11:20:56.267973+0000","last_became_peered":"2023-11-28T11:20:56.267973+0000","last_unstale":"2023-11-28T11:21:00.291211+0000","last_undegraded":"2023-11-28T11:21:00.291211+0000","last_fullsized":"2023-11-28T11:21:00.291211+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:40:42.059656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.383115+0000","last_change":"2023-11-28T11:21:00.291222+0000","last_active":"2023-11-28T11:21:01.383115+0000","last_peered":"2023-11-28T11:21:01.383115+0000","last_clean":"2023-11-28T11:21:01.383115+0000","last_became_active":"2023-11-28T11:21:00.290768+0000","last_became_peered":"2023-11-28T11:21:00.290768+0000","last_unstale":"2023-11-28T11:21:01.383115+0000","last_undegraded":"2023-11-28T11:21:01.383115+0000","last_fullsized":"2023-11-28T11:21:01.383115+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:56:39.135021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.286149+0000","last_change":"2023-11-28T11:20:55.766427+0000","last_active":"2023-11-28T11:21:00.286149+0000","last_peered":"2023-11-28T11:21:00.286149+0000","last_clean":"2023-11-28T11:21:00.286149+0000","last_became_active":"2023-11-28T11:20:55.764723+0000","last_became_peered":"2023-11-28T11:20:55.764723+0000","last_unstale":"2023-11-28T11:21:00.286149+0000","last_undegraded":"2023-11-28T11:21:00.286149+0000","last_fullsized":"2023-11-28T11:21:00.286149+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:54:56.683286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.272205+0000","last_change":"2023-11-28T11:20:56.274667+0000","last_active":"2023-11-28T11:21:00.272205+0000","last_peered":"2023-11-28T11:21:00.272205+0000","last_clean":"2023-11-28T11:21:00.272205+0000","last_became_active":"2023-11-28T11:20:56.274325+0000","last_became_peered":"2023-11-28T11:20:56.274325+0000","last_unstale":"2023-11-28T11:21:00.272205+0000","last_undegraded":"2023-11-28T11:21:00.272205+0000","last_fullsized":"2023-11-28T11:21:00.272205+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:33:54.639487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.807008+0000","last_change":"2023-11-28T11:21:00.301042+0000","last_active":"2023-11-28T11:21:01.807008+0000","last_peered":"2023-11-28T11:21:01.807008+0000","last_clean":"2023-11-28T11:21:01.807008+0000","last_became_active":"2023-11-28T11:21:00.300581+0000","last_became_peered":"2023-11-28T11:21:00.300581+0000","last_unstale":"2023-11-28T11:21:01.807008+0000","last_undegraded":"2023-11-28T11:21:01.807008+0000","last_fullsized":"2023-11-28T11:21:01.807008+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:21:12.545633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.298437+0000","last_change":"2023-11-28T11:20:55.774510+0000","last_active":"2023-11-28T11:21:00.298437+0000","last_peered":"2023-11-28T11:21:00.298437+0000","last_clean":"2023-11-28T11:21:00.298437+0000","last_became_active":"2023-11-28T11:20:55.774211+0000","last_became_peered":"2023-11-28T11:20:55.774211+0000","last_unstale":"2023-11-28T11:21:00.298437+0000","last_undegraded":"2023-11-28T11:21:00.298437+0000","last_fullsized":"2023-11-28T11:21:00.298437+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:47:23.879025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.273372+0000","last_change":"2023-11-28T11:20:56.263432+0000","last_active":"2023-11-28T11:21:00.273372+0000","last_peered":"2023-11-28T11:21:00.273372+0000","last_clean":"2023-11-28T11:21:00.273372+0000","last_became_active":"2023-11-28T11:20:56.263125+0000","last_became_peered":"2023-11-28T11:20:56.263125+0000","last_unstale":"2023-11-28T11:21:00.273372+0000","last_undegraded":"2023-11-28T11:21:00.273372+0000","last_fullsized":"2023-11-28T11:21:00.273372+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:40:42.059656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.774944+0000","last_change":"2023-11-28T11:21:00.291421+0000","last_active":"2023-11-28T11:21:00.774944+0000","last_peered":"2023-11-28T11:21:00.774944+0000","last_clean":"2023-11-28T11:21:00.774944+0000","last_became_active":"2023-11-28T11:21:00.291077+0000","last_became_peered":"2023-11-28T11:21:00.291077+0000","last_unstale":"2023-11-28T11:21:00.774944+0000","last_undegraded":"2023-11-28T11:21:00.774944+0000","last_fullsized":"2023-11-28T11:21:00.774944+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:26:35.867941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.298743+0000","last_change":"2023-11-28T11:20:55.774653+0000","last_active":"2023-11-28T11:21:00.298743+0000","last_peered":"2023-11-28T11:21:00.298743+0000","last_clean":"2023-11-28T11:21:00.298743+0000","last_became_active":"2023-11-28T11:20:55.774352+0000","last_became_peered":"2023-11-28T11:20:55.774352+0000","last_unstale":"2023-11-28T11:21:00.298743+0000","last_undegraded":"2023-11-28T11:21:00.298743+0000","last_fullsized":"2023-11-28T11:21:00.298743+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:33:55.577608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.274658+0000","last_change":"2023-11-28T11:20:56.266409+0000","last_active":"2023-11-28T11:21:00.274658+0000","last_peered":"2023-11-28T11:21:00.274658+0000","last_clean":"2023-11-28T11:21:00.274658+0000","last_became_active":"2023-11-28T11:20:56.266091+0000","last_became_peered":"2023-11-28T11:20:56.266091+0000","last_unstale":"2023-11-28T11:21:00.274658+0000","last_undegraded":"2023-11-28T11:21:00.274658+0000","last_fullsized":"2023-11-28T11:21:00.274658+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:28:30.120683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.775175+0000","last_change":"2023-11-28T11:21:00.311439+0000","last_active":"2023-11-28T11:21:00.775175+0000","last_peered":"2023-11-28T11:21:00.775175+0000","last_clean":"2023-11-28T11:21:00.775175+0000","last_became_active":"2023-11-28T11:21:00.308732+0000","last_became_peered":"2023-11-28T11:21:00.308732+0000","last_unstale":"2023-11-28T11:21:00.775175+0000","last_undegraded":"2023-11-28T11:21:00.775175+0000","last_fullsized":"2023-11-28T11:21:00.775175+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:53:19.245981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.298398+0000","last_change":"2023-11-28T11:20:55.771105+0000","last_active":"2023-11-28T11:21:00.298398+0000","last_peered":"2023-11-28T11:21:00.298398+0000","last_clean":"2023-11-28T11:21:00.298398+0000","last_became_active":"2023-11-28T11:20:55.770583+0000","last_became_peered":"2023-11-28T11:20:55.770583+0000","last_unstale":"2023-11-28T11:21:00.298398+0000","last_undegraded":"2023-11-28T11:21:00.298398+0000","last_fullsized":"2023-11-28T11:21:00.298398+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:06:01.854141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.286992+0000","last_change":"2023-11-28T11:20:56.263719+0000","last_active":"2023-11-28T11:21:00.286992+0000","last_peered":"2023-11-28T11:21:00.286992+0000","last_clean":"2023-11-28T11:21:00.286992+0000","last_became_active":"2023-11-28T11:20:56.263444+0000","last_became_peered":"2023-11-28T11:20:56.263444+0000","last_unstale":"2023-11-28T11:21:00.286992+0000","last_undegraded":"2023-11-28T11:21:00.286992+0000","last_fullsized":"2023-11-28T11:21:00.286992+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:33:56.583532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.775522+0000","last_change":"2023-11-28T11:21:00.294534+0000","last_active":"2023-11-28T11:21:00.775522+0000","last_peered":"2023-11-28T11:21:00.775522+0000","last_clean":"2023-11-28T11:21:00.775522+0000","last_became_active":"2023-11-28T11:21:00.294055+0000","last_became_peered":"2023-11-28T11:21:00.294055+0000","last_unstale":"2023-11-28T11:21:00.775522+0000","last_undegraded":"2023-11-28T11:21:00.775522+0000","last_fullsized":"2023-11-28T11:21:00.775522+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:21:24.577104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.275166+0000","last_change":"2023-11-28T11:20:55.763501+0000","last_active":"2023-11-28T11:21:00.275166+0000","last_peered":"2023-11-28T11:21:00.275166+0000","last_clean":"2023-11-28T11:21:00.275166+0000","last_became_active":"2023-11-28T11:20:55.763188+0000","last_became_peered":"2023-11-28T11:20:55.763188+0000","last_unstale":"2023-11-28T11:21:00.275166+0000","last_undegraded":"2023-11-28T11:21:00.275166+0000","last_fullsized":"2023-11-28T11:21:00.275166+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:58:46.726052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.290571+0000","last_change":"2023-11-28T11:20:56.260904+0000","last_active":"2023-11-28T11:21:00.290571+0000","last_peered":"2023-11-28T11:21:00.290571+0000","last_clean":"2023-11-28T11:21:00.290571+0000","last_became_active":"2023-11-28T11:20:56.260584+0000","last_became_peered":"2023-11-28T11:20:56.260584+0000","last_unstale":"2023-11-28T11:21:00.290571+0000","last_undegraded":"2023-11-28T11:21:00.290571+0000","last_fullsized":"2023-11-28T11:21:00.290571+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:08.079216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.775218+0000","last_change":"2023-11-28T11:21:00.299500+0000","last_active":"2023-11-28T11:21:00.775218+0000","last_peered":"2023-11-28T11:21:00.775218+0000","last_clean":"2023-11-28T11:21:00.775218+0000","last_became_active":"2023-11-28T11:21:00.299065+0000","last_became_peered":"2023-11-28T11:21:00.299065+0000","last_unstale":"2023-11-28T11:21:00.775218+0000","last_undegraded":"2023-11-28T11:21:00.775218+0000","last_fullsized":"2023-11-28T11:21:00.775218+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:59:57.725780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.271583+0000","last_change":"2023-11-28T11:20:55.767855+0000","last_active":"2023-11-28T11:21:00.271583+0000","last_peered":"2023-11-28T11:21:00.271583+0000","last_clean":"2023-11-28T11:21:00.271583+0000","last_became_active":"2023-11-28T11:20:55.767418+0000","last_became_peered":"2023-11-28T11:20:55.767418+0000","last_unstale":"2023-11-28T11:21:00.271583+0000","last_undegraded":"2023-11-28T11:21:00.271583+0000","last_fullsized":"2023-11-28T11:21:00.271583+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:06:11.286885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.275680+0000","last_change":"2023-11-28T11:20:56.282762+0000","last_active":"2023-11-28T11:21:00.275680+0000","last_peered":"2023-11-28T11:21:00.275680+0000","last_clean":"2023-11-28T11:21:00.275680+0000","last_became_active":"2023-11-28T11:20:56.280729+0000","last_became_peered":"2023-11-28T11:20:56.280729+0000","last_unstale":"2023-11-28T11:21:00.275680+0000","last_undegraded":"2023-11-28T11:21:00.275680+0000","last_fullsized":"2023-11-28T11:21:00.275680+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:53:15.721370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.883819+0000","last_change":"2023-11-28T11:21:00.305663+0000","last_active":"2023-11-28T11:21:01.883819+0000","last_peered":"2023-11-28T11:21:01.883819+0000","last_clean":"2023-11-28T11:21:01.883819+0000","last_became_active":"2023-11-28T11:21:00.305308+0000","last_became_peered":"2023-11-28T11:21:00.305308+0000","last_unstale":"2023-11-28T11:21:01.883819+0000","last_undegraded":"2023-11-28T11:21:01.883819+0000","last_fullsized":"2023-11-28T11:21:01.883819+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:05:27.287399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.287091+0000","last_change":"2023-11-28T11:20:55.774271+0000","last_active":"2023-11-28T11:21:00.287091+0000","last_peered":"2023-11-28T11:21:00.287091+0000","last_clean":"2023-11-28T11:21:00.287091+0000","last_became_active":"2023-11-28T11:20:55.773041+0000","last_became_peered":"2023-11-28T11:20:55.773041+0000","last_unstale":"2023-11-28T11:21:00.287091+0000","last_undegraded":"2023-11-28T11:21:00.287091+0000","last_fullsized":"2023-11-28T11:21:00.287091+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:06:11.286885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.273977+0000","last_change":"2023-11-28T11:20:56.269860+0000","last_active":"2023-11-28T11:21:00.273977+0000","last_peered":"2023-11-28T11:21:00.273977+0000","last_clean":"2023-11-28T11:21:00.273977+0000","last_became_active":"2023-11-28T11:20:56.269355+0000","last_became_peered":"2023-11-28T11:20:56.269355+0000","last_unstale":"2023-11-28T11:21:00.273977+0000","last_undegraded":"2023-11-28T11:21:00.273977+0000","last_fullsized":"2023-11-28T11:21:00.273977+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:24:07.384730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.383366+0000","last_change":"2023-11-28T11:21:00.297532+0000","last_active":"2023-11-28T11:21:01.383366+0000","last_peered":"2023-11-28T11:21:01.383366+0000","last_clean":"2023-11-28T11:21:01.383366+0000","last_became_active":"2023-11-28T11:21:00.295523+0000","last_became_peered":"2023-11-28T11:21:00.295523+0000","last_unstale":"2023-11-28T11:21:01.383366+0000","last_undegraded":"2023-11-28T11:21:01.383366+0000","last_fullsized":"2023-11-28T11:21:01.383366+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:41.700110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.274453+0000","last_change":"2023-11-28T11:20:55.764095+0000","last_active":"2023-11-28T11:21:00.274453+0000","last_peered":"2023-11-28T11:21:00.274453+0000","last_clean":"2023-11-28T11:21:00.274453+0000","last_became_active":"2023-11-28T11:20:55.762342+0000","last_became_peered":"2023-11-28T11:20:55.762342+0000","last_unstale":"2023-11-28T11:21:00.274453+0000","last_undegraded":"2023-11-28T11:21:00.274453+0000","last_fullsized":"2023-11-28T11:21:00.274453+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:25:25.914486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.274835+0000","last_change":"2023-11-28T11:20:56.268448+0000","last_active":"2023-11-28T11:21:00.274835+0000","last_peered":"2023-11-28T11:21:00.274835+0000","last_clean":"2023-11-28T11:21:00.274835+0000","last_became_active":"2023-11-28T11:20:56.268150+0000","last_became_peered":"2023-11-28T11:20:56.268150+0000","last_unstale":"2023-11-28T11:21:00.274835+0000","last_undegraded":"2023-11-28T11:21:00.274835+0000","last_fullsized":"2023-11-28T11:21:00.274835+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:21:21.052493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.777908+0000","last_change":"2023-11-28T11:21:00.306271+0000","last_active":"2023-11-28T11:21:00.777908+0000","last_peered":"2023-11-28T11:21:00.777908+0000","last_clean":"2023-11-28T11:21:00.777908+0000","last_became_active":"2023-11-28T11:21:00.305835+0000","last_became_peered":"2023-11-28T11:21:00.305835+0000","last_unstale":"2023-11-28T11:21:00.777908+0000","last_undegraded":"2023-11-28T11:21:00.777908+0000","last_fullsized":"2023-11-28T11:21:00.777908+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:04:07.374394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.270977+0000","last_change":"2023-11-28T11:20:55.762830+0000","last_active":"2023-11-28T11:21:00.270977+0000","last_peered":"2023-11-28T11:21:00.270977+0000","last_clean":"2023-11-28T11:21:00.270977+0000","last_became_active":"2023-11-28T11:20:55.760841+0000","last_became_peered":"2023-11-28T11:20:55.760841+0000","last_unstale":"2023-11-28T11:21:00.270977+0000","last_undegraded":"2023-11-28T11:21:00.270977+0000","last_fullsized":"2023-11-28T11:21:00.270977+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:54:56.683286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.290757+0000","last_change":"2023-11-28T11:20:56.271356+0000","last_active":"2023-11-28T11:21:00.290757+0000","last_peered":"2023-11-28T11:21:00.290757+0000","last_clean":"2023-11-28T11:21:00.290757+0000","last_became_active":"2023-11-28T11:20:56.271110+0000","last_became_peered":"2023-11-28T11:20:56.271110+0000","last_unstale":"2023-11-28T11:21:00.290757+0000","last_undegraded":"2023-11-28T11:21:00.290757+0000","last_fullsized":"2023-11-28T11:21:00.290757+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:08.079216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.884172+0000","last_change":"2023-11-28T11:21:00.295227+0000","last_active":"2023-11-28T11:21:01.884172+0000","last_peered":"2023-11-28T11:21:01.884172+0000","last_clean":"2023-11-28T11:21:01.884172+0000","last_became_active":"2023-11-28T11:21:00.293463+0000","last_became_peered":"2023-11-28T11:21:00.293463+0000","last_unstale":"2023-11-28T11:21:01.884172+0000","last_undegraded":"2023-11-28T11:21:01.884172+0000","last_fullsized":"2023-11-28T11:21:01.884172+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:41.700110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.288397+0000","last_change":"2023-11-28T11:20:51.020016+0000","last_active":"2023-11-28T11:21:00.288397+0000","last_peered":"2023-11-28T11:21:00.288397+0000","last_clean":"2023-11-28T11:21:00.288397+0000","last_became_active":"2023-11-28T11:20:51.019492+0000","last_became_peered":"2023-11-28T11:20:51.019492+0000","last_unstale":"2023-11-28T11:21:00.288397+0000","last_undegraded":"2023-11-28T11:21:00.288397+0000","last_fullsized":"2023-11-28T11:21:00.288397+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:27.884538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.273192+0000","last_change":"2023-11-28T11:20:55.770921+0000","last_active":"2023-11-28T11:21:00.273192+0000","last_peered":"2023-11-28T11:21:00.273192+0000","last_clean":"2023-11-28T11:21:00.273192+0000","last_became_active":"2023-11-28T11:20:55.770693+0000","last_became_peered":"2023-11-28T11:20:55.770693+0000","last_unstale":"2023-11-28T11:21:00.273192+0000","last_undegraded":"2023-11-28T11:21:00.273192+0000","last_fullsized":"2023-11-28T11:21:00.273192+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:37:05.627049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.285261+0000","last_change":"2023-11-28T11:20:56.257635+0000","last_active":"2023-11-28T11:21:00.285261+0000","last_peered":"2023-11-28T11:21:00.285261+0000","last_clean":"2023-11-28T11:21:00.285261+0000","last_became_active":"2023-11-28T11:20:56.255047+0000","last_became_peered":"2023-11-28T11:20:56.255047+0000","last_unstale":"2023-11-28T11:21:00.285261+0000","last_undegraded":"2023-11-28T11:21:00.285261+0000","last_fullsized":"2023-11-28T11:21:00.285261+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:27:12.297507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.884415+0000","last_change":"2023-11-28T11:21:00.311287+0000","last_active":"2023-11-28T11:21:01.884415+0000","last_peered":"2023-11-28T11:21:01.884415+0000","last_clean":"2023-11-28T11:21:01.884415+0000","last_became_active":"2023-11-28T11:21:00.305808+0000","last_became_peered":"2023-11-28T11:21:00.305808+0000","last_unstale":"2023-11-28T11:21:01.884415+0000","last_undegraded":"2023-11-28T11:21:01.884415+0000","last_fullsized":"2023-11-28T11:21:01.884415+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:56:39.135021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.276695+0000","last_change":"2023-11-28T11:20:50.521908+0000","last_active":"2023-11-28T11:21:00.276695+0000","last_peered":"2023-11-28T11:21:00.276695+0000","last_clean":"2023-11-28T11:21:00.276695+0000","last_became_active":"2023-11-28T11:20:50.521059+0000","last_became_peered":"2023-11-28T11:20:50.521059+0000","last_unstale":"2023-11-28T11:21:00.276695+0000","last_undegraded":"2023-11-28T11:21:00.276695+0000","last_fullsized":"2023-11-28T11:21:00.276695+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:05.302498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.273078+0000","last_change":"2023-11-28T11:20:55.767339+0000","last_active":"2023-11-28T11:21:00.273078+0000","last_peered":"2023-11-28T11:21:00.273078+0000","last_clean":"2023-11-28T11:21:00.273078+0000","last_became_active":"2023-11-28T11:20:55.766065+0000","last_became_peered":"2023-11-28T11:20:55.766065+0000","last_unstale":"2023-11-28T11:21:00.273078+0000","last_undegraded":"2023-11-28T11:21:00.273078+0000","last_fullsized":"2023-11-28T11:21:00.273078+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:06:11.286885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.289236+0000","last_change":"2023-11-28T11:20:56.261611+0000","last_active":"2023-11-28T11:21:00.289236+0000","last_peered":"2023-11-28T11:21:00.289236+0000","last_clean":"2023-11-28T11:21:00.289236+0000","last_became_active":"2023-11-28T11:20:56.260557+0000","last_became_peered":"2023-11-28T11:20:56.260557+0000","last_unstale":"2023-11-28T11:21:00.289236+0000","last_undegraded":"2023-11-28T11:21:00.289236+0000","last_fullsized":"2023-11-28T11:21:00.289236+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:14:34.876987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.775674+0000","last_change":"2023-11-28T11:21:00.301087+0000","last_active":"2023-11-28T11:21:00.775674+0000","last_peered":"2023-11-28T11:21:00.775674+0000","last_clean":"2023-11-28T11:21:00.775674+0000","last_became_active":"2023-11-28T11:21:00.300674+0000","last_became_peered":"2023-11-28T11:21:00.300674+0000","last_unstale":"2023-11-28T11:21:00.775674+0000","last_undegraded":"2023-11-28T11:21:00.775674+0000","last_fullsized":"2023-11-28T11:21:00.775674+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:13:50.207611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.282488+0000","last_change":"2023-11-28T11:20:50.619004+0000","last_active":"2023-11-28T11:21:00.282488+0000","last_peered":"2023-11-28T11:21:00.282488+0000","last_clean":"2023-11-28T11:21:00.282488+0000","last_became_active":"2023-11-28T11:20:50.618315+0000","last_became_peered":"2023-11-28T11:20:50.618315+0000","last_unstale":"2023-11-28T11:21:00.282488+0000","last_undegraded":"2023-11-28T11:21:00.282488+0000","last_fullsized":"2023-11-28T11:21:00.282488+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:27.884538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.283998+0000","last_change":"2023-11-28T11:20:55.758384+0000","last_active":"2023-11-28T11:21:00.283998+0000","last_peered":"2023-11-28T11:21:00.283998+0000","last_clean":"2023-11-28T11:21:00.283998+0000","last_became_active":"2023-11-28T11:20:55.758125+0000","last_became_peered":"2023-11-28T11:20:55.758125+0000","last_unstale":"2023-11-28T11:21:00.283998+0000","last_undegraded":"2023-11-28T11:21:00.283998+0000","last_fullsized":"2023-11-28T11:21:00.283998+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:47:38.102588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.290667+0000","last_change":"2023-11-28T11:20:56.253940+0000","last_active":"2023-11-28T11:21:00.290667+0000","last_peered":"2023-11-28T11:21:00.290667+0000","last_clean":"2023-11-28T11:21:00.290667+0000","last_became_active":"2023-11-28T11:20:56.253635+0000","last_became_peered":"2023-11-28T11:20:56.253635+0000","last_unstale":"2023-11-28T11:21:00.290667+0000","last_undegraded":"2023-11-28T11:21:00.290667+0000","last_fullsized":"2023-11-28T11:21:00.290667+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:00:17.958443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.806956+0000","last_change":"2023-11-28T11:21:00.311172+0000","last_active":"2023-11-28T11:21:01.806956+0000","last_peered":"2023-11-28T11:21:01.806956+0000","last_clean":"2023-11-28T11:21:01.806956+0000","last_became_active":"2023-11-28T11:21:00.310707+0000","last_became_peered":"2023-11-28T11:21:00.310707+0000","last_unstale":"2023-11-28T11:21:01.806956+0000","last_undegraded":"2023-11-28T11:21:01.806956+0000","last_fullsized":"2023-11-28T11:21:01.806956+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:41:46.264594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.273954+0000","last_change":"2023-11-28T11:20:50.522582+0000","last_active":"2023-11-28T11:21:00.273954+0000","last_peered":"2023-11-28T11:21:00.273954+0000","last_clean":"2023-11-28T11:21:00.273954+0000","last_became_active":"2023-11-28T11:20:50.522157+0000","last_became_peered":"2023-11-28T11:20:50.522157+0000","last_unstale":"2023-11-28T11:21:00.273954+0000","last_undegraded":"2023-11-28T11:21:00.273954+0000","last_fullsized":"2023-11-28T11:21:00.273954+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:27.884538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.286501+0000","last_change":"2023-11-28T11:20:43.294845+0000","last_active":"2023-11-28T11:21:00.286501+0000","last_peered":"2023-11-28T11:21:00.286501+0000","last_clean":"2023-11-28T11:21:00.286501+0000","last_became_active":"2023-11-28T11:20:43.294483+0000","last_became_peered":"2023-11-28T11:20:43.294483+0000","last_unstale":"2023-11-28T11:21:00.286501+0000","last_undegraded":"2023-11-28T11:21:00.286501+0000","last_fullsized":"2023-11-28T11:21:00.286501+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:42.270512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:42.270512+0000","last_clean_scrub_stamp":"2023-11-28T11:20:42.270512+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:17:03.376680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.270966+0000","last_change":"2023-11-28T11:20:55.767420+0000","last_active":"2023-11-28T11:21:00.270966+0000","last_peered":"2023-11-28T11:21:00.270966+0000","last_clean":"2023-11-28T11:21:00.270966+0000","last_became_active":"2023-11-28T11:20:55.766809+0000","last_became_peered":"2023-11-28T11:20:55.766809+0000","last_unstale":"2023-11-28T11:21:00.270966+0000","last_undegraded":"2023-11-28T11:21:00.270966+0000","last_fullsized":"2023-11-28T11:21:00.270966+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:37:05.627049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.286972+0000","last_change":"2023-11-28T11:20:56.276228+0000","last_active":"2023-11-28T11:21:00.286972+0000","last_peered":"2023-11-28T11:21:00.286972+0000","last_clean":"2023-11-28T11:21:00.286972+0000","last_became_active":"2023-11-28T11:20:56.275905+0000","last_became_peered":"2023-11-28T11:20:56.275905+0000","last_unstale":"2023-11-28T11:21:00.286972+0000","last_undegraded":"2023-11-28T11:21:00.286972+0000","last_fullsized":"2023-11-28T11:21:00.286972+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:47:24.884949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.383236+0000","last_change":"2023-11-28T11:21:00.305959+0000","last_active":"2023-11-28T11:21:01.383236+0000","last_peered":"2023-11-28T11:21:01.383236+0000","last_clean":"2023-11-28T11:21:01.383236+0000","last_became_active":"2023-11-28T11:21:00.305530+0000","last_became_peered":"2023-11-28T11:21:00.305530+0000","last_unstale":"2023-11-28T11:21:01.383236+0000","last_undegraded":"2023-11-28T11:21:01.383236+0000","last_fullsized":"2023-11-28T11:21:01.383236+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:22:04.224877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.286683+0000","last_change":"2023-11-28T11:20:50.526865+0000","last_active":"2023-11-28T11:21:00.286683+0000","last_peered":"2023-11-28T11:21:00.286683+0000","last_clean":"2023-11-28T11:21:00.286683+0000","last_became_active":"2023-11-28T11:20:50.526576+0000","last_became_peered":"2023-11-28T11:20:50.526576+0000","last_unstale":"2023-11-28T11:21:00.286683+0000","last_undegraded":"2023-11-28T11:21:00.286683+0000","last_fullsized":"2023-11-28T11:21:00.286683+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:09:18.460112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.289849+0000","last_change":"2023-11-28T11:20:55.772865+0000","last_active":"2023-11-28T11:21:00.289849+0000","last_peered":"2023-11-28T11:21:00.289849+0000","last_clean":"2023-11-28T11:21:00.289849+0000","last_became_active":"2023-11-28T11:20:55.772559+0000","last_became_peered":"2023-11-28T11:20:55.772559+0000","last_unstale":"2023-11-28T11:21:00.289849+0000","last_undegraded":"2023-11-28T11:21:00.289849+0000","last_fullsized":"2023-11-28T11:21:00.289849+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:37:05.627049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.290171+0000","last_change":"2023-11-28T11:20:56.268570+0000","last_active":"2023-11-28T11:21:00.290171+0000","last_peered":"2023-11-28T11:21:00.290171+0000","last_clean":"2023-11-28T11:21:00.290171+0000","last_became_active":"2023-11-28T11:20:56.264598+0000","last_became_peered":"2023-11-28T11:20:56.264598+0000","last_unstale":"2023-11-28T11:21:00.290171+0000","last_undegraded":"2023-11-28T11:21:00.290171+0000","last_fullsized":"2023-11-28T11:21:00.290171+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:14:34.876987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.778849+0000","last_change":"2023-11-28T11:21:00.311861+0000","last_active":"2023-11-28T11:21:00.778849+0000","last_peered":"2023-11-28T11:21:00.778849+0000","last_clean":"2023-11-28T11:21:00.778849+0000","last_became_active":"2023-11-28T11:21:00.311394+0000","last_became_peered":"2023-11-28T11:21:00.311394+0000","last_unstale":"2023-11-28T11:21:00.778849+0000","last_undegraded":"2023-11-28T11:21:00.778849+0000","last_fullsized":"2023-11-28T11:21:00.778849+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:25:55.767021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.298282+0000","last_change":"2023-11-28T11:20:51.020043+0000","last_active":"2023-11-28T11:21:00.298282+0000","last_peered":"2023-11-28T11:21:00.298282+0000","last_clean":"2023-11-28T11:21:00.298282+0000","last_became_active":"2023-11-28T11:20:51.019702+0000","last_became_peered":"2023-11-28T11:20:51.019702+0000","last_unstale":"2023-11-28T11:21:00.298282+0000","last_undegraded":"2023-11-28T11:21:00.298282+0000","last_fullsized":"2023-11-28T11:21:00.298282+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:26:20.246873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.288664+0000","last_change":"2023-11-28T11:20:55.760221+0000","last_active":"2023-11-28T11:21:00.288664+0000","last_peered":"2023-11-28T11:21:00.288664+0000","last_clean":"2023-11-28T11:21:00.288664+0000","last_became_active":"2023-11-28T11:20:55.759960+0000","last_became_peered":"2023-11-28T11:20:55.759960+0000","last_unstale":"2023-11-28T11:21:00.288664+0000","last_undegraded":"2023-11-28T11:21:00.288664+0000","last_fullsized":"2023-11-28T11:21:00.288664+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:54:56.683286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.290072+0000","last_change":"2023-11-28T11:20:56.251820+0000","last_active":"2023-11-28T11:21:00.290072+0000","last_peered":"2023-11-28T11:21:00.290072+0000","last_clean":"2023-11-28T11:21:00.290072+0000","last_became_active":"2023-11-28T11:20:56.251550+0000","last_became_peered":"2023-11-28T11:20:56.251550+0000","last_unstale":"2023-11-28T11:21:00.290072+0000","last_undegraded":"2023-11-28T11:21:00.290072+0000","last_fullsized":"2023-11-28T11:21:00.290072+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:45:14.478416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.777012+0000","last_change":"2023-11-28T11:21:00.298541+0000","last_active":"2023-11-28T11:21:00.777012+0000","last_peered":"2023-11-28T11:21:00.777012+0000","last_clean":"2023-11-28T11:21:00.777012+0000","last_became_active":"2023-11-28T11:21:00.298160+0000","last_became_peered":"2023-11-28T11:21:00.298160+0000","last_unstale":"2023-11-28T11:21:00.777012+0000","last_undegraded":"2023-11-28T11:21:00.777012+0000","last_fullsized":"2023-11-28T11:21:00.777012+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:16:48.936607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.272276+0000","last_change":"2023-11-28T11:20:55.766385+0000","last_active":"2023-11-28T11:21:00.272276+0000","last_peered":"2023-11-28T11:21:00.272276+0000","last_clean":"2023-11-28T11:21:00.272276+0000","last_became_active":"2023-11-28T11:20:55.766148+0000","last_became_peered":"2023-11-28T11:20:55.766148+0000","last_unstale":"2023-11-28T11:21:00.272276+0000","last_undegraded":"2023-11-28T11:21:00.272276+0000","last_fullsized":"2023-11-28T11:21:00.272276+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:33:02.765966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.288602+0000","last_change":"2023-11-28T11:20:50.524251+0000","last_active":"2023-11-28T11:21:00.288602+0000","last_peered":"2023-11-28T11:21:00.288602+0000","last_clean":"2023-11-28T11:21:00.288602+0000","last_became_active":"2023-11-28T11:20:50.524012+0000","last_became_peered":"2023-11-28T11:20:50.524012+0000","last_unstale":"2023-11-28T11:21:00.288602+0000","last_undegraded":"2023-11-28T11:21:00.288602+0000","last_fullsized":"2023-11-28T11:21:00.288602+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:05.302498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.270885+0000","last_change":"2023-11-28T11:20:56.266167+0000","last_active":"2023-11-28T11:21:00.270885+0000","last_peered":"2023-11-28T11:21:00.270885+0000","last_clean":"2023-11-28T11:21:00.270885+0000","last_became_active":"2023-11-28T11:20:56.264890+0000","last_became_peered":"2023-11-28T11:20:56.264890+0000","last_unstale":"2023-11-28T11:21:00.270885+0000","last_undegraded":"2023-11-28T11:21:00.270885+0000","last_fullsized":"2023-11-28T11:21:00.270885+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:31:26.827850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.843672+0000","last_change":"2023-11-28T11:21:00.295193+0000","last_active":"2023-11-28T11:21:01.843672+0000","last_peered":"2023-11-28T11:21:01.843672+0000","last_clean":"2023-11-28T11:21:01.843672+0000","last_became_active":"2023-11-28T11:21:00.294668+0000","last_became_peered":"2023-11-28T11:21:00.294668+0000","last_unstale":"2023-11-28T11:21:01.843672+0000","last_undegraded":"2023-11-28T11:21:01.843672+0000","last_fullsized":"2023-11-28T11:21:01.843672+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:53:58.267145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.287437+0000","last_change":"2023-11-28T11:20:50.524838+0000","last_active":"2023-11-28T11:21:00.287437+0000","last_peered":"2023-11-28T11:21:00.287437+0000","last_clean":"2023-11-28T11:21:00.287437+0000","last_became_active":"2023-11-28T11:20:50.524402+0000","last_became_peered":"2023-11-28T11:20:50.524402+0000","last_unstale":"2023-11-28T11:21:00.287437+0000","last_undegraded":"2023-11-28T11:21:00.287437+0000","last_fullsized":"2023-11-28T11:21:00.287437+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:22.699447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"26'1","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.725299+0000","last_change":"2023-11-28T11:20:55.765528+0000","last_active":"2023-11-28T11:21:00.725299+0000","last_peered":"2023-11-28T11:21:00.725299+0000","last_clean":"2023-11-28T11:21:00.725299+0000","last_became_active":"2023-11-28T11:20:55.765244+0000","last_became_peered":"2023-11-28T11:20:55.765244+0000","last_unstale":"2023-11-28T11:21:00.725299+0000","last_undegraded":"2023-11-28T11:21:00.725299+0000","last_fullsized":"2023-11-28T11:21:00.725299+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:47:38.102588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.274546+0000","last_change":"2023-11-28T11:20:56.275241+0000","last_active":"2023-11-28T11:21:00.274546+0000","last_peered":"2023-11-28T11:21:00.274546+0000","last_clean":"2023-11-28T11:21:00.274546+0000","last_became_active":"2023-11-28T11:20:56.271190+0000","last_became_peered":"2023-11-28T11:20:56.271190+0000","last_unstale":"2023-11-28T11:21:00.274546+0000","last_undegraded":"2023-11-28T11:21:00.274546+0000","last_fullsized":"2023-11-28T11:21:00.274546+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:45:14.478416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.883997+0000","last_change":"2023-11-28T11:21:00.297029+0000","last_active":"2023-11-28T11:21:01.883997+0000","last_peered":"2023-11-28T11:21:01.883997+0000","last_clean":"2023-11-28T11:21:01.883997+0000","last_became_active":"2023-11-28T11:21:00.296497+0000","last_became_peered":"2023-11-28T11:21:00.296497+0000","last_unstale":"2023-11-28T11:21:01.883997+0000","last_undegraded":"2023-11-28T11:21:01.883997+0000","last_fullsized":"2023-11-28T11:21:01.883997+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:41.700110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.290180+0000","last_change":"2023-11-28T11:20:55.773378+0000","last_active":"2023-11-28T11:21:00.290180+0000","last_peered":"2023-11-28T11:21:00.290180+0000","last_clean":"2023-11-28T11:21:00.290180+0000","last_became_active":"2023-11-28T11:20:55.773095+0000","last_became_peered":"2023-11-28T11:20:55.773095+0000","last_unstale":"2023-11-28T11:21:00.290180+0000","last_undegraded":"2023-11-28T11:21:00.290180+0000","last_fullsized":"2023-11-28T11:21:00.290180+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:54:56.683286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.280606+0000","last_change":"2023-11-28T11:20:56.258626+0000","last_active":"2023-11-28T11:21:00.280606+0000","last_peered":"2023-11-28T11:21:00.280606+0000","last_clean":"2023-11-28T11:21:00.280606+0000","last_became_active":"2023-11-28T11:20:56.258350+0000","last_became_peered":"2023-11-28T11:20:56.258350+0000","last_unstale":"2023-11-28T11:21:00.280606+0000","last_undegraded":"2023-11-28T11:21:00.280606+0000","last_fullsized":"2023-11-28T11:21:00.280606+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:33:03.771889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.806818+0000","last_change":"2023-11-28T11:21:00.306341+0000","last_active":"2023-11-28T11:21:01.806818+0000","last_peered":"2023-11-28T11:21:01.806818+0000","last_clean":"2023-11-28T11:21:01.806818+0000","last_became_active":"2023-11-28T11:21:00.304545+0000","last_became_peered":"2023-11-28T11:21:00.304545+0000","last_unstale":"2023-11-28T11:21:01.806818+0000","last_undegraded":"2023-11-28T11:21:01.806818+0000","last_fullsized":"2023-11-28T11:21:01.806818+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:53:58.267145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.275266+0000","last_change":"2023-11-28T11:20:55.766530+0000","last_active":"2023-11-28T11:21:00.275266+0000","last_peered":"2023-11-28T11:21:00.275266+0000","last_clean":"2023-11-28T11:21:00.275266+0000","last_became_active":"2023-11-28T11:20:55.766205+0000","last_became_peered":"2023-11-28T11:20:55.766205+0000","last_unstale":"2023-11-28T11:21:00.275266+0000","last_undegraded":"2023-11-28T11:21:00.275266+0000","last_fullsized":"2023-11-28T11:21:00.275266+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:33:02.765966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.273248+0000","last_change":"2023-11-28T11:20:56.279673+0000","last_active":"2023-11-28T11:21:00.273248+0000","last_peered":"2023-11-28T11:21:00.273248+0000","last_clean":"2023-11-28T11:21:00.273248+0000","last_became_active":"2023-11-28T11:20:56.276338+0000","last_became_peered":"2023-11-28T11:20:56.276338+0000","last_unstale":"2023-11-28T11:21:00.273248+0000","last_undegraded":"2023-11-28T11:21:00.273248+0000","last_fullsized":"2023-11-28T11:21:00.273248+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:59:54.201169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.382912+0000","last_change":"2023-11-28T11:21:00.308516+0000","last_active":"2023-11-28T11:21:01.382912+0000","last_peered":"2023-11-28T11:21:01.382912+0000","last_clean":"2023-11-28T11:21:01.382912+0000","last_became_active":"2023-11-28T11:21:00.307909+0000","last_became_peered":"2023-11-28T11:21:00.307909+0000","last_unstale":"2023-11-28T11:21:01.382912+0000","last_undegraded":"2023-11-28T11:21:01.382912+0000","last_fullsized":"2023-11-28T11:21:01.382912+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:53:42.920160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.273087+0000","last_change":"2023-11-28T11:20:55.772607+0000","last_active":"2023-11-28T11:21:00.273087+0000","last_peered":"2023-11-28T11:21:00.273087+0000","last_clean":"2023-11-28T11:21:00.273087+0000","last_became_active":"2023-11-28T11:20:55.772335+0000","last_became_peered":"2023-11-28T11:20:55.772335+0000","last_unstale":"2023-11-28T11:21:00.273087+0000","last_undegraded":"2023-11-28T11:21:00.273087+0000","last_fullsized":"2023-11-28T11:21:00.273087+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:23:36.801918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.274556+0000","last_change":"2023-11-28T11:20:56.279846+0000","last_active":"2023-11-28T11:21:00.274556+0000","last_peered":"2023-11-28T11:21:00.274556+0000","last_clean":"2023-11-28T11:21:00.274556+0000","last_became_active":"2023-11-28T11:20:56.276529+0000","last_became_peered":"2023-11-28T11:20:56.276529+0000","last_unstale":"2023-11-28T11:21:00.274556+0000","last_undegraded":"2023-11-28T11:21:00.274556+0000","last_fullsized":"2023-11-28T11:21:00.274556+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:29:57.585501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.776665+0000","last_change":"2023-11-28T11:21:00.294235+0000","last_active":"2023-11-28T11:21:00.776665+0000","last_peered":"2023-11-28T11:21:00.776665+0000","last_clean":"2023-11-28T11:21:00.776665+0000","last_became_active":"2023-11-28T11:21:00.293813+0000","last_became_peered":"2023-11-28T11:21:00.293813+0000","last_unstale":"2023-11-28T11:21:00.776665+0000","last_undegraded":"2023-11-28T11:21:00.776665+0000","last_fullsized":"2023-11-28T11:21:00.776665+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:44:23.280744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.271358+0000","last_change":"2023-11-28T11:20:55.767813+0000","last_active":"2023-11-28T11:21:00.271358+0000","last_peered":"2023-11-28T11:21:00.271358+0000","last_clean":"2023-11-28T11:21:00.271358+0000","last_became_active":"2023-11-28T11:20:55.765782+0000","last_became_peered":"2023-11-28T11:20:55.765782+0000","last_unstale":"2023-11-28T11:21:00.271358+0000","last_undegraded":"2023-11-28T11:21:00.271358+0000","last_fullsized":"2023-11-28T11:21:00.271358+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:47:38.102588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.271000+0000","last_change":"2023-11-28T11:20:56.265782+0000","last_active":"2023-11-28T11:21:00.271000+0000","last_peered":"2023-11-28T11:21:00.271000+0000","last_clean":"2023-11-28T11:21:00.271000+0000","last_became_active":"2023-11-28T11:20:56.265102+0000","last_became_peered":"2023-11-28T11:20:56.265102+0000","last_unstale":"2023-11-28T11:21:00.271000+0000","last_undegraded":"2023-11-28T11:21:00.271000+0000","last_fullsized":"2023-11-28T11:21:00.271000+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:45:14.478416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-11-28T11:21:01.883772+0000","last_change":"2023-11-28T11:21:00.297751+0000","last_active":"2023-11-28T11:21:01.883772+0000","last_peered":"2023-11-28T11:21:01.883772+0000","last_clean":"2023-11-28T11:21:01.883772+0000","last_became_active":"2023-11-28T11:21:00.297281+0000","last_became_peered":"2023-11-28T11:21:00.297281+0000","last_unstale":"2023-11-28T11:21:01.883772+0000","last_undegraded":"2023-11-28T11:21:01.883772+0000","last_fullsized":"2023-11-28T11:21:01.883772+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:59.255083+0000","last_clean_scrub_stamp":"2023-11-28T11:20:59.255083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:42:41.996906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.292230+0000","last_change":"2023-11-28T11:20:55.764439+0000","last_active":"2023-11-28T11:21:00.292230+0000","last_peered":"2023-11-28T11:21:00.292230+0000","last_clean":"2023-11-28T11:21:00.292230+0000","last_became_active":"2023-11-28T11:20:55.764165+0000","last_became_peered":"2023-11-28T11:20:55.764165+0000","last_unstale":"2023-11-28T11:21:00.292230+0000","last_undegraded":"2023-11-28T11:21:00.292230+0000","last_fullsized":"2023-11-28T11:21:00.292230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:54.724548+0000","last_clean_scrub_stamp":"2023-11-28T11:20:54.724548+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:33:02.765966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.273131+0000","last_change":"2023-11-28T11:20:56.265074+0000","last_active":"2023-11-28T11:21:00.273131+0000","last_peered":"2023-11-28T11:21:00.273131+0000","last_clean":"2023-11-28T11:21:00.273131+0000","last_became_active":"2023-11-28T11:20:56.264495+0000","last_became_peered":"2023-11-28T11:20:56.264495+0000","last_unstale":"2023-11-28T11:21:00.273131+0000","last_undegraded":"2023-11-28T11:21:00.273131+0000","last_fullsized":"2023-11-28T11:21:00.273131+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:31:26.827850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.286246+0000","last_change":"2023-11-28T11:20:56.260972+0000","last_active":"2023-11-28T11:21:00.286246+0000","last_peered":"2023-11-28T11:21:00.286246+0000","last_clean":"2023-11-28T11:21:00.286246+0000","last_became_active":"2023-11-28T11:20:56.259491+0000","last_became_peered":"2023-11-28T11:20:56.259491+0000","last_unstale":"2023-11-28T11:21:00.286246+0000","last_undegraded":"2023-11-28T11:21:00.286246+0000","last_fullsized":"2023-11-28T11:21:00.286246+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:06:02.860064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.271158+0000","last_change":"2023-11-28T11:20:56.263917+0000","last_active":"2023-11-28T11:21:00.271158+0000","last_peered":"2023-11-28T11:21:00.271158+0000","last_clean":"2023-11-28T11:21:00.271158+0000","last_became_active":"2023-11-28T11:20:56.261946+0000","last_became_peered":"2023-11-28T11:20:56.261946+0000","last_unstale":"2023-11-28T11:21:00.271158+0000","last_undegraded":"2023-11-28T11:21:00.271158+0000","last_fullsized":"2023-11-28T11:21:00.271158+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:40:42.059656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.274852+0000","last_change":"2023-11-28T11:20:56.272089+0000","last_active":"2023-11-28T11:21:00.274852+0000","last_peered":"2023-11-28T11:21:00.274852+0000","last_clean":"2023-11-28T11:21:00.274852+0000","last_became_active":"2023-11-28T11:20:56.271704+0000","last_became_peered":"2023-11-28T11:20:56.271704+0000","last_unstale":"2023-11-28T11:21:00.274852+0000","last_undegraded":"2023-11-28T11:21:00.274852+0000","last_fullsized":"2023-11-28T11:21:00.274852+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:33:54.639487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.270968+0000","last_change":"2023-11-28T11:20:56.270543+0000","last_active":"2023-11-28T11:21:00.270968+0000","last_peered":"2023-11-28T11:21:00.270968+0000","last_clean":"2023-11-28T11:21:00.270968+0000","last_became_active":"2023-11-28T11:20:56.270310+0000","last_became_peered":"2023-11-28T11:20:56.270310+0000","last_unstale":"2023-11-28T11:21:00.270968+0000","last_undegraded":"2023-11-28T11:21:00.270968+0000","last_fullsized":"2023-11-28T11:21:00.270968+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:21:21.052493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T11:21:00.288589+0000","last_change":"2023-11-28T11:20:56.262145+0000","last_active":"2023-11-28T11:21:00.288589+0000","last_peered":"2023-11-28T11:21:00.288589+0000","last_clean":"2023-11-28T11:21:00.288589+0000","last_became_active":"2023-11-28T11:20:56.260881+0000","last_became_peered":"2023-11-28T11:20:56.260881+0000","last_unstale":"2023-11-28T11:21:00.288589+0000","last_undegraded":"2023-11-28T11:21:00.288589+0000","last_fullsized":"2023-11-28T11:21:00.288589+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:55.730472+0000","last_clean_scrub_stamp":"2023-11-28T11:20:55.730472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:32:52.386018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4528,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574860,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9280,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362560,"statfs":{"total":96636764160,"available":96627261440,"internally_reserved":0,"allocated":909312,"data_stored":713609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574860,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8100,"kb_used_data":284,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363740,"statfs":{"total":96636764160,"available":96628469760,"internally_reserved":0,"allocated":290816,"data_stored":119901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.409}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574860,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8164,"kb_used_data":284,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363676,"statfs":{"total":96636764160,"available":96628404224,"internally_reserved":0,"allocated":290816,"data_stored":119901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88900000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574860,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9260,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362580,"statfs":{"total":96636764160,"available":96627281920,"internally_reserved":0,"allocated":888832,"data_stored":710835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96799999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.681}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.9790000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3300000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.6160000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.0099999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8100,"kb_used_data":284,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363740,"statfs":{"total":96636764160,"available":96628469760,"internally_reserved":0,"allocated":290816,"data_stored":119901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.1109999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.6640000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.1529999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.0529999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3560000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.319}]}]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8168,"kb_used_data":288,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363672,"statfs":{"total":96636764160,"available":96628400128,"internally_reserved":0,"allocated":294912,"data_stored":119991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95999999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8248,"kb_used_data":304,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363592,"statfs":{"total":96636764160,"available":96628318208,"internally_reserved":0,"allocated":311296,"data_stored":123703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8172,"kb_used_data":292,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363668,"statfs":{"total":96636764160,"available":96628396032,"internally_reserved":0,"allocated":299008,"data_stored":120537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:21:24.053 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-28T11:21:24.053 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-11-28T11:21:24.053 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-11-28T11:21:24.053 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T11:21:24.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.281+0000 7f81dddff700 1 Processor -- start 2023-11-28T11:21:24.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.282+0000 7f81dddff700 1 -- start start 2023-11-28T11:21:24.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.283+0000 7f81dddff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81d80d2df0 0x7f81d80cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:24.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.283+0000 7f81dddff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f81d80b4fc0 0x7f81d80d31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:24.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.283+0000 7f81dddff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81d80d1950 con 0x7f81d80b1c60 2023-11-28T11:21:24.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.283+0000 7f81dddff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81d80d3d30 con 0x7f81d80b4fc0 2023-11-28T11:21:24.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.283+0000 7f81dddff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81d80b2500 con 0x7f81d80d2df0 2023-11-28T11:21:24.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.283+0000 7f81d7fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f81d80b1c60 0x7f81d80cf010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47512/0 (socket says 172.21.15.92:47512) 2023-11-28T11:21:24.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.283+0000 7f81d7fff700 1 -- 172.21.15.92:0/4289309429 learned_addr learned my addr 172.21.15.92:0/4289309429 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:24.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.283+0000 7f81d77fe700 1 --2- 172.21.15.92:0/4289309429 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81d80d2df0 0x7f81d80cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:24.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.283+0000 7f81d6ffd700 1 -- 172.21.15.92:0/4289309429 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 409080103 0 0) 0x7f81d80d1950 con 0x7f81d80b1c60 2023-11-28T11:21:24.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.283+0000 7f81cffff700 1 --2- 172.21.15.92:0/4289309429 >> v2:172.21.15.169:3300/0 conn(0x7f81d80b4fc0 0x7f81d80d31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:24.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.283+0000 7f81d6ffd700 1 -- 172.21.15.92:0/4289309429 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81c40035b0 con 0x7f81d80b1c60 2023-11-28T11:21:24.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.284+0000 7f81d6ffd700 1 -- 172.21.15.92:0/4289309429 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2531032794 0 0) 0x7f81c40035b0 con 0x7f81d80b1c60 2023-11-28T11:21:24.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.284+0000 7f81d6ffd700 1 -- 172.21.15.92:0/4289309429 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81d80d2df0 msgr2=0x7f81d80cd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:24.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.284+0000 7f81d6ffd700 1 --2- 172.21.15.92:0/4289309429 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81d80d2df0 0x7f81d80cd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:24.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.284+0000 7f81d6ffd700 1 -- 172.21.15.92:0/4289309429 >> v2:172.21.15.169:3300/0 conn(0x7f81d80b4fc0 msgr2=0x7f81d80d31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:24.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.284+0000 7f81d6ffd700 1 --2- 172.21.15.92:0/4289309429 >> v2:172.21.15.169:3300/0 conn(0x7f81d80b4fc0 0x7f81d80d31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:24.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.284+0000 7f81d6ffd700 1 -- 172.21.15.92:0/4289309429 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d80d3a50 con 0x7f81d80b1c60 2023-11-28T11:21:24.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.284+0000 7f81d6ffd700 1 -- 172.21.15.92:0/4289309429 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532161587 0 0) 0x7f81d00019f0 con 0x7f81d80b1c60 2023-11-28T11:21:24.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.284+0000 7f81d6ffd700 1 -- 172.21.15.92:0/4289309429 >> v1:172.21.15.92:6789/0 conn(0x7f81d80b1c60 legacy=0x7f81d80cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:24.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.284+0000 7f81d6ffd700 1 --2- 172.21.15.92:0/4289309429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81c4003a00 0x7f81c4003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:24.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.284+0000 7f81d6ffd700 1 --2- 172.21.15.92:0/4289309429 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81d80d2df0 0x7f81c4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:24.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.284+0000 7f81d6ffd700 1 --2- 172.21.15.92:0/4289309429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81c4004d90 0x7f81c4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:24.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.284+0000 7f81d6ffd700 1 -- 172.21.15.92:0/4289309429 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d80d3d30 con 0x7f81c4004d90 2023-11-28T11:21:24.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.284+0000 7f81d6ffd700 1 -- 172.21.15.92:0/4289309429 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d80b2500 con 0x7f81c4003a00 2023-11-28T11:21:24.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.284+0000 7f81d6ffd700 1 -- 172.21.15.92:0/4289309429 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81c400ba10 con 0x7f81d80d2df0 2023-11-28T11:21:24.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.284+0000 7f81d77fe700 1 --2- 172.21.15.92:0/4289309429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81c4003a00 0x7f81c4003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:24.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.284+0000 7f81d7fff700 1 --2- 172.21.15.92:0/4289309429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81c4004d90 0x7f81c4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:24.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.284+0000 7f81cffff700 1 --2- 172.21.15.92:0/4289309429 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81d80d2df0 0x7f81c4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:24.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.285+0000 7f81d7fff700 1 -- 172.21.15.92:0/4289309429 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81d80d2df0 msgr2=0x7f81c4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:24.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.285+0000 7f81d7fff700 1 --2- 172.21.15.92:0/4289309429 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81d80d2df0 0x7f81c4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:24.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.285+0000 7f81d7fff700 1 -- 172.21.15.92:0/4289309429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81c4003a00 msgr2=0x7f81c4003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:24.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.285+0000 7f81d7fff700 1 --2- 172.21.15.92:0/4289309429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81c4003a00 0x7f81c4003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:24.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.285+0000 7f81d7fff700 1 -- 172.21.15.92:0/4289309429 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81c40035b0 con 0x7f81c4004d90 2023-11-28T11:21:24.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.285+0000 7f81d77fe700 1 --2- 172.21.15.92:0/4289309429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81c4003a00 0x7f81c4003df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:21:24.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.285+0000 7f81d7fff700 1 --2- 172.21.15.92:0/4289309429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81c4004d90 0x7f81c4005200 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f81d000ab00 tx=0x7f81c4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54bd48605ac1aaea server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:24.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.285+0000 7f81d6ffd700 1 -- 172.21.15.92:0/4289309429 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d000a570 con 0x7f81c4004d90 2023-11-28T11:21:24.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.285+0000 7f81d6ffd700 1 -- 172.21.15.92:0/4289309429 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81d000c770 con 0x7f81c4004d90 2023-11-28T11:21:24.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.285+0000 7f81d6ffd700 1 -- 172.21.15.92:0/4289309429 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d000cb20 con 0x7f81c4004d90 2023-11-28T11:21:24.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.286+0000 7f81dddff700 1 -- 172.21.15.92:0/4289309429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81c4004d90 msgr2=0x7f81c4005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:24.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.286+0000 7f81dddff700 1 --2- 172.21.15.92:0/4289309429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81c4004d90 0x7f81c4005200 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f81d000ab00 tx=0x7f81c4005880 comp rx=0 tx=0).stop 2023-11-28T11:21:24.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.286+0000 7f81d7fff700 1 -- 172.21.15.92:0/4289309429 reap_dead start 2023-11-28T11:21:24.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.286+0000 7f81dddff700 1 -- 172.21.15.92:0/4289309429 shutdown_connections 2023-11-28T11:21:24.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.286+0000 7f81dddff700 1 -- 172.21.15.92:0/4289309429 >> 172.21.15.92:0/4289309429 conn(0x7f81d80719a0 msgr2=0x7f81d8071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:24.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.286+0000 7f81dddff700 1 -- 172.21.15.92:0/4289309429 shutdown_connections 2023-11-28T11:21:24.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.286+0000 7f81dddff700 1 -- 172.21.15.92:0/4289309429 wait complete. 2023-11-28T11:21:24.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.288+0000 7f81dddff700 1 Processor -- start 2023-11-28T11:21:24.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.288+0000 7f81dddff700 1 -- start start 2023-11-28T11:21:24.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.289+0000 7f81dddff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81c4004d90 0x7f81d8157250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:24.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.289+0000 7f81dddff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81d81f52c0 0x7f81d814cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:24.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.289+0000 7f81dddff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81d8158450 0x7f81d81588a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:24.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.289+0000 7f81dddff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d8158de0 con 0x7f81d81f52c0 2023-11-28T11:21:24.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.289+0000 7f81dddff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d81f6240 con 0x7f81c4004d90 2023-11-28T11:21:24.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.289+0000 7f81dddff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81d80c9f30 con 0x7f81d8158450 2023-11-28T11:21:24.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.289+0000 7f81cffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81d81f52c0 0x7f81d814cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:24.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.289+0000 7f81cffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81d81f52c0 0x7f81d814cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49100/0 (socket says 172.21.15.92:49100) 2023-11-28T11:21:24.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.289+0000 7f81cffff700 1 -- 172.21.15.92:0/2252487922 learned_addr learned my addr 172.21.15.92:0/2252487922 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:24.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.289+0000 7f81d7fff700 1 --2- 172.21.15.92:0/2252487922 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81d8158450 0x7f81d81588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:24.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.289+0000 7f81d77fe700 1 --2- 172.21.15.92:0/2252487922 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81c4004d90 0x7f81d8157250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:24.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.289+0000 7f81cffff700 1 -- 172.21.15.92:0/2252487922 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81d8158450 msgr2=0x7f81d81588a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:24.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.289+0000 7f81cffff700 1 --2- 172.21.15.92:0/2252487922 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81d8158450 0x7f81d81588a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:24.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.289+0000 7f81cffff700 1 -- 172.21.15.92:0/2252487922 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81c4004d90 msgr2=0x7f81d8157250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:24.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.289+0000 7f81cffff700 1 --2- 172.21.15.92:0/2252487922 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81c4004d90 0x7f81d8157250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:24.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.289+0000 7f81cffff700 1 -- 172.21.15.92:0/2252487922 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d80b88f0 con 0x7f81d81f52c0 2023-11-28T11:21:24.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.289+0000 7f81d77fe700 1 --2- 172.21.15.92:0/2252487922 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81c4004d90 0x7f81d8157250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:21:24.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.289+0000 7f81d7fff700 1 --2- 172.21.15.92:0/2252487922 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81d8158450 0x7f81d81588a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:21:24.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.290+0000 7f81cffff700 1 --2- 172.21.15.92:0/2252487922 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81d81f52c0 0x7f81d814cc20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f81c0011a80 tx=0x7f81c0011e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:24.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.290+0000 7f81d4ff9700 1 -- 172.21.15.92:0/2252487922 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c000f9a0 con 0x7f81d81f52c0 2023-11-28T11:21:24.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.290+0000 7f81d4ff9700 1 -- 172.21.15.92:0/2252487922 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81c000fb00 con 0x7f81d81f52c0 2023-11-28T11:21:24.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.290+0000 7f81dddff700 1 -- 172.21.15.92:0/2252487922 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81d80d15e0 con 0x7f81d81f52c0 2023-11-28T11:21:24.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.290+0000 7f81d4ff9700 1 -- 172.21.15.92:0/2252487922 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c000fe90 con 0x7f81d81f52c0 2023-11-28T11:21:24.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.290+0000 7f81dddff700 1 -- 172.21.15.92:0/2252487922 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81d80b21b0 con 0x7f81d81f52c0 2023-11-28T11:21:24.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.292+0000 7f81dddff700 1 -- 172.21.15.92:0/2252487922 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81d8067170 con 0x7f81d81f52c0 2023-11-28T11:21:24.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.292+0000 7f81d4ff9700 1 -- 172.21.15.92:0/2252487922 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f81c0014440 con 0x7f81d81f52c0 2023-11-28T11:21:24.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.294+0000 7f81d4ff9700 1 --2- 172.21.15.92:0/2252487922 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f81b80682f0 0x7f81b806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:24.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.294+0000 7f81d4ff9700 1 -- 172.21.15.92:0/2252487922 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f81c0086970 con 0x7f81d81f52c0 2023-11-28T11:21:24.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.294+0000 7f81d77fe700 1 --2- 172.21.15.92:0/2252487922 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f81b80682f0 0x7f81b806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:24.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.295+0000 7f81d77fe700 1 --2- 172.21.15.92:0/2252487922 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f81b80682f0 0x7f81b806a7a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f81d80cc3e0 tx=0x7f81c8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:24.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.299+0000 7f81d4ff9700 1 -- 172.21.15.92:0/2252487922 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f81c0052d90 con 0x7f81d81f52c0 2023-11-28T11:21:24.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.534+0000 7f81dddff700 1 -- 172.21.15.92:0/2252487922 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f81d80ceb20 con 0x7f81d81f52c0 2023-11-28T11:21:24.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.534+0000 7f81d4ff9700 1 -- 172.21.15.92:0/2252487922 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f81c00563b0 con 0x7f81d81f52c0 2023-11-28T11:21:24.535 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:21:24.535 INFO:teuthology.orchestra.run.smithi092.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-11-28T11:21:24.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.536+0000 7f81dddff700 1 -- 172.21.15.92:0/2252487922 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f81b80682f0 msgr2=0x7f81b806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:24.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.536+0000 7f81dddff700 1 --2- 172.21.15.92:0/2252487922 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f81b80682f0 0x7f81b806a7a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f81d80cc3e0 tx=0x7f81c8008040 comp rx=0 tx=0).stop 2023-11-28T11:21:24.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.536+0000 7f81dddff700 1 -- 172.21.15.92:0/2252487922 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81d81f52c0 msgr2=0x7f81d814cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:24.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.536+0000 7f81dddff700 1 --2- 172.21.15.92:0/2252487922 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81d81f52c0 0x7f81d814cc20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f81c0011a80 tx=0x7f81c0011e40 comp rx=0 tx=0).stop 2023-11-28T11:21:24.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.537+0000 7f81dddff700 1 -- 172.21.15.92:0/2252487922 shutdown_connections 2023-11-28T11:21:24.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.537+0000 7f81dddff700 1 --2- 172.21.15.92:0/2252487922 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f81d8158450 0x7f81d81588a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:24.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.537+0000 7f81dddff700 1 --2- 172.21.15.92:0/2252487922 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f81c4004d90 0x7f81d8157250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:24.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.537+0000 7f81dddff700 1 --2- 172.21.15.92:0/2252487922 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f81b80682f0 0x7f81b806a7a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:24.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.537+0000 7f81dddff700 1 --2- 172.21.15.92:0/2252487922 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81d81f52c0 0x7f81d814cc20 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:24.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.537+0000 7f81dddff700 1 -- 172.21.15.92:0/2252487922 >> 172.21.15.92:0/2252487922 conn(0x7f81d80719a0 msgr2=0x7f81d8072cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:24.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.537+0000 7f81dddff700 1 -- 172.21.15.92:0/2252487922 shutdown_connections 2023-11-28T11:21:24.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.537+0000 7f81dddff700 1 -- 172.21.15.92:0/2252487922 wait complete. 2023-11-28T11:21:24.555 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-11-28T11:21:24.556 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:21:24.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.752+0000 7f5f8d7ff700 1 Processor -- start 2023-11-28T11:21:24.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.752+0000 7f5f8d7ff700 1 -- start start 2023-11-28T11:21:24.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.753+0000 7f5f8d7ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5f880b8510 0x7f5f8806b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:24.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.753+0000 7f5f8d7ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f880d0d60 0x7f5f880d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:24.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.753+0000 7f5f8d7ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f880b88e0 con 0x7f5f880cd460 2023-11-28T11:21:24.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.753+0000 7f5f8d7ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f880cdbd0 con 0x7f5f880b8510 2023-11-28T11:21:24.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.753+0000 7f5f8d7ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f880d02d0 con 0x7f5f880d0d60 2023-11-28T11:21:24.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.753+0000 7f5f867fc700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f5f880cd460 0x7f5f8806beb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47514/0 (socket says 172.21.15.92:47514) 2023-11-28T11:21:24.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.753+0000 7f5f867fc700 1 -- 172.21.15.92:0/494302542 learned_addr learned my addr 172.21.15.92:0/494302542 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:24.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.754+0000 7f5f86ffd700 1 --2- 172.21.15.92:0/494302542 >> v2:172.21.15.169:3300/0 conn(0x7f5f880b8510 0x7f5f8806b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:24.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.754+0000 7f5f877fe700 1 --2- 172.21.15.92:0/494302542 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f880d0d60 0x7f5f880d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:24.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.754+0000 7f5f85ffb700 1 -- 172.21.15.92:0/494302542 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3133601093 0 0) 0x7f5f880b88e0 con 0x7f5f880cd460 2023-11-28T11:21:24.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.754+0000 7f5f85ffb700 1 -- 172.21.15.92:0/494302542 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f700035b0 con 0x7f5f880cd460 2023-11-28T11:21:24.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.754+0000 7f5f86ffd700 1 -- 172.21.15.92:0/494302542 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f880d0d60 msgr2=0x7f5f880d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:24.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.754+0000 7f5f86ffd700 1 --2- 172.21.15.92:0/494302542 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f880d0d60 0x7f5f880d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:24.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.754+0000 7f5f86ffd700 1 -- 172.21.15.92:0/494302542 >> v1:172.21.15.92:6789/0 conn(0x7f5f880cd460 legacy=0x7f5f8806beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:24.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.754+0000 7f5f86ffd700 1 -- 172.21.15.92:0/494302542 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f880cd8f0 con 0x7f5f880b8510 2023-11-28T11:21:24.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.754+0000 7f5f877fe700 1 --2- 172.21.15.92:0/494302542 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f880d0d60 0x7f5f880d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:21:24.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.754+0000 7f5f86ffd700 1 --2- 172.21.15.92:0/494302542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f880b8510 0x7f5f8806b970 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5f78009870 tx=0x7f5f78009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5eb0aa3fc6315e3a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:24.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.755+0000 7f5f85ffb700 1 -- 172.21.15.92:0/494302542 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f78011830 con 0x7f5f880b8510 2023-11-28T11:21:24.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.755+0000 7f5f85ffb700 1 -- 172.21.15.92:0/494302542 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f78011990 con 0x7f5f880b8510 2023-11-28T11:21:24.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.755+0000 7f5f85ffb700 1 -- 172.21.15.92:0/494302542 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f78011d20 con 0x7f5f880b8510 2023-11-28T11:21:24.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.755+0000 7f5f8d7ff700 1 -- 172.21.15.92:0/494302542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f880b8510 msgr2=0x7f5f8806b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:24.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.755+0000 7f5f8d7ff700 1 --2- 172.21.15.92:0/494302542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f880b8510 0x7f5f8806b970 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5f78009870 tx=0x7f5f78009b80 comp rx=0 tx=0).stop 2023-11-28T11:21:24.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.755+0000 7f5f8d7ff700 1 -- 172.21.15.92:0/494302542 shutdown_connections 2023-11-28T11:21:24.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.755+0000 7f5f8d7ff700 1 --2- 172.21.15.92:0/494302542 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f880d0d60 0x7f5f880d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:24.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.755+0000 7f5f8d7ff700 1 --2- 172.21.15.92:0/494302542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f880b8510 0x7f5f8806b970 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:24.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.755+0000 7f5f8d7ff700 1 -- 172.21.15.92:0/494302542 >> 172.21.15.92:0/494302542 conn(0x7f5f880719a0 msgr2=0x7f5f88071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:24.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.755+0000 7f5f8d7ff700 1 -- 172.21.15.92:0/494302542 shutdown_connections 2023-11-28T11:21:24.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.755+0000 7f5f8d7ff700 1 -- 172.21.15.92:0/494302542 wait complete. 2023-11-28T11:21:24.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.757+0000 7f5f8d7ff700 1 Processor -- start 2023-11-28T11:21:24.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.757+0000 7f5f8d7ff700 1 -- start start 2023-11-28T11:21:24.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.758+0000 7f5f8d7ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f880b8510 0x7f5f881e6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:24.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.758+0000 7f5f8d7ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f880cd460 0x7f5f880cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:24.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.758+0000 7f5f8d7ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f880d0d60 0x7f5f881e47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:24.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.758+0000 7f5f8d7ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f881e6170 con 0x7f5f880cd460 2023-11-28T11:21:24.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.758+0000 7f5f8d7ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f881e54f0 con 0x7f5f880b8510 2023-11-28T11:21:24.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.758+0000 7f5f8d7ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f881e57d0 con 0x7f5f880d0d60 2023-11-28T11:21:24.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.758+0000 7f5f867fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f880cd460 0x7f5f880cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:24.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.759+0000 7f5f867fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f880cd460 0x7f5f880cc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49116/0 (socket says 172.21.15.92:49116) 2023-11-28T11:21:24.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.759+0000 7f5f867fc700 1 -- 172.21.15.92:0/3622514485 learned_addr learned my addr 172.21.15.92:0/3622514485 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:24.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.759+0000 7f5f86ffd700 1 --2- 172.21.15.92:0/3622514485 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f880b8510 0x7f5f881e6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:24.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.759+0000 7f5f877fe700 1 --2- 172.21.15.92:0/3622514485 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f880d0d60 0x7f5f881e47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:24.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.759+0000 7f5f867fc700 1 -- 172.21.15.92:0/3622514485 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f880d0d60 msgr2=0x7f5f881e47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:24.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.759+0000 7f5f867fc700 1 --2- 172.21.15.92:0/3622514485 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f880d0d60 0x7f5f881e47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:24.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.759+0000 7f5f867fc700 1 -- 172.21.15.92:0/3622514485 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f880b8510 msgr2=0x7f5f881e6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:24.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.759+0000 7f5f867fc700 1 --2- 172.21.15.92:0/3622514485 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f880b8510 0x7f5f881e6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:24.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.759+0000 7f5f867fc700 1 -- 172.21.15.92:0/3622514485 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f80009a70 con 0x7f5f880cd460 2023-11-28T11:21:24.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.759+0000 7f5f867fc700 1 --2- 172.21.15.92:0/3622514485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f880cd460 0x7f5f880cc6e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c00c4f0 tx=0x7f5f7c00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:24.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.759+0000 7f5f6f7fe700 1 -- 172.21.15.92:0/3622514485 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c005d70 con 0x7f5f880cd460 2023-11-28T11:21:24.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.759+0000 7f5f6f7fe700 1 -- 172.21.15.92:0/3622514485 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f7c0070d0 con 0x7f5f880cd460 2023-11-28T11:21:24.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.759+0000 7f5f8d7ff700 1 -- 172.21.15.92:0/3622514485 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f780094a0 con 0x7f5f880cd460 2023-11-28T11:21:24.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.759+0000 7f5f6f7fe700 1 -- 172.21.15.92:0/3622514485 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c007440 con 0x7f5f880cd460 2023-11-28T11:21:24.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.759+0000 7f5f8d7ff700 1 -- 172.21.15.92:0/3622514485 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f880cfff0 con 0x7f5f880cd460 2023-11-28T11:21:24.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.761+0000 7f5f8d7ff700 1 -- 172.21.15.92:0/3622514485 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f881e62b0 con 0x7f5f880cd460 2023-11-28T11:21:24.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.761+0000 7f5f6f7fe700 1 -- 172.21.15.92:0/3622514485 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5f7c019030 con 0x7f5f880cd460 2023-11-28T11:21:24.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.762+0000 7f5f6f7fe700 1 --2- 172.21.15.92:0/3622514485 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5f740682f0 0x7f5f7406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:24.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.762+0000 7f5f6f7fe700 1 -- 172.21.15.92:0/3622514485 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5f7c085190 con 0x7f5f880cd460 2023-11-28T11:21:24.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.762+0000 7f5f86ffd700 1 --2- 172.21.15.92:0/3622514485 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5f740682f0 0x7f5f7406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:24.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.763+0000 7f5f86ffd700 1 --2- 172.21.15.92:0/3622514485 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5f740682f0 0x7f5f7406a7a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5f780090b0 tx=0x7f5f78008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:24.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.766+0000 7f5f6f7fe700 1 -- 172.21.15.92:0/3622514485 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5f7c04b5e0 con 0x7f5f880cd460 2023-11-28T11:21:24.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.957+0000 7f5f8d7ff700 1 -- 172.21.15.92:0/3622514485 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5f880cfcf0 con 0x7f5f880cd460 2023-11-28T11:21:24.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.958+0000 7f5f6f7fe700 1 -- 172.21.15.92:0/3622514485 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+19748 (secure 0 0 0) 0x7f5f881e62b0 con 0x7f5f880cd460 2023-11-28T11:21:24.959 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:21:24.959 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":27,"fsid":"68956e0c-c16f-47cd-a220-f98364b0831d","created":"2023-11-28T11:20:28.697573+0000","modified":"2023-11-28T11:21:00.770781+0000","last_up_change":"2023-11-28T11:20:41.257011+0000","last_in_change":"2023-11-28T11:20:32.249767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:20:41.925237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:20:48.682497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:20:54.471398+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:20:55.204585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:20:58.297307+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6abe483f-0e03-4a26-b9cf-62041291747e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":115823},{"type":"v1","addr":"172.21.15.92:6809","nonce":115823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":115823},{"type":"v1","addr":"172.21.15.92:6813","nonce":115823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":115823},{"type":"v1","addr":"172.21.15.92:6821","nonce":115823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":115823},{"type":"v1","addr":"172.21.15.92:6817","nonce":115823}]},"public_addr":"172.21.15.92:6809/115823","cluster_addr":"172.21.15.92:6813/115823","heartbeat_back_addr":"172.21.15.92:6821/115823","heartbeat_front_addr":"172.21.15.92:6817/115823","state":["exists","up"]},{"osd":1,"uuid":"13c1de73-1a8c-4d7e-8a28-9dae52dfe089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":115824},{"type":"v1","addr":"172.21.15.92:6812","nonce":115824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":115824},{"type":"v1","addr":"172.21.15.92:6816","nonce":115824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":115824},{"type":"v1","addr":"172.21.15.92:6823","nonce":115824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":115824},{"type":"v1","addr":"172.21.15.92:6820","nonce":115824}]},"public_addr":"172.21.15.92:6812/115824","cluster_addr":"172.21.15.92:6816/115824","heartbeat_back_addr":"172.21.15.92:6823/115824","heartbeat_front_addr":"172.21.15.92:6820/115824","state":["exists","up"]},{"osd":2,"uuid":"25788a55-112f-4f00-810d-2bc37cf76778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":115821},{"type":"v1","addr":"172.21.15.92:6801","nonce":115821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":115821},{"type":"v1","addr":"172.21.15.92:6803","nonce":115821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":115821},{"type":"v1","addr":"172.21.15.92:6807","nonce":115821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":115821},{"type":"v1","addr":"172.21.15.92:6805","nonce":115821}]},"public_addr":"172.21.15.92:6801/115821","cluster_addr":"172.21.15.92:6803/115821","heartbeat_back_addr":"172.21.15.92:6807/115821","heartbeat_front_addr":"172.21.15.92:6805/115821","state":["exists","up"]},{"osd":3,"uuid":"adf54e24-33a9-41fe-9ce3-98a5ab6c158e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":115825},{"type":"v1","addr":"172.21.15.92:6825","nonce":115825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":115825},{"type":"v1","addr":"172.21.15.92:6827","nonce":115825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":115825},{"type":"v1","addr":"172.21.15.92:6831","nonce":115825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":115825},{"type":"v1","addr":"172.21.15.92:6829","nonce":115825}]},"public_addr":"172.21.15.92:6825/115825","cluster_addr":"172.21.15.92:6827/115825","heartbeat_back_addr":"172.21.15.92:6831/115825","heartbeat_front_addr":"172.21.15.92:6829/115825","state":["exists","up"]},{"osd":4,"uuid":"ccb9b747-f399-44ea-9a18-9db3152227e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":116035},{"type":"v1","addr":"172.21.15.169:6817","nonce":116035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":116035},{"type":"v1","addr":"172.21.15.169:6819","nonce":116035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":116035},{"type":"v1","addr":"172.21.15.169:6826","nonce":116035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":116035},{"type":"v1","addr":"172.21.15.169:6822","nonce":116035}]},"public_addr":"172.21.15.169:6817/116035","cluster_addr":"172.21.15.169:6819/116035","heartbeat_back_addr":"172.21.15.169:6826/116035","heartbeat_front_addr":"172.21.15.169:6822/116035","state":["exists","up"]},{"osd":5,"uuid":"308a1e63-75aa-4d7d-8a7c-cecc24714fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":116033},{"type":"v1","addr":"172.21.15.169:6801","nonce":116033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":116033},{"type":"v1","addr":"172.21.15.169:6803","nonce":116033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":116033},{"type":"v1","addr":"172.21.15.169:6807","nonce":116033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":116033},{"type":"v1","addr":"172.21.15.169:6805","nonce":116033}]},"public_addr":"172.21.15.169:6801/116033","cluster_addr":"172.21.15.169:6803/116033","heartbeat_back_addr":"172.21.15.169:6807/116033","heartbeat_front_addr":"172.21.15.169:6805/116033","state":["exists","up"]},{"osd":6,"uuid":"9c459b7b-b821-4b4b-b91e-6aaae271457c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":116034},{"type":"v1","addr":"172.21.15.169:6809","nonce":116034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":116034},{"type":"v1","addr":"172.21.15.169:6811","nonce":116034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":116034},{"type":"v1","addr":"172.21.15.169:6815","nonce":116034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":116034},{"type":"v1","addr":"172.21.15.169:6813","nonce":116034}]},"public_addr":"172.21.15.169:6809/116034","cluster_addr":"172.21.15.169:6811/116034","heartbeat_back_addr":"172.21.15.169:6815/116034","heartbeat_front_addr":"172.21.15.169:6813/116034","state":["exists","up"]},{"osd":7,"uuid":"1ca9aa4a-755d-4e28-a937-4d8dccab4df3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":116044},{"type":"v1","addr":"172.21.15.169:6823","nonce":116044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6825","nonce":116044},{"type":"v1","addr":"172.21.15.169:6827","nonce":116044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":116044},{"type":"v1","addr":"172.21.15.169:6831","nonce":116044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":116044},{"type":"v1","addr":"172.21.15.169:6829","nonce":116044}]},"public_addr":"172.21.15.169:6823/116044","cluster_addr":"172.21.15.169:6827/116044","heartbeat_back_addr":"172.21.15.169:6831/116044","heartbeat_front_addr":"172.21.15.169:6829/116044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.586781+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.869778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.508594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.860493+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.872267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.746933+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.611520+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.871446+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:21:24.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.960+0000 7f5f8d7ff700 1 -- 172.21.15.92:0/3622514485 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5f740682f0 msgr2=0x7f5f7406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:24.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.960+0000 7f5f8d7ff700 1 --2- 172.21.15.92:0/3622514485 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5f740682f0 0x7f5f7406a7a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5f780090b0 tx=0x7f5f78008030 comp rx=0 tx=0).stop 2023-11-28T11:21:24.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.960+0000 7f5f8d7ff700 1 -- 172.21.15.92:0/3622514485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f880cd460 msgr2=0x7f5f880cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:24.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.960+0000 7f5f8d7ff700 1 --2- 172.21.15.92:0/3622514485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f880cd460 0x7f5f880cc6e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c00c4f0 tx=0x7f5f7c00c800 comp rx=0 tx=0).stop 2023-11-28T11:21:24.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.960+0000 7f5f8d7ff700 1 -- 172.21.15.92:0/3622514485 shutdown_connections 2023-11-28T11:21:24.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.960+0000 7f5f8d7ff700 1 --2- 172.21.15.92:0/3622514485 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f880d0d60 0x7f5f881e47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:24.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.960+0000 7f5f8d7ff700 1 --2- 172.21.15.92:0/3622514485 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f880b8510 0x7f5f881e6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:24.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.960+0000 7f5f8d7ff700 1 --2- 172.21.15.92:0/3622514485 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5f740682f0 0x7f5f7406a7a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:24.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.960+0000 7f5f8d7ff700 1 --2- 172.21.15.92:0/3622514485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f880cd460 0x7f5f880cc6e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:24.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.960+0000 7f5f8d7ff700 1 -- 172.21.15.92:0/3622514485 >> 172.21.15.92:0/3622514485 conn(0x7f5f880719a0 msgr2=0x7f5f880b27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:24.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.960+0000 7f5f8d7ff700 1 -- 172.21.15.92:0/3622514485 shutdown_connections 2023-11-28T11:21:24.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:24.961+0000 7f5f8d7ff700 1 -- 172.21.15.92:0/3622514485 wait complete. 2023-11-28T11:21:24.980 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:20:41.925237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:20:48.682497+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:20:54.471398+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:20:55.204585+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T11:20:58.297307+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T11:21:24.980 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:21:25.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.178+0000 7f79f45ff700 1 Processor -- start 2023-11-28T11:21:25.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.179+0000 7f79f45ff700 1 -- start start 2023-11-28T11:21:25.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.180+0000 7f79f45ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79ec0d2df0 0x7f79ec0cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:25.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.180+0000 7f79f45ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f79ec0b4fc0 0x7f79ec0d31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:25.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.180+0000 7f79f45ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f79ec0b1c60 0x7f79ec0cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:25.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.180+0000 7f79f45ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79ec0d3d30 con 0x7f79ec0b1c60 2023-11-28T11:21:25.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.180+0000 7f79f45ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79ec0d1950 con 0x7f79ec0b4fc0 2023-11-28T11:21:25.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.180+0000 7f79f45ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79ec0b2500 con 0x7f79ec0d2df0 2023-11-28T11:21:25.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.180+0000 7f79f2b9c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f79ec0b1c60 0x7f79ec0cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:25.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.180+0000 7f79f2b9c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f79ec0b1c60 0x7f79ec0cf010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49120/0 (socket says 172.21.15.92:49120) 2023-11-28T11:21:25.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.180+0000 7f79f2b9c700 1 -- 172.21.15.92:0/1947119672 learned_addr learned my addr 172.21.15.92:0/1947119672 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:25.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.180+0000 7f79f1b9a700 1 --2- 172.21.15.92:0/1947119672 >> v2:172.21.15.169:3300/0 conn(0x7f79ec0b4fc0 0x7f79ec0d31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:25.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.180+0000 7f79f239b700 1 --2- 172.21.15.92:0/1947119672 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79ec0d2df0 0x7f79ec0cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:25.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.181+0000 7f79f2b9c700 1 -- 172.21.15.92:0/1947119672 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79ec0d2df0 msgr2=0x7f79ec0cd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:25.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.181+0000 7f79f2b9c700 1 --2- 172.21.15.92:0/1947119672 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79ec0d2df0 0x7f79ec0cd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:25.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.181+0000 7f79f2b9c700 1 -- 172.21.15.92:0/1947119672 >> v2:172.21.15.169:3300/0 conn(0x7f79ec0b4fc0 msgr2=0x7f79ec0d31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:25.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.181+0000 7f79f2b9c700 1 --2- 172.21.15.92:0/1947119672 >> v2:172.21.15.169:3300/0 conn(0x7f79ec0b4fc0 0x7f79ec0d31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:25.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.181+0000 7f79f2b9c700 1 -- 172.21.15.92:0/1947119672 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79ec0d3a50 con 0x7f79ec0b1c60 2023-11-28T11:21:25.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.181+0000 7f79f1b9a700 1 --2- 172.21.15.92:0/1947119672 >> v2:172.21.15.169:3300/0 conn(0x7f79ec0b4fc0 0x7f79ec0d31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:21:25.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.181+0000 7f79f2b9c700 1 --2- 172.21.15.92:0/1947119672 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79ec0b1c60 0x7f79ec0cf010 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f79ec08be30 tx=0x7f79e400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=216e30e50cdff6c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:25.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.181+0000 7f79f1399700 1 -- 172.21.15.92:0/1947119672 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e400e070 con 0x7f79ec0b1c60 2023-11-28T11:21:25.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.181+0000 7f79f1399700 1 -- 172.21.15.92:0/1947119672 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e400b910 con 0x7f79ec0b1c60 2023-11-28T11:21:25.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.181+0000 7f79f1399700 1 -- 172.21.15.92:0/1947119672 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e400bcc0 con 0x7f79ec0b1c60 2023-11-28T11:21:25.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.182+0000 7f79f45ff700 1 -- 172.21.15.92:0/1947119672 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79ec0b1c60 msgr2=0x7f79ec0cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:25.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.182+0000 7f79f45ff700 1 --2- 172.21.15.92:0/1947119672 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79ec0b1c60 0x7f79ec0cf010 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f79ec08be30 tx=0x7f79e400b330 comp rx=0 tx=0).stop 2023-11-28T11:21:25.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.182+0000 7f79f45ff700 1 -- 172.21.15.92:0/1947119672 shutdown_connections 2023-11-28T11:21:25.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.182+0000 7f79f45ff700 1 --2- 172.21.15.92:0/1947119672 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79ec0b1c60 0x7f79ec0cf010 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:25.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.182+0000 7f79f45ff700 1 --2- 172.21.15.92:0/1947119672 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79ec0d2df0 0x7f79ec0cd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:25.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.182+0000 7f79f45ff700 1 --2- 172.21.15.92:0/1947119672 >> v2:172.21.15.169:3300/0 conn(0x7f79ec0b4fc0 0x7f79ec0d31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:25.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.182+0000 7f79f45ff700 1 -- 172.21.15.92:0/1947119672 >> 172.21.15.92:0/1947119672 conn(0x7f79ec0719a0 msgr2=0x7f79ec071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:25.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.182+0000 7f79f45ff700 1 -- 172.21.15.92:0/1947119672 shutdown_connections 2023-11-28T11:21:25.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.182+0000 7f79f45ff700 1 -- 172.21.15.92:0/1947119672 wait complete. 2023-11-28T11:21:25.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.184+0000 7f79f45ff700 1 Processor -- start 2023-11-28T11:21:25.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.184+0000 7f79f45ff700 1 -- start start 2023-11-28T11:21:25.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.186+0000 7f79f45ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79ec0b1c60 0x7f79ec158f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:25.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.186+0000 7f79f45ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79ec0b4fc0 0x7f79ec0cc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:25.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.186+0000 7f79f45ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79ec0d2df0 0x7f79ec158370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:25.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.186+0000 7f79f45ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ec1e5150 con 0x7f79ec0b1c60 2023-11-28T11:21:25.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.186+0000 7f79f45ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ec15dfd0 con 0x7f79ec0d2df0 2023-11-28T11:21:25.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.186+0000 7f79f45ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79ec15e2b0 con 0x7f79ec0b4fc0 2023-11-28T11:21:25.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.186+0000 7f79f239b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79ec0b1c60 0x7f79ec158f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:25.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.186+0000 7f79f239b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79ec0b1c60 0x7f79ec158f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49132/0 (socket says 172.21.15.92:49132) 2023-11-28T11:21:25.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.186+0000 7f79f239b700 1 -- 172.21.15.92:0/1818986199 learned_addr learned my addr 172.21.15.92:0/1818986199 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:25.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.186+0000 7f79f1b9a700 1 --2- 172.21.15.92:0/1818986199 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79ec0b4fc0 0x7f79ec0cc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:25.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.186+0000 7f79f2b9c700 1 --2- 172.21.15.92:0/1818986199 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79ec0d2df0 0x7f79ec158370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:25.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.186+0000 7f79f239b700 1 -- 172.21.15.92:0/1818986199 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79ec0b4fc0 msgr2=0x7f79ec0cc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:25.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.186+0000 7f79f239b700 1 --2- 172.21.15.92:0/1818986199 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79ec0b4fc0 0x7f79ec0cc1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:25.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.186+0000 7f79f239b700 1 -- 172.21.15.92:0/1818986199 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79ec0d2df0 msgr2=0x7f79ec158370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:25.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.186+0000 7f79f239b700 1 --2- 172.21.15.92:0/1818986199 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79ec0d2df0 0x7f79ec158370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:25.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.186+0000 7f79f239b700 1 -- 172.21.15.92:0/1818986199 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79dc0094a0 con 0x7f79ec0b1c60 2023-11-28T11:21:25.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.186+0000 7f79f2b9c700 1 --2- 172.21.15.92:0/1818986199 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79ec0d2df0 0x7f79ec158370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:21:25.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.186+0000 7f79f1b9a700 1 --2- 172.21.15.92:0/1818986199 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79ec0b4fc0 0x7f79ec0cc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:21:25.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.187+0000 7f79f239b700 1 --2- 172.21.15.92:0/1818986199 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79ec0b1c60 0x7f79ec158f10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f79e800e870 tx=0x7f79e800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:25.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.187+0000 7f79e2ffd700 1 -- 172.21.15.92:0/1818986199 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e8009590 con 0x7f79ec0b1c60 2023-11-28T11:21:25.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.187+0000 7f79e2ffd700 1 -- 172.21.15.92:0/1818986199 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e800ee70 con 0x7f79ec0b1c60 2023-11-28T11:21:25.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.187+0000 7f79e2ffd700 1 -- 172.21.15.92:0/1818986199 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e800ca50 con 0x7f79ec0b1c60 2023-11-28T11:21:25.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.187+0000 7f79f45ff700 1 -- 172.21.15.92:0/1818986199 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79e4009b30 con 0x7f79ec0b1c60 2023-11-28T11:21:25.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.187+0000 7f79f45ff700 1 -- 172.21.15.92:0/1818986199 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79ec0b88f0 con 0x7f79ec0b1c60 2023-11-28T11:21:25.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.188+0000 7f79f45ff700 1 -- 172.21.15.92:0/1818986199 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79ec0b21b0 con 0x7f79ec0b1c60 2023-11-28T11:21:25.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.189+0000 7f79e2ffd700 1 -- 172.21.15.92:0/1818986199 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f79e8019030 con 0x7f79ec0b1c60 2023-11-28T11:21:25.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.190+0000 7f79e2ffd700 1 --2- 172.21.15.92:0/1818986199 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f79d00682f0 0x7f79d006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:25.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.190+0000 7f79e2ffd700 1 -- 172.21.15.92:0/1818986199 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f79e8085e40 con 0x7f79ec0b1c60 2023-11-28T11:21:25.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.190+0000 7f79f1b9a700 1 --2- 172.21.15.92:0/1818986199 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f79d00682f0 0x7f79d006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:25.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.191+0000 7f79f1b9a700 1 --2- 172.21.15.92:0/1818986199 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f79d00682f0 0x7f79d006a7a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f79ec0cceb0 tx=0x7f79dc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:25.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.194+0000 7f79e2ffd700 1 -- 172.21.15.92:0/1818986199 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f79e80522c0 con 0x7f79ec0b1c60 2023-11-28T11:21:25.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.385+0000 7f79f45ff700 1 -- 172.21.15.92:0/1818986199 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f79ec15e7a0 con 0x7f79ec0b1c60 2023-11-28T11:21:25.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.386+0000 7f79e2ffd700 1 -- 172.21.15.92:0/1818986199 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f79e80558e0 con 0x7f79ec0b1c60 2023-11-28T11:21:25.387 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-11-28T11:21:25.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.388+0000 7f79f45ff700 1 -- 172.21.15.92:0/1818986199 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f79d00682f0 msgr2=0x7f79d006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:25.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.388+0000 7f79f45ff700 1 --2- 172.21.15.92:0/1818986199 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f79d00682f0 0x7f79d006a7a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f79ec0cceb0 tx=0x7f79dc008040 comp rx=0 tx=0).stop 2023-11-28T11:21:25.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.388+0000 7f79f45ff700 1 -- 172.21.15.92:0/1818986199 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79ec0b1c60 msgr2=0x7f79ec158f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:25.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.388+0000 7f79f45ff700 1 --2- 172.21.15.92:0/1818986199 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79ec0b1c60 0x7f79ec158f10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f79e800e870 tx=0x7f79e800ec30 comp rx=0 tx=0).stop 2023-11-28T11:21:25.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.388+0000 7f79f45ff700 1 -- 172.21.15.92:0/1818986199 shutdown_connections 2023-11-28T11:21:25.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.388+0000 7f79f45ff700 1 --2- 172.21.15.92:0/1818986199 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f79ec0d2df0 0x7f79ec158370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:25.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.388+0000 7f79f45ff700 1 --2- 172.21.15.92:0/1818986199 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f79d00682f0 0x7f79d006a7a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:25.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.388+0000 7f79f45ff700 1 --2- 172.21.15.92:0/1818986199 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79ec0b1c60 0x7f79ec158f10 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:25.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.388+0000 7f79f45ff700 1 --2- 172.21.15.92:0/1818986199 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f79ec0b4fc0 0x7f79ec0cc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:25.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.388+0000 7f79f45ff700 1 -- 172.21.15.92:0/1818986199 >> 172.21.15.92:0/1818986199 conn(0x7f79ec0719a0 msgr2=0x7f79ec0b7830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:25.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.388+0000 7f79f45ff700 1 -- 172.21.15.92:0/1818986199 shutdown_connections 2023-11-28T11:21:25.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.388+0000 7f79f45ff700 1 -- 172.21.15.92:0/1818986199 wait complete. 2023-11-28T11:21:25.407 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:21:25.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.606+0000 7f77acbff700 1 Processor -- start 2023-11-28T11:21:25.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.606+0000 7f77acbff700 1 -- start start 2023-11-28T11:21:25.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.607+0000 7f77acbff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77a80d2df0 0x7f77a80cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:25.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.607+0000 7f77acbff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f77a80b4fc0 0x7f77a80d31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:25.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.607+0000 7f77acbff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77a80d1950 con 0x7f77a80b1c60 2023-11-28T11:21:25.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.607+0000 7f77acbff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77a80d3d30 con 0x7f77a80b4fc0 2023-11-28T11:21:25.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.607+0000 7f77acbff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77a80b2500 con 0x7f77a80d2df0 2023-11-28T11:21:25.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.607+0000 7f77a5d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f77a80b4fc0 0x7f77a80d31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:25.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.607+0000 7f77a6d9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f77a80b1c60 0x7f77a80cf010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47518/0 (socket says 172.21.15.92:47518) 2023-11-28T11:21:25.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.607+0000 7f77a6d9d700 1 -- 172.21.15.92:0/3812620322 learned_addr learned my addr 172.21.15.92:0/3812620322 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:25.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.608+0000 7f77a659c700 1 --2- 172.21.15.92:0/3812620322 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77a80d2df0 0x7f77a80cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:25.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.608+0000 7f77a5d9b700 1 -- 172.21.15.92:0/3812620322 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77a80d2df0 msgr2=0x7f77a80cd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:25.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.608+0000 7f77a5d9b700 1 --2- 172.21.15.92:0/3812620322 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77a80d2df0 0x7f77a80cd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:25.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.608+0000 7f77a5d9b700 1 -- 172.21.15.92:0/3812620322 >> v1:172.21.15.92:6789/0 conn(0x7f77a80b1c60 legacy=0x7f77a80cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:25.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.608+0000 7f77a5d9b700 1 -- 172.21.15.92:0/3812620322 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77a80d3a50 con 0x7f77a80b4fc0 2023-11-28T11:21:25.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.608+0000 7f77a5d9b700 1 --2- 172.21.15.92:0/3812620322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77a80b4fc0 0x7f77a80d31c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f779c009840 tx=0x7f779c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c10c0feaf9a4efca server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:25.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.608+0000 7f77a559a700 1 -- 172.21.15.92:0/3812620322 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779c011830 con 0x7f77a80b4fc0 2023-11-28T11:21:25.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.608+0000 7f77a559a700 1 -- 172.21.15.92:0/3812620322 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f779c011990 con 0x7f77a80b4fc0 2023-11-28T11:21:25.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.608+0000 7f77a559a700 1 -- 172.21.15.92:0/3812620322 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779c011d20 con 0x7f77a80b4fc0 2023-11-28T11:21:25.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.609+0000 7f77acbff700 1 -- 172.21.15.92:0/3812620322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77a80b4fc0 msgr2=0x7f77a80d31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:25.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.609+0000 7f77acbff700 1 --2- 172.21.15.92:0/3812620322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77a80b4fc0 0x7f77a80d31c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f779c009840 tx=0x7f779c009b50 comp rx=0 tx=0).stop 2023-11-28T11:21:25.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.609+0000 7f77acbff700 1 -- 172.21.15.92:0/3812620322 shutdown_connections 2023-11-28T11:21:25.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.609+0000 7f77acbff700 1 --2- 172.21.15.92:0/3812620322 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77a80d2df0 0x7f77a80cd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:25.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.609+0000 7f77acbff700 1 --2- 172.21.15.92:0/3812620322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77a80b4fc0 0x7f77a80d31c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:25.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.609+0000 7f77acbff700 1 -- 172.21.15.92:0/3812620322 >> 172.21.15.92:0/3812620322 conn(0x7f77a80719a0 msgr2=0x7f77a8071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:25.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.609+0000 7f77acbff700 1 -- 172.21.15.92:0/3812620322 shutdown_connections 2023-11-28T11:21:25.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.609+0000 7f77acbff700 1 -- 172.21.15.92:0/3812620322 wait complete. 2023-11-28T11:21:25.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.611+0000 7f77acbff700 1 Processor -- start 2023-11-28T11:21:25.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.611+0000 7f77acbff700 1 -- start start 2023-11-28T11:21:25.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.613+0000 7f77acbff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77a80b1c60 0x7f77a815a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:25.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.613+0000 7f77acbff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77a80b4fc0 0x7f77a80cc440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:25.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.613+0000 7f77acbff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77a80d2df0 0x7f77a80cc980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:25.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.613+0000 7f77acbff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a81e6750 con 0x7f77a80b1c60 2023-11-28T11:21:25.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.613+0000 7f77acbff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a815b970 con 0x7f77a80d2df0 2023-11-28T11:21:25.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.613+0000 7f77acbff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77a81e5db0 con 0x7f77a80b4fc0 2023-11-28T11:21:25.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.613+0000 7f77a659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77a80b1c60 0x7f77a815a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:25.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.613+0000 7f77a659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77a80b1c60 0x7f77a815a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49144/0 (socket says 172.21.15.92:49144) 2023-11-28T11:21:25.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.613+0000 7f77a659c700 1 -- 172.21.15.92:0/2709966535 learned_addr learned my addr 172.21.15.92:0/2709966535 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:25.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.613+0000 7f77a5d9b700 1 --2- 172.21.15.92:0/2709966535 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77a80b4fc0 0x7f77a80cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:25.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.613+0000 7f77a659c700 1 -- 172.21.15.92:0/2709966535 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77a80b4fc0 msgr2=0x7f77a80cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:25.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.613+0000 7f77a659c700 1 --2- 172.21.15.92:0/2709966535 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77a80b4fc0 0x7f77a80cc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:25.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.613+0000 7f77a659c700 1 -- 172.21.15.92:0/2709966535 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77a80d2df0 msgr2=0x7f77a80cc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:25.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.613+0000 7f77a659c700 1 --2- 172.21.15.92:0/2709966535 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77a80d2df0 0x7f77a80cc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:25.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.613+0000 7f77a659c700 1 -- 172.21.15.92:0/2709966535 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f779c0094a0 con 0x7f77a80b1c60 2023-11-28T11:21:25.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.613+0000 7f77a659c700 1 --2- 172.21.15.92:0/2709966535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77a80b1c60 0x7f77a815a830 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f779800d830 tx=0x7f779800dbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:25.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.614+0000 7f7796ffd700 1 -- 172.21.15.92:0/2709966535 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77980125f0 con 0x7f77a80b1c60 2023-11-28T11:21:25.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.614+0000 7f7796ffd700 1 -- 172.21.15.92:0/2709966535 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7798012750 con 0x7f77a80b1c60 2023-11-28T11:21:25.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.614+0000 7f77acbff700 1 -- 172.21.15.92:0/2709966535 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77a80b88f0 con 0x7f77a80b1c60 2023-11-28T11:21:25.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.614+0000 7f7796ffd700 1 -- 172.21.15.92:0/2709966535 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7798012b00 con 0x7f77a80b1c60 2023-11-28T11:21:25.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.614+0000 7f77acbff700 1 -- 172.21.15.92:0/2709966535 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77a80d15e0 con 0x7f77a80b1c60 2023-11-28T11:21:25.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.615+0000 7f77acbff700 1 -- 172.21.15.92:0/2709966535 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77a80b21b0 con 0x7f77a80b1c60 2023-11-28T11:21:25.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.616+0000 7f7796ffd700 1 -- 172.21.15.92:0/2709966535 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7798012c60 con 0x7f77a80b1c60 2023-11-28T11:21:25.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.617+0000 7f7796ffd700 1 --2- 172.21.15.92:0/2709966535 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f77880682f0 0x7f778806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:25.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.617+0000 7f7796ffd700 1 -- 172.21.15.92:0/2709966535 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7798086560 con 0x7f77a80b1c60 2023-11-28T11:21:25.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.617+0000 7f77a5d9b700 1 --2- 172.21.15.92:0/2709966535 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f77880682f0 0x7f778806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:25.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.618+0000 7f77a5d9b700 1 --2- 172.21.15.92:0/2709966535 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f77880682f0 0x7f778806a7a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f779c009840 tx=0x7f779c01c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:25.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.621+0000 7f7796ffd700 1 -- 172.21.15.92:0/2709966535 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f77980529e0 con 0x7f77a80b1c60 2023-11-28T11:21:25.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.812+0000 7f77acbff700 1 -- 172.21.15.92:0/2709966535 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f77a8067170 con 0x7f77a80b1c60 2023-11-28T11:21:25.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.812+0000 7f7796ffd700 1 -- 172.21.15.92:0/2709966535 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v27) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7798056000 con 0x7f77a80b1c60 2023-11-28T11:21:25.813 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-11-28T11:21:25.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.814+0000 7f77acbff700 1 -- 172.21.15.92:0/2709966535 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f77880682f0 msgr2=0x7f778806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:25.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.814+0000 7f77acbff700 1 --2- 172.21.15.92:0/2709966535 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f77880682f0 0x7f778806a7a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f779c009840 tx=0x7f779c01c040 comp rx=0 tx=0).stop 2023-11-28T11:21:25.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.814+0000 7f77acbff700 1 -- 172.21.15.92:0/2709966535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77a80b1c60 msgr2=0x7f77a815a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:25.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.814+0000 7f77acbff700 1 --2- 172.21.15.92:0/2709966535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77a80b1c60 0x7f77a815a830 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f779800d830 tx=0x7f779800dbf0 comp rx=0 tx=0).stop 2023-11-28T11:21:25.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.814+0000 7f77acbff700 1 -- 172.21.15.92:0/2709966535 shutdown_connections 2023-11-28T11:21:25.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.814+0000 7f77acbff700 1 --2- 172.21.15.92:0/2709966535 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f77a80d2df0 0x7f77a80cc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:25.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.814+0000 7f77acbff700 1 --2- 172.21.15.92:0/2709966535 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f77880682f0 0x7f778806a7a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:25.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.814+0000 7f77acbff700 1 --2- 172.21.15.92:0/2709966535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77a80b1c60 0x7f77a815a830 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:25.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.814+0000 7f77acbff700 1 --2- 172.21.15.92:0/2709966535 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f77a80b4fc0 0x7f77a80cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:25.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.814+0000 7f77acbff700 1 -- 172.21.15.92:0/2709966535 >> 172.21.15.92:0/2709966535 conn(0x7f77a80719a0 msgr2=0x7f77a80b7410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:25.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.814+0000 7f77acbff700 1 -- 172.21.15.92:0/2709966535 shutdown_connections 2023-11-28T11:21:25.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:25.814+0000 7f77acbff700 1 -- 172.21.15.92:0/2709966535 wait complete. 2023-11-28T11:21:25.834 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:21:26.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.033+0000 7fe1ed5ff700 1 Processor -- start 2023-11-28T11:21:26.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.033+0000 7fe1ed5ff700 1 -- start start 2023-11-28T11:21:26.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.034+0000 7fe1ed5ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1e80b4fc0 0x7fe1e80cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:26.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.034+0000 7fe1ed5ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1e80d1950 con 0x7fe1e80d2df0 2023-11-28T11:21:26.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.034+0000 7fe1ed5ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1e80b2500 con 0x7fe1e80b1c60 2023-11-28T11:21:26.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.034+0000 7fe1ed5ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e80d3d30 con 0x7fe1e80b4fc0 2023-11-28T11:21:26.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.034+0000 7fe1e6ffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe1e80d2df0 0x7fe1e80d31c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47522/0 (socket says 172.21.15.92:47522) 2023-11-28T11:21:26.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.034+0000 7fe1e6ffd700 1 -- 172.21.15.92:0/1083672114 learned_addr learned my addr 172.21.15.92:0/1083672114 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:26.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.034+0000 7fe1e67fc700 1 -- 172.21.15.92:0/1083672114 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1994602601 0 0) 0x7fe1e80d1950 con 0x7fe1e80d2df0 2023-11-28T11:21:26.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.035+0000 7fe1e67fc700 1 -- 172.21.15.92:0/1083672114 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1d40035b0 con 0x7fe1e80d2df0 2023-11-28T11:21:26.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.035+0000 7fe1dffff700 1 --2- 172.21.15.92:0/1083672114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1e80b4fc0 0x7fe1e80cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:26.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.035+0000 7fe1e67fc700 1 -- 172.21.15.92:0/1083672114 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3707332441 0 0) 0x7fe1d40035b0 con 0x7fe1e80d2df0 2023-11-28T11:21:26.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.035+0000 7fe1e67fc700 1 -- 172.21.15.92:0/1083672114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1e80b4fc0 msgr2=0x7fe1e80cd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:26.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.035+0000 7fe1e67fc700 1 --2- 172.21.15.92:0/1083672114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1e80b4fc0 0x7fe1e80cd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:26.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.035+0000 7fe1e67fc700 1 -- 172.21.15.92:0/1083672114 >> v1:172.21.15.169:6789/0 conn(0x7fe1e80b1c60 legacy=0x7fe1e80cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:26.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.035+0000 7fe1e67fc700 1 -- 172.21.15.92:0/1083672114 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e80d3a50 con 0x7fe1e80d2df0 2023-11-28T11:21:26.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.035+0000 7fe1e67fc700 1 -- 172.21.15.92:0/1083672114 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532161587 0 0) 0x7fe1d8001670 con 0x7fe1e80d2df0 2023-11-28T11:21:26.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.035+0000 7fe1e67fc700 1 -- 172.21.15.92:0/1083672114 >> v1:172.21.15.92:6789/0 conn(0x7fe1e80d2df0 legacy=0x7fe1e80d31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:26.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.035+0000 7fe1e67fc700 1 --2- 172.21.15.92:0/1083672114 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1d4003a30 0x7fe1d4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:26.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.035+0000 7fe1e67fc700 1 --2- 172.21.15.92:0/1083672114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1e80b4fc0 0x7fe1d4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:26.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.035+0000 7fe1e67fc700 1 --2- 172.21.15.92:0/1083672114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1d4004d90 0x7fe1d4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:26.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.035+0000 7fe1e67fc700 1 -- 172.21.15.92:0/1083672114 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e80d3d30 con 0x7fe1d4003a30 2023-11-28T11:21:26.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.035+0000 7fe1e67fc700 1 -- 172.21.15.92:0/1083672114 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d400ba10 con 0x7fe1d4004d90 2023-11-28T11:21:26.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.035+0000 7fe1e67fc700 1 -- 172.21.15.92:0/1083672114 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d400bba0 con 0x7fe1e80b4fc0 2023-11-28T11:21:26.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.035+0000 7fe1e6ffd700 1 --2- 172.21.15.92:0/1083672114 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1d4003a30 0x7fe1d4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:26.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.035+0000 7fe1dffff700 1 --2- 172.21.15.92:0/1083672114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1e80b4fc0 0x7fe1d4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:26.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.035+0000 7fe1e77fe700 1 --2- 172.21.15.92:0/1083672114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1d4004d90 0x7fe1d4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:26.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.036+0000 7fe1e6ffd700 1 -- 172.21.15.92:0/1083672114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1e80b4fc0 msgr2=0x7fe1d4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:26.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.036+0000 7fe1e6ffd700 1 --2- 172.21.15.92:0/1083672114 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1e80b4fc0 0x7fe1d4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:26.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.036+0000 7fe1e6ffd700 1 -- 172.21.15.92:0/1083672114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1d4004d90 msgr2=0x7fe1d4005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:26.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.036+0000 7fe1e6ffd700 1 --2- 172.21.15.92:0/1083672114 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1d4004d90 0x7fe1d4005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:26.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.036+0000 7fe1e6ffd700 1 -- 172.21.15.92:0/1083672114 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d40035b0 con 0x7fe1d4003a30 2023-11-28T11:21:26.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.036+0000 7fe1e6ffd700 1 --2- 172.21.15.92:0/1083672114 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1d4003a30 0x7fe1d4003e00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe1d800c240 tx=0x7fe1d4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eef72fd390f0d4e4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:26.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.036+0000 7fe1e67fc700 1 -- 172.21.15.92:0/1083672114 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d800cd80 con 0x7fe1d4003a30 2023-11-28T11:21:26.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.036+0000 7fe1e67fc700 1 -- 172.21.15.92:0/1083672114 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1d8005960 con 0x7fe1d4003a30 2023-11-28T11:21:26.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.036+0000 7fe1e67fc700 1 -- 172.21.15.92:0/1083672114 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d8005d10 con 0x7fe1d4003a30 2023-11-28T11:21:26.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.037+0000 7fe1ed5ff700 1 -- 172.21.15.92:0/1083672114 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1d4003a30 msgr2=0x7fe1d4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:26.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.037+0000 7fe1ed5ff700 1 --2- 172.21.15.92:0/1083672114 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1d4003a30 0x7fe1d4003e00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe1d800c240 tx=0x7fe1d4005880 comp rx=0 tx=0).stop 2023-11-28T11:21:26.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.037+0000 7fe1e77fe700 1 -- 172.21.15.92:0/1083672114 reap_dead start 2023-11-28T11:21:26.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.037+0000 7fe1ed5ff700 1 -- 172.21.15.92:0/1083672114 shutdown_connections 2023-11-28T11:21:26.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.037+0000 7fe1ed5ff700 1 -- 172.21.15.92:0/1083672114 >> 172.21.15.92:0/1083672114 conn(0x7fe1e80719a0 msgr2=0x7fe1e8071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:26.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.037+0000 7fe1ed5ff700 1 -- 172.21.15.92:0/1083672114 shutdown_connections 2023-11-28T11:21:26.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.037+0000 7fe1ed5ff700 1 -- 172.21.15.92:0/1083672114 wait complete. 2023-11-28T11:21:26.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.039+0000 7fe1ed5ff700 1 Processor -- start 2023-11-28T11:21:26.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.039+0000 7fe1ed5ff700 1 -- start start 2023-11-28T11:21:26.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.040+0000 7fe1ed5ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1d4003a30 0x7fe1e80cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:26.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.040+0000 7fe1ed5ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1e81f3330 0x7fe1e80c9f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:26.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.040+0000 7fe1ed5ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1e81f37a0 0x7fe1e80ca470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:26.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.040+0000 7fe1ed5ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e81f3110 con 0x7fe1e81f3330 2023-11-28T11:21:26.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.040+0000 7fe1ed5ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e81e5020 con 0x7fe1d4003a30 2023-11-28T11:21:26.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.040+0000 7fe1ed5ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e81e5300 con 0x7fe1e81f37a0 2023-11-28T11:21:26.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.040+0000 7fe1dffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1e81f3330 0x7fe1e80c9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:26.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.040+0000 7fe1dffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1e81f3330 0x7fe1e80c9f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49166/0 (socket says 172.21.15.92:49166) 2023-11-28T11:21:26.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.040+0000 7fe1dffff700 1 -- 172.21.15.92:0/1096099759 learned_addr learned my addr 172.21.15.92:0/1096099759 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:26.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.040+0000 7fe1e6ffd700 1 --2- 172.21.15.92:0/1096099759 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1d4003a30 0x7fe1e80cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:26.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.040+0000 7fe1e77fe700 1 --2- 172.21.15.92:0/1096099759 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1e81f37a0 0x7fe1e80ca470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:26.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.040+0000 7fe1dffff700 1 -- 172.21.15.92:0/1096099759 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1e81f37a0 msgr2=0x7fe1e80ca470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:26.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.040+0000 7fe1dffff700 1 --2- 172.21.15.92:0/1096099759 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1e81f37a0 0x7fe1e80ca470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:26.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.040+0000 7fe1dffff700 1 -- 172.21.15.92:0/1096099759 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1d4003a30 msgr2=0x7fe1e80cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:26.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.040+0000 7fe1dffff700 1 --2- 172.21.15.92:0/1096099759 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1d4003a30 0x7fe1e80cccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:26.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.040+0000 7fe1dffff700 1 -- 172.21.15.92:0/1096099759 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e80b88f0 con 0x7fe1e81f3330 2023-11-28T11:21:26.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.040+0000 7fe1e6ffd700 1 --2- 172.21.15.92:0/1096099759 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1d4003a30 0x7fe1e80cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:21:26.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.041+0000 7fe1dffff700 1 --2- 172.21.15.92:0/1096099759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1e81f3330 0x7fe1e80c9f30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe1d00119c0 tx=0x7fe1d0011d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:26.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.041+0000 7fe1df7fe700 1 -- 172.21.15.92:0/1096099759 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d000f940 con 0x7fe1e81f3330 2023-11-28T11:21:26.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.041+0000 7fe1df7fe700 1 -- 172.21.15.92:0/1096099759 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1d000faa0 con 0x7fe1e81f3330 2023-11-28T11:21:26.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.041+0000 7fe1ed5ff700 1 -- 172.21.15.92:0/1096099759 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1e80d15e0 con 0x7fe1e81f3330 2023-11-28T11:21:26.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.041+0000 7fe1df7fe700 1 -- 172.21.15.92:0/1096099759 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d000fe50 con 0x7fe1e81f3330 2023-11-28T11:21:26.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.041+0000 7fe1ed5ff700 1 -- 172.21.15.92:0/1096099759 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1e80b21b0 con 0x7fe1e81f3330 2023-11-28T11:21:26.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.042+0000 7fe1ed5ff700 1 -- 172.21.15.92:0/1096099759 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1e8067170 con 0x7fe1e81f3330 2023-11-28T11:21:26.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.043+0000 7fe1df7fe700 1 -- 172.21.15.92:0/1096099759 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe1d0014440 con 0x7fe1e81f3330 2023-11-28T11:21:26.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.044+0000 7fe1df7fe700 1 --2- 172.21.15.92:0/1096099759 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe1c80682f0 0x7fe1c806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:26.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.044+0000 7fe1df7fe700 1 -- 172.21.15.92:0/1096099759 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe1d00868d0 con 0x7fe1e81f3330 2023-11-28T11:21:26.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.044+0000 7fe1e6ffd700 1 --2- 172.21.15.92:0/1096099759 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe1c80682f0 0x7fe1c806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:26.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.045+0000 7fe1e6ffd700 1 --2- 172.21.15.92:0/1096099759 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe1c80682f0 0x7fe1c806a7a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe1e80b73d0 tx=0x7fe1d800a000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:26.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.048+0000 7fe1df7fe700 1 -- 172.21.15.92:0/1096099759 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe1d0052cf0 con 0x7fe1e81f3330 2023-11-28T11:21:26.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.238+0000 7fe1ed5ff700 1 -- 172.21.15.92:0/1096099759 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe1e80b3c30 con 0x7fe1e81f3330 2023-11-28T11:21:26.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.239+0000 7fe1df7fe700 1 -- 172.21.15.92:0/1096099759 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe1d0056310 con 0x7fe1e81f3330 2023-11-28T11:21:26.240 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-11-28T11:21:26.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.241+0000 7fe1ed5ff700 1 -- 172.21.15.92:0/1096099759 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe1c80682f0 msgr2=0x7fe1c806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:26.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.241+0000 7fe1ed5ff700 1 --2- 172.21.15.92:0/1096099759 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe1c80682f0 0x7fe1c806a7a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe1e80b73d0 tx=0x7fe1d800a000 comp rx=0 tx=0).stop 2023-11-28T11:21:26.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.241+0000 7fe1ed5ff700 1 -- 172.21.15.92:0/1096099759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1e81f3330 msgr2=0x7fe1e80c9f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:26.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.241+0000 7fe1ed5ff700 1 --2- 172.21.15.92:0/1096099759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1e81f3330 0x7fe1e80c9f30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe1d00119c0 tx=0x7fe1d0011d80 comp rx=0 tx=0).stop 2023-11-28T11:21:26.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.241+0000 7fe1ed5ff700 1 -- 172.21.15.92:0/1096099759 shutdown_connections 2023-11-28T11:21:26.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.241+0000 7fe1ed5ff700 1 --2- 172.21.15.92:0/1096099759 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe1e81f37a0 0x7fe1e80ca470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:26.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.241+0000 7fe1ed5ff700 1 --2- 172.21.15.92:0/1096099759 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe1d4003a30 0x7fe1e80cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:26.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.241+0000 7fe1ed5ff700 1 --2- 172.21.15.92:0/1096099759 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe1c80682f0 0x7fe1c806a7a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:26.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.241+0000 7fe1ed5ff700 1 --2- 172.21.15.92:0/1096099759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1e81f3330 0x7fe1e80c9f30 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:26.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.241+0000 7fe1ed5ff700 1 -- 172.21.15.92:0/1096099759 >> 172.21.15.92:0/1096099759 conn(0x7fe1e80719a0 msgr2=0x7fe1e80b2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:26.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.241+0000 7fe1ed5ff700 1 -- 172.21.15.92:0/1096099759 shutdown_connections 2023-11-28T11:21:26.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.241+0000 7fe1ed5ff700 1 -- 172.21.15.92:0/1096099759 wait complete. 2023-11-28T11:21:26.260 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:21:26.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.458+0000 7f3f553ff700 1 Processor -- start 2023-11-28T11:21:26.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.458+0000 7f3f553ff700 1 -- start start 2023-11-28T11:21:26.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.459+0000 7f3f553ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3f500b8510 0x7f3f5006b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:26.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.459+0000 7f3f553ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f500b88e0 con 0x7f3f500cd460 2023-11-28T11:21:26.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.459+0000 7f3f553ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f500cd8f0 con 0x7f3f500d0d60 2023-11-28T11:21:26.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.459+0000 7f3f553ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f500cdbd0 con 0x7f3f500b8510 2023-11-28T11:21:26.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.459+0000 7f3f47fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f3f500cd460 0x7f3f5006beb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47532/0 (socket says 172.21.15.92:47532) 2023-11-28T11:21:26.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.459+0000 7f3f47fff700 1 -- 172.21.15.92:0/78778304 learned_addr learned my addr 172.21.15.92:0/78778304 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:26.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.459+0000 7f3f4effd700 1 --2- 172.21.15.92:0/78778304 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3f500b8510 0x7f3f5006b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:26.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.459+0000 7f3f4e7fc700 1 -- 172.21.15.92:0/78778304 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1947976520 0 0) 0x7f3f500b88e0 con 0x7f3f500cd460 2023-11-28T11:21:26.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.460+0000 7f3f4e7fc700 1 -- 172.21.15.92:0/78778304 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f3c0035b0 con 0x7f3f500cd460 2023-11-28T11:21:26.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.460+0000 7f3f4e7fc700 1 -- 172.21.15.92:0/78778304 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1592963423 0 0) 0x7f3f500cd8f0 con 0x7f3f500d0d60 2023-11-28T11:21:26.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.460+0000 7f3f4e7fc700 1 -- 172.21.15.92:0/78778304 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f500b88e0 con 0x7f3f500d0d60 2023-11-28T11:21:26.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.460+0000 7f3f4effd700 1 -- 172.21.15.92:0/78778304 >> v1:172.21.15.169:6789/0 conn(0x7f3f500d0d60 legacy=0x7f3f500d11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:26.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.460+0000 7f3f4effd700 1 -- 172.21.15.92:0/78778304 >> v1:172.21.15.92:6789/0 conn(0x7f3f500cd460 legacy=0x7f3f5006beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:26.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.460+0000 7f3f4e7fc700 1 -- 172.21.15.92:0/78778304 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1830992326 0 0) 0x7f3f3c0035b0 con 0x7f3f500cd460 2023-11-28T11:21:26.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.460+0000 7f3f4effd700 1 -- 172.21.15.92:0/78778304 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f500cfd10 con 0x7f3f500b8510 2023-11-28T11:21:26.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.460+0000 7f3f4effd700 1 --2- 172.21.15.92:0/78778304 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3f500b8510 0x7f3f5006b970 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3f40009870 tx=0x7f3f40009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf5ae6ff547e55d4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:26.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.461+0000 7f3f4e7fc700 1 -- 172.21.15.92:0/78778304 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f38001670 con 0x7f3f500b8510 2023-11-28T11:21:26.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.461+0000 7f3f4e7fc700 1 -- 172.21.15.92:0/78778304 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f40004030 con 0x7f3f500b8510 2023-11-28T11:21:26.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.461+0000 7f3f4e7fc700 1 -- 172.21.15.92:0/78778304 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f400043a0 con 0x7f3f500b8510 2023-11-28T11:21:26.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.461+0000 7f3f553ff700 1 -- 172.21.15.92:0/78778304 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3f500b8510 msgr2=0x7f3f5006b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:26.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.461+0000 7f3f553ff700 1 --2- 172.21.15.92:0/78778304 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3f500b8510 0x7f3f5006b970 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3f40009870 tx=0x7f3f40009b80 comp rx=0 tx=0).stop 2023-11-28T11:21:26.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.461+0000 7f3f553ff700 1 -- 172.21.15.92:0/78778304 shutdown_connections 2023-11-28T11:21:26.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.461+0000 7f3f553ff700 1 --2- 172.21.15.92:0/78778304 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3f500b8510 0x7f3f5006b970 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:26.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.461+0000 7f3f553ff700 1 -- 172.21.15.92:0/78778304 >> 172.21.15.92:0/78778304 conn(0x7f3f500719a0 msgr2=0x7f3f50071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:26.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.461+0000 7f3f553ff700 1 -- 172.21.15.92:0/78778304 shutdown_connections 2023-11-28T11:21:26.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.461+0000 7f3f553ff700 1 -- 172.21.15.92:0/78778304 wait complete. 2023-11-28T11:21:26.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.463+0000 7f3f553ff700 1 Processor -- start 2023-11-28T11:21:26.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.463+0000 7f3f553ff700 1 -- start start 2023-11-28T11:21:26.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.464+0000 7f3f553ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f500b8510 0x7f3f501511f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:26.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.464+0000 7f3f553ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3f500cd460 0x7f3f500b6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:26.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.464+0000 7f3f553ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f500d0d60 0x7f3f500b7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:26.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.464+0000 7f3f553ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f500cd0c0 con 0x7f3f500d0d60 2023-11-28T11:21:26.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.464+0000 7f3f553ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5015a590 con 0x7f3f500b8510 2023-11-28T11:21:26.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.464+0000 7f3f553ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f50159270 con 0x7f3f500cd460 2023-11-28T11:21:26.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.464+0000 7f3f4f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f500d0d60 0x7f3f500b7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:26.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.464+0000 7f3f4f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f500d0d60 0x7f3f500b7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49174/0 (socket says 172.21.15.92:49174) 2023-11-28T11:21:26.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.464+0000 7f3f4f7fe700 1 -- 172.21.15.92:0/4194098028 learned_addr learned my addr 172.21.15.92:0/4194098028 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:26.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.465+0000 7f3f47fff700 1 --2- 172.21.15.92:0/4194098028 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3f500cd460 0x7f3f500b6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:26.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.465+0000 7f3f4effd700 1 --2- 172.21.15.92:0/4194098028 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f500b8510 0x7f3f501511f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:26.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.465+0000 7f3f4f7fe700 1 -- 172.21.15.92:0/4194098028 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3f500cd460 msgr2=0x7f3f500b6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:26.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.465+0000 7f3f4f7fe700 1 --2- 172.21.15.92:0/4194098028 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3f500cd460 0x7f3f500b6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:26.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.465+0000 7f3f4f7fe700 1 -- 172.21.15.92:0/4194098028 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f500b8510 msgr2=0x7f3f501511f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:26.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.465+0000 7f3f4f7fe700 1 --2- 172.21.15.92:0/4194098028 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f500b8510 0x7f3f501511f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:26.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.465+0000 7f3f4f7fe700 1 -- 172.21.15.92:0/4194098028 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f400094a0 con 0x7f3f500d0d60 2023-11-28T11:21:26.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.465+0000 7f3f4effd700 1 --2- 172.21.15.92:0/4194098028 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f500b8510 0x7f3f501511f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:21:26.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.465+0000 7f3f4f7fe700 1 --2- 172.21.15.92:0/4194098028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f500d0d60 0x7f3f500b7260 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3f4800cbc0 tx=0x7f3f4800ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:26.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.465+0000 7f3f477fe700 1 -- 172.21.15.92:0/4194098028 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f48003c00 con 0x7f3f500d0d60 2023-11-28T11:21:26.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.465+0000 7f3f477fe700 1 -- 172.21.15.92:0/4194098028 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f48003d60 con 0x7f3f500d0d60 2023-11-28T11:21:26.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.465+0000 7f3f553ff700 1 -- 172.21.15.92:0/4194098028 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f500d29c0 con 0x7f3f500d0d60 2023-11-28T11:21:26.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.465+0000 7f3f477fe700 1 -- 172.21.15.92:0/4194098028 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4800ab00 con 0x7f3f500d0d60 2023-11-28T11:21:26.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.466+0000 7f3f553ff700 1 -- 172.21.15.92:0/4194098028 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f500cfff0 con 0x7f3f500d0d60 2023-11-28T11:21:26.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.467+0000 7f3f553ff700 1 -- 172.21.15.92:0/4194098028 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f500d2b50 con 0x7f3f500d0d60 2023-11-28T11:21:26.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.467+0000 7f3f477fe700 1 -- 172.21.15.92:0/4194098028 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3f48010050 con 0x7f3f500d0d60 2023-11-28T11:21:26.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.468+0000 7f3f477fe700 1 --2- 172.21.15.92:0/4194098028 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3f300682f0 0x7f3f3006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:26.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.468+0000 7f3f477fe700 1 -- 172.21.15.92:0/4194098028 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3f48085ef0 con 0x7f3f500d0d60 2023-11-28T11:21:26.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.468+0000 7f3f4effd700 1 --2- 172.21.15.92:0/4194098028 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3f300682f0 0x7f3f3006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:26.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.469+0000 7f3f4effd700 1 --2- 172.21.15.92:0/4194098028 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3f300682f0 0x7f3f3006a7a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3f400090b0 tx=0x7f3f40008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:26.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.472+0000 7f3f477fe700 1 -- 172.21.15.92:0/4194098028 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3f48052400 con 0x7f3f500d0d60 2023-11-28T11:21:26.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.663+0000 7f3f553ff700 1 -- 172.21.15.92:0/4194098028 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3f5006a950 con 0x7f3f500d0d60 2023-11-28T11:21:26.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.663+0000 7f3f477fe700 1 -- 172.21.15.92:0/4194098028 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3f48055a20 con 0x7f3f500d0d60 2023-11-28T11:21:26.664 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-11-28T11:21:26.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.665+0000 7f3f553ff700 1 -- 172.21.15.92:0/4194098028 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3f300682f0 msgr2=0x7f3f3006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:26.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.665+0000 7f3f553ff700 1 --2- 172.21.15.92:0/4194098028 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3f300682f0 0x7f3f3006a7a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3f400090b0 tx=0x7f3f40008030 comp rx=0 tx=0).stop 2023-11-28T11:21:26.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.665+0000 7f3f553ff700 1 -- 172.21.15.92:0/4194098028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f500d0d60 msgr2=0x7f3f500b7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:26.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.665+0000 7f3f553ff700 1 --2- 172.21.15.92:0/4194098028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f500d0d60 0x7f3f500b7260 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3f4800cbc0 tx=0x7f3f4800ced0 comp rx=0 tx=0).stop 2023-11-28T11:21:26.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.665+0000 7f3f553ff700 1 -- 172.21.15.92:0/4194098028 shutdown_connections 2023-11-28T11:21:26.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.665+0000 7f3f553ff700 1 --2- 172.21.15.92:0/4194098028 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3f300682f0 0x7f3f3006a7a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:26.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.665+0000 7f3f553ff700 1 --2- 172.21.15.92:0/4194098028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f500d0d60 0x7f3f500b7260 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:26.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.665+0000 7f3f553ff700 1 --2- 172.21.15.92:0/4194098028 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3f500b8510 0x7f3f501511f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:26.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.666+0000 7f3f553ff700 1 --2- 172.21.15.92:0/4194098028 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3f500cd460 0x7f3f500b6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:26.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.666+0000 7f3f553ff700 1 -- 172.21.15.92:0/4194098028 >> 172.21.15.92:0/4194098028 conn(0x7f3f500719a0 msgr2=0x7f3f500b3950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:26.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.666+0000 7f3f553ff700 1 -- 172.21.15.92:0/4194098028 shutdown_connections 2023-11-28T11:21:26.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.666+0000 7f3f553ff700 1 -- 172.21.15.92:0/4194098028 wait complete. 2023-11-28T11:21:26.685 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T11:21:26.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.885+0000 7f34c91ff700 1 Processor -- start 2023-11-28T11:21:26.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.886+0000 7f34c91ff700 1 -- start start 2023-11-28T11:21:26.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.887+0000 7f34c91ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f34c415cbc0 0x7f34c40c9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:26.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.887+0000 7f34c91ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34c4160490 con 0x7f34c406b9e0 2023-11-28T11:21:26.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.887+0000 7f34c91ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34c40716e0 con 0x7f34c40cce90 2023-11-28T11:21:26.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.887+0000 7f34c91ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34c40719c0 con 0x7f34c415cbc0 2023-11-28T11:21:26.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.888+0000 7f34c359e700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f34c406b9e0 0x7f34c4160990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47544/0 (socket says 172.21.15.92:47544) 2023-11-28T11:21:26.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.888+0000 7f34c359e700 1 -- 172.21.15.92:0/3627290745 learned_addr learned my addr 172.21.15.92:0/3627290745 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:26.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.888+0000 7f34c259c700 1 --2- 172.21.15.92:0/3627290745 >> v2:172.21.15.169:3300/0 conn(0x7f34c415cbc0 0x7f34c40c9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:26.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.888+0000 7f34c1d9b700 1 -- 172.21.15.92:0/3627290745 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1040504312 0 0) 0x7f34c4160490 con 0x7f34c406b9e0 2023-11-28T11:21:26.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.888+0000 7f34c1d9b700 1 -- 172.21.15.92:0/3627290745 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34ac0035b0 con 0x7f34c406b9e0 2023-11-28T11:21:26.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.888+0000 7f34c1d9b700 1 -- 172.21.15.92:0/3627290745 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4170519578 0 0) 0x7f34c40716e0 con 0x7f34c40cce90 2023-11-28T11:21:26.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.888+0000 7f34c1d9b700 1 -- 172.21.15.92:0/3627290745 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34c4160490 con 0x7f34c40cce90 2023-11-28T11:21:26.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.888+0000 7f34c259c700 1 -- 172.21.15.92:0/3627290745 >> v1:172.21.15.169:6789/0 conn(0x7f34c40cce90 legacy=0x7f34c40cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:26.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.888+0000 7f34c259c700 1 -- 172.21.15.92:0/3627290745 >> v1:172.21.15.92:6789/0 conn(0x7f34c406b9e0 legacy=0x7f34c4160990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:26.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.888+0000 7f34c259c700 1 -- 172.21.15.92:0/3627290745 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34c4071ca0 con 0x7f34c415cbc0 2023-11-28T11:21:26.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.889+0000 7f34c259c700 1 --2- 172.21.15.92:0/3627290745 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34c415cbc0 0x7f34c40c9910 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f34b8009840 tx=0x7f34b8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e25186fea3a8fb59 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:26.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.889+0000 7f34c1d9b700 1 -- 172.21.15.92:0/3627290745 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b8011830 con 0x7f34c415cbc0 2023-11-28T11:21:26.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.889+0000 7f34c1d9b700 1 -- 172.21.15.92:0/3627290745 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34b8011990 con 0x7f34c415cbc0 2023-11-28T11:21:26.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.889+0000 7f34c1d9b700 1 -- 172.21.15.92:0/3627290745 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b8011d20 con 0x7f34c415cbc0 2023-11-28T11:21:26.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.889+0000 7f34c91ff700 1 -- 172.21.15.92:0/3627290745 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34c415cbc0 msgr2=0x7f34c40c9910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:26.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.889+0000 7f34c91ff700 1 --2- 172.21.15.92:0/3627290745 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34c415cbc0 0x7f34c40c9910 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f34b8009840 tx=0x7f34b8009b50 comp rx=0 tx=0).stop 2023-11-28T11:21:26.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.889+0000 7f34c91ff700 1 -- 172.21.15.92:0/3627290745 shutdown_connections 2023-11-28T11:21:26.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.889+0000 7f34c91ff700 1 --2- 172.21.15.92:0/3627290745 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34c415cbc0 0x7f34c40c9910 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:26.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.889+0000 7f34c91ff700 1 -- 172.21.15.92:0/3627290745 >> 172.21.15.92:0/3627290745 conn(0x7f34c40b0db0 msgr2=0x7f34c40b1180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:26.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.889+0000 7f34c91ff700 1 -- 172.21.15.92:0/3627290745 shutdown_connections 2023-11-28T11:21:26.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.889+0000 7f34c91ff700 1 -- 172.21.15.92:0/3627290745 wait complete. 2023-11-28T11:21:26.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.891+0000 7f34c91ff700 1 Processor -- start 2023-11-28T11:21:26.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.891+0000 7f34c91ff700 1 -- start start 2023-11-28T11:21:26.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.892+0000 7f34c91ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34c406b9e0 0x7f34c41f2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:26.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.892+0000 7f34c91ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34c40cce90 0x7f34c40c3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:26.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.892+0000 7f34c91ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34c415cbc0 0x7f34c40ab8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:26.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.892+0000 7f34c91ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c40c5850 con 0x7f34c406b9e0 2023-11-28T11:21:26.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.892+0000 7f34c91ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c40c3270 con 0x7f34c40cce90 2023-11-28T11:21:26.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.892+0000 7f34c91ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34c4157fc0 con 0x7f34c415cbc0 2023-11-28T11:21:26.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.893+0000 7f34c2d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34c406b9e0 0x7f34c41f2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:26.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.893+0000 7f34c2d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34c406b9e0 0x7f34c41f2c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49176/0 (socket says 172.21.15.92:49176) 2023-11-28T11:21:26.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.893+0000 7f34c2d9d700 1 -- 172.21.15.92:0/1201681311 learned_addr learned my addr 172.21.15.92:0/1201681311 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:26.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.893+0000 7f34c259c700 1 --2- 172.21.15.92:0/1201681311 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34c40cce90 0x7f34c40c3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:26.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.893+0000 7f34c359e700 1 --2- 172.21.15.92:0/1201681311 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34c415cbc0 0x7f34c40ab8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:26.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.893+0000 7f34c2d9d700 1 -- 172.21.15.92:0/1201681311 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34c415cbc0 msgr2=0x7f34c40ab8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:26.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.893+0000 7f34c2d9d700 1 --2- 172.21.15.92:0/1201681311 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34c415cbc0 0x7f34c40ab8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:26.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.893+0000 7f34c2d9d700 1 -- 172.21.15.92:0/1201681311 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34c40cce90 msgr2=0x7f34c40c3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:26.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.893+0000 7f34c2d9d700 1 --2- 172.21.15.92:0/1201681311 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34c40cce90 0x7f34c40c3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:26.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.893+0000 7f34c2d9d700 1 -- 172.21.15.92:0/1201681311 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34b80094a0 con 0x7f34c406b9e0 2023-11-28T11:21:26.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.893+0000 7f34c2d9d700 1 --2- 172.21.15.92:0/1201681311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34c406b9e0 0x7f34c41f2c90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f34b400c520 tx=0x7f34b400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:26.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.894+0000 7f34ab7fe700 1 -- 172.21.15.92:0/1201681311 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b40052d0 con 0x7f34c406b9e0 2023-11-28T11:21:26.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.894+0000 7f34ab7fe700 1 -- 172.21.15.92:0/1201681311 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34b4005430 con 0x7f34c406b9e0 2023-11-28T11:21:26.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.894+0000 7f34ab7fe700 1 -- 172.21.15.92:0/1201681311 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b40038c0 con 0x7f34c406b9e0 2023-11-28T11:21:26.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.894+0000 7f34c91ff700 1 -- 172.21.15.92:0/1201681311 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34c40c9e50 con 0x7f34c406b9e0 2023-11-28T11:21:26.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.894+0000 7f34c91ff700 1 -- 172.21.15.92:0/1201681311 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34c4071f80 con 0x7f34c406b9e0 2023-11-28T11:21:26.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.895+0000 7f34c91ff700 1 -- 172.21.15.92:0/1201681311 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34c40c9fe0 con 0x7f34c406b9e0 2023-11-28T11:21:26.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.896+0000 7f34ab7fe700 1 -- 172.21.15.92:0/1201681311 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f34b4019030 con 0x7f34c406b9e0 2023-11-28T11:21:26.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.897+0000 7f34ab7fe700 1 --2- 172.21.15.92:0/1201681311 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f34b00682f0 0x7f34b006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:26.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.897+0000 7f34ab7fe700 1 -- 172.21.15.92:0/1201681311 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f34b4085360 con 0x7f34c406b9e0 2023-11-28T11:21:26.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.897+0000 7f34c259c700 1 --2- 172.21.15.92:0/1201681311 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f34b00682f0 0x7f34b006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:26.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.898+0000 7f34c259c700 1 --2- 172.21.15.92:0/1201681311 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f34b00682f0 0x7f34b006a7a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f34b8009fd0 tx=0x7f34b801c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:26.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:26.901+0000 7f34ab7fe700 1 -- 172.21.15.92:0/1201681311 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f34b40517e0 con 0x7f34c406b9e0 2023-11-28T11:21:27.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.091+0000 7f34c91ff700 1 -- 172.21.15.92:0/1201681311 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f34c40c34e0 con 0x7f34c406b9e0 2023-11-28T11:21:27.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.092+0000 7f34ab7fe700 1 -- 172.21.15.92:0/1201681311 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f34b4054e00 con 0x7f34c406b9e0 2023-11-28T11:21:27.093 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-11-28T11:21:27.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.094+0000 7f34c91ff700 1 -- 172.21.15.92:0/1201681311 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f34b00682f0 msgr2=0x7f34b006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:27.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.094+0000 7f34c91ff700 1 --2- 172.21.15.92:0/1201681311 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f34b00682f0 0x7f34b006a7a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f34b8009fd0 tx=0x7f34b801c040 comp rx=0 tx=0).stop 2023-11-28T11:21:27.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.094+0000 7f34c91ff700 1 -- 172.21.15.92:0/1201681311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34c406b9e0 msgr2=0x7f34c41f2c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:27.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.094+0000 7f34c91ff700 1 --2- 172.21.15.92:0/1201681311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34c406b9e0 0x7f34c41f2c90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f34b400c520 tx=0x7f34b400c830 comp rx=0 tx=0).stop 2023-11-28T11:21:27.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.094+0000 7f34c91ff700 1 -- 172.21.15.92:0/1201681311 shutdown_connections 2023-11-28T11:21:27.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.094+0000 7f34c91ff700 1 --2- 172.21.15.92:0/1201681311 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f34c415cbc0 0x7f34c40ab8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:27.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.094+0000 7f34c91ff700 1 --2- 172.21.15.92:0/1201681311 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f34b00682f0 0x7f34b006a7a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:27.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.094+0000 7f34c91ff700 1 --2- 172.21.15.92:0/1201681311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34c406b9e0 0x7f34c41f2c90 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:27.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.094+0000 7f34c91ff700 1 --2- 172.21.15.92:0/1201681311 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f34c40cce90 0x7f34c40c3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:27.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.094+0000 7f34c91ff700 1 -- 172.21.15.92:0/1201681311 >> 172.21.15.92:0/1201681311 conn(0x7f34c40b0db0 msgr2=0x7f34c40b21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:27.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.094+0000 7f34c91ff700 1 -- 172.21.15.92:0/1201681311 shutdown_connections 2023-11-28T11:21:27.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.095+0000 7f34c91ff700 1 -- 172.21.15.92:0/1201681311 wait complete. 2023-11-28T11:21:27.114 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:21:27.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.309+0000 7fdae37ff700 1 Processor -- start 2023-11-28T11:21:27.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.309+0000 7fdae37ff700 1 -- start start 2023-11-28T11:21:27.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.310+0000 7fdae37ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fdadc0cce90 0x7fdadc0cd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:27.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.310+0000 7fdae37ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdadc15cbc0 0x7fdadc0c9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:27.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.310+0000 7fdae37ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdadc160490 con 0x7fdadc06b9e0 2023-11-28T11:21:27.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.310+0000 7fdae37ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdadc0cd7a0 con 0x7fdadc0cce90 2023-11-28T11:21:27.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.310+0000 7fdae37ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdadc0716e0 con 0x7fdadc15cbc0 2023-11-28T11:21:27.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.310+0000 7fdae1d9c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fdadc06b9e0 0x7fdadc160990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47552/0 (socket says 172.21.15.92:47552) 2023-11-28T11:21:27.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.310+0000 7fdae1d9c700 1 -- 172.21.15.92:0/3531365634 learned_addr learned my addr 172.21.15.92:0/3531365634 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:27.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.310+0000 7fdae159b700 1 --2- 172.21.15.92:0/3531365634 >> v2:172.21.15.169:3300/0 conn(0x7fdadc0cce90 0x7fdadc0cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:27.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.311+0000 7fdae0d9a700 1 --2- 172.21.15.92:0/3531365634 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdadc15cbc0 0x7fdadc0c9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:27.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.311+0000 7fdad3fff700 1 -- 172.21.15.92:0/3531365634 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1734388901 0 0) 0x7fdadc160490 con 0x7fdadc06b9e0 2023-11-28T11:21:27.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.311+0000 7fdad3fff700 1 -- 172.21.15.92:0/3531365634 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdac80035b0 con 0x7fdadc06b9e0 2023-11-28T11:21:27.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.311+0000 7fdae159b700 1 -- 172.21.15.92:0/3531365634 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdadc15cbc0 msgr2=0x7fdadc0c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:27.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.311+0000 7fdae159b700 1 --2- 172.21.15.92:0/3531365634 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdadc15cbc0 0x7fdadc0c9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:27.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.311+0000 7fdae159b700 1 -- 172.21.15.92:0/3531365634 >> v1:172.21.15.92:6789/0 conn(0x7fdadc06b9e0 legacy=0x7fdadc160990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:27.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.311+0000 7fdae159b700 1 -- 172.21.15.92:0/3531365634 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdadc0719c0 con 0x7fdadc0cce90 2023-11-28T11:21:27.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.311+0000 7fdae159b700 1 --2- 172.21.15.92:0/3531365634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdadc0cce90 0x7fdadc0cd260 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdad8009840 tx=0x7fdad8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe91978216a16a7c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:27.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.312+0000 7fdad3fff700 1 -- 172.21.15.92:0/3531365634 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad8011830 con 0x7fdadc0cce90 2023-11-28T11:21:27.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.312+0000 7fdad3fff700 1 -- 172.21.15.92:0/3531365634 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdad8011990 con 0x7fdadc0cce90 2023-11-28T11:21:27.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.312+0000 7fdad3fff700 1 -- 172.21.15.92:0/3531365634 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad8011d20 con 0x7fdadc0cce90 2023-11-28T11:21:27.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.312+0000 7fdae37ff700 1 -- 172.21.15.92:0/3531365634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdadc0cce90 msgr2=0x7fdadc0cd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:27.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.312+0000 7fdae37ff700 1 --2- 172.21.15.92:0/3531365634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdadc0cce90 0x7fdadc0cd260 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdad8009840 tx=0x7fdad8009b50 comp rx=0 tx=0).stop 2023-11-28T11:21:27.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.312+0000 7fdae37ff700 1 -- 172.21.15.92:0/3531365634 shutdown_connections 2023-11-28T11:21:27.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.312+0000 7fdae37ff700 1 --2- 172.21.15.92:0/3531365634 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdadc0cce90 0x7fdadc0cd260 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:27.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.312+0000 7fdae37ff700 1 --2- 172.21.15.92:0/3531365634 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdadc15cbc0 0x7fdadc0c9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:27.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.312+0000 7fdae37ff700 1 -- 172.21.15.92:0/3531365634 >> 172.21.15.92:0/3531365634 conn(0x7fdadc0b0db0 msgr2=0x7fdadc0b1180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:27.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.312+0000 7fdae37ff700 1 -- 172.21.15.92:0/3531365634 shutdown_connections 2023-11-28T11:21:27.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.312+0000 7fdae37ff700 1 -- 172.21.15.92:0/3531365634 wait complete. 2023-11-28T11:21:27.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.314+0000 7fdae37ff700 1 Processor -- start 2023-11-28T11:21:27.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.314+0000 7fdae37ff700 1 -- start start 2023-11-28T11:21:27.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.315+0000 7fdae37ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdadc06b9e0 0x7fdadc1f66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:27.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.315+0000 7fdae37ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdadc0cce90 0x7fdadc158170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:27.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.315+0000 7fdae37ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdadc15cbc0 0x7fdadc15f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:27.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.315+0000 7fdae37ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdadc0ac660 con 0x7fdadc15cbc0 2023-11-28T11:21:27.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.315+0000 7fdae37ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdadc1f6c10 con 0x7fdadc0cce90 2023-11-28T11:21:27.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.315+0000 7fdae37ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdadc0aba50 con 0x7fdadc06b9e0 2023-11-28T11:21:27.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.315+0000 7fdae1d9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdadc15cbc0 0x7fdadc15f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:27.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.315+0000 7fdae1d9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdadc15cbc0 0x7fdadc15f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49190/0 (socket says 172.21.15.92:49190) 2023-11-28T11:21:27.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.315+0000 7fdae1d9c700 1 -- 172.21.15.92:0/1336154037 learned_addr learned my addr 172.21.15.92:0/1336154037 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:27.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.315+0000 7fdae0d9a700 1 --2- 172.21.15.92:0/1336154037 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdadc0cce90 0x7fdadc158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:27.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.315+0000 7fdae159b700 1 --2- 172.21.15.92:0/1336154037 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdadc06b9e0 0x7fdadc1f66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:27.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.316+0000 7fdae1d9c700 1 -- 172.21.15.92:0/1336154037 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdadc06b9e0 msgr2=0x7fdadc1f66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:27.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.316+0000 7fdae1d9c700 1 --2- 172.21.15.92:0/1336154037 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdadc06b9e0 0x7fdadc1f66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:27.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.316+0000 7fdae1d9c700 1 -- 172.21.15.92:0/1336154037 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdadc0cce90 msgr2=0x7fdadc158170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:27.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.316+0000 7fdae1d9c700 1 --2- 172.21.15.92:0/1336154037 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdadc0cce90 0x7fdadc158170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:27.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.316+0000 7fdae1d9c700 1 -- 172.21.15.92:0/1336154037 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad80094a0 con 0x7fdadc15cbc0 2023-11-28T11:21:27.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.316+0000 7fdae1d9c700 1 --2- 172.21.15.92:0/1336154037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdadc15cbc0 0x7fdadc15f5a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fdad400caa0 tx=0x7fdad400cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:27.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.316+0000 7fdad1ffb700 1 -- 172.21.15.92:0/1336154037 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad400a9e0 con 0x7fdadc15cbc0 2023-11-28T11:21:27.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.316+0000 7fdad1ffb700 1 -- 172.21.15.92:0/1336154037 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdad400ab40 con 0x7fdadc15cbc0 2023-11-28T11:21:27.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.316+0000 7fdae37ff700 1 -- 172.21.15.92:0/1336154037 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdadc071f80 con 0x7fdadc15cbc0 2023-11-28T11:21:27.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.316+0000 7fdad1ffb700 1 -- 172.21.15.92:0/1336154037 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad4012400 con 0x7fdadc15cbc0 2023-11-28T11:21:27.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.316+0000 7fdae37ff700 1 -- 172.21.15.92:0/1336154037 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdadc071ca0 con 0x7fdadc15cbc0 2023-11-28T11:21:27.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.317+0000 7fdae37ff700 1 -- 172.21.15.92:0/1336154037 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdadc0c9e50 con 0x7fdadc15cbc0 2023-11-28T11:21:27.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.318+0000 7fdad1ffb700 1 -- 172.21.15.92:0/1336154037 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdad4019030 con 0x7fdadc15cbc0 2023-11-28T11:21:27.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.319+0000 7fdad1ffb700 1 --2- 172.21.15.92:0/1336154037 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fdac00682f0 0x7fdac006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:27.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.319+0000 7fdad1ffb700 1 -- 172.21.15.92:0/1336154037 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fdad4085a20 con 0x7fdadc15cbc0 2023-11-28T11:21:27.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.319+0000 7fdae159b700 1 --2- 172.21.15.92:0/1336154037 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fdac00682f0 0x7fdac006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:27.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.320+0000 7fdae159b700 1 --2- 172.21.15.92:0/1336154037 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fdac00682f0 0x7fdac006a7a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdad8000c00 tx=0x7fdad8008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:27.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.323+0000 7fdad1ffb700 1 -- 172.21.15.92:0/1336154037 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fdad4052050 con 0x7fdadc15cbc0 2023-11-28T11:21:27.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.554+0000 7fdae37ff700 1 -- 172.21.15.92:0/1336154037 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdadc067880 con 0x7fdadc15cbc0 2023-11-28T11:21:27.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.555+0000 7fdad1ffb700 1 -- 172.21.15.92:0/1336154037 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4647 (secure 0 0 0) 0x7fdad4055670 con 0x7fdadc15cbc0 2023-11-28T11:21:27.556 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:21:27.556 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/3257209990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3257209990},{"type":"v1","addr":"172.21.15.92:6836","nonce":3257209990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2193338949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2193338949},{"type":"v1","addr":"172.21.15.169:6834","nonce":2193338949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4308,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/471035619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":471035619},{"type":"v1","addr":"172.21.15.92:6837","nonce":471035619}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:20:56.234970+0000","modified":"2023-11-28T11:21:16.544352+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4326},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4326":{"gid":4326,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/2067251916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2067251916},{"type":"v1","addr":"172.21.15.169:6835","nonce":2067251916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T11:21:27.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.557+0000 7fdae37ff700 1 -- 172.21.15.92:0/1336154037 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fdac00682f0 msgr2=0x7fdac006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:27.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.557+0000 7fdae37ff700 1 --2- 172.21.15.92:0/1336154037 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fdac00682f0 0x7fdac006a7a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdad8000c00 tx=0x7fdad8008030 comp rx=0 tx=0).stop 2023-11-28T11:21:27.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.557+0000 7fdae37ff700 1 -- 172.21.15.92:0/1336154037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdadc15cbc0 msgr2=0x7fdadc15f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:27.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.557+0000 7fdae37ff700 1 --2- 172.21.15.92:0/1336154037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdadc15cbc0 0x7fdadc15f5a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fdad400caa0 tx=0x7fdad400cdb0 comp rx=0 tx=0).stop 2023-11-28T11:21:27.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.557+0000 7fdae37ff700 1 -- 172.21.15.92:0/1336154037 shutdown_connections 2023-11-28T11:21:27.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.557+0000 7fdae37ff700 1 --2- 172.21.15.92:0/1336154037 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fdac00682f0 0x7fdac006a7a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:27.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.557+0000 7fdae37ff700 1 --2- 172.21.15.92:0/1336154037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdadc15cbc0 0x7fdadc15f5a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:27.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.557+0000 7fdae37ff700 1 --2- 172.21.15.92:0/1336154037 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdadc06b9e0 0x7fdadc1f66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:27.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.557+0000 7fdae37ff700 1 --2- 172.21.15.92:0/1336154037 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdadc0cce90 0x7fdadc158170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:27.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.557+0000 7fdae37ff700 1 -- 172.21.15.92:0/1336154037 >> 172.21.15.92:0/1336154037 conn(0x7fdadc0b0db0 msgr2=0x7fdadc0cb340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:27.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.557+0000 7fdae37ff700 1 -- 172.21.15.92:0/1336154037 shutdown_connections 2023-11-28T11:21:27.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:27.558+0000 7fdae37ff700 1 -- 172.21.15.92:0/1336154037 wait complete. 2023-11-28T11:21:27.559 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 9 2023-11-28T11:21:27.979 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T11:20:56.234970+0000', 'modified': '2023-11-28T11:21:16.544352+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4326}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4326': {'gid': 4326, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.169:6835/2067251916', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.169:6833', 'nonce': 2067251916}, {'type': 'v1', 'addr': '172.21.15.169:6835', 'nonce': 2067251916}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4, 5], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T11:21:27.979 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T11:21:27.979 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-28T11:21:28.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.175+0000 7fc90b7ff700 1 Processor -- start 2023-11-28T11:21:28.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.176+0000 7fc90b7ff700 1 -- start start 2023-11-28T11:21:28.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.177+0000 7fc90b7ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc9040b1c60 0x7fc9040cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:28.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.177+0000 7fc90b7ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9040d1950 con 0x7fc9040b4fc0 2023-11-28T11:21:28.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.177+0000 7fc90b7ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9040b2500 con 0x7fc9040d2df0 2023-11-28T11:21:28.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.177+0000 7fc90b7ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9040d3d30 con 0x7fc9040b1c60 2023-11-28T11:21:28.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.177+0000 7fc908d9a700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc9040b4fc0 0x7fc9040cf010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47560/0 (socket says 172.21.15.92:47560) 2023-11-28T11:21:28.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.177+0000 7fc908d9a700 1 -- 172.21.15.92:0/3576624404 learned_addr learned my addr 172.21.15.92:0/3576624404 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:28.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.177+0000 7fc909d9c700 1 --2- 172.21.15.92:0/3576624404 >> v2:172.21.15.169:3300/0 conn(0x7fc9040b1c60 0x7fc9040cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:28.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.177+0000 7fc8fbfff700 1 -- 172.21.15.92:0/3576624404 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2217936106 0 0) 0x7fc9040d1950 con 0x7fc9040b4fc0 2023-11-28T11:21:28.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.177+0000 7fc8fbfff700 1 -- 172.21.15.92:0/3576624404 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8f00035b0 con 0x7fc9040b4fc0 2023-11-28T11:21:28.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.178+0000 7fc8fbfff700 1 -- 172.21.15.92:0/3576624404 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3520106359 0 0) 0x7fc9040b2500 con 0x7fc9040d2df0 2023-11-28T11:21:28.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.178+0000 7fc8fbfff700 1 -- 172.21.15.92:0/3576624404 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9040d1950 con 0x7fc9040d2df0 2023-11-28T11:21:28.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.178+0000 7fc8fbfff700 1 -- 172.21.15.92:0/3576624404 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2737690529 0 0) 0x7fc8f00035b0 con 0x7fc9040b4fc0 2023-11-28T11:21:28.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.178+0000 7fc909d9c700 1 -- 172.21.15.92:0/3576624404 >> v1:172.21.15.169:6789/0 conn(0x7fc9040d2df0 legacy=0x7fc9040d31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:28.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.178+0000 7fc909d9c700 1 -- 172.21.15.92:0/3576624404 >> v1:172.21.15.92:6789/0 conn(0x7fc9040b4fc0 legacy=0x7fc9040cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:28.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.178+0000 7fc909d9c700 1 -- 172.21.15.92:0/3576624404 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9040d3a50 con 0x7fc9040b1c60 2023-11-28T11:21:28.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.178+0000 7fc909d9c700 1 --2- 172.21.15.92:0/3576624404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9040b1c60 0x7fc9040cd620 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc009d90 tx=0x7fc8fc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2561bdb1525311ce server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:28.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.178+0000 7fc8fbfff700 1 -- 172.21.15.92:0/3576624404 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc00e070 con 0x7fc9040b1c60 2023-11-28T11:21:28.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.178+0000 7fc8fbfff700 1 -- 172.21.15.92:0/3576624404 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8fc003a20 con 0x7fc9040b1c60 2023-11-28T11:21:28.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.178+0000 7fc8fbfff700 1 -- 172.21.15.92:0/3576624404 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f4001670 con 0x7fc9040b1c60 2023-11-28T11:21:28.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.179+0000 7fc90b7ff700 1 -- 172.21.15.92:0/3576624404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9040b1c60 msgr2=0x7fc9040cd620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:28.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.179+0000 7fc90b7ff700 1 --2- 172.21.15.92:0/3576624404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9040b1c60 0x7fc9040cd620 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc009d90 tx=0x7fc8fc00b330 comp rx=0 tx=0).stop 2023-11-28T11:21:28.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.179+0000 7fc90b7ff700 1 -- 172.21.15.92:0/3576624404 shutdown_connections 2023-11-28T11:21:28.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.179+0000 7fc90b7ff700 1 --2- 172.21.15.92:0/3576624404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9040b1c60 0x7fc9040cd620 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:28.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.179+0000 7fc90b7ff700 1 -- 172.21.15.92:0/3576624404 >> 172.21.15.92:0/3576624404 conn(0x7fc9040719a0 msgr2=0x7fc904071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:28.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.179+0000 7fc90b7ff700 1 -- 172.21.15.92:0/3576624404 shutdown_connections 2023-11-28T11:21:28.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.179+0000 7fc90b7ff700 1 -- 172.21.15.92:0/3576624404 wait complete. 2023-11-28T11:21:28.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.181+0000 7fc90b7ff700 1 Processor -- start 2023-11-28T11:21:28.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.181+0000 7fc90b7ff700 1 -- start start 2023-11-28T11:21:28.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.182+0000 7fc90b7ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9040b1c60 0x7fc90415a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:28.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.182+0000 7fc90b7ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9040b4fc0 0x7fc9041e52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:28.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.182+0000 7fc90b7ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9040d2df0 0x7fc9041e7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:28.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.182+0000 7fc90b7ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9041e5d80 con 0x7fc9040b4fc0 2023-11-28T11:21:28.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.182+0000 7fc90b7ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9041e6060 con 0x7fc9040b1c60 2023-11-28T11:21:28.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.182+0000 7fc90b7ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9041e6340 con 0x7fc9040d2df0 2023-11-28T11:21:28.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.182+0000 7fc908d9a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9040b4fc0 0x7fc9041e52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:28.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.182+0000 7fc908d9a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9040b4fc0 0x7fc9041e52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49200/0 (socket says 172.21.15.92:49200) 2023-11-28T11:21:28.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.182+0000 7fc908d9a700 1 -- 172.21.15.92:0/3910479371 learned_addr learned my addr 172.21.15.92:0/3910479371 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:28.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.182+0000 7fc909d9c700 1 --2- 172.21.15.92:0/3910479371 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9040d2df0 0x7fc9041e7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:28.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.182+0000 7fc90959b700 1 --2- 172.21.15.92:0/3910479371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9040b1c60 0x7fc90415a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:28.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.183+0000 7fc908d9a700 1 -- 172.21.15.92:0/3910479371 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9040d2df0 msgr2=0x7fc9041e7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:28.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.183+0000 7fc908d9a700 1 --2- 172.21.15.92:0/3910479371 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9040d2df0 0x7fc9041e7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:28.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.183+0000 7fc908d9a700 1 -- 172.21.15.92:0/3910479371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9040b1c60 msgr2=0x7fc90415a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:28.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.183+0000 7fc908d9a700 1 --2- 172.21.15.92:0/3910479371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9040b1c60 0x7fc90415a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:28.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.183+0000 7fc908d9a700 1 -- 172.21.15.92:0/3910479371 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8fc009a70 con 0x7fc9040b4fc0 2023-11-28T11:21:28.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.183+0000 7fc908d9a700 1 --2- 172.21.15.92:0/3910479371 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9040b4fc0 0x7fc9041e52c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc8f400c6b0 tx=0x7fc8f400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:28.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.183+0000 7fc8f9ffb700 1 -- 172.21.15.92:0/3910479371 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f4004c80 con 0x7fc9040b4fc0 2023-11-28T11:21:28.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.183+0000 7fc8f9ffb700 1 -- 172.21.15.92:0/3910479371 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8f4004de0 con 0x7fc9040b4fc0 2023-11-28T11:21:28.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.183+0000 7fc90b7ff700 1 -- 172.21.15.92:0/3910479371 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9040b88f0 con 0x7fc9040b4fc0 2023-11-28T11:21:28.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.183+0000 7fc8f9ffb700 1 -- 172.21.15.92:0/3910479371 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f40072f0 con 0x7fc9040b4fc0 2023-11-28T11:21:28.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.183+0000 7fc90b7ff700 1 -- 172.21.15.92:0/3910479371 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9040d15e0 con 0x7fc9040b4fc0 2023-11-28T11:21:28.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.185+0000 7fc8f9ffb700 1 -- 172.21.15.92:0/3910479371 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc8f401a020 con 0x7fc9040b4fc0 2023-11-28T11:21:28.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.186+0000 7fc8f9ffb700 1 --2- 172.21.15.92:0/3910479371 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc8e80682f0 0x7fc8e806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:28.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.186+0000 7fc8f9ffb700 1 -- 172.21.15.92:0/3910479371 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fc8f4085c30 con 0x7fc9040b4fc0 2023-11-28T11:21:28.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.186+0000 7fc90959b700 1 --2- 172.21.15.92:0/3910479371 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc8e80682f0 0x7fc8e806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:28.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.187+0000 7fc90959b700 1 --2- 172.21.15.92:0/3910479371 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc8e80682f0 0x7fc8e806a7a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc900001220 tx=0x7fc900009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:28.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.188+0000 7fc91f041b80 1 Processor -- start 2023-11-28T11:21:28.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.188+0000 7fc91f041b80 1 -- start start 2023-11-28T11:21:28.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.189+0000 7fc91f041b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ae8787df50 0x55ae878379f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:28.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.190+0000 7fc91f041b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ae87874ac0 0x55ae8787e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:28.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.190+0000 7fc91f041b80 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ae87860660 0x55ae87838b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:28.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.190+0000 7fc91f041b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae877b5de0 con 0x55ae8787df50 2023-11-28T11:21:28.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.190+0000 7fc91f041b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae87722140 con 0x55ae87874ac0 2023-11-28T11:21:28.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.190+0000 7fc91f041b80 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ae87790db0 con 0x55ae87860660 2023-11-28T11:21:28.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.190+0000 7fc90959b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ae8787df50 0x55ae878379f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:28.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.190+0000 7fc90959b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ae8787df50 0x55ae878379f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49204/0 (socket says 172.21.15.92:49204) 2023-11-28T11:21:28.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.190+0000 7fc90959b700 1 -- 172.21.15.92:0/3974738340 learned_addr learned my addr 172.21.15.92:0/3974738340 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:28.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.190+0000 7fc908d9a700 1 --2- 172.21.15.92:0/3974738340 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ae87874ac0 0x55ae8787e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:28.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.190+0000 7fc909d9c700 1 --2- 172.21.15.92:0/3974738340 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ae87860660 0x55ae87838b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:28.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.190+0000 7fc90959b700 1 -- 172.21.15.92:0/3974738340 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ae87860660 msgr2=0x55ae87838b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:28.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.190+0000 7fc90959b700 1 --2- 172.21.15.92:0/3974738340 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ae87860660 0x55ae87838b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:28.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.190+0000 7fc90959b700 1 -- 172.21.15.92:0/3974738340 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ae87874ac0 msgr2=0x55ae8787e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:28.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.190+0000 7fc90959b700 1 --2- 172.21.15.92:0/3974738340 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ae87874ac0 0x55ae8787e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:28.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.190+0000 7fc90959b700 1 -- 172.21.15.92:0/3974738340 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae8757e790 con 0x55ae8787df50 2023-11-28T11:21:28.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.190+0000 7fc908d9a700 1 --2- 172.21.15.92:0/3974738340 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ae87874ac0 0x55ae8787e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:21:28.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.190+0000 7fc90959b700 1 --2- 172.21.15.92:0/3974738340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ae8787df50 0x55ae878379f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc90000f6f0 tx=0x7fc900012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcac3f72ae44fef7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:28.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.191+0000 7fc8eeffd700 1 -- 172.21.15.92:0/3974738340 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90000fe70 con 0x55ae8787df50 2023-11-28T11:21:28.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.191+0000 7fc8eeffd700 1 -- 172.21.15.92:0/3974738340 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc900014450 con 0x55ae8787df50 2023-11-28T11:21:28.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.191+0000 7fc8eeffd700 1 -- 172.21.15.92:0/3974738340 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc900014800 con 0x55ae8787df50 2023-11-28T11:21:28.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.191+0000 7fc91f041b80 1 -- 172.21.15.92:0/3974738340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ae8787df50 msgr2=0x55ae878379f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:28.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.191+0000 7fc91f041b80 1 --2- 172.21.15.92:0/3974738340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ae8787df50 0x55ae878379f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc90000f6f0 tx=0x7fc900012000 comp rx=0 tx=0).stop 2023-11-28T11:21:28.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.191+0000 7fc91f041b80 1 -- 172.21.15.92:0/3974738340 shutdown_connections 2023-11-28T11:21:28.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.191+0000 7fc91f041b80 1 --2- 172.21.15.92:0/3974738340 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ae87860660 0x55ae87838b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:28.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.191+0000 7fc91f041b80 1 --2- 172.21.15.92:0/3974738340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ae8787df50 0x55ae878379f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:28.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.191+0000 7fc91f041b80 1 --2- 172.21.15.92:0/3974738340 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ae87874ac0 0x55ae8787e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:28.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.191+0000 7fc91f041b80 1 -- 172.21.15.92:0/3974738340 >> 172.21.15.92:0/3974738340 conn(0x55ae8782dd20 msgr2=0x55ae87836f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:28.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.191+0000 7fc91f041b80 1 -- 172.21.15.92:0/3974738340 shutdown_connections 2023-11-28T11:21:28.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.191+0000 7fc91f041b80 1 -- 172.21.15.92:0/3974738340 wait complete. 2023-11-28T11:21:28.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.192+0000 7fc91f041b80 1 -- start start 2023-11-28T11:21:28.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.192+0000 7fc91f041b80 1 Processor -- start 2023-11-28T11:21:28.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.193+0000 7fc91f041b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ae87860660 0x55ae8785de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:28.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.193+0000 7fc91f041b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ae87874ac0 0x55ae87861ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:28.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.193+0000 7fc91f041b80 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ae8787df50 0x55ae8787c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:28.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.193+0000 7fc91f041b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae8778a120 con 0x55ae87860660 2023-11-28T11:21:28.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.193+0000 7fc91f041b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae8757f390 con 0x55ae87874ac0 2023-11-28T11:21:28.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.193+0000 7fc91f041b80 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ae87722340 con 0x55ae8787df50 2023-11-28T11:21:28.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.194+0000 7fc91f041b80 10 client.0 _ll_register_callbacks cb 0x55ae87728400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:21:28.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.194+0000 7fc90959b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ae87860660 0x55ae8785de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:28.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.194+0000 7fc90959b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ae87860660 0x55ae8785de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49214/0 (socket says 172.21.15.92:49214) 2023-11-28T11:21:28.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.194+0000 7fc90959b700 1 -- 172.21.15.92:0/4277377151 learned_addr learned my addr 172.21.15.92:0/4277377151 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:28.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.194+0000 7fc908d9a700 1 --2- 172.21.15.92:0/4277377151 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ae87874ac0 0x55ae87861ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:28.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.194+0000 7fc909d9c700 1 --2- 172.21.15.92:0/4277377151 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ae8787df50 0x55ae8787c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:28.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.194+0000 7fc90959b700 1 -- 172.21.15.92:0/4277377151 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ae8787df50 msgr2=0x55ae8787c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:28.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.194+0000 7fc90959b700 1 --2- 172.21.15.92:0/4277377151 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ae8787df50 0x55ae8787c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:28.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.194+0000 7fc90959b700 1 -- 172.21.15.92:0/4277377151 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ae87874ac0 msgr2=0x55ae87861ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:28.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.194+0000 7fc90959b700 1 --2- 172.21.15.92:0/4277377151 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ae87874ac0 0x55ae87861ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:28.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.194+0000 7fc90959b700 1 -- 172.21.15.92:0/4277377151 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9040d1950 con 0x55ae87860660 2023-11-28T11:21:28.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.194+0000 7fc90959b700 1 --2- 172.21.15.92:0/4277377151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ae87860660 0x55ae8785de80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc900014bc0 tx=0x7fc900001190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa7c9aef0b567e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:28.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.194+0000 7fc8e3fff700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-11-28T11:21:28.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.195+0000 7fc8e3fff700 1 -- 172.21.15.92:0/4277377151 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90002b620 con 0x55ae87860660 2023-11-28T11:21:28.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.195+0000 7fc8e3fff700 1 -- 172.21.15.92:0/4277377151 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fc90001c040 con 0x55ae87860660 2023-11-28T11:21:28.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.195+0000 7fc91f041b80 1 -- 172.21.15.92:0/4277377151 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ae8778e030 con 0x55ae87860660 2023-11-28T11:21:28.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.195+0000 7fc8e3fff700 1 -- 172.21.15.92:0/4277377151 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc90002b780 con 0x55ae87860660 2023-11-28T11:21:28.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.195+0000 7fc91f041b80 10 client.4797 fetch_fsmap learned FSMap version 9 2023-11-28T11:21:28.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.195+0000 7fc91f041b80 1 -- 172.21.15.92:0/4277377151 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x55ae8766b0a0 con 0x55ae87860660 2023-11-28T11:21:28.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.195+0000 7fc8e3fff700 1 -- 172.21.15.92:0/4277377151 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ae8778e030 con 0x55ae87860660 2023-11-28T11:21:28.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.196+0000 7fc8e3fff700 1 -- 172.21.15.92:0/4277377151 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 9) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fc900035070 con 0x55ae87860660 2023-11-28T11:21:28.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.196+0000 7fc91f041b80 10 client.4797 fetch_fsmap finished waiting for FSMap version 9 2023-11-28T11:21:28.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.196+0000 7fc91f041b80 10 client.4797 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-28T11:21:28.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.196+0000 7fc91f041b80 20 client.4797 populate_metadata read hostname 'smithi092' 2023-11-28T11:21:28.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.196+0000 7fc91f041b80 1 --2- 172.21.15.92:0/4277377151 >> [v2:172.21.15.169:6833/2067251916,v1:172.21.15.169:6835/2067251916] conn(0x55ae87861060 0x55ae87861430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:28.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.196+0000 7fc91f041b80 4 client.4797 mds_command: new command op to 4326 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:21:28.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.196+0000 7fc91f041b80 1 -- 172.21.15.92:0/4277377151 --> [v2:172.21.15.169:6833/2067251916,v1:172.21.15.169:6835/2067251916] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ae875d9ef0 con 0x55ae87861060 2023-11-28T11:21:28.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.196+0000 7fc908d9a700 1 --2- 172.21.15.92:0/4277377151 >> [v2:172.21.15.169:6833/2067251916,v1:172.21.15.169:6835/2067251916] conn(0x55ae87861060 0x55ae87861430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:28.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.197+0000 7fc8e3fff700 10 client.4797 ms_handle_connect on v2:172.21.15.169:6833/2067251916 2023-11-28T11:21:28.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.197+0000 7fc908d9a700 1 --2- 172.21.15.92:0/4277377151 >> [v2:172.21.15.169:6833/2067251916,v1:172.21.15.169:6835/2067251916] conn(0x55ae87861060 0x55ae87861430 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c3a4391f96ca367 server_cookie=433746ff37504e41 in_seq=0 out_seq=0 2023-11-28T11:21:28.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.198+0000 7fc8e3fff700 1 -- 172.21.15.92:0/4277377151 <== mds.0 v2:172.21.15.169:6833/2067251916 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x55ae875d9ef0 con 0x55ae87861060 2023-11-28T11:21:28.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.198+0000 7fc8e3fff700 10 client.4797 handle_command_reply: tid=0 2023-11-28T11:21:28.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.198+0000 7fc91f041b80 1 client.4797 shutdown 2023-11-28T11:21:28.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.198+0000 7fc91f041b80 1 -- 172.21.15.92:0/4277377151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ae87860660 msgr2=0x55ae8785de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:28.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.198+0000 7fc91f041b80 1 --2- 172.21.15.92:0/4277377151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ae87860660 0x55ae8785de80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc900014bc0 tx=0x7fc900001190 comp rx=0 tx=0).stop 2023-11-28T11:21:28.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.198+0000 7fc91f041b80 1 -- 172.21.15.92:0/4277377151 shutdown_connections 2023-11-28T11:21:28.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.198+0000 7fc91f041b80 1 --2- 172.21.15.92:0/4277377151 >> [v2:172.21.15.169:6833/2067251916,v1:172.21.15.169:6835/2067251916] conn(0x55ae87861060 0x55ae87861430 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:28.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.198+0000 7fc91f041b80 1 --2- 172.21.15.92:0/4277377151 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ae8787df50 0x55ae8787c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:28.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.198+0000 7fc91f041b80 1 --2- 172.21.15.92:0/4277377151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ae87860660 0x55ae8785de80 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:28.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.198+0000 7fc91f041b80 1 --2- 172.21.15.92:0/4277377151 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ae87874ac0 0x55ae87861ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:28.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.198+0000 7fc91f041b80 1 -- 172.21.15.92:0/4277377151 >> 172.21.15.92:0/4277377151 conn(0x55ae8782dd20 msgr2=0x55ae87836f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:28.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.198+0000 7fc8e3fff700 0 client.4797 ms_handle_reset on v2:172.21.15.169:6833/2067251916 2023-11-28T11:21:28.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.198+0000 7fc91f041b80 1 -- 172.21.15.92:0/4277377151 shutdown_connections 2023-11-28T11:21:28.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.199+0000 7fc91f041b80 1 -- 172.21.15.92:0/4277377151 wait complete. 2023-11-28T11:21:28.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.199+0000 7fc91f041b80 20 client.4797 trim_cache size 0 max 16384 2023-11-28T11:21:28.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.216+0000 7fc91f041b80 1 Processor -- start 2023-11-28T11:21:28.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.216+0000 7fc91f041b80 1 -- start start 2023-11-28T11:21:28.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.217+0000 7fc91f041b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ae87874ac0 0x55ae87835280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:28.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.217+0000 7fc91f041b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ae8787df50 0x55ae87838d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:28.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.217+0000 7fc91f041b80 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ae8787d990 0x55ae8787c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:28.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.217+0000 7fc91f041b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae8787d3f0 con 0x55ae87874ac0 2023-11-28T11:21:28.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.217+0000 7fc91f041b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae878357c0 con 0x55ae8787df50 2023-11-28T11:21:28.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.217+0000 7fc91f041b80 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ae878619e0 con 0x55ae8787d990 2023-11-28T11:21:28.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.217+0000 7fc90959b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ae87874ac0 0x55ae87835280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:28.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.217+0000 7fc90959b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ae87874ac0 0x55ae87835280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49230/0 (socket says 172.21.15.92:49230) 2023-11-28T11:21:28.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.217+0000 7fc90959b700 1 -- 172.21.15.92:0/2237229228 learned_addr learned my addr 172.21.15.92:0/2237229228 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:28.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.217+0000 7fc90959b700 1 -- 172.21.15.92:0/2237229228 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ae8787d990 msgr2=0x55ae8787c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:28.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.217+0000 7fc90959b700 1 --2- 172.21.15.92:0/2237229228 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ae8787d990 0x55ae8787c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:28.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.217+0000 7fc908d9a700 1 --2- 172.21.15.92:0/2237229228 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ae8787df50 0x55ae87838d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:28.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.217+0000 7fc90959b700 1 -- 172.21.15.92:0/2237229228 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ae8787df50 msgr2=0x55ae87838d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:28.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.217+0000 7fc90959b700 1 --2- 172.21.15.92:0/2237229228 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ae8787df50 0x55ae87838d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:28.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.217+0000 7fc90959b700 1 -- 172.21.15.92:0/2237229228 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae877512e0 con 0x55ae87874ac0 2023-11-28T11:21:28.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.218+0000 7fc90959b700 1 --2- 172.21.15.92:0/2237229228 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ae87874ac0 0x55ae87835280 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc900007fb0 tx=0x7fc90002b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0a3d18ae3987c27 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:28.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.218+0000 7fc8ef7fe700 1 -- 172.21.15.92:0/2237229228 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc900014bf0 con 0x55ae87874ac0 2023-11-28T11:21:28.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.218+0000 7fc8ef7fe700 1 -- 172.21.15.92:0/2237229228 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc90002b420 con 0x55ae87874ac0 2023-11-28T11:21:28.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.218+0000 7fc8ef7fe700 1 -- 172.21.15.92:0/2237229228 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90002b930 con 0x55ae87874ac0 2023-11-28T11:21:28.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.218+0000 7fc91f041b80 1 -- 172.21.15.92:0/2237229228 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ae87874ac0 msgr2=0x55ae87835280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:28.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.218+0000 7fc91f041b80 1 --2- 172.21.15.92:0/2237229228 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ae87874ac0 0x55ae87835280 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc900007fb0 tx=0x7fc90002b000 comp rx=0 tx=0).stop 2023-11-28T11:21:28.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.218+0000 7fc91f041b80 1 -- 172.21.15.92:0/2237229228 shutdown_connections 2023-11-28T11:21:28.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.218+0000 7fc91f041b80 1 --2- 172.21.15.92:0/2237229228 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ae8787d990 0x55ae8787c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:28.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.218+0000 7fc91f041b80 1 --2- 172.21.15.92:0/2237229228 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ae87874ac0 0x55ae87835280 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:28.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.218+0000 7fc91f041b80 1 --2- 172.21.15.92:0/2237229228 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ae8787df50 0x55ae87838d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:28.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.218+0000 7fc91f041b80 1 -- 172.21.15.92:0/2237229228 >> 172.21.15.92:0/2237229228 conn(0x55ae8782dd20 msgr2=0x55ae87836ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:28.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.219+0000 7fc91f041b80 1 -- 172.21.15.92:0/2237229228 shutdown_connections 2023-11-28T11:21:28.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.219+0000 7fc91f041b80 1 -- 172.21.15.92:0/2237229228 wait complete. 2023-11-28T11:21:28.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.219+0000 7fc91f041b80 1 -- start start 2023-11-28T11:21:28.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.220+0000 7fc91f041b80 1 Processor -- start 2023-11-28T11:21:28.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.221+0000 7fc91f041b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ae87874ac0 0x55ae878e27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:28.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.221+0000 7fc91f041b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ae8787d990 0x55ae87873840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:28.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.221+0000 7fc91f041b80 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ae8787df50 0x55ae87865c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:28.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.221+0000 7fc91f041b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae87838000 con 0x55ae87874ac0 2023-11-28T11:21:28.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.221+0000 7fc91f041b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae87834800 con 0x55ae8787d990 2023-11-28T11:21:28.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.221+0000 7fc91f041b80 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ae8785ef30 con 0x55ae8787df50 2023-11-28T11:21:28.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.221+0000 7fc91f041b80 10 client.0 _ll_register_callbacks cb 0x55ae87728400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:21:28.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.221+0000 7fc90959b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ae87874ac0 0x55ae878e27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:28.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.221+0000 7fc90959b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ae87874ac0 0x55ae878e27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49236/0 (socket says 172.21.15.92:49236) 2023-11-28T11:21:28.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.221+0000 7fc90959b700 1 -- 172.21.15.92:0/2294824524 learned_addr learned my addr 172.21.15.92:0/2294824524 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:28.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.222+0000 7fc908d9a700 1 --2- 172.21.15.92:0/2294824524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ae8787d990 0x55ae87873840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:28.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.222+0000 7fc909d9c700 1 --2- 172.21.15.92:0/2294824524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ae8787df50 0x55ae87865c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:28.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.222+0000 7fc90959b700 1 -- 172.21.15.92:0/2294824524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ae8787df50 msgr2=0x55ae87865c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:28.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.222+0000 7fc90959b700 1 --2- 172.21.15.92:0/2294824524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ae8787df50 0x55ae87865c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:28.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.222+0000 7fc90959b700 1 -- 172.21.15.92:0/2294824524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ae8787d990 msgr2=0x55ae87873840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:28.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.222+0000 7fc90959b700 1 --2- 172.21.15.92:0/2294824524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ae8787d990 0x55ae87873840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:28.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.222+0000 7fc90959b700 1 -- 172.21.15.92:0/2294824524 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ae8766b0a0 con 0x55ae87874ac0 2023-11-28T11:21:28.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.222+0000 7fc909d9c700 1 --2- 172.21.15.92:0/2294824524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ae8787df50 0x55ae87865c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:21:28.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.222+0000 7fc90959b700 1 --2- 172.21.15.92:0/2294824524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ae87874ac0 0x55ae878e27a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc900001030 tx=0x7fc90000a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd8af13f63a11110 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:28.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.222+0000 7fc8e3fff700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-11-28T11:21:28.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.222+0000 7fc8e3fff700 1 -- 172.21.15.92:0/2294824524 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc900045070 con 0x55ae87874ac0 2023-11-28T11:21:28.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.222+0000 7fc8e3fff700 1 -- 172.21.15.92:0/2294824524 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fc90002bb40 con 0x55ae87874ac0 2023-11-28T11:21:28.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.222+0000 7fc91f041b80 1 -- 172.21.15.92:0/2294824524 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc900014450 con 0x55ae87874ac0 2023-11-28T11:21:28.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.223+0000 7fc8e3fff700 1 -- 172.21.15.92:0/2294824524 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc900006860 con 0x55ae87874ac0 2023-11-28T11:21:28.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.223+0000 7fc91f041b80 10 client.4803 fetch_fsmap learned FSMap version 9 2023-11-28T11:21:28.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.223+0000 7fc91f041b80 1 -- 172.21.15.92:0/2294824524 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x55ae875df1d0 con 0x55ae87874ac0 2023-11-28T11:21:28.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.223+0000 7fc8e3fff700 1 -- 172.21.15.92:0/2294824524 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc900014450 con 0x55ae87874ac0 2023-11-28T11:21:28.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.223+0000 7fc8e3fff700 1 -- 172.21.15.92:0/2294824524 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 9) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fc900035070 con 0x55ae87874ac0 2023-11-28T11:21:28.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.223+0000 7fc91f041b80 10 client.4803 fetch_fsmap finished waiting for FSMap version 9 2023-11-28T11:21:28.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.223+0000 7fc91f041b80 10 client.4803 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-28T11:21:28.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.223+0000 7fc91f041b80 20 client.4803 populate_metadata read hostname 'smithi092' 2023-11-28T11:21:28.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.223+0000 7fc91f041b80 1 --2- 172.21.15.92:0/2294824524 >> [v2:172.21.15.169:6833/2067251916,v1:172.21.15.169:6835/2067251916] conn(0x55ae87837820 0x55ae87866140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:28.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.223+0000 7fc91f041b80 4 client.4803 mds_command: new command op to 4326 tid=0[{"prefix": "status"}] 2023-11-28T11:21:28.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.223+0000 7fc91f041b80 1 -- 172.21.15.92:0/2294824524 --> [v2:172.21.15.169:6833/2067251916,v1:172.21.15.169:6835/2067251916] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ae8760f040 con 0x55ae87837820 2023-11-28T11:21:28.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.224+0000 7fc908d9a700 1 --2- 172.21.15.92:0/2294824524 >> [v2:172.21.15.169:6833/2067251916,v1:172.21.15.169:6835/2067251916] conn(0x55ae87837820 0x55ae87866140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:28.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.224+0000 7fc8e3fff700 10 client.4803 ms_handle_connect on v2:172.21.15.169:6833/2067251916 2023-11-28T11:21:28.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.224+0000 7fc908d9a700 1 --2- 172.21.15.92:0/2294824524 >> [v2:172.21.15.169:6833/2067251916,v1:172.21.15.169:6835/2067251916] conn(0x55ae87837820 0x55ae87866140 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6746c2f7903ab439 server_cookie=3210c4e48d9a8e34 in_seq=0 out_seq=0 2023-11-28T11:21:28.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.224+0000 7fc8e3fff700 1 -- 172.21.15.92:0/2294824524 <== mds.0 v2:172.21.15.169:6833/2067251916 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x55ae8760f040 con 0x55ae87837820 2023-11-28T11:21:28.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.224+0000 7fc8e3fff700 10 client.4803 handle_command_reply: tid=0 2023-11-28T11:21:28.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.224+0000 7fc91f041b80 1 client.4803 shutdown 2023-11-28T11:21:28.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.225+0000 7fc91f041b80 1 -- 172.21.15.92:0/2294824524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ae87874ac0 msgr2=0x55ae878e27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:28.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.225+0000 7fc91f041b80 1 --2- 172.21.15.92:0/2294824524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ae87874ac0 0x55ae878e27a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc900001030 tx=0x7fc90000a000 comp rx=0 tx=0).stop 2023-11-28T11:21:28.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.225+0000 7fc91f041b80 1 -- 172.21.15.92:0/2294824524 shutdown_connections 2023-11-28T11:21:28.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.225+0000 7fc91f041b80 1 --2- 172.21.15.92:0/2294824524 >> [v2:172.21.15.169:6833/2067251916,v1:172.21.15.169:6835/2067251916] conn(0x55ae87837820 0x55ae87866140 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:28.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.225+0000 7fc91f041b80 1 --2- 172.21.15.92:0/2294824524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ae8787df50 0x55ae87865c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:28.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.225+0000 7fc91f041b80 1 --2- 172.21.15.92:0/2294824524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ae87874ac0 0x55ae878e27a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:28.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.225+0000 7fc91f041b80 1 --2- 172.21.15.92:0/2294824524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ae8787d990 0x55ae87873840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:28.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.225+0000 7fc8e3fff700 0 client.4803 ms_handle_reset on v2:172.21.15.169:6833/2067251916 2023-11-28T11:21:28.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.225+0000 7fc91f041b80 1 -- 172.21.15.92:0/2294824524 >> 172.21.15.92:0/2294824524 conn(0x55ae8782dd20 msgr2=0x55ae87838d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:28.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.225+0000 7fc91f041b80 1 -- 172.21.15.92:0/2294824524 shutdown_connections 2023-11-28T11:21:28.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.225+0000 7fc91f041b80 1 -- 172.21.15.92:0/2294824524 wait complete. 2023-11-28T11:21:28.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.225+0000 7fc91f041b80 20 client.4803 trim_cache size 0 max 16384 2023-11-28T11:21:28.230 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-28T11:21:28.231 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "68956e0c-c16f-47cd-a220-f98364b0831d", 2023-11-28T11:21:28.231 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 0, 2023-11-28T11:21:28.231 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 4326, 2023-11-28T11:21:28.231 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-11-28T11:21:28.231 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-11-28T11:21:28.231 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-11-28T11:21:28.231 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 31.987202799999999, 2023-11-28T11:21:28.231 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 9, 2023-11-28T11:21:28.231 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 27, 2023-11-28T11:21:28.231 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 27, 2023-11-28T11:21:28.231 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 37.506097115000003 2023-11-28T11:21:28.231 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-28T11:21:28.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.225+0000 7fc90b7ff700 1 -- 172.21.15.92:0/3910479371 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc8e80682f0 msgr2=0x7fc8e806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:28.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.225+0000 7fc90b7ff700 1 --2- 172.21.15.92:0/3910479371 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc8e80682f0 0x7fc8e806a7a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc900001220 tx=0x7fc900009040 comp rx=0 tx=0).stop 2023-11-28T11:21:28.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.225+0000 7fc90b7ff700 1 -- 172.21.15.92:0/3910479371 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9040b4fc0 msgr2=0x7fc9041e52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:28.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.225+0000 7fc90b7ff700 1 --2- 172.21.15.92:0/3910479371 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9040b4fc0 0x7fc9041e52c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc8f400c6b0 tx=0x7fc8f400c9c0 comp rx=0 tx=0).stop 2023-11-28T11:21:28.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.226+0000 7fc90b7ff700 1 -- 172.21.15.92:0/3910479371 shutdown_connections 2023-11-28T11:21:28.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.226+0000 7fc90b7ff700 1 --2- 172.21.15.92:0/3910479371 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc9040d2df0 0x7fc9041e7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:28.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.226+0000 7fc90b7ff700 1 --2- 172.21.15.92:0/3910479371 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc9040b1c60 0x7fc90415a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:28.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.226+0000 7fc90b7ff700 1 --2- 172.21.15.92:0/3910479371 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc8e80682f0 0x7fc8e806a7a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:28.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.226+0000 7fc90b7ff700 1 --2- 172.21.15.92:0/3910479371 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9040b4fc0 0x7fc9041e52c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:28.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.226+0000 7fc90b7ff700 1 -- 172.21.15.92:0/3910479371 >> 172.21.15.92:0/3910479371 conn(0x7fc9040719a0 msgr2=0x7fc9040d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:28.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.226+0000 7fc90b7ff700 1 -- 172.21.15.92:0/3910479371 shutdown_connections 2023-11-28T11:21:28.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:28.226+0000 7fc90b7ff700 1 -- 172.21.15.92:0/3910479371 wait complete. 2023-11-28T11:21:28.244 INFO:teuthology.run_tasks:Running task kclient... 2023-11-28T11:21:28.257 INFO:tasks.kclient:Mounting kernel clients... 2023-11-28T11:21:28.258 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-11-28T11:21:28.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:28.258 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-11-28T11:21:28.306 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-11-28T11:21:28.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:28.307 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-28T11:21:28.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:28.379 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link delete ceph-brx 2023-11-28T11:21:28.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:28.483 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-11-28T11:21:28.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:28.533 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link delete ceph-brx 2023-11-28T11:21:28.606 INFO:teuthology.orchestra.run.smithi092.stderr:Cannot find device "ceph-brx" 2023-11-28T11:21:28.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:21:28.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:28.609 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2023-11-28T11:21:28.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:28.662 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link delete ceph-brx 2023-11-28T11:21:28.739 INFO:teuthology.orchestra.run.smithi169.stderr:Cannot find device "ceph-brx" 2023-11-28T11:21:28.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:21:28.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:28.742 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2023-11-28T11:21:28.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:28.790 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link delete ceph-brx 2023-11-28T11:21:28.863 INFO:teuthology.orchestra.run.smithi169.stderr:Cannot find device "ceph-brx" 2023-11-28T11:21:28.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:21:28.866 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2023-11-28T11:21:28.867 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-28T11:21:28.867 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T11:21:28.867 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T11:21:29.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.062+0000 7ff6fc7ff700 1 Processor -- start 2023-11-28T11:21:29.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.062+0000 7ff6fc7ff700 1 -- start start 2023-11-28T11:21:29.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.063+0000 7ff6fc7ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff6f40cd460 0x7ff6f406c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:29.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.063+0000 7ff6fc7ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6f40b88e0 con 0x7ff6f40b8510 2023-11-28T11:21:29.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.063+0000 7ff6fc7ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6f40cd8f0 con 0x7ff6f40d0d60 2023-11-28T11:21:29.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.063+0000 7ff6fc7ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f40cdbd0 con 0x7ff6f40cd460 2023-11-28T11:21:29.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.063+0000 7ff6fa59b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff6f40b8510 0x7ff6f406b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59822/0 (socket says 172.21.15.92:59822) 2023-11-28T11:21:29.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.063+0000 7ff6fa59b700 1 -- 172.21.15.92:0/1359459191 learned_addr learned my addr 172.21.15.92:0/1359459191 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:29.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.064+0000 7ff6f9d9a700 1 --2- 172.21.15.92:0/1359459191 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff6f40cd460 0x7ff6f406c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:29.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.064+0000 7ff6f9599700 1 -- 172.21.15.92:0/1359459191 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2476170687 0 0) 0x7ff6f40b88e0 con 0x7ff6f40b8510 2023-11-28T11:21:29.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.064+0000 7ff6f9599700 1 -- 172.21.15.92:0/1359459191 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6e00035b0 con 0x7ff6f40b8510 2023-11-28T11:21:29.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.064+0000 7ff6f9599700 1 -- 172.21.15.92:0/1359459191 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3831691097 0 0) 0x7ff6f40cd8f0 con 0x7ff6f40d0d60 2023-11-28T11:21:29.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.064+0000 7ff6f9599700 1 -- 172.21.15.92:0/1359459191 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6f40b88e0 con 0x7ff6f40d0d60 2023-11-28T11:21:29.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.064+0000 7ff6f9599700 1 -- 172.21.15.92:0/1359459191 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 667974472 0 0) 0x7ff6e00035b0 con 0x7ff6f40b8510 2023-11-28T11:21:29.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.064+0000 7ff6f9d9a700 1 -- 172.21.15.92:0/1359459191 >> v1:172.21.15.169:6789/0 conn(0x7ff6f40d0d60 legacy=0x7ff6f40d11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:29.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.064+0000 7ff6f9d9a700 1 -- 172.21.15.92:0/1359459191 >> v1:172.21.15.92:6789/0 conn(0x7ff6f40b8510 legacy=0x7ff6f406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:29.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.064+0000 7ff6f9d9a700 1 -- 172.21.15.92:0/1359459191 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f40cfd10 con 0x7ff6f40cd460 2023-11-28T11:21:29.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.064+0000 7ff6f9d9a700 1 --2- 172.21.15.92:0/1359459191 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff6f40cd460 0x7ff6f406c080 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4009870 tx=0x7ff6e4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=824b7e54cb7a131e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:29.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.065+0000 7ff6f9599700 1 -- 172.21.15.92:0/1359459191 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f00019f0 con 0x7ff6f40cd460 2023-11-28T11:21:29.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.065+0000 7ff6f9599700 1 -- 172.21.15.92:0/1359459191 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e4004030 con 0x7ff6f40cd460 2023-11-28T11:21:29.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.065+0000 7ff6f9599700 1 -- 172.21.15.92:0/1359459191 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e40043c0 con 0x7ff6f40cd460 2023-11-28T11:21:29.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.065+0000 7ff6fc7ff700 1 -- 172.21.15.92:0/1359459191 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff6f40cd460 msgr2=0x7ff6f406c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:29.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.065+0000 7ff6fc7ff700 1 --2- 172.21.15.92:0/1359459191 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff6f40cd460 0x7ff6f406c080 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4009870 tx=0x7ff6e4009b80 comp rx=0 tx=0).stop 2023-11-28T11:21:29.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.065+0000 7ff6fc7ff700 1 -- 172.21.15.92:0/1359459191 shutdown_connections 2023-11-28T11:21:29.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.065+0000 7ff6fc7ff700 1 --2- 172.21.15.92:0/1359459191 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff6f40cd460 0x7ff6f406c080 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.065+0000 7ff6fc7ff700 1 -- 172.21.15.92:0/1359459191 >> 172.21.15.92:0/1359459191 conn(0x7ff6f40719a0 msgr2=0x7ff6f4071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:29.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.065+0000 7ff6fc7ff700 1 -- 172.21.15.92:0/1359459191 shutdown_connections 2023-11-28T11:21:29.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.065+0000 7ff6fc7ff700 1 -- 172.21.15.92:0/1359459191 wait complete. 2023-11-28T11:21:29.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.067+0000 7ff6fc7ff700 1 Processor -- start 2023-11-28T11:21:29.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.067+0000 7ff6fc7ff700 1 -- start start 2023-11-28T11:21:29.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.068+0000 7ff6fc7ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6f40b8510 0x7ff6f4158900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:29.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.068+0000 7ff6fc7ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff6f40cd460 0x7ff6f41f1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:29.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.068+0000 7ff6fc7ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6f40d0d60 0x7ff6f415a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:29.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.068+0000 7ff6fc7ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f4159150 con 0x7ff6f40d0d60 2023-11-28T11:21:29.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.068+0000 7ff6fc7ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f4159430 con 0x7ff6f40b8510 2023-11-28T11:21:29.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.068+0000 7ff6fc7ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f4159710 con 0x7ff6f40cd460 2023-11-28T11:21:29.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.068+0000 7ff6fad9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6f40d0d60 0x7ff6f415a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:29.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.068+0000 7ff6fad9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6f40d0d60 0x7ff6f415a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57080/0 (socket says 172.21.15.92:57080) 2023-11-28T11:21:29.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.068+0000 7ff6fad9c700 1 -- 172.21.15.92:0/3157876847 learned_addr learned my addr 172.21.15.92:0/3157876847 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:29.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.068+0000 7ff6fa59b700 1 --2- 172.21.15.92:0/3157876847 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6f40b8510 0x7ff6f4158900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:29.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.068+0000 7ff6f9d9a700 1 --2- 172.21.15.92:0/3157876847 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff6f40cd460 0x7ff6f41f1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:29.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.069+0000 7ff6fad9c700 1 -- 172.21.15.92:0/3157876847 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff6f40cd460 msgr2=0x7ff6f41f1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:29.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.069+0000 7ff6fad9c700 1 --2- 172.21.15.92:0/3157876847 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff6f40cd460 0x7ff6f41f1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.069+0000 7ff6fad9c700 1 -- 172.21.15.92:0/3157876847 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6f40b8510 msgr2=0x7ff6f4158900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:29.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.069+0000 7ff6fad9c700 1 --2- 172.21.15.92:0/3157876847 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6f40b8510 0x7ff6f4158900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.069+0000 7ff6fad9c700 1 -- 172.21.15.92:0/3157876847 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e40094a0 con 0x7ff6f40d0d60 2023-11-28T11:21:29.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.069+0000 7ff6fad9c700 1 --2- 172.21.15.92:0/3157876847 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6f40d0d60 0x7ff6f415a1f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec00c430 tx=0x7ff6ec00c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:29.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.069+0000 7ff6eaffd700 1 -- 172.21.15.92:0/3157876847 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6ec0050d0 con 0x7ff6f40d0d60 2023-11-28T11:21:29.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.069+0000 7ff6eaffd700 1 -- 172.21.15.92:0/3157876847 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6ec005250 con 0x7ff6f40d0d60 2023-11-28T11:21:29.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.069+0000 7ff6eaffd700 1 -- 172.21.15.92:0/3157876847 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6ec003680 con 0x7ff6f40d0d60 2023-11-28T11:21:29.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.069+0000 7ff6fc7ff700 1 -- 172.21.15.92:0/3157876847 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6f40d29c0 con 0x7ff6f40d0d60 2023-11-28T11:21:29.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.070+0000 7ff6fc7ff700 1 -- 172.21.15.92:0/3157876847 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6f40cfff0 con 0x7ff6f40d0d60 2023-11-28T11:21:29.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.070+0000 7ff6fc7ff700 1 -- 172.21.15.92:0/3157876847 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6f40d2b50 con 0x7ff6f40d0d60 2023-11-28T11:21:29.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.071+0000 7ff6eaffd700 1 -- 172.21.15.92:0/3157876847 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff6ec019030 con 0x7ff6f40d0d60 2023-11-28T11:21:29.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.072+0000 7ff6eaffd700 1 --2- 172.21.15.92:0/3157876847 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff6d80682f0 0x7ff6d806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:29.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.072+0000 7ff6eaffd700 1 -- 172.21.15.92:0/3157876847 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff6ec085230 con 0x7ff6f40d0d60 2023-11-28T11:21:29.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.072+0000 7ff6fa59b700 1 --2- 172.21.15.92:0/3157876847 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff6d80682f0 0x7ff6d806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:29.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.073+0000 7ff6fa59b700 1 --2- 172.21.15.92:0/3157876847 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff6d80682f0 0x7ff6d806a7a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff6f00012e0 tx=0x7ff6f0000f40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:29.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.077+0000 7ff6eaffd700 1 -- 172.21.15.92:0/3157876847 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff6ec0516b0 con 0x7ff6f40d0d60 2023-11-28T11:21:29.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.313+0000 7ff6fc7ff700 1 -- 172.21.15.92:0/3157876847 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff6f4067880 con 0x7ff6f40d0d60 2023-11-28T11:21:29.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.313+0000 7ff6eaffd700 1 -- 172.21.15.92:0/3157876847 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7ff6ec054cd0 con 0x7ff6f40d0d60 2023-11-28T11:21:29.314 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-28T11:21:29.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.315+0000 7ff6fc7ff700 1 -- 172.21.15.92:0/3157876847 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff6d80682f0 msgr2=0x7ff6d806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:29.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.315+0000 7ff6fc7ff700 1 --2- 172.21.15.92:0/3157876847 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff6d80682f0 0x7ff6d806a7a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff6f00012e0 tx=0x7ff6f0000f40 comp rx=0 tx=0).stop 2023-11-28T11:21:29.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.315+0000 7ff6fc7ff700 1 -- 172.21.15.92:0/3157876847 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6f40d0d60 msgr2=0x7ff6f415a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:29.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.315+0000 7ff6fc7ff700 1 --2- 172.21.15.92:0/3157876847 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6f40d0d60 0x7ff6f415a1f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec00c430 tx=0x7ff6ec00c740 comp rx=0 tx=0).stop 2023-11-28T11:21:29.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.315+0000 7ff6fc7ff700 1 -- 172.21.15.92:0/3157876847 shutdown_connections 2023-11-28T11:21:29.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.315+0000 7ff6fc7ff700 1 --2- 172.21.15.92:0/3157876847 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff6d80682f0 0x7ff6d806a7a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.315+0000 7ff6fc7ff700 1 --2- 172.21.15.92:0/3157876847 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6f40d0d60 0x7ff6f415a1f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.315+0000 7ff6fc7ff700 1 --2- 172.21.15.92:0/3157876847 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6f40b8510 0x7ff6f4158900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.315+0000 7ff6fc7ff700 1 --2- 172.21.15.92:0/3157876847 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff6f40cd460 0x7ff6f41f1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:29.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.315+0000 7ff6fc7ff700 1 -- 172.21.15.92:0/3157876847 >> 172.21.15.92:0/3157876847 conn(0x7ff6f40719a0 msgr2=0x7ff6f415d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:29.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.316+0000 7ff6fc7ff700 1 -- 172.21.15.92:0/3157876847 shutdown_connections 2023-11-28T11:21:29.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:29.316+0000 7ff6fc7ff700 1 -- 172.21.15.92:0/3157876847 wait complete. 2023-11-28T11:21:29.334 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T11:21:29.334 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T11:21:29.334 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi092.front.sepia.ceph.com 2023-11-28T11:21:29.335 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T11:21:29.335 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T11:21:29.335 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T11:21:29.335 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T11:21:29.335 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T11:21:29.335 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:21:29.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:29.335 DEBUG:teuthology.orchestra.run.smithi092:> ip addr 2023-11-28T11:21:29.383 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:21:29.383 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:21:29.383 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:21:29.383 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:21:29.383 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-11-28T11:21:29.383 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:21:29.384 INFO:teuthology.orchestra.run.smithi092.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:21:29.384 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b2 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:21:29.384 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f0 2023-11-28T11:21:29.384 INFO:teuthology.orchestra.run.smithi092.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:21:29.384 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b3 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:21:29.384 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f1 2023-11-28T11:21:29.384 INFO:teuthology.orchestra.run.smithi092.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:21:29.384 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:44 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:21:29.384 INFO:teuthology.orchestra.run.smithi092.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T11:21:29.384 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:45 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:21:29.384 INFO:teuthology.orchestra.run.smithi092.stdout: inet 172.21.15.92/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T11:21:29.384 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft 41965sec preferred_lft 41965sec 2023-11-28T11:21:29.384 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::ec4:7aff:fe8f:cd45/64 scope link 2023-11-28T11:21:29.384 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:21:29.385 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-28T11:21:29.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:29.385 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-11-28T11:21:29.385 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-11-28T11:21:29.385 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link add name ceph-brx type bridge 2023-11-28T11:21:29.385 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip addr flush dev ceph-brx 2023-11-28T11:21:29.385 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set ceph-brx up 2023-11-28T11:21:29.386 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-28T11:21:29.386 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-11-28T11:21:29.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:29.619 DEBUG:teuthology.orchestra.run.smithi092:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T11:21:29.690 INFO:teuthology.orchestra.run.smithi092.stdout:1 2023-11-28T11:21:29.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:29.693 DEBUG:teuthology.orchestra.run.smithi092:> route 2023-11-28T11:21:29.742 INFO:teuthology.orchestra.run.smithi092.stdout:Kernel IP routing table 2023-11-28T11:21:29.742 INFO:teuthology.orchestra.run.smithi092.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T11:21:29.742 INFO:teuthology.orchestra.run.smithi092.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T11:21:29.743 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T11:21:29.743 INFO:teuthology.orchestra.run.smithi092.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T11:21:29.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:29.743 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-11-28T11:21:29.743 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-11-28T11:21:29.743 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T11:21:29.743 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T11:21:29.743 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T11:21:29.743 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-11-28T11:21:29.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:29.915 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-11-28T11:21:29.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:29.963 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list-id 2023-11-28T11:21:30.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:30.013 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-11-28T11:21:30.013 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-11-28T11:21:30.013 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T11:21:30.013 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-11-28T11:21:30.013 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-11-28T11:21:30.156 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-11-28T11:21:30.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:30.157 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-11-28T11:21:30.157 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-11-28T11:21:30.157 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-11-28T11:21:30.157 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-28T11:21:30.157 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-11-28T11:21:30.157 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-11-28T11:21:30.157 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-11-28T11:21:30.157 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-11-28T11:21:30.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:30.408 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-11-28T11:21:30.408 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-11-28T11:21:30.408 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set brx.0 up 2023-11-28T11:21:30.408 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set dev brx.0 master ceph-brx 2023-11-28T11:21:30.408 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-11-28T11:21:30.579 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:21:30.579 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-28T11:21:30.625 INFO:teuthology.orchestra.run.smithi092.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:21:30.625 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:21:30.626 DEBUG:teuthology.orchestra.run.smithi092:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-28T11:21:30.671 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-11-28T11:21:30.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:30.672 DEBUG:teuthology.orchestra.run.smithi092:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:21:30.817 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:21:30.817 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T11:21:30.817 INFO:teuthology.orchestra.run.smithi092.stdout:invalid new device string format 2023-11-28T11:21:31.246 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:21:31.247 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T11:21:31.247 INFO:teuthology.orchestra.run.smithi092.stdout:invalid new device string format 2023-11-28T11:21:31.247 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.169:3300,172.21.15.169:3301" 2023-11-28T11:21:31.247 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:21:31.272 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:21:31.273 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T11:21:31.273 INFO:teuthology.orchestra.run.smithi092.stdout:invalid new device string format 2023-11-28T11:21:31.273 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.169:3300,172.21.15.169:3301" 2023-11-28T11:21:31.273 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:21:31.273 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:6789,172.21.15.169:6789,172.21.15.169:6790" 2023-11-28T11:21:31.273 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.92:6789,172.21.15.169:6789,172.21.15.169:6790:/ 2023-11-28T11:21:31.273 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=68956e0c-c16f-47cd-a220-f98364b0831d" will pass to kernel 2023-11-28T11:21:31.279 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T11:21:31.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:31.279 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T11:21:31.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:31.349 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:21:31.437 INFO:teuthology.orchestra.run.smithi092.stdout:client4637 2023-11-28T11:21:31.437 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-28T11:21:31.437 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:21:31.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:31.483 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:21:31.612 INFO:teuthology.orchestra.run.smithi092.stdout:client4637 2023-11-28T11:21:31.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:31.612 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:21:31.700 INFO:teuthology.orchestra.run.smithi092.stdout:client4637 2023-11-28T11:21:31.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:31.700 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/68956e0c-c16f-47cd-a220-f98364b0831d.client4637/status') 2023-11-28T11:21:31.809 INFO:teuthology.orchestra.run.smithi092.stdout:instance: client.4637 (3)172.21.15.92:0/812841678 2023-11-28T11:21:31.809 INFO:teuthology.orchestra.run.smithi092.stdout:blocklisted: false 2023-11-28T11:21:31.809 INFO:teuthology.orchestra.run.smithi092.stderr:0+1 records in 2023-11-28T11:21:31.810 INFO:teuthology.orchestra.run.smithi092.stderr:0+1 records out 2023-11-28T11:21:31.810 INFO:teuthology.orchestra.run.smithi092.stderr:69 bytes copied, 5.2686e-05 s, 1.3 MB/s 2023-11-28T11:21:31.812 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2023-11-28T11:21:31.812 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-28T11:21:31.812 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-28T11:21:31.813 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T11:21:32.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.009+0000 7f01f7dff700 1 Processor -- start 2023-11-28T11:21:32.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.009+0000 7f01f7dff700 1 -- start start 2023-11-28T11:21:32.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.011+0000 7f01f7dff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f01f00cd460 0x7f01f006c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:32.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.011+0000 7f01f7dff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01f00b88e0 con 0x7f01f00b8510 2023-11-28T11:21:32.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.011+0000 7f01f7dff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01f00cd8f0 con 0x7f01f00d0d60 2023-11-28T11:21:32.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.011+0000 7f01f7dff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01f00cdbd0 con 0x7f01f00cd460 2023-11-28T11:21:32.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.011+0000 7f01f5b9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f01f00b8510 0x7f01f006b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59836/0 (socket says 172.21.15.92:59836) 2023-11-28T11:21:32.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.011+0000 7f01f5b9b700 1 -- 172.21.15.92:0/4058405057 learned_addr learned my addr 172.21.15.92:0/4058405057 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:32.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.011+0000 7f01f539a700 1 --2- 172.21.15.92:0/4058405057 >> v2:172.21.15.92:3300/0 conn(0x7f01f00cd460 0x7f01f006c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:32.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.011+0000 7f01f539a700 1 -- 172.21.15.92:0/4058405057 >> v1:172.21.15.169:6789/0 conn(0x7f01f00d0d60 legacy=0x7f01f00d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:32.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.012+0000 7f01f539a700 1 -- 172.21.15.92:0/4058405057 >> v1:172.21.15.92:6789/0 conn(0x7f01f00b8510 legacy=0x7f01f006b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:32.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.012+0000 7f01f539a700 1 -- 172.21.15.92:0/4058405057 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f00cfd10 con 0x7f01f00cd460 2023-11-28T11:21:32.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.012+0000 7f01f539a700 1 --2- 172.21.15.92:0/4058405057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01f00cd460 0x7f01f006c080 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f01e0009840 tx=0x7f01e0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b18654998e21c428 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:32.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.012+0000 7f01f4b99700 1 -- 172.21.15.92:0/4058405057 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e0011830 con 0x7f01f00cd460 2023-11-28T11:21:32.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.012+0000 7f01f4b99700 1 -- 172.21.15.92:0/4058405057 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e0011990 con 0x7f01f00cd460 2023-11-28T11:21:32.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.012+0000 7f01f4b99700 1 -- 172.21.15.92:0/4058405057 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e0011d20 con 0x7f01f00cd460 2023-11-28T11:21:32.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.013+0000 7f01f7dff700 1 -- 172.21.15.92:0/4058405057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01f00cd460 msgr2=0x7f01f006c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:32.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.013+0000 7f01f7dff700 1 --2- 172.21.15.92:0/4058405057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01f00cd460 0x7f01f006c080 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f01e0009840 tx=0x7f01e0009b50 comp rx=0 tx=0).stop 2023-11-28T11:21:32.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.013+0000 7f01f7dff700 1 -- 172.21.15.92:0/4058405057 shutdown_connections 2023-11-28T11:21:32.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.013+0000 7f01f7dff700 1 --2- 172.21.15.92:0/4058405057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01f00cd460 0x7f01f006c080 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:32.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.013+0000 7f01f7dff700 1 -- 172.21.15.92:0/4058405057 >> 172.21.15.92:0/4058405057 conn(0x7f01f00719a0 msgr2=0x7f01f0071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:32.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.013+0000 7f01f7dff700 1 -- 172.21.15.92:0/4058405057 shutdown_connections 2023-11-28T11:21:32.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.013+0000 7f01f7dff700 1 -- 172.21.15.92:0/4058405057 wait complete. 2023-11-28T11:21:32.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.014+0000 7f01f7dff700 1 Processor -- start 2023-11-28T11:21:32.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.015+0000 7f01f7dff700 1 -- start start 2023-11-28T11:21:32.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.015+0000 7f01f7dff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01f00b8510 0x7f01f01e55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:32.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.015+0000 7f01f7dff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01f00cd460 0x7f01f01e6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:32.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.015+0000 7f01f7dff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01f00d0d60 0x7f01f01f3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:32.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.015+0000 7f01f7dff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f01e6880 con 0x7f01f00b8510 2023-11-28T11:21:32.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.015+0000 7f01f7dff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f01e5c00 con 0x7f01f00d0d60 2023-11-28T11:21:32.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.015+0000 7f01f7dff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01f01e5ee0 con 0x7f01f00cd460 2023-11-28T11:21:32.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.016+0000 7f01f5b9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01f00b8510 0x7f01f01e55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:32.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.016+0000 7f01f5b9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01f00b8510 0x7f01f01e55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57096/0 (socket says 172.21.15.92:57096) 2023-11-28T11:21:32.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.016+0000 7f01f5b9b700 1 -- 172.21.15.92:0/1410955918 learned_addr learned my addr 172.21.15.92:0/1410955918 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:32.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.016+0000 7f01f539a700 1 --2- 172.21.15.92:0/1410955918 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01f00cd460 0x7f01f01e6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:32.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.016+0000 7f01f639c700 1 --2- 172.21.15.92:0/1410955918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01f00d0d60 0x7f01f01f3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:32.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.016+0000 7f01f5b9b700 1 -- 172.21.15.92:0/1410955918 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01f00cd460 msgr2=0x7f01f01e6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:32.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.016+0000 7f01f5b9b700 1 --2- 172.21.15.92:0/1410955918 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01f00cd460 0x7f01f01e6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:32.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.016+0000 7f01f5b9b700 1 -- 172.21.15.92:0/1410955918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01f00d0d60 msgr2=0x7f01f01f3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:32.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.016+0000 7f01f5b9b700 1 --2- 172.21.15.92:0/1410955918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01f00d0d60 0x7f01f01f3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:32.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.016+0000 7f01f5b9b700 1 -- 172.21.15.92:0/1410955918 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e00094a0 con 0x7f01f00b8510 2023-11-28T11:21:32.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.016+0000 7f01f5b9b700 1 --2- 172.21.15.92:0/1410955918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01f00b8510 0x7f01f01e55c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f01ec00c520 tx=0x7f01ec00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:32.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.017+0000 7f01e67fc700 1 -- 172.21.15.92:0/1410955918 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ec005d70 con 0x7f01f00b8510 2023-11-28T11:21:32.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.017+0000 7f01e67fc700 1 -- 172.21.15.92:0/1410955918 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01ec0070d0 con 0x7f01f00b8510 2023-11-28T11:21:32.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.017+0000 7f01e67fc700 1 -- 172.21.15.92:0/1410955918 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ec007440 con 0x7f01f00b8510 2023-11-28T11:21:32.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.017+0000 7f01f7dff700 1 -- 172.21.15.92:0/1410955918 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01f00d29c0 con 0x7f01f00b8510 2023-11-28T11:21:32.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.017+0000 7f01f7dff700 1 -- 172.21.15.92:0/1410955918 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01f00cfff0 con 0x7f01f00b8510 2023-11-28T11:21:32.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.018+0000 7f01f7dff700 1 -- 172.21.15.92:0/1410955918 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01f00d2b50 con 0x7f01f00b8510 2023-11-28T11:21:32.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.019+0000 7f01e67fc700 1 -- 172.21.15.92:0/1410955918 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f01ec019030 con 0x7f01f00b8510 2023-11-28T11:21:32.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.019+0000 7f01e67fc700 1 --2- 172.21.15.92:0/1410955918 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f01d40682f0 0x7f01d406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:32.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.019+0000 7f01e67fc700 1 -- 172.21.15.92:0/1410955918 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f01ec0851e0 con 0x7f01f00b8510 2023-11-28T11:21:32.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.020+0000 7f01f539a700 1 --2- 172.21.15.92:0/1410955918 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f01d40682f0 0x7f01d406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:32.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.020+0000 7f01f539a700 1 --2- 172.21.15.92:0/1410955918 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f01d40682f0 0x7f01d406a7a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f01f00b1de0 tx=0x7f01e0008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:32.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.023+0000 7f01e67fc700 1 -- 172.21.15.92:0/1410955918 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f01ec04b5e0 con 0x7f01f00b8510 2023-11-28T11:21:32.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.257+0000 7f01f7dff700 1 -- 172.21.15.92:0/1410955918 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f01f0067880 con 0x7f01f00b8510 2023-11-28T11:21:32.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.258+0000 7f01e67fc700 1 -- 172.21.15.92:0/1410955918 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f01ec017070 con 0x7f01f00b8510 2023-11-28T11:21:32.259 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-28T11:21:32.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.260+0000 7f01f7dff700 1 -- 172.21.15.92:0/1410955918 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f01d40682f0 msgr2=0x7f01d406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:32.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.260+0000 7f01f7dff700 1 --2- 172.21.15.92:0/1410955918 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f01d40682f0 0x7f01d406a7a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f01f00b1de0 tx=0x7f01e0008030 comp rx=0 tx=0).stop 2023-11-28T11:21:32.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.260+0000 7f01f7dff700 1 -- 172.21.15.92:0/1410955918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01f00b8510 msgr2=0x7f01f01e55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:32.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.260+0000 7f01f7dff700 1 --2- 172.21.15.92:0/1410955918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01f00b8510 0x7f01f01e55c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f01ec00c520 tx=0x7f01ec00c830 comp rx=0 tx=0).stop 2023-11-28T11:21:32.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.260+0000 7f01f7dff700 1 -- 172.21.15.92:0/1410955918 shutdown_connections 2023-11-28T11:21:32.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.260+0000 7f01f7dff700 1 --2- 172.21.15.92:0/1410955918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f01f00d0d60 0x7f01f01f3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:32.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.260+0000 7f01f7dff700 1 --2- 172.21.15.92:0/1410955918 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f01d40682f0 0x7f01d406a7a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:32.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.260+0000 7f01f7dff700 1 --2- 172.21.15.92:0/1410955918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01f00b8510 0x7f01f01e55c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:32.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.260+0000 7f01f7dff700 1 --2- 172.21.15.92:0/1410955918 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f01f00cd460 0x7f01f01e6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:32.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.260+0000 7f01f7dff700 1 -- 172.21.15.92:0/1410955918 >> 172.21.15.92:0/1410955918 conn(0x7f01f00719a0 msgr2=0x7f01f00b29f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:32.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.260+0000 7f01f7dff700 1 -- 172.21.15.92:0/1410955918 shutdown_connections 2023-11-28T11:21:32.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:32.260+0000 7f01f7dff700 1 -- 172.21.15.92:0/1410955918 wait complete. 2023-11-28T11:21:32.279 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T11:21:32.279 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T11:21:32.279 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi092.front.sepia.ceph.com 2023-11-28T11:21:32.279 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-11-28T11:21:32.279 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-28T11:21:32.279 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T11:21:32.279 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T11:21:32.279 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T11:21:32.279 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-11-28T11:21:32.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:32.280 DEBUG:teuthology.orchestra.run.smithi092:> ip addr 2023-11-28T11:21:32.329 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:21:32.330 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:21:32.330 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:21:32.330 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:21:32.330 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-11-28T11:21:32.330 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:21:32.330 INFO:teuthology.orchestra.run.smithi092.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:21:32.330 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b2 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:21:32.330 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f0 2023-11-28T11:21:32.330 INFO:teuthology.orchestra.run.smithi092.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:21:32.330 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b3 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:21:32.331 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f1 2023-11-28T11:21:32.331 INFO:teuthology.orchestra.run.smithi092.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:21:32.331 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:44 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:21:32.331 INFO:teuthology.orchestra.run.smithi092.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T11:21:32.331 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:45 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:21:32.331 INFO:teuthology.orchestra.run.smithi092.stdout: inet 172.21.15.92/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T11:21:32.331 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft 41962sec preferred_lft 41962sec 2023-11-28T11:21:32.331 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::ec4:7aff:fe8f:cd45/64 scope link 2023-11-28T11:21:32.331 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:21:32.331 INFO:teuthology.orchestra.run.smithi092.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T11:21:32.331 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 4e:8f:e4:5b:18:ce brd ff:ff:ff:ff:ff:ff 2023-11-28T11:21:32.331 INFO:teuthology.orchestra.run.smithi092.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T11:21:32.331 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:21:32.332 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::3ca6:3fff:feb6:2c13/64 scope link 2023-11-28T11:21:32.332 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:21:32.332 INFO:teuthology.orchestra.run.smithi092.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T11:21:32.332 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 4e:8f:e4:5b:18:ce brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T11:21:32.332 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::4c8f:e4ff:fe5b:18ce/64 scope link 2023-11-28T11:21:32.332 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:21:32.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:32.332 DEBUG:teuthology.orchestra.run.smithi092:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T11:21:32.399 INFO:teuthology.orchestra.run.smithi092.stdout:1 2023-11-28T11:21:32.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:32.403 DEBUG:teuthology.orchestra.run.smithi092:> route 2023-11-28T11:21:32.451 INFO:teuthology.orchestra.run.smithi092.stdout:Kernel IP routing table 2023-11-28T11:21:32.452 INFO:teuthology.orchestra.run.smithi092.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T11:21:32.452 INFO:teuthology.orchestra.run.smithi092.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T11:21:32.452 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T11:21:32.452 INFO:teuthology.orchestra.run.smithi092.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T11:21:32.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:32.453 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-11-28T11:21:32.453 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-11-28T11:21:32.453 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T11:21:32.453 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T11:21:32.453 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T11:21:32.453 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-11-28T11:21:32.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:32.621 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-11-28T11:21:32.667 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T11:21:32.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:32.669 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list-id 2023-11-28T11:21:32.716 INFO:teuthology.orchestra.run.smithi092.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-11-28T11:21:32.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:32.717 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-11-28T11:21:32.717 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-11-28T11:21:32.717 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T11:21:32.717 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-11-28T11:21:32.717 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-11-28T11:21:32.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:32.860 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-28T11:21:32.930 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:21:32.931 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:21:32.931 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:21:32.931 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:21:32.931 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-11-28T11:21:32.931 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:21:32.931 INFO:teuthology.orchestra.run.smithi092.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T11:21:32.931 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether da:0a:03:32:9f:61 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-28T11:21:32.931 INFO:teuthology.orchestra.run.smithi092.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-28T11:21:32.931 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:21:32.931 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::d80a:3ff:fe32:9f61/64 scope link 2023-11-28T11:21:32.932 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:21:32.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:32.934 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-28T11:21:33.005 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:21:33.005 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:21:33.005 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:21:33.005 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:21:33.005 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-11-28T11:21:33.005 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:21:33.006 INFO:teuthology.orchestra.run.smithi092.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T11:21:33.006 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether da:0a:03:32:9f:61 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-28T11:21:33.006 INFO:teuthology.orchestra.run.smithi092.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-28T11:21:33.006 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:21:33.006 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::d80a:3ff:fe32:9f61/64 scope link 2023-11-28T11:21:33.006 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:21:33.008 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-11-28T11:21:33.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:33.008 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-11-28T11:21:33.009 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-11-28T11:21:33.009 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-11-28T11:21:33.009 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-28T11:21:33.009 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-11-28T11:21:33.009 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-11-28T11:21:33.009 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-11-28T11:21:33.009 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-11-28T11:21:33.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:33.255 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-11-28T11:21:33.255 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-11-28T11:21:33.255 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set brx.1 up 2023-11-28T11:21:33.255 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set dev brx.1 master ceph-brx 2023-11-28T11:21:33.255 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-11-28T11:21:33.427 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:21:33.428 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-11-28T11:21:33.474 INFO:teuthology.orchestra.run.smithi092.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-11-28T11:21:33.474 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:21:33.474 DEBUG:teuthology.orchestra.run.smithi092:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-11-28T11:21:33.559 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-11-28T11:21:33.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:33.560 DEBUG:teuthology.orchestra.run.smithi092:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-11-28T11:21:33.705 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-11-28T11:21:33.705 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-11-28T11:21:33.705 INFO:teuthology.orchestra.run.smithi092.stdout:invalid new device string format 2023-11-28T11:21:33.721 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-11-28T11:21:33.721 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-11-28T11:21:33.721 INFO:teuthology.orchestra.run.smithi092.stdout:invalid new device string format 2023-11-28T11:21:33.721 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.169:3300,172.21.15.169:3301" 2023-11-28T11:21:33.721 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:21:33.742 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-11-28T11:21:33.742 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-11-28T11:21:33.742 INFO:teuthology.orchestra.run.smithi092.stdout:invalid new device string format 2023-11-28T11:21:33.743 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.169:3300,172.21.15.169:3301" 2023-11-28T11:21:33.743 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:21:33.743 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:6789,172.21.15.169:6789,172.21.15.169:6790" 2023-11-28T11:21:33.743 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.92:6789,172.21.15.169:6789,172.21.15.169:6790:/ 2023-11-28T11:21:33.743 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=68956e0c-c16f-47cd-a220-f98364b0831d" will pass to kernel 2023-11-28T11:21:33.747 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T11:21:33.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:33.747 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-11-28T11:21:33.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:33.816 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:21:33.906 INFO:teuthology.orchestra.run.smithi092.stdout:client4821 2023-11-28T11:21:33.906 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-28T11:21:33.906 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:21:33.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:33.952 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:21:34.081 INFO:teuthology.orchestra.run.smithi092.stdout:client4821 2023-11-28T11:21:34.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:34.082 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:21:34.170 INFO:teuthology.orchestra.run.smithi092.stdout:client4821 2023-11-28T11:21:34.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:34.171 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/68956e0c-c16f-47cd-a220-f98364b0831d.client4821/status') 2023-11-28T11:21:34.278 INFO:teuthology.orchestra.run.smithi092.stdout:instance: client.4821 (3)192.168.0.2:0/2076871904 2023-11-28T11:21:34.279 INFO:teuthology.orchestra.run.smithi092.stdout:blocklisted: false 2023-11-28T11:21:34.279 INFO:teuthology.orchestra.run.smithi092.stderr:0+1 records in 2023-11-28T11:21:34.279 INFO:teuthology.orchestra.run.smithi092.stderr:0+1 records out 2023-11-28T11:21:34.279 INFO:teuthology.orchestra.run.smithi092.stderr:69 bytes copied, 8.6222e-05 s, 800 kB/s 2023-11-28T11:21:34.281 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2023-11-28T11:21:34.281 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-28T11:21:34.281 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-28T11:21:34.282 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T11:21:34.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.476+0000 7fa6e71ff700 1 Processor -- start 2023-11-28T11:21:34.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.476+0000 7fa6e71ff700 1 -- start start 2023-11-28T11:21:34.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.477+0000 7fa6e71ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa6e00cd460 0x7fa6e006c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:34.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.477+0000 7fa6e71ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa6e00d0d60 0x7fa6e00d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:34.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.477+0000 7fa6e71ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6e00b88e0 con 0x7fa6e00b8510 2023-11-28T11:21:34.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.477+0000 7fa6e71ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6e00cdbd0 con 0x7fa6e00d0d60 2023-11-28T11:21:34.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.477+0000 7fa6e71ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6e00d02d0 con 0x7fa6e00cd460 2023-11-28T11:21:34.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.477+0000 7fa6e579c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa6e00d0d60 0x7fa6e00d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:34.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.477+0000 7fa6e4f9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fa6e00b8510 0x7fa6e006b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59848/0 (socket says 172.21.15.92:59848) 2023-11-28T11:21:34.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.477+0000 7fa6e4f9b700 1 -- 172.21.15.92:0/1057184453 learned_addr learned my addr 172.21.15.92:0/1057184453 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:34.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.477+0000 7fa6dffff700 1 --2- 172.21.15.92:0/1057184453 >> v2:172.21.15.169:3300/0 conn(0x7fa6e00cd460 0x7fa6e006c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:34.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.478+0000 7fa6e579c700 1 -- 172.21.15.92:0/1057184453 >> v2:172.21.15.169:3300/0 conn(0x7fa6e00cd460 msgr2=0x7fa6e006c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:34.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.478+0000 7fa6e579c700 1 --2- 172.21.15.92:0/1057184453 >> v2:172.21.15.169:3300/0 conn(0x7fa6e00cd460 0x7fa6e006c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:34.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.478+0000 7fa6e579c700 1 -- 172.21.15.92:0/1057184453 >> v1:172.21.15.92:6789/0 conn(0x7fa6e00b8510 legacy=0x7fa6e006b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:34.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.478+0000 7fa6e579c700 1 -- 172.21.15.92:0/1057184453 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6e00cd8f0 con 0x7fa6e00d0d60 2023-11-28T11:21:34.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.478+0000 7fa6e579c700 1 --2- 172.21.15.92:0/1057184453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6e00d0d60 0x7fa6e00d1200 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa6e00ceba0 tx=0x7fa6d800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0c9294d6a4a862 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:34.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.478+0000 7fa6df7fe700 1 -- 172.21.15.92:0/1057184453 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d800e070 con 0x7fa6e00d0d60 2023-11-28T11:21:34.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.478+0000 7fa6df7fe700 1 -- 172.21.15.92:0/1057184453 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6d800bae0 con 0x7fa6e00d0d60 2023-11-28T11:21:34.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.478+0000 7fa6df7fe700 1 -- 172.21.15.92:0/1057184453 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d800be90 con 0x7fa6e00d0d60 2023-11-28T11:21:34.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.479+0000 7fa6e71ff700 1 -- 172.21.15.92:0/1057184453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6e00d0d60 msgr2=0x7fa6e00d1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:34.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.479+0000 7fa6e71ff700 1 --2- 172.21.15.92:0/1057184453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6e00d0d60 0x7fa6e00d1200 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa6e00ceba0 tx=0x7fa6d800b330 comp rx=0 tx=0).stop 2023-11-28T11:21:34.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.479+0000 7fa6e71ff700 1 -- 172.21.15.92:0/1057184453 shutdown_connections 2023-11-28T11:21:34.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.479+0000 7fa6e71ff700 1 --2- 172.21.15.92:0/1057184453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6e00d0d60 0x7fa6e00d1200 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:34.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.479+0000 7fa6e71ff700 1 --2- 172.21.15.92:0/1057184453 >> v2:172.21.15.169:3300/0 conn(0x7fa6e00cd460 0x7fa6e006c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:34.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.479+0000 7fa6e71ff700 1 -- 172.21.15.92:0/1057184453 >> 172.21.15.92:0/1057184453 conn(0x7fa6e00719a0 msgr2=0x7fa6e0071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:34.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.479+0000 7fa6e71ff700 1 -- 172.21.15.92:0/1057184453 shutdown_connections 2023-11-28T11:21:34.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.479+0000 7fa6e71ff700 1 -- 172.21.15.92:0/1057184453 wait complete. 2023-11-28T11:21:34.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.481+0000 7fa6e71ff700 1 Processor -- start 2023-11-28T11:21:34.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.481+0000 7fa6e71ff700 1 -- start start 2023-11-28T11:21:34.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.482+0000 7fa6e71ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6e00b8510 0x7fa6e00ccb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:34.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.482+0000 7fa6e71ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6e00cd460 0x7fa6e00aaee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:34.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.482+0000 7fa6e71ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6e00d0d60 0x7fa6e0159160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:34.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.482+0000 7fa6e71ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e00aa850 con 0x7fa6e00b8510 2023-11-28T11:21:34.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.482+0000 7fa6e71ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e00cc6f0 con 0x7fa6e00d0d60 2023-11-28T11:21:34.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.482+0000 7fa6e71ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e00cbdc0 con 0x7fa6e00cd460 2023-11-28T11:21:34.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.482+0000 7fa6e4f9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6e00b8510 0x7fa6e00ccb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:34.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.482+0000 7fa6e4f9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6e00b8510 0x7fa6e00ccb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57110/0 (socket says 172.21.15.92:57110) 2023-11-28T11:21:34.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.482+0000 7fa6e4f9b700 1 -- 172.21.15.92:0/1444379771 learned_addr learned my addr 172.21.15.92:0/1444379771 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:34.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.482+0000 7fa6e579c700 1 --2- 172.21.15.92:0/1444379771 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6e00d0d60 0x7fa6e0159160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:34.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.482+0000 7fa6dffff700 1 --2- 172.21.15.92:0/1444379771 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6e00cd460 0x7fa6e00aaee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:34.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.482+0000 7fa6e4f9b700 1 -- 172.21.15.92:0/1444379771 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6e00cd460 msgr2=0x7fa6e00aaee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:34.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.482+0000 7fa6e4f9b700 1 --2- 172.21.15.92:0/1444379771 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6e00cd460 0x7fa6e00aaee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:34.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.482+0000 7fa6e4f9b700 1 -- 172.21.15.92:0/1444379771 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6e00d0d60 msgr2=0x7fa6e0159160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:34.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.482+0000 7fa6e4f9b700 1 --2- 172.21.15.92:0/1444379771 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6e00d0d60 0x7fa6e0159160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:34.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.482+0000 7fa6e4f9b700 1 -- 172.21.15.92:0/1444379771 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d8009a70 con 0x7fa6e00b8510 2023-11-28T11:21:34.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.482+0000 7fa6dffff700 1 --2- 172.21.15.92:0/1444379771 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6e00cd460 0x7fa6e00aaee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:21:34.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.483+0000 7fa6e4f9b700 1 --2- 172.21.15.92:0/1444379771 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6e00b8510 0x7fa6e00ccb80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa6d400c520 tx=0x7fa6d400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:34.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.483+0000 7fa6dd7fa700 1 -- 172.21.15.92:0/1444379771 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d4005d70 con 0x7fa6e00b8510 2023-11-28T11:21:34.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.483+0000 7fa6dd7fa700 1 -- 172.21.15.92:0/1444379771 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6d40070d0 con 0x7fa6e00b8510 2023-11-28T11:21:34.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.483+0000 7fa6e71ff700 1 -- 172.21.15.92:0/1444379771 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6e00cfff0 con 0x7fa6e00b8510 2023-11-28T11:21:34.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.483+0000 7fa6dd7fa700 1 -- 172.21.15.92:0/1444379771 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d4007440 con 0x7fa6e00b8510 2023-11-28T11:21:34.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.483+0000 7fa6e71ff700 1 -- 172.21.15.92:0/1444379771 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6e00cfd10 con 0x7fa6e00b8510 2023-11-28T11:21:34.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.484+0000 7fa6e71ff700 1 -- 172.21.15.92:0/1444379771 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6e00cc9d0 con 0x7fa6e00b8510 2023-11-28T11:21:34.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.485+0000 7fa6dd7fa700 1 -- 172.21.15.92:0/1444379771 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa6d4019030 con 0x7fa6e00b8510 2023-11-28T11:21:34.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.486+0000 7fa6dd7fa700 1 --2- 172.21.15.92:0/1444379771 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa6c40682f0 0x7fa6c406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:34.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.486+0000 7fa6dd7fa700 1 -- 172.21.15.92:0/1444379771 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa6d4085300 con 0x7fa6e00b8510 2023-11-28T11:21:34.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.486+0000 7fa6dffff700 1 --2- 172.21.15.92:0/1444379771 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa6c40682f0 0x7fa6c406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:34.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.487+0000 7fa6dffff700 1 --2- 172.21.15.92:0/1444379771 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa6c40682f0 0x7fa6c406a7a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa6e00cd3f0 tx=0x7fa6cc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:34.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.490+0000 7fa6dd7fa700 1 -- 172.21.15.92:0/1444379771 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa6d404b5e0 con 0x7fa6e00b8510 2023-11-28T11:21:34.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.725+0000 7fa6e71ff700 1 -- 172.21.15.92:0/1444379771 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa6e0067880 con 0x7fa6e00b8510 2023-11-28T11:21:34.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.726+0000 7fa6dd7fa700 1 -- 172.21.15.92:0/1444379771 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7fa6d4017070 con 0x7fa6e00b8510 2023-11-28T11:21:34.727 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-28T11:21:34.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.728+0000 7fa6e71ff700 1 -- 172.21.15.92:0/1444379771 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa6c40682f0 msgr2=0x7fa6c406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:34.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.728+0000 7fa6e71ff700 1 --2- 172.21.15.92:0/1444379771 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa6c40682f0 0x7fa6c406a7a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa6e00cd3f0 tx=0x7fa6cc008040 comp rx=0 tx=0).stop 2023-11-28T11:21:34.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.728+0000 7fa6e71ff700 1 -- 172.21.15.92:0/1444379771 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6e00b8510 msgr2=0x7fa6e00ccb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:34.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.728+0000 7fa6e71ff700 1 --2- 172.21.15.92:0/1444379771 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6e00b8510 0x7fa6e00ccb80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa6d400c520 tx=0x7fa6d400c8e0 comp rx=0 tx=0).stop 2023-11-28T11:21:34.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.728+0000 7fa6e71ff700 1 -- 172.21.15.92:0/1444379771 shutdown_connections 2023-11-28T11:21:34.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.728+0000 7fa6e71ff700 1 --2- 172.21.15.92:0/1444379771 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6e00d0d60 0x7fa6e0159160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:34.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.728+0000 7fa6e71ff700 1 --2- 172.21.15.92:0/1444379771 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa6c40682f0 0x7fa6c406a7a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:34.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.728+0000 7fa6e71ff700 1 --2- 172.21.15.92:0/1444379771 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6e00b8510 0x7fa6e00ccb80 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:34.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.728+0000 7fa6e71ff700 1 --2- 172.21.15.92:0/1444379771 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6e00cd460 0x7fa6e00aaee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:34.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.728+0000 7fa6e71ff700 1 -- 172.21.15.92:0/1444379771 >> 172.21.15.92:0/1444379771 conn(0x7fa6e00719a0 msgr2=0x7fa6e00b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:34.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.728+0000 7fa6e71ff700 1 -- 172.21.15.92:0/1444379771 shutdown_connections 2023-11-28T11:21:34.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:34.728+0000 7fa6e71ff700 1 -- 172.21.15.92:0/1444379771 wait complete. 2023-11-28T11:21:34.747 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T11:21:34.747 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T11:21:34.747 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi169.front.sepia.ceph.com 2023-11-28T11:21:34.747 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-28T11:21:34.747 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-28T11:21:34.747 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T11:21:34.748 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T11:21:34.748 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T11:21:34.748 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-28T11:21:34.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:34.748 DEBUG:teuthology.orchestra.run.smithi169:> ip addr 2023-11-28T11:21:34.795 INFO:teuthology.orchestra.run.smithi169.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:21:34.796 INFO:teuthology.orchestra.run.smithi169.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:21:34.796 INFO:teuthology.orchestra.run.smithi169.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:21:34.796 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:21:34.796 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 ::1/128 scope host 2023-11-28T11:21:34.796 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:21:34.797 INFO:teuthology.orchestra.run.smithi169.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:21:34.797 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:d9:3b:b4 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:21:34.797 INFO:teuthology.orchestra.run.smithi169.stdout: altname enp7s0f0 2023-11-28T11:21:34.797 INFO:teuthology.orchestra.run.smithi169.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:21:34.797 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:d9:3b:b5 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:21:34.797 INFO:teuthology.orchestra.run.smithi169.stdout: altname enp7s0f1 2023-11-28T11:21:34.797 INFO:teuthology.orchestra.run.smithi169.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:21:34.797 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:88:7f:4c brd ff:ff:ff:ff:ff:ff 2023-11-28T11:21:34.797 INFO:teuthology.orchestra.run.smithi169.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T11:21:34.797 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:88:7f:4d brd ff:ff:ff:ff:ff:ff 2023-11-28T11:21:34.798 INFO:teuthology.orchestra.run.smithi169.stdout: inet 172.21.15.169/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T11:21:34.798 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft 41938sec preferred_lft 41938sec 2023-11-28T11:21:34.798 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::ec4:7aff:fe88:7f4d/64 scope link 2023-11-28T11:21:34.798 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:21:34.799 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-28T11:21:34.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:34.799 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2023-11-28T11:21:34.799 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2023-11-28T11:21:34.799 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link add name ceph-brx type bridge 2023-11-28T11:21:34.799 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip addr flush dev ceph-brx 2023-11-28T11:21:34.799 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set ceph-brx up 2023-11-28T11:21:34.799 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-28T11:21:34.800 DEBUG:teuthology.orchestra.run.smithi169:> ') 2023-11-28T11:21:35.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:35.237 DEBUG:teuthology.orchestra.run.smithi169:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T11:21:35.350 INFO:teuthology.orchestra.run.smithi169.stdout:1 2023-11-28T11:21:35.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:35.351 DEBUG:teuthology.orchestra.run.smithi169:> route 2023-11-28T11:21:35.419 INFO:teuthology.orchestra.run.smithi169.stdout:Kernel IP routing table 2023-11-28T11:21:35.419 INFO:teuthology.orchestra.run.smithi169.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T11:21:35.419 INFO:teuthology.orchestra.run.smithi169.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T11:21:35.420 INFO:teuthology.orchestra.run.smithi169.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T11:21:35.420 INFO:teuthology.orchestra.run.smithi169.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T11:21:35.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:35.421 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2023-11-28T11:21:35.421 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2023-11-28T11:21:35.421 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T11:21:35.421 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T11:21:35.422 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T11:21:35.422 DEBUG:teuthology.orchestra.run.smithi169:> ') 2023-11-28T11:21:36.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:36.099 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2023-11-28T11:21:36.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:36.148 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list-id 2023-11-28T11:21:36.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:36.195 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2023-11-28T11:21:36.196 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2023-11-28T11:21:36.196 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-28T11:21:36.196 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-11-28T11:21:36.196 DEBUG:teuthology.orchestra.run.smithi169:> ') 2023-11-28T11:21:36.342 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-11-28T11:21:36.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:36.343 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2023-11-28T11:21:36.343 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2023-11-28T11:21:36.343 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-11-28T11:21:36.343 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-28T11:21:36.343 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-11-28T11:21:36.343 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-11-28T11:21:36.343 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-11-28T11:21:36.343 DEBUG:teuthology.orchestra.run.smithi169:> ') 2023-11-28T11:21:36.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:36.632 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2023-11-28T11:21:36.632 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2023-11-28T11:21:36.633 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set brx.0 up 2023-11-28T11:21:36.633 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set dev brx.0 master ceph-brx 2023-11-28T11:21:36.633 DEBUG:teuthology.orchestra.run.smithi169:> ') 2023-11-28T11:21:36.811 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:21:36.811 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-28T11:21:36.857 INFO:teuthology.orchestra.run.smithi169.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-28T11:21:36.857 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:21:36.857 DEBUG:teuthology.orchestra.run.smithi169:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-28T11:21:36.903 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-11-28T11:21:36.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:36.903 DEBUG:teuthology.orchestra.run.smithi169:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-11-28T11:21:37.050 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-11-28T11:21:37.050 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-11-28T11:21:37.050 INFO:teuthology.orchestra.run.smithi169.stdout:invalid new device string format 2023-11-28T11:21:37.498 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-11-28T11:21:37.498 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-11-28T11:21:37.498 INFO:teuthology.orchestra.run.smithi169.stdout:invalid new device string format 2023-11-28T11:21:37.498 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.169:3300,172.21.15.169:3301" 2023-11-28T11:21:37.498 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:21:37.523 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-11-28T11:21:37.523 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-11-28T11:21:37.523 INFO:teuthology.orchestra.run.smithi169.stdout:invalid new device string format 2023-11-28T11:21:37.523 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.169:3300,172.21.15.169:3301" 2023-11-28T11:21:37.523 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:21:37.523 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.92:6789,172.21.15.169:6789,172.21.15.169:6790" 2023-11-28T11:21:37.523 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.92:6789,172.21.15.169:6789,172.21.15.169:6790:/ 2023-11-28T11:21:37.523 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=68956e0c-c16f-47cd-a220-f98364b0831d" will pass to kernel 2023-11-28T11:21:37.529 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T11:21:37.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:37.530 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-28T11:21:37.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:37.600 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:21:37.710 INFO:teuthology.orchestra.run.smithi169.stdout:client4643 2023-11-28T11:21:37.711 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-28T11:21:37.711 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:21:37.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:37.758 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:21:37.887 INFO:teuthology.orchestra.run.smithi169.stdout:client4643 2023-11-28T11:21:37.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:37.887 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:21:37.977 INFO:teuthology.orchestra.run.smithi169.stdout:client4643 2023-11-28T11:21:37.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:37.977 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/68956e0c-c16f-47cd-a220-f98364b0831d.client4643/status') 2023-11-28T11:21:38.087 INFO:teuthology.orchestra.run.smithi169.stdout:instance: client.4643 (3)192.168.0.1:0/2031634591 2023-11-28T11:21:38.087 INFO:teuthology.orchestra.run.smithi169.stdout:blocklisted: false 2023-11-28T11:21:38.088 INFO:teuthology.orchestra.run.smithi169.stderr:0+1 records in 2023-11-28T11:21:38.088 INFO:teuthology.orchestra.run.smithi169.stderr:0+1 records out 2023-11-28T11:21:38.088 INFO:teuthology.orchestra.run.smithi169.stderr:69 bytes copied, 5.2753e-05 s, 1.3 MB/s 2023-11-28T11:21:38.091 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2023-11-28T11:21:38.091 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-28T11:21:38.091 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-28T11:21:38.091 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T11:21:38.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.296+0000 7f880b5ff700 1 Processor -- start 2023-11-28T11:21:38.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.296+0000 7f880b5ff700 1 -- start start 2023-11-28T11:21:38.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.297+0000 7f880b5ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8804159080 0x7f88040ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:38.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.297+0000 7f880b5ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8804071020 0x7f880406dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:38.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.297+0000 7f880b5ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f880406d870 con 0x7f8804158cb0 2023-11-28T11:21:38.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.297+0000 7f880b5ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f880406cc60 con 0x7f8804071020 2023-11-28T11:21:38.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.297+0000 7f880b5ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f880406cf40 con 0x7f8804159080 2023-11-28T11:21:38.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.297+0000 7f8809b9c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8804071020 0x7f880406dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:38.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.297+0000 7f8809b9c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8804071020 0x7f880406dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57124/0 (socket says 172.21.15.92:57124) 2023-11-28T11:21:38.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.297+0000 7f8809b9c700 1 -- 172.21.15.92:0/4022007841 learned_addr learned my addr 172.21.15.92:0/4022007841 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:38.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.297+0000 7f8808b9a700 1 --2- 172.21.15.92:0/4022007841 >> v2:172.21.15.169:3300/0 conn(0x7f8804159080 0x7f88040ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:38.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.298+0000 7f8809b9c700 1 -- 172.21.15.92:0/4022007841 >> v2:172.21.15.169:3300/0 conn(0x7f8804159080 msgr2=0x7f88040ca1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:38.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.298+0000 7f8809b9c700 1 --2- 172.21.15.92:0/4022007841 >> v2:172.21.15.169:3300/0 conn(0x7f8804159080 0x7f88040ca1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:38.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.298+0000 7f8809b9c700 1 -- 172.21.15.92:0/4022007841 >> v1:172.21.15.169:6789/0 conn(0x7f8804158cb0 legacy=0x7f8804070910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:38.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.298+0000 7f8809b9c700 1 -- 172.21.15.92:0/4022007841 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f880406d220 con 0x7f8804071020 2023-11-28T11:21:38.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.298+0000 7f8809b9c700 1 --2- 172.21.15.92:0/4022007841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8804071020 0x7f880406dd00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f880415c8c0 tx=0x7f87fc00d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4726b86361baeaa server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:38.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.298+0000 7f87fbfff700 1 -- 172.21.15.92:0/4022007841 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87fc010070 con 0x7f8804071020 2023-11-28T11:21:38.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.298+0000 7f87fbfff700 1 -- 172.21.15.92:0/4022007841 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87fc00dad0 con 0x7f8804071020 2023-11-28T11:21:38.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.298+0000 7f87fbfff700 1 -- 172.21.15.92:0/4022007841 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87fc00de80 con 0x7f8804071020 2023-11-28T11:21:38.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.299+0000 7f880b5ff700 1 -- 172.21.15.92:0/4022007841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8804071020 msgr2=0x7f880406dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:38.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.299+0000 7f880b5ff700 1 --2- 172.21.15.92:0/4022007841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8804071020 0x7f880406dd00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f880415c8c0 tx=0x7f87fc00d430 comp rx=0 tx=0).stop 2023-11-28T11:21:38.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.299+0000 7f880b5ff700 1 -- 172.21.15.92:0/4022007841 shutdown_connections 2023-11-28T11:21:38.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.299+0000 7f880b5ff700 1 --2- 172.21.15.92:0/4022007841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8804071020 0x7f880406dd00 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:38.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.299+0000 7f880b5ff700 1 --2- 172.21.15.92:0/4022007841 >> v2:172.21.15.169:3300/0 conn(0x7f8804159080 0x7f88040ca1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:38.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.299+0000 7f880b5ff700 1 -- 172.21.15.92:0/4022007841 >> 172.21.15.92:0/4022007841 conn(0x7f88040cb490 msgr2=0x7f88040cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:38.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.299+0000 7f880b5ff700 1 -- 172.21.15.92:0/4022007841 shutdown_connections 2023-11-28T11:21:38.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.299+0000 7f880b5ff700 1 -- 172.21.15.92:0/4022007841 wait complete. 2023-11-28T11:21:38.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.301+0000 7f880b5ff700 1 Processor -- start 2023-11-28T11:21:38.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.301+0000 7f880b5ff700 1 -- start start 2023-11-28T11:21:38.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.302+0000 7f880b5ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8804071020 0x7f88040ac920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:38.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.302+0000 7f880b5ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8804158cb0 0x7f88040ace60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:38.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.302+0000 7f880b5ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88040c21b0 0x7f8804144790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:38.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.302+0000 7f880b5ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88040b2c30 con 0x7f88040c21b0 2023-11-28T11:21:38.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.302+0000 7f880b5ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88040c37b0 con 0x7f8804071020 2023-11-28T11:21:38.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.302+0000 7f880b5ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88040c26b0 con 0x7f8804158cb0 2023-11-28T11:21:38.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.302+0000 7f8809b9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88040c21b0 0x7f8804144790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:38.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.302+0000 7f8809b9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88040c21b0 0x7f8804144790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57128/0 (socket says 172.21.15.92:57128) 2023-11-28T11:21:38.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.302+0000 7f8809b9c700 1 -- 172.21.15.92:0/1030942088 learned_addr learned my addr 172.21.15.92:0/1030942088 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:38.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.302+0000 7f8808b9a700 1 --2- 172.21.15.92:0/1030942088 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8804158cb0 0x7f88040ace60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:38.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.302+0000 7f880939b700 1 --2- 172.21.15.92:0/1030942088 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8804071020 0x7f88040ac920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:38.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.302+0000 7f8809b9c700 1 -- 172.21.15.92:0/1030942088 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8804158cb0 msgr2=0x7f88040ace60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:38.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.302+0000 7f8809b9c700 1 --2- 172.21.15.92:0/1030942088 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8804158cb0 0x7f88040ace60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:38.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.302+0000 7f8809b9c700 1 -- 172.21.15.92:0/1030942088 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8804071020 msgr2=0x7f88040ac920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:38.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.302+0000 7f8809b9c700 1 --2- 172.21.15.92:0/1030942088 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8804071020 0x7f88040ac920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:38.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.302+0000 7f8809b9c700 1 -- 172.21.15.92:0/1030942088 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87fc00bd40 con 0x7f88040c21b0 2023-11-28T11:21:38.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.303+0000 7f8809b9c700 1 --2- 172.21.15.92:0/1030942088 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88040c21b0 0x7f8804144790 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f87fc016000 tx=0x7f87fc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:38.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.303+0000 7f87f9ffb700 1 -- 172.21.15.92:0/1030942088 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87fc004140 con 0x7f88040c21b0 2023-11-28T11:21:38.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.303+0000 7f87f9ffb700 1 -- 172.21.15.92:0/1030942088 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87fc0042e0 con 0x7f88040c21b0 2023-11-28T11:21:38.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.303+0000 7f880b5ff700 1 -- 172.21.15.92:0/1030942088 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8804071b20 con 0x7f88040c21b0 2023-11-28T11:21:38.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.303+0000 7f87f9ffb700 1 -- 172.21.15.92:0/1030942088 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87fc004690 con 0x7f88040c21b0 2023-11-28T11:21:38.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.303+0000 7f880b5ff700 1 -- 172.21.15.92:0/1030942088 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f880406d500 con 0x7f88040c21b0 2023-11-28T11:21:38.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.304+0000 7f880b5ff700 1 -- 172.21.15.92:0/1030942088 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88040c9300 con 0x7f88040c21b0 2023-11-28T11:21:38.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.305+0000 7f87f9ffb700 1 -- 172.21.15.92:0/1030942088 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f87fc014070 con 0x7f88040c21b0 2023-11-28T11:21:38.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.306+0000 7f87f9ffb700 1 --2- 172.21.15.92:0/1030942088 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f87e80682f0 0x7f87e806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:38.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.306+0000 7f87f9ffb700 1 -- 172.21.15.92:0/1030942088 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f87fc087580 con 0x7f88040c21b0 2023-11-28T11:21:38.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.306+0000 7f880939b700 1 --2- 172.21.15.92:0/1030942088 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f87e80682f0 0x7f87e806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:38.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.307+0000 7f880939b700 1 --2- 172.21.15.92:0/1030942088 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f87e80682f0 0x7f87e806a7a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8800007fd0 tx=0x7f8800009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:38.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.310+0000 7f87f9ffb700 1 -- 172.21.15.92:0/1030942088 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f87fc0539c0 con 0x7f88040c21b0 2023-11-28T11:21:38.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.545+0000 7f880b5ff700 1 -- 172.21.15.92:0/1030942088 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f88040c3920 con 0x7f88040c21b0 2023-11-28T11:21:38.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.545+0000 7f87f9ffb700 1 -- 172.21.15.92:0/1030942088 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f87fc056fe0 con 0x7f88040c21b0 2023-11-28T11:21:38.546 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-28T11:21:38.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.547+0000 7f880b5ff700 1 -- 172.21.15.92:0/1030942088 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f87e80682f0 msgr2=0x7f87e806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:38.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.547+0000 7f880b5ff700 1 --2- 172.21.15.92:0/1030942088 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f87e80682f0 0x7f87e806a7a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8800007fd0 tx=0x7f8800009040 comp rx=0 tx=0).stop 2023-11-28T11:21:38.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.547+0000 7f880b5ff700 1 -- 172.21.15.92:0/1030942088 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88040c21b0 msgr2=0x7f8804144790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:38.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.547+0000 7f880b5ff700 1 --2- 172.21.15.92:0/1030942088 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88040c21b0 0x7f8804144790 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f87fc016000 tx=0x7f87fc004ad0 comp rx=0 tx=0).stop 2023-11-28T11:21:38.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.547+0000 7f880b5ff700 1 -- 172.21.15.92:0/1030942088 shutdown_connections 2023-11-28T11:21:38.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.547+0000 7f880b5ff700 1 --2- 172.21.15.92:0/1030942088 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f87e80682f0 0x7f87e806a7a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:38.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.547+0000 7f880b5ff700 1 --2- 172.21.15.92:0/1030942088 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88040c21b0 0x7f8804144790 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:38.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.547+0000 7f880b5ff700 1 --2- 172.21.15.92:0/1030942088 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8804071020 0x7f88040ac920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:38.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.547+0000 7f880b5ff700 1 --2- 172.21.15.92:0/1030942088 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8804158cb0 0x7f88040ace60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:38.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.547+0000 7f880b5ff700 1 -- 172.21.15.92:0/1030942088 >> 172.21.15.92:0/1030942088 conn(0x7f88040cb490 msgr2=0x7f88040afd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:38.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.547+0000 7f880b5ff700 1 -- 172.21.15.92:0/1030942088 shutdown_connections 2023-11-28T11:21:38.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:38.547+0000 7f880b5ff700 1 -- 172.21.15.92:0/1030942088 wait complete. 2023-11-28T11:21:38.567 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T11:21:38.567 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T11:21:38.567 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi169.front.sepia.ceph.com 2023-11-28T11:21:38.567 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-11-28T11:21:38.567 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-28T11:21:38.567 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T11:21:38.567 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T11:21:38.567 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T11:21:38.567 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-11-28T11:21:38.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:38.568 DEBUG:teuthology.orchestra.run.smithi169:> ip addr 2023-11-28T11:21:38.617 INFO:teuthology.orchestra.run.smithi169.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:21:38.617 INFO:teuthology.orchestra.run.smithi169.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:21:38.617 INFO:teuthology.orchestra.run.smithi169.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:21:38.618 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:21:38.618 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 ::1/128 scope host 2023-11-28T11:21:38.618 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:21:38.618 INFO:teuthology.orchestra.run.smithi169.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:21:38.618 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:d9:3b:b4 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:21:38.619 INFO:teuthology.orchestra.run.smithi169.stdout: altname enp7s0f0 2023-11-28T11:21:38.619 INFO:teuthology.orchestra.run.smithi169.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:21:38.619 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:d9:3b:b5 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:21:38.619 INFO:teuthology.orchestra.run.smithi169.stdout: altname enp7s0f1 2023-11-28T11:21:38.619 INFO:teuthology.orchestra.run.smithi169.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:21:38.619 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:88:7f:4c brd ff:ff:ff:ff:ff:ff 2023-11-28T11:21:38.620 INFO:teuthology.orchestra.run.smithi169.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T11:21:38.620 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 0c:c4:7a:88:7f:4d brd ff:ff:ff:ff:ff:ff 2023-11-28T11:21:38.620 INFO:teuthology.orchestra.run.smithi169.stdout: inet 172.21.15.169/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T11:21:38.620 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft 41934sec preferred_lft 41934sec 2023-11-28T11:21:38.620 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::ec4:7aff:fe88:7f4d/64 scope link 2023-11-28T11:21:38.620 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:21:38.621 INFO:teuthology.orchestra.run.smithi169.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T11:21:38.621 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether e6:52:e1:2b:0a:a8 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:21:38.621 INFO:teuthology.orchestra.run.smithi169.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T11:21:38.621 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:21:38.621 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::48c9:43ff:fe8b:ed95/64 scope link 2023-11-28T11:21:38.621 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:21:38.622 INFO:teuthology.orchestra.run.smithi169.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T11:21:38.622 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether e6:52:e1:2b:0a:a8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-28T11:21:38.622 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::e452:e1ff:fe2b:aa8/64 scope link tentative 2023-11-28T11:21:38.622 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:21:38.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:38.623 DEBUG:teuthology.orchestra.run.smithi169:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T11:21:38.691 INFO:teuthology.orchestra.run.smithi169.stdout:1 2023-11-28T11:21:38.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:38.695 DEBUG:teuthology.orchestra.run.smithi169:> route 2023-11-28T11:21:38.743 INFO:teuthology.orchestra.run.smithi169.stdout:Kernel IP routing table 2023-11-28T11:21:38.743 INFO:teuthology.orchestra.run.smithi169.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T11:21:38.744 INFO:teuthology.orchestra.run.smithi169.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T11:21:38.744 INFO:teuthology.orchestra.run.smithi169.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T11:21:38.744 INFO:teuthology.orchestra.run.smithi169.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T11:21:38.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:38.745 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2023-11-28T11:21:38.745 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2023-11-28T11:21:38.745 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T11:21:38.745 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T11:21:38.745 DEBUG:teuthology.orchestra.run.smithi169:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T11:21:38.745 DEBUG:teuthology.orchestra.run.smithi169:> ') 2023-11-28T11:21:38.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:38.916 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2023-11-28T11:21:38.962 INFO:teuthology.orchestra.run.smithi169.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-28T11:21:38.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:38.964 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list-id 2023-11-28T11:21:39.010 INFO:teuthology.orchestra.run.smithi169.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-11-28T11:21:39.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:39.011 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2023-11-28T11:21:39.011 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2023-11-28T11:21:39.011 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-28T11:21:39.011 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-11-28T11:21:39.011 DEBUG:teuthology.orchestra.run.smithi169:> ') 2023-11-28T11:21:39.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:39.194 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-28T11:21:39.265 INFO:teuthology.orchestra.run.smithi169.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:21:39.266 INFO:teuthology.orchestra.run.smithi169.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:21:39.266 INFO:teuthology.orchestra.run.smithi169.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:21:39.266 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:21:39.266 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 ::1/128 scope host 2023-11-28T11:21:39.266 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:21:39.266 INFO:teuthology.orchestra.run.smithi169.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T11:21:39.266 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 92:92:44:11:c2:45 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-28T11:21:39.266 INFO:teuthology.orchestra.run.smithi169.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-28T11:21:39.266 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:21:39.266 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::9092:44ff:fe11:c245/64 scope link 2023-11-28T11:21:39.267 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:21:39.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:39.269 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-28T11:21:39.340 INFO:teuthology.orchestra.run.smithi169.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:21:39.340 INFO:teuthology.orchestra.run.smithi169.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:21:39.341 INFO:teuthology.orchestra.run.smithi169.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:21:39.341 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:21:39.341 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 ::1/128 scope host 2023-11-28T11:21:39.341 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:21:39.341 INFO:teuthology.orchestra.run.smithi169.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T11:21:39.341 INFO:teuthology.orchestra.run.smithi169.stdout: link/ether 92:92:44:11:c2:45 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-28T11:21:39.341 INFO:teuthology.orchestra.run.smithi169.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-28T11:21:39.341 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:21:39.341 INFO:teuthology.orchestra.run.smithi169.stdout: inet6 fe80::9092:44ff:fe11:c245/64 scope link 2023-11-28T11:21:39.341 INFO:teuthology.orchestra.run.smithi169.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:21:39.343 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-11-28T11:21:39.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:39.344 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2023-11-28T11:21:39.344 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2023-11-28T11:21:39.344 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-11-28T11:21:39.344 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-28T11:21:39.344 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-11-28T11:21:39.344 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-11-28T11:21:39.344 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-11-28T11:21:39.344 DEBUG:teuthology.orchestra.run.smithi169:> ') 2023-11-28T11:21:39.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:39.589 DEBUG:teuthology.orchestra.run.smithi169:> (cd / && exec bash -c ' 2023-11-28T11:21:39.589 DEBUG:teuthology.orchestra.run.smithi169:> set -e 2023-11-28T11:21:39.589 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set brx.1 up 2023-11-28T11:21:39.589 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link set dev brx.1 master ceph-brx 2023-11-28T11:21:39.589 DEBUG:teuthology.orchestra.run.smithi169:> ') 2023-11-28T11:21:39.763 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:21:39.763 DEBUG:teuthology.orchestra.run.smithi169:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-11-28T11:21:39.810 INFO:teuthology.orchestra.run.smithi169.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-11-28T11:21:39.810 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:21:39.810 DEBUG:teuthology.orchestra.run.smithi169:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-11-28T11:21:39.897 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-11-28T11:21:39.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:39.897 DEBUG:teuthology.orchestra.run.smithi169:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-11-28T11:21:40.042 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-11-28T11:21:40.043 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-11-28T11:21:40.043 INFO:teuthology.orchestra.run.smithi169.stdout:invalid new device string format 2023-11-28T11:21:40.058 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-11-28T11:21:40.059 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-11-28T11:21:40.059 INFO:teuthology.orchestra.run.smithi169.stdout:invalid new device string format 2023-11-28T11:21:40.059 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.169:3300,172.21.15.169:3301" 2023-11-28T11:21:40.059 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:21:40.080 INFO:teuthology.orchestra.run.smithi169.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-11-28T11:21:40.081 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-11-28T11:21:40.081 INFO:teuthology.orchestra.run.smithi169.stdout:invalid new device string format 2023-11-28T11:21:40.081 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.169:3300,172.21.15.169:3301" 2023-11-28T11:21:40.081 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:21:40.081 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: resolved to: "172.21.15.92:6789,172.21.15.169:6789,172.21.15.169:6790" 2023-11-28T11:21:40.081 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.92:6789,172.21.15.169:6789,172.21.15.169:6790:/ 2023-11-28T11:21:40.081 INFO:teuthology.orchestra.run.smithi169.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=68956e0c-c16f-47cd-a220-f98364b0831d" will pass to kernel 2023-11-28T11:21:40.084 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T11:21:40.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:40.085 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-11-28T11:21:40.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:40.156 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:21:40.244 INFO:teuthology.orchestra.run.smithi169.stdout:client4649 2023-11-28T11:21:40.245 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-28T11:21:40.245 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:21:40.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:40.291 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:21:40.420 INFO:teuthology.orchestra.run.smithi169.stdout:client4649 2023-11-28T11:21:40.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:40.420 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:21:40.509 INFO:teuthology.orchestra.run.smithi169.stdout:client4649 2023-11-28T11:21:40.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:40.509 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/68956e0c-c16f-47cd-a220-f98364b0831d.client4649/status') 2023-11-28T11:21:40.618 INFO:teuthology.orchestra.run.smithi169.stdout:instance: client.4649 (3)192.168.0.2:0/1581754056 2023-11-28T11:21:40.618 INFO:teuthology.orchestra.run.smithi169.stdout:blocklisted: false 2023-11-28T11:21:40.618 INFO:teuthology.orchestra.run.smithi169.stderr:0+1 records in 2023-11-28T11:21:40.618 INFO:teuthology.orchestra.run.smithi169.stderr:0+1 records out 2023-11-28T11:21:40.618 INFO:teuthology.orchestra.run.smithi169.stderr:69 bytes copied, 7.2074e-05 s, 957 kB/s 2023-11-28T11:21:40.621 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-11-28T11:21:40.639 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:21:40.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.848+0000 7f757adff700 1 Processor -- start 2023-11-28T11:21:40.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.848+0000 7f757adff700 1 -- start start 2023-11-28T11:21:40.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.849+0000 7f757adff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75740b8510 0x7f757406b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:40.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.849+0000 7f757adff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f75740d0d60 0x7f75740d11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:40.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.849+0000 7f757adff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75740b88e0 con 0x7f75740cd460 2023-11-28T11:21:40.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.849+0000 7f757adff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75740cdbd0 con 0x7f75740d0d60 2023-11-28T11:21:40.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.850+0000 7f757adff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75740d02d0 con 0x7f75740b8510 2023-11-28T11:21:40.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.850+0000 7f757939c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f75740d0d60 0x7f75740d11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:40.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.850+0000 7f757939c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f75740d0d60 0x7f75740d11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51266/0 (socket says 172.21.15.92:51266) 2023-11-28T11:21:40.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.850+0000 7f757939c700 1 -- 172.21.15.92:0/4114746762 learned_addr learned my addr 172.21.15.92:0/4114746762 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:40.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.850+0000 7f7578b9b700 1 --2- 172.21.15.92:0/4114746762 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75740b8510 0x7f757406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:40.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.850+0000 7f757939c700 1 -- 172.21.15.92:0/4114746762 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75740b8510 msgr2=0x7f757406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:40.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.850+0000 7f757939c700 1 --2- 172.21.15.92:0/4114746762 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75740b8510 0x7f757406b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:40.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.850+0000 7f757939c700 1 -- 172.21.15.92:0/4114746762 >> v1:172.21.15.169:6789/0 conn(0x7f75740cd460 legacy=0x7f757406beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:40.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.850+0000 7f75737fe700 1 -- 172.21.15.92:0/4114746762 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4172982829 0 0) 0x7f75740b88e0 con 0x7f75740cd460 2023-11-28T11:21:40.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.850+0000 7f757939c700 1 -- 172.21.15.92:0/4114746762 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75740cd8f0 con 0x7f75740d0d60 2023-11-28T11:21:40.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.850+0000 7f757939c700 1 --2- 172.21.15.92:0/4114746762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75740d0d60 0x7f75740d11a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f756c009e80 tx=0x7f756c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c36f830a8e4d997e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:40.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.851+0000 7f75737fe700 1 -- 172.21.15.92:0/4114746762 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c00e070 con 0x7f75740d0d60 2023-11-28T11:21:40.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.851+0000 7f75737fe700 1 -- 172.21.15.92:0/4114746762 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f756c00b8d0 con 0x7f75740d0d60 2023-11-28T11:21:40.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.851+0000 7f75737fe700 1 -- 172.21.15.92:0/4114746762 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c00bc80 con 0x7f75740d0d60 2023-11-28T11:21:40.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.851+0000 7f757adff700 1 -- 172.21.15.92:0/4114746762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75740d0d60 msgr2=0x7f75740d11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:40.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.851+0000 7f757adff700 1 --2- 172.21.15.92:0/4114746762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75740d0d60 0x7f75740d11a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f756c009e80 tx=0x7f756c00b330 comp rx=0 tx=0).stop 2023-11-28T11:21:40.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.851+0000 7f757adff700 1 -- 172.21.15.92:0/4114746762 shutdown_connections 2023-11-28T11:21:40.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.851+0000 7f757adff700 1 --2- 172.21.15.92:0/4114746762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75740d0d60 0x7f75740d11a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:40.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.851+0000 7f757adff700 1 --2- 172.21.15.92:0/4114746762 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75740b8510 0x7f757406b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:40.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.851+0000 7f757adff700 1 -- 172.21.15.92:0/4114746762 >> 172.21.15.92:0/4114746762 conn(0x7f75740719a0 msgr2=0x7f7574071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:40.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.851+0000 7f757adff700 1 -- 172.21.15.92:0/4114746762 shutdown_connections 2023-11-28T11:21:40.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.851+0000 7f757adff700 1 -- 172.21.15.92:0/4114746762 wait complete. 2023-11-28T11:21:40.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.854+0000 7f757adff700 1 Processor -- start 2023-11-28T11:21:40.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.854+0000 7f757adff700 1 -- start start 2023-11-28T11:21:40.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.855+0000 7f757adff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75740b8510 0x7f75740c9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:40.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.855+0000 7f757adff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75740cd460 0x7f757415acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:40.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.855+0000 7f757adff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75740d0d60 0x7f75741e6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:40.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.855+0000 7f757adff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757415b8b0 con 0x7f75740cd460 2023-11-28T11:21:40.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.855+0000 7f757adff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75740c7350 con 0x7f75740d0d60 2023-11-28T11:21:40.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.855+0000 7f757adff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75741e3630 con 0x7f75740b8510 2023-11-28T11:21:40.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.855+0000 7f7573fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75740cd460 0x7f757415acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:40.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.856+0000 7f7573fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75740cd460 0x7f757415acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51268/0 (socket says 172.21.15.92:51268) 2023-11-28T11:21:40.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.856+0000 7f7573fff700 1 -- 172.21.15.92:0/1637015084 learned_addr learned my addr 172.21.15.92:0/1637015084 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:40.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.856+0000 7f7578b9b700 1 --2- 172.21.15.92:0/1637015084 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75740b8510 0x7f75740c9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:40.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.856+0000 7f757939c700 1 --2- 172.21.15.92:0/1637015084 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75740d0d60 0x7f75741e6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:40.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.856+0000 7f7573fff700 1 -- 172.21.15.92:0/1637015084 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75740b8510 msgr2=0x7f75740c9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:40.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.856+0000 7f7573fff700 1 --2- 172.21.15.92:0/1637015084 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75740b8510 0x7f75740c9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:40.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.856+0000 7f7573fff700 1 -- 172.21.15.92:0/1637015084 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75740d0d60 msgr2=0x7f75741e6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:40.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.856+0000 7f7573fff700 1 --2- 172.21.15.92:0/1637015084 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75740d0d60 0x7f75741e6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:40.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.856+0000 7f7573fff700 1 -- 172.21.15.92:0/1637015084 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f756c009b30 con 0x7f75740cd460 2023-11-28T11:21:40.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.856+0000 7f7573fff700 1 --2- 172.21.15.92:0/1637015084 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75740cd460 0x7f757415acb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f75600011e0 tx=0x7f756000cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:40.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.856+0000 7f75717fa700 1 -- 172.21.15.92:0/1637015084 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75600054e0 con 0x7f75740cd460 2023-11-28T11:21:40.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.856+0000 7f75717fa700 1 -- 172.21.15.92:0/1637015084 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7560003680 con 0x7f75740cd460 2023-11-28T11:21:40.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.857+0000 7f757adff700 1 -- 172.21.15.92:0/1637015084 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75740cfff0 con 0x7f75740cd460 2023-11-28T11:21:40.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.857+0000 7f75717fa700 1 -- 172.21.15.92:0/1637015084 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7560003a30 con 0x7f75740cd460 2023-11-28T11:21:40.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.857+0000 7f757adff700 1 -- 172.21.15.92:0/1637015084 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75740cfd10 con 0x7f75740cd460 2023-11-28T11:21:40.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.858+0000 7f757adff700 1 -- 172.21.15.92:0/1637015084 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7574067880 con 0x7f75740cd460 2023-11-28T11:21:40.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.858+0000 7f75717fa700 1 -- 172.21.15.92:0/1637015084 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7560019030 con 0x7f75740cd460 2023-11-28T11:21:40.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.860+0000 7f75717fa700 1 --2- 172.21.15.92:0/1637015084 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f75580682f0 0x7f755806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:40.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.860+0000 7f75717fa700 1 -- 172.21.15.92:0/1637015084 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f75600854a0 con 0x7f75740cd460 2023-11-28T11:21:40.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.860+0000 7f7578b9b700 1 --2- 172.21.15.92:0/1637015084 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f75580682f0 0x7f755806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:40.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.861+0000 7f7578b9b700 1 --2- 172.21.15.92:0/1637015084 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f75580682f0 0x7f755806a7a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7568009b70 tx=0x7f7568008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:40.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:40.864+0000 7f75717fa700 1 -- 172.21.15.92:0/1637015084 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f75600518e0 con 0x7f75740cd460 2023-11-28T11:21:41.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.060+0000 7f757adff700 1 -- 172.21.15.92:0/1637015084 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f75740d2940 con 0x7f75740cd460 2023-11-28T11:21:41.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.061+0000 7f75717fa700 1 -- 172.21.15.92:0/1637015084 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+19844 (secure 0 0 0) 0x7f7560017090 con 0x7f75740cd460 2023-11-28T11:21:41.062 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:21:41.064 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":29,"fsid":"68956e0c-c16f-47cd-a220-f98364b0831d","created":"2023-11-28T11:20:28.697573+0000","modified":"2023-11-28T11:21:37.638578+0000","last_up_change":"2023-11-28T11:20:41.257011+0000","last_in_change":"2023-11-28T11:20:32.249767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:20:41.925237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:20:48.682497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:20:54.471398+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:20:55.204585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:20:58.297307+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6abe483f-0e03-4a26-b9cf-62041291747e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":115823},{"type":"v1","addr":"172.21.15.92:6809","nonce":115823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":115823},{"type":"v1","addr":"172.21.15.92:6813","nonce":115823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":115823},{"type":"v1","addr":"172.21.15.92:6821","nonce":115823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":115823},{"type":"v1","addr":"172.21.15.92:6817","nonce":115823}]},"public_addr":"172.21.15.92:6809/115823","cluster_addr":"172.21.15.92:6813/115823","heartbeat_back_addr":"172.21.15.92:6821/115823","heartbeat_front_addr":"172.21.15.92:6817/115823","state":["exists","up"]},{"osd":1,"uuid":"13c1de73-1a8c-4d7e-8a28-9dae52dfe089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":115824},{"type":"v1","addr":"172.21.15.92:6812","nonce":115824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":115824},{"type":"v1","addr":"172.21.15.92:6816","nonce":115824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":115824},{"type":"v1","addr":"172.21.15.92:6823","nonce":115824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":115824},{"type":"v1","addr":"172.21.15.92:6820","nonce":115824}]},"public_addr":"172.21.15.92:6812/115824","cluster_addr":"172.21.15.92:6816/115824","heartbeat_back_addr":"172.21.15.92:6823/115824","heartbeat_front_addr":"172.21.15.92:6820/115824","state":["exists","up"]},{"osd":2,"uuid":"25788a55-112f-4f00-810d-2bc37cf76778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":115821},{"type":"v1","addr":"172.21.15.92:6801","nonce":115821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":115821},{"type":"v1","addr":"172.21.15.92:6803","nonce":115821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":115821},{"type":"v1","addr":"172.21.15.92:6807","nonce":115821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":115821},{"type":"v1","addr":"172.21.15.92:6805","nonce":115821}]},"public_addr":"172.21.15.92:6801/115821","cluster_addr":"172.21.15.92:6803/115821","heartbeat_back_addr":"172.21.15.92:6807/115821","heartbeat_front_addr":"172.21.15.92:6805/115821","state":["exists","up"]},{"osd":3,"uuid":"adf54e24-33a9-41fe-9ce3-98a5ab6c158e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":115825},{"type":"v1","addr":"172.21.15.92:6825","nonce":115825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":115825},{"type":"v1","addr":"172.21.15.92:6827","nonce":115825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":115825},{"type":"v1","addr":"172.21.15.92:6831","nonce":115825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":115825},{"type":"v1","addr":"172.21.15.92:6829","nonce":115825}]},"public_addr":"172.21.15.92:6825/115825","cluster_addr":"172.21.15.92:6827/115825","heartbeat_back_addr":"172.21.15.92:6831/115825","heartbeat_front_addr":"172.21.15.92:6829/115825","state":["exists","up"]},{"osd":4,"uuid":"ccb9b747-f399-44ea-9a18-9db3152227e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":116035},{"type":"v1","addr":"172.21.15.169:6817","nonce":116035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":116035},{"type":"v1","addr":"172.21.15.169:6819","nonce":116035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":116035},{"type":"v1","addr":"172.21.15.169:6826","nonce":116035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":116035},{"type":"v1","addr":"172.21.15.169:6822","nonce":116035}]},"public_addr":"172.21.15.169:6817/116035","cluster_addr":"172.21.15.169:6819/116035","heartbeat_back_addr":"172.21.15.169:6826/116035","heartbeat_front_addr":"172.21.15.169:6822/116035","state":["exists","up"]},{"osd":5,"uuid":"308a1e63-75aa-4d7d-8a7c-cecc24714fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":116033},{"type":"v1","addr":"172.21.15.169:6801","nonce":116033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":116033},{"type":"v1","addr":"172.21.15.169:6803","nonce":116033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":116033},{"type":"v1","addr":"172.21.15.169:6807","nonce":116033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":116033},{"type":"v1","addr":"172.21.15.169:6805","nonce":116033}]},"public_addr":"172.21.15.169:6801/116033","cluster_addr":"172.21.15.169:6803/116033","heartbeat_back_addr":"172.21.15.169:6807/116033","heartbeat_front_addr":"172.21.15.169:6805/116033","state":["exists","up"]},{"osd":6,"uuid":"9c459b7b-b821-4b4b-b91e-6aaae271457c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":116034},{"type":"v1","addr":"172.21.15.169:6809","nonce":116034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":116034},{"type":"v1","addr":"172.21.15.169:6811","nonce":116034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":116034},{"type":"v1","addr":"172.21.15.169:6815","nonce":116034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":116034},{"type":"v1","addr":"172.21.15.169:6813","nonce":116034}]},"public_addr":"172.21.15.169:6809/116034","cluster_addr":"172.21.15.169:6811/116034","heartbeat_back_addr":"172.21.15.169:6815/116034","heartbeat_front_addr":"172.21.15.169:6813/116034","state":["exists","up"]},{"osd":7,"uuid":"1ca9aa4a-755d-4e28-a937-4d8dccab4df3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":116044},{"type":"v1","addr":"172.21.15.169:6823","nonce":116044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6825","nonce":116044},{"type":"v1","addr":"172.21.15.169:6827","nonce":116044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":116044},{"type":"v1","addr":"172.21.15.169:6831","nonce":116044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":116044},{"type":"v1","addr":"172.21.15.169:6829","nonce":116044}]},"public_addr":"172.21.15.169:6823/116044","cluster_addr":"172.21.15.169:6827/116044","heartbeat_back_addr":"172.21.15.169:6831/116044","heartbeat_front_addr":"172.21.15.169:6829/116044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.586781+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.869778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.508594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.860493+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.872267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.746933+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.611520+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.871446+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.1d","mappings":[{"from":3,"to":2}]},{"pgid":"4.21","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:21:41.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.063+0000 7f757adff700 1 -- 172.21.15.92:0/1637015084 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f75580682f0 msgr2=0x7f755806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:41.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.063+0000 7f757adff700 1 --2- 172.21.15.92:0/1637015084 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f75580682f0 0x7f755806a7a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7568009b70 tx=0x7f7568008040 comp rx=0 tx=0).stop 2023-11-28T11:21:41.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.063+0000 7f757adff700 1 -- 172.21.15.92:0/1637015084 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75740cd460 msgr2=0x7f757415acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:41.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.063+0000 7f757adff700 1 --2- 172.21.15.92:0/1637015084 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75740cd460 0x7f757415acb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f75600011e0 tx=0x7f756000cae0 comp rx=0 tx=0).stop 2023-11-28T11:21:41.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.063+0000 7f757adff700 1 -- 172.21.15.92:0/1637015084 shutdown_connections 2023-11-28T11:21:41.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.063+0000 7f757adff700 1 --2- 172.21.15.92:0/1637015084 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f75740d0d60 0x7f75741e6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:41.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.063+0000 7f757adff700 1 --2- 172.21.15.92:0/1637015084 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f75740b8510 0x7f75740c9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:41.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.064+0000 7f757adff700 1 --2- 172.21.15.92:0/1637015084 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f75580682f0 0x7f755806a7a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:41.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.064+0000 7f757adff700 1 --2- 172.21.15.92:0/1637015084 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75740cd460 0x7f757415acb0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:41.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.064+0000 7f757adff700 1 -- 172.21.15.92:0/1637015084 >> 172.21.15.92:0/1637015084 conn(0x7f75740719a0 msgr2=0x7f75740b2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:41.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.064+0000 7f757adff700 1 -- 172.21.15.92:0/1637015084 shutdown_connections 2023-11-28T11:21:41.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.064+0000 7f757adff700 1 -- 172.21.15.92:0/1637015084 wait complete. 2023-11-28T11:21:41.083 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:20:41.925237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:20:48.682497+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:20:54.471398+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:20:55.204585+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T11:20:58.297307+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T11:21:41.084 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:21:41.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.281+0000 7fc6f29ff700 1 Processor -- start 2023-11-28T11:21:41.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.281+0000 7fc6f29ff700 1 -- start start 2023-11-28T11:21:41.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.282+0000 7fc6f29ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc6ec0cd460 0x7fc6ec06c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:41.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.282+0000 7fc6f29ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc6ec0d0d60 0x7fc6ec0d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:41.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.282+0000 7fc6f29ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6ec0b88e0 con 0x7fc6ec0b8510 2023-11-28T11:21:41.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.282+0000 7fc6f29ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6ec0cdbd0 con 0x7fc6ec0cd460 2023-11-28T11:21:41.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.282+0000 7fc6f29ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6ec0d02d0 con 0x7fc6ec0d0d60 2023-11-28T11:21:41.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.283+0000 7fc6eb7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc6ec0cd460 0x7fc6ec06c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:41.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.283+0000 7fc6eb7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc6ec0cd460 0x7fc6ec06c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51278/0 (socket says 172.21.15.92:51278) 2023-11-28T11:21:41.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.283+0000 7fc6eb7fe700 1 -- 172.21.15.92:0/3625090750 learned_addr learned my addr 172.21.15.92:0/3625090750 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:41.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.283+0000 7fc6f0f9c700 1 --2- 172.21.15.92:0/3625090750 >> v2:172.21.15.169:3300/0 conn(0x7fc6ec0d0d60 0x7fc6ec0d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:41.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.283+0000 7fc6eb7fe700 1 -- 172.21.15.92:0/3625090750 >> v2:172.21.15.169:3300/0 conn(0x7fc6ec0d0d60 msgr2=0x7fc6ec0d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:41.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.283+0000 7fc6eb7fe700 1 --2- 172.21.15.92:0/3625090750 >> v2:172.21.15.169:3300/0 conn(0x7fc6ec0d0d60 0x7fc6ec0d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:41.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.283+0000 7fc6eb7fe700 1 -- 172.21.15.92:0/3625090750 >> v1:172.21.15.169:6789/0 conn(0x7fc6ec0b8510 legacy=0x7fc6ec06b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:41.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.283+0000 7fc6eb7fe700 1 -- 172.21.15.92:0/3625090750 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6ec0cd8f0 con 0x7fc6ec0cd460 2023-11-28T11:21:41.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.283+0000 7fc6eb7fe700 1 --2- 172.21.15.92:0/3625090750 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6ec0cd460 0x7fc6ec06c080 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec0ce8c0 tx=0x7fc6e0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f096d4e33a8d7b89 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:41.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.284+0000 7fc6eaffd700 1 -- 172.21.15.92:0/3625090750 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e000b860 con 0x7fc6ec0cd460 2023-11-28T11:21:41.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.284+0000 7fc6eaffd700 1 -- 172.21.15.92:0/3625090750 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e000b9c0 con 0x7fc6ec0cd460 2023-11-28T11:21:41.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.284+0000 7fc6eaffd700 1 -- 172.21.15.92:0/3625090750 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e000bd50 con 0x7fc6ec0cd460 2023-11-28T11:21:41.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.284+0000 7fc6f29ff700 1 -- 172.21.15.92:0/3625090750 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6ec0cd460 msgr2=0x7fc6ec06c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:41.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.284+0000 7fc6f29ff700 1 --2- 172.21.15.92:0/3625090750 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6ec0cd460 0x7fc6ec06c080 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec0ce8c0 tx=0x7fc6e0009c00 comp rx=0 tx=0).stop 2023-11-28T11:21:41.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.284+0000 7fc6f29ff700 1 -- 172.21.15.92:0/3625090750 shutdown_connections 2023-11-28T11:21:41.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.284+0000 7fc6f29ff700 1 --2- 172.21.15.92:0/3625090750 >> v2:172.21.15.169:3300/0 conn(0x7fc6ec0d0d60 0x7fc6ec0d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:41.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.284+0000 7fc6f29ff700 1 --2- 172.21.15.92:0/3625090750 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6ec0cd460 0x7fc6ec06c080 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:41.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.284+0000 7fc6f29ff700 1 -- 172.21.15.92:0/3625090750 >> 172.21.15.92:0/3625090750 conn(0x7fc6ec0719a0 msgr2=0x7fc6ec071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:41.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.284+0000 7fc6f29ff700 1 -- 172.21.15.92:0/3625090750 shutdown_connections 2023-11-28T11:21:41.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.285+0000 7fc6f29ff700 1 -- 172.21.15.92:0/3625090750 wait complete. 2023-11-28T11:21:41.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.286+0000 7fc6f29ff700 1 Processor -- start 2023-11-28T11:21:41.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.286+0000 7fc6f29ff700 1 -- start start 2023-11-28T11:21:41.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.287+0000 7fc6f29ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6ec0b8510 0x7fc6ec1e2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:41.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.287+0000 7fc6f29ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6ec0cd460 0x7fc6ec1e4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:41.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.287+0000 7fc6f29ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6ec0d0d60 0x7fc6ec15a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:41.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.287+0000 7fc6f29ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ec1e1ca0 con 0x7fc6ec0b8510 2023-11-28T11:21:41.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.287+0000 7fc6f29ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ec1e1f80 con 0x7fc6ec0cd460 2023-11-28T11:21:41.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.287+0000 7fc6f29ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ec1e2260 con 0x7fc6ec0d0d60 2023-11-28T11:21:41.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.287+0000 7fc6ebfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6ec0b8510 0x7fc6ec1e2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:41.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.288+0000 7fc6ebfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6ec0b8510 0x7fc6ec1e2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51282/0 (socket says 172.21.15.92:51282) 2023-11-28T11:21:41.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.288+0000 7fc6ebfff700 1 -- 172.21.15.92:0/4022599127 learned_addr learned my addr 172.21.15.92:0/4022599127 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:41.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.288+0000 7fc6f0f9c700 1 --2- 172.21.15.92:0/4022599127 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6ec0d0d60 0x7fc6ec15a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:41.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.288+0000 7fc6eb7fe700 1 --2- 172.21.15.92:0/4022599127 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6ec0cd460 0x7fc6ec1e4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:41.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.288+0000 7fc6ebfff700 1 -- 172.21.15.92:0/4022599127 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6ec0d0d60 msgr2=0x7fc6ec15a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:41.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.288+0000 7fc6ebfff700 1 --2- 172.21.15.92:0/4022599127 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6ec0d0d60 0x7fc6ec15a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:41.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.288+0000 7fc6ebfff700 1 -- 172.21.15.92:0/4022599127 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6ec0cd460 msgr2=0x7fc6ec1e4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:41.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.288+0000 7fc6ebfff700 1 --2- 172.21.15.92:0/4022599127 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6ec0cd460 0x7fc6ec1e4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:41.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.288+0000 7fc6ebfff700 1 -- 172.21.15.92:0/4022599127 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e0009550 con 0x7fc6ec0b8510 2023-11-28T11:21:41.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.288+0000 7fc6f0f9c700 1 --2- 172.21.15.92:0/4022599127 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6ec0d0d60 0x7fc6ec15a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:21:41.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.288+0000 7fc6eb7fe700 1 --2- 172.21.15.92:0/4022599127 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6ec0cd460 0x7fc6ec1e4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:21:41.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.288+0000 7fc6ebfff700 1 --2- 172.21.15.92:0/4022599127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6ec0b8510 0x7fc6ec1e2d40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc00c4e0 tx=0x7fc6dc00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:41.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.288+0000 7fc6e8ff9700 1 -- 172.21.15.92:0/4022599127 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc0052d0 con 0x7fc6ec0b8510 2023-11-28T11:21:41.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.288+0000 7fc6e8ff9700 1 -- 172.21.15.92:0/4022599127 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6dc005430 con 0x7fc6ec0b8510 2023-11-28T11:21:41.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.288+0000 7fc6f29ff700 1 -- 172.21.15.92:0/4022599127 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6ec0cfff0 con 0x7fc6ec0b8510 2023-11-28T11:21:41.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.288+0000 7fc6e8ff9700 1 -- 172.21.15.92:0/4022599127 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc0038c0 con 0x7fc6ec0b8510 2023-11-28T11:21:41.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.288+0000 7fc6f29ff700 1 -- 172.21.15.92:0/4022599127 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6ec0cfd10 con 0x7fc6ec0b8510 2023-11-28T11:21:41.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.290+0000 7fc6f29ff700 1 -- 172.21.15.92:0/4022599127 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6ec067880 con 0x7fc6ec0b8510 2023-11-28T11:21:41.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.290+0000 7fc6e8ff9700 1 -- 172.21.15.92:0/4022599127 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc6dc019030 con 0x7fc6ec0b8510 2023-11-28T11:21:41.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.292+0000 7fc6e8ff9700 1 --2- 172.21.15.92:0/4022599127 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc6d80682f0 0x7fc6d806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:41.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.292+0000 7fc6e8ff9700 1 -- 172.21.15.92:0/4022599127 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fc6dc085370 con 0x7fc6ec0b8510 2023-11-28T11:21:41.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.292+0000 7fc6eb7fe700 1 --2- 172.21.15.92:0/4022599127 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc6d80682f0 0x7fc6d806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:41.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.292+0000 7fc6eb7fe700 1 --2- 172.21.15.92:0/4022599127 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc6d80682f0 0x7fc6d806a7a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec0b1de0 tx=0x7fc6e0008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:41.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.296+0000 7fc6e8ff9700 1 -- 172.21.15.92:0/4022599127 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc6dc0518a0 con 0x7fc6ec0b8510 2023-11-28T11:21:41.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.487+0000 7fc6f29ff700 1 -- 172.21.15.92:0/4022599127 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc6ec0d2910 con 0x7fc6ec0b8510 2023-11-28T11:21:41.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.488+0000 7fc6e8ff9700 1 -- 172.21.15.92:0/4022599127 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc6dc054ec0 con 0x7fc6ec0b8510 2023-11-28T11:21:41.489 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-11-28T11:21:41.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.490+0000 7fc6f29ff700 1 -- 172.21.15.92:0/4022599127 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc6d80682f0 msgr2=0x7fc6d806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:41.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.490+0000 7fc6f29ff700 1 --2- 172.21.15.92:0/4022599127 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc6d80682f0 0x7fc6d806a7a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec0b1de0 tx=0x7fc6e0008030 comp rx=0 tx=0).stop 2023-11-28T11:21:41.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.490+0000 7fc6f29ff700 1 -- 172.21.15.92:0/4022599127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6ec0b8510 msgr2=0x7fc6ec1e2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:41.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.490+0000 7fc6f29ff700 1 --2- 172.21.15.92:0/4022599127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6ec0b8510 0x7fc6ec1e2d40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc00c4e0 tx=0x7fc6dc00c8a0 comp rx=0 tx=0).stop 2023-11-28T11:21:41.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.490+0000 7fc6f29ff700 1 -- 172.21.15.92:0/4022599127 shutdown_connections 2023-11-28T11:21:41.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.490+0000 7fc6f29ff700 1 --2- 172.21.15.92:0/4022599127 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc6ec0d0d60 0x7fc6ec15a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:41.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.490+0000 7fc6f29ff700 1 --2- 172.21.15.92:0/4022599127 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc6d80682f0 0x7fc6d806a7a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:41.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.490+0000 7fc6f29ff700 1 --2- 172.21.15.92:0/4022599127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6ec0b8510 0x7fc6ec1e2d40 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:41.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.490+0000 7fc6f29ff700 1 --2- 172.21.15.92:0/4022599127 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc6ec0cd460 0x7fc6ec1e4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:41.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.490+0000 7fc6f29ff700 1 -- 172.21.15.92:0/4022599127 >> 172.21.15.92:0/4022599127 conn(0x7fc6ec0719a0 msgr2=0x7fc6ec0b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:41.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.490+0000 7fc6f29ff700 1 -- 172.21.15.92:0/4022599127 shutdown_connections 2023-11-28T11:21:41.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.490+0000 7fc6f29ff700 1 -- 172.21.15.92:0/4022599127 wait complete. 2023-11-28T11:21:41.509 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:21:41.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.708+0000 7f8e045ff700 1 Processor -- start 2023-11-28T11:21:41.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.708+0000 7f8e045ff700 1 -- start start 2023-11-28T11:21:41.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.710+0000 7f8e045ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8dfc0cd460 0x7f8dfc06c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:41.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.710+0000 7f8e045ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dfc0b88e0 con 0x7f8dfc0d0d60 2023-11-28T11:21:41.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.710+0000 7f8e045ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dfc0cd8f0 con 0x7f8dfc0b8510 2023-11-28T11:21:41.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.710+0000 7f8e045ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dfc0cdbd0 con 0x7f8dfc0cd460 2023-11-28T11:21:41.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.710+0000 7f8e02b9c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f8dfc0d0d60 0x7f8dfc0d11e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:58002/0 (socket says 172.21.15.92:58002) 2023-11-28T11:21:41.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.710+0000 7f8e02b9c700 1 -- 172.21.15.92:0/1728146372 learned_addr learned my addr 172.21.15.92:0/1728146372 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:41.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.710+0000 7f8e01b9a700 1 --2- 172.21.15.92:0/1728146372 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8dfc0cd460 0x7f8dfc06c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:41.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.710+0000 7f8e01399700 1 -- 172.21.15.92:0/1728146372 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3655107417 0 0) 0x7f8dfc0b88e0 con 0x7f8dfc0d0d60 2023-11-28T11:21:41.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.710+0000 7f8e01399700 1 -- 172.21.15.92:0/1728146372 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8de80035b0 con 0x7f8dfc0d0d60 2023-11-28T11:21:41.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.710+0000 7f8e01399700 1 -- 172.21.15.92:0/1728146372 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1275254227 0 0) 0x7f8dfc0cd8f0 con 0x7f8dfc0b8510 2023-11-28T11:21:41.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.710+0000 7f8e01399700 1 -- 172.21.15.92:0/1728146372 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8dfc0b88e0 con 0x7f8dfc0b8510 2023-11-28T11:21:41.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.711+0000 7f8e01b9a700 1 -- 172.21.15.92:0/1728146372 >> v1:172.21.15.169:6789/0 conn(0x7f8dfc0b8510 legacy=0x7f8dfc06b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:41.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.711+0000 7f8e01b9a700 1 -- 172.21.15.92:0/1728146372 >> v1:172.21.15.92:6789/0 conn(0x7f8dfc0d0d60 legacy=0x7f8dfc0d11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:41.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.711+0000 7f8e01b9a700 1 -- 172.21.15.92:0/1728146372 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dfc0cfd10 con 0x7f8dfc0cd460 2023-11-28T11:21:41.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.711+0000 7f8e01b9a700 1 --2- 172.21.15.92:0/1728146372 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8dfc0cd460 0x7f8dfc06c080 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8dec009870 tx=0x7f8dec009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a289e9af8e7d859 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:41.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.711+0000 7f8e01399700 1 -- 172.21.15.92:0/1728146372 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dec011830 con 0x7f8dfc0cd460 2023-11-28T11:21:41.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.711+0000 7f8e01399700 1 -- 172.21.15.92:0/1728146372 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dec011990 con 0x7f8dfc0cd460 2023-11-28T11:21:41.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.711+0000 7f8e01399700 1 -- 172.21.15.92:0/1728146372 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dec011d00 con 0x7f8dfc0cd460 2023-11-28T11:21:41.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.712+0000 7f8e045ff700 1 -- 172.21.15.92:0/1728146372 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8dfc0cd460 msgr2=0x7f8dfc06c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:41.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.712+0000 7f8e045ff700 1 --2- 172.21.15.92:0/1728146372 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8dfc0cd460 0x7f8dfc06c080 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8dec009870 tx=0x7f8dec009b80 comp rx=0 tx=0).stop 2023-11-28T11:21:41.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.712+0000 7f8e045ff700 1 -- 172.21.15.92:0/1728146372 shutdown_connections 2023-11-28T11:21:41.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.712+0000 7f8e045ff700 1 --2- 172.21.15.92:0/1728146372 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8dfc0cd460 0x7f8dfc06c080 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:41.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.712+0000 7f8e045ff700 1 -- 172.21.15.92:0/1728146372 >> 172.21.15.92:0/1728146372 conn(0x7f8dfc0719a0 msgr2=0x7f8dfc071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:41.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.712+0000 7f8e045ff700 1 -- 172.21.15.92:0/1728146372 shutdown_connections 2023-11-28T11:21:41.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.712+0000 7f8e045ff700 1 -- 172.21.15.92:0/1728146372 wait complete. 2023-11-28T11:21:41.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.714+0000 7f8e045ff700 1 Processor -- start 2023-11-28T11:21:41.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.714+0000 7f8e045ff700 1 -- start start 2023-11-28T11:21:41.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.715+0000 7f8e045ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8dfc0b8510 0x7f8dfc159eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:41.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.715+0000 7f8e045ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8dfc0cd460 0x7f8dfc14d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:41.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.715+0000 7f8e045ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8dfc0d0d60 0x7f8dfc0ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:41.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.715+0000 7f8e045ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dfc14d900 con 0x7f8dfc0cd460 2023-11-28T11:21:41.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.715+0000 7f8e045ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dfc1f6340 con 0x7f8dfc0d0d60 2023-11-28T11:21:41.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.715+0000 7f8e045ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dfc1f5730 con 0x7f8dfc0b8510 2023-11-28T11:21:41.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.715+0000 7f8e01b9a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8dfc0cd460 0x7f8dfc14d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:41.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.715+0000 7f8e01b9a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8dfc0cd460 0x7f8dfc14d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51286/0 (socket says 172.21.15.92:51286) 2023-11-28T11:21:41.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.715+0000 7f8e01b9a700 1 -- 172.21.15.92:0/1511886706 learned_addr learned my addr 172.21.15.92:0/1511886706 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:41.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.715+0000 7f8e0239b700 1 --2- 172.21.15.92:0/1511886706 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8dfc0b8510 0x7f8dfc159eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:41.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.715+0000 7f8e02b9c700 1 --2- 172.21.15.92:0/1511886706 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8dfc0d0d60 0x7f8dfc0ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:41.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.715+0000 7f8e01b9a700 1 -- 172.21.15.92:0/1511886706 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8dfc0b8510 msgr2=0x7f8dfc159eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:41.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.715+0000 7f8e01b9a700 1 --2- 172.21.15.92:0/1511886706 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8dfc0b8510 0x7f8dfc159eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:41.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.715+0000 7f8e01b9a700 1 -- 172.21.15.92:0/1511886706 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8dfc0d0d60 msgr2=0x7f8dfc0ccc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:41.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.715+0000 7f8e01b9a700 1 --2- 172.21.15.92:0/1511886706 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8dfc0d0d60 0x7f8dfc0ccc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:41.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.715+0000 7f8e01b9a700 1 -- 172.21.15.92:0/1511886706 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dec0094a0 con 0x7f8dfc0cd460 2023-11-28T11:21:41.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.715+0000 7f8e02b9c700 1 --2- 172.21.15.92:0/1511886706 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8dfc0d0d60 0x7f8dfc0ccc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:21:41.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.716+0000 7f8e01b9a700 1 --2- 172.21.15.92:0/1511886706 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8dfc0cd460 0x7f8dfc14d3c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8dec0097c0 tx=0x7f8dec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:41.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.716+0000 7f8df2ffd700 1 -- 172.21.15.92:0/1511886706 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dec0044d0 con 0x7f8dfc0cd460 2023-11-28T11:21:41.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.716+0000 7f8df2ffd700 1 -- 172.21.15.92:0/1511886706 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dec01f070 con 0x7f8dfc0cd460 2023-11-28T11:21:41.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.716+0000 7f8df2ffd700 1 -- 172.21.15.92:0/1511886706 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dec0038e0 con 0x7f8dfc0cd460 2023-11-28T11:21:41.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.716+0000 7f8e045ff700 1 -- 172.21.15.92:0/1511886706 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dfc0d29c0 con 0x7f8dfc0cd460 2023-11-28T11:21:41.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.716+0000 7f8e045ff700 1 -- 172.21.15.92:0/1511886706 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dfc0cfff0 con 0x7f8dfc0cd460 2023-11-28T11:21:41.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.717+0000 7f8e045ff700 1 -- 172.21.15.92:0/1511886706 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dfc0d2b50 con 0x7f8dfc0cd460 2023-11-28T11:21:41.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.718+0000 7f8df2ffd700 1 -- 172.21.15.92:0/1511886706 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8dec00f050 con 0x7f8dfc0cd460 2023-11-28T11:21:41.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.719+0000 7f8df2ffd700 1 --2- 172.21.15.92:0/1511886706 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8de00682f0 0x7f8de006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:41.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.719+0000 7f8df2ffd700 1 -- 172.21.15.92:0/1511886706 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f8dec021070 con 0x7f8dfc0cd460 2023-11-28T11:21:41.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.719+0000 7f8e0239b700 1 --2- 172.21.15.92:0/1511886706 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8de00682f0 0x7f8de006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:41.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.720+0000 7f8e0239b700 1 --2- 172.21.15.92:0/1511886706 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8de00682f0 0x7f8de006a7a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8df80011e0 tx=0x7f8df8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:41.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.723+0000 7f8df2ffd700 1 -- 172.21.15.92:0/1511886706 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8dec053c50 con 0x7f8dfc0cd460 2023-11-28T11:21:41.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.915+0000 7f8e045ff700 1 -- 172.21.15.92:0/1511886706 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8dfc1585d0 con 0x7f8dfc0cd460 2023-11-28T11:21:41.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.915+0000 7f8df2ffd700 1 -- 172.21.15.92:0/1511886706 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8dfc1585d0 con 0x7f8dfc0cd460 2023-11-28T11:21:41.917 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-11-28T11:21:41.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.917+0000 7f8e045ff700 1 -- 172.21.15.92:0/1511886706 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8de00682f0 msgr2=0x7f8de006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:41.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.917+0000 7f8e045ff700 1 --2- 172.21.15.92:0/1511886706 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8de00682f0 0x7f8de006a7a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8df80011e0 tx=0x7f8df8009040 comp rx=0 tx=0).stop 2023-11-28T11:21:41.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.917+0000 7f8e045ff700 1 -- 172.21.15.92:0/1511886706 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8dfc0cd460 msgr2=0x7f8dfc14d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:41.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.917+0000 7f8e045ff700 1 --2- 172.21.15.92:0/1511886706 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8dfc0cd460 0x7f8dfc14d3c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8dec0097c0 tx=0x7f8dec004750 comp rx=0 tx=0).stop 2023-11-28T11:21:41.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.917+0000 7f8e045ff700 1 -- 172.21.15.92:0/1511886706 shutdown_connections 2023-11-28T11:21:41.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.917+0000 7f8e045ff700 1 --2- 172.21.15.92:0/1511886706 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8dfc0d0d60 0x7f8dfc0ccc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:41.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.917+0000 7f8e045ff700 1 --2- 172.21.15.92:0/1511886706 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8dfc0b8510 0x7f8dfc159eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:41.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.917+0000 7f8e045ff700 1 --2- 172.21.15.92:0/1511886706 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8de00682f0 0x7f8de006a7a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:41.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.917+0000 7f8e045ff700 1 --2- 172.21.15.92:0/1511886706 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8dfc0cd460 0x7f8dfc14d3c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:41.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.918+0000 7f8e045ff700 1 -- 172.21.15.92:0/1511886706 >> 172.21.15.92:0/1511886706 conn(0x7f8dfc0719a0 msgr2=0x7f8dfc15c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:41.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.918+0000 7f8e045ff700 1 -- 172.21.15.92:0/1511886706 shutdown_connections 2023-11-28T11:21:41.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:41.918+0000 7f8e045ff700 1 -- 172.21.15.92:0/1511886706 wait complete. 2023-11-28T11:21:41.937 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:21:42.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.135+0000 7ff73fbff700 1 Processor -- start 2023-11-28T11:21:42.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.135+0000 7ff73fbff700 1 -- start start 2023-11-28T11:21:42.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.137+0000 7ff73fbff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7380b4fc0 0x7ff7380cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:42.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.137+0000 7ff73fbff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff7380b1c60 0x7ff7380cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:42.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.137+0000 7ff73fbff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7380d1950 con 0x7ff7380d2df0 2023-11-28T11:21:42.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.137+0000 7ff73fbff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7380d3d30 con 0x7ff7380b1c60 2023-11-28T11:21:42.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.137+0000 7ff73fbff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7380b2500 con 0x7ff7380b4fc0 2023-11-28T11:21:42.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.137+0000 7ff73d99b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff7380d2df0 0x7ff7380d31c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:58014/0 (socket says 172.21.15.92:58014) 2023-11-28T11:21:42.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.137+0000 7ff73d99b700 1 -- 172.21.15.92:0/3050112733 learned_addr learned my addr 172.21.15.92:0/3050112733 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:42.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.137+0000 7ff73d19a700 1 --2- 172.21.15.92:0/3050112733 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7380b4fc0 0x7ff7380cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:42.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.137+0000 7ff73e19c700 1 --2- 172.21.15.92:0/3050112733 >> v2:172.21.15.169:3300/0 conn(0x7ff7380b1c60 0x7ff7380cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:42.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.137+0000 7ff73c999700 1 -- 172.21.15.92:0/3050112733 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2438448793 0 0) 0x7ff7380d1950 con 0x7ff7380d2df0 2023-11-28T11:21:42.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.137+0000 7ff73c999700 1 -- 172.21.15.92:0/3050112733 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7240035b0 con 0x7ff7380d2df0 2023-11-28T11:21:42.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.138+0000 7ff73c999700 1 -- 172.21.15.92:0/3050112733 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2710324214 0 0) 0x7ff7240035b0 con 0x7ff7380d2df0 2023-11-28T11:21:42.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.138+0000 7ff73c999700 1 -- 172.21.15.92:0/3050112733 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7380b4fc0 msgr2=0x7ff7380cd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:42.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.138+0000 7ff73c999700 1 --2- 172.21.15.92:0/3050112733 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7380b4fc0 0x7ff7380cd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:42.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.138+0000 7ff73c999700 1 -- 172.21.15.92:0/3050112733 >> v2:172.21.15.169:3300/0 conn(0x7ff7380b1c60 msgr2=0x7ff7380cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:42.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.138+0000 7ff73c999700 1 --2- 172.21.15.92:0/3050112733 >> v2:172.21.15.169:3300/0 conn(0x7ff7380b1c60 0x7ff7380cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:42.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.138+0000 7ff73c999700 1 -- 172.21.15.92:0/3050112733 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7380d3a50 con 0x7ff7380d2df0 2023-11-28T11:21:42.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.138+0000 7ff73c999700 1 -- 172.21.15.92:0/3050112733 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532161587 0 0) 0x7ff734001670 con 0x7ff7380d2df0 2023-11-28T11:21:42.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.138+0000 7ff73c999700 1 -- 172.21.15.92:0/3050112733 >> v1:172.21.15.92:6789/0 conn(0x7ff7380d2df0 legacy=0x7ff7380d31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:42.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.138+0000 7ff73c999700 1 --2- 172.21.15.92:0/3050112733 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7380b4fc0 0x7ff724003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:42.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.138+0000 7ff73c999700 1 --2- 172.21.15.92:0/3050112733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff724003f80 0x7ff724004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:42.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.138+0000 7ff73c999700 1 --2- 172.21.15.92:0/3050112733 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff724004d90 0x7ff724005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:42.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.138+0000 7ff73c999700 1 -- 172.21.15.92:0/3050112733 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7380d3d30 con 0x7ff724003f80 2023-11-28T11:21:42.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.138+0000 7ff73c999700 1 -- 172.21.15.92:0/3050112733 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7380b2500 con 0x7ff724004d90 2023-11-28T11:21:42.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.138+0000 7ff73c999700 1 -- 172.21.15.92:0/3050112733 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff72400ba10 con 0x7ff7380b4fc0 2023-11-28T11:21:42.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.138+0000 7ff73d19a700 1 --2- 172.21.15.92:0/3050112733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff724003f80 0x7ff724004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:42.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.138+0000 7ff73d99b700 1 --2- 172.21.15.92:0/3050112733 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7380b4fc0 0x7ff724003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:42.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.138+0000 7ff73e19c700 1 --2- 172.21.15.92:0/3050112733 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff724004d90 0x7ff724005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:42.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.138+0000 7ff73d19a700 1 -- 172.21.15.92:0/3050112733 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7380b4fc0 msgr2=0x7ff724003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:42.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.138+0000 7ff73d19a700 1 --2- 172.21.15.92:0/3050112733 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff7380b4fc0 0x7ff724003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:42.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.138+0000 7ff73d19a700 1 -- 172.21.15.92:0/3050112733 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff724004d90 msgr2=0x7ff724005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:42.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.138+0000 7ff73d19a700 1 --2- 172.21.15.92:0/3050112733 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff724004d90 0x7ff724005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:42.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.138+0000 7ff73d19a700 1 -- 172.21.15.92:0/3050112733 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7240035b0 con 0x7ff724003f80 2023-11-28T11:21:42.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.139+0000 7ff73d19a700 1 --2- 172.21.15.92:0/3050112733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff724003f80 0x7ff724004850 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff728009b30 tx=0x7ff724005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=384d535a580224b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:42.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.139+0000 7ff73c999700 1 -- 172.21.15.92:0/3050112733 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72800e6c0 con 0x7ff724003f80 2023-11-28T11:21:42.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.139+0000 7ff73c999700 1 -- 172.21.15.92:0/3050112733 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff72800e820 con 0x7ff724003f80 2023-11-28T11:21:42.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.139+0000 7ff73c999700 1 -- 172.21.15.92:0/3050112733 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72800eb90 con 0x7ff724003f80 2023-11-28T11:21:42.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.139+0000 7ff73fbff700 1 -- 172.21.15.92:0/3050112733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff724003f80 msgr2=0x7ff724004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:42.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.139+0000 7ff73fbff700 1 --2- 172.21.15.92:0/3050112733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff724003f80 0x7ff724004850 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff728009b30 tx=0x7ff724005880 comp rx=0 tx=0).stop 2023-11-28T11:21:42.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.139+0000 7ff73e19c700 1 -- 172.21.15.92:0/3050112733 reap_dead start 2023-11-28T11:21:42.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.139+0000 7ff73fbff700 1 -- 172.21.15.92:0/3050112733 shutdown_connections 2023-11-28T11:21:42.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.139+0000 7ff73fbff700 1 -- 172.21.15.92:0/3050112733 >> 172.21.15.92:0/3050112733 conn(0x7ff7380719a0 msgr2=0x7ff738071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:42.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.139+0000 7ff73fbff700 1 -- 172.21.15.92:0/3050112733 shutdown_connections 2023-11-28T11:21:42.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.139+0000 7ff73fbff700 1 -- 172.21.15.92:0/3050112733 wait complete. 2023-11-28T11:21:42.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.142+0000 7ff73fbff700 1 Processor -- start 2023-11-28T11:21:42.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.142+0000 7ff73fbff700 1 -- start start 2023-11-28T11:21:42.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.143+0000 7ff73fbff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff724003f80 0x7ff7380cc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:42.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.143+0000 7ff73fbff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff738156c70 0x7ff7380c8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:42.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.143+0000 7ff73fbff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7381570e0 0x7ff7380c90c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:42.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.143+0000 7ff73fbff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73815cde0 con 0x7ff738156c70 2023-11-28T11:21:42.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.143+0000 7ff73fbff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73815d0c0 con 0x7ff7381570e0 2023-11-28T11:21:42.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.143+0000 7ff73fbff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff73815d3a0 con 0x7ff724003f80 2023-11-28T11:21:42.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.144+0000 7ff73d19a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff738156c70 0x7ff7380c8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:42.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.144+0000 7ff73d19a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff738156c70 0x7ff7380c8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51314/0 (socket says 172.21.15.92:51314) 2023-11-28T11:21:42.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.144+0000 7ff73d19a700 1 -- 172.21.15.92:0/129710385 learned_addr learned my addr 172.21.15.92:0/129710385 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:42.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.144+0000 7ff73d99b700 1 --2- 172.21.15.92:0/129710385 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff724003f80 0x7ff7380cc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:42.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.144+0000 7ff73e19c700 1 --2- 172.21.15.92:0/129710385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7381570e0 0x7ff7380c90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:42.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.144+0000 7ff73d19a700 1 -- 172.21.15.92:0/129710385 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff724003f80 msgr2=0x7ff7380cc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:42.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.144+0000 7ff73d19a700 1 --2- 172.21.15.92:0/129710385 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff724003f80 0x7ff7380cc1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:42.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.144+0000 7ff73d19a700 1 -- 172.21.15.92:0/129710385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7381570e0 msgr2=0x7ff7380c90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:42.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.144+0000 7ff73d19a700 1 --2- 172.21.15.92:0/129710385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7381570e0 0x7ff7380c90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:42.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.144+0000 7ff73d19a700 1 -- 172.21.15.92:0/129710385 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7380b88f0 con 0x7ff738156c70 2023-11-28T11:21:42.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.144+0000 7ff73d99b700 1 --2- 172.21.15.92:0/129710385 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff724003f80 0x7ff7380cc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:21:42.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.144+0000 7ff73e19c700 1 --2- 172.21.15.92:0/129710385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7381570e0 0x7ff7380c90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:21:42.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.144+0000 7ff73d19a700 1 --2- 172.21.15.92:0/129710385 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff738156c70 0x7ff7380c8b80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff7240046d0 tx=0x7ff7280047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:42.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.144+0000 7ff72e7fc700 1 -- 172.21.15.92:0/129710385 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72800e6c0 con 0x7ff738156c70 2023-11-28T11:21:42.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.145+0000 7ff72e7fc700 1 -- 172.21.15.92:0/129710385 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff728013e30 con 0x7ff738156c70 2023-11-28T11:21:42.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.145+0000 7ff72e7fc700 1 -- 172.21.15.92:0/129710385 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff728028690 con 0x7ff738156c70 2023-11-28T11:21:42.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.145+0000 7ff73fbff700 1 -- 172.21.15.92:0/129710385 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7380d15e0 con 0x7ff738156c70 2023-11-28T11:21:42.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.145+0000 7ff73fbff700 1 -- 172.21.15.92:0/129710385 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7380b21b0 con 0x7ff738156c70 2023-11-28T11:21:42.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.146+0000 7ff73fbff700 1 -- 172.21.15.92:0/129710385 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff738067170 con 0x7ff738156c70 2023-11-28T11:21:42.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.146+0000 7ff72e7fc700 1 -- 172.21.15.92:0/129710385 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff728025070 con 0x7ff738156c70 2023-11-28T11:21:42.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.147+0000 7ff72e7fc700 1 --2- 172.21.15.92:0/129710385 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff71c0682f0 0x7ff71c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:42.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.147+0000 7ff72e7fc700 1 -- 172.21.15.92:0/129710385 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7ff7280878c0 con 0x7ff738156c70 2023-11-28T11:21:42.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.147+0000 7ff73d99b700 1 --2- 172.21.15.92:0/129710385 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff71c0682f0 0x7ff71c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:42.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.148+0000 7ff73d99b700 1 --2- 172.21.15.92:0/129710385 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff71c0682f0 0x7ff71c06a7a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff734001eb0 tx=0x7ff734009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:42.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.151+0000 7ff72e7fc700 1 -- 172.21.15.92:0/129710385 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff728053db0 con 0x7ff738156c70 2023-11-28T11:21:42.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.340+0000 7ff73fbff700 1 -- 172.21.15.92:0/129710385 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff73808aa60 con 0x7ff738156c70 2023-11-28T11:21:42.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.340+0000 7ff72e7fc700 1 -- 172.21.15.92:0/129710385 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff7280573d0 con 0x7ff738156c70 2023-11-28T11:21:42.342 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-11-28T11:21:42.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.342+0000 7ff73fbff700 1 -- 172.21.15.92:0/129710385 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff71c0682f0 msgr2=0x7ff71c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:42.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.342+0000 7ff73fbff700 1 --2- 172.21.15.92:0/129710385 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff71c0682f0 0x7ff71c06a7a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff734001eb0 tx=0x7ff734009040 comp rx=0 tx=0).stop 2023-11-28T11:21:42.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.342+0000 7ff73fbff700 1 -- 172.21.15.92:0/129710385 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff738156c70 msgr2=0x7ff7380c8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:42.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.342+0000 7ff73fbff700 1 --2- 172.21.15.92:0/129710385 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff738156c70 0x7ff7380c8b80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff7240046d0 tx=0x7ff7280047c0 comp rx=0 tx=0).stop 2023-11-28T11:21:42.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.342+0000 7ff73fbff700 1 -- 172.21.15.92:0/129710385 shutdown_connections 2023-11-28T11:21:42.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.342+0000 7ff73fbff700 1 --2- 172.21.15.92:0/129710385 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff7381570e0 0x7ff7380c90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:42.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.342+0000 7ff73fbff700 1 --2- 172.21.15.92:0/129710385 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff724003f80 0x7ff7380cc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:42.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.342+0000 7ff73fbff700 1 --2- 172.21.15.92:0/129710385 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff71c0682f0 0x7ff71c06a7a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:42.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.342+0000 7ff73fbff700 1 --2- 172.21.15.92:0/129710385 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff738156c70 0x7ff7380c8b80 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:42.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.342+0000 7ff73fbff700 1 -- 172.21.15.92:0/129710385 >> 172.21.15.92:0/129710385 conn(0x7ff7380719a0 msgr2=0x7ff7380b7b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:42.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.342+0000 7ff73fbff700 1 -- 172.21.15.92:0/129710385 shutdown_connections 2023-11-28T11:21:42.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.343+0000 7ff73fbff700 1 -- 172.21.15.92:0/129710385 wait complete. 2023-11-28T11:21:42.362 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:21:42.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.561+0000 7ff389bff700 1 Processor -- start 2023-11-28T11:21:42.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.561+0000 7ff389bff700 1 -- start start 2023-11-28T11:21:42.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.563+0000 7ff389bff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff3840cd460 0x7ff38406c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:42.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.563+0000 7ff389bff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3840d0d60 0x7ff3840d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:42.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.563+0000 7ff389bff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3840b88e0 con 0x7ff3840b8510 2023-11-28T11:21:42.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.563+0000 7ff389bff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3840cdbd0 con 0x7ff3840cd460 2023-11-28T11:21:42.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.563+0000 7ff389bff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3840d02d0 con 0x7ff3840d0d60 2023-11-28T11:21:42.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.563+0000 7ff3837fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff3840b8510 0x7ff38406b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:58028/0 (socket says 172.21.15.92:58028) 2023-11-28T11:21:42.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.563+0000 7ff3837fe700 1 -- 172.21.15.92:0/3869157462 learned_addr learned my addr 172.21.15.92:0/3869157462 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:42.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.563+0000 7ff37bfff700 1 --2- 172.21.15.92:0/3869157462 >> v2:172.21.15.169:3300/0 conn(0x7ff3840cd460 0x7ff38406c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:42.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.563+0000 7ff383fff700 1 --2- 172.21.15.92:0/3869157462 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3840d0d60 0x7ff3840d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:42.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.563+0000 7ff382ffd700 1 -- 172.21.15.92:0/3869157462 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 207035418 0 0) 0x7ff3840b88e0 con 0x7ff3840b8510 2023-11-28T11:21:42.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.563+0000 7ff382ffd700 1 -- 172.21.15.92:0/3869157462 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3700035b0 con 0x7ff3840b8510 2023-11-28T11:21:42.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.563+0000 7ff37bfff700 1 -- 172.21.15.92:0/3869157462 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3840d0d60 msgr2=0x7ff3840d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:42.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.563+0000 7ff37bfff700 1 --2- 172.21.15.92:0/3869157462 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3840d0d60 0x7ff3840d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:42.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.563+0000 7ff37bfff700 1 -- 172.21.15.92:0/3869157462 >> v1:172.21.15.92:6789/0 conn(0x7ff3840b8510 legacy=0x7ff38406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:42.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.563+0000 7ff37bfff700 1 -- 172.21.15.92:0/3869157462 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3840cd8f0 con 0x7ff3840cd460 2023-11-28T11:21:42.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.564+0000 7ff37bfff700 1 --2- 172.21.15.92:0/3869157462 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3840cd460 0x7ff38406c080 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff36c009870 tx=0x7ff36c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a3a4c49a9e5b103 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:42.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.564+0000 7ff382ffd700 1 -- 172.21.15.92:0/3869157462 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36c011830 con 0x7ff3840cd460 2023-11-28T11:21:42.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.564+0000 7ff382ffd700 1 -- 172.21.15.92:0/3869157462 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff36c011990 con 0x7ff3840cd460 2023-11-28T11:21:42.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.564+0000 7ff382ffd700 1 -- 172.21.15.92:0/3869157462 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36c011d20 con 0x7ff3840cd460 2023-11-28T11:21:42.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.564+0000 7ff389bff700 1 -- 172.21.15.92:0/3869157462 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3840cd460 msgr2=0x7ff38406c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:42.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.564+0000 7ff389bff700 1 --2- 172.21.15.92:0/3869157462 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3840cd460 0x7ff38406c080 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff36c009870 tx=0x7ff36c009b80 comp rx=0 tx=0).stop 2023-11-28T11:21:42.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.565+0000 7ff389bff700 1 -- 172.21.15.92:0/3869157462 shutdown_connections 2023-11-28T11:21:42.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.565+0000 7ff389bff700 1 --2- 172.21.15.92:0/3869157462 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3840d0d60 0x7ff3840d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:42.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.565+0000 7ff389bff700 1 --2- 172.21.15.92:0/3869157462 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3840cd460 0x7ff38406c080 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:42.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.565+0000 7ff389bff700 1 -- 172.21.15.92:0/3869157462 >> 172.21.15.92:0/3869157462 conn(0x7ff3840719a0 msgr2=0x7ff384071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:42.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.565+0000 7ff389bff700 1 -- 172.21.15.92:0/3869157462 shutdown_connections 2023-11-28T11:21:42.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.565+0000 7ff389bff700 1 -- 172.21.15.92:0/3869157462 wait complete. 2023-11-28T11:21:42.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.566+0000 7ff389bff700 1 Processor -- start 2023-11-28T11:21:42.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.567+0000 7ff389bff700 1 -- start start 2023-11-28T11:21:42.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.567+0000 7ff389bff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3840b8510 0x7ff3840cb200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:42.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.567+0000 7ff389bff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3840cd460 0x7ff3841e5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:42.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.567+0000 7ff389bff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3840d0d60 0x7ff3841f1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:42.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.567+0000 7ff389bff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3841e5a80 con 0x7ff3840d0d60 2023-11-28T11:21:42.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.567+0000 7ff389bff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3840c9cf0 con 0x7ff3840b8510 2023-11-28T11:21:42.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.567+0000 7ff389bff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3841e4e70 con 0x7ff3840cd460 2023-11-28T11:21:42.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.568+0000 7ff383fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3840d0d60 0x7ff3841f1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:42.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.568+0000 7ff383fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3840d0d60 0x7ff3841f1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51324/0 (socket says 172.21.15.92:51324) 2023-11-28T11:21:42.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.568+0000 7ff383fff700 1 -- 172.21.15.92:0/2928624808 learned_addr learned my addr 172.21.15.92:0/2928624808 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:42.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.568+0000 7ff3837fe700 1 --2- 172.21.15.92:0/2928624808 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3840b8510 0x7ff3840cb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:42.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.568+0000 7ff37bfff700 1 --2- 172.21.15.92:0/2928624808 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3840cd460 0x7ff3841e5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:42.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.568+0000 7ff383fff700 1 -- 172.21.15.92:0/2928624808 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3840cd460 msgr2=0x7ff3841e5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:42.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.568+0000 7ff383fff700 1 --2- 172.21.15.92:0/2928624808 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3840cd460 0x7ff3841e5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:42.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.568+0000 7ff383fff700 1 -- 172.21.15.92:0/2928624808 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3840b8510 msgr2=0x7ff3840cb200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:42.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.568+0000 7ff383fff700 1 --2- 172.21.15.92:0/2928624808 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3840b8510 0x7ff3840cb200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:42.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.568+0000 7ff383fff700 1 -- 172.21.15.92:0/2928624808 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff36c0094a0 con 0x7ff3840d0d60 2023-11-28T11:21:42.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.568+0000 7ff383fff700 1 --2- 172.21.15.92:0/2928624808 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3840d0d60 0x7ff3841f1cc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff37c009d30 tx=0x7ff37c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:42.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.569+0000 7ff380ff9700 1 -- 172.21.15.92:0/2928624808 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37c009ba0 con 0x7ff3840d0d60 2023-11-28T11:21:42.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.569+0000 7ff380ff9700 1 -- 172.21.15.92:0/2928624808 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff37c00f040 con 0x7ff3840d0d60 2023-11-28T11:21:42.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.569+0000 7ff389bff700 1 -- 172.21.15.92:0/2928624808 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3840cfff0 con 0x7ff3840d0d60 2023-11-28T11:21:42.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.569+0000 7ff380ff9700 1 -- 172.21.15.92:0/2928624808 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37c014480 con 0x7ff3840d0d60 2023-11-28T11:21:42.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.569+0000 7ff389bff700 1 -- 172.21.15.92:0/2928624808 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3840cfd10 con 0x7ff3840d0d60 2023-11-28T11:21:42.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.570+0000 7ff389bff700 1 -- 172.21.15.92:0/2928624808 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff384067880 con 0x7ff3840d0d60 2023-11-28T11:21:42.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.571+0000 7ff380ff9700 1 -- 172.21.15.92:0/2928624808 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff37c0145e0 con 0x7ff3840d0d60 2023-11-28T11:21:42.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.571+0000 7ff380ff9700 1 --2- 172.21.15.92:0/2928624808 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff3640682f0 0x7ff36406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:42.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.572+0000 7ff380ff9700 1 -- 172.21.15.92:0/2928624808 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7ff37c086b80 con 0x7ff3840d0d60 2023-11-28T11:21:42.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.572+0000 7ff3837fe700 1 --2- 172.21.15.92:0/2928624808 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff3640682f0 0x7ff36406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:42.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.572+0000 7ff3837fe700 1 --2- 172.21.15.92:0/2928624808 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff3640682f0 0x7ff36406a7a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff3740011e0 tx=0x7ff374009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:42.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.576+0000 7ff380ff9700 1 -- 172.21.15.92:0/2928624808 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff37c052f40 con 0x7ff3840d0d60 2023-11-28T11:21:42.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.769+0000 7ff389bff700 1 -- 172.21.15.92:0/2928624808 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff3841e5bc0 con 0x7ff3840d0d60 2023-11-28T11:21:42.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.770+0000 7ff380ff9700 1 -- 172.21.15.92:0/2928624808 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff37c056560 con 0x7ff3840d0d60 2023-11-28T11:21:42.771 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-11-28T11:21:42.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.772+0000 7ff389bff700 1 -- 172.21.15.92:0/2928624808 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff3640682f0 msgr2=0x7ff36406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:42.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.772+0000 7ff389bff700 1 --2- 172.21.15.92:0/2928624808 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff3640682f0 0x7ff36406a7a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff3740011e0 tx=0x7ff374009040 comp rx=0 tx=0).stop 2023-11-28T11:21:42.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.772+0000 7ff389bff700 1 -- 172.21.15.92:0/2928624808 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3840d0d60 msgr2=0x7ff3841f1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:42.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.772+0000 7ff389bff700 1 --2- 172.21.15.92:0/2928624808 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3840d0d60 0x7ff3841f1cc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff37c009d30 tx=0x7ff37c00c3a0 comp rx=0 tx=0).stop 2023-11-28T11:21:42.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.772+0000 7ff389bff700 1 -- 172.21.15.92:0/2928624808 shutdown_connections 2023-11-28T11:21:42.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.772+0000 7ff389bff700 1 --2- 172.21.15.92:0/2928624808 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff3640682f0 0x7ff36406a7a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:42.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.772+0000 7ff389bff700 1 --2- 172.21.15.92:0/2928624808 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3840d0d60 0x7ff3841f1cc0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:42.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.772+0000 7ff389bff700 1 --2- 172.21.15.92:0/2928624808 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff3840b8510 0x7ff3840cb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:42.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.772+0000 7ff389bff700 1 --2- 172.21.15.92:0/2928624808 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff3840cd460 0x7ff3841e5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:42.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.772+0000 7ff389bff700 1 -- 172.21.15.92:0/2928624808 >> 172.21.15.92:0/2928624808 conn(0x7ff3840719a0 msgr2=0x7ff3840b6650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:42.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.772+0000 7ff389bff700 1 -- 172.21.15.92:0/2928624808 shutdown_connections 2023-11-28T11:21:42.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.772+0000 7ff389bff700 1 -- 172.21.15.92:0/2928624808 wait complete. 2023-11-28T11:21:42.791 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T11:21:42.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.989+0000 7fc8db1ff700 1 Processor -- start 2023-11-28T11:21:42.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.989+0000 7fc8db1ff700 1 -- start start 2023-11-28T11:21:42.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.990+0000 7fc8db1ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc8d40cd460 0x7fc8d406c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:42.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.990+0000 7fc8db1ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8d40b88e0 con 0x7fc8d40d0d60 2023-11-28T11:21:42.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.990+0000 7fc8db1ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8d40cd8f0 con 0x7fc8d40b8510 2023-11-28T11:21:42.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.990+0000 7fc8db1ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8d40cdbd0 con 0x7fc8d40cd460 2023-11-28T11:21:42.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.991+0000 7fc8d3fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc8d40cd460 0x7fc8d406c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:42.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.991+0000 7fc8d3fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc8d40cd460 0x7fc8d406c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51328/0 (socket says 172.21.15.92:51328) 2023-11-28T11:21:42.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.991+0000 7fc8d979c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc8d40d0d60 0x7fc8d40d1200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:58040/0 (socket says 172.21.15.92:58040) 2023-11-28T11:21:42.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.991+0000 7fc8d3fff700 1 -- 172.21.15.92:0/1060364023 learned_addr learned my addr 172.21.15.92:0/1060364023 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:42.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.991+0000 7fc8d3fff700 1 -- 172.21.15.92:0/1060364023 >> v1:172.21.15.169:6789/0 conn(0x7fc8d40b8510 legacy=0x7fc8d406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:42.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.991+0000 7fc8d3fff700 1 -- 172.21.15.92:0/1060364023 >> v1:172.21.15.92:6789/0 conn(0x7fc8d40d0d60 legacy=0x7fc8d40d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:42.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.991+0000 7fc8d3fff700 1 -- 172.21.15.92:0/1060364023 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8d40cfd10 con 0x7fc8d40cd460 2023-11-28T11:21:42.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.991+0000 7fc8d37fe700 1 -- 172.21.15.92:0/1060364023 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 840654582 0 0) 0x7fc8d40b88e0 con 0x7fc8d40d0d60 2023-11-28T11:21:42.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.991+0000 7fc8d3fff700 1 --2- 172.21.15.92:0/1060364023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8d40cd460 0x7fc8d406c080 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc8c00098f0 tx=0x7fc8c0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8ba84ed37f3745d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:42.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.992+0000 7fc8d37fe700 1 -- 172.21.15.92:0/1060364023 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c000b8a0 con 0x7fc8d40cd460 2023-11-28T11:21:42.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.992+0000 7fc8d37fe700 1 -- 172.21.15.92:0/1060364023 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8c000ba00 con 0x7fc8d40cd460 2023-11-28T11:21:42.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.992+0000 7fc8d37fe700 1 -- 172.21.15.92:0/1060364023 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c000bd90 con 0x7fc8d40cd460 2023-11-28T11:21:42.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.992+0000 7fc8db1ff700 1 -- 172.21.15.92:0/1060364023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8d40cd460 msgr2=0x7fc8d406c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:42.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.992+0000 7fc8db1ff700 1 --2- 172.21.15.92:0/1060364023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8d40cd460 0x7fc8d406c080 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc8c00098f0 tx=0x7fc8c0009c00 comp rx=0 tx=0).stop 2023-11-28T11:21:42.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.992+0000 7fc8db1ff700 1 -- 172.21.15.92:0/1060364023 shutdown_connections 2023-11-28T11:21:42.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.992+0000 7fc8db1ff700 1 --2- 172.21.15.92:0/1060364023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8d40cd460 0x7fc8d406c080 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:42.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.992+0000 7fc8db1ff700 1 -- 172.21.15.92:0/1060364023 >> 172.21.15.92:0/1060364023 conn(0x7fc8d40719a0 msgr2=0x7fc8d4071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:42.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.992+0000 7fc8db1ff700 1 -- 172.21.15.92:0/1060364023 shutdown_connections 2023-11-28T11:21:42.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.992+0000 7fc8db1ff700 1 -- 172.21.15.92:0/1060364023 wait complete. 2023-11-28T11:21:42.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.994+0000 7fc8db1ff700 1 Processor -- start 2023-11-28T11:21:42.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.994+0000 7fc8db1ff700 1 -- start start 2023-11-28T11:21:42.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.995+0000 7fc8db1ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8d40b8510 0x7fc8d415b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:43.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.995+0000 7fc8db1ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8d40cd460 0x7fc8d40b7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:43.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.995+0000 7fc8db1ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8d40d0d60 0x7fc8d40b77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:43.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.995+0000 7fc8db1ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d41e6d70 con 0x7fc8d40d0d60 2023-11-28T11:21:43.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.995+0000 7fc8db1ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d41f2080 con 0x7fc8d40cd460 2023-11-28T11:21:43.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.995+0000 7fc8db1ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d415ba00 con 0x7fc8d40b8510 2023-11-28T11:21:43.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.995+0000 7fc8d979c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8d40d0d60 0x7fc8d40b77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:43.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.995+0000 7fc8d979c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8d40d0d60 0x7fc8d40b77c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51342/0 (socket says 172.21.15.92:51342) 2023-11-28T11:21:43.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.995+0000 7fc8d979c700 1 -- 172.21.15.92:0/1426921531 learned_addr learned my addr 172.21.15.92:0/1426921531 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:43.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.996+0000 7fc8d3fff700 1 --2- 172.21.15.92:0/1426921531 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8d40cd460 0x7fc8d40b7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:43.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.996+0000 7fc8d8f9b700 1 --2- 172.21.15.92:0/1426921531 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8d40b8510 0x7fc8d415b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:43.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.996+0000 7fc8d979c700 1 -- 172.21.15.92:0/1426921531 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8d40b8510 msgr2=0x7fc8d415b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:43.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.996+0000 7fc8d979c700 1 --2- 172.21.15.92:0/1426921531 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8d40b8510 0x7fc8d415b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:43.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.996+0000 7fc8d979c700 1 -- 172.21.15.92:0/1426921531 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8d40cd460 msgr2=0x7fc8d40b7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:43.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.996+0000 7fc8d979c700 1 --2- 172.21.15.92:0/1426921531 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8d40cd460 0x7fc8d40b7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:43.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.996+0000 7fc8d979c700 1 -- 172.21.15.92:0/1426921531 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8c0009550 con 0x7fc8d40d0d60 2023-11-28T11:21:43.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.996+0000 7fc8d3fff700 1 --2- 172.21.15.92:0/1426921531 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8d40cd460 0x7fc8d40b7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:21:43.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.996+0000 7fc8d979c700 1 --2- 172.21.15.92:0/1426921531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8d40d0d60 0x7fc8d40b77c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc00cc00 tx=0x7fc8cc00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:43.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.996+0000 7fc8d17fa700 1 -- 172.21.15.92:0/1426921531 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc00aba0 con 0x7fc8d40d0d60 2023-11-28T11:21:43.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.996+0000 7fc8d17fa700 1 -- 172.21.15.92:0/1426921531 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8cc00ad20 con 0x7fc8d40d0d60 2023-11-28T11:21:43.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.996+0000 7fc8d17fa700 1 -- 172.21.15.92:0/1426921531 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc007310 con 0x7fc8d40d0d60 2023-11-28T11:21:43.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.996+0000 7fc8db1ff700 1 -- 172.21.15.92:0/1426921531 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8d40d29c0 con 0x7fc8d40d0d60 2023-11-28T11:21:43.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.996+0000 7fc8db1ff700 1 -- 172.21.15.92:0/1426921531 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8d40cfff0 con 0x7fc8d40d0d60 2023-11-28T11:21:43.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.997+0000 7fc8db1ff700 1 -- 172.21.15.92:0/1426921531 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8d40d2b50 con 0x7fc8d40d0d60 2023-11-28T11:21:43.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.998+0000 7fc8d17fa700 1 -- 172.21.15.92:0/1426921531 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc8cc010070 con 0x7fc8d40d0d60 2023-11-28T11:21:43.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:42.999+0000 7fc8d17fa700 1 --2- 172.21.15.92:0/1426921531 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc8b80682f0 0x7fc8b806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:43.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.000+0000 7fc8d17fa700 1 -- 172.21.15.92:0/1426921531 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fc8cc085a40 con 0x7fc8d40d0d60 2023-11-28T11:21:43.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.000+0000 7fc8d8f9b700 1 --2- 172.21.15.92:0/1426921531 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc8b80682f0 0x7fc8b806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:43.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.000+0000 7fc8d8f9b700 1 --2- 172.21.15.92:0/1426921531 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc8b80682f0 0x7fc8b806a7a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc8c8000f60 tx=0x7fc8c8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:43.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.003+0000 7fc8d17fa700 1 -- 172.21.15.92:0/1426921531 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc8cc051e20 con 0x7fc8d40d0d60 2023-11-28T11:21:43.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.194+0000 7fc8db1ff700 1 -- 172.21.15.92:0/1426921531 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fc8d4067880 con 0x7fc8d40d0d60 2023-11-28T11:21:43.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.195+0000 7fc8d17fa700 1 -- 172.21.15.92:0/1426921531 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7fc8cc018020 con 0x7fc8d40d0d60 2023-11-28T11:21:43.196 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-11-28T11:21:43.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.197+0000 7fc8db1ff700 1 -- 172.21.15.92:0/1426921531 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc8b80682f0 msgr2=0x7fc8b806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:43.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.197+0000 7fc8db1ff700 1 --2- 172.21.15.92:0/1426921531 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc8b80682f0 0x7fc8b806a7a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc8c8000f60 tx=0x7fc8c8009040 comp rx=0 tx=0).stop 2023-11-28T11:21:43.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.197+0000 7fc8db1ff700 1 -- 172.21.15.92:0/1426921531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8d40d0d60 msgr2=0x7fc8d40b77c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:43.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.197+0000 7fc8db1ff700 1 --2- 172.21.15.92:0/1426921531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8d40d0d60 0x7fc8d40b77c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc00cc00 tx=0x7fc8cc00a060 comp rx=0 tx=0).stop 2023-11-28T11:21:43.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.197+0000 7fc8db1ff700 1 -- 172.21.15.92:0/1426921531 shutdown_connections 2023-11-28T11:21:43.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.197+0000 7fc8db1ff700 1 --2- 172.21.15.92:0/1426921531 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc8b80682f0 0x7fc8b806a7a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:43.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.197+0000 7fc8db1ff700 1 --2- 172.21.15.92:0/1426921531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8d40d0d60 0x7fc8d40b77c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:43.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.197+0000 7fc8db1ff700 1 --2- 172.21.15.92:0/1426921531 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc8d40b8510 0x7fc8d415b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:43.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.197+0000 7fc8db1ff700 1 --2- 172.21.15.92:0/1426921531 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc8d40cd460 0x7fc8d40b7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:43.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.197+0000 7fc8db1ff700 1 -- 172.21.15.92:0/1426921531 >> 172.21.15.92:0/1426921531 conn(0x7fc8d40719a0 msgr2=0x7fc8d40cbef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:43.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.197+0000 7fc8db1ff700 1 -- 172.21.15.92:0/1426921531 shutdown_connections 2023-11-28T11:21:43.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.197+0000 7fc8db1ff700 1 -- 172.21.15.92:0/1426921531 wait complete. 2023-11-28T11:21:43.217 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:21:43.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.412+0000 7f0e631ff700 1 Processor -- start 2023-11-28T11:21:43.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.412+0000 7f0e631ff700 1 -- start start 2023-11-28T11:21:43.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.413+0000 7f0e631ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0e5c0b8510 0x7f0e5c06b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:43.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.413+0000 7f0e631ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0e5c0cd460 0x7f0e5c06beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:43.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.413+0000 7f0e631ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e5c0b88e0 con 0x7f0e5c06c3f0 2023-11-28T11:21:43.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.413+0000 7f0e631ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e5c0cdbd0 con 0x7f0e5c0cd460 2023-11-28T11:21:43.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.413+0000 7f0e631ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e5c0d02d0 con 0x7f0e5c0b8510 2023-11-28T11:21:43.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.413+0000 7f0e5bfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0e5c0cd460 0x7f0e5c06beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:43.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.413+0000 7f0e5bfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0e5c0cd460 0x7f0e5c06beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51358/0 (socket says 172.21.15.92:51358) 2023-11-28T11:21:43.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.413+0000 7f0e5bfff700 1 -- 172.21.15.92:0/3682567170 learned_addr learned my addr 172.21.15.92:0/3682567170 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:43.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.413+0000 7f0e5bfff700 1 -- 172.21.15.92:0/3682567170 >> v2:172.21.15.169:3300/0 conn(0x7f0e5c0b8510 msgr2=0x7f0e5c06b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:43.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.413+0000 7f0e5bfff700 1 --2- 172.21.15.92:0/3682567170 >> v2:172.21.15.169:3300/0 conn(0x7f0e5c0b8510 0x7f0e5c06b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:43.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.414+0000 7f0e5bfff700 1 -- 172.21.15.92:0/3682567170 >> v1:172.21.15.169:6789/0 conn(0x7f0e5c06c3f0 legacy=0x7f0e5c15cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:43.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.414+0000 7f0e5bfff700 1 -- 172.21.15.92:0/3682567170 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e5c0cd8f0 con 0x7f0e5c0cd460 2023-11-28T11:21:43.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.414+0000 7f0e5bfff700 1 --2- 172.21.15.92:0/3682567170 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e5c0cd460 0x7f0e5c06beb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0e48009920 tx=0x7f0e48009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edc5722cc4838c64 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:43.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.414+0000 7f0e5b7fe700 1 -- 172.21.15.92:0/3682567170 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e48004230 con 0x7f0e5c0cd460 2023-11-28T11:21:43.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.414+0000 7f0e5b7fe700 1 -- 172.21.15.92:0/3682567170 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e48004390 con 0x7f0e5c0cd460 2023-11-28T11:21:43.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.414+0000 7f0e5b7fe700 1 -- 172.21.15.92:0/3682567170 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e48009e70 con 0x7f0e5c0cd460 2023-11-28T11:21:43.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.414+0000 7f0e631ff700 1 -- 172.21.15.92:0/3682567170 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e5c0cd460 msgr2=0x7f0e5c06beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:43.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.415+0000 7f0e631ff700 1 --2- 172.21.15.92:0/3682567170 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e5c0cd460 0x7f0e5c06beb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0e48009920 tx=0x7f0e48009c30 comp rx=0 tx=0).stop 2023-11-28T11:21:43.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.415+0000 7f0e631ff700 1 -- 172.21.15.92:0/3682567170 shutdown_connections 2023-11-28T11:21:43.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.415+0000 7f0e631ff700 1 --2- 172.21.15.92:0/3682567170 >> v2:172.21.15.169:3300/0 conn(0x7f0e5c0b8510 0x7f0e5c06b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:43.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.415+0000 7f0e631ff700 1 --2- 172.21.15.92:0/3682567170 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e5c0cd460 0x7f0e5c06beb0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:43.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.415+0000 7f0e631ff700 1 -- 172.21.15.92:0/3682567170 >> 172.21.15.92:0/3682567170 conn(0x7f0e5c0719a0 msgr2=0x7f0e5c071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:43.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.415+0000 7f0e631ff700 1 -- 172.21.15.92:0/3682567170 shutdown_connections 2023-11-28T11:21:43.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.415+0000 7f0e631ff700 1 -- 172.21.15.92:0/3682567170 wait complete. 2023-11-28T11:21:43.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.416+0000 7f0e631ff700 1 Processor -- start 2023-11-28T11:21:43.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.417+0000 7f0e631ff700 1 -- start start 2023-11-28T11:21:43.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.417+0000 7f0e631ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e5c06c3f0 0x7f0e5c0ca880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:43.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.418+0000 7f0e631ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e5c0b8510 0x7f0e5c156620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:43.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.418+0000 7f0e631ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e5c0cd460 0x7f0e5c1f1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:43.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.418+0000 7f0e631ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5c1f2ce0 con 0x7f0e5c0cd460 2023-11-28T11:21:43.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.418+0000 7f0e631ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5c1f2720 con 0x7f0e5c06c3f0 2023-11-28T11:21:43.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.418+0000 7f0e631ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5c1f2a00 con 0x7f0e5c0b8510 2023-11-28T11:21:43.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.418+0000 7f0e6179c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e5c0cd460 0x7f0e5c1f1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:43.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.418+0000 7f0e6179c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e5c0cd460 0x7f0e5c1f1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51364/0 (socket says 172.21.15.92:51364) 2023-11-28T11:21:43.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.418+0000 7f0e6179c700 1 -- 172.21.15.92:0/3561863338 learned_addr learned my addr 172.21.15.92:0/3561863338 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:43.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.418+0000 7f0e5bfff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e5c0b8510 0x7f0e5c156620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:43.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.418+0000 7f0e60f9b700 1 --2- 172.21.15.92:0/3561863338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e5c06c3f0 0x7f0e5c0ca880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:43.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.418+0000 7f0e6179c700 1 -- 172.21.15.92:0/3561863338 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e5c0b8510 msgr2=0x7f0e5c156620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:43.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.418+0000 7f0e6179c700 1 --2- 172.21.15.92:0/3561863338 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e5c0b8510 0x7f0e5c156620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:43.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.418+0000 7f0e6179c700 1 -- 172.21.15.92:0/3561863338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e5c06c3f0 msgr2=0x7f0e5c0ca880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:43.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.418+0000 7f0e6179c700 1 --2- 172.21.15.92:0/3561863338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e5c06c3f0 0x7f0e5c0ca880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:43.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.418+0000 7f0e6179c700 1 -- 172.21.15.92:0/3561863338 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e48009550 con 0x7f0e5c0cd460 2023-11-28T11:21:43.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.419+0000 7f0e6179c700 1 --2- 172.21.15.92:0/3561863338 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e5c0cd460 0x7f0e5c1f1cd0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0e5400c7a0 tx=0x7f0e5400cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:43.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.419+0000 7f0e597fa700 1 -- 172.21.15.92:0/3561863338 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e54003e10 con 0x7f0e5c0cd460 2023-11-28T11:21:43.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.419+0000 7f0e597fa700 1 -- 172.21.15.92:0/3561863338 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e54005070 con 0x7f0e5c0cd460 2023-11-28T11:21:43.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.419+0000 7f0e631ff700 1 -- 172.21.15.92:0/3561863338 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e5c0cfff0 con 0x7f0e5c0cd460 2023-11-28T11:21:43.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.419+0000 7f0e597fa700 1 -- 172.21.15.92:0/3561863338 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e54005420 con 0x7f0e5c0cd460 2023-11-28T11:21:43.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.419+0000 7f0e631ff700 1 -- 172.21.15.92:0/3561863338 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e5c0cfd10 con 0x7f0e5c0cd460 2023-11-28T11:21:43.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.420+0000 7f0e631ff700 1 -- 172.21.15.92:0/3561863338 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e5c0c9ee0 con 0x7f0e5c0cd460 2023-11-28T11:21:43.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.421+0000 7f0e597fa700 1 -- 172.21.15.92:0/3561863338 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0e54019030 con 0x7f0e5c0cd460 2023-11-28T11:21:43.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.422+0000 7f0e597fa700 1 --2- 172.21.15.92:0/3561863338 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f0e400682f0 0x7f0e4006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:43.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.422+0000 7f0e597fa700 1 -- 172.21.15.92:0/3561863338 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f0e54085580 con 0x7f0e5c0cd460 2023-11-28T11:21:43.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.422+0000 7f0e60f9b700 1 --2- 172.21.15.92:0/3561863338 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f0e400682f0 0x7f0e4006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:43.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.423+0000 7f0e60f9b700 1 --2- 172.21.15.92:0/3561863338 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f0e400682f0 0x7f0e4006a7a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0e50009a10 tx=0x7f0e50009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:43.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.426+0000 7f0e597fa700 1 -- 172.21.15.92:0/3561863338 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0e5c0c9ee0 con 0x7f0e5c0cd460 2023-11-28T11:21:43.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.621+0000 7f0e631ff700 1 -- 172.21.15.92:0/3561863338 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0e5c0c9ee0 con 0x7f0e5c0cd460 2023-11-28T11:21:43.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.623+0000 7f0e597fa700 1 -- 172.21.15.92:0/3561863338 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+19844 (secure 0 0 0) 0x7f0e5c0c9ee0 con 0x7f0e5c0cd460 2023-11-28T11:21:43.624 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:21:43.624 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":29,"fsid":"68956e0c-c16f-47cd-a220-f98364b0831d","created":"2023-11-28T11:20:28.697573+0000","modified":"2023-11-28T11:21:37.638578+0000","last_up_change":"2023-11-28T11:20:41.257011+0000","last_in_change":"2023-11-28T11:20:32.249767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:20:41.925237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:20:48.682497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:20:54.471398+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:20:55.204585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:20:58.297307+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6abe483f-0e03-4a26-b9cf-62041291747e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":115823},{"type":"v1","addr":"172.21.15.92:6809","nonce":115823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":115823},{"type":"v1","addr":"172.21.15.92:6813","nonce":115823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":115823},{"type":"v1","addr":"172.21.15.92:6821","nonce":115823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":115823},{"type":"v1","addr":"172.21.15.92:6817","nonce":115823}]},"public_addr":"172.21.15.92:6809/115823","cluster_addr":"172.21.15.92:6813/115823","heartbeat_back_addr":"172.21.15.92:6821/115823","heartbeat_front_addr":"172.21.15.92:6817/115823","state":["exists","up"]},{"osd":1,"uuid":"13c1de73-1a8c-4d7e-8a28-9dae52dfe089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":115824},{"type":"v1","addr":"172.21.15.92:6812","nonce":115824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":115824},{"type":"v1","addr":"172.21.15.92:6816","nonce":115824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":115824},{"type":"v1","addr":"172.21.15.92:6823","nonce":115824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":115824},{"type":"v1","addr":"172.21.15.92:6820","nonce":115824}]},"public_addr":"172.21.15.92:6812/115824","cluster_addr":"172.21.15.92:6816/115824","heartbeat_back_addr":"172.21.15.92:6823/115824","heartbeat_front_addr":"172.21.15.92:6820/115824","state":["exists","up"]},{"osd":2,"uuid":"25788a55-112f-4f00-810d-2bc37cf76778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":115821},{"type":"v1","addr":"172.21.15.92:6801","nonce":115821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":115821},{"type":"v1","addr":"172.21.15.92:6803","nonce":115821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":115821},{"type":"v1","addr":"172.21.15.92:6807","nonce":115821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":115821},{"type":"v1","addr":"172.21.15.92:6805","nonce":115821}]},"public_addr":"172.21.15.92:6801/115821","cluster_addr":"172.21.15.92:6803/115821","heartbeat_back_addr":"172.21.15.92:6807/115821","heartbeat_front_addr":"172.21.15.92:6805/115821","state":["exists","up"]},{"osd":3,"uuid":"adf54e24-33a9-41fe-9ce3-98a5ab6c158e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":115825},{"type":"v1","addr":"172.21.15.92:6825","nonce":115825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":115825},{"type":"v1","addr":"172.21.15.92:6827","nonce":115825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":115825},{"type":"v1","addr":"172.21.15.92:6831","nonce":115825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":115825},{"type":"v1","addr":"172.21.15.92:6829","nonce":115825}]},"public_addr":"172.21.15.92:6825/115825","cluster_addr":"172.21.15.92:6827/115825","heartbeat_back_addr":"172.21.15.92:6831/115825","heartbeat_front_addr":"172.21.15.92:6829/115825","state":["exists","up"]},{"osd":4,"uuid":"ccb9b747-f399-44ea-9a18-9db3152227e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":116035},{"type":"v1","addr":"172.21.15.169:6817","nonce":116035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":116035},{"type":"v1","addr":"172.21.15.169:6819","nonce":116035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":116035},{"type":"v1","addr":"172.21.15.169:6826","nonce":116035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":116035},{"type":"v1","addr":"172.21.15.169:6822","nonce":116035}]},"public_addr":"172.21.15.169:6817/116035","cluster_addr":"172.21.15.169:6819/116035","heartbeat_back_addr":"172.21.15.169:6826/116035","heartbeat_front_addr":"172.21.15.169:6822/116035","state":["exists","up"]},{"osd":5,"uuid":"308a1e63-75aa-4d7d-8a7c-cecc24714fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":116033},{"type":"v1","addr":"172.21.15.169:6801","nonce":116033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":116033},{"type":"v1","addr":"172.21.15.169:6803","nonce":116033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":116033},{"type":"v1","addr":"172.21.15.169:6807","nonce":116033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":116033},{"type":"v1","addr":"172.21.15.169:6805","nonce":116033}]},"public_addr":"172.21.15.169:6801/116033","cluster_addr":"172.21.15.169:6803/116033","heartbeat_back_addr":"172.21.15.169:6807/116033","heartbeat_front_addr":"172.21.15.169:6805/116033","state":["exists","up"]},{"osd":6,"uuid":"9c459b7b-b821-4b4b-b91e-6aaae271457c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":116034},{"type":"v1","addr":"172.21.15.169:6809","nonce":116034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":116034},{"type":"v1","addr":"172.21.15.169:6811","nonce":116034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":116034},{"type":"v1","addr":"172.21.15.169:6815","nonce":116034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":116034},{"type":"v1","addr":"172.21.15.169:6813","nonce":116034}]},"public_addr":"172.21.15.169:6809/116034","cluster_addr":"172.21.15.169:6811/116034","heartbeat_back_addr":"172.21.15.169:6815/116034","heartbeat_front_addr":"172.21.15.169:6813/116034","state":["exists","up"]},{"osd":7,"uuid":"1ca9aa4a-755d-4e28-a937-4d8dccab4df3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":116044},{"type":"v1","addr":"172.21.15.169:6823","nonce":116044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6825","nonce":116044},{"type":"v1","addr":"172.21.15.169:6827","nonce":116044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":116044},{"type":"v1","addr":"172.21.15.169:6831","nonce":116044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":116044},{"type":"v1","addr":"172.21.15.169:6829","nonce":116044}]},"public_addr":"172.21.15.169:6823/116044","cluster_addr":"172.21.15.169:6827/116044","heartbeat_back_addr":"172.21.15.169:6831/116044","heartbeat_front_addr":"172.21.15.169:6829/116044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.586781+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.869778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.508594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.860493+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.872267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.746933+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.611520+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.871446+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.1d","mappings":[{"from":3,"to":2}]},{"pgid":"4.21","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:21:43.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.625+0000 7f0e631ff700 1 -- 172.21.15.92:0/3561863338 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f0e400682f0 msgr2=0x7f0e4006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:43.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.625+0000 7f0e631ff700 1 --2- 172.21.15.92:0/3561863338 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f0e400682f0 0x7f0e4006a7a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0e50009a10 tx=0x7f0e50009190 comp rx=0 tx=0).stop 2023-11-28T11:21:43.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.625+0000 7f0e631ff700 1 -- 172.21.15.92:0/3561863338 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e5c0cd460 msgr2=0x7f0e5c1f1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:43.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.625+0000 7f0e631ff700 1 --2- 172.21.15.92:0/3561863338 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e5c0cd460 0x7f0e5c1f1cd0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0e5400c7a0 tx=0x7f0e5400cab0 comp rx=0 tx=0).stop 2023-11-28T11:21:43.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.625+0000 7f0e631ff700 1 -- 172.21.15.92:0/3561863338 shutdown_connections 2023-11-28T11:21:43.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.625+0000 7f0e631ff700 1 --2- 172.21.15.92:0/3561863338 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f0e400682f0 0x7f0e4006a7a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:43.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.625+0000 7f0e631ff700 1 --2- 172.21.15.92:0/3561863338 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e5c0cd460 0x7f0e5c1f1cd0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:43.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.625+0000 7f0e631ff700 1 --2- 172.21.15.92:0/3561863338 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e5c06c3f0 0x7f0e5c0ca880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:43.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.625+0000 7f0e631ff700 1 --2- 172.21.15.92:0/3561863338 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e5c0b8510 0x7f0e5c156620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:43.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.625+0000 7f0e631ff700 1 -- 172.21.15.92:0/3561863338 >> 172.21.15.92:0/3561863338 conn(0x7f0e5c0719a0 msgr2=0x7f0e5c0b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:43.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.625+0000 7f0e631ff700 1 -- 172.21.15.92:0/3561863338 shutdown_connections 2023-11-28T11:21:43.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.625+0000 7f0e631ff700 1 -- 172.21.15.92:0/3561863338 wait complete. 2023-11-28T11:21:43.645 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:20:41.925237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:20:48.682497+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:20:54.471398+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:20:55.204585+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T11:20:58.297307+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T11:21:43.645 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:21:43.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.846+0000 7ff41f7ff700 1 Processor -- start 2023-11-28T11:21:43.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.847+0000 7ff41f7ff700 1 -- start start 2023-11-28T11:21:43.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.848+0000 7ff41f7ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff4180ca600 0x7ff418164220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:43.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.848+0000 7ff41f7ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff418159b10 con 0x7ff418142e80 2023-11-28T11:21:43.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.848+0000 7ff41f7ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff418143250 con 0x7ff41806a980 2023-11-28T11:21:43.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.848+0000 7ff41f7ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4180c9380 con 0x7ff4180ca600 2023-11-28T11:21:43.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.848+0000 7ff41d59b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff418142e80 0x7ff41806a270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:58048/0 (socket says 172.21.15.92:58048) 2023-11-28T11:21:43.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.848+0000 7ff41d59b700 1 -- 172.21.15.92:0/3916383112 learned_addr learned my addr 172.21.15.92:0/3916383112 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:43.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.848+0000 7ff41dd9c700 1 --2- 172.21.15.92:0/3916383112 >> v2:172.21.15.92:3300/0 conn(0x7ff4180ca600 0x7ff418164220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:43.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.848+0000 7ff40ffff700 1 -- 172.21.15.92:0/3916383112 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2286097407 0 0) 0x7ff418159b10 con 0x7ff418142e80 2023-11-28T11:21:43.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.848+0000 7ff40ffff700 1 -- 172.21.15.92:0/3916383112 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4040035b0 con 0x7ff418142e80 2023-11-28T11:21:43.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.849+0000 7ff41dd9c700 1 -- 172.21.15.92:0/3916383112 >> v1:172.21.15.169:6789/0 conn(0x7ff41806a980 legacy=0x7ff4180c9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:43.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.849+0000 7ff41dd9c700 1 -- 172.21.15.92:0/3916383112 >> v1:172.21.15.92:6789/0 conn(0x7ff418142e80 legacy=0x7ff41806a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:43.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.849+0000 7ff41dd9c700 1 -- 172.21.15.92:0/3916383112 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4180cab00 con 0x7ff4180ca600 2023-11-28T11:21:43.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.849+0000 7ff41dd9c700 1 --2- 172.21.15.92:0/3916383112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4180ca600 0x7ff418164220 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff410009d90 tx=0x7ff41000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3f6580df4485a04 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:43.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.849+0000 7ff40ffff700 1 -- 172.21.15.92:0/3916383112 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff41000e070 con 0x7ff4180ca600 2023-11-28T11:21:43.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.849+0000 7ff40ffff700 1 -- 172.21.15.92:0/3916383112 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff41000bb20 con 0x7ff4180ca600 2023-11-28T11:21:43.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.849+0000 7ff40ffff700 1 -- 172.21.15.92:0/3916383112 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff410012430 con 0x7ff4180ca600 2023-11-28T11:21:43.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.849+0000 7ff41f7ff700 1 -- 172.21.15.92:0/3916383112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4180ca600 msgr2=0x7ff418164220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:43.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.849+0000 7ff41f7ff700 1 --2- 172.21.15.92:0/3916383112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4180ca600 0x7ff418164220 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff410009d90 tx=0x7ff41000b330 comp rx=0 tx=0).stop 2023-11-28T11:21:43.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.850+0000 7ff41f7ff700 1 -- 172.21.15.92:0/3916383112 shutdown_connections 2023-11-28T11:21:43.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.850+0000 7ff41f7ff700 1 --2- 172.21.15.92:0/3916383112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4180ca600 0x7ff418164220 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:43.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.850+0000 7ff41f7ff700 1 -- 172.21.15.92:0/3916383112 >> 172.21.15.92:0/3916383112 conn(0x7ff4180cb490 msgr2=0x7ff4180cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:43.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.850+0000 7ff41f7ff700 1 -- 172.21.15.92:0/3916383112 shutdown_connections 2023-11-28T11:21:43.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.850+0000 7ff41f7ff700 1 -- 172.21.15.92:0/3916383112 wait complete. 2023-11-28T11:21:43.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.851+0000 7ff41f7ff700 1 Processor -- start 2023-11-28T11:21:43.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.852+0000 7ff41f7ff700 1 -- start start 2023-11-28T11:21:43.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.852+0000 7ff41f7ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff41806a980 0x7ff4181e4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:43.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.852+0000 7ff41f7ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4180ca600 0x7ff418154400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:43.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.852+0000 7ff41f7ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff418142e80 0x7ff4181e2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:43.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.852+0000 7ff41f7ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff418157d20 con 0x7ff4180ca600 2023-11-28T11:21:43.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.852+0000 7ff41f7ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4181577e0 con 0x7ff418142e80 2023-11-28T11:21:43.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.852+0000 7ff41f7ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff418156fd0 con 0x7ff41806a980 2023-11-28T11:21:43.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.853+0000 7ff41cd9a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4180ca600 0x7ff418154400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:43.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.853+0000 7ff41cd9a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4180ca600 0x7ff418154400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51376/0 (socket says 172.21.15.92:51376) 2023-11-28T11:21:43.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.853+0000 7ff41cd9a700 1 -- 172.21.15.92:0/2441912947 learned_addr learned my addr 172.21.15.92:0/2441912947 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:43.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.853+0000 7ff41d59b700 1 --2- 172.21.15.92:0/2441912947 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff41806a980 0x7ff4181e4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:43.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.853+0000 7ff41dd9c700 1 --2- 172.21.15.92:0/2441912947 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff418142e80 0x7ff4181e2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:43.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.853+0000 7ff41cd9a700 1 -- 172.21.15.92:0/2441912947 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff41806a980 msgr2=0x7ff4181e4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:43.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.853+0000 7ff41cd9a700 1 --2- 172.21.15.92:0/2441912947 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff41806a980 0x7ff4181e4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:43.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.853+0000 7ff41cd9a700 1 -- 172.21.15.92:0/2441912947 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff418142e80 msgr2=0x7ff4181e2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:43.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.853+0000 7ff41cd9a700 1 --2- 172.21.15.92:0/2441912947 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff418142e80 0x7ff4181e2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:43.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.853+0000 7ff41cd9a700 1 -- 172.21.15.92:0/2441912947 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff410009a70 con 0x7ff4180ca600 2023-11-28T11:21:43.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.853+0000 7ff41cd9a700 1 --2- 172.21.15.92:0/2441912947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4180ca600 0x7ff418154400 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff40800c4e0 tx=0x7ff40800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:43.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.854+0000 7ff40dffb700 1 -- 172.21.15.92:0/2441912947 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4080052d0 con 0x7ff4180ca600 2023-11-28T11:21:43.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.854+0000 7ff40dffb700 1 -- 172.21.15.92:0/2441912947 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff408005430 con 0x7ff4180ca600 2023-11-28T11:21:43.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.854+0000 7ff41f7ff700 1 -- 172.21.15.92:0/2441912947 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4180cc8d0 con 0x7ff4180ca600 2023-11-28T11:21:43.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.854+0000 7ff40dffb700 1 -- 172.21.15.92:0/2441912947 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4080038c0 con 0x7ff4180ca600 2023-11-28T11:21:43.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.854+0000 7ff41f7ff700 1 -- 172.21.15.92:0/2441912947 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4180cc5f0 con 0x7ff4180ca600 2023-11-28T11:21:43.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.855+0000 7ff41f7ff700 1 -- 172.21.15.92:0/2441912947 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4180ccbb0 con 0x7ff4180ca600 2023-11-28T11:21:43.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.856+0000 7ff40dffb700 1 -- 172.21.15.92:0/2441912947 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff408019030 con 0x7ff4180ca600 2023-11-28T11:21:43.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.856+0000 7ff40dffb700 1 --2- 172.21.15.92:0/2441912947 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff3fc0682f0 0x7ff3fc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:43.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.857+0000 7ff40dffb700 1 -- 172.21.15.92:0/2441912947 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7ff408085220 con 0x7ff4180ca600 2023-11-28T11:21:43.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.857+0000 7ff41d59b700 1 --2- 172.21.15.92:0/2441912947 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff3fc0682f0 0x7ff3fc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:43.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.857+0000 7ff41d59b700 1 --2- 172.21.15.92:0/2441912947 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff3fc0682f0 0x7ff3fc06a7a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff414007fd0 tx=0x7ff414007dc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:43.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:43.861+0000 7ff40dffb700 1 -- 172.21.15.92:0/2441912947 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff408051750 con 0x7ff4180ca600 2023-11-28T11:21:44.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.052+0000 7ff41f7ff700 1 -- 172.21.15.92:0/2441912947 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff418067880 con 0x7ff4180ca600 2023-11-28T11:21:44.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.053+0000 7ff40dffb700 1 -- 172.21.15.92:0/2441912947 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff408054d70 con 0x7ff4180ca600 2023-11-28T11:21:44.054 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-11-28T11:21:44.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.055+0000 7ff41f7ff700 1 -- 172.21.15.92:0/2441912947 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff3fc0682f0 msgr2=0x7ff3fc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:44.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.055+0000 7ff41f7ff700 1 --2- 172.21.15.92:0/2441912947 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff3fc0682f0 0x7ff3fc06a7a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff414007fd0 tx=0x7ff414007dc0 comp rx=0 tx=0).stop 2023-11-28T11:21:44.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.055+0000 7ff41f7ff700 1 -- 172.21.15.92:0/2441912947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4180ca600 msgr2=0x7ff418154400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:44.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.055+0000 7ff41f7ff700 1 --2- 172.21.15.92:0/2441912947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4180ca600 0x7ff418154400 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff40800c4e0 tx=0x7ff40800c7f0 comp rx=0 tx=0).stop 2023-11-28T11:21:44.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.055+0000 7ff41f7ff700 1 -- 172.21.15.92:0/2441912947 shutdown_connections 2023-11-28T11:21:44.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.055+0000 7ff41f7ff700 1 --2- 172.21.15.92:0/2441912947 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff418142e80 0x7ff4181e2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:44.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.055+0000 7ff41f7ff700 1 --2- 172.21.15.92:0/2441912947 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff41806a980 0x7ff4181e4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:44.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.055+0000 7ff41f7ff700 1 --2- 172.21.15.92:0/2441912947 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff3fc0682f0 0x7ff3fc06a7a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:44.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.055+0000 7ff41f7ff700 1 --2- 172.21.15.92:0/2441912947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4180ca600 0x7ff418154400 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:44.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.055+0000 7ff41f7ff700 1 -- 172.21.15.92:0/2441912947 >> 172.21.15.92:0/2441912947 conn(0x7ff4180cb490 msgr2=0x7ff4180acaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:44.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.055+0000 7ff41f7ff700 1 -- 172.21.15.92:0/2441912947 shutdown_connections 2023-11-28T11:21:44.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.056+0000 7ff41f7ff700 1 -- 172.21.15.92:0/2441912947 wait complete. 2023-11-28T11:21:44.075 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:21:44.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.274+0000 7f7103fff700 1 Processor -- start 2023-11-28T11:21:44.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.274+0000 7f7103fff700 1 -- start start 2023-11-28T11:21:44.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.275+0000 7f7103fff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f70fc0d2df0 0x7f70fc0cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:44.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.275+0000 7f7103fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70fc0b4fc0 0x7f70fc0d31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:44.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.275+0000 7f7103fff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70fc0d1950 con 0x7f70fc0b1c60 2023-11-28T11:21:44.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.275+0000 7f7103fff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70fc0d3d30 con 0x7f70fc0d2df0 2023-11-28T11:21:44.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.275+0000 7f7103fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70fc0b2500 con 0x7f70fc0b4fc0 2023-11-28T11:21:44.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.276+0000 7f710259c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f70fc0b1c60 0x7f70fc0cf010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:58054/0 (socket says 172.21.15.92:58054) 2023-11-28T11:21:44.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.276+0000 7f710259c700 1 -- 172.21.15.92:0/2858048732 learned_addr learned my addr 172.21.15.92:0/2858048732 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:44.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.276+0000 7f7101d9b700 1 --2- 172.21.15.92:0/2858048732 >> v2:172.21.15.169:3300/0 conn(0x7f70fc0d2df0 0x7f70fc0cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:44.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.276+0000 7f710159a700 1 --2- 172.21.15.92:0/2858048732 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70fc0b4fc0 0x7f70fc0d31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:44.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.276+0000 7f7100d99700 1 -- 172.21.15.92:0/2858048732 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3459753605 0 0) 0x7f70fc0d1950 con 0x7f70fc0b1c60 2023-11-28T11:21:44.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.276+0000 7f7100d99700 1 -- 172.21.15.92:0/2858048732 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70e80035b0 con 0x7f70fc0b1c60 2023-11-28T11:21:44.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.276+0000 7f7101d9b700 1 -- 172.21.15.92:0/2858048732 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70fc0b4fc0 msgr2=0x7f70fc0d31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:44.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.276+0000 7f7101d9b700 1 --2- 172.21.15.92:0/2858048732 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70fc0b4fc0 0x7f70fc0d31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:44.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.276+0000 7f7101d9b700 1 -- 172.21.15.92:0/2858048732 >> v1:172.21.15.92:6789/0 conn(0x7f70fc0b1c60 legacy=0x7f70fc0cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:44.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.276+0000 7f7101d9b700 1 -- 172.21.15.92:0/2858048732 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70fc0d3a50 con 0x7f70fc0d2df0 2023-11-28T11:21:44.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.277+0000 7f7101d9b700 1 --2- 172.21.15.92:0/2858048732 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70fc0d2df0 0x7f70fc0cd620 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f70f8009840 tx=0x7f70f8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92deed0f07c19588 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:44.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.277+0000 7f7100d99700 1 -- 172.21.15.92:0/2858048732 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f8011830 con 0x7f70fc0d2df0 2023-11-28T11:21:44.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.277+0000 7f7100d99700 1 -- 172.21.15.92:0/2858048732 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70f8011990 con 0x7f70fc0d2df0 2023-11-28T11:21:44.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.277+0000 7f7100d99700 1 -- 172.21.15.92:0/2858048732 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f8011d20 con 0x7f70fc0d2df0 2023-11-28T11:21:44.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.277+0000 7f7103fff700 1 -- 172.21.15.92:0/2858048732 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70fc0d2df0 msgr2=0x7f70fc0cd620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:44.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.277+0000 7f7103fff700 1 --2- 172.21.15.92:0/2858048732 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70fc0d2df0 0x7f70fc0cd620 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f70f8009840 tx=0x7f70f8009b50 comp rx=0 tx=0).stop 2023-11-28T11:21:44.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.278+0000 7f7103fff700 1 -- 172.21.15.92:0/2858048732 shutdown_connections 2023-11-28T11:21:44.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.278+0000 7f7103fff700 1 --2- 172.21.15.92:0/2858048732 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70fc0d2df0 0x7f70fc0cd620 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:44.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.278+0000 7f7103fff700 1 --2- 172.21.15.92:0/2858048732 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70fc0b4fc0 0x7f70fc0d31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:44.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.278+0000 7f7103fff700 1 -- 172.21.15.92:0/2858048732 >> 172.21.15.92:0/2858048732 conn(0x7f70fc0719a0 msgr2=0x7f70fc071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:44.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.278+0000 7f7103fff700 1 -- 172.21.15.92:0/2858048732 shutdown_connections 2023-11-28T11:21:44.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.278+0000 7f7103fff700 1 -- 172.21.15.92:0/2858048732 wait complete. 2023-11-28T11:21:44.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.279+0000 7f7103fff700 1 Processor -- start 2023-11-28T11:21:44.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.279+0000 7f7103fff700 1 -- start start 2023-11-28T11:21:44.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.280+0000 7f7103fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70fc0b1c60 0x7f70fc0cc990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:44.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.280+0000 7f7103fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70fc0b4fc0 0x7f70fc153b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:44.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.280+0000 7f7103fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70fc0d2df0 0x7f70fc0b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:44.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.280+0000 7f7103fff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70fc1540b0 con 0x7f70fc0d2df0 2023-11-28T11:21:44.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.280+0000 7f7103fff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70fc15d050 con 0x7f70fc0b4fc0 2023-11-28T11:21:44.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.280+0000 7f7103fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70fc15d330 con 0x7f70fc0b1c60 2023-11-28T11:21:44.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.281+0000 7f710259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70fc0d2df0 0x7f70fc0b3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:44.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.281+0000 7f710259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70fc0d2df0 0x7f70fc0b3c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51390/0 (socket says 172.21.15.92:51390) 2023-11-28T11:21:44.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.281+0000 7f710259c700 1 -- 172.21.15.92:0/3656582507 learned_addr learned my addr 172.21.15.92:0/3656582507 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:44.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.281+0000 7f7101d9b700 1 --2- 172.21.15.92:0/3656582507 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70fc0b1c60 0x7f70fc0cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:44.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.281+0000 7f710159a700 1 --2- 172.21.15.92:0/3656582507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70fc0b4fc0 0x7f70fc153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:44.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.281+0000 7f710259c700 1 -- 172.21.15.92:0/3656582507 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70fc0b1c60 msgr2=0x7f70fc0cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:44.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.281+0000 7f710259c700 1 --2- 172.21.15.92:0/3656582507 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70fc0b1c60 0x7f70fc0cc990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:44.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.281+0000 7f710259c700 1 -- 172.21.15.92:0/3656582507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70fc0b4fc0 msgr2=0x7f70fc153b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:44.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.281+0000 7f710259c700 1 --2- 172.21.15.92:0/3656582507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70fc0b4fc0 0x7f70fc153b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:44.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.281+0000 7f710259c700 1 -- 172.21.15.92:0/3656582507 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70f80094a0 con 0x7f70fc0d2df0 2023-11-28T11:21:44.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.281+0000 7f7101d9b700 1 --2- 172.21.15.92:0/3656582507 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70fc0b1c60 0x7f70fc0cc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:21:44.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.281+0000 7f710159a700 1 --2- 172.21.15.92:0/3656582507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70fc0b4fc0 0x7f70fc153b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:21:44.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.281+0000 7f710259c700 1 --2- 172.21.15.92:0/3656582507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70fc0d2df0 0x7f70fc0b3c60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f70f400caa0 tx=0x7f70f400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:44.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.281+0000 7f70f27fc700 1 -- 172.21.15.92:0/3656582507 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f400aa80 con 0x7f70fc0d2df0 2023-11-28T11:21:44.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.282+0000 7f70f27fc700 1 -- 172.21.15.92:0/3656582507 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70f400abe0 con 0x7f70fc0d2df0 2023-11-28T11:21:44.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.282+0000 7f70f27fc700 1 -- 172.21.15.92:0/3656582507 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f4012400 con 0x7f70fc0d2df0 2023-11-28T11:21:44.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.282+0000 7f7103fff700 1 -- 172.21.15.92:0/3656582507 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70fc0b88f0 con 0x7f70fc0d2df0 2023-11-28T11:21:44.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.282+0000 7f7103fff700 1 -- 172.21.15.92:0/3656582507 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70fc0d15e0 con 0x7f70fc0d2df0 2023-11-28T11:21:44.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.283+0000 7f7103fff700 1 -- 172.21.15.92:0/3656582507 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70fc0b21b0 con 0x7f70fc0d2df0 2023-11-28T11:21:44.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.284+0000 7f70f27fc700 1 -- 172.21.15.92:0/3656582507 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f70f4019030 con 0x7f70fc0d2df0 2023-11-28T11:21:44.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.284+0000 7f70f27fc700 1 --2- 172.21.15.92:0/3656582507 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f70e00682f0 0x7f70e006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:44.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.285+0000 7f70f27fc700 1 -- 172.21.15.92:0/3656582507 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f70f4085ce0 con 0x7f70fc0d2df0 2023-11-28T11:21:44.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.285+0000 7f7101d9b700 1 --2- 172.21.15.92:0/3656582507 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f70e00682f0 0x7f70e006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:44.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.285+0000 7f7101d9b700 1 --2- 172.21.15.92:0/3656582507 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f70e00682f0 0x7f70e006a7a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f70f8009fd0 tx=0x7f70f801c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:44.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.289+0000 7f70f27fc700 1 -- 172.21.15.92:0/3656582507 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f70f4052120 con 0x7f70fc0d2df0 2023-11-28T11:21:44.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.476+0000 7f7103fff700 1 -- 172.21.15.92:0/3656582507 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f70fc067170 con 0x7f70fc0d2df0 2023-11-28T11:21:44.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.476+0000 7f70f27fc700 1 -- 172.21.15.92:0/3656582507 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7f70f4017020 con 0x7f70fc0d2df0 2023-11-28T11:21:44.477 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-11-28T11:21:44.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.478+0000 7f7103fff700 1 -- 172.21.15.92:0/3656582507 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f70e00682f0 msgr2=0x7f70e006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:44.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.478+0000 7f7103fff700 1 --2- 172.21.15.92:0/3656582507 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f70e00682f0 0x7f70e006a7a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f70f8009fd0 tx=0x7f70f801c040 comp rx=0 tx=0).stop 2023-11-28T11:21:44.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.478+0000 7f7103fff700 1 -- 172.21.15.92:0/3656582507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70fc0d2df0 msgr2=0x7f70fc0b3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:44.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.478+0000 7f7103fff700 1 --2- 172.21.15.92:0/3656582507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70fc0d2df0 0x7f70fc0b3c60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f70f400caa0 tx=0x7f70f400ce60 comp rx=0 tx=0).stop 2023-11-28T11:21:44.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.478+0000 7f7103fff700 1 -- 172.21.15.92:0/3656582507 shutdown_connections 2023-11-28T11:21:44.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.478+0000 7f7103fff700 1 --2- 172.21.15.92:0/3656582507 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f70e00682f0 0x7f70e006a7a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:44.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.478+0000 7f7103fff700 1 --2- 172.21.15.92:0/3656582507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70fc0d2df0 0x7f70fc0b3c60 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:44.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.478+0000 7f7103fff700 1 --2- 172.21.15.92:0/3656582507 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f70fc0b1c60 0x7f70fc0cc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:44.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.478+0000 7f7103fff700 1 --2- 172.21.15.92:0/3656582507 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f70fc0b4fc0 0x7f70fc153b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:44.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.478+0000 7f7103fff700 1 -- 172.21.15.92:0/3656582507 >> 172.21.15.92:0/3656582507 conn(0x7f70fc0719a0 msgr2=0x7f70fc0b71d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:44.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.478+0000 7f7103fff700 1 -- 172.21.15.92:0/3656582507 shutdown_connections 2023-11-28T11:21:44.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.479+0000 7f7103fff700 1 -- 172.21.15.92:0/3656582507 wait complete. 2023-11-28T11:21:44.497 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:21:44.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.697+0000 7f68e0dff700 1 Processor -- start 2023-11-28T11:21:44.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.697+0000 7f68e0dff700 1 -- start start 2023-11-28T11:21:44.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.698+0000 7f68e0dff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68dc0b8510 0x7f68dc06b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:44.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.698+0000 7f68e0dff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f68dc0d0d60 0x7f68dc0d11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:44.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.698+0000 7f68e0dff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68dc0b88e0 con 0x7f68dc0cd460 2023-11-28T11:21:44.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.698+0000 7f68e0dff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68dc0cdbd0 con 0x7f68dc0d0d60 2023-11-28T11:21:44.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.698+0000 7f68e0dff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68dc0d02d0 con 0x7f68dc0b8510 2023-11-28T11:21:44.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.699+0000 7f68dad9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f68dc0d0d60 0x7f68dc0d11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:44.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.699+0000 7f68d9d9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f68dc0cd460 0x7f68dc06beb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:58060/0 (socket says 172.21.15.92:58060) 2023-11-28T11:21:44.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.699+0000 7f68d9d9b700 1 -- 172.21.15.92:0/2320937559 learned_addr learned my addr 172.21.15.92:0/2320937559 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:44.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.699+0000 7f68da59c700 1 --2- 172.21.15.92:0/2320937559 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68dc0b8510 0x7f68dc06b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:44.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.699+0000 7f68dad9d700 1 -- 172.21.15.92:0/2320937559 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68dc0b8510 msgr2=0x7f68dc06b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:44.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.699+0000 7f68dad9d700 1 --2- 172.21.15.92:0/2320937559 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68dc0b8510 0x7f68dc06b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:44.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.699+0000 7f68dad9d700 1 -- 172.21.15.92:0/2320937559 >> v1:172.21.15.92:6789/0 conn(0x7f68dc0cd460 legacy=0x7f68dc06beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:44.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.699+0000 7f68dad9d700 1 -- 172.21.15.92:0/2320937559 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68dc0cd8f0 con 0x7f68dc0d0d60 2023-11-28T11:21:44.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.699+0000 7f68d959a700 1 -- 172.21.15.92:0/2320937559 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2495832501 0 0) 0x7f68dc0b88e0 con 0x7f68dc0cd460 2023-11-28T11:21:44.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.699+0000 7f68dad9d700 1 --2- 172.21.15.92:0/2320937559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68dc0d0d60 0x7f68dc0d11a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f68d4009dc0 tx=0x7f68d400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=414f6cf505b82769 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:44.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.700+0000 7f68d959a700 1 -- 172.21.15.92:0/2320937559 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d400e070 con 0x7f68dc0d0d60 2023-11-28T11:21:44.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.700+0000 7f68d959a700 1 -- 172.21.15.92:0/2320937559 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68d400bb90 con 0x7f68dc0d0d60 2023-11-28T11:21:44.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.700+0000 7f68d959a700 1 -- 172.21.15.92:0/2320937559 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d4012430 con 0x7f68dc0d0d60 2023-11-28T11:21:44.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.700+0000 7f68e0dff700 1 -- 172.21.15.92:0/2320937559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68dc0d0d60 msgr2=0x7f68dc0d11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:44.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.700+0000 7f68e0dff700 1 --2- 172.21.15.92:0/2320937559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68dc0d0d60 0x7f68dc0d11a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f68d4009dc0 tx=0x7f68d400b330 comp rx=0 tx=0).stop 2023-11-28T11:21:44.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.700+0000 7f68e0dff700 1 -- 172.21.15.92:0/2320937559 shutdown_connections 2023-11-28T11:21:44.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.700+0000 7f68e0dff700 1 --2- 172.21.15.92:0/2320937559 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68dc0b8510 0x7f68dc06b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:44.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.700+0000 7f68e0dff700 1 --2- 172.21.15.92:0/2320937559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68dc0d0d60 0x7f68dc0d11a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:44.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.700+0000 7f68e0dff700 1 -- 172.21.15.92:0/2320937559 >> 172.21.15.92:0/2320937559 conn(0x7f68dc0719a0 msgr2=0x7f68dc071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:44.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.700+0000 7f68e0dff700 1 -- 172.21.15.92:0/2320937559 shutdown_connections 2023-11-28T11:21:44.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.700+0000 7f68e0dff700 1 -- 172.21.15.92:0/2320937559 wait complete. 2023-11-28T11:21:44.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.702+0000 7f68e0dff700 1 Processor -- start 2023-11-28T11:21:44.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.702+0000 7f68e0dff700 1 -- start start 2023-11-28T11:21:44.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.703+0000 7f68e0dff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68dc0b8510 0x7f68dc157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:44.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.703+0000 7f68e0dff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68dc0cd460 0x7f68dc1e3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:44.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.703+0000 7f68e0dff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68dc0d0d60 0x7f68dc1e41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:44.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.703+0000 7f68e0dff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68dc0cb5e0 con 0x7f68dc0d0d60 2023-11-28T11:21:44.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.703+0000 7f68e0dff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68dc0cb8c0 con 0x7f68dc0cd460 2023-11-28T11:21:44.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.703+0000 7f68e0dff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68dc0cbba0 con 0x7f68dc0b8510 2023-11-28T11:21:44.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.703+0000 7f68dad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68dc0d0d60 0x7f68dc1e41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:44.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.703+0000 7f68dad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68dc0d0d60 0x7f68dc1e41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51412/0 (socket says 172.21.15.92:51412) 2023-11-28T11:21:44.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.703+0000 7f68d9d9b700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68dc0cd460 0x7f68dc1e3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:44.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.703+0000 7f68da59c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68dc0b8510 0x7f68dc157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:44.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.703+0000 7f68dad9d700 1 -- 172.21.15.92:0/1564804999 learned_addr learned my addr 172.21.15.92:0/1564804999 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:44.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.704+0000 7f68dad9d700 1 -- 172.21.15.92:0/1564804999 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68dc0b8510 msgr2=0x7f68dc157f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:44.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.704+0000 7f68dad9d700 1 --2- 172.21.15.92:0/1564804999 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68dc0b8510 0x7f68dc157f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:44.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.704+0000 7f68dad9d700 1 -- 172.21.15.92:0/1564804999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68dc0cd460 msgr2=0x7f68dc1e3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:44.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.704+0000 7f68dad9d700 1 --2- 172.21.15.92:0/1564804999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68dc0cd460 0x7f68dc1e3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:44.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.704+0000 7f68dad9d700 1 -- 172.21.15.92:0/1564804999 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d4009a70 con 0x7f68dc0d0d60 2023-11-28T11:21:44.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.704+0000 7f68dad9d700 1 --2- 172.21.15.92:0/1564804999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68dc0d0d60 0x7f68dc1e41c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f68d4014040 tx=0x7f68d4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:44.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.704+0000 7f68caffd700 1 -- 172.21.15.92:0/1564804999 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d4012430 con 0x7f68dc0d0d60 2023-11-28T11:21:44.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.704+0000 7f68caffd700 1 -- 172.21.15.92:0/1564804999 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68d4020070 con 0x7f68dc0d0d60 2023-11-28T11:21:44.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.704+0000 7f68e0dff700 1 -- 172.21.15.92:0/1564804999 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68dc0cfff0 con 0x7f68dc0d0d60 2023-11-28T11:21:44.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.704+0000 7f68caffd700 1 -- 172.21.15.92:0/1564804999 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d40098f0 con 0x7f68dc0d0d60 2023-11-28T11:21:44.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.705+0000 7f68e0dff700 1 -- 172.21.15.92:0/1564804999 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68dc0cfd10 con 0x7f68dc0d0d60 2023-11-28T11:21:44.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.706+0000 7f68e0dff700 1 -- 172.21.15.92:0/1564804999 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68dc067880 con 0x7f68dc0d0d60 2023-11-28T11:21:44.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.706+0000 7f68caffd700 1 -- 172.21.15.92:0/1564804999 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f68d4004040 con 0x7f68dc0d0d60 2023-11-28T11:21:44.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.707+0000 7f68caffd700 1 --2- 172.21.15.92:0/1564804999 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f68bc0682f0 0x7f68bc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:44.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.707+0000 7f68caffd700 1 -- 172.21.15.92:0/1564804999 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f68d4022070 con 0x7f68dc0d0d60 2023-11-28T11:21:44.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.707+0000 7f68da59c700 1 --2- 172.21.15.92:0/1564804999 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f68bc0682f0 0x7f68bc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:44.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.708+0000 7f68da59c700 1 --2- 172.21.15.92:0/1564804999 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f68bc0682f0 0x7f68bc06a7a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f68cc006b20 tx=0x7f68cc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:44.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.712+0000 7f68caffd700 1 -- 172.21.15.92:0/1564804999 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f68d4054c20 con 0x7f68dc0d0d60 2023-11-28T11:21:44.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.900+0000 7f68e0dff700 1 -- 172.21.15.92:0/1564804999 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f68dc0d2940 con 0x7f68dc0d0d60 2023-11-28T11:21:44.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.900+0000 7f68caffd700 1 -- 172.21.15.92:0/1564804999 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7f68d4058240 con 0x7f68dc0d0d60 2023-11-28T11:21:44.901 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-11-28T11:21:44.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.902+0000 7f68e0dff700 1 -- 172.21.15.92:0/1564804999 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f68bc0682f0 msgr2=0x7f68bc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:44.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.902+0000 7f68e0dff700 1 --2- 172.21.15.92:0/1564804999 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f68bc0682f0 0x7f68bc06a7a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f68cc006b20 tx=0x7f68cc008040 comp rx=0 tx=0).stop 2023-11-28T11:21:44.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.902+0000 7f68e0dff700 1 -- 172.21.15.92:0/1564804999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68dc0d0d60 msgr2=0x7f68dc1e41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:44.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.902+0000 7f68e0dff700 1 --2- 172.21.15.92:0/1564804999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68dc0d0d60 0x7f68dc1e41c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f68d4014040 tx=0x7f68d4004ad0 comp rx=0 tx=0).stop 2023-11-28T11:21:44.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.903+0000 7f68e0dff700 1 -- 172.21.15.92:0/1564804999 shutdown_connections 2023-11-28T11:21:44.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.903+0000 7f68e0dff700 1 --2- 172.21.15.92:0/1564804999 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f68bc0682f0 0x7f68bc06a7a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:44.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.903+0000 7f68e0dff700 1 --2- 172.21.15.92:0/1564804999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68dc0d0d60 0x7f68dc1e41c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:44.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.903+0000 7f68e0dff700 1 --2- 172.21.15.92:0/1564804999 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68dc0b8510 0x7f68dc157f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:44.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.903+0000 7f68e0dff700 1 --2- 172.21.15.92:0/1564804999 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68dc0cd460 0x7f68dc1e3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:44.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.903+0000 7f68e0dff700 1 -- 172.21.15.92:0/1564804999 >> 172.21.15.92:0/1564804999 conn(0x7f68dc0719a0 msgr2=0x7f68dc0b27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:44.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.903+0000 7f68e0dff700 1 -- 172.21.15.92:0/1564804999 shutdown_connections 2023-11-28T11:21:44.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:44.903+0000 7f68e0dff700 1 -- 172.21.15.92:0/1564804999 wait complete. 2023-11-28T11:21:44.922 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:21:45.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.120+0000 7f253b5ff700 1 Processor -- start 2023-11-28T11:21:45.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.120+0000 7f253b5ff700 1 -- start start 2023-11-28T11:21:45.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.121+0000 7f253b5ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25340cd460 0x7f253406c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:45.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.121+0000 7f253b5ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25340b88e0 con 0x7f25340b8510 2023-11-28T11:21:45.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.121+0000 7f253b5ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25340cd8f0 con 0x7f25340d0d60 2023-11-28T11:21:45.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.121+0000 7f253b5ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25340cdbd0 con 0x7f25340cd460 2023-11-28T11:21:45.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.121+0000 7f253939b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f25340b8510 0x7f253406b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:58062/0 (socket says 172.21.15.92:58062) 2023-11-28T11:21:45.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.121+0000 7f253939b700 1 -- 172.21.15.92:0/2928609129 learned_addr learned my addr 172.21.15.92:0/2928609129 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:45.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.122+0000 7f2538b9a700 1 --2- 172.21.15.92:0/2928609129 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25340cd460 0x7f253406c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:45.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.122+0000 7f252bfff700 1 -- 172.21.15.92:0/2928609129 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1003439449 0 0) 0x7f25340b88e0 con 0x7f25340b8510 2023-11-28T11:21:45.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.122+0000 7f252bfff700 1 -- 172.21.15.92:0/2928609129 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25200035b0 con 0x7f25340b8510 2023-11-28T11:21:45.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.122+0000 7f252bfff700 1 -- 172.21.15.92:0/2928609129 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3006258648 0 0) 0x7f25340cd8f0 con 0x7f25340d0d60 2023-11-28T11:21:45.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.122+0000 7f252bfff700 1 -- 172.21.15.92:0/2928609129 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25340b88e0 con 0x7f25340d0d60 2023-11-28T11:21:45.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.122+0000 7f2538b9a700 1 -- 172.21.15.92:0/2928609129 >> v1:172.21.15.169:6789/0 conn(0x7f25340d0d60 legacy=0x7f25340d11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:45.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.122+0000 7f2538b9a700 1 -- 172.21.15.92:0/2928609129 >> v1:172.21.15.92:6789/0 conn(0x7f25340b8510 legacy=0x7f253406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:45.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.122+0000 7f2538b9a700 1 -- 172.21.15.92:0/2928609129 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25340cfd10 con 0x7f25340cd460 2023-11-28T11:21:45.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.122+0000 7f2538b9a700 1 --2- 172.21.15.92:0/2928609129 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25340cd460 0x7f253406c080 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2524009870 tx=0x7f2524009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1aa0f5629018b2f0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:45.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.123+0000 7f252bfff700 1 -- 172.21.15.92:0/2928609129 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2524011830 con 0x7f25340cd460 2023-11-28T11:21:45.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.123+0000 7f252bfff700 1 -- 172.21.15.92:0/2928609129 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2524011990 con 0x7f25340cd460 2023-11-28T11:21:45.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.123+0000 7f252bfff700 1 -- 172.21.15.92:0/2928609129 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2524011d00 con 0x7f25340cd460 2023-11-28T11:21:45.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.123+0000 7f253b5ff700 1 -- 172.21.15.92:0/2928609129 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25340cd460 msgr2=0x7f253406c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:45.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.123+0000 7f253b5ff700 1 --2- 172.21.15.92:0/2928609129 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25340cd460 0x7f253406c080 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2524009870 tx=0x7f2524009b80 comp rx=0 tx=0).stop 2023-11-28T11:21:45.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.123+0000 7f253b5ff700 1 -- 172.21.15.92:0/2928609129 shutdown_connections 2023-11-28T11:21:45.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.123+0000 7f253b5ff700 1 --2- 172.21.15.92:0/2928609129 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25340cd460 0x7f253406c080 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:45.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.123+0000 7f253b5ff700 1 -- 172.21.15.92:0/2928609129 >> 172.21.15.92:0/2928609129 conn(0x7f25340719a0 msgr2=0x7f2534071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:45.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.123+0000 7f253b5ff700 1 -- 172.21.15.92:0/2928609129 shutdown_connections 2023-11-28T11:21:45.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.123+0000 7f253b5ff700 1 -- 172.21.15.92:0/2928609129 wait complete. 2023-11-28T11:21:45.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.125+0000 7f253b5ff700 1 Processor -- start 2023-11-28T11:21:45.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.125+0000 7f253b5ff700 1 -- start start 2023-11-28T11:21:45.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.126+0000 7f253b5ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25340b8510 0x7f253414d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:45.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.126+0000 7f253b5ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25340cd460 0x7f25341f1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:45.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.126+0000 7f253b5ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25340d0d60 0x7f253415b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:45.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.126+0000 7f253b5ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2534156560 con 0x7f25340b8510 2023-11-28T11:21:45.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.126+0000 7f253b5ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2534156840 con 0x7f25340d0d60 2023-11-28T11:21:45.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.126+0000 7f253b5ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2534156b20 con 0x7f25340cd460 2023-11-28T11:21:45.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.126+0000 7f253939b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25340b8510 0x7f253414d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:45.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.126+0000 7f253939b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25340b8510 0x7f253414d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51420/0 (socket says 172.21.15.92:51420) 2023-11-28T11:21:45.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.126+0000 7f253939b700 1 -- 172.21.15.92:0/2554865775 learned_addr learned my addr 172.21.15.92:0/2554865775 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:45.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.127+0000 7f2538b9a700 1 --2- 172.21.15.92:0/2554865775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25340cd460 0x7f25341f1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:45.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.127+0000 7f2539b9c700 1 --2- 172.21.15.92:0/2554865775 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25340d0d60 0x7f253415b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:45.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.127+0000 7f253939b700 1 -- 172.21.15.92:0/2554865775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25340cd460 msgr2=0x7f25341f1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:45.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.127+0000 7f253939b700 1 --2- 172.21.15.92:0/2554865775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25340cd460 0x7f25341f1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:45.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.127+0000 7f253939b700 1 -- 172.21.15.92:0/2554865775 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25340d0d60 msgr2=0x7f253415b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:45.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.127+0000 7f253939b700 1 --2- 172.21.15.92:0/2554865775 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25340d0d60 0x7f253415b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:45.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.127+0000 7f253939b700 1 -- 172.21.15.92:0/2554865775 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25240094a0 con 0x7f25340b8510 2023-11-28T11:21:45.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.127+0000 7f253939b700 1 --2- 172.21.15.92:0/2554865775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25340b8510 0x7f253414d0b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f253000c700 tx=0x7f253000cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:45.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.127+0000 7f2529ffb700 1 -- 172.21.15.92:0/2554865775 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2530005d70 con 0x7f25340b8510 2023-11-28T11:21:45.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.127+0000 7f2529ffb700 1 -- 172.21.15.92:0/2554865775 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25300070d0 con 0x7f25340b8510 2023-11-28T11:21:45.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.127+0000 7f253b5ff700 1 -- 172.21.15.92:0/2554865775 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25340d29c0 con 0x7f25340b8510 2023-11-28T11:21:45.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.127+0000 7f2529ffb700 1 -- 172.21.15.92:0/2554865775 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2530007460 con 0x7f25340b8510 2023-11-28T11:21:45.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.127+0000 7f253b5ff700 1 -- 172.21.15.92:0/2554865775 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25340cfff0 con 0x7f25340b8510 2023-11-28T11:21:45.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.128+0000 7f253b5ff700 1 -- 172.21.15.92:0/2554865775 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25340d2b50 con 0x7f25340b8510 2023-11-28T11:21:45.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.129+0000 7f2529ffb700 1 -- 172.21.15.92:0/2554865775 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2530019030 con 0x7f25340b8510 2023-11-28T11:21:45.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.130+0000 7f2529ffb700 1 --2- 172.21.15.92:0/2554865775 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f25180682f0 0x7f251806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:45.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.130+0000 7f2529ffb700 1 -- 172.21.15.92:0/2554865775 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f25300855f0 con 0x7f25340b8510 2023-11-28T11:21:45.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.130+0000 7f2538b9a700 1 --2- 172.21.15.92:0/2554865775 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f25180682f0 0x7f251806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:45.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.131+0000 7f2538b9a700 1 --2- 172.21.15.92:0/2554865775 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f25180682f0 0x7f251806a7a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f25240090b0 tx=0x7f2524008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:45.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.134+0000 7f2529ffb700 1 -- 172.21.15.92:0/2554865775 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f25300519b0 con 0x7f25340b8510 2023-11-28T11:21:45.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.321+0000 7f253b5ff700 1 -- 172.21.15.92:0/2554865775 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2534067880 con 0x7f25340b8510 2023-11-28T11:21:45.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.322+0000 7f2529ffb700 1 -- 172.21.15.92:0/2554865775 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2530054fd0 con 0x7f25340b8510 2023-11-28T11:21:45.323 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-11-28T11:21:45.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.324+0000 7f253b5ff700 1 -- 172.21.15.92:0/2554865775 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f25180682f0 msgr2=0x7f251806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:45.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.324+0000 7f253b5ff700 1 --2- 172.21.15.92:0/2554865775 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f25180682f0 0x7f251806a7a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f25240090b0 tx=0x7f2524008030 comp rx=0 tx=0).stop 2023-11-28T11:21:45.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.324+0000 7f253b5ff700 1 -- 172.21.15.92:0/2554865775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25340b8510 msgr2=0x7f253414d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:45.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.324+0000 7f253b5ff700 1 --2- 172.21.15.92:0/2554865775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25340b8510 0x7f253414d0b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f253000c700 tx=0x7f253000cac0 comp rx=0 tx=0).stop 2023-11-28T11:21:45.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.324+0000 7f253b5ff700 1 -- 172.21.15.92:0/2554865775 shutdown_connections 2023-11-28T11:21:45.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.324+0000 7f253b5ff700 1 --2- 172.21.15.92:0/2554865775 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f25340d0d60 0x7f253415b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:45.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.324+0000 7f253b5ff700 1 --2- 172.21.15.92:0/2554865775 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f25180682f0 0x7f251806a7a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:45.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.324+0000 7f253b5ff700 1 --2- 172.21.15.92:0/2554865775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25340b8510 0x7f253414d0b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:45.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.324+0000 7f253b5ff700 1 --2- 172.21.15.92:0/2554865775 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f25340cd460 0x7f25341f1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:45.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.324+0000 7f253b5ff700 1 -- 172.21.15.92:0/2554865775 >> 172.21.15.92:0/2554865775 conn(0x7f25340719a0 msgr2=0x7f25340b27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:45.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.324+0000 7f253b5ff700 1 -- 172.21.15.92:0/2554865775 shutdown_connections 2023-11-28T11:21:45.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.324+0000 7f253b5ff700 1 -- 172.21.15.92:0/2554865775 wait complete. 2023-11-28T11:21:45.343 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T11:21:45.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.540+0000 7f41833ff700 1 Processor -- start 2023-11-28T11:21:45.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.541+0000 7f41833ff700 1 -- start start 2023-11-28T11:21:45.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.542+0000 7f41833ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f417c15eee0 0x7f417c06a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:45.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.542+0000 7f41833ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f417c06a7b0 0x7f417c0c9b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:45.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.542+0000 7f41833ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f417c06ac10 con 0x7f417c15a7a0 2023-11-28T11:21:45.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.542+0000 7f41833ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f417c06cc10 con 0x7f417c15eee0 2023-11-28T11:21:45.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.542+0000 7f41833ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f417c06cef0 con 0x7f417c06a7b0 2023-11-28T11:21:45.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.542+0000 7f418099a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f417c15eee0 0x7f417c06a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:45.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.542+0000 7f418099a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f417c15eee0 0x7f417c06a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51424/0 (socket says 172.21.15.92:51424) 2023-11-28T11:21:45.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.542+0000 7f418099a700 1 -- 172.21.15.92:0/3271220191 learned_addr learned my addr 172.21.15.92:0/3271220191 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:45.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.542+0000 7f418199c700 1 --2- 172.21.15.92:0/3271220191 >> v2:172.21.15.169:3300/0 conn(0x7f417c06a7b0 0x7f417c0c9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:45.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.542+0000 7f418099a700 1 -- 172.21.15.92:0/3271220191 >> v2:172.21.15.169:3300/0 conn(0x7f417c06a7b0 msgr2=0x7f417c0c9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:45.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.542+0000 7f418099a700 1 --2- 172.21.15.92:0/3271220191 >> v2:172.21.15.169:3300/0 conn(0x7f417c06a7b0 0x7f417c0c9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:45.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.542+0000 7f418099a700 1 -- 172.21.15.92:0/3271220191 >> v1:172.21.15.169:6789/0 conn(0x7f417c15a7a0 legacy=0x7f417c06c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:45.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.543+0000 7f418099a700 1 -- 172.21.15.92:0/3271220191 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f417c0c9070 con 0x7f417c15eee0 2023-11-28T11:21:45.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.543+0000 7f418199c700 1 --2- 172.21.15.92:0/3271220191 >> v2:172.21.15.169:3300/0 conn(0x7f417c06a7b0 0x7f417c0c9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:21:45.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.543+0000 7f418099a700 1 --2- 172.21.15.92:0/3271220191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f417c15eee0 0x7f417c06a270 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f417c06b3f0 tx=0x7f416c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56c384e4a186b725 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:45.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.543+0000 7f4173fff700 1 -- 172.21.15.92:0/3271220191 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416c004230 con 0x7f417c15eee0 2023-11-28T11:21:45.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.543+0000 7f4173fff700 1 -- 172.21.15.92:0/3271220191 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f416c004390 con 0x7f417c15eee0 2023-11-28T11:21:45.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.543+0000 7f4173fff700 1 -- 172.21.15.92:0/3271220191 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416c0036a0 con 0x7f417c15eee0 2023-11-28T11:21:45.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.543+0000 7f41833ff700 1 -- 172.21.15.92:0/3271220191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f417c15eee0 msgr2=0x7f417c06a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:45.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.543+0000 7f41833ff700 1 --2- 172.21.15.92:0/3271220191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f417c15eee0 0x7f417c06a270 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f417c06b3f0 tx=0x7f416c009bd0 comp rx=0 tx=0).stop 2023-11-28T11:21:45.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.544+0000 7f41833ff700 1 -- 172.21.15.92:0/3271220191 shutdown_connections 2023-11-28T11:21:45.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.544+0000 7f41833ff700 1 --2- 172.21.15.92:0/3271220191 >> v2:172.21.15.169:3300/0 conn(0x7f417c06a7b0 0x7f417c0c9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:45.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.544+0000 7f41833ff700 1 --2- 172.21.15.92:0/3271220191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f417c15eee0 0x7f417c06a270 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:45.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.544+0000 7f41833ff700 1 -- 172.21.15.92:0/3271220191 >> 172.21.15.92:0/3271220191 conn(0x7f417c0cb490 msgr2=0x7f417c0cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:45.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.544+0000 7f41833ff700 1 -- 172.21.15.92:0/3271220191 shutdown_connections 2023-11-28T11:21:45.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.544+0000 7f41833ff700 1 -- 172.21.15.92:0/3271220191 wait complete. 2023-11-28T11:21:45.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.545+0000 7f41833ff700 1 Processor -- start 2023-11-28T11:21:45.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.546+0000 7f41833ff700 1 -- start start 2023-11-28T11:21:45.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.546+0000 7f41833ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f417c06a7b0 0x7f417c1558d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:45.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.546+0000 7f41833ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f417c15a7a0 0x7f417c1f5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:45.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.546+0000 7f41833ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f417c15eee0 0x7f417c0b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:45.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.546+0000 7f41833ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417c0b2790 con 0x7f417c15eee0 2023-11-28T11:21:45.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.546+0000 7f41833ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417c156170 con 0x7f417c06a7b0 2023-11-28T11:21:45.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.547+0000 7f41833ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f417c0c8260 con 0x7f417c15a7a0 2023-11-28T11:21:45.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.547+0000 7f418199c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f417c15eee0 0x7f417c0b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:45.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.547+0000 7f418199c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f417c15eee0 0x7f417c0b31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51440/0 (socket says 172.21.15.92:51440) 2023-11-28T11:21:45.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.547+0000 7f418199c700 1 -- 172.21.15.92:0/2366990582 learned_addr learned my addr 172.21.15.92:0/2366990582 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:45.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.547+0000 7f418119b700 1 --2- 172.21.15.92:0/2366990582 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f417c06a7b0 0x7f417c1558d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:45.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.547+0000 7f418099a700 1 --2- 172.21.15.92:0/2366990582 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f417c15a7a0 0x7f417c1f5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:45.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.547+0000 7f418199c700 1 -- 172.21.15.92:0/2366990582 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f417c15a7a0 msgr2=0x7f417c1f5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:45.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.547+0000 7f418199c700 1 --2- 172.21.15.92:0/2366990582 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f417c15a7a0 0x7f417c1f5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:45.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.547+0000 7f418199c700 1 -- 172.21.15.92:0/2366990582 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f417c06a7b0 msgr2=0x7f417c1558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:45.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.547+0000 7f418199c700 1 --2- 172.21.15.92:0/2366990582 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f417c06a7b0 0x7f417c1558d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:45.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.547+0000 7f418199c700 1 -- 172.21.15.92:0/2366990582 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f416c009550 con 0x7f417c15eee0 2023-11-28T11:21:45.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.547+0000 7f418199c700 1 --2- 172.21.15.92:0/2366990582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f417c15eee0 0x7f417c0b31e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f417400cd50 tx=0x7f417400a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:45.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.548+0000 7f4171ffb700 1 -- 172.21.15.92:0/2366990582 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417400f480 con 0x7f417c15eee0 2023-11-28T11:21:45.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.548+0000 7f4171ffb700 1 -- 172.21.15.92:0/2366990582 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f417400f600 con 0x7f417c15eee0 2023-11-28T11:21:45.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.548+0000 7f41833ff700 1 -- 172.21.15.92:0/2366990582 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f417c0c9630 con 0x7f417c15eee0 2023-11-28T11:21:45.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.548+0000 7f4171ffb700 1 -- 172.21.15.92:0/2366990582 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417400f9b0 con 0x7f417c15eee0 2023-11-28T11:21:45.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.548+0000 7f41833ff700 1 -- 172.21.15.92:0/2366990582 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f417c0c9350 con 0x7f417c15eee0 2023-11-28T11:21:45.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.549+0000 7f41833ff700 1 -- 172.21.15.92:0/2366990582 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f417c15ab70 con 0x7f417c15eee0 2023-11-28T11:21:45.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.550+0000 7f4171ffb700 1 -- 172.21.15.92:0/2366990582 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4174009030 con 0x7f417c15eee0 2023-11-28T11:21:45.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.550+0000 7f4171ffb700 1 --2- 172.21.15.92:0/2366990582 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f41600682f0 0x7f416006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:45.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.551+0000 7f4171ffb700 1 -- 172.21.15.92:0/2366990582 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f4174086ab0 con 0x7f417c15eee0 2023-11-28T11:21:45.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.551+0000 7f418119b700 1 --2- 172.21.15.92:0/2366990582 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f41600682f0 0x7f416006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:45.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.551+0000 7f418119b700 1 --2- 172.21.15.92:0/2366990582 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f41600682f0 0x7f416006a7a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4178004f30 tx=0x7f4178009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:45.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.554+0000 7f4171ffb700 1 -- 172.21.15.92:0/2366990582 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4174052ef0 con 0x7f417c15eee0 2023-11-28T11:21:45.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.746+0000 7f41833ff700 1 -- 172.21.15.92:0/2366990582 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f417c067880 con 0x7f417c15eee0 2023-11-28T11:21:45.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.746+0000 7f4171ffb700 1 -- 172.21.15.92:0/2366990582 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7f4174056510 con 0x7f417c15eee0 2023-11-28T11:21:45.747 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-11-28T11:21:45.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.749+0000 7f41833ff700 1 -- 172.21.15.92:0/2366990582 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f41600682f0 msgr2=0x7f416006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:45.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.749+0000 7f41833ff700 1 --2- 172.21.15.92:0/2366990582 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f41600682f0 0x7f416006a7a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4178004f30 tx=0x7f4178009040 comp rx=0 tx=0).stop 2023-11-28T11:21:45.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.749+0000 7f41833ff700 1 -- 172.21.15.92:0/2366990582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f417c15eee0 msgr2=0x7f417c0b31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:45.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.749+0000 7f41833ff700 1 --2- 172.21.15.92:0/2366990582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f417c15eee0 0x7f417c0b31e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f417400cd50 tx=0x7f417400a560 comp rx=0 tx=0).stop 2023-11-28T11:21:45.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.749+0000 7f41833ff700 1 -- 172.21.15.92:0/2366990582 shutdown_connections 2023-11-28T11:21:45.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.749+0000 7f41833ff700 1 --2- 172.21.15.92:0/2366990582 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f41600682f0 0x7f416006a7a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:45.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.749+0000 7f41833ff700 1 --2- 172.21.15.92:0/2366990582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f417c15eee0 0x7f417c0b31e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:45.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.749+0000 7f41833ff700 1 --2- 172.21.15.92:0/2366990582 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f417c06a7b0 0x7f417c1558d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:45.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.749+0000 7f41833ff700 1 --2- 172.21.15.92:0/2366990582 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f417c15a7a0 0x7f417c1f5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:45.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.749+0000 7f41833ff700 1 -- 172.21.15.92:0/2366990582 >> 172.21.15.92:0/2366990582 conn(0x7f417c0cb490 msgr2=0x7f417c0ae810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:45.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.749+0000 7f41833ff700 1 -- 172.21.15.92:0/2366990582 shutdown_connections 2023-11-28T11:21:45.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.749+0000 7f41833ff700 1 -- 172.21.15.92:0/2366990582 wait complete. 2023-11-28T11:21:45.768 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:21:45.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.965+0000 7f623e3ff700 1 Processor -- start 2023-11-28T11:21:45.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.965+0000 7f623e3ff700 1 -- start start 2023-11-28T11:21:45.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.966+0000 7f623e3ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f62380cd460 0x7f623806c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:45.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.966+0000 7f623e3ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f62380d0d60 0x7f62380d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:45.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.966+0000 7f623e3ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62380b88e0 con 0x7f62380b8510 2023-11-28T11:21:45.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.966+0000 7f623e3ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62380cdbd0 con 0x7f62380d0d60 2023-11-28T11:21:45.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.966+0000 7f623e3ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62380d02d0 con 0x7f62380cd460 2023-11-28T11:21:45.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.967+0000 7f623c99c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f62380d0d60 0x7f62380d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:45.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.967+0000 7f6237fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f62380b8510 0x7f623806b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:58064/0 (socket says 172.21.15.92:58064) 2023-11-28T11:21:45.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.967+0000 7f6237fff700 1 -- 172.21.15.92:0/2878732027 learned_addr learned my addr 172.21.15.92:0/2878732027 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:45.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.967+0000 7f62377fe700 1 --2- 172.21.15.92:0/2878732027 >> v2:172.21.15.169:3300/0 conn(0x7f62380cd460 0x7f623806c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:45.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.967+0000 7f6236ffd700 1 -- 172.21.15.92:0/2878732027 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2736322688 0 0) 0x7f62380b88e0 con 0x7f62380b8510 2023-11-28T11:21:45.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.967+0000 7f6236ffd700 1 -- 172.21.15.92:0/2878732027 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62200035b0 con 0x7f62380b8510 2023-11-28T11:21:45.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.967+0000 7f623c99c700 1 -- 172.21.15.92:0/2878732027 >> v2:172.21.15.169:3300/0 conn(0x7f62380cd460 msgr2=0x7f623806c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:45.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.967+0000 7f623c99c700 1 --2- 172.21.15.92:0/2878732027 >> v2:172.21.15.169:3300/0 conn(0x7f62380cd460 0x7f623806c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:45.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.967+0000 7f623c99c700 1 -- 172.21.15.92:0/2878732027 >> v1:172.21.15.92:6789/0 conn(0x7f62380b8510 legacy=0x7f623806b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:45.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.967+0000 7f623c99c700 1 -- 172.21.15.92:0/2878732027 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62380cd8f0 con 0x7f62380d0d60 2023-11-28T11:21:45.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.967+0000 7f62377fe700 1 --2- 172.21.15.92:0/2878732027 >> v2:172.21.15.169:3300/0 conn(0x7f62380cd460 0x7f623806c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:21:45.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.967+0000 7f623c99c700 1 --2- 172.21.15.92:0/2878732027 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62380d0d60 0x7f62380d1200 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f62380ceba0 tx=0x7f623000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2adb4750990cd4a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:45.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.968+0000 7f6236ffd700 1 -- 172.21.15.92:0/2878732027 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623000e070 con 0x7f62380d0d60 2023-11-28T11:21:45.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.968+0000 7f6236ffd700 1 -- 172.21.15.92:0/2878732027 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f623000bae0 con 0x7f62380d0d60 2023-11-28T11:21:45.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.968+0000 7f6236ffd700 1 -- 172.21.15.92:0/2878732027 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623000be90 con 0x7f62380d0d60 2023-11-28T11:21:45.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.968+0000 7f623e3ff700 1 -- 172.21.15.92:0/2878732027 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62380d0d60 msgr2=0x7f62380d1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:45.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.968+0000 7f623e3ff700 1 --2- 172.21.15.92:0/2878732027 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62380d0d60 0x7f62380d1200 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f62380ceba0 tx=0x7f623000b330 comp rx=0 tx=0).stop 2023-11-28T11:21:45.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.968+0000 7f623e3ff700 1 -- 172.21.15.92:0/2878732027 shutdown_connections 2023-11-28T11:21:45.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.968+0000 7f623e3ff700 1 --2- 172.21.15.92:0/2878732027 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62380d0d60 0x7f62380d1200 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:45.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.968+0000 7f623e3ff700 1 --2- 172.21.15.92:0/2878732027 >> v2:172.21.15.169:3300/0 conn(0x7f62380cd460 0x7f623806c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:45.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.968+0000 7f623e3ff700 1 -- 172.21.15.92:0/2878732027 >> 172.21.15.92:0/2878732027 conn(0x7f62380719a0 msgr2=0x7f6238071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:45.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.968+0000 7f623e3ff700 1 -- 172.21.15.92:0/2878732027 shutdown_connections 2023-11-28T11:21:45.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.968+0000 7f623e3ff700 1 -- 172.21.15.92:0/2878732027 wait complete. 2023-11-28T11:21:45.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.970+0000 7f623e3ff700 1 Processor -- start 2023-11-28T11:21:45.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.970+0000 7f623e3ff700 1 -- start start 2023-11-28T11:21:45.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.971+0000 7f623e3ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62380b8510 0x7f6238155f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:45.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.971+0000 7f623e3ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62380cd460 0x7f62381f1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:45.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.971+0000 7f623e3ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62380d0d60 0x7f62381f24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:45.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.971+0000 7f623e3ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62381e1a80 con 0x7f62380d0d60 2023-11-28T11:21:45.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.971+0000 7f623e3ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62381e1d60 con 0x7f62380b8510 2023-11-28T11:21:45.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.971+0000 7f623e3ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62381e2040 con 0x7f62380cd460 2023-11-28T11:21:45.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.971+0000 7f623c99c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62380d0d60 0x7f62381f24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:45.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.971+0000 7f623c99c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62380d0d60 0x7f62381f24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51462/0 (socket says 172.21.15.92:51462) 2023-11-28T11:21:45.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.971+0000 7f623c99c700 1 -- 172.21.15.92:0/517251854 learned_addr learned my addr 172.21.15.92:0/517251854 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:45.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.972+0000 7f62377fe700 1 --2- 172.21.15.92:0/517251854 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62380cd460 0x7f62381f1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:45.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.972+0000 7f6237fff700 1 --2- 172.21.15.92:0/517251854 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62380b8510 0x7f6238155f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:45.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.972+0000 7f623c99c700 1 -- 172.21.15.92:0/517251854 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62380cd460 msgr2=0x7f62381f1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:45.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.972+0000 7f623c99c700 1 --2- 172.21.15.92:0/517251854 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62380cd460 0x7f62381f1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:45.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.972+0000 7f623c99c700 1 -- 172.21.15.92:0/517251854 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62380b8510 msgr2=0x7f6238155f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:45.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.972+0000 7f623c99c700 1 --2- 172.21.15.92:0/517251854 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62380b8510 0x7f6238155f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:45.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.972+0000 7f623c99c700 1 -- 172.21.15.92:0/517251854 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6230009a70 con 0x7f62380d0d60 2023-11-28T11:21:45.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.972+0000 7f62377fe700 1 --2- 172.21.15.92:0/517251854 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62380cd460 0x7f62381f1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:21:45.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.972+0000 7f6237fff700 1 --2- 172.21.15.92:0/517251854 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62380b8510 0x7f6238155f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:21:45.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.972+0000 7f623c99c700 1 --2- 172.21.15.92:0/517251854 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62380d0d60 0x7f62381f24a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6230009f00 tx=0x7f6230004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:45.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.972+0000 7f6234ff9700 1 -- 172.21.15.92:0/517251854 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62300046b0 con 0x7f62380d0d60 2023-11-28T11:21:45.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.972+0000 7f6234ff9700 1 -- 172.21.15.92:0/517251854 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6230020070 con 0x7f62380d0d60 2023-11-28T11:21:45.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.972+0000 7f623e3ff700 1 -- 172.21.15.92:0/517251854 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62380cfff0 con 0x7f62380d0d60 2023-11-28T11:21:45.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.972+0000 7f6234ff9700 1 -- 172.21.15.92:0/517251854 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6230027480 con 0x7f62380d0d60 2023-11-28T11:21:45.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.972+0000 7f623e3ff700 1 -- 172.21.15.92:0/517251854 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62380cfd10 con 0x7f62380d0d60 2023-11-28T11:21:45.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.973+0000 7f623e3ff700 1 -- 172.21.15.92:0/517251854 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6238067880 con 0x7f62380d0d60 2023-11-28T11:21:45.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.974+0000 7f6234ff9700 1 -- 172.21.15.92:0/517251854 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f62300275e0 con 0x7f62380d0d60 2023-11-28T11:21:45.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.975+0000 7f6234ff9700 1 --2- 172.21.15.92:0/517251854 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f62240682f0 0x7f622406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:45.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.975+0000 7f6234ff9700 1 -- 172.21.15.92:0/517251854 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f6230022070 con 0x7f62380d0d60 2023-11-28T11:21:45.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.975+0000 7f6237fff700 1 --2- 172.21.15.92:0/517251854 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f62240682f0 0x7f622406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:45.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.976+0000 7f6237fff700 1 --2- 172.21.15.92:0/517251854 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f62240682f0 0x7f622406a7a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f62280011e0 tx=0x7f6228009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:45.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:45.980+0000 7f6234ff9700 1 -- 172.21.15.92:0/517251854 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6230054870 con 0x7f62380d0d60 2023-11-28T11:21:46.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.172+0000 7f623e3ff700 1 -- 172.21.15.92:0/517251854 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f62380d2940 con 0x7f62380d0d60 2023-11-28T11:21:46.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.173+0000 7f6234ff9700 1 -- 172.21.15.92:0/517251854 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+19844 (secure 0 0 0) 0x7f6230057e90 con 0x7f62380d0d60 2023-11-28T11:21:46.174 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:21:46.175 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":29,"fsid":"68956e0c-c16f-47cd-a220-f98364b0831d","created":"2023-11-28T11:20:28.697573+0000","modified":"2023-11-28T11:21:37.638578+0000","last_up_change":"2023-11-28T11:20:41.257011+0000","last_in_change":"2023-11-28T11:20:32.249767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:20:41.925237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:20:48.682497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:20:54.471398+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:20:55.204585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:20:58.297307+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6abe483f-0e03-4a26-b9cf-62041291747e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":115823},{"type":"v1","addr":"172.21.15.92:6809","nonce":115823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":115823},{"type":"v1","addr":"172.21.15.92:6813","nonce":115823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":115823},{"type":"v1","addr":"172.21.15.92:6821","nonce":115823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":115823},{"type":"v1","addr":"172.21.15.92:6817","nonce":115823}]},"public_addr":"172.21.15.92:6809/115823","cluster_addr":"172.21.15.92:6813/115823","heartbeat_back_addr":"172.21.15.92:6821/115823","heartbeat_front_addr":"172.21.15.92:6817/115823","state":["exists","up"]},{"osd":1,"uuid":"13c1de73-1a8c-4d7e-8a28-9dae52dfe089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":115824},{"type":"v1","addr":"172.21.15.92:6812","nonce":115824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":115824},{"type":"v1","addr":"172.21.15.92:6816","nonce":115824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":115824},{"type":"v1","addr":"172.21.15.92:6823","nonce":115824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":115824},{"type":"v1","addr":"172.21.15.92:6820","nonce":115824}]},"public_addr":"172.21.15.92:6812/115824","cluster_addr":"172.21.15.92:6816/115824","heartbeat_back_addr":"172.21.15.92:6823/115824","heartbeat_front_addr":"172.21.15.92:6820/115824","state":["exists","up"]},{"osd":2,"uuid":"25788a55-112f-4f00-810d-2bc37cf76778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":115821},{"type":"v1","addr":"172.21.15.92:6801","nonce":115821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":115821},{"type":"v1","addr":"172.21.15.92:6803","nonce":115821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":115821},{"type":"v1","addr":"172.21.15.92:6807","nonce":115821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":115821},{"type":"v1","addr":"172.21.15.92:6805","nonce":115821}]},"public_addr":"172.21.15.92:6801/115821","cluster_addr":"172.21.15.92:6803/115821","heartbeat_back_addr":"172.21.15.92:6807/115821","heartbeat_front_addr":"172.21.15.92:6805/115821","state":["exists","up"]},{"osd":3,"uuid":"adf54e24-33a9-41fe-9ce3-98a5ab6c158e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":115825},{"type":"v1","addr":"172.21.15.92:6825","nonce":115825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":115825},{"type":"v1","addr":"172.21.15.92:6827","nonce":115825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":115825},{"type":"v1","addr":"172.21.15.92:6831","nonce":115825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":115825},{"type":"v1","addr":"172.21.15.92:6829","nonce":115825}]},"public_addr":"172.21.15.92:6825/115825","cluster_addr":"172.21.15.92:6827/115825","heartbeat_back_addr":"172.21.15.92:6831/115825","heartbeat_front_addr":"172.21.15.92:6829/115825","state":["exists","up"]},{"osd":4,"uuid":"ccb9b747-f399-44ea-9a18-9db3152227e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":116035},{"type":"v1","addr":"172.21.15.169:6817","nonce":116035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":116035},{"type":"v1","addr":"172.21.15.169:6819","nonce":116035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":116035},{"type":"v1","addr":"172.21.15.169:6826","nonce":116035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":116035},{"type":"v1","addr":"172.21.15.169:6822","nonce":116035}]},"public_addr":"172.21.15.169:6817/116035","cluster_addr":"172.21.15.169:6819/116035","heartbeat_back_addr":"172.21.15.169:6826/116035","heartbeat_front_addr":"172.21.15.169:6822/116035","state":["exists","up"]},{"osd":5,"uuid":"308a1e63-75aa-4d7d-8a7c-cecc24714fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":116033},{"type":"v1","addr":"172.21.15.169:6801","nonce":116033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":116033},{"type":"v1","addr":"172.21.15.169:6803","nonce":116033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":116033},{"type":"v1","addr":"172.21.15.169:6807","nonce":116033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":116033},{"type":"v1","addr":"172.21.15.169:6805","nonce":116033}]},"public_addr":"172.21.15.169:6801/116033","cluster_addr":"172.21.15.169:6803/116033","heartbeat_back_addr":"172.21.15.169:6807/116033","heartbeat_front_addr":"172.21.15.169:6805/116033","state":["exists","up"]},{"osd":6,"uuid":"9c459b7b-b821-4b4b-b91e-6aaae271457c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":116034},{"type":"v1","addr":"172.21.15.169:6809","nonce":116034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":116034},{"type":"v1","addr":"172.21.15.169:6811","nonce":116034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":116034},{"type":"v1","addr":"172.21.15.169:6815","nonce":116034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":116034},{"type":"v1","addr":"172.21.15.169:6813","nonce":116034}]},"public_addr":"172.21.15.169:6809/116034","cluster_addr":"172.21.15.169:6811/116034","heartbeat_back_addr":"172.21.15.169:6815/116034","heartbeat_front_addr":"172.21.15.169:6813/116034","state":["exists","up"]},{"osd":7,"uuid":"1ca9aa4a-755d-4e28-a937-4d8dccab4df3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":116044},{"type":"v1","addr":"172.21.15.169:6823","nonce":116044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6825","nonce":116044},{"type":"v1","addr":"172.21.15.169:6827","nonce":116044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":116044},{"type":"v1","addr":"172.21.15.169:6831","nonce":116044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":116044},{"type":"v1","addr":"172.21.15.169:6829","nonce":116044}]},"public_addr":"172.21.15.169:6823/116044","cluster_addr":"172.21.15.169:6827/116044","heartbeat_back_addr":"172.21.15.169:6831/116044","heartbeat_front_addr":"172.21.15.169:6829/116044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.586781+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.869778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.508594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.860493+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.872267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.746933+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.611520+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.871446+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.1d","mappings":[{"from":3,"to":2}]},{"pgid":"4.21","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:21:46.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.176+0000 7f623e3ff700 1 -- 172.21.15.92:0/517251854 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f62240682f0 msgr2=0x7f622406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:46.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.176+0000 7f623e3ff700 1 --2- 172.21.15.92:0/517251854 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f62240682f0 0x7f622406a7a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f62280011e0 tx=0x7f6228009040 comp rx=0 tx=0).stop 2023-11-28T11:21:46.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.176+0000 7f623e3ff700 1 -- 172.21.15.92:0/517251854 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62380d0d60 msgr2=0x7f62381f24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:46.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.176+0000 7f623e3ff700 1 --2- 172.21.15.92:0/517251854 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62380d0d60 0x7f62381f24a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6230009f00 tx=0x7f6230004ad0 comp rx=0 tx=0).stop 2023-11-28T11:21:46.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.176+0000 7f623e3ff700 1 -- 172.21.15.92:0/517251854 shutdown_connections 2023-11-28T11:21:46.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.176+0000 7f623e3ff700 1 --2- 172.21.15.92:0/517251854 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f62240682f0 0x7f622406a7a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:46.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.176+0000 7f623e3ff700 1 --2- 172.21.15.92:0/517251854 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62380d0d60 0x7f62381f24a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:46.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.176+0000 7f623e3ff700 1 --2- 172.21.15.92:0/517251854 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f62380b8510 0x7f6238155f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:46.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.176+0000 7f623e3ff700 1 --2- 172.21.15.92:0/517251854 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f62380cd460 0x7f62381f1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:46.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.176+0000 7f623e3ff700 1 -- 172.21.15.92:0/517251854 >> 172.21.15.92:0/517251854 conn(0x7f62380719a0 msgr2=0x7f62380b3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:46.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.176+0000 7f623e3ff700 1 -- 172.21.15.92:0/517251854 shutdown_connections 2023-11-28T11:21:46.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.176+0000 7f623e3ff700 1 -- 172.21.15.92:0/517251854 wait complete. 2023-11-28T11:21:46.194 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:20:41.925237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:20:48.682497+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:20:54.471398+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:20:55.204585+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T11:20:58.297307+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T11:21:46.195 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:21:46.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.393+0000 7fdba11ff700 1 Processor -- start 2023-11-28T11:21:46.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.393+0000 7fdba11ff700 1 -- start start 2023-11-28T11:21:46.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.394+0000 7fdba11ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb9c15a110 0x7fdb9c160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:46.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.394+0000 7fdba11ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fdb9c15d820 0x7fdb9c0cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:46.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.394+0000 7fdba11ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb9c161070 con 0x7fdb9c0cfcf0 2023-11-28T11:21:46.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.394+0000 7fdba11ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb9c0cdae0 con 0x7fdb9c15d820 2023-11-28T11:21:46.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.394+0000 7fdba11ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9c0ce4f0 con 0x7fdb9c15a110 2023-11-28T11:21:46.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.394+0000 7fdb9b59e700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fdb9c0cfcf0 0x7fdb9c0b19b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:58076/0 (socket says 172.21.15.92:58076) 2023-11-28T11:21:46.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.394+0000 7fdb9b59e700 1 -- 172.21.15.92:0/562667706 learned_addr learned my addr 172.21.15.92:0/562667706 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:46.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.394+0000 7fdb9ad9d700 1 --2- 172.21.15.92:0/562667706 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb9c15a110 0x7fdb9c160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:46.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.394+0000 7fdb93fff700 1 --2- 172.21.15.92:0/562667706 >> v2:172.21.15.169:3300/0 conn(0x7fdb9c15d820 0x7fdb9c0cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:46.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.395+0000 7fdb9a59c700 1 -- 172.21.15.92:0/562667706 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3407204682 0 0) 0x7fdb9c161070 con 0x7fdb9c0cfcf0 2023-11-28T11:21:46.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.395+0000 7fdb9a59c700 1 -- 172.21.15.92:0/562667706 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb880035b0 con 0x7fdb9c0cfcf0 2023-11-28T11:21:46.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.395+0000 7fdb9ad9d700 1 -- 172.21.15.92:0/562667706 >> v2:172.21.15.169:3300/0 conn(0x7fdb9c15d820 msgr2=0x7fdb9c0cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:46.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.395+0000 7fdb9ad9d700 1 --2- 172.21.15.92:0/562667706 >> v2:172.21.15.169:3300/0 conn(0x7fdb9c15d820 0x7fdb9c0cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:46.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.395+0000 7fdb9ad9d700 1 -- 172.21.15.92:0/562667706 >> v1:172.21.15.92:6789/0 conn(0x7fdb9c0cfcf0 legacy=0x7fdb9c0b19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:46.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.395+0000 7fdb9ad9d700 1 -- 172.21.15.92:0/562667706 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb9c0d0170 con 0x7fdb9c15a110 2023-11-28T11:21:46.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.395+0000 7fdb9ad9d700 1 --2- 172.21.15.92:0/562667706 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb9c15a110 0x7fdb9c160b30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c009840 tx=0x7fdb8c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e13311adbad29dc9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:46.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.396+0000 7fdb9a59c700 1 -- 172.21.15.92:0/562667706 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c011830 con 0x7fdb9c15a110 2023-11-28T11:21:46.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.396+0000 7fdb9a59c700 1 -- 172.21.15.92:0/562667706 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb8c011990 con 0x7fdb9c15a110 2023-11-28T11:21:46.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.396+0000 7fdb9a59c700 1 -- 172.21.15.92:0/562667706 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c011d20 con 0x7fdb9c15a110 2023-11-28T11:21:46.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.396+0000 7fdba11ff700 1 -- 172.21.15.92:0/562667706 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb9c15a110 msgr2=0x7fdb9c160b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:46.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.396+0000 7fdba11ff700 1 --2- 172.21.15.92:0/562667706 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb9c15a110 0x7fdb9c160b30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c009840 tx=0x7fdb8c009b50 comp rx=0 tx=0).stop 2023-11-28T11:21:46.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.396+0000 7fdba11ff700 1 -- 172.21.15.92:0/562667706 shutdown_connections 2023-11-28T11:21:46.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.396+0000 7fdba11ff700 1 --2- 172.21.15.92:0/562667706 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb9c15a110 0x7fdb9c160b30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:46.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.396+0000 7fdba11ff700 1 --2- 172.21.15.92:0/562667706 >> v2:172.21.15.169:3300/0 conn(0x7fdb9c15d820 0x7fdb9c0cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:46.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.396+0000 7fdba11ff700 1 -- 172.21.15.92:0/562667706 >> 172.21.15.92:0/562667706 conn(0x7fdb9c0719a0 msgr2=0x7fdb9c071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:46.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.396+0000 7fdba11ff700 1 -- 172.21.15.92:0/562667706 shutdown_connections 2023-11-28T11:21:46.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.396+0000 7fdba11ff700 1 -- 172.21.15.92:0/562667706 wait complete. 2023-11-28T11:21:46.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.398+0000 7fdba11ff700 1 Processor -- start 2023-11-28T11:21:46.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.398+0000 7fdba11ff700 1 -- start start 2023-11-28T11:21:46.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.399+0000 7fdba11ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb9c0cfcf0 0x7fdb9c157430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:46.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.399+0000 7fdba11ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb9c15a110 0x7fdb9c0b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:46.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.399+0000 7fdba11ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb9c15d820 0x7fdb9c0b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:46.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.399+0000 7fdba11ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9c0cb940 con 0x7fdb9c0cfcf0 2023-11-28T11:21:46.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.399+0000 7fdba11ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9c0cad30 con 0x7fdb9c15d820 2023-11-28T11:21:46.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.399+0000 7fdba11ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9c0cb010 con 0x7fdb9c15a110 2023-11-28T11:21:46.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.399+0000 7fdb9ad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb9c0cfcf0 0x7fdb9c157430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:46.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.399+0000 7fdb9ad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb9c0cfcf0 0x7fdb9c157430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51472/0 (socket says 172.21.15.92:51472) 2023-11-28T11:21:46.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.399+0000 7fdb9ad9d700 1 -- 172.21.15.92:0/3131033406 learned_addr learned my addr 172.21.15.92:0/3131033406 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:46.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.399+0000 7fdb9b59e700 1 --2- 172.21.15.92:0/3131033406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb9c15d820 0x7fdb9c0b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:46.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.399+0000 7fdb93fff700 1 --2- 172.21.15.92:0/3131033406 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb9c15a110 0x7fdb9c0b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:46.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.400+0000 7fdb9ad9d700 1 -- 172.21.15.92:0/3131033406 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb9c15a110 msgr2=0x7fdb9c0b38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:46.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.400+0000 7fdb9ad9d700 1 --2- 172.21.15.92:0/3131033406 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb9c15a110 0x7fdb9c0b38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:46.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.400+0000 7fdb9ad9d700 1 -- 172.21.15.92:0/3131033406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb9c15d820 msgr2=0x7fdb9c0b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:46.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.400+0000 7fdb9ad9d700 1 --2- 172.21.15.92:0/3131033406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb9c15d820 0x7fdb9c0b3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:46.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.400+0000 7fdb9ad9d700 1 -- 172.21.15.92:0/3131033406 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb8c0094a0 con 0x7fdb9c0cfcf0 2023-11-28T11:21:46.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.400+0000 7fdb9ad9d700 1 --2- 172.21.15.92:0/3131033406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb9c0cfcf0 0x7fdb9c157430 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c009fd0 tx=0x7fdb8c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:46.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.400+0000 7fdb937fe700 1 -- 172.21.15.92:0/3131033406 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c004590 con 0x7fdb9c0cfcf0 2023-11-28T11:21:46.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.400+0000 7fdb937fe700 1 -- 172.21.15.92:0/3131033406 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb8c01e070 con 0x7fdb9c0cfcf0 2023-11-28T11:21:46.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.400+0000 7fdb937fe700 1 -- 172.21.15.92:0/3131033406 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c0038e0 con 0x7fdb9c0cfcf0 2023-11-28T11:21:46.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.400+0000 7fdba11ff700 1 -- 172.21.15.92:0/3131033406 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb9c0cd800 con 0x7fdb9c0cfcf0 2023-11-28T11:21:46.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.400+0000 7fdba11ff700 1 -- 172.21.15.92:0/3131033406 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb9c0cf2d0 con 0x7fdb9c0cfcf0 2023-11-28T11:21:46.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.401+0000 7fdba11ff700 1 -- 172.21.15.92:0/3131033406 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb9c15cd40 con 0x7fdb9c0cfcf0 2023-11-28T11:21:46.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.402+0000 7fdb937fe700 1 -- 172.21.15.92:0/3131033406 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdb8c00f050 con 0x7fdb9c0cfcf0 2023-11-28T11:21:46.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.403+0000 7fdb937fe700 1 --2- 172.21.15.92:0/3131033406 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fdb7c0682f0 0x7fdb7c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:46.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.403+0000 7fdb937fe700 1 -- 172.21.15.92:0/3131033406 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fdb8c086ed0 con 0x7fdb9c0cfcf0 2023-11-28T11:21:46.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.403+0000 7fdb93fff700 1 --2- 172.21.15.92:0/3131033406 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fdb7c0682f0 0x7fdb7c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:46.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.404+0000 7fdb93fff700 1 --2- 172.21.15.92:0/3131033406 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fdb7c0682f0 0x7fdb7c06a7a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c15fa60 tx=0x7fdb84009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:46.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.408+0000 7fdb937fe700 1 -- 172.21.15.92:0/3131033406 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fdb8c053290 con 0x7fdb9c0cfcf0 2023-11-28T11:21:46.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.597+0000 7fdba11ff700 1 -- 172.21.15.92:0/3131033406 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdb9c067880 con 0x7fdb9c0cfcf0 2023-11-28T11:21:46.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.598+0000 7fdb937fe700 1 -- 172.21.15.92:0/3131033406 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdb8c0150a0 con 0x7fdb9c0cfcf0 2023-11-28T11:21:46.599 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-11-28T11:21:46.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.600+0000 7fdba11ff700 1 -- 172.21.15.92:0/3131033406 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fdb7c0682f0 msgr2=0x7fdb7c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:46.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.600+0000 7fdba11ff700 1 --2- 172.21.15.92:0/3131033406 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fdb7c0682f0 0x7fdb7c06a7a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c15fa60 tx=0x7fdb84009210 comp rx=0 tx=0).stop 2023-11-28T11:21:46.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.600+0000 7fdba11ff700 1 -- 172.21.15.92:0/3131033406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb9c0cfcf0 msgr2=0x7fdb9c157430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:46.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.600+0000 7fdba11ff700 1 --2- 172.21.15.92:0/3131033406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb9c0cfcf0 0x7fdb9c157430 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c009fd0 tx=0x7fdb8c004750 comp rx=0 tx=0).stop 2023-11-28T11:21:46.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.600+0000 7fdba11ff700 1 -- 172.21.15.92:0/3131033406 shutdown_connections 2023-11-28T11:21:46.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.600+0000 7fdba11ff700 1 --2- 172.21.15.92:0/3131033406 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fdb9c15d820 0x7fdb9c0b3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:46.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.600+0000 7fdba11ff700 1 --2- 172.21.15.92:0/3131033406 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fdb7c0682f0 0x7fdb7c06a7a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:46.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.600+0000 7fdba11ff700 1 --2- 172.21.15.92:0/3131033406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb9c0cfcf0 0x7fdb9c157430 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:46.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.600+0000 7fdba11ff700 1 --2- 172.21.15.92:0/3131033406 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fdb9c15a110 0x7fdb9c0b38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:46.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.600+0000 7fdba11ff700 1 -- 172.21.15.92:0/3131033406 >> 172.21.15.92:0/3131033406 conn(0x7fdb9c0719a0 msgr2=0x7fdb9c15ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:46.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.600+0000 7fdba11ff700 1 -- 172.21.15.92:0/3131033406 shutdown_connections 2023-11-28T11:21:46.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.600+0000 7fdba11ff700 1 -- 172.21.15.92:0/3131033406 wait complete. 2023-11-28T11:21:46.619 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:21:46.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.818+0000 7f32a5fff700 1 Processor -- start 2023-11-28T11:21:46.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.819+0000 7f32a5fff700 1 -- start start 2023-11-28T11:21:46.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.820+0000 7f32a5fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32a00b8510 0x7f32a006b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:46.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.820+0000 7f32a5fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f32a00d0d60 0x7f32a00d11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:46.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.820+0000 7f32a5fff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32a00b88e0 con 0x7f32a00cd460 2023-11-28T11:21:46.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.820+0000 7f32a5fff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32a00cdbd0 con 0x7f32a00d0d60 2023-11-28T11:21:46.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.820+0000 7f32a5fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32a00d02d0 con 0x7f32a00b8510 2023-11-28T11:21:46.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.820+0000 7f329ffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f32a00d0d60 0x7f32a00d11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:46.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.820+0000 7f3297fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f32a00cd460 0x7f32a006beb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:58084/0 (socket says 172.21.15.92:58084) 2023-11-28T11:21:46.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.820+0000 7f3297fff700 1 -- 172.21.15.92:0/34953773 learned_addr learned my addr 172.21.15.92:0/34953773 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:46.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.820+0000 7f329f7fe700 1 --2- 172.21.15.92:0/34953773 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32a00b8510 0x7f32a006b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:46.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.820+0000 7f329ffff700 1 -- 172.21.15.92:0/34953773 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32a00b8510 msgr2=0x7f32a006b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:46.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.820+0000 7f329ffff700 1 --2- 172.21.15.92:0/34953773 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32a00b8510 0x7f32a006b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:46.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.820+0000 7f329ffff700 1 -- 172.21.15.92:0/34953773 >> v1:172.21.15.92:6789/0 conn(0x7f32a00cd460 legacy=0x7f32a006beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:46.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.820+0000 7f329ffff700 1 -- 172.21.15.92:0/34953773 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a00cd8f0 con 0x7f32a00d0d60 2023-11-28T11:21:46.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.820+0000 7f329effd700 1 -- 172.21.15.92:0/34953773 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3360888451 0 0) 0x7f32a00b88e0 con 0x7f32a00cd460 2023-11-28T11:21:46.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.821+0000 7f329ffff700 1 --2- 172.21.15.92:0/34953773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32a00d0d60 0x7f32a00d11a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3298009dc0 tx=0x7f329800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a2e43f47e268127 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:46.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.821+0000 7f329effd700 1 -- 172.21.15.92:0/34953773 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329800e070 con 0x7f32a00d0d60 2023-11-28T11:21:46.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.821+0000 7f329effd700 1 -- 172.21.15.92:0/34953773 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f329800bb90 con 0x7f32a00d0d60 2023-11-28T11:21:46.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.821+0000 7f329effd700 1 -- 172.21.15.92:0/34953773 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3298012430 con 0x7f32a00d0d60 2023-11-28T11:21:46.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.821+0000 7f32a5fff700 1 -- 172.21.15.92:0/34953773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32a00d0d60 msgr2=0x7f32a00d11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:46.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.821+0000 7f32a5fff700 1 --2- 172.21.15.92:0/34953773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32a00d0d60 0x7f32a00d11a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3298009dc0 tx=0x7f329800b330 comp rx=0 tx=0).stop 2023-11-28T11:21:46.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.822+0000 7f32a5fff700 1 -- 172.21.15.92:0/34953773 shutdown_connections 2023-11-28T11:21:46.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.822+0000 7f32a5fff700 1 --2- 172.21.15.92:0/34953773 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32a00b8510 0x7f32a006b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:46.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.822+0000 7f32a5fff700 1 --2- 172.21.15.92:0/34953773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32a00d0d60 0x7f32a00d11a0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:46.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.822+0000 7f32a5fff700 1 -- 172.21.15.92:0/34953773 >> 172.21.15.92:0/34953773 conn(0x7f32a00719a0 msgr2=0x7f32a0071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:46.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.822+0000 7f32a5fff700 1 -- 172.21.15.92:0/34953773 shutdown_connections 2023-11-28T11:21:46.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.822+0000 7f32a5fff700 1 -- 172.21.15.92:0/34953773 wait complete. 2023-11-28T11:21:46.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.823+0000 7f32a5fff700 1 Processor -- start 2023-11-28T11:21:46.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.823+0000 7f32a5fff700 1 -- start start 2023-11-28T11:21:46.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.824+0000 7f32a5fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32a00b8510 0x7f32a0157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:46.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.824+0000 7f32a5fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32a00cd460 0x7f32a01e3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:46.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.824+0000 7f32a5fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32a00d0d60 0x7f32a01e41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:46.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.824+0000 7f32a5fff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a00cb5e0 con 0x7f32a00cd460 2023-11-28T11:21:46.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.824+0000 7f32a5fff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a00cb8c0 con 0x7f32a00d0d60 2023-11-28T11:21:46.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.824+0000 7f32a5fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32a00cbba0 con 0x7f32a00b8510 2023-11-28T11:21:46.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.825+0000 7f3297fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32a00cd460 0x7f32a01e3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:46.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.825+0000 7f3297fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32a00cd460 0x7f32a01e3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51486/0 (socket says 172.21.15.92:51486) 2023-11-28T11:21:46.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.825+0000 7f3297fff700 1 -- 172.21.15.92:0/807378953 learned_addr learned my addr 172.21.15.92:0/807378953 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:46.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.825+0000 7f329ffff700 1 --2- 172.21.15.92:0/807378953 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32a00d0d60 0x7f32a01e41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:46.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.825+0000 7f329f7fe700 1 --2- 172.21.15.92:0/807378953 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32a00b8510 0x7f32a0157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:46.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.825+0000 7f3297fff700 1 -- 172.21.15.92:0/807378953 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32a00b8510 msgr2=0x7f32a0157f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:46.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.825+0000 7f3297fff700 1 --2- 172.21.15.92:0/807378953 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32a00b8510 0x7f32a0157f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:46.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.825+0000 7f3297fff700 1 -- 172.21.15.92:0/807378953 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32a00d0d60 msgr2=0x7f32a01e41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:46.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.825+0000 7f3297fff700 1 --2- 172.21.15.92:0/807378953 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32a00d0d60 0x7f32a01e41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:46.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.825+0000 7f3297fff700 1 -- 172.21.15.92:0/807378953 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3298009a70 con 0x7f32a00cd460 2023-11-28T11:21:46.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.825+0000 7f3297fff700 1 --2- 172.21.15.92:0/807378953 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32a00cd460 0x7f32a01e3c80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f32880011e0 tx=0x7f328800ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:46.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.825+0000 7f329cff9700 1 -- 172.21.15.92:0/807378953 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3288004c80 con 0x7f32a00cd460 2023-11-28T11:21:46.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.826+0000 7f329cff9700 1 -- 172.21.15.92:0/807378953 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3288004e00 con 0x7f32a00cd460 2023-11-28T11:21:46.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.826+0000 7f32a5fff700 1 -- 172.21.15.92:0/807378953 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32a00cfff0 con 0x7f32a00cd460 2023-11-28T11:21:46.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.826+0000 7f329cff9700 1 -- 172.21.15.92:0/807378953 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3288007310 con 0x7f32a00cd460 2023-11-28T11:21:46.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.826+0000 7f32a5fff700 1 -- 172.21.15.92:0/807378953 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32a00cfd10 con 0x7f32a00cd460 2023-11-28T11:21:46.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.827+0000 7f32a5fff700 1 -- 172.21.15.92:0/807378953 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32a0067880 con 0x7f32a00cd460 2023-11-28T11:21:46.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.827+0000 7f329cff9700 1 -- 172.21.15.92:0/807378953 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f328801a020 con 0x7f32a00cd460 2023-11-28T11:21:46.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.829+0000 7f329cff9700 1 --2- 172.21.15.92:0/807378953 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f32800682f0 0x7f328006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:46.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.829+0000 7f329cff9700 1 -- 172.21.15.92:0/807378953 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f3288085b50 con 0x7f32a00cd460 2023-11-28T11:21:46.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.829+0000 7f329f7fe700 1 --2- 172.21.15.92:0/807378953 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f32800682f0 0x7f328006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:46.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.830+0000 7f329f7fe700 1 --2- 172.21.15.92:0/807378953 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f32800682f0 0x7f328006a7a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3290006b20 tx=0x7f3290008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:46.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:46.833+0000 7f329cff9700 1 -- 172.21.15.92:0/807378953 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3288051f90 con 0x7f32a00cd460 2023-11-28T11:21:47.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.024+0000 7f32a5fff700 1 -- 172.21.15.92:0/807378953 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f32a00d2940 con 0x7f32a00cd460 2023-11-28T11:21:47.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.024+0000 7f329cff9700 1 -- 172.21.15.92:0/807378953 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7f32880555b0 con 0x7f32a00cd460 2023-11-28T11:21:47.025 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-11-28T11:21:47.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.026+0000 7f32a5fff700 1 -- 172.21.15.92:0/807378953 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f32800682f0 msgr2=0x7f328006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:47.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.026+0000 7f32a5fff700 1 --2- 172.21.15.92:0/807378953 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f32800682f0 0x7f328006a7a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3290006b20 tx=0x7f3290008040 comp rx=0 tx=0).stop 2023-11-28T11:21:47.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.026+0000 7f32a5fff700 1 -- 172.21.15.92:0/807378953 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32a00cd460 msgr2=0x7f32a01e3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:47.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.026+0000 7f32a5fff700 1 --2- 172.21.15.92:0/807378953 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32a00cd460 0x7f32a01e3c80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f32880011e0 tx=0x7f328800ca30 comp rx=0 tx=0).stop 2023-11-28T11:21:47.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.026+0000 7f32a5fff700 1 -- 172.21.15.92:0/807378953 shutdown_connections 2023-11-28T11:21:47.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.026+0000 7f32a5fff700 1 --2- 172.21.15.92:0/807378953 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f32a00d0d60 0x7f32a01e41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:47.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.026+0000 7f32a5fff700 1 --2- 172.21.15.92:0/807378953 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f32a00b8510 0x7f32a0157f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:47.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.026+0000 7f32a5fff700 1 --2- 172.21.15.92:0/807378953 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f32800682f0 0x7f328006a7a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:47.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.026+0000 7f32a5fff700 1 --2- 172.21.15.92:0/807378953 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32a00cd460 0x7f32a01e3c80 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:47.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.026+0000 7f32a5fff700 1 -- 172.21.15.92:0/807378953 >> 172.21.15.92:0/807378953 conn(0x7f32a00719a0 msgr2=0x7f32a00b27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:47.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.027+0000 7f32a5fff700 1 -- 172.21.15.92:0/807378953 shutdown_connections 2023-11-28T11:21:47.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.027+0000 7f32a5fff700 1 -- 172.21.15.92:0/807378953 wait complete. 2023-11-28T11:21:47.046 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:21:47.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.243+0000 7fb17a9ff700 1 Processor -- start 2023-11-28T11:21:47.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.243+0000 7fb17a9ff700 1 -- start start 2023-11-28T11:21:47.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.244+0000 7fb17a9ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb1740b8510 0x7fb17406b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:47.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.244+0000 7fb17a9ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1740cd460 0x7fb17406beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:47.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.244+0000 7fb17a9ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1740b88e0 con 0x7fb17406c3f0 2023-11-28T11:21:47.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.244+0000 7fb17a9ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1740cdbd0 con 0x7fb1740b8510 2023-11-28T11:21:47.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.244+0000 7fb17a9ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1740d02d0 con 0x7fb1740cd460 2023-11-28T11:21:47.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.245+0000 7fb173fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb1740b8510 0x7fb17406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:47.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.245+0000 7fb173fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb1740b8510 0x7fb17406b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51494/0 (socket says 172.21.15.92:51494) 2023-11-28T11:21:47.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.245+0000 7fb173fff700 1 -- 172.21.15.92:0/230429847 learned_addr learned my addr 172.21.15.92:0/230429847 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:47.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.245+0000 7fb173fff700 1 -- 172.21.15.92:0/230429847 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1740cd460 msgr2=0x7fb17406beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:47.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.245+0000 7fb1737fe700 1 --2- 172.21.15.92:0/230429847 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1740cd460 0x7fb17406beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:47.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.245+0000 7fb173fff700 1 --2- 172.21.15.92:0/230429847 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1740cd460 0x7fb17406beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:47.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.245+0000 7fb173fff700 1 -- 172.21.15.92:0/230429847 >> v1:172.21.15.169:6789/0 conn(0x7fb17406c3f0 legacy=0x7fb17415cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:47.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.245+0000 7fb173fff700 1 -- 172.21.15.92:0/230429847 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1740cd8f0 con 0x7fb1740b8510 2023-11-28T11:21:47.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.245+0000 7fb173fff700 1 --2- 172.21.15.92:0/230429847 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1740b8510 0x7fb17406b970 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb164009920 tx=0x7fb164009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69d85f2f51424566 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:47.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.246+0000 7fb172ffd700 1 -- 172.21.15.92:0/230429847 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb164004230 con 0x7fb1740b8510 2023-11-28T11:21:47.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.246+0000 7fb172ffd700 1 -- 172.21.15.92:0/230429847 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb164004390 con 0x7fb1740b8510 2023-11-28T11:21:47.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.246+0000 7fb172ffd700 1 -- 172.21.15.92:0/230429847 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb164009e70 con 0x7fb1740b8510 2023-11-28T11:21:47.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.246+0000 7fb17a9ff700 1 -- 172.21.15.92:0/230429847 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1740b8510 msgr2=0x7fb17406b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:47.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.246+0000 7fb17a9ff700 1 --2- 172.21.15.92:0/230429847 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1740b8510 0x7fb17406b970 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb164009920 tx=0x7fb164009c30 comp rx=0 tx=0).stop 2023-11-28T11:21:47.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.246+0000 7fb17a9ff700 1 -- 172.21.15.92:0/230429847 shutdown_connections 2023-11-28T11:21:47.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.246+0000 7fb17a9ff700 1 --2- 172.21.15.92:0/230429847 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1740b8510 0x7fb17406b970 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:47.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.246+0000 7fb17a9ff700 1 --2- 172.21.15.92:0/230429847 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb1740cd460 0x7fb17406beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:47.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.246+0000 7fb17a9ff700 1 -- 172.21.15.92:0/230429847 >> 172.21.15.92:0/230429847 conn(0x7fb1740719a0 msgr2=0x7fb174071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:47.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.246+0000 7fb17a9ff700 1 -- 172.21.15.92:0/230429847 shutdown_connections 2023-11-28T11:21:47.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.246+0000 7fb17a9ff700 1 -- 172.21.15.92:0/230429847 wait complete. 2023-11-28T11:21:47.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.248+0000 7fb17a9ff700 1 Processor -- start 2023-11-28T11:21:47.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.248+0000 7fb17a9ff700 1 -- start start 2023-11-28T11:21:47.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.249+0000 7fb17a9ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb17406c3f0 0x7fb1740d10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:47.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.249+0000 7fb17a9ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1740b8510 0x7fb1740d1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:47.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.249+0000 7fb17a9ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1740cd460 0x7fb174155f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:47.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.249+0000 7fb17a9ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1740cbed0 con 0x7fb1740cd460 2023-11-28T11:21:47.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.249+0000 7fb17a9ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1740cc1b0 con 0x7fb1740b8510 2023-11-28T11:21:47.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.249+0000 7fb17a9ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1740cc490 con 0x7fb17406c3f0 2023-11-28T11:21:47.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.249+0000 7fb178f9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1740cd460 0x7fb174155f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:47.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.249+0000 7fb178f9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1740cd460 0x7fb174155f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51508/0 (socket says 172.21.15.92:51508) 2023-11-28T11:21:47.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.249+0000 7fb178f9c700 1 -- 172.21.15.92:0/4263483707 learned_addr learned my addr 172.21.15.92:0/4263483707 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:47.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.250+0000 7fb1737fe700 1 --2- 172.21.15.92:0/4263483707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1740b8510 0x7fb1740d1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:47.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.250+0000 7fb173fff700 1 --2- 172.21.15.92:0/4263483707 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb17406c3f0 0x7fb1740d10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:47.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.250+0000 7fb178f9c700 1 -- 172.21.15.92:0/4263483707 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb17406c3f0 msgr2=0x7fb1740d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:47.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.250+0000 7fb178f9c700 1 --2- 172.21.15.92:0/4263483707 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb17406c3f0 0x7fb1740d10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:47.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.250+0000 7fb178f9c700 1 -- 172.21.15.92:0/4263483707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1740b8510 msgr2=0x7fb1740d1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:47.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.250+0000 7fb178f9c700 1 --2- 172.21.15.92:0/4263483707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1740b8510 0x7fb1740d1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:47.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.250+0000 7fb178f9c700 1 -- 172.21.15.92:0/4263483707 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb164009550 con 0x7fb1740cd460 2023-11-28T11:21:47.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.250+0000 7fb1737fe700 1 --2- 172.21.15.92:0/4263483707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1740b8510 0x7fb1740d1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:21:47.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.250+0000 7fb178f9c700 1 --2- 172.21.15.92:0/4263483707 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1740cd460 0x7fb174155f00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb16c00cc10 tx=0x7fb16c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:47.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.250+0000 7fb170ff9700 1 -- 172.21.15.92:0/4263483707 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16c00abb0 con 0x7fb1740cd460 2023-11-28T11:21:47.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.250+0000 7fb170ff9700 1 -- 172.21.15.92:0/4263483707 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb16c00ad10 con 0x7fb1740cd460 2023-11-28T11:21:47.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.250+0000 7fb17a9ff700 1 -- 172.21.15.92:0/4263483707 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1740cfff0 con 0x7fb1740cd460 2023-11-28T11:21:47.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.250+0000 7fb170ff9700 1 -- 172.21.15.92:0/4263483707 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16c012660 con 0x7fb1740cd460 2023-11-28T11:21:47.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.251+0000 7fb17a9ff700 1 -- 172.21.15.92:0/4263483707 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1740cfd10 con 0x7fb1740cd460 2023-11-28T11:21:47.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.251+0000 7fb17a9ff700 1 -- 172.21.15.92:0/4263483707 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb174067880 con 0x7fb1740cd460 2023-11-28T11:21:47.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.252+0000 7fb170ff9700 1 -- 172.21.15.92:0/4263483707 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb16c022070 con 0x7fb1740cd460 2023-11-28T11:21:47.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.253+0000 7fb170ff9700 1 --2- 172.21.15.92:0/4263483707 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb1600682f0 0x7fb16006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:47.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.253+0000 7fb170ff9700 1 -- 172.21.15.92:0/4263483707 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fb16c086760 con 0x7fb1740cd460 2023-11-28T11:21:47.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.254+0000 7fb173fff700 1 --2- 172.21.15.92:0/4263483707 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb1600682f0 0x7fb16006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:47.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.254+0000 7fb173fff700 1 --2- 172.21.15.92:0/4263483707 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb1600682f0 0x7fb16006a7a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb164009130 tx=0x7fb164008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:47.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.257+0000 7fb170ff9700 1 -- 172.21.15.92:0/4263483707 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb16c052ba0 con 0x7fb1740cd460 2023-11-28T11:21:47.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.448+0000 7fb17a9ff700 1 -- 172.21.15.92:0/4263483707 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb1740d2940 con 0x7fb1740cd460 2023-11-28T11:21:47.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.448+0000 7fb170ff9700 1 -- 172.21.15.92:0/4263483707 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb16c016020 con 0x7fb1740cd460 2023-11-28T11:21:47.449 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-11-28T11:21:47.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.450+0000 7fb17a9ff700 1 -- 172.21.15.92:0/4263483707 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb1600682f0 msgr2=0x7fb16006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:47.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.450+0000 7fb17a9ff700 1 --2- 172.21.15.92:0/4263483707 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb1600682f0 0x7fb16006a7a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb164009130 tx=0x7fb164008030 comp rx=0 tx=0).stop 2023-11-28T11:21:47.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.450+0000 7fb17a9ff700 1 -- 172.21.15.92:0/4263483707 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1740cd460 msgr2=0x7fb174155f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:47.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.450+0000 7fb17a9ff700 1 --2- 172.21.15.92:0/4263483707 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1740cd460 0x7fb174155f00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb16c00cc10 tx=0x7fb16c00a060 comp rx=0 tx=0).stop 2023-11-28T11:21:47.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.450+0000 7fb17a9ff700 1 -- 172.21.15.92:0/4263483707 shutdown_connections 2023-11-28T11:21:47.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.450+0000 7fb17a9ff700 1 --2- 172.21.15.92:0/4263483707 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb1600682f0 0x7fb16006a7a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:47.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.450+0000 7fb17a9ff700 1 --2- 172.21.15.92:0/4263483707 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1740cd460 0x7fb174155f00 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:47.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.450+0000 7fb17a9ff700 1 --2- 172.21.15.92:0/4263483707 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb17406c3f0 0x7fb1740d10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:47.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.450+0000 7fb17a9ff700 1 --2- 172.21.15.92:0/4263483707 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb1740b8510 0x7fb1740d1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:47.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.450+0000 7fb17a9ff700 1 -- 172.21.15.92:0/4263483707 >> 172.21.15.92:0/4263483707 conn(0x7fb1740719a0 msgr2=0x7fb1740b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:47.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.450+0000 7fb17a9ff700 1 -- 172.21.15.92:0/4263483707 shutdown_connections 2023-11-28T11:21:47.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.451+0000 7fb17a9ff700 1 -- 172.21.15.92:0/4263483707 wait complete. 2023-11-28T11:21:47.469 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:21:47.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.669+0000 7fda48dff700 1 Processor -- start 2023-11-28T11:21:47.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.669+0000 7fda48dff700 1 -- start start 2023-11-28T11:21:47.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.670+0000 7fda48dff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fda440b8510 0x7fda4406b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:47.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.670+0000 7fda48dff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda440b88e0 con 0x7fda440d0d60 2023-11-28T11:21:47.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.670+0000 7fda48dff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda440cd8f0 con 0x7fda440cd460 2023-11-28T11:21:47.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.670+0000 7fda48dff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda440cdbd0 con 0x7fda440b8510 2023-11-28T11:21:47.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.670+0000 7fda42d9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fda440d0d60 0x7fda440d1200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:58086/0 (socket says 172.21.15.92:58086) 2023-11-28T11:21:47.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.670+0000 7fda42d9d700 1 -- 172.21.15.92:0/2809730516 learned_addr learned my addr 172.21.15.92:0/2809730516 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:47.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.671+0000 7fda4259c700 1 --2- 172.21.15.92:0/2809730516 >> v2:172.21.15.169:3300/0 conn(0x7fda440b8510 0x7fda4406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:47.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.671+0000 7fda4159a700 1 -- 172.21.15.92:0/2809730516 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1298991072 0 0) 0x7fda440b88e0 con 0x7fda440d0d60 2023-11-28T11:21:47.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.671+0000 7fda4159a700 1 -- 172.21.15.92:0/2809730516 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda2c0035b0 con 0x7fda440d0d60 2023-11-28T11:21:47.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.671+0000 7fda4159a700 1 -- 172.21.15.92:0/2809730516 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 827512064 0 0) 0x7fda440cd8f0 con 0x7fda440cd460 2023-11-28T11:21:47.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.671+0000 7fda4159a700 1 -- 172.21.15.92:0/2809730516 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda440b88e0 con 0x7fda440cd460 2023-11-28T11:21:47.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.671+0000 7fda4259c700 1 -- 172.21.15.92:0/2809730516 >> v1:172.21.15.169:6789/0 conn(0x7fda440cd460 legacy=0x7fda4406beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:47.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.671+0000 7fda4259c700 1 -- 172.21.15.92:0/2809730516 >> v1:172.21.15.92:6789/0 conn(0x7fda440d0d60 legacy=0x7fda440d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:47.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.671+0000 7fda4259c700 1 -- 172.21.15.92:0/2809730516 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda440cfd10 con 0x7fda440b8510 2023-11-28T11:21:47.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.671+0000 7fda4259c700 1 --2- 172.21.15.92:0/2809730516 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda440b8510 0x7fda4406b970 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fda34009840 tx=0x7fda34009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ed997572359a238 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:47.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.672+0000 7fda4159a700 1 -- 172.21.15.92:0/2809730516 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda34011830 con 0x7fda440b8510 2023-11-28T11:21:47.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.672+0000 7fda4159a700 1 -- 172.21.15.92:0/2809730516 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda34011990 con 0x7fda440b8510 2023-11-28T11:21:47.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.672+0000 7fda4159a700 1 -- 172.21.15.92:0/2809730516 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda34011d00 con 0x7fda440b8510 2023-11-28T11:21:47.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.672+0000 7fda48dff700 1 -- 172.21.15.92:0/2809730516 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda440b8510 msgr2=0x7fda4406b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:47.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.672+0000 7fda48dff700 1 --2- 172.21.15.92:0/2809730516 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda440b8510 0x7fda4406b970 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fda34009840 tx=0x7fda34009b50 comp rx=0 tx=0).stop 2023-11-28T11:21:47.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.672+0000 7fda48dff700 1 -- 172.21.15.92:0/2809730516 shutdown_connections 2023-11-28T11:21:47.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.672+0000 7fda48dff700 1 --2- 172.21.15.92:0/2809730516 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda440b8510 0x7fda4406b970 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:47.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.672+0000 7fda48dff700 1 -- 172.21.15.92:0/2809730516 >> 172.21.15.92:0/2809730516 conn(0x7fda440719a0 msgr2=0x7fda44071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:47.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.672+0000 7fda48dff700 1 -- 172.21.15.92:0/2809730516 shutdown_connections 2023-11-28T11:21:47.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.672+0000 7fda48dff700 1 -- 172.21.15.92:0/2809730516 wait complete. 2023-11-28T11:21:47.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.674+0000 7fda48dff700 1 Processor -- start 2023-11-28T11:21:47.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.674+0000 7fda48dff700 1 -- start start 2023-11-28T11:21:47.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.675+0000 7fda48dff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda440b8510 0x7fda441f5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:47.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.675+0000 7fda48dff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda440cd460 0x7fda440b7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:47.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.675+0000 7fda48dff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda440d0d60 0x7fda44156000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:47.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.675+0000 7fda48dff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda441f6300 con 0x7fda440d0d60 2023-11-28T11:21:47.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.675+0000 7fda48dff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda440cb8e0 con 0x7fda440cd460 2023-11-28T11:21:47.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.675+0000 7fda48dff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda440aa7b0 con 0x7fda440b8510 2023-11-28T11:21:47.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.675+0000 7fda42d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda440d0d60 0x7fda44156000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:47.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.675+0000 7fda42d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda440d0d60 0x7fda44156000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51514/0 (socket says 172.21.15.92:51514) 2023-11-28T11:21:47.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.675+0000 7fda42d9d700 1 -- 172.21.15.92:0/3723117757 learned_addr learned my addr 172.21.15.92:0/3723117757 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:47.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.676+0000 7fda42d9d700 1 -- 172.21.15.92:0/3723117757 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda440b8510 msgr2=0x7fda441f5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:47.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.676+0000 7fda42d9d700 1 --2- 172.21.15.92:0/3723117757 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda440b8510 0x7fda441f5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:47.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.676+0000 7fda42d9d700 1 -- 172.21.15.92:0/3723117757 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda440cd460 msgr2=0x7fda440b7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:47.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.676+0000 7fda42d9d700 1 --2- 172.21.15.92:0/3723117757 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda440cd460 0x7fda440b7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:47.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.676+0000 7fda42d9d700 1 -- 172.21.15.92:0/3723117757 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda340094a0 con 0x7fda440d0d60 2023-11-28T11:21:47.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.676+0000 7fda42d9d700 1 --2- 172.21.15.92:0/3723117757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda440d0d60 0x7fda44156000 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fda3c00cb10 tx=0x7fda3c00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:47.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.676+0000 7fda32ffd700 1 -- 172.21.15.92:0/3723117757 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3c00ab00 con 0x7fda440d0d60 2023-11-28T11:21:47.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.676+0000 7fda32ffd700 1 -- 172.21.15.92:0/3723117757 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda3c00ac60 con 0x7fda440d0d60 2023-11-28T11:21:47.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.676+0000 7fda32ffd700 1 -- 172.21.15.92:0/3723117757 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3c012400 con 0x7fda440d0d60 2023-11-28T11:21:47.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.676+0000 7fda48dff700 1 -- 172.21.15.92:0/3723117757 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda440d29c0 con 0x7fda440d0d60 2023-11-28T11:21:47.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.676+0000 7fda48dff700 1 -- 172.21.15.92:0/3723117757 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda440cfff0 con 0x7fda440d0d60 2023-11-28T11:21:47.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.677+0000 7fda48dff700 1 -- 172.21.15.92:0/3723117757 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda440d2b50 con 0x7fda440d0d60 2023-11-28T11:21:47.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.678+0000 7fda32ffd700 1 -- 172.21.15.92:0/3723117757 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fda3c019030 con 0x7fda440d0d60 2023-11-28T11:21:47.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.679+0000 7fda32ffd700 1 --2- 172.21.15.92:0/3723117757 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fda240682f0 0x7fda2406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:47.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.679+0000 7fda32ffd700 1 -- 172.21.15.92:0/3723117757 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fda3c085b60 con 0x7fda440d0d60 2023-11-28T11:21:47.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.679+0000 7fda4259c700 1 --2- 172.21.15.92:0/3723117757 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fda240682f0 0x7fda2406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:47.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.680+0000 7fda4259c700 1 --2- 172.21.15.92:0/3723117757 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fda240682f0 0x7fda2406a7a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fda34000c00 tx=0x7fda34010000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:47.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.683+0000 7fda32ffd700 1 -- 172.21.15.92:0/3723117757 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fda3c04b5e0 con 0x7fda440d0d60 2023-11-28T11:21:47.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.875+0000 7fda48dff700 1 -- 172.21.15.92:0/3723117757 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fda440cba50 con 0x7fda440d0d60 2023-11-28T11:21:47.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.875+0000 7fda32ffd700 1 -- 172.21.15.92:0/3723117757 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7fda3c052050 con 0x7fda440d0d60 2023-11-28T11:21:47.877 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-11-28T11:21:47.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.878+0000 7fda48dff700 1 -- 172.21.15.92:0/3723117757 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fda240682f0 msgr2=0x7fda2406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:47.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.878+0000 7fda48dff700 1 --2- 172.21.15.92:0/3723117757 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fda240682f0 0x7fda2406a7a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fda34000c00 tx=0x7fda34010000 comp rx=0 tx=0).stop 2023-11-28T11:21:47.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.878+0000 7fda48dff700 1 -- 172.21.15.92:0/3723117757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda440d0d60 msgr2=0x7fda44156000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:47.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.878+0000 7fda48dff700 1 --2- 172.21.15.92:0/3723117757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda440d0d60 0x7fda44156000 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fda3c00cb10 tx=0x7fda3c00ced0 comp rx=0 tx=0).stop 2023-11-28T11:21:47.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.878+0000 7fda48dff700 1 -- 172.21.15.92:0/3723117757 shutdown_connections 2023-11-28T11:21:47.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.878+0000 7fda48dff700 1 --2- 172.21.15.92:0/3723117757 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fda240682f0 0x7fda2406a7a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:47.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.878+0000 7fda48dff700 1 --2- 172.21.15.92:0/3723117757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda440d0d60 0x7fda44156000 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:47.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.878+0000 7fda48dff700 1 --2- 172.21.15.92:0/3723117757 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fda440b8510 0x7fda441f5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:47.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.878+0000 7fda48dff700 1 --2- 172.21.15.92:0/3723117757 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fda440cd460 0x7fda440b7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:47.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.878+0000 7fda48dff700 1 -- 172.21.15.92:0/3723117757 >> 172.21.15.92:0/3723117757 conn(0x7fda440719a0 msgr2=0x7fda440b5d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:47.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.878+0000 7fda48dff700 1 -- 172.21.15.92:0/3723117757 shutdown_connections 2023-11-28T11:21:47.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:47.878+0000 7fda48dff700 1 -- 172.21.15.92:0/3723117757 wait complete. 2023-11-28T11:21:47.897 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T11:21:47.945 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:21:47.950 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:21:48.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.100+0000 7f68fc7ff700 1 Processor -- start 2023-11-28T11:21:48.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.100+0000 7f68fc7ff700 1 -- start start 2023-11-28T11:21:48.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.101+0000 7f68fc7ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68f40cd460 0x7f68f406c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:48.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.101+0000 7f68fc7ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f68f40d0d60 0x7f68f40d11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:48.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.101+0000 7f68fc7ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68f40b88e0 con 0x7f68f40b8510 2023-11-28T11:21:48.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.101+0000 7f68fc7ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68f40cdbd0 con 0x7f68f40d0d60 2023-11-28T11:21:48.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.101+0000 7f68fc7ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68f40d02d0 con 0x7f68f40cd460 2023-11-28T11:21:48.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.101+0000 7f68fa59b700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f68f40b8510 0x7f68f406b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.92:52884/0 (socket says 172.21.15.92:52884) 2023-11-28T11:21:48.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.101+0000 7f68fa59b700 1 -- 172.21.15.92:0/3234311302 learned_addr learned my addr 172.21.15.92:0/3234311302 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:48.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.101+0000 7f68f9d9a700 1 --2- 172.21.15.92:0/3234311302 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68f40cd460 0x7f68f406c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:48.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.102+0000 7f68fad9c700 1 --2- 172.21.15.92:0/3234311302 >> v2:172.21.15.169:3300/0 conn(0x7f68f40d0d60 0x7f68f40d11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:48.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.102+0000 7f68f9599700 1 -- 172.21.15.92:0/3234311302 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1533451096 0 0) 0x7f68f40b88e0 con 0x7f68f40b8510 2023-11-28T11:21:48.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.102+0000 7f68f9599700 1 -- 172.21.15.92:0/3234311302 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68e00035b0 con 0x7f68f40b8510 2023-11-28T11:21:48.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.102+0000 7f68f9d9a700 1 -- 172.21.15.92:0/3234311302 >> v2:172.21.15.169:3300/0 conn(0x7f68f40d0d60 msgr2=0x7f68f40d11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:48.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.102+0000 7f68f9d9a700 1 --2- 172.21.15.92:0/3234311302 >> v2:172.21.15.169:3300/0 conn(0x7f68f40d0d60 0x7f68f40d11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:48.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.102+0000 7f68f9d9a700 1 -- 172.21.15.92:0/3234311302 >> v1:172.21.15.169:6789/0 conn(0x7f68f40b8510 legacy=0x7f68f406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:48.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.102+0000 7f68f9d9a700 1 -- 172.21.15.92:0/3234311302 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f40cd8f0 con 0x7f68f40cd460 2023-11-28T11:21:48.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.102+0000 7f68f9d9a700 1 --2- 172.21.15.92:0/3234311302 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68f40cd460 0x7f68f406c080 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f68e4009840 tx=0x7f68e4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86a972ffa2d8e29 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:48.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.102+0000 7f68f9599700 1 -- 172.21.15.92:0/3234311302 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e4011830 con 0x7f68f40cd460 2023-11-28T11:21:48.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.103+0000 7f68f9599700 1 -- 172.21.15.92:0/3234311302 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68e4011990 con 0x7f68f40cd460 2023-11-28T11:21:48.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.103+0000 7f68f9599700 1 -- 172.21.15.92:0/3234311302 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e4011d20 con 0x7f68f40cd460 2023-11-28T11:21:48.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.103+0000 7f68fc7ff700 1 -- 172.21.15.92:0/3234311302 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68f40cd460 msgr2=0x7f68f406c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:48.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.103+0000 7f68fc7ff700 1 --2- 172.21.15.92:0/3234311302 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68f40cd460 0x7f68f406c080 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f68e4009840 tx=0x7f68e4009b50 comp rx=0 tx=0).stop 2023-11-28T11:21:48.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.103+0000 7f68fc7ff700 1 -- 172.21.15.92:0/3234311302 shutdown_connections 2023-11-28T11:21:48.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.103+0000 7f68fc7ff700 1 --2- 172.21.15.92:0/3234311302 >> v2:172.21.15.169:3300/0 conn(0x7f68f40d0d60 0x7f68f40d11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:48.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.103+0000 7f68fc7ff700 1 --2- 172.21.15.92:0/3234311302 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68f40cd460 0x7f68f406c080 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:48.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.103+0000 7f68fc7ff700 1 -- 172.21.15.92:0/3234311302 >> 172.21.15.92:0/3234311302 conn(0x7f68f40719a0 msgr2=0x7f68f4071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:48.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.103+0000 7f68fc7ff700 1 -- 172.21.15.92:0/3234311302 shutdown_connections 2023-11-28T11:21:48.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.103+0000 7f68fc7ff700 1 -- 172.21.15.92:0/3234311302 wait complete. 2023-11-28T11:21:48.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.105+0000 7f68fc7ff700 1 Processor -- start 2023-11-28T11:21:48.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.105+0000 7f68fc7ff700 1 -- start start 2023-11-28T11:21:48.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.106+0000 7f68fc7ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68f40b8510 0x7f68f4157d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:48.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.106+0000 7f68fc7ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68f40cd460 0x7f68f4159370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:48.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.106+0000 7f68fc7ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68f40d0d60 0x7f68f41f4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:48.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.106+0000 7f68fc7ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f41598b0 con 0x7f68f40d0d60 2023-11-28T11:21:48.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.106+0000 7f68fc7ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f41e3dd0 con 0x7f68f40cd460 2023-11-28T11:21:48.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.106+0000 7f68fc7ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68f4158cf0 con 0x7f68f40b8510 2023-11-28T11:21:48.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.106+0000 7f68fad9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68f40d0d60 0x7f68f41f4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:48.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.106+0000 7f68fa59b700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68f40b8510 0x7f68f4157d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:48.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.106+0000 7f68fad9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68f40d0d60 0x7f68f41f4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51528/0 (socket says 172.21.15.92:51528) 2023-11-28T11:21:48.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.106+0000 7f68fad9c700 1 -- 172.21.15.92:0/343450184 learned_addr learned my addr 172.21.15.92:0/343450184 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:48.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.106+0000 7f68f9d9a700 1 --2- 172.21.15.92:0/343450184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68f40cd460 0x7f68f4159370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:48.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.107+0000 7f68fad9c700 1 -- 172.21.15.92:0/343450184 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68f40b8510 msgr2=0x7f68f4157d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:48.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.107+0000 7f68fad9c700 1 --2- 172.21.15.92:0/343450184 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68f40b8510 0x7f68f4157d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:48.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.107+0000 7f68fad9c700 1 -- 172.21.15.92:0/343450184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68f40cd460 msgr2=0x7f68f4159370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:48.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.107+0000 7f68fad9c700 1 --2- 172.21.15.92:0/343450184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68f40cd460 0x7f68f4159370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:48.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.107+0000 7f68fad9c700 1 -- 172.21.15.92:0/343450184 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e40094a0 con 0x7f68f40d0d60 2023-11-28T11:21:48.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.107+0000 7f68fad9c700 1 --2- 172.21.15.92:0/343450184 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68f40d0d60 0x7f68f41f4f50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f68ec00efc0 tx=0x7f68ec00c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:48.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.107+0000 7f68eaffd700 1 -- 172.21.15.92:0/343450184 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ec00cad0 con 0x7f68f40d0d60 2023-11-28T11:21:48.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.107+0000 7f68eaffd700 1 -- 172.21.15.92:0/343450184 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68ec00f040 con 0x7f68f40d0d60 2023-11-28T11:21:48.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.107+0000 7f68eaffd700 1 -- 172.21.15.92:0/343450184 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ec00ce70 con 0x7f68f40d0d60 2023-11-28T11:21:48.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.107+0000 7f68fc7ff700 1 -- 172.21.15.92:0/343450184 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68f40cfff0 con 0x7f68f40d0d60 2023-11-28T11:21:48.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.107+0000 7f68fc7ff700 1 -- 172.21.15.92:0/343450184 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68f40cfd10 con 0x7f68f40d0d60 2023-11-28T11:21:48.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.108+0000 7f68fc7ff700 1 -- 172.21.15.92:0/343450184 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68f41e3f60 con 0x7f68f40d0d60 2023-11-28T11:21:48.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.109+0000 7f68eaffd700 1 -- 172.21.15.92:0/343450184 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f68ec014480 con 0x7f68f40d0d60 2023-11-28T11:21:48.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.110+0000 7f68eaffd700 1 --2- 172.21.15.92:0/343450184 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f68d8068320 0x7f68d806a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:48.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.110+0000 7f68eaffd700 1 -- 172.21.15.92:0/343450184 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f68ec086930 con 0x7f68f40d0d60 2023-11-28T11:21:48.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.110+0000 7f68fa59b700 1 --2- 172.21.15.92:0/343450184 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f68d8068320 0x7f68d806a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:48.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.111+0000 7f68fa59b700 1 --2- 172.21.15.92:0/343450184 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f68d8068320 0x7f68d806a7d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f68f0007fd0 tx=0x7f68f0007dc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:48.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.114+0000 7f68eaffd700 1 -- 172.21.15.92:0/343450184 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f68ec052d10 con 0x7f68f40d0d60 2023-11-28T11:21:48.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.307+0000 7f68fc7ff700 1 -- 172.21.15.92:0/343450184 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f68f4067880 con 0x7f68f40d0d60 2023-11-28T11:21:48.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.308+0000 7f68eaffd700 1 -- 172.21.15.92:0/343450184 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7f68ec056330 con 0x7f68f40d0d60 2023-11-28T11:21:48.309 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-11-28T11:21:48.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.310+0000 7f68fc7ff700 1 -- 172.21.15.92:0/343450184 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f68d8068320 msgr2=0x7f68d806a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:48.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.310+0000 7f68fc7ff700 1 --2- 172.21.15.92:0/343450184 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f68d8068320 0x7f68d806a7d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f68f0007fd0 tx=0x7f68f0007dc0 comp rx=0 tx=0).stop 2023-11-28T11:21:48.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.310+0000 7f68fc7ff700 1 -- 172.21.15.92:0/343450184 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68f40d0d60 msgr2=0x7f68f41f4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:48.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.310+0000 7f68fc7ff700 1 --2- 172.21.15.92:0/343450184 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68f40d0d60 0x7f68f41f4f50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f68ec00efc0 tx=0x7f68ec00c0c0 comp rx=0 tx=0).stop 2023-11-28T11:21:48.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.310+0000 7f68fc7ff700 1 -- 172.21.15.92:0/343450184 shutdown_connections 2023-11-28T11:21:48.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.310+0000 7f68fc7ff700 1 --2- 172.21.15.92:0/343450184 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f68d8068320 0x7f68d806a7d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:48.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.311+0000 7f68fc7ff700 1 --2- 172.21.15.92:0/343450184 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68f40d0d60 0x7f68f41f4f50 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:48.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.311+0000 7f68fc7ff700 1 --2- 172.21.15.92:0/343450184 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f68f40b8510 0x7f68f4157d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:48.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.311+0000 7f68fc7ff700 1 --2- 172.21.15.92:0/343450184 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f68f40cd460 0x7f68f4159370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:48.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.311+0000 7f68fc7ff700 1 -- 172.21.15.92:0/343450184 >> 172.21.15.92:0/343450184 conn(0x7f68f40719a0 msgr2=0x7f68f41e7910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:48.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.311+0000 7f68fc7ff700 1 -- 172.21.15.92:0/343450184 shutdown_connections 2023-11-28T11:21:48.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.311+0000 7f68fc7ff700 1 -- 172.21.15.92:0/343450184 wait complete. 2023-11-28T11:21:48.330 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:21:48.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.567+0000 7fcbc75ff700 1 Processor -- start 2023-11-28T11:21:48.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.567+0000 7fcbc75ff700 1 -- start start 2023-11-28T11:21:48.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.568+0000 7fcbc75ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcbc00b8510 0x7fcbc006b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:48.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.568+0000 7fcbc75ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbc00d0d60 0x7fcbc00d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:48.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.568+0000 7fcbc75ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbc00b88e0 con 0x7fcbc00cd460 2023-11-28T11:21:48.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.568+0000 7fcbc75ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbc00cdbd0 con 0x7fcbc00b8510 2023-11-28T11:21:48.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.568+0000 7fcbc75ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc00d02d0 con 0x7fcbc00d0d60 2023-11-28T11:21:48.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.569+0000 7fcbc4b9a700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fcbc00cd460 0x7fcbc006beb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:58094/0 (socket says 172.21.15.92:58094) 2023-11-28T11:21:48.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.569+0000 7fcbc4b9a700 1 -- 172.21.15.92:0/2642480524 learned_addr learned my addr 172.21.15.92:0/2642480524 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:48.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.569+0000 7fcbc539b700 1 --2- 172.21.15.92:0/2642480524 >> v2:172.21.15.169:3300/0 conn(0x7fcbc00b8510 0x7fcbc006b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:48.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.569+0000 7fcbc5b9c700 1 --2- 172.21.15.92:0/2642480524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbc00d0d60 0x7fcbc00d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:48.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.569+0000 7fcbb7fff700 1 -- 172.21.15.92:0/2642480524 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3145568183 0 0) 0x7fcbc00b88e0 con 0x7fcbc00cd460 2023-11-28T11:21:48.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.569+0000 7fcbb7fff700 1 -- 172.21.15.92:0/2642480524 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbac0035b0 con 0x7fcbc00cd460 2023-11-28T11:21:48.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.569+0000 7fcbb7fff700 1 -- 172.21.15.92:0/2642480524 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1509652744 0 0) 0x7fcbac0035b0 con 0x7fcbc00cd460 2023-11-28T11:21:48.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.569+0000 7fcbb7fff700 1 -- 172.21.15.92:0/2642480524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbc00d0d60 msgr2=0x7fcbc00d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:48.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.569+0000 7fcbb7fff700 1 --2- 172.21.15.92:0/2642480524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbc00d0d60 0x7fcbc00d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:48.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.569+0000 7fcbb7fff700 1 -- 172.21.15.92:0/2642480524 >> v2:172.21.15.169:3300/0 conn(0x7fcbc00b8510 msgr2=0x7fcbc006b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:48.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.569+0000 7fcbb7fff700 1 --2- 172.21.15.92:0/2642480524 >> v2:172.21.15.169:3300/0 conn(0x7fcbc00b8510 0x7fcbc006b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:48.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.569+0000 7fcbb7fff700 1 -- 172.21.15.92:0/2642480524 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc00cd8f0 con 0x7fcbc00cd460 2023-11-28T11:21:48.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.569+0000 7fcbb7fff700 1 -- 172.21.15.92:0/2642480524 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532161587 0 0) 0x7fcbb0001670 con 0x7fcbc00cd460 2023-11-28T11:21:48.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.570+0000 7fcbb7fff700 1 -- 172.21.15.92:0/2642480524 >> v1:172.21.15.92:6789/0 conn(0x7fcbc00cd460 legacy=0x7fcbc006beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:48.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.570+0000 7fcbb7fff700 1 --2- 172.21.15.92:0/2642480524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcbac003a30 0x7fcbac003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:48.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.570+0000 7fcbb7fff700 1 --2- 172.21.15.92:0/2642480524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbac004850 0x7fcbac004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:48.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.570+0000 7fcbb7fff700 1 --2- 172.21.15.92:0/2642480524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbc00d0d60 0x7fcbac005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:48.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.570+0000 7fcbb7fff700 1 -- 172.21.15.92:0/2642480524 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc00cdbd0 con 0x7fcbac003a30 2023-11-28T11:21:48.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.570+0000 7fcbb7fff700 1 -- 172.21.15.92:0/2642480524 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc00d02d0 con 0x7fcbac004850 2023-11-28T11:21:48.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.570+0000 7fcbb7fff700 1 -- 172.21.15.92:0/2642480524 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbac00ba10 con 0x7fcbc00d0d60 2023-11-28T11:21:48.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.570+0000 7fcbc539b700 1 --2- 172.21.15.92:0/2642480524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcbac003a30 0x7fcbac003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:48.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.570+0000 7fcbc4b9a700 1 --2- 172.21.15.92:0/2642480524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbac004850 0x7fcbac004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:48.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.570+0000 7fcbc5b9c700 1 --2- 172.21.15.92:0/2642480524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbc00d0d60 0x7fcbac005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:48.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.570+0000 7fcbc539b700 1 -- 172.21.15.92:0/2642480524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbc00d0d60 msgr2=0x7fcbac005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:48.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.570+0000 7fcbc539b700 1 --2- 172.21.15.92:0/2642480524 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbc00d0d60 0x7fcbac005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:48.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.570+0000 7fcbc539b700 1 -- 172.21.15.92:0/2642480524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbac004850 msgr2=0x7fcbac004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:48.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.570+0000 7fcbc539b700 1 --2- 172.21.15.92:0/2642480524 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbac004850 0x7fcbac004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:48.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.570+0000 7fcbc539b700 1 -- 172.21.15.92:0/2642480524 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbac0035b0 con 0x7fcbac003a30 2023-11-28T11:21:48.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.571+0000 7fcbc539b700 1 --2- 172.21.15.92:0/2642480524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcbac003a30 0x7fcbac003e40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc009b60 tx=0x7fcbac005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e96d6fd879e1dfa8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:48.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.571+0000 7fcbb7fff700 1 -- 172.21.15.92:0/2642480524 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc00e6c0 con 0x7fcbac003a30 2023-11-28T11:21:48.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.571+0000 7fcbb7fff700 1 -- 172.21.15.92:0/2642480524 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbbc00e820 con 0x7fcbac003a30 2023-11-28T11:21:48.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.571+0000 7fcbb7fff700 1 -- 172.21.15.92:0/2642480524 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc00eb90 con 0x7fcbac003a30 2023-11-28T11:21:48.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.571+0000 7fcbc75ff700 1 -- 172.21.15.92:0/2642480524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcbac003a30 msgr2=0x7fcbac003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:48.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.571+0000 7fcbc75ff700 1 --2- 172.21.15.92:0/2642480524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcbac003a30 0x7fcbac003e40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc009b60 tx=0x7fcbac005880 comp rx=0 tx=0).stop 2023-11-28T11:21:48.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.571+0000 7fcbc5b9c700 1 -- 172.21.15.92:0/2642480524 reap_dead start 2023-11-28T11:21:48.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.571+0000 7fcbc75ff700 1 -- 172.21.15.92:0/2642480524 shutdown_connections 2023-11-28T11:21:48.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.571+0000 7fcbc75ff700 1 -- 172.21.15.92:0/2642480524 >> 172.21.15.92:0/2642480524 conn(0x7fcbc00719a0 msgr2=0x7fcbc0071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:48.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.571+0000 7fcbc75ff700 1 -- 172.21.15.92:0/2642480524 shutdown_connections 2023-11-28T11:21:48.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.572+0000 7fcbc75ff700 1 -- 172.21.15.92:0/2642480524 wait complete. 2023-11-28T11:21:48.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.573+0000 7fcbc75ff700 1 Processor -- start 2023-11-28T11:21:48.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.573+0000 7fcbc75ff700 1 -- start start 2023-11-28T11:21:48.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.573+0000 7fcbc75ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcbac003a30 0x7fcbc01e5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:48.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.573+0000 7fcbc75ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbc0154200 0x7fcbc01e4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:48.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.573+0000 7fcbc75ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbc0154670 0x7fcbc00cbac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:48.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.573+0000 7fcbc75ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc00c6940 con 0x7fcbac003a30 2023-11-28T11:21:48.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.573+0000 7fcbc75ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc00c6b20 con 0x7fcbc0154670 2023-11-28T11:21:48.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.573+0000 7fcbc75ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc00c6e00 con 0x7fcbc0154200 2023-11-28T11:21:48.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.574+0000 7fcbc539b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcbac003a30 0x7fcbc01e5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:48.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.574+0000 7fcbc539b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcbac003a30 0x7fcbc01e5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51534/0 (socket says 172.21.15.92:51534) 2023-11-28T11:21:48.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.574+0000 7fcbc539b700 1 -- 172.21.15.92:0/4203851004 learned_addr learned my addr 172.21.15.92:0/4203851004 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:48.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.574+0000 7fcbc4b9a700 1 --2- 172.21.15.92:0/4203851004 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbc0154200 0x7fcbc01e4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:48.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.574+0000 7fcbc5b9c700 1 --2- 172.21.15.92:0/4203851004 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbc0154670 0x7fcbc00cbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:48.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.574+0000 7fcbc539b700 1 -- 172.21.15.92:0/4203851004 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbc0154200 msgr2=0x7fcbc01e4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:48.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.574+0000 7fcbc539b700 1 --2- 172.21.15.92:0/4203851004 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbc0154200 0x7fcbc01e4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:48.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.574+0000 7fcbc539b700 1 -- 172.21.15.92:0/4203851004 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbc0154670 msgr2=0x7fcbc00cbac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:48.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.574+0000 7fcbc539b700 1 --2- 172.21.15.92:0/4203851004 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbc0154670 0x7fcbc00cbac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:48.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.574+0000 7fcbc539b700 1 -- 172.21.15.92:0/4203851004 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc00cfff0 con 0x7fcbac003a30 2023-11-28T11:21:48.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.574+0000 7fcbc539b700 1 --2- 172.21.15.92:0/4203851004 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcbac003a30 0x7fcbc01e5ca0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fcbac0046a0 tx=0x7fcbbc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:48.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.575+0000 7fcbb5ffb700 1 -- 172.21.15.92:0/4203851004 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc00e6c0 con 0x7fcbac003a30 2023-11-28T11:21:48.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.575+0000 7fcbb5ffb700 1 -- 172.21.15.92:0/4203851004 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbbc003ab0 con 0x7fcbac003a30 2023-11-28T11:21:48.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.575+0000 7fcbb5ffb700 1 -- 172.21.15.92:0/4203851004 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc003e60 con 0x7fcbac003a30 2023-11-28T11:21:48.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.575+0000 7fcbc75ff700 1 -- 172.21.15.92:0/4203851004 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbc00cfd10 con 0x7fcbac003a30 2023-11-28T11:21:48.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.575+0000 7fcbc75ff700 1 -- 172.21.15.92:0/4203851004 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbc00d29c0 con 0x7fcbac003a30 2023-11-28T11:21:48.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.576+0000 7fcbc75ff700 1 -- 172.21.15.92:0/4203851004 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbc0067880 con 0x7fcbac003a30 2023-11-28T11:21:48.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.577+0000 7fcbb5ffb700 1 -- 172.21.15.92:0/4203851004 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcbbc025070 con 0x7fcbac003a30 2023-11-28T11:21:48.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.577+0000 7fcbb5ffb700 1 --2- 172.21.15.92:0/4203851004 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fcba40682f0 0x7fcba406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:48.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.578+0000 7fcbb5ffb700 1 -- 172.21.15.92:0/4203851004 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fcbbc0876f0 con 0x7fcbac003a30 2023-11-28T11:21:48.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.578+0000 7fcbc4b9a700 1 --2- 172.21.15.92:0/4203851004 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fcba40682f0 0x7fcba406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:48.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.578+0000 7fcbc4b9a700 1 --2- 172.21.15.92:0/4203851004 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fcba40682f0 0x7fcba406a7a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fcbc008af10 tx=0x7fcbb0000ed0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:48.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.582+0000 7fcbb5ffb700 1 -- 172.21.15.92:0/4203851004 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fcbbc053c20 con 0x7fcbac003a30 2023-11-28T11:21:48.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.778+0000 7fcbc75ff700 1 -- 172.21.15.92:0/4203851004 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcbc00cd140 con 0x7fcbac003a30 2023-11-28T11:21:48.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.779+0000 7fcbb5ffb700 1 -- 172.21.15.92:0/4203851004 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+19844 (secure 0 0 0) 0x7fcbbc057240 con 0x7fcbac003a30 2023-11-28T11:21:48.781 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:21:48.781 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":29,"fsid":"68956e0c-c16f-47cd-a220-f98364b0831d","created":"2023-11-28T11:20:28.697573+0000","modified":"2023-11-28T11:21:37.638578+0000","last_up_change":"2023-11-28T11:20:41.257011+0000","last_in_change":"2023-11-28T11:20:32.249767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:20:41.925237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:20:48.682497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:20:54.471398+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:20:55.204585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:20:58.297307+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6abe483f-0e03-4a26-b9cf-62041291747e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":115823},{"type":"v1","addr":"172.21.15.92:6809","nonce":115823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":115823},{"type":"v1","addr":"172.21.15.92:6813","nonce":115823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":115823},{"type":"v1","addr":"172.21.15.92:6821","nonce":115823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":115823},{"type":"v1","addr":"172.21.15.92:6817","nonce":115823}]},"public_addr":"172.21.15.92:6809/115823","cluster_addr":"172.21.15.92:6813/115823","heartbeat_back_addr":"172.21.15.92:6821/115823","heartbeat_front_addr":"172.21.15.92:6817/115823","state":["exists","up"]},{"osd":1,"uuid":"13c1de73-1a8c-4d7e-8a28-9dae52dfe089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":115824},{"type":"v1","addr":"172.21.15.92:6812","nonce":115824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":115824},{"type":"v1","addr":"172.21.15.92:6816","nonce":115824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":115824},{"type":"v1","addr":"172.21.15.92:6823","nonce":115824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":115824},{"type":"v1","addr":"172.21.15.92:6820","nonce":115824}]},"public_addr":"172.21.15.92:6812/115824","cluster_addr":"172.21.15.92:6816/115824","heartbeat_back_addr":"172.21.15.92:6823/115824","heartbeat_front_addr":"172.21.15.92:6820/115824","state":["exists","up"]},{"osd":2,"uuid":"25788a55-112f-4f00-810d-2bc37cf76778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":115821},{"type":"v1","addr":"172.21.15.92:6801","nonce":115821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":115821},{"type":"v1","addr":"172.21.15.92:6803","nonce":115821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":115821},{"type":"v1","addr":"172.21.15.92:6807","nonce":115821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":115821},{"type":"v1","addr":"172.21.15.92:6805","nonce":115821}]},"public_addr":"172.21.15.92:6801/115821","cluster_addr":"172.21.15.92:6803/115821","heartbeat_back_addr":"172.21.15.92:6807/115821","heartbeat_front_addr":"172.21.15.92:6805/115821","state":["exists","up"]},{"osd":3,"uuid":"adf54e24-33a9-41fe-9ce3-98a5ab6c158e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":115825},{"type":"v1","addr":"172.21.15.92:6825","nonce":115825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":115825},{"type":"v1","addr":"172.21.15.92:6827","nonce":115825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":115825},{"type":"v1","addr":"172.21.15.92:6831","nonce":115825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":115825},{"type":"v1","addr":"172.21.15.92:6829","nonce":115825}]},"public_addr":"172.21.15.92:6825/115825","cluster_addr":"172.21.15.92:6827/115825","heartbeat_back_addr":"172.21.15.92:6831/115825","heartbeat_front_addr":"172.21.15.92:6829/115825","state":["exists","up"]},{"osd":4,"uuid":"ccb9b747-f399-44ea-9a18-9db3152227e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":116035},{"type":"v1","addr":"172.21.15.169:6817","nonce":116035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":116035},{"type":"v1","addr":"172.21.15.169:6819","nonce":116035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":116035},{"type":"v1","addr":"172.21.15.169:6826","nonce":116035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":116035},{"type":"v1","addr":"172.21.15.169:6822","nonce":116035}]},"public_addr":"172.21.15.169:6817/116035","cluster_addr":"172.21.15.169:6819/116035","heartbeat_back_addr":"172.21.15.169:6826/116035","heartbeat_front_addr":"172.21.15.169:6822/116035","state":["exists","up"]},{"osd":5,"uuid":"308a1e63-75aa-4d7d-8a7c-cecc24714fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":116033},{"type":"v1","addr":"172.21.15.169:6801","nonce":116033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":116033},{"type":"v1","addr":"172.21.15.169:6803","nonce":116033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":116033},{"type":"v1","addr":"172.21.15.169:6807","nonce":116033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":116033},{"type":"v1","addr":"172.21.15.169:6805","nonce":116033}]},"public_addr":"172.21.15.169:6801/116033","cluster_addr":"172.21.15.169:6803/116033","heartbeat_back_addr":"172.21.15.169:6807/116033","heartbeat_front_addr":"172.21.15.169:6805/116033","state":["exists","up"]},{"osd":6,"uuid":"9c459b7b-b821-4b4b-b91e-6aaae271457c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":116034},{"type":"v1","addr":"172.21.15.169:6809","nonce":116034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":116034},{"type":"v1","addr":"172.21.15.169:6811","nonce":116034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":116034},{"type":"v1","addr":"172.21.15.169:6815","nonce":116034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":116034},{"type":"v1","addr":"172.21.15.169:6813","nonce":116034}]},"public_addr":"172.21.15.169:6809/116034","cluster_addr":"172.21.15.169:6811/116034","heartbeat_back_addr":"172.21.15.169:6815/116034","heartbeat_front_addr":"172.21.15.169:6813/116034","state":["exists","up"]},{"osd":7,"uuid":"1ca9aa4a-755d-4e28-a937-4d8dccab4df3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":116044},{"type":"v1","addr":"172.21.15.169:6823","nonce":116044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6825","nonce":116044},{"type":"v1","addr":"172.21.15.169:6827","nonce":116044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":116044},{"type":"v1","addr":"172.21.15.169:6831","nonce":116044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":116044},{"type":"v1","addr":"172.21.15.169:6829","nonce":116044}]},"public_addr":"172.21.15.169:6823/116044","cluster_addr":"172.21.15.169:6827/116044","heartbeat_back_addr":"172.21.15.169:6831/116044","heartbeat_front_addr":"172.21.15.169:6829/116044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.586781+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.869778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.508594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.860493+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.872267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.746933+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.611520+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.871446+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.1d","mappings":[{"from":3,"to":2}]},{"pgid":"4.21","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:21:48.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.781+0000 7fcbc75ff700 1 -- 172.21.15.92:0/4203851004 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fcba40682f0 msgr2=0x7fcba406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:48.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.781+0000 7fcbc75ff700 1 --2- 172.21.15.92:0/4203851004 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fcba40682f0 0x7fcba406a7a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fcbc008af10 tx=0x7fcbb0000ed0 comp rx=0 tx=0).stop 2023-11-28T11:21:48.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.781+0000 7fcbc75ff700 1 -- 172.21.15.92:0/4203851004 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcbac003a30 msgr2=0x7fcbc01e5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:48.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.781+0000 7fcbc75ff700 1 --2- 172.21.15.92:0/4203851004 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcbac003a30 0x7fcbc01e5ca0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fcbac0046a0 tx=0x7fcbbc0047c0 comp rx=0 tx=0).stop 2023-11-28T11:21:48.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.781+0000 7fcbc75ff700 1 -- 172.21.15.92:0/4203851004 shutdown_connections 2023-11-28T11:21:48.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.781+0000 7fcbc75ff700 1 --2- 172.21.15.92:0/4203851004 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcbc0154670 0x7fcbc00cbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:48.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.781+0000 7fcbc75ff700 1 --2- 172.21.15.92:0/4203851004 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fcba40682f0 0x7fcba406a7a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:48.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.781+0000 7fcbc75ff700 1 --2- 172.21.15.92:0/4203851004 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcbac003a30 0x7fcbc01e5ca0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:48.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.781+0000 7fcbc75ff700 1 --2- 172.21.15.92:0/4203851004 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcbc0154200 0x7fcbc01e4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:48.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.781+0000 7fcbc75ff700 1 -- 172.21.15.92:0/4203851004 >> 172.21.15.92:0/4203851004 conn(0x7fcbc00719a0 msgr2=0x7fcbc00cf220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:48.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.782+0000 7fcbc75ff700 1 -- 172.21.15.92:0/4203851004 shutdown_connections 2023-11-28T11:21:48.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.782+0000 7fcbc75ff700 1 -- 172.21.15.92:0/4203851004 wait complete. 2023-11-28T11:21:48.802 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:20:41.925237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:20:48.682497+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:20:54.471398+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:20:55.204585+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T11:20:58.297307+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T11:21:48.802 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:21:49.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.999+0000 7f86747ff700 1 Processor -- start 2023-11-28T11:21:49.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:48.999+0000 7f86747ff700 1 -- start start 2023-11-28T11:21:49.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.000+0000 7f86747ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f866c0b8510 0x7f866c06b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:49.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.000+0000 7f86747ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f866c0cd460 0x7f866c06beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:49.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.000+0000 7f86747ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f866c0b88e0 con 0x7f866c06c3f0 2023-11-28T11:21:49.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.000+0000 7f86747ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f866c0cdbd0 con 0x7f866c0b8510 2023-11-28T11:21:49.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.000+0000 7f86747ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f866c0d02d0 con 0x7f866c0cd460 2023-11-28T11:21:49.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.001+0000 7f867259b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f866c0b8510 0x7f866c06b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:49.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.001+0000 7f867259b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f866c0b8510 0x7f866c06b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53996/0 (socket says 172.21.15.92:53996) 2023-11-28T11:21:49.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.001+0000 7f867259b700 1 -- 172.21.15.92:0/3055525573 learned_addr learned my addr 172.21.15.92:0/3055525573 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:49.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.001+0000 7f8671d9a700 1 --2- 172.21.15.92:0/3055525573 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f866c0cd460 0x7f866c06beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:49.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.001+0000 7f867259b700 1 -- 172.21.15.92:0/3055525573 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f866c0cd460 msgr2=0x7f866c06beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:49.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.001+0000 7f867259b700 1 --2- 172.21.15.92:0/3055525573 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f866c0cd460 0x7f866c06beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:49.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.001+0000 7f867259b700 1 -- 172.21.15.92:0/3055525573 >> v1:172.21.15.169:6789/0 conn(0x7f866c06c3f0 legacy=0x7f866c15cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:49.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.001+0000 7f867259b700 1 -- 172.21.15.92:0/3055525573 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f866c0cd8f0 con 0x7f866c0b8510 2023-11-28T11:21:49.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.001+0000 7f867259b700 1 --2- 172.21.15.92:0/3055525573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f866c0b8510 0x7f866c06b970 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8668009920 tx=0x7f8668009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bc192fc342c2106 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:49.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.002+0000 7f8671599700 1 -- 172.21.15.92:0/3055525573 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8668011830 con 0x7f866c0b8510 2023-11-28T11:21:49.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.002+0000 7f8671599700 1 -- 172.21.15.92:0/3055525573 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8668011990 con 0x7f866c0b8510 2023-11-28T11:21:49.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.002+0000 7f8671599700 1 -- 172.21.15.92:0/3055525573 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8668011d20 con 0x7f866c0b8510 2023-11-28T11:21:49.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.002+0000 7f86747ff700 1 -- 172.21.15.92:0/3055525573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f866c0b8510 msgr2=0x7f866c06b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:49.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.002+0000 7f86747ff700 1 --2- 172.21.15.92:0/3055525573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f866c0b8510 0x7f866c06b970 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8668009920 tx=0x7f8668009c30 comp rx=0 tx=0).stop 2023-11-28T11:21:49.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.002+0000 7f86747ff700 1 -- 172.21.15.92:0/3055525573 shutdown_connections 2023-11-28T11:21:49.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.002+0000 7f86747ff700 1 --2- 172.21.15.92:0/3055525573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f866c0b8510 0x7f866c06b970 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:49.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.002+0000 7f86747ff700 1 --2- 172.21.15.92:0/3055525573 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f866c0cd460 0x7f866c06beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:49.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.002+0000 7f86747ff700 1 -- 172.21.15.92:0/3055525573 >> 172.21.15.92:0/3055525573 conn(0x7f866c0719a0 msgr2=0x7f866c071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:49.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.002+0000 7f86747ff700 1 -- 172.21.15.92:0/3055525573 shutdown_connections 2023-11-28T11:21:49.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.002+0000 7f86747ff700 1 -- 172.21.15.92:0/3055525573 wait complete. 2023-11-28T11:21:49.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.005+0000 7f86747ff700 1 Processor -- start 2023-11-28T11:21:49.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.005+0000 7f86747ff700 1 -- start start 2023-11-28T11:21:49.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.006+0000 7f86747ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f866c06c3f0 0x7f866c0c8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:49.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.006+0000 7f86747ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f866c0b8510 0x7f866c1f2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:49.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.006+0000 7f86747ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f866c0cd460 0x7f866c157810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:49.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.006+0000 7f86747ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866c159220 con 0x7f866c0cd460 2023-11-28T11:21:49.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.006+0000 7f86747ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866c158610 con 0x7f866c0b8510 2023-11-28T11:21:49.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.006+0000 7f86747ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f866c1588f0 con 0x7f866c06c3f0 2023-11-28T11:21:49.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.006+0000 7f8672d9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f866c0cd460 0x7f866c157810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:49.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.006+0000 7f8672d9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f866c0cd460 0x7f866c157810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54000/0 (socket says 172.21.15.92:54000) 2023-11-28T11:21:49.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.006+0000 7f8672d9c700 1 -- 172.21.15.92:0/3185478490 learned_addr learned my addr 172.21.15.92:0/3185478490 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:49.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.007+0000 7f867259b700 1 --2- 172.21.15.92:0/3185478490 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f866c06c3f0 0x7f866c0c8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:49.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.007+0000 7f8671d9a700 1 --2- 172.21.15.92:0/3185478490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f866c0b8510 0x7f866c1f2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:49.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.007+0000 7f8672d9c700 1 -- 172.21.15.92:0/3185478490 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f866c06c3f0 msgr2=0x7f866c0c8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:49.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.007+0000 7f8672d9c700 1 --2- 172.21.15.92:0/3185478490 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f866c06c3f0 0x7f866c0c8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:49.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.007+0000 7f8672d9c700 1 -- 172.21.15.92:0/3185478490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f866c0b8510 msgr2=0x7f866c1f2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:49.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.007+0000 7f8672d9c700 1 --2- 172.21.15.92:0/3185478490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f866c0b8510 0x7f866c1f2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:49.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.007+0000 7f8672d9c700 1 -- 172.21.15.92:0/3185478490 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8668009550 con 0x7f866c0cd460 2023-11-28T11:21:49.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.007+0000 7f867259b700 1 --2- 172.21.15.92:0/3185478490 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f866c06c3f0 0x7f866c0c8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:21:49.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.007+0000 7f8672d9c700 1 --2- 172.21.15.92:0/3185478490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f866c0cd460 0x7f866c157810 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f866400cc50 tx=0x7f866400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:49.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.007+0000 7f8662ffd700 1 -- 172.21.15.92:0/3185478490 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866400ad80 con 0x7f866c0cd460 2023-11-28T11:21:49.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.007+0000 7f8662ffd700 1 -- 172.21.15.92:0/3185478490 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f866400ce50 con 0x7f866c0cd460 2023-11-28T11:21:49.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.007+0000 7f86747ff700 1 -- 172.21.15.92:0/3185478490 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f866c0cfff0 con 0x7f866c0cd460 2023-11-28T11:21:49.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.007+0000 7f8662ffd700 1 -- 172.21.15.92:0/3185478490 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8664003c40 con 0x7f866c0cd460 2023-11-28T11:21:49.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.007+0000 7f86747ff700 1 -- 172.21.15.92:0/3185478490 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f866c0cfd10 con 0x7f866c0cd460 2023-11-28T11:21:49.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.009+0000 7f86747ff700 1 -- 172.21.15.92:0/3185478490 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f866c159390 con 0x7f866c0cd460 2023-11-28T11:21:49.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.009+0000 7f8662ffd700 1 -- 172.21.15.92:0/3185478490 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8664010050 con 0x7f866c0cd460 2023-11-28T11:21:49.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.010+0000 7f8662ffd700 1 --2- 172.21.15.92:0/3185478490 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f86500682f0 0x7f865006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:49.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.010+0000 7f8662ffd700 1 -- 172.21.15.92:0/3185478490 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f8664085f50 con 0x7f866c0cd460 2023-11-28T11:21:49.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.010+0000 7f867259b700 1 --2- 172.21.15.92:0/3185478490 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f86500682f0 0x7f865006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:49.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.011+0000 7f867259b700 1 --2- 172.21.15.92:0/3185478490 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f86500682f0 0x7f865006a7a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8668009c70 tx=0x7f8668008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:49.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.014+0000 7f8662ffd700 1 -- 172.21.15.92:0/3185478490 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8664052480 con 0x7f866c0cd460 2023-11-28T11:21:49.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.202+0000 7f86747ff700 1 -- 172.21.15.92:0/3185478490 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f866c067880 con 0x7f866c0cd460 2023-11-28T11:21:49.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.202+0000 7f8662ffd700 1 -- 172.21.15.92:0/3185478490 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8664055aa0 con 0x7f866c0cd460 2023-11-28T11:21:49.205 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-11-28T11:21:49.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.205+0000 7f86747ff700 1 -- 172.21.15.92:0/3185478490 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f86500682f0 msgr2=0x7f865006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:49.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.205+0000 7f86747ff700 1 --2- 172.21.15.92:0/3185478490 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f86500682f0 0x7f865006a7a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8668009c70 tx=0x7f8668008030 comp rx=0 tx=0).stop 2023-11-28T11:21:49.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.205+0000 7f86747ff700 1 -- 172.21.15.92:0/3185478490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f866c0cd460 msgr2=0x7f866c157810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:49.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.205+0000 7f86747ff700 1 --2- 172.21.15.92:0/3185478490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f866c0cd460 0x7f866c157810 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f866400cc50 tx=0x7f866400a340 comp rx=0 tx=0).stop 2023-11-28T11:21:49.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.205+0000 7f86747ff700 1 -- 172.21.15.92:0/3185478490 shutdown_connections 2023-11-28T11:21:49.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.205+0000 7f86747ff700 1 --2- 172.21.15.92:0/3185478490 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f86500682f0 0x7f865006a7a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:49.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.205+0000 7f86747ff700 1 --2- 172.21.15.92:0/3185478490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f866c0cd460 0x7f866c157810 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:49.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.205+0000 7f86747ff700 1 --2- 172.21.15.92:0/3185478490 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f866c06c3f0 0x7f866c0c8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:49.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.205+0000 7f86747ff700 1 --2- 172.21.15.92:0/3185478490 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f866c0b8510 0x7f866c1f2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:49.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.205+0000 7f86747ff700 1 -- 172.21.15.92:0/3185478490 >> 172.21.15.92:0/3185478490 conn(0x7f866c0719a0 msgr2=0x7f866c0b5d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:49.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.205+0000 7f86747ff700 1 -- 172.21.15.92:0/3185478490 shutdown_connections 2023-11-28T11:21:49.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.205+0000 7f86747ff700 1 -- 172.21.15.92:0/3185478490 wait complete. 2023-11-28T11:21:49.225 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:21:49.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.422+0000 7f1b4fdff700 1 Processor -- start 2023-11-28T11:21:49.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.422+0000 7f1b4fdff700 1 -- start start 2023-11-28T11:21:49.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.423+0000 7f1b4fdff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1b480b8510 0x7f1b4806b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:49.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.423+0000 7f1b4fdff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b480cd460 0x7f1b4806beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:49.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.423+0000 7f1b4fdff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1b4806c3f0 0x7f1b4815cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:49.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.423+0000 7f1b4fdff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b480cdbd0 con 0x7f1b480b8510 2023-11-28T11:21:49.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.423+0000 7f1b4fdff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b480d02d0 con 0x7f1b4806c3f0 2023-11-28T11:21:49.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.423+0000 7f1b4fdff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b480b88e0 con 0x7f1b480cd460 2023-11-28T11:21:49.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.423+0000 7f1b4db9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1b480b8510 0x7f1b4806b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:49.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.423+0000 7f1b4db9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1b480b8510 0x7f1b4806b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54006/0 (socket says 172.21.15.92:54006) 2023-11-28T11:21:49.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.423+0000 7f1b4db9b700 1 -- 172.21.15.92:0/1863971781 learned_addr learned my addr 172.21.15.92:0/1863971781 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:49.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.423+0000 7f1b4e39c700 1 --2- 172.21.15.92:0/1863971781 >> v2:172.21.15.169:3300/0 conn(0x7f1b4806c3f0 0x7f1b4815cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:49.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.423+0000 7f1b4d39a700 1 --2- 172.21.15.92:0/1863971781 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b480cd460 0x7f1b4806beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:49.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.424+0000 7f1b4db9b700 1 -- 172.21.15.92:0/1863971781 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b480cd460 msgr2=0x7f1b4806beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:49.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.424+0000 7f1b4db9b700 1 --2- 172.21.15.92:0/1863971781 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b480cd460 0x7f1b4806beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:49.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.424+0000 7f1b4db9b700 1 -- 172.21.15.92:0/1863971781 >> v2:172.21.15.169:3300/0 conn(0x7f1b4806c3f0 msgr2=0x7f1b4815cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:49.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.424+0000 7f1b4db9b700 1 --2- 172.21.15.92:0/1863971781 >> v2:172.21.15.169:3300/0 conn(0x7f1b4806c3f0 0x7f1b4815cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:49.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.424+0000 7f1b4db9b700 1 -- 172.21.15.92:0/1863971781 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b480cd8f0 con 0x7f1b480b8510 2023-11-28T11:21:49.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.424+0000 7f1b4d39a700 1 --2- 172.21.15.92:0/1863971781 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b480cd460 0x7f1b4806beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:21:49.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.424+0000 7f1b4db9b700 1 --2- 172.21.15.92:0/1863971781 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b480b8510 0x7f1b4806b970 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1b480b4c80 tx=0x7f1b44009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71a2d4762bcb88ea server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:49.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.424+0000 7f1b4cb99700 1 -- 172.21.15.92:0/1863971781 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b44011830 con 0x7f1b480b8510 2023-11-28T11:21:49.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.424+0000 7f1b4cb99700 1 -- 172.21.15.92:0/1863971781 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b44011990 con 0x7f1b480b8510 2023-11-28T11:21:49.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.424+0000 7f1b4cb99700 1 -- 172.21.15.92:0/1863971781 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b44011d20 con 0x7f1b480b8510 2023-11-28T11:21:49.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.425+0000 7f1b4fdff700 1 -- 172.21.15.92:0/1863971781 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b480b8510 msgr2=0x7f1b4806b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:49.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.425+0000 7f1b4fdff700 1 --2- 172.21.15.92:0/1863971781 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b480b8510 0x7f1b4806b970 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1b480b4c80 tx=0x7f1b44009c00 comp rx=0 tx=0).stop 2023-11-28T11:21:49.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.425+0000 7f1b4fdff700 1 -- 172.21.15.92:0/1863971781 shutdown_connections 2023-11-28T11:21:49.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.425+0000 7f1b4fdff700 1 --2- 172.21.15.92:0/1863971781 >> v2:172.21.15.169:3300/0 conn(0x7f1b4806c3f0 0x7f1b4815cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:49.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.425+0000 7f1b4fdff700 1 --2- 172.21.15.92:0/1863971781 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b480b8510 0x7f1b4806b970 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:49.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.425+0000 7f1b4fdff700 1 --2- 172.21.15.92:0/1863971781 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b480cd460 0x7f1b4806beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:49.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.425+0000 7f1b4fdff700 1 -- 172.21.15.92:0/1863971781 >> 172.21.15.92:0/1863971781 conn(0x7f1b480719a0 msgr2=0x7f1b48071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:49.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.425+0000 7f1b4fdff700 1 -- 172.21.15.92:0/1863971781 shutdown_connections 2023-11-28T11:21:49.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.425+0000 7f1b4fdff700 1 -- 172.21.15.92:0/1863971781 wait complete. 2023-11-28T11:21:49.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.427+0000 7f1b4fdff700 1 Processor -- start 2023-11-28T11:21:49.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.428+0000 7f1b4fdff700 1 -- start start 2023-11-28T11:21:49.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.429+0000 7f1b4fdff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b4806c3f0 0x7f1b480cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:49.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.429+0000 7f1b4fdff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b480b8510 0x7f1b48156600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:49.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.429+0000 7f1b4fdff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b480cd460 0x7f1b481f2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:49.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.429+0000 7f1b4fdff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b48158900 con 0x7f1b480b8510 2023-11-28T11:21:49.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.429+0000 7f1b4fdff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b48159f10 con 0x7f1b4806c3f0 2023-11-28T11:21:49.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.429+0000 7f1b4fdff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b481e1970 con 0x7f1b480cd460 2023-11-28T11:21:49.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.429+0000 7f1b4d39a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b480b8510 0x7f1b48156600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:49.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.429+0000 7f1b4d39a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b480b8510 0x7f1b48156600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54008/0 (socket says 172.21.15.92:54008) 2023-11-28T11:21:49.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.429+0000 7f1b4d39a700 1 -- 172.21.15.92:0/3412140403 learned_addr learned my addr 172.21.15.92:0/3412140403 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:49.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.429+0000 7f1b4db9b700 1 --2- 172.21.15.92:0/3412140403 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b4806c3f0 0x7f1b480cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:49.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.429+0000 7f1b4e39c700 1 --2- 172.21.15.92:0/3412140403 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b480cd460 0x7f1b481f2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:49.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.429+0000 7f1b4d39a700 1 -- 172.21.15.92:0/3412140403 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b480cd460 msgr2=0x7f1b481f2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:49.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.429+0000 7f1b4d39a700 1 --2- 172.21.15.92:0/3412140403 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b480cd460 0x7f1b481f2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:49.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.430+0000 7f1b4d39a700 1 -- 172.21.15.92:0/3412140403 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b4806c3f0 msgr2=0x7f1b480cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:49.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.430+0000 7f1b4d39a700 1 --2- 172.21.15.92:0/3412140403 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b4806c3f0 0x7f1b480cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:49.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.430+0000 7f1b4d39a700 1 -- 172.21.15.92:0/3412140403 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b44009550 con 0x7f1b480b8510 2023-11-28T11:21:49.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.430+0000 7f1b4db9b700 1 --2- 172.21.15.92:0/3412140403 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b4806c3f0 0x7f1b480cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:21:49.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.430+0000 7f1b4d39a700 1 --2- 172.21.15.92:0/3412140403 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b480b8510 0x7f1b48156600 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1b3800e810 tx=0x7f1b3800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:49.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.430+0000 7f1b3e7fc700 1 -- 172.21.15.92:0/3412140403 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3800c020 con 0x7f1b480b8510 2023-11-28T11:21:49.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.430+0000 7f1b3e7fc700 1 -- 172.21.15.92:0/3412140403 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b38004250 con 0x7f1b480b8510 2023-11-28T11:21:49.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.430+0000 7f1b3e7fc700 1 -- 172.21.15.92:0/3412140403 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b380036a0 con 0x7f1b480b8510 2023-11-28T11:21:49.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.430+0000 7f1b4fdff700 1 -- 172.21.15.92:0/3412140403 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b480cfff0 con 0x7f1b480b8510 2023-11-28T11:21:49.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.430+0000 7f1b4fdff700 1 -- 172.21.15.92:0/3412140403 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b480cfd10 con 0x7f1b480b8510 2023-11-28T11:21:49.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.431+0000 7f1b4fdff700 1 -- 172.21.15.92:0/3412140403 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b48157ce0 con 0x7f1b480b8510 2023-11-28T11:21:49.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.432+0000 7f1b3e7fc700 1 -- 172.21.15.92:0/3412140403 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1b38003800 con 0x7f1b480b8510 2023-11-28T11:21:49.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.433+0000 7f1b3e7fc700 1 --2- 172.21.15.92:0/3412140403 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f1b2c0682f0 0x7f1b2c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:49.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.434+0000 7f1b3e7fc700 1 -- 172.21.15.92:0/3412140403 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f1b38085d90 con 0x7f1b480b8510 2023-11-28T11:21:49.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.434+0000 7f1b4db9b700 1 --2- 172.21.15.92:0/3412140403 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f1b2c0682f0 0x7f1b2c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:49.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.434+0000 7f1b4db9b700 1 --2- 172.21.15.92:0/3412140403 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f1b2c0682f0 0x7f1b2c06a7a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1b440098f0 tx=0x7f1b44008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:49.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.438+0000 7f1b3e7fc700 1 -- 172.21.15.92:0/3412140403 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1b380521d0 con 0x7f1b480b8510 2023-11-28T11:21:49.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.626+0000 7f1b4fdff700 1 -- 172.21.15.92:0/3412140403 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1b48067880 con 0x7f1b480b8510 2023-11-28T11:21:49.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.627+0000 7f1b3e7fc700 1 -- 172.21.15.92:0/3412140403 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1b3800f040 con 0x7f1b480b8510 2023-11-28T11:21:49.629 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-11-28T11:21:49.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.629+0000 7f1b4fdff700 1 -- 172.21.15.92:0/3412140403 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f1b2c0682f0 msgr2=0x7f1b2c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:49.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.629+0000 7f1b4fdff700 1 --2- 172.21.15.92:0/3412140403 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f1b2c0682f0 0x7f1b2c06a7a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1b440098f0 tx=0x7f1b44008030 comp rx=0 tx=0).stop 2023-11-28T11:21:49.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.629+0000 7f1b4fdff700 1 -- 172.21.15.92:0/3412140403 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b480b8510 msgr2=0x7f1b48156600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:49.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.629+0000 7f1b4fdff700 1 --2- 172.21.15.92:0/3412140403 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b480b8510 0x7f1b48156600 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1b3800e810 tx=0x7f1b3800ebd0 comp rx=0 tx=0).stop 2023-11-28T11:21:49.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.629+0000 7f1b4fdff700 1 -- 172.21.15.92:0/3412140403 shutdown_connections 2023-11-28T11:21:49.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.629+0000 7f1b4fdff700 1 --2- 172.21.15.92:0/3412140403 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1b480cd460 0x7f1b481f2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:49.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.629+0000 7f1b4fdff700 1 --2- 172.21.15.92:0/3412140403 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1b4806c3f0 0x7f1b480cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:49.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.629+0000 7f1b4fdff700 1 --2- 172.21.15.92:0/3412140403 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f1b2c0682f0 0x7f1b2c06a7a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:49.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.629+0000 7f1b4fdff700 1 --2- 172.21.15.92:0/3412140403 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b480b8510 0x7f1b48156600 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:49.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.629+0000 7f1b4fdff700 1 -- 172.21.15.92:0/3412140403 >> 172.21.15.92:0/3412140403 conn(0x7f1b480719a0 msgr2=0x7f1b480b4040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:49.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.629+0000 7f1b4fdff700 1 -- 172.21.15.92:0/3412140403 shutdown_connections 2023-11-28T11:21:49.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.629+0000 7f1b4fdff700 1 -- 172.21.15.92:0/3412140403 wait complete. 2023-11-28T11:21:49.649 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:21:49.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.847+0000 7f609ddff700 1 Processor -- start 2023-11-28T11:21:49.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.847+0000 7f609ddff700 1 -- start start 2023-11-28T11:21:49.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.848+0000 7f609ddff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f60980cd460 0x7f609806c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:49.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.849+0000 7f609ddff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60980b88e0 con 0x7f60980b8510 2023-11-28T11:21:49.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.849+0000 7f609ddff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60980cd8f0 con 0x7f60980d0d60 2023-11-28T11:21:49.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.849+0000 7f609ddff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60980cdbd0 con 0x7f60980cd460 2023-11-28T11:21:49.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.849+0000 7f60977fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f60980b8510 0x7f609806b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55560/0 (socket says 172.21.15.92:55560) 2023-11-28T11:21:49.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.849+0000 7f60977fe700 1 -- 172.21.15.92:0/249916819 learned_addr learned my addr 172.21.15.92:0/249916819 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:49.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.849+0000 7f608ffff700 1 --2- 172.21.15.92:0/249916819 >> v2:172.21.15.169:3300/0 conn(0x7f60980cd460 0x7f609806c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:49.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.849+0000 7f6096ffd700 1 -- 172.21.15.92:0/249916819 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3964996871 0 0) 0x7f60980b88e0 con 0x7f60980b8510 2023-11-28T11:21:49.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.849+0000 7f6096ffd700 1 -- 172.21.15.92:0/249916819 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60840035b0 con 0x7f60980b8510 2023-11-28T11:21:49.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.849+0000 7f6096ffd700 1 -- 172.21.15.92:0/249916819 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3343769871 0 0) 0x7f60840035b0 con 0x7f60980b8510 2023-11-28T11:21:49.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.849+0000 7f6096ffd700 1 -- 172.21.15.92:0/249916819 >> v2:172.21.15.169:3300/0 conn(0x7f60980cd460 msgr2=0x7f609806c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:49.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.849+0000 7f6096ffd700 1 --2- 172.21.15.92:0/249916819 >> v2:172.21.15.169:3300/0 conn(0x7f60980cd460 0x7f609806c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:49.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.849+0000 7f6096ffd700 1 -- 172.21.15.92:0/249916819 >> v1:172.21.15.169:6789/0 conn(0x7f60980d0d60 legacy=0x7f60980d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:49.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.849+0000 7f6096ffd700 1 -- 172.21.15.92:0/249916819 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60980cfd10 con 0x7f60980b8510 2023-11-28T11:21:49.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.849+0000 7f6096ffd700 1 -- 172.21.15.92:0/249916819 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532161587 0 0) 0x7f6088001670 con 0x7f60980b8510 2023-11-28T11:21:49.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.850+0000 7f6096ffd700 1 -- 172.21.15.92:0/249916819 >> v1:172.21.15.92:6789/0 conn(0x7f60980b8510 legacy=0x7f609806b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:49.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.850+0000 7f6096ffd700 1 --2- 172.21.15.92:0/249916819 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6084003a50 0x7f6084003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:49.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.850+0000 7f6096ffd700 1 --2- 172.21.15.92:0/249916819 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6084004850 0x7f6084004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:49.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.850+0000 7f6096ffd700 1 --2- 172.21.15.92:0/249916819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6084005200 0x7f608400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:49.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.850+0000 7f6096ffd700 1 -- 172.21.15.92:0/249916819 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60980cdbd0 con 0x7f6084005200 2023-11-28T11:21:49.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.850+0000 7f6096ffd700 1 -- 172.21.15.92:0/249916819 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f608400c0e0 con 0x7f6084003a50 2023-11-28T11:21:49.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.850+0000 7f6096ffd700 1 -- 172.21.15.92:0/249916819 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f608400c2a0 con 0x7f6084004850 2023-11-28T11:21:49.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.850+0000 7f6097fff700 1 --2- 172.21.15.92:0/249916819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6084005200 0x7f608400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:49.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.850+0000 7f60977fe700 1 --2- 172.21.15.92:0/249916819 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6084003a50 0x7f6084003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:49.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.850+0000 7f608ffff700 1 --2- 172.21.15.92:0/249916819 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6084004850 0x7f6084004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:49.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.850+0000 7f6097fff700 1 -- 172.21.15.92:0/249916819 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6084004850 msgr2=0x7f6084004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:49.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.850+0000 7f6097fff700 1 --2- 172.21.15.92:0/249916819 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6084004850 0x7f6084004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:49.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.850+0000 7f6097fff700 1 -- 172.21.15.92:0/249916819 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6084003a50 msgr2=0x7f6084003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:49.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.850+0000 7f6097fff700 1 --2- 172.21.15.92:0/249916819 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6084003a50 0x7f6084003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:49.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.850+0000 7f6097fff700 1 -- 172.21.15.92:0/249916819 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60840035b0 con 0x7f6084005200 2023-11-28T11:21:49.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.850+0000 7f6097fff700 1 -- 172.21.15.92:0/249916819 reap_dead start 2023-11-28T11:21:49.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.851+0000 7f6097fff700 1 --2- 172.21.15.92:0/249916819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6084005200 0x7f608400ba10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f609000c160 tx=0x7f6084005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7180a08921563f45 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:49.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.851+0000 7f6096ffd700 1 -- 172.21.15.92:0/249916819 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6090001db0 con 0x7f6084005200 2023-11-28T11:21:49.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.851+0000 7f6096ffd700 1 -- 172.21.15.92:0/249916819 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f609000cca0 con 0x7f6084005200 2023-11-28T11:21:49.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.851+0000 7f6096ffd700 1 -- 172.21.15.92:0/249916819 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6090003c00 con 0x7f6084005200 2023-11-28T11:21:49.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.851+0000 7f609ddff700 1 -- 172.21.15.92:0/249916819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6084005200 msgr2=0x7f608400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:49.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.851+0000 7f609ddff700 1 --2- 172.21.15.92:0/249916819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6084005200 0x7f608400ba10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f609000c160 tx=0x7f6084005880 comp rx=0 tx=0).stop 2023-11-28T11:21:49.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.851+0000 7f609ddff700 1 -- 172.21.15.92:0/249916819 shutdown_connections 2023-11-28T11:21:49.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.851+0000 7f609ddff700 1 --2- 172.21.15.92:0/249916819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6084005200 0x7f608400ba10 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:49.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.851+0000 7f609ddff700 1 -- 172.21.15.92:0/249916819 >> 172.21.15.92:0/249916819 conn(0x7f60980719a0 msgr2=0x7f6098071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:49.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.851+0000 7f609ddff700 1 -- 172.21.15.92:0/249916819 shutdown_connections 2023-11-28T11:21:49.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.851+0000 7f609ddff700 1 -- 172.21.15.92:0/249916819 wait complete. 2023-11-28T11:21:49.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.853+0000 7f609ddff700 1 Processor -- start 2023-11-28T11:21:49.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.853+0000 7f609ddff700 1 -- start start 2023-11-28T11:21:49.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.854+0000 7f609ddff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6084005200 0x7f609815b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:49.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.854+0000 7f609ddff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60981542f0 0x7f60981f2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:49.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.854+0000 7f609ddff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6098159590 0x7f609814ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:49.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.854+0000 7f609ddff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6098156a90 con 0x7f60981542f0 2023-11-28T11:21:49.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.854+0000 7f609ddff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6098156d70 con 0x7f6098159590 2023-11-28T11:21:49.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.854+0000 7f609ddff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6098157050 con 0x7f6084005200 2023-11-28T11:21:49.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.854+0000 7f608ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60981542f0 0x7f60981f2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:49.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.854+0000 7f608ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60981542f0 0x7f60981f2020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54022/0 (socket says 172.21.15.92:54022) 2023-11-28T11:21:49.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.854+0000 7f608ffff700 1 -- 172.21.15.92:0/3740927055 learned_addr learned my addr 172.21.15.92:0/3740927055 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:49.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.855+0000 7f60977fe700 1 --2- 172.21.15.92:0/3740927055 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6084005200 0x7f609815b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:49.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.855+0000 7f6097fff700 1 --2- 172.21.15.92:0/3740927055 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6098159590 0x7f609814ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:49.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.855+0000 7f608ffff700 1 -- 172.21.15.92:0/3740927055 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6084005200 msgr2=0x7f609815b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:49.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.855+0000 7f608ffff700 1 --2- 172.21.15.92:0/3740927055 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6084005200 0x7f609815b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:49.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.855+0000 7f608ffff700 1 -- 172.21.15.92:0/3740927055 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6098159590 msgr2=0x7f609814ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:49.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.855+0000 7f608ffff700 1 --2- 172.21.15.92:0/3740927055 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6098159590 0x7f609814ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:49.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.855+0000 7f608ffff700 1 -- 172.21.15.92:0/3740927055 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60980d29c0 con 0x7f60981542f0 2023-11-28T11:21:49.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.855+0000 7f6097fff700 1 --2- 172.21.15.92:0/3740927055 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6098159590 0x7f609814ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:21:49.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.855+0000 7f60977fe700 1 --2- 172.21.15.92:0/3740927055 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6084005200 0x7f609815b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:21:49.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.855+0000 7f608ffff700 1 --2- 172.21.15.92:0/3740927055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60981542f0 0x7f60981f2020 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6080011960 tx=0x7f6080011d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:49.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.855+0000 7f6094ff9700 1 -- 172.21.15.92:0/3740927055 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6080004240 con 0x7f60981542f0 2023-11-28T11:21:49.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.855+0000 7f6094ff9700 1 -- 172.21.15.92:0/3740927055 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60800043a0 con 0x7f60981542f0 2023-11-28T11:21:49.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.855+0000 7f609ddff700 1 -- 172.21.15.92:0/3740927055 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60980cfff0 con 0x7f60981542f0 2023-11-28T11:21:49.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.855+0000 7f6094ff9700 1 -- 172.21.15.92:0/3740927055 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60800036a0 con 0x7f60981542f0 2023-11-28T11:21:49.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.856+0000 7f609ddff700 1 -- 172.21.15.92:0/3740927055 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60980d2b50 con 0x7f60981542f0 2023-11-28T11:21:49.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.856+0000 7f609ddff700 1 -- 172.21.15.92:0/3740927055 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6098067880 con 0x7f60981542f0 2023-11-28T11:21:49.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.857+0000 7f6094ff9700 1 -- 172.21.15.92:0/3740927055 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6080025070 con 0x7f60981542f0 2023-11-28T11:21:49.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.858+0000 7f6094ff9700 1 --2- 172.21.15.92:0/3740927055 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f60780682f0 0x7f607806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:49.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.858+0000 7f6094ff9700 1 -- 172.21.15.92:0/3740927055 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f60800864b0 con 0x7f60981542f0 2023-11-28T11:21:49.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.858+0000 7f60977fe700 1 --2- 172.21.15.92:0/3740927055 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f60780682f0 0x7f607806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:49.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.859+0000 7f60977fe700 1 --2- 172.21.15.92:0/3740927055 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f60780682f0 0x7f607806a7a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6098156840 tx=0x7f6088009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:49.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:49.862+0000 7f6094ff9700 1 -- 172.21.15.92:0/3740927055 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f60800528f0 con 0x7f60981542f0 2023-11-28T11:21:50.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.052+0000 7f609ddff700 1 -- 172.21.15.92:0/3740927055 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f60980d1f50 con 0x7f60981542f0 2023-11-28T11:21:50.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.053+0000 7f6094ff9700 1 -- 172.21.15.92:0/3740927055 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6080055f10 con 0x7f60981542f0 2023-11-28T11:21:50.055 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-11-28T11:21:50.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.055+0000 7f609ddff700 1 -- 172.21.15.92:0/3740927055 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f60780682f0 msgr2=0x7f607806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:50.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.055+0000 7f609ddff700 1 --2- 172.21.15.92:0/3740927055 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f60780682f0 0x7f607806a7a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6098156840 tx=0x7f6088009040 comp rx=0 tx=0).stop 2023-11-28T11:21:50.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.055+0000 7f609ddff700 1 -- 172.21.15.92:0/3740927055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60981542f0 msgr2=0x7f60981f2020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:50.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.055+0000 7f609ddff700 1 --2- 172.21.15.92:0/3740927055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60981542f0 0x7f60981f2020 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6080011960 tx=0x7f6080011d20 comp rx=0 tx=0).stop 2023-11-28T11:21:50.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.055+0000 7f609ddff700 1 -- 172.21.15.92:0/3740927055 shutdown_connections 2023-11-28T11:21:50.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.055+0000 7f609ddff700 1 --2- 172.21.15.92:0/3740927055 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6098159590 0x7f609814ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:50.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.055+0000 7f609ddff700 1 --2- 172.21.15.92:0/3740927055 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6084005200 0x7f609815b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:50.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.055+0000 7f609ddff700 1 --2- 172.21.15.92:0/3740927055 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f60780682f0 0x7f607806a7a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:50.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.055+0000 7f609ddff700 1 --2- 172.21.15.92:0/3740927055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60981542f0 0x7f60981f2020 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:50.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.055+0000 7f609ddff700 1 -- 172.21.15.92:0/3740927055 >> 172.21.15.92:0/3740927055 conn(0x7f60980719a0 msgr2=0x7f60980b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:50.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.055+0000 7f609ddff700 1 -- 172.21.15.92:0/3740927055 shutdown_connections 2023-11-28T11:21:50.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.055+0000 7f609ddff700 1 -- 172.21.15.92:0/3740927055 wait complete. 2023-11-28T11:21:50.075 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:21:50.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.272+0000 7f4fb95ff700 1 Processor -- start 2023-11-28T11:21:50.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.272+0000 7f4fb95ff700 1 -- start start 2023-11-28T11:21:50.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.274+0000 7f4fb95ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4fb40d1d70 0x7f4fb4167270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:50.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.274+0000 7f4fb95ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fb406c790 con 0x7f4fb40b8510 2023-11-28T11:21:50.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.274+0000 7f4fb95ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fb40b88e0 con 0x7f4fb40cd460 2023-11-28T11:21:50.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.274+0000 7f4fb95ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb40cdbd0 con 0x7f4fb40d1d70 2023-11-28T11:21:50.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.274+0000 7f4fb2ffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f4fb40b8510 0x7f4fb406b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55570/0 (socket says 172.21.15.92:55570) 2023-11-28T11:21:50.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.274+0000 7f4fb2ffd700 1 -- 172.21.15.92:0/228657708 learned_addr learned my addr 172.21.15.92:0/228657708 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:50.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.274+0000 7f4fb37fe700 1 --2- 172.21.15.92:0/228657708 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4fb40d1d70 0x7f4fb4167270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:50.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.274+0000 7f4fb27fc700 1 -- 172.21.15.92:0/228657708 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3958463246 0 0) 0x7f4fb406c790 con 0x7f4fb40b8510 2023-11-28T11:21:50.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.274+0000 7f4fb27fc700 1 -- 172.21.15.92:0/228657708 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fa00035b0 con 0x7f4fb40b8510 2023-11-28T11:21:50.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.274+0000 7f4fb27fc700 1 -- 172.21.15.92:0/228657708 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3632384278 0 0) 0x7f4fb40b88e0 con 0x7f4fb40cd460 2023-11-28T11:21:50.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.274+0000 7f4fb27fc700 1 -- 172.21.15.92:0/228657708 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fb406c790 con 0x7f4fb40cd460 2023-11-28T11:21:50.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.274+0000 7f4fb27fc700 1 -- 172.21.15.92:0/228657708 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3374418868 0 0) 0x7f4fa00035b0 con 0x7f4fb40b8510 2023-11-28T11:21:50.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.274+0000 7f4fb27fc700 1 -- 172.21.15.92:0/228657708 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4fb40d1d70 msgr2=0x7f4fb4167270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:50.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.274+0000 7f4fb27fc700 1 --2- 172.21.15.92:0/228657708 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4fb40d1d70 0x7f4fb4167270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:50.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.274+0000 7f4fb27fc700 1 -- 172.21.15.92:0/228657708 >> v1:172.21.15.169:6789/0 conn(0x7f4fb40cd460 legacy=0x7f4fb406c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:50.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.275+0000 7f4fb27fc700 1 -- 172.21.15.92:0/228657708 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb40cd8f0 con 0x7f4fb40b8510 2023-11-28T11:21:50.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.275+0000 7f4fb27fc700 1 -- 172.21.15.92:0/228657708 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532161587 0 0) 0x7f4fa4001670 con 0x7f4fb40b8510 2023-11-28T11:21:50.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.275+0000 7f4fb37fe700 1 --2- 172.21.15.92:0/228657708 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4fb40d1d70 0x7f4fb4167270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:21:50.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.275+0000 7f4fb27fc700 1 -- 172.21.15.92:0/228657708 >> v1:172.21.15.92:6789/0 conn(0x7f4fb40b8510 legacy=0x7f4fb406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:50.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.275+0000 7f4fb27fc700 1 --2- 172.21.15.92:0/228657708 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fa0003bc0 0x7f4fa0003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:50.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.275+0000 7f4fb27fc700 1 --2- 172.21.15.92:0/228657708 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4fa0004850 0x7f4fa0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:50.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.275+0000 7f4fb27fc700 1 --2- 172.21.15.92:0/228657708 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4fb40d1d70 0x7f4fa0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:50.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.275+0000 7f4fb27fc700 1 -- 172.21.15.92:0/228657708 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb40cdbd0 con 0x7f4fa0003bc0 2023-11-28T11:21:50.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.275+0000 7f4fb27fc700 1 -- 172.21.15.92:0/228657708 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa000ba70 con 0x7f4fa0004850 2023-11-28T11:21:50.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.275+0000 7f4fb27fc700 1 -- 172.21.15.92:0/228657708 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa000bc30 con 0x7f4fb40d1d70 2023-11-28T11:21:50.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.275+0000 7f4fb2ffd700 1 --2- 172.21.15.92:0/228657708 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fa0003bc0 0x7f4fa0003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:50.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.275+0000 7f4fabfff700 1 --2- 172.21.15.92:0/228657708 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4fa0004850 0x7f4fa0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:50.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.275+0000 7f4fb37fe700 1 --2- 172.21.15.92:0/228657708 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4fb40d1d70 0x7f4fa0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:50.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.275+0000 7f4fb2ffd700 1 -- 172.21.15.92:0/228657708 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4fb40d1d70 msgr2=0x7f4fa0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:50.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.275+0000 7f4fb2ffd700 1 --2- 172.21.15.92:0/228657708 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4fb40d1d70 0x7f4fa0005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:50.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.275+0000 7f4fb2ffd700 1 -- 172.21.15.92:0/228657708 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4fa0004850 msgr2=0x7f4fa0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:50.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.275+0000 7f4fb2ffd700 1 --2- 172.21.15.92:0/228657708 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4fa0004850 0x7f4fa0004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:50.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.275+0000 7f4fb2ffd700 1 -- 172.21.15.92:0/228657708 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fac009a70 con 0x7f4fa0003bc0 2023-11-28T11:21:50.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.275+0000 7f4fb37fe700 1 --2- 172.21.15.92:0/228657708 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4fb40d1d70 0x7f4fa0005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:21:50.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.276+0000 7f4fb2ffd700 1 --2- 172.21.15.92:0/228657708 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fa0003bc0 0x7f4fa0003fb0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4fa400c210 tx=0x7f4fa0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe87aa0dff402944 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:50.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.276+0000 7f4fb27fc700 1 -- 172.21.15.92:0/228657708 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa400cc90 con 0x7f4fa0003bc0 2023-11-28T11:21:50.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.276+0000 7f4fb27fc700 1 -- 172.21.15.92:0/228657708 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fa400ce10 con 0x7f4fa0003bc0 2023-11-28T11:21:50.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.276+0000 7f4fb27fc700 1 -- 172.21.15.92:0/228657708 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa4005ba0 con 0x7f4fa0003bc0 2023-11-28T11:21:50.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.276+0000 7f4fb95ff700 1 -- 172.21.15.92:0/228657708 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fa0003bc0 msgr2=0x7f4fa0003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:50.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.276+0000 7f4fb95ff700 1 --2- 172.21.15.92:0/228657708 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fa0003bc0 0x7f4fa0003fb0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4fa400c210 tx=0x7f4fa0005880 comp rx=0 tx=0).stop 2023-11-28T11:21:50.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.276+0000 7f4fb37fe700 1 -- 172.21.15.92:0/228657708 reap_dead start 2023-11-28T11:21:50.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.276+0000 7f4fb95ff700 1 -- 172.21.15.92:0/228657708 shutdown_connections 2023-11-28T11:21:50.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.276+0000 7f4fb95ff700 1 -- 172.21.15.92:0/228657708 >> 172.21.15.92:0/228657708 conn(0x7f4fb40719a0 msgr2=0x7f4fb4071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:50.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.276+0000 7f4fb95ff700 1 -- 172.21.15.92:0/228657708 shutdown_connections 2023-11-28T11:21:50.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.277+0000 7f4fb95ff700 1 -- 172.21.15.92:0/228657708 wait complete. 2023-11-28T11:21:50.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.278+0000 7f4fb95ff700 1 Processor -- start 2023-11-28T11:21:50.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.278+0000 7f4fb95ff700 1 -- start start 2023-11-28T11:21:50.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.279+0000 7f4fb95ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fa0003bc0 0x7f4fb40d2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:50.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.279+0000 7f4fb95ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4fb41e50e0 0x7f4fb41e5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:50.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.279+0000 7f4fb95ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4fb406b970 0x7f4fb406bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:50.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.279+0000 7f4fb95ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb41e4cc0 con 0x7f4fa0003bc0 2023-11-28T11:21:50.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.279+0000 7f4fb95ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb406c320 con 0x7f4fb406b970 2023-11-28T11:21:50.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.279+0000 7f4fb95ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb40d37a0 con 0x7f4fb41e50e0 2023-11-28T11:21:50.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.280+0000 7f4fb2ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fa0003bc0 0x7f4fb40d2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:50.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.280+0000 7f4fb2ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fa0003bc0 0x7f4fb40d2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54036/0 (socket says 172.21.15.92:54036) 2023-11-28T11:21:50.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.280+0000 7f4fb2ffd700 1 -- 172.21.15.92:0/1223345429 learned_addr learned my addr 172.21.15.92:0/1223345429 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:50.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.280+0000 7f4fb2ffd700 1 -- 172.21.15.92:0/1223345429 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4fb41e50e0 msgr2=0x7f4fb41e5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:50.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.280+0000 7f4fb2ffd700 1 --2- 172.21.15.92:0/1223345429 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4fb41e50e0 0x7f4fb41e5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:50.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.280+0000 7f4fb2ffd700 1 -- 172.21.15.92:0/1223345429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4fb406b970 msgr2=0x7f4fb406bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:50.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.280+0000 7f4fb2ffd700 1 --2- 172.21.15.92:0/1223345429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4fb406b970 0x7f4fb406bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:50.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.280+0000 7f4fb2ffd700 1 -- 172.21.15.92:0/1223345429 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb40cfff0 con 0x7f4fa0003bc0 2023-11-28T11:21:50.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.280+0000 7f4fb2ffd700 1 --2- 172.21.15.92:0/1223345429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fa0003bc0 0x7f4fb40d2e30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f4fa400c800 tx=0x7f4fa4001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:50.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.280+0000 7f4fab7fe700 1 -- 172.21.15.92:0/1223345429 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa40072d0 con 0x7f4fa0003bc0 2023-11-28T11:21:50.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.280+0000 7f4fab7fe700 1 -- 172.21.15.92:0/1223345429 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fa4007430 con 0x7f4fa0003bc0 2023-11-28T11:21:50.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.280+0000 7f4fab7fe700 1 -- 172.21.15.92:0/1223345429 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa40077a0 con 0x7f4fa0003bc0 2023-11-28T11:21:50.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.280+0000 7f4fb95ff700 1 -- 172.21.15.92:0/1223345429 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fb40cfd10 con 0x7f4fa0003bc0 2023-11-28T11:21:50.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.281+0000 7f4fb95ff700 1 -- 172.21.15.92:0/1223345429 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fb40d29c0 con 0x7f4fa0003bc0 2023-11-28T11:21:50.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.282+0000 7f4fb95ff700 1 -- 172.21.15.92:0/1223345429 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fb40d3cc0 con 0x7f4fa0003bc0 2023-11-28T11:21:50.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.282+0000 7f4fab7fe700 1 -- 172.21.15.92:0/1223345429 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4fa402e070 con 0x7f4fa0003bc0 2023-11-28T11:21:50.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.284+0000 7f4fab7fe700 1 --2- 172.21.15.92:0/1223345429 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f4f940682f0 0x7f4f9406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:50.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.284+0000 7f4fab7fe700 1 -- 172.21.15.92:0/1223345429 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f4fa4087560 con 0x7f4fa0003bc0 2023-11-28T11:21:50.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.284+0000 7f4fabfff700 1 --2- 172.21.15.92:0/1223345429 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f4f940682f0 0x7f4f9406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:50.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.284+0000 7f4fabfff700 1 --2- 172.21.15.92:0/1223345429 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f4f940682f0 0x7f4f9406a7a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4154a70 tx=0x7f4f9c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:50.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.288+0000 7f4fab7fe700 1 -- 172.21.15.92:0/1223345429 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4fb40d3cc0 con 0x7f4fa0003bc0 2023-11-28T11:21:50.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.476+0000 7f4fb95ff700 1 -- 172.21.15.92:0/1223345429 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4fb40d3cc0 con 0x7f4fa0003bc0 2023-11-28T11:21:50.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.476+0000 7f4fab7fe700 1 -- 172.21.15.92:0/1223345429 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4fb40d3cc0 con 0x7f4fa0003bc0 2023-11-28T11:21:50.478 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-11-28T11:21:50.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.478+0000 7f4fb95ff700 1 -- 172.21.15.92:0/1223345429 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f4f940682f0 msgr2=0x7f4f9406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:50.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.478+0000 7f4fb95ff700 1 --2- 172.21.15.92:0/1223345429 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f4f940682f0 0x7f4f9406a7a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4154a70 tx=0x7f4f9c009040 comp rx=0 tx=0).stop 2023-11-28T11:21:50.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.478+0000 7f4fb95ff700 1 -- 172.21.15.92:0/1223345429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fa0003bc0 msgr2=0x7f4fb40d2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:50.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.478+0000 7f4fb95ff700 1 --2- 172.21.15.92:0/1223345429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fa0003bc0 0x7f4fb40d2e30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f4fa400c800 tx=0x7f4fa4001980 comp rx=0 tx=0).stop 2023-11-28T11:21:50.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.479+0000 7f4fb95ff700 1 -- 172.21.15.92:0/1223345429 shutdown_connections 2023-11-28T11:21:50.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.479+0000 7f4fb95ff700 1 --2- 172.21.15.92:0/1223345429 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4fb406b970 0x7f4fb406bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:50.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.479+0000 7f4fb95ff700 1 --2- 172.21.15.92:0/1223345429 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f4f940682f0 0x7f4f9406a7a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:50.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.479+0000 7f4fb95ff700 1 --2- 172.21.15.92:0/1223345429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fa0003bc0 0x7f4fb40d2e30 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:50.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.479+0000 7f4fb95ff700 1 --2- 172.21.15.92:0/1223345429 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4fb41e50e0 0x7f4fb41e5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:50.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.479+0000 7f4fb95ff700 1 -- 172.21.15.92:0/1223345429 >> 172.21.15.92:0/1223345429 conn(0x7f4fb40719a0 msgr2=0x7f4fb415d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:50.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.479+0000 7f4fb95ff700 1 -- 172.21.15.92:0/1223345429 shutdown_connections 2023-11-28T11:21:50.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.479+0000 7f4fb95ff700 1 -- 172.21.15.92:0/1223345429 wait complete. 2023-11-28T11:21:50.498 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T11:21:50.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.695+0000 7fe51d7ff700 1 Processor -- start 2023-11-28T11:21:50.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.696+0000 7fe51d7ff700 1 -- start start 2023-11-28T11:21:50.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.697+0000 7fe51d7ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5180cd460 0x7fe51806c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:50.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.697+0000 7fe51d7ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5180b88e0 con 0x7fe5180b8510 2023-11-28T11:21:50.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.697+0000 7fe51d7ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5180cd8f0 con 0x7fe5180d0d60 2023-11-28T11:21:50.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.697+0000 7fe51d7ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5180cdbd0 con 0x7fe5180cd460 2023-11-28T11:21:50.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.697+0000 7fe516ffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe5180b8510 0x7fe51806b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55578/0 (socket says 172.21.15.92:55578) 2023-11-28T11:21:50.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.697+0000 7fe516ffd700 1 -- 172.21.15.92:0/3128001817 learned_addr learned my addr 172.21.15.92:0/3128001817 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:50.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.697+0000 7fe5167fc700 1 --2- 172.21.15.92:0/3128001817 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5180cd460 0x7fe51806c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:50.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.698+0000 7fe5167fc700 1 -- 172.21.15.92:0/3128001817 >> v1:172.21.15.169:6789/0 conn(0x7fe5180d0d60 legacy=0x7fe5180d11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:50.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.698+0000 7fe5167fc700 1 -- 172.21.15.92:0/3128001817 >> v1:172.21.15.92:6789/0 conn(0x7fe5180b8510 legacy=0x7fe51806b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:50.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.698+0000 7fe5167fc700 1 -- 172.21.15.92:0/3128001817 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5180cfd10 con 0x7fe5180cd460 2023-11-28T11:21:50.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.698+0000 7fe5167fc700 1 --2- 172.21.15.92:0/3128001817 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5180cd460 0x7fe51806c080 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe50c009870 tx=0x7fe50c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d3d8eeb73ef38d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:50.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.701+0000 7fe515ffb700 1 -- 172.21.15.92:0/3128001817 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50c011830 con 0x7fe5180cd460 2023-11-28T11:21:50.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.701+0000 7fe515ffb700 1 -- 172.21.15.92:0/3128001817 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe50c011990 con 0x7fe5180cd460 2023-11-28T11:21:50.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.701+0000 7fe515ffb700 1 -- 172.21.15.92:0/3128001817 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50c011d20 con 0x7fe5180cd460 2023-11-28T11:21:50.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.701+0000 7fe51d7ff700 1 -- 172.21.15.92:0/3128001817 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5180cd460 msgr2=0x7fe51806c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:50.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.701+0000 7fe51d7ff700 1 --2- 172.21.15.92:0/3128001817 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5180cd460 0x7fe51806c080 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe50c009870 tx=0x7fe50c009b80 comp rx=0 tx=0).stop 2023-11-28T11:21:50.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.701+0000 7fe51d7ff700 1 -- 172.21.15.92:0/3128001817 shutdown_connections 2023-11-28T11:21:50.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.701+0000 7fe51d7ff700 1 --2- 172.21.15.92:0/3128001817 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5180cd460 0x7fe51806c080 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:50.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.701+0000 7fe51d7ff700 1 -- 172.21.15.92:0/3128001817 >> 172.21.15.92:0/3128001817 conn(0x7fe5180719a0 msgr2=0x7fe518071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:50.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.701+0000 7fe51d7ff700 1 -- 172.21.15.92:0/3128001817 shutdown_connections 2023-11-28T11:21:50.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.701+0000 7fe51d7ff700 1 -- 172.21.15.92:0/3128001817 wait complete. 2023-11-28T11:21:50.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.703+0000 7fe51d7ff700 1 Processor -- start 2023-11-28T11:21:50.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.703+0000 7fe51d7ff700 1 -- start start 2023-11-28T11:21:50.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.704+0000 7fe51d7ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5180b8510 0x7fe5180c8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:50.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.704+0000 7fe51d7ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5180cd460 0x7fe5180cb150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:50.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.704+0000 7fe51d7ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5180d0d60 0x7fe5181534f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:50.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.704+0000 7fe51d7ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5180cbb30 con 0x7fe5180d0d60 2023-11-28T11:21:50.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.704+0000 7fe51d7ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5180c9ee0 con 0x7fe5180b8510 2023-11-28T11:21:50.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.704+0000 7fe51d7ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5180ca1c0 con 0x7fe5180cd460 2023-11-28T11:21:50.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.705+0000 7fe5177fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5180d0d60 0x7fe5181534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:50.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.705+0000 7fe5177fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5180d0d60 0x7fe5181534f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54048/0 (socket says 172.21.15.92:54048) 2023-11-28T11:21:50.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.705+0000 7fe5177fe700 1 -- 172.21.15.92:0/2635611909 learned_addr learned my addr 172.21.15.92:0/2635611909 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:50.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.705+0000 7fe516ffd700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5180b8510 0x7fe5180c8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:50.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.705+0000 7fe5167fc700 1 --2- 172.21.15.92:0/2635611909 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5180cd460 0x7fe5180cb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:50.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.705+0000 7fe5177fe700 1 -- 172.21.15.92:0/2635611909 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5180cd460 msgr2=0x7fe5180cb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:50.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.705+0000 7fe5177fe700 1 --2- 172.21.15.92:0/2635611909 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5180cd460 0x7fe5180cb150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:50.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.705+0000 7fe5177fe700 1 -- 172.21.15.92:0/2635611909 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5180b8510 msgr2=0x7fe5180c8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:50.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.705+0000 7fe5177fe700 1 --2- 172.21.15.92:0/2635611909 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5180b8510 0x7fe5180c8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:50.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.705+0000 7fe5177fe700 1 -- 172.21.15.92:0/2635611909 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe50c0094a0 con 0x7fe5180d0d60 2023-11-28T11:21:50.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.705+0000 7fe5177fe700 1 --2- 172.21.15.92:0/2635611909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5180d0d60 0x7fe5181534f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe51000cb50 tx=0x7fe51000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:50.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.705+0000 7fe4ff7fe700 1 -- 172.21.15.92:0/2635611909 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe510003c00 con 0x7fe5180d0d60 2023-11-28T11:21:50.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.706+0000 7fe4ff7fe700 1 -- 172.21.15.92:0/2635611909 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe510003d60 con 0x7fe5180d0d60 2023-11-28T11:21:50.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.706+0000 7fe4ff7fe700 1 -- 172.21.15.92:0/2635611909 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51000aa20 con 0x7fe5180d0d60 2023-11-28T11:21:50.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.706+0000 7fe51d7ff700 1 -- 172.21.15.92:0/2635611909 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5180d29c0 con 0x7fe5180d0d60 2023-11-28T11:21:50.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.706+0000 7fe51d7ff700 1 -- 172.21.15.92:0/2635611909 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5180cfff0 con 0x7fe5180d0d60 2023-11-28T11:21:50.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.707+0000 7fe51d7ff700 1 -- 172.21.15.92:0/2635611909 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5180d2b50 con 0x7fe5180d0d60 2023-11-28T11:21:50.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.707+0000 7fe4ff7fe700 1 -- 172.21.15.92:0/2635611909 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe510010050 con 0x7fe5180d0d60 2023-11-28T11:21:50.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.708+0000 7fe4ff7fe700 1 --2- 172.21.15.92:0/2635611909 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe5040682f0 0x7fe50406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:50.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.708+0000 7fe4ff7fe700 1 -- 172.21.15.92:0/2635611909 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fe510085f40 con 0x7fe5180d0d60 2023-11-28T11:21:50.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.709+0000 7fe516ffd700 1 --2- 172.21.15.92:0/2635611909 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe5040682f0 0x7fe50406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:50.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.709+0000 7fe516ffd700 1 --2- 172.21.15.92:0/2635611909 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe5040682f0 0x7fe50406a7a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe508005910 tx=0x7fe508009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:50.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.712+0000 7fe4ff7fe700 1 -- 172.21.15.92:0/2635611909 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe510052380 con 0x7fe5180d0d60 2023-11-28T11:21:50.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.902+0000 7fe51d7ff700 1 -- 172.21.15.92:0/2635611909 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe518067880 con 0x7fe5180d0d60 2023-11-28T11:21:50.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.903+0000 7fe4ff7fe700 1 -- 172.21.15.92:0/2635611909 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe5100559a0 con 0x7fe5180d0d60 2023-11-28T11:21:50.905 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-11-28T11:21:50.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.905+0000 7fe51d7ff700 1 -- 172.21.15.92:0/2635611909 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe5040682f0 msgr2=0x7fe50406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:50.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.905+0000 7fe51d7ff700 1 --2- 172.21.15.92:0/2635611909 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe5040682f0 0x7fe50406a7a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe508005910 tx=0x7fe508009040 comp rx=0 tx=0).stop 2023-11-28T11:21:50.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.905+0000 7fe51d7ff700 1 -- 172.21.15.92:0/2635611909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5180d0d60 msgr2=0x7fe5181534f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:50.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.905+0000 7fe51d7ff700 1 --2- 172.21.15.92:0/2635611909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5180d0d60 0x7fe5181534f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe51000cb50 tx=0x7fe51000ce60 comp rx=0 tx=0).stop 2023-11-28T11:21:50.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.905+0000 7fe51d7ff700 1 -- 172.21.15.92:0/2635611909 shutdown_connections 2023-11-28T11:21:50.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.905+0000 7fe51d7ff700 1 --2- 172.21.15.92:0/2635611909 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe5040682f0 0x7fe50406a7a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:50.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.905+0000 7fe51d7ff700 1 --2- 172.21.15.92:0/2635611909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5180d0d60 0x7fe5181534f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:50.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.905+0000 7fe51d7ff700 1 --2- 172.21.15.92:0/2635611909 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5180b8510 0x7fe5180c8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:50.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.905+0000 7fe51d7ff700 1 --2- 172.21.15.92:0/2635611909 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5180cd460 0x7fe5180cb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:50.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.905+0000 7fe51d7ff700 1 -- 172.21.15.92:0/2635611909 >> 172.21.15.92:0/2635611909 conn(0x7fe5180719a0 msgr2=0x7fe5180b2b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:50.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.905+0000 7fe51d7ff700 1 -- 172.21.15.92:0/2635611909 shutdown_connections 2023-11-28T11:21:50.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:50.905+0000 7fe51d7ff700 1 -- 172.21.15.92:0/2635611909 wait complete. 2023-11-28T11:21:50.933 INFO:tasks.cephfs_test_runner:Starting test: test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T11:21:50.933 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2023-11-28T11:21:51.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.128+0000 7f9d131ff700 1 Processor -- start 2023-11-28T11:21:51.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.128+0000 7f9d131ff700 1 -- start start 2023-11-28T11:21:51.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.129+0000 7f9d131ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d0c0d2df0 0x7f9d0c0cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:51.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.129+0000 7f9d131ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9d0c0b4fc0 0x7f9d0c0d31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:51.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.129+0000 7f9d131ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d0c0d1950 con 0x7f9d0c0b1c60 2023-11-28T11:21:51.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.129+0000 7f9d131ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d0c0d3d30 con 0x7f9d0c0b4fc0 2023-11-28T11:21:51.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.129+0000 7f9d131ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c0b2500 con 0x7f9d0c0d2df0 2023-11-28T11:21:51.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.130+0000 7f9d10f9b700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d0c0d2df0 0x7f9d0c0cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:51.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.130+0000 7f9d10f9b700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d0c0d2df0 0x7f9d0c0cd620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.92:48336/0 (socket says 172.21.15.92:48336) 2023-11-28T11:21:51.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.130+0000 7f9d10f9b700 1 -- 172.21.15.92:0/835445728 learned_addr learned my addr 172.21.15.92:0/835445728 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:51.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.130+0000 7f9d0bfff700 1 --2- 172.21.15.92:0/835445728 >> v2:172.21.15.169:3300/0 conn(0x7f9d0c0b4fc0 0x7f9d0c0d31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:51.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.130+0000 7f9d10f9b700 1 -- 172.21.15.92:0/835445728 >> v2:172.21.15.169:3300/0 conn(0x7f9d0c0b4fc0 msgr2=0x7f9d0c0d31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:51.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.130+0000 7f9d10f9b700 1 --2- 172.21.15.92:0/835445728 >> v2:172.21.15.169:3300/0 conn(0x7f9d0c0b4fc0 0x7f9d0c0d31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:51.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.130+0000 7f9d10f9b700 1 -- 172.21.15.92:0/835445728 >> v1:172.21.15.169:6789/0 conn(0x7f9d0c0b1c60 legacy=0x7f9d0c0cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:51.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.130+0000 7f9d10f9b700 1 -- 172.21.15.92:0/835445728 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d0c0d3a50 con 0x7f9d0c0d2df0 2023-11-28T11:21:51.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.131+0000 7f9d10f9b700 1 --2- 172.21.15.92:0/835445728 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d0c0d2df0 0x7f9d0c0cd620 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9d000098f0 tx=0x7f9d00009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6607c48877d259dc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:51.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.131+0000 7f9d0b7fe700 1 -- 172.21.15.92:0/835445728 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d00011830 con 0x7f9d0c0d2df0 2023-11-28T11:21:51.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.131+0000 7f9d0b7fe700 1 -- 172.21.15.92:0/835445728 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d00011990 con 0x7f9d0c0d2df0 2023-11-28T11:21:51.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.131+0000 7f9d0b7fe700 1 -- 172.21.15.92:0/835445728 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d00011d20 con 0x7f9d0c0d2df0 2023-11-28T11:21:51.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.131+0000 7f9d131ff700 1 -- 172.21.15.92:0/835445728 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d0c0d2df0 msgr2=0x7f9d0c0cd620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:51.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.131+0000 7f9d131ff700 1 --2- 172.21.15.92:0/835445728 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d0c0d2df0 0x7f9d0c0cd620 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9d000098f0 tx=0x7f9d00009c00 comp rx=0 tx=0).stop 2023-11-28T11:21:51.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.132+0000 7f9d131ff700 1 -- 172.21.15.92:0/835445728 shutdown_connections 2023-11-28T11:21:51.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.132+0000 7f9d131ff700 1 --2- 172.21.15.92:0/835445728 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d0c0d2df0 0x7f9d0c0cd620 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:51.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.132+0000 7f9d131ff700 1 --2- 172.21.15.92:0/835445728 >> v2:172.21.15.169:3300/0 conn(0x7f9d0c0b4fc0 0x7f9d0c0d31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:51.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.132+0000 7f9d131ff700 1 -- 172.21.15.92:0/835445728 >> 172.21.15.92:0/835445728 conn(0x7f9d0c0719a0 msgr2=0x7f9d0c071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:51.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.132+0000 7f9d131ff700 1 -- 172.21.15.92:0/835445728 shutdown_connections 2023-11-28T11:21:51.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.132+0000 7f9d131ff700 1 -- 172.21.15.92:0/835445728 wait complete. 2023-11-28T11:21:51.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.133+0000 7f9d131ff700 1 Processor -- start 2023-11-28T11:21:51.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.133+0000 7f9d131ff700 1 -- start start 2023-11-28T11:21:51.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.134+0000 7f9d131ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d0c0b1c60 0x7f9d0c1563c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:51.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.134+0000 7f9d131ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d0c0b4fc0 0x7f9d0c0cb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:51.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.134+0000 7f9d131ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d0c0d2df0 0x7f9d0c15adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:51.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.134+0000 7f9d131ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c0cc110 con 0x7f9d0c0b4fc0 2023-11-28T11:21:51.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.134+0000 7f9d131ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c0cc3f0 con 0x7f9d0c0d2df0 2023-11-28T11:21:51.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.134+0000 7f9d131ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c0cc6d0 con 0x7f9d0c0b1c60 2023-11-28T11:21:51.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.135+0000 7f9d0bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d0c0b4fc0 0x7f9d0c0cb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:51.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.135+0000 7f9d0bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d0c0b4fc0 0x7f9d0c0cb6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54064/0 (socket says 172.21.15.92:54064) 2023-11-28T11:21:51.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.135+0000 7f9d0bfff700 1 -- 172.21.15.92:0/2464933829 learned_addr learned my addr 172.21.15.92:0/2464933829 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:51.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.135+0000 7f9d10f9b700 1 --2- 172.21.15.92:0/2464933829 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d0c0b1c60 0x7f9d0c1563c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:51.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.135+0000 7f9d1179c700 1 --2- 172.21.15.92:0/2464933829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d0c0d2df0 0x7f9d0c15adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:51.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.135+0000 7f9d0bfff700 1 -- 172.21.15.92:0/2464933829 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d0c0b1c60 msgr2=0x7f9d0c1563c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:51.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.135+0000 7f9d0bfff700 1 --2- 172.21.15.92:0/2464933829 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d0c0b1c60 0x7f9d0c1563c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:51.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.135+0000 7f9d0bfff700 1 -- 172.21.15.92:0/2464933829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d0c0d2df0 msgr2=0x7f9d0c15adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:51.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.135+0000 7f9d0bfff700 1 --2- 172.21.15.92:0/2464933829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d0c0d2df0 0x7f9d0c15adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:51.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.135+0000 7f9d0bfff700 1 -- 172.21.15.92:0/2464933829 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d00009550 con 0x7f9d0c0b4fc0 2023-11-28T11:21:51.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.135+0000 7f9d1179c700 1 --2- 172.21.15.92:0/2464933829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d0c0d2df0 0x7f9d0c15adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:21:51.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.135+0000 7f9d0bfff700 1 --2- 172.21.15.92:0/2464933829 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d0c0b4fc0 0x7f9d0c0cb6c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9cf800e840 tx=0x7f9cf800eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:51.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.135+0000 7f9d097fa700 1 -- 172.21.15.92:0/2464933829 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf8009590 con 0x7f9d0c0b4fc0 2023-11-28T11:21:51.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.136+0000 7f9d097fa700 1 -- 172.21.15.92:0/2464933829 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cf8013680 con 0x7f9d0c0b4fc0 2023-11-28T11:21:51.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.136+0000 7f9d097fa700 1 -- 172.21.15.92:0/2464933829 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf8013a10 con 0x7f9d0c0b4fc0 2023-11-28T11:21:51.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.136+0000 7f9d131ff700 1 -- 172.21.15.92:0/2464933829 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d0c0b88f0 con 0x7f9d0c0b4fc0 2023-11-28T11:21:51.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.136+0000 7f9d131ff700 1 -- 172.21.15.92:0/2464933829 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d0c0d15e0 con 0x7f9d0c0b4fc0 2023-11-28T11:21:51.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.137+0000 7f9d131ff700 1 -- 172.21.15.92:0/2464933829 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d0c0b21b0 con 0x7f9d0c0b4fc0 2023-11-28T11:21:51.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.138+0000 7f9d097fa700 1 -- 172.21.15.92:0/2464933829 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9cf8019030 con 0x7f9d0c0b4fc0 2023-11-28T11:21:51.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.139+0000 7f9d097fa700 1 --2- 172.21.15.92:0/2464933829 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9cf00682f0 0x7f9cf006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:51.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.139+0000 7f9d097fa700 1 -- 172.21.15.92:0/2464933829 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f9cf8085cf0 con 0x7f9d0c0b4fc0 2023-11-28T11:21:51.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.139+0000 7f9d10f9b700 1 --2- 172.21.15.92:0/2464933829 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9cf00682f0 0x7f9cf006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:51.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.140+0000 7f9d10f9b700 1 --2- 172.21.15.92:0/2464933829 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9cf00682f0 0x7f9cf006a7a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9d00000c00 tx=0x7f9d00008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:51.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.143+0000 7f9d097fa700 1 -- 172.21.15.92:0/2464933829 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9cf8052220 con 0x7f9d0c0b4fc0 2023-11-28T11:21:51.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.410+0000 7f9d131ff700 1 -- 172.21.15.92:0/2464933829 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) v1 -- 0x7f9d0c067170 con 0x7f9d0c0b4fc0 2023-11-28T11:21:51.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.706+0000 7f9d097fa700 1 -- 172.21.15.92:0/2464933829 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v74) v1 ==== 145+0+0 (secure 0 0 0) 0x7f9cf8055840 con 0x7f9d0c0b4fc0 2023-11-28T11:21:51.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.708+0000 7f9d131ff700 1 -- 172.21.15.92:0/2464933829 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9cf00682f0 msgr2=0x7f9cf006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:51.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.708+0000 7f9d131ff700 1 --2- 172.21.15.92:0/2464933829 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9cf00682f0 0x7f9cf006a7a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9d00000c00 tx=0x7f9d00008030 comp rx=0 tx=0).stop 2023-11-28T11:21:51.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.708+0000 7f9d131ff700 1 -- 172.21.15.92:0/2464933829 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d0c0b4fc0 msgr2=0x7f9d0c0cb6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:51.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.708+0000 7f9d131ff700 1 --2- 172.21.15.92:0/2464933829 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d0c0b4fc0 0x7f9d0c0cb6c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9cf800e840 tx=0x7f9cf800eb50 comp rx=0 tx=0).stop 2023-11-28T11:21:51.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.708+0000 7f9d131ff700 1 -- 172.21.15.92:0/2464933829 shutdown_connections 2023-11-28T11:21:51.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.708+0000 7f9d131ff700 1 --2- 172.21.15.92:0/2464933829 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d0c0d2df0 0x7f9d0c15adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:51.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.708+0000 7f9d131ff700 1 --2- 172.21.15.92:0/2464933829 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d0c0b1c60 0x7f9d0c1563c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:51.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.708+0000 7f9d131ff700 1 --2- 172.21.15.92:0/2464933829 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9cf00682f0 0x7f9cf006a7a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:51.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.708+0000 7f9d131ff700 1 --2- 172.21.15.92:0/2464933829 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d0c0b4fc0 0x7f9d0c0cb6c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:51.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.708+0000 7f9d131ff700 1 -- 172.21.15.92:0/2464933829 >> 172.21.15.92:0/2464933829 conn(0x7f9d0c0719a0 msgr2=0x7f9d0c0b31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:51.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.709+0000 7f9d131ff700 1 -- 172.21.15.92:0/2464933829 shutdown_connections 2023-11-28T11:21:51.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.709+0000 7f9d131ff700 1 -- 172.21.15.92:0/2464933829 wait complete. 2023-11-28T11:21:51.730 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-28T11:21:51.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.929+0000 7faa785ff700 1 Processor -- start 2023-11-28T11:21:51.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.929+0000 7faa785ff700 1 -- start start 2023-11-28T11:21:51.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.930+0000 7faa785ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa700d2df0 0x7faa700cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:51.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.930+0000 7faa785ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7faa700b1c60 0x7faa700cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:51.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.930+0000 7faa785ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa700d1950 con 0x7faa700b4fc0 2023-11-28T11:21:51.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.930+0000 7faa785ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa700d3d30 con 0x7faa700b1c60 2023-11-28T11:21:51.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.930+0000 7faa785ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa700b2500 con 0x7faa700d2df0 2023-11-28T11:21:51.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.931+0000 7faa7639b700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa700d2df0 0x7faa700cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:51.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.931+0000 7faa75b9a700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7faa700b4fc0 0x7faa700d31c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.92:38716/0 (socket says 172.21.15.92:38716) 2023-11-28T11:21:51.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.931+0000 7faa75b9a700 1 -- 172.21.15.92:0/3142642761 learned_addr learned my addr 172.21.15.92:0/3142642761 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:51.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.931+0000 7faa76b9c700 1 --2- 172.21.15.92:0/3142642761 >> v2:172.21.15.169:3300/0 conn(0x7faa700b1c60 0x7faa700cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:51.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.931+0000 7faa75399700 1 -- 172.21.15.92:0/3142642761 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3816415720 0 0) 0x7faa700d1950 con 0x7faa700b4fc0 2023-11-28T11:21:51.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.932+0000 7faa75399700 1 -- 172.21.15.92:0/3142642761 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa5c0035b0 con 0x7faa700b4fc0 2023-11-28T11:21:51.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.932+0000 7faa7639b700 1 -- 172.21.15.92:0/3142642761 >> v2:172.21.15.169:3300/0 conn(0x7faa700b1c60 msgr2=0x7faa700cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:51.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.932+0000 7faa7639b700 1 --2- 172.21.15.92:0/3142642761 >> v2:172.21.15.169:3300/0 conn(0x7faa700b1c60 0x7faa700cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:51.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.932+0000 7faa7639b700 1 -- 172.21.15.92:0/3142642761 >> v1:172.21.15.169:6789/0 conn(0x7faa700b4fc0 legacy=0x7faa700d31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:51.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.932+0000 7faa7639b700 1 -- 172.21.15.92:0/3142642761 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa700d3a50 con 0x7faa700d2df0 2023-11-28T11:21:51.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.932+0000 7faa76b9c700 1 --2- 172.21.15.92:0/3142642761 >> v2:172.21.15.169:3300/0 conn(0x7faa700b1c60 0x7faa700cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:21:51.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.932+0000 7faa7639b700 1 --2- 172.21.15.92:0/3142642761 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa700d2df0 0x7faa700cd620 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7faa6c009840 tx=0x7faa6c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78302a60e39da691 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:51.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.932+0000 7faa75399700 1 -- 172.21.15.92:0/3142642761 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa6c011830 con 0x7faa700d2df0 2023-11-28T11:21:51.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.932+0000 7faa75399700 1 -- 172.21.15.92:0/3142642761 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa6c011990 con 0x7faa700d2df0 2023-11-28T11:21:51.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.932+0000 7faa75399700 1 -- 172.21.15.92:0/3142642761 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa6c011d20 con 0x7faa700d2df0 2023-11-28T11:21:51.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.933+0000 7faa785ff700 1 -- 172.21.15.92:0/3142642761 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa700d2df0 msgr2=0x7faa700cd620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:51.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.933+0000 7faa785ff700 1 --2- 172.21.15.92:0/3142642761 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa700d2df0 0x7faa700cd620 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7faa6c009840 tx=0x7faa6c009b50 comp rx=0 tx=0).stop 2023-11-28T11:21:51.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.933+0000 7faa785ff700 1 -- 172.21.15.92:0/3142642761 shutdown_connections 2023-11-28T11:21:51.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.933+0000 7faa785ff700 1 --2- 172.21.15.92:0/3142642761 >> v2:172.21.15.169:3300/0 conn(0x7faa700b1c60 0x7faa700cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:51.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.933+0000 7faa785ff700 1 --2- 172.21.15.92:0/3142642761 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa700d2df0 0x7faa700cd620 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:51.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.933+0000 7faa785ff700 1 -- 172.21.15.92:0/3142642761 >> 172.21.15.92:0/3142642761 conn(0x7faa700719a0 msgr2=0x7faa70071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:51.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.933+0000 7faa785ff700 1 -- 172.21.15.92:0/3142642761 shutdown_connections 2023-11-28T11:21:51.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.933+0000 7faa785ff700 1 -- 172.21.15.92:0/3142642761 wait complete. 2023-11-28T11:21:51.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.935+0000 7faa785ff700 1 Processor -- start 2023-11-28T11:21:51.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.936+0000 7faa785ff700 1 -- start start 2023-11-28T11:21:51.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.937+0000 7faa785ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa700b1c60 0x7faa701584e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:51.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.937+0000 7faa785ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa700b4fc0 0x7faa7015f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:51.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.937+0000 7faa785ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa700d2df0 0x7faa701f2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:51.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.937+0000 7faa785ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa700ccec0 con 0x7faa700b1c60 2023-11-28T11:21:51.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.937+0000 7faa785ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7015ef30 con 0x7faa700d2df0 2023-11-28T11:21:51.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.937+0000 7faa785ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa700ccbe0 con 0x7faa700b4fc0 2023-11-28T11:21:51.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.937+0000 7faa7639b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa700b1c60 0x7faa701584e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:51.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.937+0000 7faa7639b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa700b1c60 0x7faa701584e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54072/0 (socket says 172.21.15.92:54072) 2023-11-28T11:21:51.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.937+0000 7faa7639b700 1 -- 172.21.15.92:0/2061183624 learned_addr learned my addr 172.21.15.92:0/2061183624 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:51.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.937+0000 7faa75b9a700 1 --2- 172.21.15.92:0/2061183624 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa700b4fc0 0x7faa7015f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:51.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.937+0000 7faa76b9c700 1 --2- 172.21.15.92:0/2061183624 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa700d2df0 0x7faa701f2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:51.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.937+0000 7faa7639b700 1 -- 172.21.15.92:0/2061183624 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa700b4fc0 msgr2=0x7faa7015f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:51.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.937+0000 7faa7639b700 1 --2- 172.21.15.92:0/2061183624 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa700b4fc0 0x7faa7015f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:51.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.938+0000 7faa7639b700 1 -- 172.21.15.92:0/2061183624 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa700d2df0 msgr2=0x7faa701f2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:51.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.938+0000 7faa7639b700 1 --2- 172.21.15.92:0/2061183624 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa700d2df0 0x7faa701f2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:51.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.938+0000 7faa7639b700 1 -- 172.21.15.92:0/2061183624 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa68009a70 con 0x7faa700b1c60 2023-11-28T11:21:51.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.938+0000 7faa7639b700 1 --2- 172.21.15.92:0/2061183624 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa700b1c60 0x7faa701584e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7faa6c0090b0 tx=0x7faa6c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:51.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.938+0000 7faa66ffd700 1 -- 172.21.15.92:0/2061183624 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa6c0036a0 con 0x7faa700b1c60 2023-11-28T11:21:51.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.938+0000 7faa66ffd700 1 -- 172.21.15.92:0/2061183624 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa6c01f070 con 0x7faa700b1c60 2023-11-28T11:21:51.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.938+0000 7faa66ffd700 1 -- 172.21.15.92:0/2061183624 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa6c003a00 con 0x7faa700b1c60 2023-11-28T11:21:51.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.938+0000 7faa785ff700 1 -- 172.21.15.92:0/2061183624 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa6c0094a0 con 0x7faa700b1c60 2023-11-28T11:21:51.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.938+0000 7faa785ff700 1 -- 172.21.15.92:0/2061183624 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa700b88f0 con 0x7faa700b1c60 2023-11-28T11:21:51.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.939+0000 7faa785ff700 1 -- 172.21.15.92:0/2061183624 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa700d15e0 con 0x7faa700b1c60 2023-11-28T11:21:51.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.940+0000 7faa66ffd700 1 -- 172.21.15.92:0/2061183624 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7faa6c00f050 con 0x7faa700b1c60 2023-11-28T11:21:51.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.941+0000 7faa66ffd700 1 --2- 172.21.15.92:0/2061183624 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7faa540682f0 0x7faa5406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:51.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.941+0000 7faa66ffd700 1 -- 172.21.15.92:0/2061183624 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7faa6c021070 con 0x7faa700b1c60 2023-11-28T11:21:51.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.941+0000 7faa75b9a700 1 --2- 172.21.15.92:0/2061183624 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7faa540682f0 0x7faa5406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:51.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.941+0000 7faa75b9a700 1 --2- 172.21.15.92:0/2061183624 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7faa540682f0 0x7faa5406a7a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7faa700ce250 tx=0x7faa60009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:51.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:51.945+0000 7faa66ffd700 1 -- 172.21.15.92:0/2061183624 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7faa6c053da0 con 0x7faa700b1c60 2023-11-28T11:21:52.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.137+0000 7faa785ff700 1 -- 172.21.15.92:0/2061183624 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7faa700b21b0 con 0x7faa700b1c60 2023-11-28T11:21:52.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.141+0000 7faa66ffd700 1 -- 172.21.15.92:0/2061183624 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7faa6c0573c0 con 0x7faa700b1c60 2023-11-28T11:21:52.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.144+0000 7faa785ff700 1 -- 172.21.15.92:0/2061183624 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7faa540682f0 msgr2=0x7faa5406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:52.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.144+0000 7faa785ff700 1 --2- 172.21.15.92:0/2061183624 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7faa540682f0 0x7faa5406a7a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7faa700ce250 tx=0x7faa60009040 comp rx=0 tx=0).stop 2023-11-28T11:21:52.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.144+0000 7faa785ff700 1 -- 172.21.15.92:0/2061183624 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa700b1c60 msgr2=0x7faa701584e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:52.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.144+0000 7faa785ff700 1 --2- 172.21.15.92:0/2061183624 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa700b1c60 0x7faa701584e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7faa6c0090b0 tx=0x7faa6c004750 comp rx=0 tx=0).stop 2023-11-28T11:21:52.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.144+0000 7faa785ff700 1 -- 172.21.15.92:0/2061183624 shutdown_connections 2023-11-28T11:21:52.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.144+0000 7faa785ff700 1 --2- 172.21.15.92:0/2061183624 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faa700d2df0 0x7faa701f2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:52.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.144+0000 7faa785ff700 1 --2- 172.21.15.92:0/2061183624 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7faa540682f0 0x7faa5406a7a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:52.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.144+0000 7faa785ff700 1 --2- 172.21.15.92:0/2061183624 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa700b1c60 0x7faa701584e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:52.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.144+0000 7faa785ff700 1 --2- 172.21.15.92:0/2061183624 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faa700b4fc0 0x7faa7015f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:52.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.144+0000 7faa785ff700 1 -- 172.21.15.92:0/2061183624 >> 172.21.15.92:0/2061183624 conn(0x7faa700719a0 msgr2=0x7faa700b2670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:52.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.144+0000 7faa785ff700 1 -- 172.21.15.92:0/2061183624 shutdown_connections 2023-11-28T11:21:52.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.144+0000 7faa785ff700 1 -- 172.21.15.92:0/2061183624 wait complete. 2023-11-28T11:21:52.164 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T11:21:52.164 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:21:52.259 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:21:52.361 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T11:21:52.362 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-28T11:21:52.362 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:21:52.409 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-28T11:21:52.409 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:21:52.494 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-28T11:21:52.495 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:21:52.582 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:21:52.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.816+0000 7fb94a1ff700 1 Processor -- start 2023-11-28T11:21:52.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.817+0000 7fb94a1ff700 1 -- start start 2023-11-28T11:21:52.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.818+0000 7fb94a1ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb9440b8510 0x7fb94406b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:52.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.818+0000 7fb94a1ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb9440cd460 0x7fb94406beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:52.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.818+0000 7fb94a1ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9440b88e0 con 0x7fb94406c3f0 2023-11-28T11:21:52.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.818+0000 7fb94a1ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9440cdbd0 con 0x7fb9440b8510 2023-11-28T11:21:52.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.818+0000 7fb94a1ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9440d02d0 con 0x7fb9440cd460 2023-11-28T11:21:52.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.818+0000 7fb9437fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb9440b8510 0x7fb94406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:52.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.818+0000 7fb943fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb94406c3f0 0x7fb94415cc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55586/0 (socket says 172.21.15.92:55586) 2023-11-28T11:21:52.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.818+0000 7fb943fff700 1 -- 172.21.15.92:0/516409072 learned_addr learned my addr 172.21.15.92:0/516409072 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:52.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.818+0000 7fb93bfff700 1 --2- 172.21.15.92:0/516409072 >> v2:172.21.15.169:3300/0 conn(0x7fb9440cd460 0x7fb94406beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:52.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.818+0000 7fb9437fe700 1 -- 172.21.15.92:0/516409072 >> v2:172.21.15.169:3300/0 conn(0x7fb9440cd460 msgr2=0x7fb94406beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:52.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.818+0000 7fb9437fe700 1 --2- 172.21.15.92:0/516409072 >> v2:172.21.15.169:3300/0 conn(0x7fb9440cd460 0x7fb94406beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:52.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.818+0000 7fb9437fe700 1 -- 172.21.15.92:0/516409072 >> v1:172.21.15.92:6789/0 conn(0x7fb94406c3f0 legacy=0x7fb94415cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:52.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.818+0000 7fb9437fe700 1 -- 172.21.15.92:0/516409072 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9440cd8f0 con 0x7fb9440b8510 2023-11-28T11:21:52.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.819+0000 7fb9437fe700 1 --2- 172.21.15.92:0/516409072 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9440b8510 0x7fb94406b970 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb934009870 tx=0x7fb934009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33ae50ca23bb70b3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:52.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.819+0000 7fb942ffd700 1 -- 172.21.15.92:0/516409072 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb934011830 con 0x7fb9440b8510 2023-11-28T11:21:52.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.819+0000 7fb942ffd700 1 -- 172.21.15.92:0/516409072 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb934011990 con 0x7fb9440b8510 2023-11-28T11:21:52.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.819+0000 7fb942ffd700 1 -- 172.21.15.92:0/516409072 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb934011d20 con 0x7fb9440b8510 2023-11-28T11:21:52.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.819+0000 7fb94a1ff700 1 -- 172.21.15.92:0/516409072 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9440b8510 msgr2=0x7fb94406b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:52.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.819+0000 7fb94a1ff700 1 --2- 172.21.15.92:0/516409072 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9440b8510 0x7fb94406b970 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb934009870 tx=0x7fb934009b80 comp rx=0 tx=0).stop 2023-11-28T11:21:52.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.820+0000 7fb94a1ff700 1 -- 172.21.15.92:0/516409072 shutdown_connections 2023-11-28T11:21:52.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.820+0000 7fb94a1ff700 1 --2- 172.21.15.92:0/516409072 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9440b8510 0x7fb94406b970 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:52.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.820+0000 7fb94a1ff700 1 --2- 172.21.15.92:0/516409072 >> v2:172.21.15.169:3300/0 conn(0x7fb9440cd460 0x7fb94406beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:52.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.820+0000 7fb94a1ff700 1 -- 172.21.15.92:0/516409072 >> 172.21.15.92:0/516409072 conn(0x7fb9440719a0 msgr2=0x7fb944071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:52.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.820+0000 7fb94a1ff700 1 -- 172.21.15.92:0/516409072 shutdown_connections 2023-11-28T11:21:52.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.820+0000 7fb94a1ff700 1 -- 172.21.15.92:0/516409072 wait complete. 2023-11-28T11:21:52.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.821+0000 7fb94a1ff700 1 Processor -- start 2023-11-28T11:21:52.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.821+0000 7fb94a1ff700 1 -- start start 2023-11-28T11:21:52.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.822+0000 7fb94a1ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb94406c3f0 0x7fb9440c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:52.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.822+0000 7fb94a1ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9440b8510 0x7fb9440cb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:52.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.822+0000 7fb94a1ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9440cd460 0x7fb9440d10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:52.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.822+0000 7fb94a1ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9440ccb40 con 0x7fb9440cd460 2023-11-28T11:21:52.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.822+0000 7fb94a1ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9440c9300 con 0x7fb94406c3f0 2023-11-28T11:21:52.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.822+0000 7fb94a1ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9440cbec0 con 0x7fb9440b8510 2023-11-28T11:21:52.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.823+0000 7fb943fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9440cd460 0x7fb9440d10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:52.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.823+0000 7fb943fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9440cd460 0x7fb9440d10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54086/0 (socket says 172.21.15.92:54086) 2023-11-28T11:21:52.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.823+0000 7fb943fff700 1 -- 172.21.15.92:0/3124527508 learned_addr learned my addr 172.21.15.92:0/3124527508 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:52.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.823+0000 7fb9437fe700 1 --2- 172.21.15.92:0/3124527508 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb94406c3f0 0x7fb9440c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:52.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.823+0000 7fb93bfff700 1 --2- 172.21.15.92:0/3124527508 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9440b8510 0x7fb9440cb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:52.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.823+0000 7fb943fff700 1 -- 172.21.15.92:0/3124527508 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9440b8510 msgr2=0x7fb9440cb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:52.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.823+0000 7fb943fff700 1 --2- 172.21.15.92:0/3124527508 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9440b8510 0x7fb9440cb5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:52.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.823+0000 7fb943fff700 1 -- 172.21.15.92:0/3124527508 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb94406c3f0 msgr2=0x7fb9440c8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:52.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.823+0000 7fb943fff700 1 --2- 172.21.15.92:0/3124527508 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb94406c3f0 0x7fb9440c8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:52.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.823+0000 7fb943fff700 1 -- 172.21.15.92:0/3124527508 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9340094a0 con 0x7fb9440cd460 2023-11-28T11:21:52.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.823+0000 7fb93bfff700 1 --2- 172.21.15.92:0/3124527508 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9440b8510 0x7fb9440cb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:21:52.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.823+0000 7fb9437fe700 1 --2- 172.21.15.92:0/3124527508 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb94406c3f0 0x7fb9440c8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:21:52.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.823+0000 7fb943fff700 1 --2- 172.21.15.92:0/3124527508 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9440cd460 0x7fb9440d10d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb93c00cbc0 tx=0x7fb93c00cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:52.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.823+0000 7fb940ff9700 1 -- 172.21.15.92:0/3124527508 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93c00ab50 con 0x7fb9440cd460 2023-11-28T11:21:52.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.824+0000 7fb940ff9700 1 -- 172.21.15.92:0/3124527508 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb93c00acb0 con 0x7fb9440cd460 2023-11-28T11:21:52.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.824+0000 7fb940ff9700 1 -- 172.21.15.92:0/3124527508 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93c012600 con 0x7fb9440cd460 2023-11-28T11:21:52.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.824+0000 7fb94a1ff700 1 -- 172.21.15.92:0/3124527508 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9440cfff0 con 0x7fb9440cd460 2023-11-28T11:21:52.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.824+0000 7fb94a1ff700 1 -- 172.21.15.92:0/3124527508 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9440cfd10 con 0x7fb9440cd460 2023-11-28T11:21:52.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.825+0000 7fb94a1ff700 1 -- 172.21.15.92:0/3124527508 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9440ccc80 con 0x7fb9440cd460 2023-11-28T11:21:52.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.825+0000 7fb940ff9700 1 -- 172.21.15.92:0/3124527508 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb93c019030 con 0x7fb9440cd460 2023-11-28T11:21:52.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.827+0000 7fb940ff9700 1 --2- 172.21.15.92:0/3124527508 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb9240682f0 0x7fb92406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:52.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.827+0000 7fb940ff9700 1 -- 172.21.15.92:0/3124527508 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fb93c085b40 con 0x7fb9440cd460 2023-11-28T11:21:52.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.827+0000 7fb9437fe700 1 --2- 172.21.15.92:0/3124527508 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb9240682f0 0x7fb92406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:52.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.828+0000 7fb9437fe700 1 --2- 172.21.15.92:0/3124527508 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb9240682f0 0x7fb92406a7a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb934000c00 tx=0x7fb934008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:52.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:52.831+0000 7fb940ff9700 1 -- 172.21.15.92:0/3124527508 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb93c04b5e0 con 0x7fb9440cd460 2023-11-28T11:21:53.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.024+0000 7fb94a1ff700 1 -- 172.21.15.92:0/3124527508 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb944067880 con 0x7fb9440cd460 2023-11-28T11:21:53.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.025+0000 7fb940ff9700 1 -- 172.21.15.92:0/3124527508 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+19844 (secure 0 0 0) 0x7fb9440ccc80 con 0x7fb9440cd460 2023-11-28T11:21:53.027 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:21:53.028 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":29,"fsid":"68956e0c-c16f-47cd-a220-f98364b0831d","created":"2023-11-28T11:20:28.697573+0000","modified":"2023-11-28T11:21:37.638578+0000","last_up_change":"2023-11-28T11:20:41.257011+0000","last_in_change":"2023-11-28T11:20:32.249767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:20:41.925237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:20:48.682497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:20:54.471398+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:20:55.204585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:20:58.297307+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6abe483f-0e03-4a26-b9cf-62041291747e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":115823},{"type":"v1","addr":"172.21.15.92:6809","nonce":115823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":115823},{"type":"v1","addr":"172.21.15.92:6813","nonce":115823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":115823},{"type":"v1","addr":"172.21.15.92:6821","nonce":115823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":115823},{"type":"v1","addr":"172.21.15.92:6817","nonce":115823}]},"public_addr":"172.21.15.92:6809/115823","cluster_addr":"172.21.15.92:6813/115823","heartbeat_back_addr":"172.21.15.92:6821/115823","heartbeat_front_addr":"172.21.15.92:6817/115823","state":["exists","up"]},{"osd":1,"uuid":"13c1de73-1a8c-4d7e-8a28-9dae52dfe089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":115824},{"type":"v1","addr":"172.21.15.92:6812","nonce":115824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":115824},{"type":"v1","addr":"172.21.15.92:6816","nonce":115824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":115824},{"type":"v1","addr":"172.21.15.92:6823","nonce":115824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":115824},{"type":"v1","addr":"172.21.15.92:6820","nonce":115824}]},"public_addr":"172.21.15.92:6812/115824","cluster_addr":"172.21.15.92:6816/115824","heartbeat_back_addr":"172.21.15.92:6823/115824","heartbeat_front_addr":"172.21.15.92:6820/115824","state":["exists","up"]},{"osd":2,"uuid":"25788a55-112f-4f00-810d-2bc37cf76778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":115821},{"type":"v1","addr":"172.21.15.92:6801","nonce":115821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":115821},{"type":"v1","addr":"172.21.15.92:6803","nonce":115821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":115821},{"type":"v1","addr":"172.21.15.92:6807","nonce":115821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":115821},{"type":"v1","addr":"172.21.15.92:6805","nonce":115821}]},"public_addr":"172.21.15.92:6801/115821","cluster_addr":"172.21.15.92:6803/115821","heartbeat_back_addr":"172.21.15.92:6807/115821","heartbeat_front_addr":"172.21.15.92:6805/115821","state":["exists","up"]},{"osd":3,"uuid":"adf54e24-33a9-41fe-9ce3-98a5ab6c158e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":115825},{"type":"v1","addr":"172.21.15.92:6825","nonce":115825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":115825},{"type":"v1","addr":"172.21.15.92:6827","nonce":115825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":115825},{"type":"v1","addr":"172.21.15.92:6831","nonce":115825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":115825},{"type":"v1","addr":"172.21.15.92:6829","nonce":115825}]},"public_addr":"172.21.15.92:6825/115825","cluster_addr":"172.21.15.92:6827/115825","heartbeat_back_addr":"172.21.15.92:6831/115825","heartbeat_front_addr":"172.21.15.92:6829/115825","state":["exists","up"]},{"osd":4,"uuid":"ccb9b747-f399-44ea-9a18-9db3152227e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":116035},{"type":"v1","addr":"172.21.15.169:6817","nonce":116035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":116035},{"type":"v1","addr":"172.21.15.169:6819","nonce":116035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":116035},{"type":"v1","addr":"172.21.15.169:6826","nonce":116035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":116035},{"type":"v1","addr":"172.21.15.169:6822","nonce":116035}]},"public_addr":"172.21.15.169:6817/116035","cluster_addr":"172.21.15.169:6819/116035","heartbeat_back_addr":"172.21.15.169:6826/116035","heartbeat_front_addr":"172.21.15.169:6822/116035","state":["exists","up"]},{"osd":5,"uuid":"308a1e63-75aa-4d7d-8a7c-cecc24714fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":116033},{"type":"v1","addr":"172.21.15.169:6801","nonce":116033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":116033},{"type":"v1","addr":"172.21.15.169:6803","nonce":116033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":116033},{"type":"v1","addr":"172.21.15.169:6807","nonce":116033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":116033},{"type":"v1","addr":"172.21.15.169:6805","nonce":116033}]},"public_addr":"172.21.15.169:6801/116033","cluster_addr":"172.21.15.169:6803/116033","heartbeat_back_addr":"172.21.15.169:6807/116033","heartbeat_front_addr":"172.21.15.169:6805/116033","state":["exists","up"]},{"osd":6,"uuid":"9c459b7b-b821-4b4b-b91e-6aaae271457c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":116034},{"type":"v1","addr":"172.21.15.169:6809","nonce":116034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":116034},{"type":"v1","addr":"172.21.15.169:6811","nonce":116034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":116034},{"type":"v1","addr":"172.21.15.169:6815","nonce":116034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":116034},{"type":"v1","addr":"172.21.15.169:6813","nonce":116034}]},"public_addr":"172.21.15.169:6809/116034","cluster_addr":"172.21.15.169:6811/116034","heartbeat_back_addr":"172.21.15.169:6815/116034","heartbeat_front_addr":"172.21.15.169:6813/116034","state":["exists","up"]},{"osd":7,"uuid":"1ca9aa4a-755d-4e28-a937-4d8dccab4df3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":116044},{"type":"v1","addr":"172.21.15.169:6823","nonce":116044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6825","nonce":116044},{"type":"v1","addr":"172.21.15.169:6827","nonce":116044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":116044},{"type":"v1","addr":"172.21.15.169:6831","nonce":116044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":116044},{"type":"v1","addr":"172.21.15.169:6829","nonce":116044}]},"public_addr":"172.21.15.169:6823/116044","cluster_addr":"172.21.15.169:6827/116044","heartbeat_back_addr":"172.21.15.169:6831/116044","heartbeat_front_addr":"172.21.15.169:6829/116044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.586781+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.869778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.508594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.860493+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.872267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.746933+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.611520+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.871446+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.1d","mappings":[{"from":3,"to":2}]},{"pgid":"4.21","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:21:53.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.027+0000 7fb94a1ff700 1 -- 172.21.15.92:0/3124527508 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb9240682f0 msgr2=0x7fb92406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:53.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.027+0000 7fb94a1ff700 1 --2- 172.21.15.92:0/3124527508 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb9240682f0 0x7fb92406a7a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb934000c00 tx=0x7fb934008030 comp rx=0 tx=0).stop 2023-11-28T11:21:53.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.027+0000 7fb94a1ff700 1 -- 172.21.15.92:0/3124527508 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9440cd460 msgr2=0x7fb9440d10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:53.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.027+0000 7fb94a1ff700 1 --2- 172.21.15.92:0/3124527508 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9440cd460 0x7fb9440d10d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb93c00cbc0 tx=0x7fb93c00cf80 comp rx=0 tx=0).stop 2023-11-28T11:21:53.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.028+0000 7fb94a1ff700 1 -- 172.21.15.92:0/3124527508 shutdown_connections 2023-11-28T11:21:53.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.028+0000 7fb94a1ff700 1 --2- 172.21.15.92:0/3124527508 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb9240682f0 0x7fb92406a7a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:53.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.028+0000 7fb94a1ff700 1 --2- 172.21.15.92:0/3124527508 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9440cd460 0x7fb9440d10d0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:53.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.028+0000 7fb94a1ff700 1 --2- 172.21.15.92:0/3124527508 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb94406c3f0 0x7fb9440c8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:53.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.028+0000 7fb94a1ff700 1 --2- 172.21.15.92:0/3124527508 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb9440b8510 0x7fb9440cb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:53.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.028+0000 7fb94a1ff700 1 -- 172.21.15.92:0/3124527508 >> 172.21.15.92:0/3124527508 conn(0x7fb9440719a0 msgr2=0x7fb9440c96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:53.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.028+0000 7fb94a1ff700 1 -- 172.21.15.92:0/3124527508 shutdown_connections 2023-11-28T11:21:53.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.028+0000 7fb94a1ff700 1 -- 172.21.15.92:0/3124527508 wait complete. 2023-11-28T11:21:53.049 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:20:41.925237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:20:48.682497+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:20:54.471398+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:20:55.204585+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T11:20:58.297307+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T11:21:53.049 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:21:53.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.245+0000 7f90a13ff700 1 Processor -- start 2023-11-28T11:21:53.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.245+0000 7f90a13ff700 1 -- start start 2023-11-28T11:21:53.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.247+0000 7f90a13ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f909c0b8510 0x7f909c06b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:53.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.247+0000 7f90a13ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f909c0d0d60 0x7f909c0d11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:53.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.247+0000 7f90a13ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f909c0b88e0 con 0x7f909c0cd460 2023-11-28T11:21:53.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.247+0000 7f90a13ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f909c0cdbd0 con 0x7f909c0d0d60 2023-11-28T11:21:53.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.247+0000 7f90a13ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f909c0d02d0 con 0x7f909c0b8510 2023-11-28T11:21:53.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.247+0000 7f9093fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f909c0cd460 0x7f909c06beb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55594/0 (socket says 172.21.15.92:55594) 2023-11-28T11:21:53.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.247+0000 7f9093fff700 1 -- 172.21.15.92:0/1656876812 learned_addr learned my addr 172.21.15.92:0/1656876812 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:53.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.247+0000 7f909affd700 1 --2- 172.21.15.92:0/1656876812 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f909c0b8510 0x7f909c06b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:53.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.247+0000 7f909b7fe700 1 --2- 172.21.15.92:0/1656876812 >> v2:172.21.15.169:3300/0 conn(0x7f909c0d0d60 0x7f909c0d11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:53.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.247+0000 7f909a7fc700 1 -- 172.21.15.92:0/1656876812 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1960817661 0 0) 0x7f909c0b88e0 con 0x7f909c0cd460 2023-11-28T11:21:53.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.247+0000 7f909a7fc700 1 -- 172.21.15.92:0/1656876812 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90880035b0 con 0x7f909c0cd460 2023-11-28T11:21:53.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.248+0000 7f909b7fe700 1 -- 172.21.15.92:0/1656876812 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f909c0b8510 msgr2=0x7f909c06b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:53.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.248+0000 7f909b7fe700 1 --2- 172.21.15.92:0/1656876812 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f909c0b8510 0x7f909c06b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:53.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.248+0000 7f909b7fe700 1 -- 172.21.15.92:0/1656876812 >> v1:172.21.15.92:6789/0 conn(0x7f909c0cd460 legacy=0x7f909c06beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:53.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.248+0000 7f909b7fe700 1 -- 172.21.15.92:0/1656876812 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f909c0cd8f0 con 0x7f909c0d0d60 2023-11-28T11:21:53.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.248+0000 7f909affd700 1 --2- 172.21.15.92:0/1656876812 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f909c0b8510 0x7f909c06b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:21:53.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.248+0000 7f909b7fe700 1 --2- 172.21.15.92:0/1656876812 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f909c0d0d60 0x7f909c0d11a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9094009dc0 tx=0x7f909400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da4d4832a64c6828 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:53.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.248+0000 7f909a7fc700 1 -- 172.21.15.92:0/1656876812 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909400e070 con 0x7f909c0d0d60 2023-11-28T11:21:53.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.248+0000 7f909a7fc700 1 -- 172.21.15.92:0/1656876812 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f909400bae0 con 0x7f909c0d0d60 2023-11-28T11:21:53.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.248+0000 7f909a7fc700 1 -- 172.21.15.92:0/1656876812 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909400be90 con 0x7f909c0d0d60 2023-11-28T11:21:53.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.249+0000 7f90a13ff700 1 -- 172.21.15.92:0/1656876812 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f909c0d0d60 msgr2=0x7f909c0d11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:53.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.249+0000 7f90a13ff700 1 --2- 172.21.15.92:0/1656876812 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f909c0d0d60 0x7f909c0d11a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9094009dc0 tx=0x7f909400b330 comp rx=0 tx=0).stop 2023-11-28T11:21:53.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.249+0000 7f90a13ff700 1 -- 172.21.15.92:0/1656876812 shutdown_connections 2023-11-28T11:21:53.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.249+0000 7f90a13ff700 1 --2- 172.21.15.92:0/1656876812 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f909c0d0d60 0x7f909c0d11a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:53.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.249+0000 7f90a13ff700 1 --2- 172.21.15.92:0/1656876812 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f909c0b8510 0x7f909c06b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:53.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.249+0000 7f90a13ff700 1 -- 172.21.15.92:0/1656876812 >> 172.21.15.92:0/1656876812 conn(0x7f909c0719a0 msgr2=0x7f909c071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:53.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.249+0000 7f90a13ff700 1 -- 172.21.15.92:0/1656876812 shutdown_connections 2023-11-28T11:21:53.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.249+0000 7f90a13ff700 1 -- 172.21.15.92:0/1656876812 wait complete. 2023-11-28T11:21:53.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.251+0000 7f90a13ff700 1 Processor -- start 2023-11-28T11:21:53.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.251+0000 7f90a13ff700 1 -- start start 2023-11-28T11:21:53.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.252+0000 7f90a13ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f909c0b8510 0x7f909c14e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:53.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.252+0000 7f90a13ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f909c0cd460 0x7f909c0d1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:53.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.252+0000 7f90a13ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f909c0d0d60 0x7f909c0d16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:53.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.252+0000 7f90a13ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f909c1f2120 con 0x7f909c0b8510 2023-11-28T11:21:53.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.252+0000 7f90a13ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f909c15b7d0 con 0x7f909c0cd460 2023-11-28T11:21:53.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.252+0000 7f90a13ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f909c1f65a0 con 0x7f909c0d0d60 2023-11-28T11:21:53.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.252+0000 7f909affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f909c0b8510 0x7f909c14e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:53.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.252+0000 7f909affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f909c0b8510 0x7f909c14e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54098/0 (socket says 172.21.15.92:54098) 2023-11-28T11:21:53.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.252+0000 7f909affd700 1 -- 172.21.15.92:0/1697081142 learned_addr learned my addr 172.21.15.92:0/1697081142 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:53.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.252+0000 7f9093fff700 1 --2- 172.21.15.92:0/1697081142 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f909c0cd460 0x7f909c0d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:53.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.252+0000 7f909b7fe700 1 --2- 172.21.15.92:0/1697081142 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f909c0d0d60 0x7f909c0d16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:53.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.252+0000 7f909affd700 1 -- 172.21.15.92:0/1697081142 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f909c0d0d60 msgr2=0x7f909c0d16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:53.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.252+0000 7f909affd700 1 --2- 172.21.15.92:0/1697081142 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f909c0d0d60 0x7f909c0d16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:53.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.252+0000 7f909affd700 1 -- 172.21.15.92:0/1697081142 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f909c0cd460 msgr2=0x7f909c0d1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:53.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.252+0000 7f909affd700 1 --2- 172.21.15.92:0/1697081142 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f909c0cd460 0x7f909c0d1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:53.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.252+0000 7f909affd700 1 -- 172.21.15.92:0/1697081142 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f908c0094a0 con 0x7f909c0b8510 2023-11-28T11:21:53.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.253+0000 7f909affd700 1 --2- 172.21.15.92:0/1697081142 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f909c0b8510 0x7f909c14e260 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f908c00ebe0 tx=0x7f908c009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:53.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.253+0000 7f90937fe700 1 -- 172.21.15.92:0/1697081142 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908c00c910 con 0x7f909c0b8510 2023-11-28T11:21:53.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.253+0000 7f90937fe700 1 -- 172.21.15.92:0/1697081142 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f908c00ca70 con 0x7f909c0b8510 2023-11-28T11:21:53.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.253+0000 7f90937fe700 1 -- 172.21.15.92:0/1697081142 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908c00cde0 con 0x7f909c0b8510 2023-11-28T11:21:53.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.253+0000 7f90a13ff700 1 -- 172.21.15.92:0/1697081142 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9094009a70 con 0x7f909c0b8510 2023-11-28T11:21:53.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.253+0000 7f90a13ff700 1 -- 172.21.15.92:0/1697081142 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f909c0cfff0 con 0x7f909c0b8510 2023-11-28T11:21:53.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.254+0000 7f90a13ff700 1 -- 172.21.15.92:0/1697081142 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f909c0cfd10 con 0x7f909c0b8510 2023-11-28T11:21:53.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.255+0000 7f90937fe700 1 -- 172.21.15.92:0/1697081142 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f908c019030 con 0x7f909c0b8510 2023-11-28T11:21:53.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.256+0000 7f90937fe700 1 --2- 172.21.15.92:0/1697081142 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f907c0682f0 0x7f907c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:53.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.256+0000 7f90937fe700 1 -- 172.21.15.92:0/1697081142 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f908c086080 con 0x7f909c0b8510 2023-11-28T11:21:53.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.256+0000 7f9093fff700 1 --2- 172.21.15.92:0/1697081142 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f907c0682f0 0x7f907c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:53.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.257+0000 7f9093fff700 1 --2- 172.21.15.92:0/1697081142 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f907c0682f0 0x7f907c06a7a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f909c0b7090 tx=0x7f9084009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:53.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.260+0000 7f90937fe700 1 -- 172.21.15.92:0/1697081142 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f908c0525b0 con 0x7f909c0b8510 2023-11-28T11:21:53.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.449+0000 7f90a13ff700 1 -- 172.21.15.92:0/1697081142 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f909c1e33a0 con 0x7f909c0b8510 2023-11-28T11:21:53.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.450+0000 7f90937fe700 1 -- 172.21.15.92:0/1697081142 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f908c0170a0 con 0x7f909c0b8510 2023-11-28T11:21:53.452 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-11-28T11:21:53.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.452+0000 7f90a13ff700 1 -- 172.21.15.92:0/1697081142 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f907c0682f0 msgr2=0x7f907c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:53.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.452+0000 7f90a13ff700 1 --2- 172.21.15.92:0/1697081142 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f907c0682f0 0x7f907c06a7a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f909c0b7090 tx=0x7f9084009040 comp rx=0 tx=0).stop 2023-11-28T11:21:53.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.452+0000 7f90a13ff700 1 -- 172.21.15.92:0/1697081142 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f909c0b8510 msgr2=0x7f909c14e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:53.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.452+0000 7f90a13ff700 1 --2- 172.21.15.92:0/1697081142 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f909c0b8510 0x7f909c14e260 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f908c00ebe0 tx=0x7f908c009ec0 comp rx=0 tx=0).stop 2023-11-28T11:21:53.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.452+0000 7f90a13ff700 1 -- 172.21.15.92:0/1697081142 shutdown_connections 2023-11-28T11:21:53.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.452+0000 7f90a13ff700 1 --2- 172.21.15.92:0/1697081142 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f909c0d0d60 0x7f909c0d16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:53.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.452+0000 7f90a13ff700 1 --2- 172.21.15.92:0/1697081142 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f907c0682f0 0x7f907c06a7a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:53.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.452+0000 7f90a13ff700 1 --2- 172.21.15.92:0/1697081142 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f909c0b8510 0x7f909c14e260 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:53.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.452+0000 7f90a13ff700 1 --2- 172.21.15.92:0/1697081142 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f909c0cd460 0x7f909c0d1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:53.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.452+0000 7f90a13ff700 1 -- 172.21.15.92:0/1697081142 >> 172.21.15.92:0/1697081142 conn(0x7f909c0719a0 msgr2=0x7f909c15c480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:53.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.452+0000 7f90a13ff700 1 -- 172.21.15.92:0/1697081142 shutdown_connections 2023-11-28T11:21:53.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.453+0000 7f90a13ff700 1 -- 172.21.15.92:0/1697081142 wait complete. 2023-11-28T11:21:53.472 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:21:53.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.669+0000 7fb6557ff700 1 Processor -- start 2023-11-28T11:21:53.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.669+0000 7fb6557ff700 1 -- start start 2023-11-28T11:21:53.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.670+0000 7fb6557ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6500b8510 0x7fb65006b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:53.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.670+0000 7fb6557ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb6500d0d60 0x7fb6500d11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:53.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.670+0000 7fb6557ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6500b88e0 con 0x7fb6500cd460 2023-11-28T11:21:53.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.670+0000 7fb6557ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6500cdbd0 con 0x7fb6500d0d60 2023-11-28T11:21:53.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.670+0000 7fb6557ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6500d02d0 con 0x7fb6500b8510 2023-11-28T11:21:53.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.671+0000 7fb64f7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb6500d0d60 0x7fb6500d11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:53.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.671+0000 7fb64f7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb6500d0d60 0x7fb6500d11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54102/0 (socket says 172.21.15.92:54102) 2023-11-28T11:21:53.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.671+0000 7fb64f7fe700 1 -- 172.21.15.92:0/989816700 learned_addr learned my addr 172.21.15.92:0/989816700 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:53.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.671+0000 7fb64e7fc700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb6500cd460 0x7fb65006beb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55602/0 (socket says 172.21.15.92:55602) 2023-11-28T11:21:53.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.671+0000 7fb64effd700 1 --2- 172.21.15.92:0/989816700 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6500b8510 0x7fb65006b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:53.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.671+0000 7fb64f7fe700 1 -- 172.21.15.92:0/989816700 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6500b8510 msgr2=0x7fb65006b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:53.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.671+0000 7fb64f7fe700 1 --2- 172.21.15.92:0/989816700 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6500b8510 0x7fb65006b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:53.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.671+0000 7fb64f7fe700 1 -- 172.21.15.92:0/989816700 >> v1:172.21.15.92:6789/0 conn(0x7fb6500cd460 legacy=0x7fb65006beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:53.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.671+0000 7fb64f7fe700 1 -- 172.21.15.92:0/989816700 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6500cd8f0 con 0x7fb6500d0d60 2023-11-28T11:21:53.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.671+0000 7fb64f7fe700 1 --2- 172.21.15.92:0/989816700 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6500d0d60 0x7fb6500d11a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb648009e80 tx=0x7fb64800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7f53fc50c88f76b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:53.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.671+0000 7fb64dffb700 1 -- 172.21.15.92:0/989816700 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64800e070 con 0x7fb6500d0d60 2023-11-28T11:21:53.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.672+0000 7fb64dffb700 1 -- 172.21.15.92:0/989816700 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb64800b980 con 0x7fb6500d0d60 2023-11-28T11:21:53.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.672+0000 7fb64dffb700 1 -- 172.21.15.92:0/989816700 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64800bd30 con 0x7fb6500d0d60 2023-11-28T11:21:53.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.672+0000 7fb6557ff700 1 -- 172.21.15.92:0/989816700 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6500d0d60 msgr2=0x7fb6500d11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:53.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.672+0000 7fb6557ff700 1 --2- 172.21.15.92:0/989816700 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6500d0d60 0x7fb6500d11a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb648009e80 tx=0x7fb64800b330 comp rx=0 tx=0).stop 2023-11-28T11:21:53.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.672+0000 7fb6557ff700 1 -- 172.21.15.92:0/989816700 shutdown_connections 2023-11-28T11:21:53.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.672+0000 7fb6557ff700 1 --2- 172.21.15.92:0/989816700 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6500b8510 0x7fb65006b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:53.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.672+0000 7fb6557ff700 1 --2- 172.21.15.92:0/989816700 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6500d0d60 0x7fb6500d11a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:53.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.672+0000 7fb6557ff700 1 -- 172.21.15.92:0/989816700 >> 172.21.15.92:0/989816700 conn(0x7fb6500719a0 msgr2=0x7fb650071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:53.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.672+0000 7fb6557ff700 1 -- 172.21.15.92:0/989816700 shutdown_connections 2023-11-28T11:21:53.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.672+0000 7fb6557ff700 1 -- 172.21.15.92:0/989816700 wait complete. 2023-11-28T11:21:53.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.674+0000 7fb6557ff700 1 Processor -- start 2023-11-28T11:21:53.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.674+0000 7fb6557ff700 1 -- start start 2023-11-28T11:21:53.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.675+0000 7fb6557ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6500b8510 0x7fb650154820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:53.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.675+0000 7fb6557ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6500cd460 0x7fb65014ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:53.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.675+0000 7fb6557ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6500d0d60 0x7fb65014a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:53.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.675+0000 7fb6557ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65014dbf0 con 0x7fb6500cd460 2023-11-28T11:21:53.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.675+0000 7fb6557ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65014a860 con 0x7fb6500d0d60 2023-11-28T11:21:53.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.675+0000 7fb6557ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6501591f0 con 0x7fb6500b8510 2023-11-28T11:21:53.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.675+0000 7fb64e7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6500cd460 0x7fb65014ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:53.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.675+0000 7fb64e7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6500cd460 0x7fb65014ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54114/0 (socket says 172.21.15.92:54114) 2023-11-28T11:21:53.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.675+0000 7fb64e7fc700 1 -- 172.21.15.92:0/2589577525 learned_addr learned my addr 172.21.15.92:0/2589577525 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:53.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.675+0000 7fb64effd700 1 --2- 172.21.15.92:0/2589577525 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6500b8510 0x7fb650154820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:53.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.675+0000 7fb64f7fe700 1 --2- 172.21.15.92:0/2589577525 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6500d0d60 0x7fb65014a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:53.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.675+0000 7fb64e7fc700 1 -- 172.21.15.92:0/2589577525 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6500b8510 msgr2=0x7fb650154820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:53.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.675+0000 7fb64e7fc700 1 --2- 172.21.15.92:0/2589577525 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6500b8510 0x7fb650154820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:53.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.675+0000 7fb64e7fc700 1 -- 172.21.15.92:0/2589577525 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6500d0d60 msgr2=0x7fb65014a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:53.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.675+0000 7fb64e7fc700 1 --2- 172.21.15.92:0/2589577525 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6500d0d60 0x7fb65014a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:53.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.675+0000 7fb64e7fc700 1 -- 172.21.15.92:0/2589577525 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb648009b30 con 0x7fb6500cd460 2023-11-28T11:21:53.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.676+0000 7fb64e7fc700 1 --2- 172.21.15.92:0/2589577525 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6500cd460 0x7fb65014ad00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb64400c520 tx=0x7fb64400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:53.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.676+0000 7fb6377fe700 1 -- 172.21.15.92:0/2589577525 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb644005cb0 con 0x7fb6500cd460 2023-11-28T11:21:53.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.676+0000 7fb6377fe700 1 -- 172.21.15.92:0/2589577525 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb644005e10 con 0x7fb6500cd460 2023-11-28T11:21:53.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.676+0000 7fb6557ff700 1 -- 172.21.15.92:0/2589577525 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6500cfff0 con 0x7fb6500cd460 2023-11-28T11:21:53.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.676+0000 7fb6377fe700 1 -- 172.21.15.92:0/2589577525 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6440072d0 con 0x7fb6500cd460 2023-11-28T11:21:53.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.676+0000 7fb6557ff700 1 -- 172.21.15.92:0/2589577525 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6500cfd10 con 0x7fb6500cd460 2023-11-28T11:21:53.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.677+0000 7fb6557ff700 1 -- 172.21.15.92:0/2589577525 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6501593a0 con 0x7fb6500cd460 2023-11-28T11:21:53.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.678+0000 7fb6377fe700 1 -- 172.21.15.92:0/2589577525 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb644019030 con 0x7fb6500cd460 2023-11-28T11:21:53.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.679+0000 7fb6377fe700 1 --2- 172.21.15.92:0/2589577525 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb63c068320 0x7fb63c06a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:53.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.679+0000 7fb6377fe700 1 -- 172.21.15.92:0/2589577525 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fb644085330 con 0x7fb6500cd460 2023-11-28T11:21:53.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.679+0000 7fb64effd700 1 --2- 172.21.15.92:0/2589577525 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb63c068320 0x7fb63c06a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:53.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.680+0000 7fb64effd700 1 --2- 172.21.15.92:0/2589577525 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb63c068320 0x7fb63c06a7d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb640009930 tx=0x7fb640008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:53.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.683+0000 7fb6377fe700 1 -- 172.21.15.92:0/2589577525 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb64404b5e0 con 0x7fb6500cd460 2023-11-28T11:21:53.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.875+0000 7fb6557ff700 1 -- 172.21.15.92:0/2589577525 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb6500d2920 con 0x7fb6500cd460 2023-11-28T11:21:53.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.876+0000 7fb6377fe700 1 -- 172.21.15.92:0/2589577525 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb644017020 con 0x7fb6500cd460 2023-11-28T11:21:53.878 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-11-28T11:21:53.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.878+0000 7fb6557ff700 1 -- 172.21.15.92:0/2589577525 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb63c068320 msgr2=0x7fb63c06a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:53.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.878+0000 7fb6557ff700 1 --2- 172.21.15.92:0/2589577525 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb63c068320 0x7fb63c06a7d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb640009930 tx=0x7fb640008040 comp rx=0 tx=0).stop 2023-11-28T11:21:53.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.878+0000 7fb6557ff700 1 -- 172.21.15.92:0/2589577525 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6500cd460 msgr2=0x7fb65014ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:53.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.878+0000 7fb6557ff700 1 --2- 172.21.15.92:0/2589577525 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6500cd460 0x7fb65014ad00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb64400c520 tx=0x7fb64400c8e0 comp rx=0 tx=0).stop 2023-11-28T11:21:53.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.878+0000 7fb6557ff700 1 -- 172.21.15.92:0/2589577525 shutdown_connections 2023-11-28T11:21:53.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.878+0000 7fb6557ff700 1 --2- 172.21.15.92:0/2589577525 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6500d0d60 0x7fb65014a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:53.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.878+0000 7fb6557ff700 1 --2- 172.21.15.92:0/2589577525 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6500b8510 0x7fb650154820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:53.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.878+0000 7fb6557ff700 1 --2- 172.21.15.92:0/2589577525 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb63c068320 0x7fb63c06a7d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:53.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.878+0000 7fb6557ff700 1 --2- 172.21.15.92:0/2589577525 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6500cd460 0x7fb65014ad00 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:53.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.878+0000 7fb6557ff700 1 -- 172.21.15.92:0/2589577525 >> 172.21.15.92:0/2589577525 conn(0x7fb6500719a0 msgr2=0x7fb6500b1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:53.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.878+0000 7fb6557ff700 1 -- 172.21.15.92:0/2589577525 shutdown_connections 2023-11-28T11:21:53.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:53.879+0000 7fb6557ff700 1 -- 172.21.15.92:0/2589577525 wait complete. 2023-11-28T11:21:53.898 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:21:54.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.096+0000 7fad7b7ff700 1 Processor -- start 2023-11-28T11:21:54.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.097+0000 7fad7b7ff700 1 -- start start 2023-11-28T11:21:54.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.098+0000 7fad7b7ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad740cd460 0x7fad7406c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:54.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.098+0000 7fad7b7ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fad740d0d60 0x7fad740d11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:54.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.098+0000 7fad7b7ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad740b88e0 con 0x7fad740b8510 2023-11-28T11:21:54.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.098+0000 7fad7b7ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad740cdbd0 con 0x7fad740d0d60 2023-11-28T11:21:54.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.098+0000 7fad7b7ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad740d02d0 con 0x7fad740cd460 2023-11-28T11:21:54.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.098+0000 7fad78d9a700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad740cd460 0x7fad7406c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:54.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.098+0000 7fad79d9c700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fad740d0d60 0x7fad740d11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:54.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.098+0000 7fad7959b700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fad740b8510 0x7fad7406b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.92:38722/0 (socket says 172.21.15.92:38722) 2023-11-28T11:21:54.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.098+0000 7fad7959b700 1 -- 172.21.15.92:0/808773459 learned_addr learned my addr 172.21.15.92:0/808773459 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:54.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.099+0000 7fad79d9c700 1 -- 172.21.15.92:0/808773459 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad740cd460 msgr2=0x7fad7406c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:54.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.099+0000 7fad79d9c700 1 --2- 172.21.15.92:0/808773459 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad740cd460 0x7fad7406c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:54.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.099+0000 7fad79d9c700 1 -- 172.21.15.92:0/808773459 >> v1:172.21.15.169:6789/0 conn(0x7fad740b8510 legacy=0x7fad7406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:54.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.099+0000 7fad79d9c700 1 -- 172.21.15.92:0/808773459 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad740cd8f0 con 0x7fad740d0d60 2023-11-28T11:21:54.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.099+0000 7fad78d9a700 1 --2- 172.21.15.92:0/808773459 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad740cd460 0x7fad7406c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:21:54.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.099+0000 7fad79d9c700 1 --2- 172.21.15.92:0/808773459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad740d0d60 0x7fad740d11e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fad6c009dc0 tx=0x7fad6c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8042c2043c967043 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:54.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.099+0000 7fad6bfff700 1 -- 172.21.15.92:0/808773459 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c00e070 con 0x7fad740d0d60 2023-11-28T11:21:54.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.099+0000 7fad6bfff700 1 -- 172.21.15.92:0/808773459 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad6c00bae0 con 0x7fad740d0d60 2023-11-28T11:21:54.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.100+0000 7fad6bfff700 1 -- 172.21.15.92:0/808773459 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c00be90 con 0x7fad740d0d60 2023-11-28T11:21:54.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.100+0000 7fad7b7ff700 1 -- 172.21.15.92:0/808773459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad740d0d60 msgr2=0x7fad740d11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:54.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.100+0000 7fad7b7ff700 1 --2- 172.21.15.92:0/808773459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad740d0d60 0x7fad740d11e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fad6c009dc0 tx=0x7fad6c00b330 comp rx=0 tx=0).stop 2023-11-28T11:21:54.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.100+0000 7fad7b7ff700 1 -- 172.21.15.92:0/808773459 shutdown_connections 2023-11-28T11:21:54.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.100+0000 7fad7b7ff700 1 --2- 172.21.15.92:0/808773459 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad740d0d60 0x7fad740d11e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:54.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.100+0000 7fad7b7ff700 1 --2- 172.21.15.92:0/808773459 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad740cd460 0x7fad7406c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:54.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.100+0000 7fad7b7ff700 1 -- 172.21.15.92:0/808773459 >> 172.21.15.92:0/808773459 conn(0x7fad740719a0 msgr2=0x7fad74071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:54.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.100+0000 7fad7b7ff700 1 -- 172.21.15.92:0/808773459 shutdown_connections 2023-11-28T11:21:54.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.100+0000 7fad7b7ff700 1 -- 172.21.15.92:0/808773459 wait complete. 2023-11-28T11:21:54.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.102+0000 7fad7b7ff700 1 Processor -- start 2023-11-28T11:21:54.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.102+0000 7fad7b7ff700 1 -- start start 2023-11-28T11:21:54.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.103+0000 7fad7b7ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad740b8510 0x7fad74157680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:54.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.103+0000 7fad7b7ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad740cd460 0x7fad741f29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:54.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.103+0000 7fad7b7ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad740d0d60 0x7fad7415a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:54.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.103+0000 7fad7b7ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad741585b0 con 0x7fad740d0d60 2023-11-28T11:21:54.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.103+0000 7fad7b7ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad74157d10 con 0x7fad740b8510 2023-11-28T11:21:54.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.103+0000 7fad7b7ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad74157ff0 con 0x7fad740cd460 2023-11-28T11:21:54.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.103+0000 7fad79d9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad740d0d60 0x7fad7415a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:54.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.103+0000 7fad79d9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad740d0d60 0x7fad7415a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54122/0 (socket says 172.21.15.92:54122) 2023-11-28T11:21:54.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.103+0000 7fad79d9c700 1 -- 172.21.15.92:0/2678150235 learned_addr learned my addr 172.21.15.92:0/2678150235 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:54.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.103+0000 7fad78d9a700 1 --2- 172.21.15.92:0/2678150235 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad740cd460 0x7fad741f29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:54.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.103+0000 7fad7959b700 1 --2- 172.21.15.92:0/2678150235 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad740b8510 0x7fad74157680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:54.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.104+0000 7fad79d9c700 1 -- 172.21.15.92:0/2678150235 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad740cd460 msgr2=0x7fad741f29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:54.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.104+0000 7fad79d9c700 1 --2- 172.21.15.92:0/2678150235 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad740cd460 0x7fad741f29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:54.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.104+0000 7fad79d9c700 1 -- 172.21.15.92:0/2678150235 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad740b8510 msgr2=0x7fad74157680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:54.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.104+0000 7fad79d9c700 1 --2- 172.21.15.92:0/2678150235 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad740b8510 0x7fad74157680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:54.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.104+0000 7fad79d9c700 1 -- 172.21.15.92:0/2678150235 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad640094a0 con 0x7fad740d0d60 2023-11-28T11:21:54.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.104+0000 7fad79d9c700 1 --2- 172.21.15.92:0/2678150235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad740d0d60 0x7fad7415a1f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00b370 tx=0x7fad6c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:54.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.104+0000 7fad69ffb700 1 -- 172.21.15.92:0/2678150235 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c004550 con 0x7fad740d0d60 2023-11-28T11:21:54.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.104+0000 7fad69ffb700 1 -- 172.21.15.92:0/2678150235 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad6c020070 con 0x7fad740d0d60 2023-11-28T11:21:54.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.104+0000 7fad7b7ff700 1 -- 172.21.15.92:0/2678150235 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad6c009a70 con 0x7fad740d0d60 2023-11-28T11:21:54.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.104+0000 7fad69ffb700 1 -- 172.21.15.92:0/2678150235 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c004910 con 0x7fad740d0d60 2023-11-28T11:21:54.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.104+0000 7fad7b7ff700 1 -- 172.21.15.92:0/2678150235 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad740cfff0 con 0x7fad740d0d60 2023-11-28T11:21:54.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.105+0000 7fad7b7ff700 1 -- 172.21.15.92:0/2678150235 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad740cfd10 con 0x7fad740d0d60 2023-11-28T11:21:54.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.106+0000 7fad69ffb700 1 -- 172.21.15.92:0/2678150235 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fad6c02e070 con 0x7fad740d0d60 2023-11-28T11:21:54.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.107+0000 7fad69ffb700 1 --2- 172.21.15.92:0/2678150235 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fad580682f0 0x7fad5806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:54.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.107+0000 7fad69ffb700 1 -- 172.21.15.92:0/2678150235 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fad6c022070 con 0x7fad740d0d60 2023-11-28T11:21:54.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.107+0000 7fad7959b700 1 --2- 172.21.15.92:0/2678150235 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fad580682f0 0x7fad5806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:54.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.108+0000 7fad7959b700 1 --2- 172.21.15.92:0/2678150235 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fad580682f0 0x7fad5806a7a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fad70005910 tx=0x7fad70009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:54.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.112+0000 7fad69ffb700 1 -- 172.21.15.92:0/2678150235 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fad6c054120 con 0x7fad740d0d60 2023-11-28T11:21:54.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.300+0000 7fad7b7ff700 1 -- 172.21.15.92:0/2678150235 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fad740ce360 con 0x7fad740d0d60 2023-11-28T11:21:54.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.301+0000 7fad69ffb700 1 -- 172.21.15.92:0/2678150235 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7fad6c02c020 con 0x7fad740d0d60 2023-11-28T11:21:54.303 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-11-28T11:21:54.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.303+0000 7fad7b7ff700 1 -- 172.21.15.92:0/2678150235 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fad580682f0 msgr2=0x7fad5806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:54.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.303+0000 7fad7b7ff700 1 --2- 172.21.15.92:0/2678150235 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fad580682f0 0x7fad5806a7a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fad70005910 tx=0x7fad70009040 comp rx=0 tx=0).stop 2023-11-28T11:21:54.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.303+0000 7fad7b7ff700 1 -- 172.21.15.92:0/2678150235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad740d0d60 msgr2=0x7fad7415a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:54.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.303+0000 7fad7b7ff700 1 --2- 172.21.15.92:0/2678150235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad740d0d60 0x7fad7415a1f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00b370 tx=0x7fad6c004ad0 comp rx=0 tx=0).stop 2023-11-28T11:21:54.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.303+0000 7fad7b7ff700 1 -- 172.21.15.92:0/2678150235 shutdown_connections 2023-11-28T11:21:54.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.303+0000 7fad7b7ff700 1 --2- 172.21.15.92:0/2678150235 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fad580682f0 0x7fad5806a7a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:54.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.303+0000 7fad7b7ff700 1 --2- 172.21.15.92:0/2678150235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad740d0d60 0x7fad7415a1f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:54.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.303+0000 7fad7b7ff700 1 --2- 172.21.15.92:0/2678150235 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad740b8510 0x7fad74157680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:54.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.303+0000 7fad7b7ff700 1 --2- 172.21.15.92:0/2678150235 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad740cd460 0x7fad741f29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:54.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.303+0000 7fad7b7ff700 1 -- 172.21.15.92:0/2678150235 >> 172.21.15.92:0/2678150235 conn(0x7fad740719a0 msgr2=0x7fad740b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:54.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.303+0000 7fad7b7ff700 1 -- 172.21.15.92:0/2678150235 shutdown_connections 2023-11-28T11:21:54.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.303+0000 7fad7b7ff700 1 -- 172.21.15.92:0/2678150235 wait complete. 2023-11-28T11:21:54.323 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:21:54.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.520+0000 7f9ed75ff700 1 Processor -- start 2023-11-28T11:21:54.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.520+0000 7f9ed75ff700 1 -- start start 2023-11-28T11:21:54.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.521+0000 7f9ed75ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9ed00b8510 0x7f9ed006b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:54.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.522+0000 7f9ed75ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ed00b88e0 con 0x7f9ed00d0d60 2023-11-28T11:21:54.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.522+0000 7f9ed75ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ed00cd8f0 con 0x7f9ed00cd460 2023-11-28T11:21:54.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.522+0000 7f9ed75ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ed00cdbd0 con 0x7f9ed00b8510 2023-11-28T11:21:54.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.522+0000 7f9ed5b9c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f9ed00d0d60 0x7f9ed00d1200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55610/0 (socket says 172.21.15.92:55610) 2023-11-28T11:21:54.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.522+0000 7f9ed5b9c700 1 -- 172.21.15.92:0/2444672596 learned_addr learned my addr 172.21.15.92:0/2444672596 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:54.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.522+0000 7f9ed539b700 1 --2- 172.21.15.92:0/2444672596 >> v2:172.21.15.169:3300/0 conn(0x7f9ed00b8510 0x7f9ed006b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:54.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.522+0000 7f9ec7fff700 1 -- 172.21.15.92:0/2444672596 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4143439510 0 0) 0x7f9ed00cd8f0 con 0x7f9ed00cd460 2023-11-28T11:21:54.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.522+0000 7f9ec7fff700 1 -- 172.21.15.92:0/2444672596 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ebc0035b0 con 0x7f9ed00cd460 2023-11-28T11:21:54.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.522+0000 7f9ec7fff700 1 -- 172.21.15.92:0/2444672596 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 261839884 0 0) 0x7f9ed00b88e0 con 0x7f9ed00d0d60 2023-11-28T11:21:54.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.522+0000 7f9ec7fff700 1 -- 172.21.15.92:0/2444672596 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ed00cd8f0 con 0x7f9ed00d0d60 2023-11-28T11:21:54.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.522+0000 7f9ed539b700 1 -- 172.21.15.92:0/2444672596 >> v1:172.21.15.169:6789/0 conn(0x7f9ed00cd460 legacy=0x7f9ed006beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:54.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.522+0000 7f9ed539b700 1 -- 172.21.15.92:0/2444672596 >> v1:172.21.15.92:6789/0 conn(0x7f9ed00d0d60 legacy=0x7f9ed00d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:54.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.522+0000 7f9ed539b700 1 -- 172.21.15.92:0/2444672596 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed00cfd10 con 0x7f9ed00b8510 2023-11-28T11:21:54.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.523+0000 7f9ed539b700 1 --2- 172.21.15.92:0/2444672596 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ed00b8510 0x7f9ed006b970 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc009840 tx=0x7f9ecc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a55a3e2b79e8f1e0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:54.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.523+0000 7f9ec7fff700 1 -- 172.21.15.92:0/2444672596 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc011830 con 0x7f9ed00b8510 2023-11-28T11:21:54.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.523+0000 7f9ec7fff700 1 -- 172.21.15.92:0/2444672596 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ecc011990 con 0x7f9ed00b8510 2023-11-28T11:21:54.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.523+0000 7f9ec7fff700 1 -- 172.21.15.92:0/2444672596 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc011d20 con 0x7f9ed00b8510 2023-11-28T11:21:54.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.524+0000 7f9ed75ff700 1 -- 172.21.15.92:0/2444672596 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ed00b8510 msgr2=0x7f9ed006b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:54.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.524+0000 7f9ed75ff700 1 --2- 172.21.15.92:0/2444672596 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ed00b8510 0x7f9ed006b970 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc009840 tx=0x7f9ecc009b50 comp rx=0 tx=0).stop 2023-11-28T11:21:54.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.524+0000 7f9ed75ff700 1 -- 172.21.15.92:0/2444672596 shutdown_connections 2023-11-28T11:21:54.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.524+0000 7f9ed75ff700 1 --2- 172.21.15.92:0/2444672596 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ed00b8510 0x7f9ed006b970 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:54.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.524+0000 7f9ed75ff700 1 -- 172.21.15.92:0/2444672596 >> 172.21.15.92:0/2444672596 conn(0x7f9ed00719a0 msgr2=0x7f9ed0071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:54.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.524+0000 7f9ed75ff700 1 -- 172.21.15.92:0/2444672596 shutdown_connections 2023-11-28T11:21:54.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.524+0000 7f9ed75ff700 1 -- 172.21.15.92:0/2444672596 wait complete. 2023-11-28T11:21:54.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.526+0000 7f9ed75ff700 1 Processor -- start 2023-11-28T11:21:54.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.526+0000 7f9ed75ff700 1 -- start start 2023-11-28T11:21:54.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.527+0000 7f9ed75ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ed00b8510 0x7f9ed015af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:54.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.527+0000 7f9ed75ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ed00cd460 0x7f9ed01e2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:54.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.527+0000 7f9ed75ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ed00d0d60 0x7f9ed01f4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:54.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.527+0000 7f9ed75ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed01f3a50 con 0x7f9ed00cd460 2023-11-28T11:21:54.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.527+0000 7f9ed75ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed01f3d30 con 0x7f9ed00d0d60 2023-11-28T11:21:54.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.527+0000 7f9ed75ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed01f4010 con 0x7f9ed00b8510 2023-11-28T11:21:54.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.527+0000 7f9ed4b9a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ed00cd460 0x7f9ed01e2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:54.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.527+0000 7f9ed4b9a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ed00cd460 0x7f9ed01e2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54134/0 (socket says 172.21.15.92:54134) 2023-11-28T11:21:54.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.527+0000 7f9ed4b9a700 1 -- 172.21.15.92:0/108958294 learned_addr learned my addr 172.21.15.92:0/108958294 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:54.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.527+0000 7f9ed539b700 1 --2- 172.21.15.92:0/108958294 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ed00b8510 0x7f9ed015af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:54.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.527+0000 7f9ed5b9c700 1 --2- 172.21.15.92:0/108958294 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ed00d0d60 0x7f9ed01f4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:54.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.527+0000 7f9ed4b9a700 1 -- 172.21.15.92:0/108958294 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ed00b8510 msgr2=0x7f9ed015af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:54.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.527+0000 7f9ed4b9a700 1 --2- 172.21.15.92:0/108958294 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ed00b8510 0x7f9ed015af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:54.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.527+0000 7f9ed4b9a700 1 -- 172.21.15.92:0/108958294 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ed00d0d60 msgr2=0x7f9ed01f4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:54.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.527+0000 7f9ed4b9a700 1 --2- 172.21.15.92:0/108958294 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ed00d0d60 0x7f9ed01f4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:54.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.527+0000 7f9ed4b9a700 1 -- 172.21.15.92:0/108958294 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ecc0094a0 con 0x7f9ed00cd460 2023-11-28T11:21:54.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.528+0000 7f9ed4b9a700 1 --2- 172.21.15.92:0/108958294 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ed00cd460 0x7f9ed01e2f00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9ec00010e0 tx=0x7f9ec000c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:54.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.528+0000 7f9ec5ffb700 1 -- 172.21.15.92:0/108958294 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec00052d0 con 0x7f9ed00cd460 2023-11-28T11:21:54.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.528+0000 7f9ec5ffb700 1 -- 172.21.15.92:0/108958294 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec0005430 con 0x7f9ed00cd460 2023-11-28T11:21:54.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.528+0000 7f9ec5ffb700 1 -- 172.21.15.92:0/108958294 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec00038c0 con 0x7f9ed00cd460 2023-11-28T11:21:54.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.528+0000 7f9ed75ff700 1 -- 172.21.15.92:0/108958294 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ed00d29c0 con 0x7f9ed00cd460 2023-11-28T11:21:54.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.528+0000 7f9ed75ff700 1 -- 172.21.15.92:0/108958294 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ed00cfff0 con 0x7f9ed00cd460 2023-11-28T11:21:54.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.529+0000 7f9ed75ff700 1 -- 172.21.15.92:0/108958294 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ed00d2b50 con 0x7f9ed00cd460 2023-11-28T11:21:54.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.530+0000 7f9ec5ffb700 1 -- 172.21.15.92:0/108958294 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9ec0019030 con 0x7f9ed00cd460 2023-11-28T11:21:54.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.531+0000 7f9ec5ffb700 1 --2- 172.21.15.92:0/108958294 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9eb40682f0 0x7f9eb406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:54.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.531+0000 7f9ec5ffb700 1 -- 172.21.15.92:0/108958294 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f9ec0085360 con 0x7f9ed00cd460 2023-11-28T11:21:54.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.531+0000 7f9ed539b700 1 --2- 172.21.15.92:0/108958294 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9eb40682f0 0x7f9eb406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:54.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.532+0000 7f9ed539b700 1 --2- 172.21.15.92:0/108958294 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9eb40682f0 0x7f9eb406a7a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc009fd0 tx=0x7f9ecc01c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:54.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.535+0000 7f9ec5ffb700 1 -- 172.21.15.92:0/108958294 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9ec00517a0 con 0x7f9ed00cd460 2023-11-28T11:21:54.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.728+0000 7f9ed75ff700 1 -- 172.21.15.92:0/108958294 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9ed0067880 con 0x7f9ed00cd460 2023-11-28T11:21:54.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.729+0000 7f9ec5ffb700 1 -- 172.21.15.92:0/108958294 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9ec0054dc0 con 0x7f9ed00cd460 2023-11-28T11:21:54.731 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-11-28T11:21:54.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.731+0000 7f9ed75ff700 1 -- 172.21.15.92:0/108958294 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9eb40682f0 msgr2=0x7f9eb406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:54.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.731+0000 7f9ed75ff700 1 --2- 172.21.15.92:0/108958294 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9eb40682f0 0x7f9eb406a7a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc009fd0 tx=0x7f9ecc01c040 comp rx=0 tx=0).stop 2023-11-28T11:21:54.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.731+0000 7f9ed75ff700 1 -- 172.21.15.92:0/108958294 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ed00cd460 msgr2=0x7f9ed01e2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:54.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.731+0000 7f9ed75ff700 1 --2- 172.21.15.92:0/108958294 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ed00cd460 0x7f9ed01e2f00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9ec00010e0 tx=0x7f9ec000c7d0 comp rx=0 tx=0).stop 2023-11-28T11:21:54.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.731+0000 7f9ed75ff700 1 -- 172.21.15.92:0/108958294 shutdown_connections 2023-11-28T11:21:54.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.731+0000 7f9ed75ff700 1 --2- 172.21.15.92:0/108958294 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9ed00d0d60 0x7f9ed01f4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:54.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.731+0000 7f9ed75ff700 1 --2- 172.21.15.92:0/108958294 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9ed00b8510 0x7f9ed015af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:54.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.731+0000 7f9ed75ff700 1 --2- 172.21.15.92:0/108958294 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9eb40682f0 0x7f9eb406a7a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:54.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.731+0000 7f9ed75ff700 1 --2- 172.21.15.92:0/108958294 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ed00cd460 0x7f9ed01e2f00 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:54.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.731+0000 7f9ed75ff700 1 -- 172.21.15.92:0/108958294 >> 172.21.15.92:0/108958294 conn(0x7f9ed00719a0 msgr2=0x7f9ed00ccd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:54.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.731+0000 7f9ed75ff700 1 -- 172.21.15.92:0/108958294 shutdown_connections 2023-11-28T11:21:54.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.731+0000 7f9ed75ff700 1 -- 172.21.15.92:0/108958294 wait complete. 2023-11-28T11:21:54.752 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T11:21:54.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.949+0000 7fad589ff700 1 Processor -- start 2023-11-28T11:21:54.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.949+0000 7fad589ff700 1 -- start start 2023-11-28T11:21:54.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.950+0000 7fad589ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fad540b8510 0x7fad5406b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:54.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.950+0000 7fad589ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad540d0d60 0x7fad540d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:54.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.950+0000 7fad589ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad540b88e0 con 0x7fad540cd460 2023-11-28T11:21:54.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.950+0000 7fad589ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad540cdbd0 con 0x7fad540b8510 2023-11-28T11:21:54.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.950+0000 7fad589ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad540d02d0 con 0x7fad540d0d60 2023-11-28T11:21:54.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.950+0000 7fad5259c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fad540b8510 0x7fad5406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:54.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.950+0000 7fad5259c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fad540b8510 0x7fad5406b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54140/0 (socket says 172.21.15.92:54140) 2023-11-28T11:21:54.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.950+0000 7fad5259c700 1 -- 172.21.15.92:0/1446505166 learned_addr learned my addr 172.21.15.92:0/1446505166 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:54.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.950+0000 7fad4bfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fad540cd460 0x7fad5406beb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55612/0 (socket says 172.21.15.92:55612) 2023-11-28T11:21:54.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.951+0000 7fad52d9d700 1 --2- 172.21.15.92:0/1446505166 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad540d0d60 0x7fad540d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:54.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.951+0000 7fad5259c700 1 -- 172.21.15.92:0/1446505166 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad540d0d60 msgr2=0x7fad540d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:54.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.951+0000 7fad5259c700 1 --2- 172.21.15.92:0/1446505166 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad540d0d60 0x7fad540d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:54.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.951+0000 7fad5259c700 1 -- 172.21.15.92:0/1446505166 >> v1:172.21.15.92:6789/0 conn(0x7fad540cd460 legacy=0x7fad5406beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:54.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.951+0000 7fad5259c700 1 -- 172.21.15.92:0/1446505166 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad540cd8f0 con 0x7fad540b8510 2023-11-28T11:21:54.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.951+0000 7fad51d9b700 1 -- 172.21.15.92:0/1446505166 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4215817936 0 0) 0x7fad540b88e0 con 0x7fad540cd460 2023-11-28T11:21:54.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.951+0000 7fad5259c700 1 --2- 172.21.15.92:0/1446505166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad540b8510 0x7fad5406b970 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fad44009920 tx=0x7fad44009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=799e61573399aeca server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:54.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.951+0000 7fad51d9b700 1 -- 172.21.15.92:0/1446505166 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4400ba20 con 0x7fad540b8510 2023-11-28T11:21:54.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.951+0000 7fad51d9b700 1 -- 172.21.15.92:0/1446505166 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad4400bb80 con 0x7fad540b8510 2023-11-28T11:21:54.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.951+0000 7fad51d9b700 1 -- 172.21.15.92:0/1446505166 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad44009e70 con 0x7fad540b8510 2023-11-28T11:21:54.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.952+0000 7fad589ff700 1 -- 172.21.15.92:0/1446505166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad540b8510 msgr2=0x7fad5406b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:54.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.952+0000 7fad589ff700 1 --2- 172.21.15.92:0/1446505166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad540b8510 0x7fad5406b970 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fad44009920 tx=0x7fad44009c30 comp rx=0 tx=0).stop 2023-11-28T11:21:54.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.952+0000 7fad589ff700 1 -- 172.21.15.92:0/1446505166 shutdown_connections 2023-11-28T11:21:54.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.952+0000 7fad589ff700 1 --2- 172.21.15.92:0/1446505166 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad540d0d60 0x7fad540d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:54.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.952+0000 7fad589ff700 1 --2- 172.21.15.92:0/1446505166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad540b8510 0x7fad5406b970 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:54.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.952+0000 7fad589ff700 1 -- 172.21.15.92:0/1446505166 >> 172.21.15.92:0/1446505166 conn(0x7fad540719a0 msgr2=0x7fad54071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:54.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.952+0000 7fad589ff700 1 -- 172.21.15.92:0/1446505166 shutdown_connections 2023-11-28T11:21:54.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.952+0000 7fad589ff700 1 -- 172.21.15.92:0/1446505166 wait complete. 2023-11-28T11:21:54.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.954+0000 7fad589ff700 1 Processor -- start 2023-11-28T11:21:54.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.954+0000 7fad589ff700 1 -- start start 2023-11-28T11:21:54.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.955+0000 7fad589ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad540b8510 0x7fad541f5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:54.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.955+0000 7fad589ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad540cd460 0x7fad541e3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:54.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.955+0000 7fad589ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad540d0d60 0x7fad540b3260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:54.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.955+0000 7fad589ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad541e4af0 con 0x7fad540cd460 2023-11-28T11:21:54.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.955+0000 7fad589ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad541e5f30 con 0x7fad540d0d60 2023-11-28T11:21:54.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.955+0000 7fad589ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad541e4810 con 0x7fad540b8510 2023-11-28T11:21:54.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.955+0000 7fad4bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad540cd460 0x7fad541e3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:54.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.955+0000 7fad4bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad540cd460 0x7fad541e3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54144/0 (socket says 172.21.15.92:54144) 2023-11-28T11:21:54.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.955+0000 7fad4bfff700 1 -- 172.21.15.92:0/3127072931 learned_addr learned my addr 172.21.15.92:0/3127072931 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:54.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.955+0000 7fad5259c700 1 --2- 172.21.15.92:0/3127072931 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad540b8510 0x7fad541f5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:54.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.955+0000 7fad52d9d700 1 --2- 172.21.15.92:0/3127072931 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad540d0d60 0x7fad540b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:54.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.955+0000 7fad4bfff700 1 -- 172.21.15.92:0/3127072931 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad540b8510 msgr2=0x7fad541f5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:54.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.955+0000 7fad4bfff700 1 --2- 172.21.15.92:0/3127072931 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad540b8510 0x7fad541f5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:54.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.955+0000 7fad4bfff700 1 -- 172.21.15.92:0/3127072931 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad540d0d60 msgr2=0x7fad540b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:54.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.955+0000 7fad4bfff700 1 --2- 172.21.15.92:0/3127072931 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad540d0d60 0x7fad540b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:54.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.955+0000 7fad4bfff700 1 -- 172.21.15.92:0/3127072931 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad44009550 con 0x7fad540cd460 2023-11-28T11:21:54.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.956+0000 7fad4bfff700 1 --2- 172.21.15.92:0/3127072931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad540cd460 0x7fad541e3d10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fad3c00c520 tx=0x7fad3c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:54.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.956+0000 7fad4affd700 1 -- 172.21.15.92:0/3127072931 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad3c004c80 con 0x7fad540cd460 2023-11-28T11:21:54.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.956+0000 7fad4affd700 1 -- 172.21.15.92:0/3127072931 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad3c004e00 con 0x7fad540cd460 2023-11-28T11:21:54.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.956+0000 7fad589ff700 1 -- 172.21.15.92:0/3127072931 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad540cfff0 con 0x7fad540cd460 2023-11-28T11:21:54.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.956+0000 7fad4affd700 1 -- 172.21.15.92:0/3127072931 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad3c007310 con 0x7fad540cd460 2023-11-28T11:21:54.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.956+0000 7fad589ff700 1 -- 172.21.15.92:0/3127072931 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad540cfd10 con 0x7fad540cd460 2023-11-28T11:21:54.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.957+0000 7fad589ff700 1 -- 172.21.15.92:0/3127072931 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad54067880 con 0x7fad540cd460 2023-11-28T11:21:54.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.958+0000 7fad4affd700 1 -- 172.21.15.92:0/3127072931 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fad3c01a020 con 0x7fad540cd460 2023-11-28T11:21:54.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.959+0000 7fad4affd700 1 --2- 172.21.15.92:0/3127072931 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fad340682f0 0x7fad3406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:54.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.959+0000 7fad4affd700 1 -- 172.21.15.92:0/3127072931 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fad3c085cb0 con 0x7fad540cd460 2023-11-28T11:21:54.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.959+0000 7fad5259c700 1 --2- 172.21.15.92:0/3127072931 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fad340682f0 0x7fad3406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:54.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.959+0000 7fad5259c700 1 --2- 172.21.15.92:0/3127072931 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fad340682f0 0x7fad3406a7a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fad44009130 tx=0x7fad44008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:54.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:54.963+0000 7fad4affd700 1 -- 172.21.15.92:0/3127072931 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fad3c0521a0 con 0x7fad540cd460 2023-11-28T11:21:55.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.154+0000 7fad589ff700 1 -- 172.21.15.92:0/3127072931 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fad540d2940 con 0x7fad540cd460 2023-11-28T11:21:55.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.155+0000 7fad4affd700 1 -- 172.21.15.92:0/3127072931 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7fad3c0557c0 con 0x7fad540cd460 2023-11-28T11:21:55.157 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-11-28T11:21:55.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.157+0000 7fad589ff700 1 -- 172.21.15.92:0/3127072931 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fad340682f0 msgr2=0x7fad3406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:55.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.157+0000 7fad589ff700 1 --2- 172.21.15.92:0/3127072931 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fad340682f0 0x7fad3406a7a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fad44009130 tx=0x7fad44008030 comp rx=0 tx=0).stop 2023-11-28T11:21:55.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.157+0000 7fad589ff700 1 -- 172.21.15.92:0/3127072931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad540cd460 msgr2=0x7fad541e3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:55.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.157+0000 7fad589ff700 1 --2- 172.21.15.92:0/3127072931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad540cd460 0x7fad541e3d10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fad3c00c520 tx=0x7fad3c00c8e0 comp rx=0 tx=0).stop 2023-11-28T11:21:55.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.157+0000 7fad589ff700 1 -- 172.21.15.92:0/3127072931 shutdown_connections 2023-11-28T11:21:55.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.157+0000 7fad589ff700 1 --2- 172.21.15.92:0/3127072931 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad540d0d60 0x7fad540b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:55.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.157+0000 7fad589ff700 1 --2- 172.21.15.92:0/3127072931 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad540b8510 0x7fad541f5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:55.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.157+0000 7fad589ff700 1 --2- 172.21.15.92:0/3127072931 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fad340682f0 0x7fad3406a7a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:55.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.157+0000 7fad589ff700 1 --2- 172.21.15.92:0/3127072931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad540cd460 0x7fad541e3d10 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:55.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.157+0000 7fad589ff700 1 -- 172.21.15.92:0/3127072931 >> 172.21.15.92:0/3127072931 conn(0x7fad540719a0 msgr2=0x7fad54071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:55.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.157+0000 7fad589ff700 1 -- 172.21.15.92:0/3127072931 shutdown_connections 2023-11-28T11:21:55.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.157+0000 7fad589ff700 1 -- 172.21.15.92:0/3127072931 wait complete. 2023-11-28T11:21:55.178 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T11:21:55.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.374+0000 7fc821bff700 1 Processor -- start 2023-11-28T11:21:55.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.374+0000 7fc821bff700 1 -- start start 2023-11-28T11:21:55.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.375+0000 7fc821bff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc81c0cd460 0x7fc81c06c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:55.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.375+0000 7fc821bff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc81c0b88e0 con 0x7fc81c0b8510 2023-11-28T11:21:55.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.375+0000 7fc821bff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc81c0cd8f0 con 0x7fc81c0d0d60 2023-11-28T11:21:55.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.375+0000 7fc821bff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc81c0cdbd0 con 0x7fc81c0cd460 2023-11-28T11:21:55.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.375+0000 7fc81affd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc81c0cd460 0x7fc81c06c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:55.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.375+0000 7fc81affd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc81c0cd460 0x7fc81c06c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54160/0 (socket says 172.21.15.92:54160) 2023-11-28T11:21:55.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.375+0000 7fc81b7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc81c0b8510 0x7fc81c06b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55622/0 (socket says 172.21.15.92:55622) 2023-11-28T11:21:55.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.375+0000 7fc81affd700 1 -- 172.21.15.92:0/671782787 learned_addr learned my addr 172.21.15.92:0/671782787 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:55.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.376+0000 7fc81affd700 1 -- 172.21.15.92:0/671782787 >> v1:172.21.15.169:6789/0 conn(0x7fc81c0d0d60 legacy=0x7fc81c0d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:55.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.376+0000 7fc81affd700 1 -- 172.21.15.92:0/671782787 >> v1:172.21.15.92:6789/0 conn(0x7fc81c0b8510 legacy=0x7fc81c06b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:55.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.376+0000 7fc81affd700 1 -- 172.21.15.92:0/671782787 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc81c0cfd10 con 0x7fc81c0cd460 2023-11-28T11:21:55.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.376+0000 7fc81a7fc700 1 -- 172.21.15.92:0/671782787 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1896626724 0 0) 0x7fc81c0b88e0 con 0x7fc81c0b8510 2023-11-28T11:21:55.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.376+0000 7fc81affd700 1 --2- 172.21.15.92:0/671782787 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc81c0cd460 0x7fc81c06c080 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc8100098f0 tx=0x7fc810009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd546f688f6e10d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:55.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.376+0000 7fc81a7fc700 1 -- 172.21.15.92:0/671782787 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81000b8a0 con 0x7fc81c0cd460 2023-11-28T11:21:55.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.376+0000 7fc81a7fc700 1 -- 172.21.15.92:0/671782787 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc81000ba00 con 0x7fc81c0cd460 2023-11-28T11:21:55.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.376+0000 7fc81a7fc700 1 -- 172.21.15.92:0/671782787 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81000bd90 con 0x7fc81c0cd460 2023-11-28T11:21:55.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.377+0000 7fc821bff700 1 -- 172.21.15.92:0/671782787 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc81c0cd460 msgr2=0x7fc81c06c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:55.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.377+0000 7fc821bff700 1 --2- 172.21.15.92:0/671782787 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc81c0cd460 0x7fc81c06c080 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc8100098f0 tx=0x7fc810009c00 comp rx=0 tx=0).stop 2023-11-28T11:21:55.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.377+0000 7fc821bff700 1 -- 172.21.15.92:0/671782787 shutdown_connections 2023-11-28T11:21:55.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.377+0000 7fc821bff700 1 --2- 172.21.15.92:0/671782787 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc81c0cd460 0x7fc81c06c080 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:55.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.377+0000 7fc821bff700 1 -- 172.21.15.92:0/671782787 >> 172.21.15.92:0/671782787 conn(0x7fc81c0719a0 msgr2=0x7fc81c071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:55.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.377+0000 7fc821bff700 1 -- 172.21.15.92:0/671782787 shutdown_connections 2023-11-28T11:21:55.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.377+0000 7fc821bff700 1 -- 172.21.15.92:0/671782787 wait complete. 2023-11-28T11:21:55.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.379+0000 7fc821bff700 1 Processor -- start 2023-11-28T11:21:55.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.379+0000 7fc821bff700 1 -- start start 2023-11-28T11:21:55.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.380+0000 7fc821bff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc81c0b8510 0x7fc81c1e3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:55.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.380+0000 7fc821bff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc81c0cd460 0x7fc81c15b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:55.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.380+0000 7fc821bff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc81c0d0d60 0x7fc81c1e1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:55.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.380+0000 7fc821bff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc81c15ba30 con 0x7fc81c0b8510 2023-11-28T11:21:55.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.380+0000 7fc821bff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc81c1e2570 con 0x7fc81c0cd460 2023-11-28T11:21:55.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.380+0000 7fc821bff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc81c1e2850 con 0x7fc81c0d0d60 2023-11-28T11:21:55.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.380+0000 7fc81b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc81c0b8510 0x7fc81c1e3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:55.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.380+0000 7fc81b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc81c0b8510 0x7fc81c1e3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54168/0 (socket says 172.21.15.92:54168) 2023-11-28T11:21:55.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.380+0000 7fc81b7fe700 1 -- 172.21.15.92:0/844698071 learned_addr learned my addr 172.21.15.92:0/844698071 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:55.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.380+0000 7fc81affd700 1 --2- 172.21.15.92:0/844698071 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc81c0cd460 0x7fc81c15b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:55.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.380+0000 7fc81bfff700 1 --2- 172.21.15.92:0/844698071 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc81c0d0d60 0x7fc81c1e1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:55.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.380+0000 7fc81b7fe700 1 -- 172.21.15.92:0/844698071 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc81c0d0d60 msgr2=0x7fc81c1e1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:55.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.380+0000 7fc81b7fe700 1 --2- 172.21.15.92:0/844698071 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc81c0d0d60 0x7fc81c1e1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:55.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.381+0000 7fc81b7fe700 1 -- 172.21.15.92:0/844698071 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc81c0cd460 msgr2=0x7fc81c15b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:55.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.381+0000 7fc81b7fe700 1 --2- 172.21.15.92:0/844698071 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc81c0cd460 0x7fc81c15b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:55.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.381+0000 7fc81b7fe700 1 -- 172.21.15.92:0/844698071 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc810009550 con 0x7fc81c0b8510 2023-11-28T11:21:55.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.381+0000 7fc81bfff700 1 --2- 172.21.15.92:0/844698071 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc81c0d0d60 0x7fc81c1e1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:21:55.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.381+0000 7fc81b7fe700 1 --2- 172.21.15.92:0/844698071 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc81c0b8510 0x7fc81c1e3610 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc80c00c520 tx=0x7fc80c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:55.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.381+0000 7fc803fff700 1 -- 172.21.15.92:0/844698071 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80c004c80 con 0x7fc81c0b8510 2023-11-28T11:21:55.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.381+0000 7fc803fff700 1 -- 172.21.15.92:0/844698071 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc80c004de0 con 0x7fc81c0b8510 2023-11-28T11:21:55.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.381+0000 7fc821bff700 1 -- 172.21.15.92:0/844698071 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc81c0d29c0 con 0x7fc81c0b8510 2023-11-28T11:21:55.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.381+0000 7fc803fff700 1 -- 172.21.15.92:0/844698071 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80c007310 con 0x7fc81c0b8510 2023-11-28T11:21:55.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.381+0000 7fc821bff700 1 -- 172.21.15.92:0/844698071 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc81c0cfff0 con 0x7fc81c0b8510 2023-11-28T11:21:55.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.382+0000 7fc821bff700 1 -- 172.21.15.92:0/844698071 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc81c0d2b50 con 0x7fc81c0b8510 2023-11-28T11:21:55.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.383+0000 7fc803fff700 1 -- 172.21.15.92:0/844698071 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc80c01a020 con 0x7fc81c0b8510 2023-11-28T11:21:55.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.384+0000 7fc803fff700 1 --2- 172.21.15.92:0/844698071 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc8080682f0 0x7fc80806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:55.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.384+0000 7fc803fff700 1 -- 172.21.15.92:0/844698071 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fc80c085b40 con 0x7fc81c0b8510 2023-11-28T11:21:55.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.384+0000 7fc81affd700 1 --2- 172.21.15.92:0/844698071 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc8080682f0 0x7fc80806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:55.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.385+0000 7fc81affd700 1 --2- 172.21.15.92:0/844698071 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc8080682f0 0x7fc80806a7a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc81c0b1de0 tx=0x7fc810008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:55.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.388+0000 7fc803fff700 1 -- 172.21.15.92:0/844698071 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc80c052010 con 0x7fc81c0b8510 2023-11-28T11:21:55.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.592+0000 7fc821bff700 1 -- 172.21.15.92:0/844698071 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc81c067880 con 0x7fc81c0b8510 2023-11-28T11:21:55.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.593+0000 7fc803fff700 1 -- 172.21.15.92:0/844698071 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc80c055630 con 0x7fc81c0b8510 2023-11-28T11:21:55.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.595+0000 7fc821bff700 1 -- 172.21.15.92:0/844698071 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc8080682f0 msgr2=0x7fc80806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:55.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.595+0000 7fc821bff700 1 --2- 172.21.15.92:0/844698071 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc8080682f0 0x7fc80806a7a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc81c0b1de0 tx=0x7fc810008030 comp rx=0 tx=0).stop 2023-11-28T11:21:55.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.595+0000 7fc821bff700 1 -- 172.21.15.92:0/844698071 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc81c0b8510 msgr2=0x7fc81c1e3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:55.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.595+0000 7fc821bff700 1 --2- 172.21.15.92:0/844698071 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc81c0b8510 0x7fc81c1e3610 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc80c00c520 tx=0x7fc80c00c830 comp rx=0 tx=0).stop 2023-11-28T11:21:55.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.595+0000 7fc821bff700 1 -- 172.21.15.92:0/844698071 shutdown_connections 2023-11-28T11:21:55.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.595+0000 7fc821bff700 1 --2- 172.21.15.92:0/844698071 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc81c0d0d60 0x7fc81c1e1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:55.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.595+0000 7fc821bff700 1 --2- 172.21.15.92:0/844698071 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc8080682f0 0x7fc80806a7a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:55.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.595+0000 7fc821bff700 1 --2- 172.21.15.92:0/844698071 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc81c0b8510 0x7fc81c1e3610 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:55.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.595+0000 7fc821bff700 1 --2- 172.21.15.92:0/844698071 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc81c0cd460 0x7fc81c15b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:55.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.595+0000 7fc821bff700 1 -- 172.21.15.92:0/844698071 >> 172.21.15.92:0/844698071 conn(0x7fc81c0719a0 msgr2=0x7fc81c0b7920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:55.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.595+0000 7fc821bff700 1 -- 172.21.15.92:0/844698071 shutdown_connections 2023-11-28T11:21:55.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:55.595+0000 7fc821bff700 1 -- 172.21.15.92:0/844698071 wait complete. 2023-11-28T11:21:55.599 INFO:teuthology.orchestra.run.smithi092.stderr:listed 0 entries 2023-11-28T11:21:55.615 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-28T11:21:55.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:55.616 DEBUG:teuthology.orchestra.run.smithi092:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-11-28T11:21:55.691 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2023-11-28T11:21:55.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:55.691 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T11:21:55.739 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-28T11:21:55.739 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:21:55.791 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-28T11:21:55.792 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:21:55.841 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-28T11:21:55.841 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:21:55.889 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:21:56.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.085+0000 7f6f015ff700 1 Processor -- start 2023-11-28T11:21:56.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.085+0000 7f6f015ff700 1 -- start start 2023-11-28T11:21:56.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.086+0000 7f6f015ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6efc0cd460 0x7f6efc06c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:56.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.086+0000 7f6f015ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6efc0d0d60 0x7f6efc0d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:56.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.086+0000 7f6f015ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6efc0b88e0 con 0x7f6efc0b8510 2023-11-28T11:21:56.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.086+0000 7f6f015ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6efc0cdbd0 con 0x7f6efc0cd460 2023-11-28T11:21:56.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.086+0000 7f6f015ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6efc0d02d0 con 0x7f6efc0d0d60 2023-11-28T11:21:56.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.087+0000 7f6efb7fe700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6efc0d0d60 0x7f6efc0d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:56.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.087+0000 7f6efaffd700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f6efc0b8510 0x7f6efc06b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.92:38736/0 (socket says 172.21.15.92:38736) 2023-11-28T11:21:56.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.087+0000 7f6efaffd700 1 -- 172.21.15.92:0/3188224473 learned_addr learned my addr 172.21.15.92:0/3188224473 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:56.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.087+0000 7f6efa7fc700 1 --2- 172.21.15.92:0/3188224473 >> v2:172.21.15.169:3300/0 conn(0x7f6efc0cd460 0x7f6efc06c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:56.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.087+0000 7f6efb7fe700 1 -- 172.21.15.92:0/3188224473 >> v2:172.21.15.169:3300/0 conn(0x7f6efc0cd460 msgr2=0x7f6efc06c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:56.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.087+0000 7f6ef9ffb700 1 -- 172.21.15.92:0/3188224473 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 907408599 0 0) 0x7f6efc0b88e0 con 0x7f6efc0b8510 2023-11-28T11:21:56.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.087+0000 7f6efb7fe700 1 --2- 172.21.15.92:0/3188224473 >> v2:172.21.15.169:3300/0 conn(0x7f6efc0cd460 0x7f6efc06c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:56.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.087+0000 7f6efb7fe700 1 -- 172.21.15.92:0/3188224473 >> v1:172.21.15.169:6789/0 conn(0x7f6efc0b8510 legacy=0x7f6efc06b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:56.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.087+0000 7f6efb7fe700 1 -- 172.21.15.92:0/3188224473 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6efc0cd8f0 con 0x7f6efc0d0d60 2023-11-28T11:21:56.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.088+0000 7f6efb7fe700 1 --2- 172.21.15.92:0/3188224473 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6efc0d0d60 0x7f6efc0d1200 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4009d90 tx=0x7f6ef400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e39d7cffe2809bf7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:56.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.088+0000 7f6ef9ffb700 1 -- 172.21.15.92:0/3188224473 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef400e070 con 0x7f6efc0d0d60 2023-11-28T11:21:56.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.088+0000 7f6ef9ffb700 1 -- 172.21.15.92:0/3188224473 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ef400bb20 con 0x7f6efc0d0d60 2023-11-28T11:21:56.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.088+0000 7f6ef9ffb700 1 -- 172.21.15.92:0/3188224473 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef4012430 con 0x7f6efc0d0d60 2023-11-28T11:21:56.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.088+0000 7f6f015ff700 1 -- 172.21.15.92:0/3188224473 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6efc0d0d60 msgr2=0x7f6efc0d1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:56.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.088+0000 7f6f015ff700 1 --2- 172.21.15.92:0/3188224473 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6efc0d0d60 0x7f6efc0d1200 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4009d90 tx=0x7f6ef400b330 comp rx=0 tx=0).stop 2023-11-28T11:21:56.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.088+0000 7f6f015ff700 1 -- 172.21.15.92:0/3188224473 shutdown_connections 2023-11-28T11:21:56.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.088+0000 7f6f015ff700 1 --2- 172.21.15.92:0/3188224473 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6efc0d0d60 0x7f6efc0d1200 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:56.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.088+0000 7f6f015ff700 1 --2- 172.21.15.92:0/3188224473 >> v2:172.21.15.169:3300/0 conn(0x7f6efc0cd460 0x7f6efc06c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:56.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.089+0000 7f6f015ff700 1 -- 172.21.15.92:0/3188224473 >> 172.21.15.92:0/3188224473 conn(0x7f6efc0719a0 msgr2=0x7f6efc071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:56.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.089+0000 7f6f015ff700 1 -- 172.21.15.92:0/3188224473 shutdown_connections 2023-11-28T11:21:56.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.089+0000 7f6f015ff700 1 -- 172.21.15.92:0/3188224473 wait complete. 2023-11-28T11:21:56.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.090+0000 7f6f015ff700 1 Processor -- start 2023-11-28T11:21:56.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.090+0000 7f6f015ff700 1 -- start start 2023-11-28T11:21:56.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.091+0000 7f6f015ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6efc0b8510 0x7f6efc14d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:56.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.091+0000 7f6f015ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6efc0cd460 0x7f6efc159d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:56.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.091+0000 7f6f015ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6efc0d0d60 0x7f6efc15a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:56.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.091+0000 7f6f015ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6efc1f5e30 con 0x7f6efc0d0d60 2023-11-28T11:21:56.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.091+0000 7f6f015ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6efc1f6bb0 con 0x7f6efc0b8510 2023-11-28T11:21:56.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.091+0000 7f6f015ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6efc1f60d0 con 0x7f6efc0cd460 2023-11-28T11:21:56.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.092+0000 7f6efb7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6efc0d0d60 0x7f6efc15a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:56.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.092+0000 7f6efb7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6efc0d0d60 0x7f6efc15a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54176/0 (socket says 172.21.15.92:54176) 2023-11-28T11:21:56.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.092+0000 7f6efb7fe700 1 -- 172.21.15.92:0/3950503348 learned_addr learned my addr 172.21.15.92:0/3950503348 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:56.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.092+0000 7f6efaffd700 1 --2- 172.21.15.92:0/3950503348 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6efc0b8510 0x7f6efc14d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:56.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.092+0000 7f6efa7fc700 1 --2- 172.21.15.92:0/3950503348 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6efc0cd460 0x7f6efc159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:56.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.092+0000 7f6efb7fe700 1 -- 172.21.15.92:0/3950503348 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6efc0cd460 msgr2=0x7f6efc159d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:56.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.092+0000 7f6efb7fe700 1 --2- 172.21.15.92:0/3950503348 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6efc0cd460 0x7f6efc159d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:56.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.092+0000 7f6efb7fe700 1 -- 172.21.15.92:0/3950503348 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6efc0b8510 msgr2=0x7f6efc14d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:56.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.092+0000 7f6efb7fe700 1 --2- 172.21.15.92:0/3950503348 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6efc0b8510 0x7f6efc14d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:56.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.092+0000 7f6efb7fe700 1 -- 172.21.15.92:0/3950503348 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef4009a70 con 0x7f6efc0d0d60 2023-11-28T11:21:56.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.092+0000 7f6efb7fe700 1 --2- 172.21.15.92:0/3950503348 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6efc0d0d60 0x7f6efc15a290 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4014040 tx=0x7f6ef4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:56.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.093+0000 7f6ee37fe700 1 -- 172.21.15.92:0/3950503348 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef401b810 con 0x7f6efc0d0d60 2023-11-28T11:21:56.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.093+0000 7f6ee37fe700 1 -- 172.21.15.92:0/3950503348 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ef4020070 con 0x7f6efc0d0d60 2023-11-28T11:21:56.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.093+0000 7f6ee37fe700 1 -- 172.21.15.92:0/3950503348 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef401bbd0 con 0x7f6efc0d0d60 2023-11-28T11:21:56.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.093+0000 7f6f015ff700 1 -- 172.21.15.92:0/3950503348 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6efc0cfff0 con 0x7f6efc0d0d60 2023-11-28T11:21:56.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.093+0000 7f6f015ff700 1 -- 172.21.15.92:0/3950503348 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6efc0cfd10 con 0x7f6efc0d0d60 2023-11-28T11:21:56.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.094+0000 7f6f015ff700 1 -- 172.21.15.92:0/3950503348 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6efc067880 con 0x7f6efc0d0d60 2023-11-28T11:21:56.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.095+0000 7f6ee37fe700 1 -- 172.21.15.92:0/3950503348 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6ef402e070 con 0x7f6efc0d0d60 2023-11-28T11:21:56.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.096+0000 7f6ee37fe700 1 --2- 172.21.15.92:0/3950503348 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6ee80682f0 0x7f6ee806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:56.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.096+0000 7f6ee37fe700 1 -- 172.21.15.92:0/3950503348 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f6ef4022070 con 0x7f6efc0d0d60 2023-11-28T11:21:56.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.096+0000 7f6efaffd700 1 --2- 172.21.15.92:0/3950503348 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6ee80682f0 0x7f6ee806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:56.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.096+0000 7f6efaffd700 1 --2- 172.21.15.92:0/3950503348 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6ee80682f0 0x7f6ee806a7a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6eec007fd0 tx=0x7f6eec007de0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:56.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.100+0000 7f6ee37fe700 1 -- 172.21.15.92:0/3950503348 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6ef4054510 con 0x7f6efc0d0d60 2023-11-28T11:21:56.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.293+0000 7f6f015ff700 1 -- 172.21.15.92:0/3950503348 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6efc0d2940 con 0x7f6efc0d0d60 2023-11-28T11:21:56.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.295+0000 7f6ee37fe700 1 -- 172.21.15.92:0/3950503348 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+19844 (secure 0 0 0) 0x7f6ef402c090 con 0x7f6efc0d0d60 2023-11-28T11:21:56.297 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:21:56.298 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":29,"fsid":"68956e0c-c16f-47cd-a220-f98364b0831d","created":"2023-11-28T11:20:28.697573+0000","modified":"2023-11-28T11:21:37.638578+0000","last_up_change":"2023-11-28T11:20:41.257011+0000","last_in_change":"2023-11-28T11:20:32.249767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:20:41.925237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:20:48.682497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:20:54.471398+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:20:55.204585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:20:58.297307+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6abe483f-0e03-4a26-b9cf-62041291747e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":115823},{"type":"v1","addr":"172.21.15.92:6809","nonce":115823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":115823},{"type":"v1","addr":"172.21.15.92:6813","nonce":115823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":115823},{"type":"v1","addr":"172.21.15.92:6821","nonce":115823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":115823},{"type":"v1","addr":"172.21.15.92:6817","nonce":115823}]},"public_addr":"172.21.15.92:6809/115823","cluster_addr":"172.21.15.92:6813/115823","heartbeat_back_addr":"172.21.15.92:6821/115823","heartbeat_front_addr":"172.21.15.92:6817/115823","state":["exists","up"]},{"osd":1,"uuid":"13c1de73-1a8c-4d7e-8a28-9dae52dfe089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":115824},{"type":"v1","addr":"172.21.15.92:6812","nonce":115824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":115824},{"type":"v1","addr":"172.21.15.92:6816","nonce":115824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":115824},{"type":"v1","addr":"172.21.15.92:6823","nonce":115824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":115824},{"type":"v1","addr":"172.21.15.92:6820","nonce":115824}]},"public_addr":"172.21.15.92:6812/115824","cluster_addr":"172.21.15.92:6816/115824","heartbeat_back_addr":"172.21.15.92:6823/115824","heartbeat_front_addr":"172.21.15.92:6820/115824","state":["exists","up"]},{"osd":2,"uuid":"25788a55-112f-4f00-810d-2bc37cf76778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":115821},{"type":"v1","addr":"172.21.15.92:6801","nonce":115821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":115821},{"type":"v1","addr":"172.21.15.92:6803","nonce":115821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":115821},{"type":"v1","addr":"172.21.15.92:6807","nonce":115821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":115821},{"type":"v1","addr":"172.21.15.92:6805","nonce":115821}]},"public_addr":"172.21.15.92:6801/115821","cluster_addr":"172.21.15.92:6803/115821","heartbeat_back_addr":"172.21.15.92:6807/115821","heartbeat_front_addr":"172.21.15.92:6805/115821","state":["exists","up"]},{"osd":3,"uuid":"adf54e24-33a9-41fe-9ce3-98a5ab6c158e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":115825},{"type":"v1","addr":"172.21.15.92:6825","nonce":115825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":115825},{"type":"v1","addr":"172.21.15.92:6827","nonce":115825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":115825},{"type":"v1","addr":"172.21.15.92:6831","nonce":115825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":115825},{"type":"v1","addr":"172.21.15.92:6829","nonce":115825}]},"public_addr":"172.21.15.92:6825/115825","cluster_addr":"172.21.15.92:6827/115825","heartbeat_back_addr":"172.21.15.92:6831/115825","heartbeat_front_addr":"172.21.15.92:6829/115825","state":["exists","up"]},{"osd":4,"uuid":"ccb9b747-f399-44ea-9a18-9db3152227e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":116035},{"type":"v1","addr":"172.21.15.169:6817","nonce":116035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":116035},{"type":"v1","addr":"172.21.15.169:6819","nonce":116035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":116035},{"type":"v1","addr":"172.21.15.169:6826","nonce":116035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":116035},{"type":"v1","addr":"172.21.15.169:6822","nonce":116035}]},"public_addr":"172.21.15.169:6817/116035","cluster_addr":"172.21.15.169:6819/116035","heartbeat_back_addr":"172.21.15.169:6826/116035","heartbeat_front_addr":"172.21.15.169:6822/116035","state":["exists","up"]},{"osd":5,"uuid":"308a1e63-75aa-4d7d-8a7c-cecc24714fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":116033},{"type":"v1","addr":"172.21.15.169:6801","nonce":116033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":116033},{"type":"v1","addr":"172.21.15.169:6803","nonce":116033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":116033},{"type":"v1","addr":"172.21.15.169:6807","nonce":116033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":116033},{"type":"v1","addr":"172.21.15.169:6805","nonce":116033}]},"public_addr":"172.21.15.169:6801/116033","cluster_addr":"172.21.15.169:6803/116033","heartbeat_back_addr":"172.21.15.169:6807/116033","heartbeat_front_addr":"172.21.15.169:6805/116033","state":["exists","up"]},{"osd":6,"uuid":"9c459b7b-b821-4b4b-b91e-6aaae271457c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":116034},{"type":"v1","addr":"172.21.15.169:6809","nonce":116034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":116034},{"type":"v1","addr":"172.21.15.169:6811","nonce":116034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":116034},{"type":"v1","addr":"172.21.15.169:6815","nonce":116034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":116034},{"type":"v1","addr":"172.21.15.169:6813","nonce":116034}]},"public_addr":"172.21.15.169:6809/116034","cluster_addr":"172.21.15.169:6811/116034","heartbeat_back_addr":"172.21.15.169:6815/116034","heartbeat_front_addr":"172.21.15.169:6813/116034","state":["exists","up"]},{"osd":7,"uuid":"1ca9aa4a-755d-4e28-a937-4d8dccab4df3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":116044},{"type":"v1","addr":"172.21.15.169:6823","nonce":116044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6825","nonce":116044},{"type":"v1","addr":"172.21.15.169:6827","nonce":116044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":116044},{"type":"v1","addr":"172.21.15.169:6831","nonce":116044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":116044},{"type":"v1","addr":"172.21.15.169:6829","nonce":116044}]},"public_addr":"172.21.15.169:6823/116044","cluster_addr":"172.21.15.169:6827/116044","heartbeat_back_addr":"172.21.15.169:6831/116044","heartbeat_front_addr":"172.21.15.169:6829/116044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.586781+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.869778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.508594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.860493+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.872267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.746933+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.611520+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.871446+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.1d","mappings":[{"from":3,"to":2}]},{"pgid":"4.21","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:21:56.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.297+0000 7f6f015ff700 1 -- 172.21.15.92:0/3950503348 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6ee80682f0 msgr2=0x7f6ee806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:56.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.297+0000 7f6f015ff700 1 --2- 172.21.15.92:0/3950503348 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6ee80682f0 0x7f6ee806a7a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6eec007fd0 tx=0x7f6eec007de0 comp rx=0 tx=0).stop 2023-11-28T11:21:56.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.297+0000 7f6f015ff700 1 -- 172.21.15.92:0/3950503348 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6efc0d0d60 msgr2=0x7f6efc15a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:56.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.297+0000 7f6f015ff700 1 --2- 172.21.15.92:0/3950503348 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6efc0d0d60 0x7f6efc15a290 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4014040 tx=0x7f6ef4004ad0 comp rx=0 tx=0).stop 2023-11-28T11:21:56.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.297+0000 7f6f015ff700 1 -- 172.21.15.92:0/3950503348 shutdown_connections 2023-11-28T11:21:56.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.297+0000 7f6f015ff700 1 --2- 172.21.15.92:0/3950503348 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6ee80682f0 0x7f6ee806a7a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:56.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.297+0000 7f6f015ff700 1 --2- 172.21.15.92:0/3950503348 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6efc0d0d60 0x7f6efc15a290 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:56.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.297+0000 7f6f015ff700 1 --2- 172.21.15.92:0/3950503348 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6efc0b8510 0x7f6efc14d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:56.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.297+0000 7f6f015ff700 1 --2- 172.21.15.92:0/3950503348 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6efc0cd460 0x7f6efc159d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:56.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.297+0000 7f6f015ff700 1 -- 172.21.15.92:0/3950503348 >> 172.21.15.92:0/3950503348 conn(0x7f6efc0719a0 msgr2=0x7f6efc071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:56.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.298+0000 7f6f015ff700 1 -- 172.21.15.92:0/3950503348 shutdown_connections 2023-11-28T11:21:56.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.298+0000 7f6f015ff700 1 -- 172.21.15.92:0/3950503348 wait complete. 2023-11-28T11:21:56.320 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:20:41.925237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:20:48.682497+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:20:54.471398+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:20:55.204585+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T11:20:58.297307+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T11:21:56.321 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:21:56.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.518+0000 7fb3e97ff700 1 Processor -- start 2023-11-28T11:21:56.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.518+0000 7fb3e97ff700 1 -- start start 2023-11-28T11:21:56.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.519+0000 7fb3e97ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb3e40cd460 0x7fb3e406c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:56.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.519+0000 7fb3e97ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb3e40d0d60 0x7fb3e40d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:56.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.519+0000 7fb3e97ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3e40b88e0 con 0x7fb3e40b8510 2023-11-28T11:21:56.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.519+0000 7fb3e97ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3e40cdbd0 con 0x7fb3e40d0d60 2023-11-28T11:21:56.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.519+0000 7fb3e97ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3e40d02d0 con 0x7fb3e40cd460 2023-11-28T11:21:56.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.520+0000 7fb3e37fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb3e40d0d60 0x7fb3e40d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:56.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.520+0000 7fb3e37fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb3e40d0d60 0x7fb3e40d1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54180/0 (socket says 172.21.15.92:54180) 2023-11-28T11:21:56.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.520+0000 7fb3e37fe700 1 -- 172.21.15.92:0/20220531 learned_addr learned my addr 172.21.15.92:0/20220531 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:56.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.520+0000 7fb3dbfff700 1 --2- 172.21.15.92:0/20220531 >> v2:172.21.15.169:3300/0 conn(0x7fb3e40cd460 0x7fb3e406c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:56.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.520+0000 7fb3e37fe700 1 -- 172.21.15.92:0/20220531 >> v2:172.21.15.169:3300/0 conn(0x7fb3e40cd460 msgr2=0x7fb3e406c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:56.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.520+0000 7fb3e37fe700 1 --2- 172.21.15.92:0/20220531 >> v2:172.21.15.169:3300/0 conn(0x7fb3e40cd460 0x7fb3e406c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:56.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.520+0000 7fb3e37fe700 1 -- 172.21.15.92:0/20220531 >> v1:172.21.15.169:6789/0 conn(0x7fb3e40b8510 legacy=0x7fb3e406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:56.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.520+0000 7fb3e37fe700 1 -- 172.21.15.92:0/20220531 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e40cd8f0 con 0x7fb3e40d0d60 2023-11-28T11:21:56.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.520+0000 7fb3e37fe700 1 --2- 172.21.15.92:0/20220531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3e40d0d60 0x7fb3e40d1200 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb3e40ceba0 tx=0x7fb3dc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c3da27613f4ace9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:56.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.521+0000 7fb3e27fc700 1 -- 172.21.15.92:0/20220531 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc00e070 con 0x7fb3e40d0d60 2023-11-28T11:21:56.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.521+0000 7fb3e27fc700 1 -- 172.21.15.92:0/20220531 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3dc00b8d0 con 0x7fb3e40d0d60 2023-11-28T11:21:56.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.521+0000 7fb3e27fc700 1 -- 172.21.15.92:0/20220531 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc00bc80 con 0x7fb3e40d0d60 2023-11-28T11:21:56.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.521+0000 7fb3e97ff700 1 -- 172.21.15.92:0/20220531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3e40d0d60 msgr2=0x7fb3e40d1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:56.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.521+0000 7fb3e97ff700 1 --2- 172.21.15.92:0/20220531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3e40d0d60 0x7fb3e40d1200 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb3e40ceba0 tx=0x7fb3dc00b330 comp rx=0 tx=0).stop 2023-11-28T11:21:56.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.521+0000 7fb3e97ff700 1 -- 172.21.15.92:0/20220531 shutdown_connections 2023-11-28T11:21:56.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.521+0000 7fb3e97ff700 1 --2- 172.21.15.92:0/20220531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3e40d0d60 0x7fb3e40d1200 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:56.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.521+0000 7fb3e97ff700 1 --2- 172.21.15.92:0/20220531 >> v2:172.21.15.169:3300/0 conn(0x7fb3e40cd460 0x7fb3e406c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:56.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.521+0000 7fb3e97ff700 1 -- 172.21.15.92:0/20220531 >> 172.21.15.92:0/20220531 conn(0x7fb3e40719a0 msgr2=0x7fb3e4071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:56.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.521+0000 7fb3e97ff700 1 -- 172.21.15.92:0/20220531 shutdown_connections 2023-11-28T11:21:56.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.521+0000 7fb3e97ff700 1 -- 172.21.15.92:0/20220531 wait complete. 2023-11-28T11:21:56.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.523+0000 7fb3e97ff700 1 Processor -- start 2023-11-28T11:21:56.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.523+0000 7fb3e97ff700 1 -- start start 2023-11-28T11:21:56.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.524+0000 7fb3e97ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3e40b8510 0x7fb3e41e4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:56.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.524+0000 7fb3e97ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3e40cd460 0x7fb3e41e6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:56.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.524+0000 7fb3e97ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3e40d0d60 0x7fb3e41e78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:56.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.524+0000 7fb3e97ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e41e6210 con 0x7fb3e40b8510 2023-11-28T11:21:56.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.524+0000 7fb3e97ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e41e5590 con 0x7fb3e40d0d60 2023-11-28T11:21:56.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.524+0000 7fb3e97ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e41e5870 con 0x7fb3e40cd460 2023-11-28T11:21:56.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.524+0000 7fb3e2ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3e40b8510 0x7fb3e41e4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:56.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.524+0000 7fb3e2ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3e40b8510 0x7fb3e41e4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54196/0 (socket says 172.21.15.92:54196) 2023-11-28T11:21:56.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.524+0000 7fb3e2ffd700 1 -- 172.21.15.92:0/2856151142 learned_addr learned my addr 172.21.15.92:0/2856151142 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:56.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.525+0000 7fb3dbfff700 1 --2- 172.21.15.92:0/2856151142 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3e40cd460 0x7fb3e41e6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:56.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.525+0000 7fb3e37fe700 1 --2- 172.21.15.92:0/2856151142 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3e40d0d60 0x7fb3e41e78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:56.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.525+0000 7fb3e2ffd700 1 -- 172.21.15.92:0/2856151142 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3e40cd460 msgr2=0x7fb3e41e6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:56.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.525+0000 7fb3e2ffd700 1 --2- 172.21.15.92:0/2856151142 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3e40cd460 0x7fb3e41e6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:56.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.525+0000 7fb3e2ffd700 1 -- 172.21.15.92:0/2856151142 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3e40d0d60 msgr2=0x7fb3e41e78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:56.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.525+0000 7fb3e2ffd700 1 --2- 172.21.15.92:0/2856151142 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3e40d0d60 0x7fb3e41e78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:56.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.525+0000 7fb3e2ffd700 1 -- 172.21.15.92:0/2856151142 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3dc009b30 con 0x7fb3e40b8510 2023-11-28T11:21:56.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.525+0000 7fb3e2ffd700 1 --2- 172.21.15.92:0/2856151142 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3e40b8510 0x7fb3e41e4fb0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb3d400c490 tx=0x7fb3d400c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:56.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.525+0000 7fb3db7fe700 1 -- 172.21.15.92:0/2856151142 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d400cea0 con 0x7fb3e40b8510 2023-11-28T11:21:56.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.525+0000 7fb3db7fe700 1 -- 172.21.15.92:0/2856151142 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3d40052f0 con 0x7fb3e40b8510 2023-11-28T11:21:56.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.525+0000 7fb3e97ff700 1 -- 172.21.15.92:0/2856151142 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3e40cfff0 con 0x7fb3e40b8510 2023-11-28T11:21:56.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.525+0000 7fb3db7fe700 1 -- 172.21.15.92:0/2856151142 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d4006860 con 0x7fb3e40b8510 2023-11-28T11:21:56.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.526+0000 7fb3e97ff700 1 -- 172.21.15.92:0/2856151142 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3e40cfd10 con 0x7fb3e40b8510 2023-11-28T11:21:56.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.526+0000 7fb3e97ff700 1 -- 172.21.15.92:0/2856151142 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3e4067880 con 0x7fb3e40b8510 2023-11-28T11:21:56.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.527+0000 7fb3db7fe700 1 -- 172.21.15.92:0/2856151142 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb3d4010050 con 0x7fb3e40b8510 2023-11-28T11:21:56.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.528+0000 7fb3db7fe700 1 --2- 172.21.15.92:0/2856151142 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb3c40682f0 0x7fb3c406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:56.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.528+0000 7fb3db7fe700 1 -- 172.21.15.92:0/2856151142 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fb3d4085110 con 0x7fb3e40b8510 2023-11-28T11:21:56.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.528+0000 7fb3dbfff700 1 --2- 172.21.15.92:0/2856151142 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb3c40682f0 0x7fb3c406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:56.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.529+0000 7fb3dbfff700 1 --2- 172.21.15.92:0/2856151142 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb3c40682f0 0x7fb3c406a7a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb3e415ae80 tx=0x7fb3cc009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:56.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.532+0000 7fb3db7fe700 1 -- 172.21.15.92:0/2856151142 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb3d404a5e0 con 0x7fb3e40b8510 2023-11-28T11:21:56.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.721+0000 7fb3e97ff700 1 -- 172.21.15.92:0/2856151142 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb3e40d2940 con 0x7fb3e40b8510 2023-11-28T11:21:56.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.721+0000 7fb3db7fe700 1 -- 172.21.15.92:0/2856151142 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb3d405a020 con 0x7fb3e40b8510 2023-11-28T11:21:56.723 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-11-28T11:21:56.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.724+0000 7fb3e97ff700 1 -- 172.21.15.92:0/2856151142 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb3c40682f0 msgr2=0x7fb3c406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:56.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.724+0000 7fb3e97ff700 1 --2- 172.21.15.92:0/2856151142 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb3c40682f0 0x7fb3c406a7a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb3e415ae80 tx=0x7fb3cc009210 comp rx=0 tx=0).stop 2023-11-28T11:21:56.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.724+0000 7fb3e97ff700 1 -- 172.21.15.92:0/2856151142 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3e40b8510 msgr2=0x7fb3e41e4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:56.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.724+0000 7fb3e97ff700 1 --2- 172.21.15.92:0/2856151142 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3e40b8510 0x7fb3e41e4fb0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb3d400c490 tx=0x7fb3d400c7a0 comp rx=0 tx=0).stop 2023-11-28T11:21:56.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.724+0000 7fb3e97ff700 1 -- 172.21.15.92:0/2856151142 shutdown_connections 2023-11-28T11:21:56.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.724+0000 7fb3e97ff700 1 --2- 172.21.15.92:0/2856151142 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3e40d0d60 0x7fb3e41e78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:56.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.724+0000 7fb3e97ff700 1 --2- 172.21.15.92:0/2856151142 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb3c40682f0 0x7fb3c406a7a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:56.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.724+0000 7fb3e97ff700 1 --2- 172.21.15.92:0/2856151142 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3e40b8510 0x7fb3e41e4fb0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:56.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.724+0000 7fb3e97ff700 1 --2- 172.21.15.92:0/2856151142 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3e40cd460 0x7fb3e41e6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:56.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.724+0000 7fb3e97ff700 1 -- 172.21.15.92:0/2856151142 >> 172.21.15.92:0/2856151142 conn(0x7fb3e40719a0 msgr2=0x7fb3e40b6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:56.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.724+0000 7fb3e97ff700 1 -- 172.21.15.92:0/2856151142 shutdown_connections 2023-11-28T11:21:56.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.724+0000 7fb3e97ff700 1 -- 172.21.15.92:0/2856151142 wait complete. 2023-11-28T11:21:56.745 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:21:56.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.942+0000 7fbda29ff700 1 Processor -- start 2023-11-28T11:21:56.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.942+0000 7fbda29ff700 1 -- start start 2023-11-28T11:21:56.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.943+0000 7fbda29ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbd9c0b4fc0 0x7fbd9c0cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:56.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.943+0000 7fbda29ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd9c0d1950 con 0x7fbd9c0d2df0 2023-11-28T11:21:56.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.943+0000 7fbda29ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd9c0b2500 con 0x7fbd9c0b1c60 2023-11-28T11:21:56.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.943+0000 7fbda29ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd9c0d3d30 con 0x7fbd9c0b4fc0 2023-11-28T11:21:56.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.944+0000 7fbd9b7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbd9c0b4fc0 0x7fbd9c0cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:56.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.944+0000 7fbd9bfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fbd9c0d2df0 0x7fbd9c0d31c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55630/0 (socket says 172.21.15.92:55630) 2023-11-28T11:21:56.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.944+0000 7fbd9bfff700 1 -- 172.21.15.92:0/2166634055 learned_addr learned my addr 172.21.15.92:0/2166634055 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:56.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.944+0000 7fbd9b7fe700 1 -- 172.21.15.92:0/2166634055 >> v1:172.21.15.169:6789/0 conn(0x7fbd9c0b1c60 legacy=0x7fbd9c0cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:56.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.944+0000 7fbd9b7fe700 1 -- 172.21.15.92:0/2166634055 >> v1:172.21.15.92:6789/0 conn(0x7fbd9c0d2df0 legacy=0x7fbd9c0d31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:56.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.944+0000 7fbd9b7fe700 1 -- 172.21.15.92:0/2166634055 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd9c0d3a50 con 0x7fbd9c0b4fc0 2023-11-28T11:21:56.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.944+0000 7fbd9affd700 1 -- 172.21.15.92:0/2166634055 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3501930302 0 0) 0x7fbd9c0d1950 con 0x7fbd9c0d2df0 2023-11-28T11:21:56.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.944+0000 7fbd9b7fe700 1 --2- 172.21.15.92:0/2166634055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd9c0b4fc0 0x7fbd9c0cd620 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbd90009840 tx=0x7fbd90009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db5c7f69bdf38424 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:56.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.944+0000 7fbd9affd700 1 -- 172.21.15.92:0/2166634055 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd90011830 con 0x7fbd9c0b4fc0 2023-11-28T11:21:56.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.945+0000 7fbd9affd700 1 -- 172.21.15.92:0/2166634055 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd90011990 con 0x7fbd9c0b4fc0 2023-11-28T11:21:56.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.945+0000 7fbd9affd700 1 -- 172.21.15.92:0/2166634055 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd90011d20 con 0x7fbd9c0b4fc0 2023-11-28T11:21:56.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.945+0000 7fbda29ff700 1 -- 172.21.15.92:0/2166634055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd9c0b4fc0 msgr2=0x7fbd9c0cd620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:56.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.945+0000 7fbda29ff700 1 --2- 172.21.15.92:0/2166634055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd9c0b4fc0 0x7fbd9c0cd620 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbd90009840 tx=0x7fbd90009b50 comp rx=0 tx=0).stop 2023-11-28T11:21:56.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.945+0000 7fbda29ff700 1 -- 172.21.15.92:0/2166634055 shutdown_connections 2023-11-28T11:21:56.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.945+0000 7fbda29ff700 1 --2- 172.21.15.92:0/2166634055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd9c0b4fc0 0x7fbd9c0cd620 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:56.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.945+0000 7fbda29ff700 1 -- 172.21.15.92:0/2166634055 >> 172.21.15.92:0/2166634055 conn(0x7fbd9c0719a0 msgr2=0x7fbd9c071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:56.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.945+0000 7fbda29ff700 1 -- 172.21.15.92:0/2166634055 shutdown_connections 2023-11-28T11:21:56.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.945+0000 7fbda29ff700 1 -- 172.21.15.92:0/2166634055 wait complete. 2023-11-28T11:21:56.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.948+0000 7fbda29ff700 1 Processor -- start 2023-11-28T11:21:56.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.948+0000 7fbda29ff700 1 -- start start 2023-11-28T11:21:56.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.949+0000 7fbda29ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd9c0b1c60 0x7fbd9c0b4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:56.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.949+0000 7fbda29ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd9c0b4fc0 0x7fbd9c15ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:56.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.949+0000 7fbda29ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd9c0d2df0 0x7fbd9c14ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:56.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.949+0000 7fbda29ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9c15d770 con 0x7fbd9c0b4fc0 2023-11-28T11:21:56.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.949+0000 7fbda29ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9c15da50 con 0x7fbd9c0d2df0 2023-11-28T11:21:56.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.949+0000 7fbda29ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9c15dd30 con 0x7fbd9c0b1c60 2023-11-28T11:21:56.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.949+0000 7fbd9b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd9c0b4fc0 0x7fbd9c15ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:56.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.949+0000 7fbd9b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd9c0b4fc0 0x7fbd9c15ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54216/0 (socket says 172.21.15.92:54216) 2023-11-28T11:21:56.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.949+0000 7fbd9b7fe700 1 -- 172.21.15.92:0/507032161 learned_addr learned my addr 172.21.15.92:0/507032161 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:56.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.950+0000 7fbd9bfff700 1 --2- 172.21.15.92:0/507032161 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd9c0b1c60 0x7fbd9c0b4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:56.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.950+0000 7fbda0f9c700 1 --2- 172.21.15.92:0/507032161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd9c0d2df0 0x7fbd9c14ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:56.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.950+0000 7fbd9b7fe700 1 -- 172.21.15.92:0/507032161 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd9c0b1c60 msgr2=0x7fbd9c0b4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:56.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.950+0000 7fbd9b7fe700 1 --2- 172.21.15.92:0/507032161 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd9c0b1c60 0x7fbd9c0b4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:56.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.950+0000 7fbd9b7fe700 1 -- 172.21.15.92:0/507032161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd9c0d2df0 msgr2=0x7fbd9c14ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:56.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.950+0000 7fbd9b7fe700 1 --2- 172.21.15.92:0/507032161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd9c0d2df0 0x7fbd9c14ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:56.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.950+0000 7fbd9b7fe700 1 -- 172.21.15.92:0/507032161 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd900094a0 con 0x7fbd9c0b4fc0 2023-11-28T11:21:56.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.950+0000 7fbd9b7fe700 1 --2- 172.21.15.92:0/507032161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd9c0b4fc0 0x7fbd9c15ccb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbd90014040 tx=0x7fbd90004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:56.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.950+0000 7fbd98ff9700 1 -- 172.21.15.92:0/507032161 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9001a7e0 con 0x7fbd9c0b4fc0 2023-11-28T11:21:56.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.950+0000 7fbd98ff9700 1 -- 172.21.15.92:0/507032161 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd9001a940 con 0x7fbd9c0b4fc0 2023-11-28T11:21:56.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.950+0000 7fbda29ff700 1 -- 172.21.15.92:0/507032161 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd9c0b88f0 con 0x7fbd9c0b4fc0 2023-11-28T11:21:56.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.950+0000 7fbd98ff9700 1 -- 172.21.15.92:0/507032161 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9001acb0 con 0x7fbd9c0b4fc0 2023-11-28T11:21:56.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.950+0000 7fbda29ff700 1 -- 172.21.15.92:0/507032161 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd9c0d15e0 con 0x7fbd9c0b4fc0 2023-11-28T11:21:56.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.952+0000 7fbda29ff700 1 -- 172.21.15.92:0/507032161 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd9c0b21b0 con 0x7fbd9c0b4fc0 2023-11-28T11:21:56.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.952+0000 7fbd98ff9700 1 -- 172.21.15.92:0/507032161 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbd9002b030 con 0x7fbd9c0b4fc0 2023-11-28T11:21:56.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.954+0000 7fbd98ff9700 1 --2- 172.21.15.92:0/507032161 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fbd880682f0 0x7fbd8806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:56.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.954+0000 7fbd98ff9700 1 -- 172.21.15.92:0/507032161 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fbd90087360 con 0x7fbd9c0b4fc0 2023-11-28T11:21:56.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.954+0000 7fbd9bfff700 1 --2- 172.21.15.92:0/507032161 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fbd880682f0 0x7fbd8806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:56.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.955+0000 7fbd9bfff700 1 --2- 172.21.15.92:0/507032161 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fbd880682f0 0x7fbd8806a7a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c000f60 tx=0x7fbd8c007d90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:56.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:56.958+0000 7fbd98ff9700 1 -- 172.21.15.92:0/507032161 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbd900537a0 con 0x7fbd9c0b4fc0 2023-11-28T11:21:57.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.150+0000 7fbda29ff700 1 -- 172.21.15.92:0/507032161 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fbd9c067170 con 0x7fbd9c0b4fc0 2023-11-28T11:21:57.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.151+0000 7fbd98ff9700 1 -- 172.21.15.92:0/507032161 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7fbd900150a0 con 0x7fbd9c0b4fc0 2023-11-28T11:21:57.153 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-11-28T11:21:57.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.153+0000 7fbda29ff700 1 -- 172.21.15.92:0/507032161 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fbd880682f0 msgr2=0x7fbd8806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:57.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.153+0000 7fbda29ff700 1 --2- 172.21.15.92:0/507032161 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fbd880682f0 0x7fbd8806a7a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c000f60 tx=0x7fbd8c007d90 comp rx=0 tx=0).stop 2023-11-28T11:21:57.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.153+0000 7fbda29ff700 1 -- 172.21.15.92:0/507032161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd9c0b4fc0 msgr2=0x7fbd9c15ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:57.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.153+0000 7fbda29ff700 1 --2- 172.21.15.92:0/507032161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd9c0b4fc0 0x7fbd9c15ccb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbd90014040 tx=0x7fbd90004750 comp rx=0 tx=0).stop 2023-11-28T11:21:57.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.153+0000 7fbda29ff700 1 -- 172.21.15.92:0/507032161 shutdown_connections 2023-11-28T11:21:57.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.153+0000 7fbda29ff700 1 --2- 172.21.15.92:0/507032161 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbd9c0d2df0 0x7fbd9c14ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:57.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.153+0000 7fbda29ff700 1 --2- 172.21.15.92:0/507032161 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbd9c0b1c60 0x7fbd9c0b4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:57.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.153+0000 7fbda29ff700 1 --2- 172.21.15.92:0/507032161 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fbd880682f0 0x7fbd8806a7a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:57.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.153+0000 7fbda29ff700 1 --2- 172.21.15.92:0/507032161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd9c0b4fc0 0x7fbd9c15ccb0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:57.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.153+0000 7fbda29ff700 1 -- 172.21.15.92:0/507032161 >> 172.21.15.92:0/507032161 conn(0x7fbd9c0719a0 msgr2=0x7fbd9c0b7ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:57.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.153+0000 7fbda29ff700 1 -- 172.21.15.92:0/507032161 shutdown_connections 2023-11-28T11:21:57.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.153+0000 7fbda29ff700 1 -- 172.21.15.92:0/507032161 wait complete. 2023-11-28T11:21:57.173 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:21:57.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.369+0000 7f22657ff700 1 Processor -- start 2023-11-28T11:21:57.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.370+0000 7f22657ff700 1 -- start start 2023-11-28T11:21:57.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.371+0000 7f22657ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f22600b8510 0x7f226006b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:57.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.371+0000 7f22657ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22600d0d60 0x7f22600d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:57.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.371+0000 7f22657ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22600b88e0 con 0x7f22600cd460 2023-11-28T11:21:57.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.371+0000 7f22657ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22600cdbd0 con 0x7f22600b8510 2023-11-28T11:21:57.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.371+0000 7f22657ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22600d02d0 con 0x7f22600d0d60 2023-11-28T11:21:57.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.371+0000 7f225effd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f22600b8510 0x7f226006b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:57.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.371+0000 7f2257fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f22600cd460 0x7f226006beb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55640/0 (socket says 172.21.15.92:55640) 2023-11-28T11:21:57.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.371+0000 7f2257fff700 1 -- 172.21.15.92:0/407072702 learned_addr learned my addr 172.21.15.92:0/407072702 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:57.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.371+0000 7f225f7fe700 1 --2- 172.21.15.92:0/407072702 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22600d0d60 0x7f22600d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:57.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.371+0000 7f225effd700 1 -- 172.21.15.92:0/407072702 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22600d0d60 msgr2=0x7f22600d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:57.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.371+0000 7f225effd700 1 --2- 172.21.15.92:0/407072702 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22600d0d60 0x7f22600d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:57.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.371+0000 7f225effd700 1 -- 172.21.15.92:0/407072702 >> v1:172.21.15.92:6789/0 conn(0x7f22600cd460 legacy=0x7f226006beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:57.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.371+0000 7f225e7fc700 1 -- 172.21.15.92:0/407072702 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2842383373 0 0) 0x7f22600b88e0 con 0x7f22600cd460 2023-11-28T11:21:57.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.371+0000 7f225effd700 1 -- 172.21.15.92:0/407072702 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22600cd8f0 con 0x7f22600b8510 2023-11-28T11:21:57.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.371+0000 7f225f7fe700 1 --2- 172.21.15.92:0/407072702 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22600d0d60 0x7f22600d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:21:57.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.372+0000 7f225effd700 1 --2- 172.21.15.92:0/407072702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22600b8510 0x7f226006b970 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2250009870 tx=0x7f2250009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af894ee70113c505 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:57.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.372+0000 7f225e7fc700 1 -- 172.21.15.92:0/407072702 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2250011830 con 0x7f22600b8510 2023-11-28T11:21:57.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.372+0000 7f225e7fc700 1 -- 172.21.15.92:0/407072702 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2250011990 con 0x7f22600b8510 2023-11-28T11:21:57.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.372+0000 7f225e7fc700 1 -- 172.21.15.92:0/407072702 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2250011d20 con 0x7f22600b8510 2023-11-28T11:21:57.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.372+0000 7f22657ff700 1 -- 172.21.15.92:0/407072702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22600b8510 msgr2=0x7f226006b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:57.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.372+0000 7f22657ff700 1 --2- 172.21.15.92:0/407072702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22600b8510 0x7f226006b970 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2250009870 tx=0x7f2250009b80 comp rx=0 tx=0).stop 2023-11-28T11:21:57.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.373+0000 7f22657ff700 1 -- 172.21.15.92:0/407072702 shutdown_connections 2023-11-28T11:21:57.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.373+0000 7f22657ff700 1 --2- 172.21.15.92:0/407072702 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22600d0d60 0x7f22600d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:57.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.373+0000 7f22657ff700 1 --2- 172.21.15.92:0/407072702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22600b8510 0x7f226006b970 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:57.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.373+0000 7f22657ff700 1 -- 172.21.15.92:0/407072702 >> 172.21.15.92:0/407072702 conn(0x7f22600719a0 msgr2=0x7f2260071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:57.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.373+0000 7f22657ff700 1 -- 172.21.15.92:0/407072702 shutdown_connections 2023-11-28T11:21:57.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.373+0000 7f22657ff700 1 -- 172.21.15.92:0/407072702 wait complete. 2023-11-28T11:21:57.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.374+0000 7f22657ff700 1 Processor -- start 2023-11-28T11:21:57.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.375+0000 7f22657ff700 1 -- start start 2023-11-28T11:21:57.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.375+0000 7f22657ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22600b8510 0x7f22601e5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:57.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.375+0000 7f22657ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22600cd460 0x7f22601e6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:57.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.375+0000 7f22657ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22600d0d60 0x7f22601e69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:57.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.376+0000 7f22657ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22601e6f10 con 0x7f22600b8510 2023-11-28T11:21:57.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.376+0000 7f22657ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22601f2280 con 0x7f22600cd460 2023-11-28T11:21:57.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.376+0000 7f22657ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22601f2560 con 0x7f22600d0d60 2023-11-28T11:21:57.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.376+0000 7f225effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22600b8510 0x7f22601e5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:57.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.376+0000 7f225effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22600b8510 0x7f22601e5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54226/0 (socket says 172.21.15.92:54226) 2023-11-28T11:21:57.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.376+0000 7f225effd700 1 -- 172.21.15.92:0/1230590255 learned_addr learned my addr 172.21.15.92:0/1230590255 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:57.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.376+0000 7f225effd700 1 -- 172.21.15.92:0/1230590255 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22600d0d60 msgr2=0x7f22601e69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:57.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.376+0000 7f225effd700 1 --2- 172.21.15.92:0/1230590255 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22600d0d60 0x7f22601e69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:57.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.376+0000 7f225effd700 1 -- 172.21.15.92:0/1230590255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22600cd460 msgr2=0x7f22601e6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:57.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.376+0000 7f225effd700 1 --2- 172.21.15.92:0/1230590255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22600cd460 0x7f22601e6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:57.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.376+0000 7f225effd700 1 -- 172.21.15.92:0/1230590255 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22500094a0 con 0x7f22600b8510 2023-11-28T11:21:57.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.376+0000 7f225effd700 1 --2- 172.21.15.92:0/1230590255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22600b8510 0x7f22601e5b60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2250009bc0 tx=0x7f2250004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:57.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.377+0000 7f22577fe700 1 -- 172.21.15.92:0/1230590255 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22500036a0 con 0x7f22600b8510 2023-11-28T11:21:57.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.377+0000 7f22577fe700 1 -- 172.21.15.92:0/1230590255 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f225001f070 con 0x7f22600b8510 2023-11-28T11:21:57.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.377+0000 7f22657ff700 1 -- 172.21.15.92:0/1230590255 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22600cfff0 con 0x7f22600b8510 2023-11-28T11:21:57.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.377+0000 7f22577fe700 1 -- 172.21.15.92:0/1230590255 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2250003a60 con 0x7f22600b8510 2023-11-28T11:21:57.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.377+0000 7f22657ff700 1 -- 172.21.15.92:0/1230590255 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22600cfd10 con 0x7f22600b8510 2023-11-28T11:21:57.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.378+0000 7f22657ff700 1 -- 172.21.15.92:0/1230590255 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2260067880 con 0x7f22600b8510 2023-11-28T11:21:57.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.379+0000 7f22577fe700 1 -- 172.21.15.92:0/1230590255 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f225000f050 con 0x7f22600b8510 2023-11-28T11:21:57.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.379+0000 7f22577fe700 1 --2- 172.21.15.92:0/1230590255 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f22400682f0 0x7f224006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:57.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.380+0000 7f22577fe700 1 -- 172.21.15.92:0/1230590255 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f2250021070 con 0x7f22600b8510 2023-11-28T11:21:57.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.380+0000 7f2257fff700 1 --2- 172.21.15.92:0/1230590255 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f22400682f0 0x7f224006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:57.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.380+0000 7f2257fff700 1 --2- 172.21.15.92:0/1230590255 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f22400682f0 0x7f224006a7a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f22480011e0 tx=0x7f22480079b0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:57.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.384+0000 7f22577fe700 1 -- 172.21.15.92:0/1230590255 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2250053e30 con 0x7f22600b8510 2023-11-28T11:21:57.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.575+0000 7f22657ff700 1 -- 172.21.15.92:0/1230590255 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f22600d2940 con 0x7f22600b8510 2023-11-28T11:21:57.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.575+0000 7f22577fe700 1 -- 172.21.15.92:0/1230590255 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2250057450 con 0x7f22600b8510 2023-11-28T11:21:57.577 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-11-28T11:21:57.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.578+0000 7f22657ff700 1 -- 172.21.15.92:0/1230590255 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f22400682f0 msgr2=0x7f224006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:57.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.578+0000 7f22657ff700 1 --2- 172.21.15.92:0/1230590255 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f22400682f0 0x7f224006a7a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f22480011e0 tx=0x7f22480079b0 comp rx=0 tx=0).stop 2023-11-28T11:21:57.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.578+0000 7f22657ff700 1 -- 172.21.15.92:0/1230590255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22600b8510 msgr2=0x7f22601e5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:57.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.578+0000 7f22657ff700 1 --2- 172.21.15.92:0/1230590255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22600b8510 0x7f22601e5b60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2250009bc0 tx=0x7f2250004750 comp rx=0 tx=0).stop 2023-11-28T11:21:57.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.578+0000 7f22657ff700 1 -- 172.21.15.92:0/1230590255 shutdown_connections 2023-11-28T11:21:57.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.578+0000 7f22657ff700 1 --2- 172.21.15.92:0/1230590255 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22600d0d60 0x7f22601e69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:57.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.578+0000 7f22657ff700 1 --2- 172.21.15.92:0/1230590255 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f22400682f0 0x7f224006a7a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:57.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.578+0000 7f22657ff700 1 --2- 172.21.15.92:0/1230590255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22600b8510 0x7f22601e5b60 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:57.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.578+0000 7f22657ff700 1 --2- 172.21.15.92:0/1230590255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22600cd460 0x7f22601e6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:57.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.578+0000 7f22657ff700 1 -- 172.21.15.92:0/1230590255 >> 172.21.15.92:0/1230590255 conn(0x7f22600719a0 msgr2=0x7f22600cccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:57.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.578+0000 7f22657ff700 1 -- 172.21.15.92:0/1230590255 shutdown_connections 2023-11-28T11:21:57.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.578+0000 7f22657ff700 1 -- 172.21.15.92:0/1230590255 wait complete. 2023-11-28T11:21:57.598 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:21:57.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.796+0000 7fe2c41ff700 1 Processor -- start 2023-11-28T11:21:57.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.796+0000 7fe2c41ff700 1 -- start start 2023-11-28T11:21:57.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.798+0000 7fe2c41ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe2bc0cd460 0x7fe2bc06c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:57.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.798+0000 7fe2c41ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2bc0d0d60 0x7fe2bc0d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:57.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.798+0000 7fe2c41ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2bc0b88e0 con 0x7fe2bc0b8510 2023-11-28T11:21:57.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.798+0000 7fe2c41ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2bc0cdbd0 con 0x7fe2bc0cd460 2023-11-28T11:21:57.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.798+0000 7fe2c41ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc0d02d0 con 0x7fe2bc0d0d60 2023-11-28T11:21:57.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.798+0000 7fe2c279c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2bc0d0d60 0x7fe2bc0d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:57.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.798+0000 7fe2c279c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2bc0d0d60 0x7fe2bc0d1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.92:48526/0 (socket says 172.21.15.92:48526) 2023-11-28T11:21:57.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.798+0000 7fe2c279c700 1 -- 172.21.15.92:0/3166939149 learned_addr learned my addr 172.21.15.92:0/3166939149 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:57.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.798+0000 7fe2c179a700 1 --2- 172.21.15.92:0/3166939149 >> v2:172.21.15.169:3300/0 conn(0x7fe2bc0cd460 0x7fe2bc06c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:57.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.799+0000 7fe2c0f99700 1 -- 172.21.15.92:0/3166939149 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 60161865 0 0) 0x7fe2bc0b88e0 con 0x7fe2bc0b8510 2023-11-28T11:21:57.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.799+0000 7fe2c279c700 1 -- 172.21.15.92:0/3166939149 >> v2:172.21.15.169:3300/0 conn(0x7fe2bc0cd460 msgr2=0x7fe2bc06c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:57.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.799+0000 7fe2c279c700 1 --2- 172.21.15.92:0/3166939149 >> v2:172.21.15.169:3300/0 conn(0x7fe2bc0cd460 0x7fe2bc06c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:57.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.799+0000 7fe2c279c700 1 -- 172.21.15.92:0/3166939149 >> v1:172.21.15.169:6789/0 conn(0x7fe2bc0b8510 legacy=0x7fe2bc06b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:57.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.799+0000 7fe2c279c700 1 -- 172.21.15.92:0/3166939149 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2bc0cd8f0 con 0x7fe2bc0d0d60 2023-11-28T11:21:57.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.799+0000 7fe2c279c700 1 --2- 172.21.15.92:0/3166939149 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2bc0d0d60 0x7fe2bc0d1200 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4009e50 tx=0x7fe2b400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cee0e40ce1c132d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:57.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.799+0000 7fe2c0f99700 1 -- 172.21.15.92:0/3166939149 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b400e070 con 0x7fe2bc0d0d60 2023-11-28T11:21:57.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.799+0000 7fe2c0f99700 1 -- 172.21.15.92:0/3166939149 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b400b910 con 0x7fe2bc0d0d60 2023-11-28T11:21:57.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.799+0000 7fe2c0f99700 1 -- 172.21.15.92:0/3166939149 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b400bcc0 con 0x7fe2bc0d0d60 2023-11-28T11:21:57.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.800+0000 7fe2c41ff700 1 -- 172.21.15.92:0/3166939149 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2bc0d0d60 msgr2=0x7fe2bc0d1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:57.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.800+0000 7fe2c41ff700 1 --2- 172.21.15.92:0/3166939149 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2bc0d0d60 0x7fe2bc0d1200 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4009e50 tx=0x7fe2b400b330 comp rx=0 tx=0).stop 2023-11-28T11:21:57.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.800+0000 7fe2c41ff700 1 -- 172.21.15.92:0/3166939149 shutdown_connections 2023-11-28T11:21:57.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.800+0000 7fe2c41ff700 1 --2- 172.21.15.92:0/3166939149 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2bc0d0d60 0x7fe2bc0d1200 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:57.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.800+0000 7fe2c41ff700 1 --2- 172.21.15.92:0/3166939149 >> v2:172.21.15.169:3300/0 conn(0x7fe2bc0cd460 0x7fe2bc06c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:57.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.800+0000 7fe2c41ff700 1 -- 172.21.15.92:0/3166939149 >> 172.21.15.92:0/3166939149 conn(0x7fe2bc0719a0 msgr2=0x7fe2bc071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:57.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.800+0000 7fe2c41ff700 1 -- 172.21.15.92:0/3166939149 shutdown_connections 2023-11-28T11:21:57.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.800+0000 7fe2c41ff700 1 -- 172.21.15.92:0/3166939149 wait complete. 2023-11-28T11:21:57.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.802+0000 7fe2c41ff700 1 Processor -- start 2023-11-28T11:21:57.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.802+0000 7fe2c41ff700 1 -- start start 2023-11-28T11:21:57.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.803+0000 7fe2c41ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2bc0b8510 0x7fe2bc1581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:57.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.803+0000 7fe2c41ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2bc0cd460 0x7fe2bc1e4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:57.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.803+0000 7fe2c41ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2bc0d0d60 0x7fe2bc1e73f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:57.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.803+0000 7fe2c41ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc1e56a0 con 0x7fe2bc0b8510 2023-11-28T11:21:57.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.803+0000 7fe2c41ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc1e5980 con 0x7fe2bc0d0d60 2023-11-28T11:21:57.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.803+0000 7fe2c41ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc1e5c60 con 0x7fe2bc0cd460 2023-11-28T11:21:57.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.803+0000 7fe2c1f9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2bc0b8510 0x7fe2bc1581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:57.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.803+0000 7fe2c1f9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2bc0b8510 0x7fe2bc1581d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54236/0 (socket says 172.21.15.92:54236) 2023-11-28T11:21:57.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.803+0000 7fe2c1f9b700 1 -- 172.21.15.92:0/1861598694 learned_addr learned my addr 172.21.15.92:0/1861598694 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:57.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.803+0000 7fe2c1f9b700 1 -- 172.21.15.92:0/1861598694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2bc0cd460 msgr2=0x7fe2bc1e4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:57.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.803+0000 7fe2c1f9b700 1 --2- 172.21.15.92:0/1861598694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2bc0cd460 0x7fe2bc1e4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:57.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.803+0000 7fe2c1f9b700 1 -- 172.21.15.92:0/1861598694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2bc0d0d60 msgr2=0x7fe2bc1e73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:57.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.803+0000 7fe2c1f9b700 1 --2- 172.21.15.92:0/1861598694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2bc0d0d60 0x7fe2bc1e73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:57.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.803+0000 7fe2c1f9b700 1 -- 172.21.15.92:0/1861598694 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b4009b30 con 0x7fe2bc0b8510 2023-11-28T11:21:57.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.804+0000 7fe2c1f9b700 1 --2- 172.21.15.92:0/1861598694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2bc0b8510 0x7fe2bc1581d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe2b80011e0 tx=0x7fe2b800cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:57.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.804+0000 7fe2b27fc700 1 -- 172.21.15.92:0/1861598694 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b80054e0 con 0x7fe2bc0b8510 2023-11-28T11:21:57.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.804+0000 7fe2b27fc700 1 -- 172.21.15.92:0/1861598694 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b8003680 con 0x7fe2bc0b8510 2023-11-28T11:21:57.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.804+0000 7fe2b27fc700 1 -- 172.21.15.92:0/1861598694 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b8003a30 con 0x7fe2bc0b8510 2023-11-28T11:21:57.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.804+0000 7fe2c41ff700 1 -- 172.21.15.92:0/1861598694 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2bc0cfff0 con 0x7fe2bc0b8510 2023-11-28T11:21:57.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.804+0000 7fe2c41ff700 1 -- 172.21.15.92:0/1861598694 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2bc0cfd10 con 0x7fe2bc0b8510 2023-11-28T11:21:57.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.805+0000 7fe2c41ff700 1 -- 172.21.15.92:0/1861598694 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2bc067880 con 0x7fe2bc0b8510 2023-11-28T11:21:57.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.806+0000 7fe2b27fc700 1 -- 172.21.15.92:0/1861598694 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe2b8019030 con 0x7fe2bc0b8510 2023-11-28T11:21:57.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.807+0000 7fe2b27fc700 1 --2- 172.21.15.92:0/1861598694 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe2a00682f0 0x7fe2a006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:57.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.807+0000 7fe2b27fc700 1 -- 172.21.15.92:0/1861598694 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fe2b80854a0 con 0x7fe2bc0b8510 2023-11-28T11:21:57.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.807+0000 7fe2c179a700 1 --2- 172.21.15.92:0/1861598694 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe2a00682f0 0x7fe2a006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:57.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.808+0000 7fe2c179a700 1 --2- 172.21.15.92:0/1861598694 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe2a00682f0 0x7fe2a006a7a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac006b20 tx=0x7fe2ac006ab0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:57.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:57.811+0000 7fe2b27fc700 1 -- 172.21.15.92:0/1861598694 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe2b80518e0 con 0x7fe2bc0b8510 2023-11-28T11:21:58.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.001+0000 7fe2c41ff700 1 -- 172.21.15.92:0/1861598694 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe2bc0d2940 con 0x7fe2bc0b8510 2023-11-28T11:21:58.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.001+0000 7fe2b27fc700 1 -- 172.21.15.92:0/1861598694 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe2b80170a0 con 0x7fe2bc0b8510 2023-11-28T11:21:58.003 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-11-28T11:21:58.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.004+0000 7fe2c41ff700 1 -- 172.21.15.92:0/1861598694 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe2a00682f0 msgr2=0x7fe2a006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:58.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.004+0000 7fe2c41ff700 1 --2- 172.21.15.92:0/1861598694 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe2a00682f0 0x7fe2a006a7a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac006b20 tx=0x7fe2ac006ab0 comp rx=0 tx=0).stop 2023-11-28T11:21:58.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.004+0000 7fe2c41ff700 1 -- 172.21.15.92:0/1861598694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2bc0b8510 msgr2=0x7fe2bc1581d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:58.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.004+0000 7fe2c41ff700 1 --2- 172.21.15.92:0/1861598694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2bc0b8510 0x7fe2bc1581d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe2b80011e0 tx=0x7fe2b800cae0 comp rx=0 tx=0).stop 2023-11-28T11:21:58.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.004+0000 7fe2c41ff700 1 -- 172.21.15.92:0/1861598694 shutdown_connections 2023-11-28T11:21:58.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.004+0000 7fe2c41ff700 1 --2- 172.21.15.92:0/1861598694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe2bc0d0d60 0x7fe2bc1e73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:58.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.004+0000 7fe2c41ff700 1 --2- 172.21.15.92:0/1861598694 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe2a00682f0 0x7fe2a006a7a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:58.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.004+0000 7fe2c41ff700 1 --2- 172.21.15.92:0/1861598694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2bc0b8510 0x7fe2bc1581d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:58.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.004+0000 7fe2c41ff700 1 --2- 172.21.15.92:0/1861598694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe2bc0cd460 0x7fe2bc1e4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:58.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.004+0000 7fe2c41ff700 1 -- 172.21.15.92:0/1861598694 >> 172.21.15.92:0/1861598694 conn(0x7fe2bc0719a0 msgr2=0x7fe2bc0b4180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:58.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.004+0000 7fe2c41ff700 1 -- 172.21.15.92:0/1861598694 shutdown_connections 2023-11-28T11:21:58.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.004+0000 7fe2c41ff700 1 -- 172.21.15.92:0/1861598694 wait complete. 2023-11-28T11:21:58.024 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T11:21:58.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.220+0000 7f0e1fbff700 1 Processor -- start 2023-11-28T11:21:58.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.221+0000 7f0e1fbff700 1 -- start start 2023-11-28T11:21:58.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.222+0000 7f0e1fbff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0e180cd460 0x7f0e1806c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:58.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.222+0000 7f0e1fbff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0e180d0d60 0x7f0e180d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:58.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.222+0000 7f0e1fbff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e180b88e0 con 0x7f0e180b8510 2023-11-28T11:21:58.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.222+0000 7f0e1fbff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e180cdbd0 con 0x7f0e180d0d60 2023-11-28T11:21:58.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.222+0000 7f0e1fbff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e180d02d0 con 0x7f0e180cd460 2023-11-28T11:21:58.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.222+0000 7f0e1e19c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0e180d0d60 0x7f0e180d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:58.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.222+0000 7f0e1e19c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0e180d0d60 0x7f0e180d1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54238/0 (socket says 172.21.15.92:54238) 2023-11-28T11:21:58.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.222+0000 7f0e1e19c700 1 -- 172.21.15.92:0/3839977278 learned_addr learned my addr 172.21.15.92:0/3839977278 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:58.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.222+0000 7f0e1d19a700 1 --2- 172.21.15.92:0/3839977278 >> v2:172.21.15.169:3300/0 conn(0x7f0e180cd460 0x7f0e1806c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:58.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.222+0000 7f0e1e19c700 1 -- 172.21.15.92:0/3839977278 >> v2:172.21.15.169:3300/0 conn(0x7f0e180cd460 msgr2=0x7f0e1806c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:58.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.222+0000 7f0e1e19c700 1 --2- 172.21.15.92:0/3839977278 >> v2:172.21.15.169:3300/0 conn(0x7f0e180cd460 0x7f0e1806c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:58.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.222+0000 7f0e1e19c700 1 -- 172.21.15.92:0/3839977278 >> v1:172.21.15.169:6789/0 conn(0x7f0e180b8510 legacy=0x7f0e1806b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:58.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.222+0000 7f0e1e19c700 1 -- 172.21.15.92:0/3839977278 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e180cd8f0 con 0x7f0e180d0d60 2023-11-28T11:21:58.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.222+0000 7f0e1d19a700 1 --2- 172.21.15.92:0/3839977278 >> v2:172.21.15.169:3300/0 conn(0x7f0e180cd460 0x7f0e1806c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:21:58.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.223+0000 7f0e1e19c700 1 --2- 172.21.15.92:0/3839977278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e180d0d60 0x7f0e180d1200 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0e180ceba0 tx=0x7f0e1000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=556d88a2be964658 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:58.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.223+0000 7f0e1c999700 1 -- 172.21.15.92:0/3839977278 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1000e070 con 0x7f0e180d0d60 2023-11-28T11:21:58.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.223+0000 7f0e1c999700 1 -- 172.21.15.92:0/3839977278 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e1000b980 con 0x7f0e180d0d60 2023-11-28T11:21:58.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.223+0000 7f0e1c999700 1 -- 172.21.15.92:0/3839977278 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1000bd30 con 0x7f0e180d0d60 2023-11-28T11:21:58.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.223+0000 7f0e1fbff700 1 -- 172.21.15.92:0/3839977278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e180d0d60 msgr2=0x7f0e180d1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:58.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.223+0000 7f0e1fbff700 1 --2- 172.21.15.92:0/3839977278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e180d0d60 0x7f0e180d1200 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0e180ceba0 tx=0x7f0e1000b330 comp rx=0 tx=0).stop 2023-11-28T11:21:58.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.223+0000 7f0e1fbff700 1 -- 172.21.15.92:0/3839977278 shutdown_connections 2023-11-28T11:21:58.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.223+0000 7f0e1fbff700 1 --2- 172.21.15.92:0/3839977278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e180d0d60 0x7f0e180d1200 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:58.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.224+0000 7f0e1fbff700 1 --2- 172.21.15.92:0/3839977278 >> v2:172.21.15.169:3300/0 conn(0x7f0e180cd460 0x7f0e1806c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:58.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.224+0000 7f0e1fbff700 1 -- 172.21.15.92:0/3839977278 >> 172.21.15.92:0/3839977278 conn(0x7f0e180719a0 msgr2=0x7f0e18071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:58.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.224+0000 7f0e1fbff700 1 -- 172.21.15.92:0/3839977278 shutdown_connections 2023-11-28T11:21:58.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.224+0000 7f0e1fbff700 1 -- 172.21.15.92:0/3839977278 wait complete. 2023-11-28T11:21:58.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.226+0000 7f0e1fbff700 1 Processor -- start 2023-11-28T11:21:58.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.226+0000 7f0e1fbff700 1 -- start start 2023-11-28T11:21:58.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.227+0000 7f0e1fbff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e180b8510 0x7f0e181e4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:58.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.227+0000 7f0e1fbff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e180cd460 0x7f0e181e6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:58.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.227+0000 7f0e1fbff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e180d0d60 0x7f0e181e78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:58.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.227+0000 7f0e1fbff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e181e6210 con 0x7f0e180b8510 2023-11-28T11:21:58.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.227+0000 7f0e1fbff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e181e5590 con 0x7f0e180d0d60 2023-11-28T11:21:58.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.227+0000 7f0e1fbff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e181e5870 con 0x7f0e180cd460 2023-11-28T11:21:58.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.228+0000 7f0e1d99b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e180b8510 0x7f0e181e4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:58.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.228+0000 7f0e1d99b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e180b8510 0x7f0e181e4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54248/0 (socket says 172.21.15.92:54248) 2023-11-28T11:21:58.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.228+0000 7f0e1d99b700 1 -- 172.21.15.92:0/2834579854 learned_addr learned my addr 172.21.15.92:0/2834579854 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:58.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.228+0000 7f0e1e19c700 1 --2- 172.21.15.92:0/2834579854 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e180d0d60 0x7f0e181e78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:58.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.228+0000 7f0e1d19a700 1 --2- 172.21.15.92:0/2834579854 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e180cd460 0x7f0e181e6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:58.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.228+0000 7f0e1d99b700 1 -- 172.21.15.92:0/2834579854 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e180cd460 msgr2=0x7f0e181e6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:58.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.228+0000 7f0e1d99b700 1 --2- 172.21.15.92:0/2834579854 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e180cd460 0x7f0e181e6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:58.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.228+0000 7f0e1d99b700 1 -- 172.21.15.92:0/2834579854 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e180d0d60 msgr2=0x7f0e181e78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:58.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.228+0000 7f0e1d99b700 1 --2- 172.21.15.92:0/2834579854 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e180d0d60 0x7f0e181e78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:58.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.228+0000 7f0e1d99b700 1 -- 172.21.15.92:0/2834579854 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e10009b30 con 0x7f0e180b8510 2023-11-28T11:21:58.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.228+0000 7f0e1d19a700 1 --2- 172.21.15.92:0/2834579854 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e180cd460 0x7f0e181e6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:21:58.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.228+0000 7f0e1d99b700 1 --2- 172.21.15.92:0/2834579854 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e180b8510 0x7f0e181e4fb0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f0e1400c490 tx=0x7f0e1400c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:58.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.228+0000 7f0e0e7fc700 1 -- 172.21.15.92:0/2834579854 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e140052d0 con 0x7f0e180b8510 2023-11-28T11:21:58.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.228+0000 7f0e0e7fc700 1 -- 172.21.15.92:0/2834579854 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e14005450 con 0x7f0e180b8510 2023-11-28T11:21:58.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.228+0000 7f0e1fbff700 1 -- 172.21.15.92:0/2834579854 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e180cfff0 con 0x7f0e180b8510 2023-11-28T11:21:58.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.229+0000 7f0e0e7fc700 1 -- 172.21.15.92:0/2834579854 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e14006aa0 con 0x7f0e180b8510 2023-11-28T11:21:58.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.229+0000 7f0e1fbff700 1 -- 172.21.15.92:0/2834579854 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e180cfd10 con 0x7f0e180b8510 2023-11-28T11:21:58.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.230+0000 7f0e1fbff700 1 -- 172.21.15.92:0/2834579854 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e18067880 con 0x7f0e180b8510 2023-11-28T11:21:58.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.230+0000 7f0e0e7fc700 1 -- 172.21.15.92:0/2834579854 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0e14010050 con 0x7f0e180b8510 2023-11-28T11:21:58.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.231+0000 7f0e0e7fc700 1 --2- 172.21.15.92:0/2834579854 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f0dfc0682f0 0x7f0dfc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:58.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.231+0000 7f0e0e7fc700 1 -- 172.21.15.92:0/2834579854 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f0e140851a0 con 0x7f0e180b8510 2023-11-28T11:21:58.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.232+0000 7f0e1d19a700 1 --2- 172.21.15.92:0/2834579854 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f0dfc0682f0 0x7f0dfc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:58.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.232+0000 7f0e1d19a700 1 --2- 172.21.15.92:0/2834579854 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f0dfc0682f0 0x7f0dfc06a7a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0e1815ae80 tx=0x7f0e08008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:58.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.235+0000 7f0e0e7fc700 1 -- 172.21.15.92:0/2834579854 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0e1404a5e0 con 0x7f0e180b8510 2023-11-28T11:21:58.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.424+0000 7f0e1fbff700 1 -- 172.21.15.92:0/2834579854 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f0e180d2940 con 0x7f0e180b8510 2023-11-28T11:21:58.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.425+0000 7f0e0e7fc700 1 -- 172.21.15.92:0/2834579854 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7f0e140515e0 con 0x7f0e180b8510 2023-11-28T11:21:58.427 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-11-28T11:21:58.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.428+0000 7f0e1fbff700 1 -- 172.21.15.92:0/2834579854 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f0dfc0682f0 msgr2=0x7f0dfc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:58.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.428+0000 7f0e1fbff700 1 --2- 172.21.15.92:0/2834579854 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f0dfc0682f0 0x7f0dfc06a7a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0e1815ae80 tx=0x7f0e08008040 comp rx=0 tx=0).stop 2023-11-28T11:21:58.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.428+0000 7f0e1fbff700 1 -- 172.21.15.92:0/2834579854 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e180b8510 msgr2=0x7f0e181e4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:58.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.428+0000 7f0e1fbff700 1 --2- 172.21.15.92:0/2834579854 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e180b8510 0x7f0e181e4fb0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f0e1400c490 tx=0x7f0e1400c7a0 comp rx=0 tx=0).stop 2023-11-28T11:21:58.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.428+0000 7f0e1fbff700 1 -- 172.21.15.92:0/2834579854 shutdown_connections 2023-11-28T11:21:58.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.428+0000 7f0e1fbff700 1 --2- 172.21.15.92:0/2834579854 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0e180d0d60 0x7f0e181e78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:58.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.428+0000 7f0e1fbff700 1 --2- 172.21.15.92:0/2834579854 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f0dfc0682f0 0x7f0dfc06a7a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:58.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.428+0000 7f0e1fbff700 1 --2- 172.21.15.92:0/2834579854 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e180b8510 0x7f0e181e4fb0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:58.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.428+0000 7f0e1fbff700 1 --2- 172.21.15.92:0/2834579854 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0e180cd460 0x7f0e181e6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:58.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.428+0000 7f0e1fbff700 1 -- 172.21.15.92:0/2834579854 >> 172.21.15.92:0/2834579854 conn(0x7f0e180719a0 msgr2=0x7f0e180b6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:58.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.428+0000 7f0e1fbff700 1 -- 172.21.15.92:0/2834579854 shutdown_connections 2023-11-28T11:21:58.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.428+0000 7f0e1fbff700 1 -- 172.21.15.92:0/2834579854 wait complete. 2023-11-28T11:21:58.448 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T11:21:58.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.642+0000 7fc7b07ff700 1 Processor -- start 2023-11-28T11:21:58.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.642+0000 7fc7b07ff700 1 -- start start 2023-11-28T11:21:58.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.643+0000 7fc7b07ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7a80cd460 0x7fc7a806c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:58.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.643+0000 7fc7b07ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc7a80d0d60 0x7fc7a80d11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:58.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.644+0000 7fc7b07ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7a80b88e0 con 0x7fc7a80b8510 2023-11-28T11:21:58.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.644+0000 7fc7b07ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7a80cdbd0 con 0x7fc7a80d0d60 2023-11-28T11:21:58.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.644+0000 7fc7b07ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a80d02d0 con 0x7fc7a80cd460 2023-11-28T11:21:58.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.644+0000 7fc7ae59b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc7a80b8510 0x7fc7a806b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55648/0 (socket says 172.21.15.92:55648) 2023-11-28T11:21:58.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.644+0000 7fc7ae59b700 1 -- 172.21.15.92:0/1638395669 learned_addr learned my addr 172.21.15.92:0/1638395669 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:58.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.644+0000 7fc7aed9c700 1 --2- 172.21.15.92:0/1638395669 >> v2:172.21.15.169:3300/0 conn(0x7fc7a80d0d60 0x7fc7a80d11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:58.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.644+0000 7fc7add9a700 1 --2- 172.21.15.92:0/1638395669 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7a80cd460 0x7fc7a806c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:58.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.644+0000 7fc7ad599700 1 -- 172.21.15.92:0/1638395669 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 21516167 0 0) 0x7fc7a80b88e0 con 0x7fc7a80b8510 2023-11-28T11:21:58.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.644+0000 7fc7ad599700 1 -- 172.21.15.92:0/1638395669 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7940035b0 con 0x7fc7a80b8510 2023-11-28T11:21:58.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.645+0000 7fc7add9a700 1 -- 172.21.15.92:0/1638395669 >> v2:172.21.15.169:3300/0 conn(0x7fc7a80d0d60 msgr2=0x7fc7a80d11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:58.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.645+0000 7fc7add9a700 1 --2- 172.21.15.92:0/1638395669 >> v2:172.21.15.169:3300/0 conn(0x7fc7a80d0d60 0x7fc7a80d11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:58.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.645+0000 7fc7add9a700 1 -- 172.21.15.92:0/1638395669 >> v1:172.21.15.92:6789/0 conn(0x7fc7a80b8510 legacy=0x7fc7a806b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:58.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.645+0000 7fc7add9a700 1 -- 172.21.15.92:0/1638395669 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7a80cd8f0 con 0x7fc7a80cd460 2023-11-28T11:21:58.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.645+0000 7fc7aed9c700 1 --2- 172.21.15.92:0/1638395669 >> v2:172.21.15.169:3300/0 conn(0x7fc7a80d0d60 0x7fc7a80d11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:21:58.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.645+0000 7fc7add9a700 1 --2- 172.21.15.92:0/1638395669 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7a80cd460 0x7fc7a806c080 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc798009840 tx=0x7fc798009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c6cde482d381bbd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:58.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.645+0000 7fc7ad599700 1 -- 172.21.15.92:0/1638395669 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc798011830 con 0x7fc7a80cd460 2023-11-28T11:21:58.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.645+0000 7fc7ad599700 1 -- 172.21.15.92:0/1638395669 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc798011990 con 0x7fc7a80cd460 2023-11-28T11:21:58.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.645+0000 7fc7ad599700 1 -- 172.21.15.92:0/1638395669 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc798011d20 con 0x7fc7a80cd460 2023-11-28T11:21:58.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.646+0000 7fc7b07ff700 1 -- 172.21.15.92:0/1638395669 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7a80cd460 msgr2=0x7fc7a806c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:58.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.646+0000 7fc7b07ff700 1 --2- 172.21.15.92:0/1638395669 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7a80cd460 0x7fc7a806c080 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc798009840 tx=0x7fc798009b50 comp rx=0 tx=0).stop 2023-11-28T11:21:58.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.646+0000 7fc7b07ff700 1 -- 172.21.15.92:0/1638395669 shutdown_connections 2023-11-28T11:21:58.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.646+0000 7fc7b07ff700 1 --2- 172.21.15.92:0/1638395669 >> v2:172.21.15.169:3300/0 conn(0x7fc7a80d0d60 0x7fc7a80d11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:58.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.646+0000 7fc7b07ff700 1 --2- 172.21.15.92:0/1638395669 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7a80cd460 0x7fc7a806c080 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:58.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.646+0000 7fc7b07ff700 1 -- 172.21.15.92:0/1638395669 >> 172.21.15.92:0/1638395669 conn(0x7fc7a80719a0 msgr2=0x7fc7a8071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:58.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.646+0000 7fc7b07ff700 1 -- 172.21.15.92:0/1638395669 shutdown_connections 2023-11-28T11:21:58.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.646+0000 7fc7b07ff700 1 -- 172.21.15.92:0/1638395669 wait complete. 2023-11-28T11:21:58.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.648+0000 7fc7b07ff700 1 Processor -- start 2023-11-28T11:21:58.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.648+0000 7fc7b07ff700 1 -- start start 2023-11-28T11:21:58.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.649+0000 7fc7b07ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7a80b8510 0x7fc7a80cc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:58.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.649+0000 7fc7b07ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7a80cd460 0x7fc7a80ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:58.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.649+0000 7fc7b07ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7a80d0d60 0x7fc7a80c8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:58.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.649+0000 7fc7b07ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a80cae30 con 0x7fc7a80cd460 2023-11-28T11:21:58.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.649+0000 7fc7b07ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a80cb060 con 0x7fc7a80b8510 2023-11-28T11:21:58.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.649+0000 7fc7b07ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a80cb340 con 0x7fc7a80d0d60 2023-11-28T11:21:58.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.649+0000 7fc7add9a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7a80cd460 0x7fc7a80ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:58.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.649+0000 7fc7add9a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7a80cd460 0x7fc7a80ca8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54260/0 (socket says 172.21.15.92:54260) 2023-11-28T11:21:58.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.649+0000 7fc7add9a700 1 -- 172.21.15.92:0/4162017592 learned_addr learned my addr 172.21.15.92:0/4162017592 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:58.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.649+0000 7fc7ae59b700 1 --2- 172.21.15.92:0/4162017592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7a80b8510 0x7fc7a80cc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:58.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.649+0000 7fc7aed9c700 1 --2- 172.21.15.92:0/4162017592 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7a80d0d60 0x7fc7a80c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:58.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.649+0000 7fc7add9a700 1 -- 172.21.15.92:0/4162017592 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7a80d0d60 msgr2=0x7fc7a80c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:58.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.649+0000 7fc7add9a700 1 --2- 172.21.15.92:0/4162017592 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7a80d0d60 0x7fc7a80c8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:58.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.649+0000 7fc7add9a700 1 -- 172.21.15.92:0/4162017592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7a80b8510 msgr2=0x7fc7a80cc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:58.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.649+0000 7fc7add9a700 1 --2- 172.21.15.92:0/4162017592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7a80b8510 0x7fc7a80cc8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:58.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.649+0000 7fc7add9a700 1 -- 172.21.15.92:0/4162017592 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7a0009a70 con 0x7fc7a80cd460 2023-11-28T11:21:58.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.649+0000 7fc7add9a700 1 --2- 172.21.15.92:0/4162017592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7a80cd460 0x7fc7a80ca8f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc798009fd0 tx=0x7fc798004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:58.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.650+0000 7fc79effd700 1 -- 172.21.15.92:0/4162017592 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7980036a0 con 0x7fc7a80cd460 2023-11-28T11:21:58.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.650+0000 7fc79effd700 1 -- 172.21.15.92:0/4162017592 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc79801f070 con 0x7fc7a80cd460 2023-11-28T11:21:58.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.650+0000 7fc79effd700 1 -- 172.21.15.92:0/4162017592 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc798003a20 con 0x7fc7a80cd460 2023-11-28T11:21:58.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.650+0000 7fc7b07ff700 1 -- 172.21.15.92:0/4162017592 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7980094a0 con 0x7fc7a80cd460 2023-11-28T11:21:58.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.650+0000 7fc7b07ff700 1 -- 172.21.15.92:0/4162017592 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7a80cfff0 con 0x7fc7a80cd460 2023-11-28T11:21:58.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.651+0000 7fc7b07ff700 1 -- 172.21.15.92:0/4162017592 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7a80cfd10 con 0x7fc7a80cd460 2023-11-28T11:21:58.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.652+0000 7fc79effd700 1 -- 172.21.15.92:0/4162017592 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc79800f050 con 0x7fc7a80cd460 2023-11-28T11:21:58.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.653+0000 7fc79effd700 1 --2- 172.21.15.92:0/4162017592 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc78c0682f0 0x7fc78c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:58.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.653+0000 7fc79effd700 1 -- 172.21.15.92:0/4162017592 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fc798021070 con 0x7fc7a80cd460 2023-11-28T11:21:58.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.653+0000 7fc7ae59b700 1 --2- 172.21.15.92:0/4162017592 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc78c0682f0 0x7fc78c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:58.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.654+0000 7fc7ae59b700 1 --2- 172.21.15.92:0/4162017592 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc78c0682f0 0x7fc78c06a7a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc7a40011e0 tx=0x7fc7a4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:58.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.658+0000 7fc79effd700 1 -- 172.21.15.92:0/4162017592 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc798053ea0 con 0x7fc7a80cd460 2023-11-28T11:21:58.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.869+0000 7fc7b07ff700 1 -- 172.21.15.92:0/4162017592 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc7a8067880 con 0x7fc7a80cd460 2023-11-28T11:21:58.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.870+0000 7fc79effd700 1 -- 172.21.15.92:0/4162017592 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc7980574c0 con 0x7fc7a80cd460 2023-11-28T11:21:58.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.872+0000 7fc7b07ff700 1 -- 172.21.15.92:0/4162017592 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc78c0682f0 msgr2=0x7fc78c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:58.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.872+0000 7fc7b07ff700 1 --2- 172.21.15.92:0/4162017592 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc78c0682f0 0x7fc78c06a7a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc7a40011e0 tx=0x7fc7a4009040 comp rx=0 tx=0).stop 2023-11-28T11:21:58.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.872+0000 7fc7b07ff700 1 -- 172.21.15.92:0/4162017592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7a80cd460 msgr2=0x7fc7a80ca8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:58.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.872+0000 7fc7b07ff700 1 --2- 172.21.15.92:0/4162017592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7a80cd460 0x7fc7a80ca8f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc798009fd0 tx=0x7fc798004750 comp rx=0 tx=0).stop 2023-11-28T11:21:58.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.872+0000 7fc7b07ff700 1 -- 172.21.15.92:0/4162017592 shutdown_connections 2023-11-28T11:21:58.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.872+0000 7fc7b07ff700 1 --2- 172.21.15.92:0/4162017592 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc7a80d0d60 0x7fc7a80c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:58.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.872+0000 7fc7b07ff700 1 --2- 172.21.15.92:0/4162017592 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc7a80b8510 0x7fc7a80cc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:58.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.872+0000 7fc7b07ff700 1 --2- 172.21.15.92:0/4162017592 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc78c0682f0 0x7fc78c06a7a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:58.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.872+0000 7fc7b07ff700 1 --2- 172.21.15.92:0/4162017592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7a80cd460 0x7fc7a80ca8f0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:58.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.872+0000 7fc7b07ff700 1 -- 172.21.15.92:0/4162017592 >> 172.21.15.92:0/4162017592 conn(0x7fc7a80719a0 msgr2=0x7fc7a80d11e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:58.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.872+0000 7fc7b07ff700 1 -- 172.21.15.92:0/4162017592 shutdown_connections 2023-11-28T11:21:58.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:58.872+0000 7fc7b07ff700 1 -- 172.21.15.92:0/4162017592 wait complete. 2023-11-28T11:21:58.875 INFO:teuthology.orchestra.run.smithi092.stderr:listed 0 entries 2023-11-28T11:21:58.894 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-11-28T11:21:58.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:58.894 DEBUG:teuthology.orchestra.run.smithi169:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-11-28T11:21:58.978 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi169.front.sepia.ceph.com 2023-11-28T11:21:58.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:21:58.978 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-28T11:21:59.031 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-28T11:21:59.031 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:21:59.078 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-28T11:21:59.078 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:21:59.165 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-28T11:21:59.165 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:21:59.251 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:21:59.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.486+0000 7f7cb93ff700 1 Processor -- start 2023-11-28T11:21:59.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.486+0000 7f7cb93ff700 1 -- start start 2023-11-28T11:21:59.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.487+0000 7f7cb93ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7cb40b8510 0x7f7cb406b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:59.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.488+0000 7f7cb93ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7cb40cd460 0x7f7cb406beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:59.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.488+0000 7f7cb93ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cb40b88e0 con 0x7f7cb406c3f0 2023-11-28T11:21:59.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.488+0000 7f7cb93ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cb40cdbd0 con 0x7f7cb40cd460 2023-11-28T11:21:59.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.488+0000 7f7cb93ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb406c830 con 0x7f7cb40b8510 2023-11-28T11:21:59.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.488+0000 7f7cabfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7cb40cd460 0x7f7cb406beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:59.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.488+0000 7f7cabfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7cb40cd460 0x7f7cb406beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56312/0 (socket says 172.21.15.92:56312) 2023-11-28T11:21:59.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.488+0000 7f7cabfff700 1 -- 172.21.15.92:0/2233794860 learned_addr learned my addr 172.21.15.92:0/2233794860 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:59.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.488+0000 7f7cb2ffd700 1 --2- 172.21.15.92:0/2233794860 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7cb40b8510 0x7f7cb406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:59.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.488+0000 7f7cabfff700 1 -- 172.21.15.92:0/2233794860 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7cb40b8510 msgr2=0x7f7cb406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:59.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.488+0000 7f7cabfff700 1 --2- 172.21.15.92:0/2233794860 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7cb40b8510 0x7f7cb406b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:59.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.488+0000 7f7cabfff700 1 -- 172.21.15.92:0/2233794860 >> v1:172.21.15.169:6789/0 conn(0x7f7cb406c3f0 legacy=0x7f7cb415cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:59.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.488+0000 7f7cabfff700 1 -- 172.21.15.92:0/2233794860 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb40cd8f0 con 0x7f7cb40cd460 2023-11-28T11:21:59.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.489+0000 7f7cabfff700 1 --2- 172.21.15.92:0/2233794860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cb40cd460 0x7f7cb406beb0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c009920 tx=0x7f7c9c009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8307e95cecec79b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:59.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.489+0000 7f7cb27fc700 1 -- 172.21.15.92:0/2233794860 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c00ba20 con 0x7f7cb40cd460 2023-11-28T11:21:59.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.489+0000 7f7cb27fc700 1 -- 172.21.15.92:0/2233794860 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c9c00bb80 con 0x7f7cb40cd460 2023-11-28T11:21:59.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.489+0000 7f7cb27fc700 1 -- 172.21.15.92:0/2233794860 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c009e70 con 0x7f7cb40cd460 2023-11-28T11:21:59.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.489+0000 7f7cb93ff700 1 -- 172.21.15.92:0/2233794860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cb40cd460 msgr2=0x7f7cb406beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:59.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.489+0000 7f7cb93ff700 1 --2- 172.21.15.92:0/2233794860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cb40cd460 0x7f7cb406beb0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c009920 tx=0x7f7c9c009c30 comp rx=0 tx=0).stop 2023-11-28T11:21:59.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.490+0000 7f7cb93ff700 1 -- 172.21.15.92:0/2233794860 shutdown_connections 2023-11-28T11:21:59.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.490+0000 7f7cb93ff700 1 --2- 172.21.15.92:0/2233794860 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7cb40b8510 0x7f7cb406b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:59.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.490+0000 7f7cb93ff700 1 --2- 172.21.15.92:0/2233794860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cb40cd460 0x7f7cb406beb0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:59.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.490+0000 7f7cb93ff700 1 -- 172.21.15.92:0/2233794860 >> 172.21.15.92:0/2233794860 conn(0x7f7cb40719a0 msgr2=0x7f7cb4071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:59.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.490+0000 7f7cb93ff700 1 -- 172.21.15.92:0/2233794860 shutdown_connections 2023-11-28T11:21:59.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.490+0000 7f7cb93ff700 1 -- 172.21.15.92:0/2233794860 wait complete. 2023-11-28T11:21:59.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.491+0000 7f7cb93ff700 1 Processor -- start 2023-11-28T11:21:59.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.491+0000 7f7cb93ff700 1 -- start start 2023-11-28T11:21:59.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.492+0000 7f7cb93ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7cb406c3f0 0x7f7cb4155990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:59.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.492+0000 7f7cb93ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cb40b8510 0x7f7cb4159630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:59.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.492+0000 7f7cb93ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cb40cd460 0x7f7cb40b6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:59.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.492+0000 7f7cb93ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb415a110 con 0x7f7cb40b8510 2023-11-28T11:21:59.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.492+0000 7f7cb93ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb4155ed0 con 0x7f7cb40cd460 2023-11-28T11:21:59.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.492+0000 7f7cb93ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb41f3360 con 0x7f7cb406c3f0 2023-11-28T11:21:59.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.492+0000 7f7cabfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cb40b8510 0x7f7cb4159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:59.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.493+0000 7f7cabfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cb40b8510 0x7f7cb4159630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56328/0 (socket says 172.21.15.92:56328) 2023-11-28T11:21:59.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.493+0000 7f7cabfff700 1 -- 172.21.15.92:0/3524220766 learned_addr learned my addr 172.21.15.92:0/3524220766 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:59.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.493+0000 7f7cb2ffd700 1 --2- 172.21.15.92:0/3524220766 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7cb406c3f0 0x7f7cb4155990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:59.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.493+0000 7f7cb37fe700 1 --2- 172.21.15.92:0/3524220766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cb40cd460 0x7f7cb40b6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:59.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.493+0000 7f7cabfff700 1 -- 172.21.15.92:0/3524220766 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7cb406c3f0 msgr2=0x7f7cb4155990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:59.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.493+0000 7f7cabfff700 1 --2- 172.21.15.92:0/3524220766 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7cb406c3f0 0x7f7cb4155990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:59.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.493+0000 7f7cabfff700 1 -- 172.21.15.92:0/3524220766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cb40cd460 msgr2=0x7f7cb40b6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:59.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.493+0000 7f7cabfff700 1 --2- 172.21.15.92:0/3524220766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cb40cd460 0x7f7cb40b6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:59.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.493+0000 7f7cabfff700 1 -- 172.21.15.92:0/3524220766 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c9c009550 con 0x7f7cb40b8510 2023-11-28T11:21:59.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.493+0000 7f7cabfff700 1 --2- 172.21.15.92:0/3524220766 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cb40b8510 0x7f7cb4159630 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c009130 tx=0x7f7c9c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:59.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.493+0000 7f7cab7fe700 1 -- 172.21.15.92:0/3524220766 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c004550 con 0x7f7cb40b8510 2023-11-28T11:21:59.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.493+0000 7f7cab7fe700 1 -- 172.21.15.92:0/3524220766 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c9c01e070 con 0x7f7cb40b8510 2023-11-28T11:21:59.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.493+0000 7f7cb93ff700 1 -- 172.21.15.92:0/3524220766 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cb40cfff0 con 0x7f7cb40b8510 2023-11-28T11:21:59.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.494+0000 7f7cab7fe700 1 -- 172.21.15.92:0/3524220766 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c0038e0 con 0x7f7cb40b8510 2023-11-28T11:21:59.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.494+0000 7f7cb93ff700 1 -- 172.21.15.92:0/3524220766 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cb40cfd10 con 0x7f7cb40b8510 2023-11-28T11:21:59.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.495+0000 7f7cb93ff700 1 -- 172.21.15.92:0/3524220766 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cb41f34a0 con 0x7f7cb40b8510 2023-11-28T11:21:59.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.495+0000 7f7cab7fe700 1 -- 172.21.15.92:0/3524220766 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7c9c00f050 con 0x7f7cb40b8510 2023-11-28T11:21:59.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.496+0000 7f7cab7fe700 1 --2- 172.21.15.92:0/3524220766 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7c940682f0 0x7f7c9406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:59.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.496+0000 7f7cab7fe700 1 -- 172.21.15.92:0/3524220766 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f7c9c020070 con 0x7f7cb40b8510 2023-11-28T11:21:59.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.497+0000 7f7cb2ffd700 1 --2- 172.21.15.92:0/3524220766 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7c940682f0 0x7f7c9406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:59.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.497+0000 7f7cb2ffd700 1 --2- 172.21.15.92:0/3524220766 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7c940682f0 0x7f7c9406a7a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4156d30 tx=0x7f7ca4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:59.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.500+0000 7f7cab7fe700 1 -- 172.21.15.92:0/3524220766 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7cb41f34a0 con 0x7f7cb40b8510 2023-11-28T11:21:59.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.692+0000 7f7cb93ff700 1 -- 172.21.15.92:0/3524220766 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7cb41f34a0 con 0x7f7cb40b8510 2023-11-28T11:21:59.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.693+0000 7f7cab7fe700 1 -- 172.21.15.92:0/3524220766 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+19844 (secure 0 0 0) 0x7f7cb41f34a0 con 0x7f7cb40b8510 2023-11-28T11:21:59.694 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:21:59.694 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":29,"fsid":"68956e0c-c16f-47cd-a220-f98364b0831d","created":"2023-11-28T11:20:28.697573+0000","modified":"2023-11-28T11:21:37.638578+0000","last_up_change":"2023-11-28T11:20:41.257011+0000","last_in_change":"2023-11-28T11:20:32.249767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:20:41.925237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:20:48.682497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:20:54.471398+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:20:55.204585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:20:58.297307+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6abe483f-0e03-4a26-b9cf-62041291747e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":115823},{"type":"v1","addr":"172.21.15.92:6809","nonce":115823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":115823},{"type":"v1","addr":"172.21.15.92:6813","nonce":115823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":115823},{"type":"v1","addr":"172.21.15.92:6821","nonce":115823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":115823},{"type":"v1","addr":"172.21.15.92:6817","nonce":115823}]},"public_addr":"172.21.15.92:6809/115823","cluster_addr":"172.21.15.92:6813/115823","heartbeat_back_addr":"172.21.15.92:6821/115823","heartbeat_front_addr":"172.21.15.92:6817/115823","state":["exists","up"]},{"osd":1,"uuid":"13c1de73-1a8c-4d7e-8a28-9dae52dfe089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":115824},{"type":"v1","addr":"172.21.15.92:6812","nonce":115824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":115824},{"type":"v1","addr":"172.21.15.92:6816","nonce":115824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":115824},{"type":"v1","addr":"172.21.15.92:6823","nonce":115824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":115824},{"type":"v1","addr":"172.21.15.92:6820","nonce":115824}]},"public_addr":"172.21.15.92:6812/115824","cluster_addr":"172.21.15.92:6816/115824","heartbeat_back_addr":"172.21.15.92:6823/115824","heartbeat_front_addr":"172.21.15.92:6820/115824","state":["exists","up"]},{"osd":2,"uuid":"25788a55-112f-4f00-810d-2bc37cf76778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":115821},{"type":"v1","addr":"172.21.15.92:6801","nonce":115821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":115821},{"type":"v1","addr":"172.21.15.92:6803","nonce":115821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":115821},{"type":"v1","addr":"172.21.15.92:6807","nonce":115821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":115821},{"type":"v1","addr":"172.21.15.92:6805","nonce":115821}]},"public_addr":"172.21.15.92:6801/115821","cluster_addr":"172.21.15.92:6803/115821","heartbeat_back_addr":"172.21.15.92:6807/115821","heartbeat_front_addr":"172.21.15.92:6805/115821","state":["exists","up"]},{"osd":3,"uuid":"adf54e24-33a9-41fe-9ce3-98a5ab6c158e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":115825},{"type":"v1","addr":"172.21.15.92:6825","nonce":115825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":115825},{"type":"v1","addr":"172.21.15.92:6827","nonce":115825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":115825},{"type":"v1","addr":"172.21.15.92:6831","nonce":115825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":115825},{"type":"v1","addr":"172.21.15.92:6829","nonce":115825}]},"public_addr":"172.21.15.92:6825/115825","cluster_addr":"172.21.15.92:6827/115825","heartbeat_back_addr":"172.21.15.92:6831/115825","heartbeat_front_addr":"172.21.15.92:6829/115825","state":["exists","up"]},{"osd":4,"uuid":"ccb9b747-f399-44ea-9a18-9db3152227e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":116035},{"type":"v1","addr":"172.21.15.169:6817","nonce":116035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":116035},{"type":"v1","addr":"172.21.15.169:6819","nonce":116035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":116035},{"type":"v1","addr":"172.21.15.169:6826","nonce":116035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":116035},{"type":"v1","addr":"172.21.15.169:6822","nonce":116035}]},"public_addr":"172.21.15.169:6817/116035","cluster_addr":"172.21.15.169:6819/116035","heartbeat_back_addr":"172.21.15.169:6826/116035","heartbeat_front_addr":"172.21.15.169:6822/116035","state":["exists","up"]},{"osd":5,"uuid":"308a1e63-75aa-4d7d-8a7c-cecc24714fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":116033},{"type":"v1","addr":"172.21.15.169:6801","nonce":116033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":116033},{"type":"v1","addr":"172.21.15.169:6803","nonce":116033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":116033},{"type":"v1","addr":"172.21.15.169:6807","nonce":116033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":116033},{"type":"v1","addr":"172.21.15.169:6805","nonce":116033}]},"public_addr":"172.21.15.169:6801/116033","cluster_addr":"172.21.15.169:6803/116033","heartbeat_back_addr":"172.21.15.169:6807/116033","heartbeat_front_addr":"172.21.15.169:6805/116033","state":["exists","up"]},{"osd":6,"uuid":"9c459b7b-b821-4b4b-b91e-6aaae271457c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":116034},{"type":"v1","addr":"172.21.15.169:6809","nonce":116034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":116034},{"type":"v1","addr":"172.21.15.169:6811","nonce":116034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":116034},{"type":"v1","addr":"172.21.15.169:6815","nonce":116034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":116034},{"type":"v1","addr":"172.21.15.169:6813","nonce":116034}]},"public_addr":"172.21.15.169:6809/116034","cluster_addr":"172.21.15.169:6811/116034","heartbeat_back_addr":"172.21.15.169:6815/116034","heartbeat_front_addr":"172.21.15.169:6813/116034","state":["exists","up"]},{"osd":7,"uuid":"1ca9aa4a-755d-4e28-a937-4d8dccab4df3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":116044},{"type":"v1","addr":"172.21.15.169:6823","nonce":116044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6825","nonce":116044},{"type":"v1","addr":"172.21.15.169:6827","nonce":116044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":116044},{"type":"v1","addr":"172.21.15.169:6831","nonce":116044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":116044},{"type":"v1","addr":"172.21.15.169:6829","nonce":116044}]},"public_addr":"172.21.15.169:6823/116044","cluster_addr":"172.21.15.169:6827/116044","heartbeat_back_addr":"172.21.15.169:6831/116044","heartbeat_front_addr":"172.21.15.169:6829/116044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.586781+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.869778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.508594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.860493+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.872267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.746933+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.611520+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.871446+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.1d","mappings":[{"from":3,"to":2}]},{"pgid":"4.21","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:21:59.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.695+0000 7f7cb93ff700 1 -- 172.21.15.92:0/3524220766 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7c940682f0 msgr2=0x7f7c9406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:59.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.695+0000 7f7cb93ff700 1 --2- 172.21.15.92:0/3524220766 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7c940682f0 0x7f7c9406a7a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4156d30 tx=0x7f7ca4008040 comp rx=0 tx=0).stop 2023-11-28T11:21:59.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.695+0000 7f7cb93ff700 1 -- 172.21.15.92:0/3524220766 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cb40b8510 msgr2=0x7f7cb4159630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:59.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.695+0000 7f7cb93ff700 1 --2- 172.21.15.92:0/3524220766 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cb40b8510 0x7f7cb4159630 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c009130 tx=0x7f7c9c004750 comp rx=0 tx=0).stop 2023-11-28T11:21:59.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.695+0000 7f7cb93ff700 1 -- 172.21.15.92:0/3524220766 shutdown_connections 2023-11-28T11:21:59.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.695+0000 7f7cb93ff700 1 --2- 172.21.15.92:0/3524220766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7cb40cd460 0x7f7cb40b6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:59.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.695+0000 7f7cb93ff700 1 --2- 172.21.15.92:0/3524220766 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7cb406c3f0 0x7f7cb4155990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:59.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.695+0000 7f7cb93ff700 1 --2- 172.21.15.92:0/3524220766 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7c940682f0 0x7f7c9406a7a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:59.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.695+0000 7f7cb93ff700 1 --2- 172.21.15.92:0/3524220766 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cb40b8510 0x7f7cb4159630 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:59.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.695+0000 7f7cb93ff700 1 -- 172.21.15.92:0/3524220766 >> 172.21.15.92:0/3524220766 conn(0x7f7cb40719a0 msgr2=0x7f7cb40b3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:59.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.695+0000 7f7cb93ff700 1 -- 172.21.15.92:0/3524220766 shutdown_connections 2023-11-28T11:21:59.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.695+0000 7f7cb93ff700 1 -- 172.21.15.92:0/3524220766 wait complete. 2023-11-28T11:21:59.714 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:20:41.925237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:20:48.682497+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:20:54.471398+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:20:55.204585+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T11:20:58.297307+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T11:21:59.715 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:21:59.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.914+0000 7f48d13ff700 1 Processor -- start 2023-11-28T11:21:59.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.914+0000 7f48d13ff700 1 -- start start 2023-11-28T11:21:59.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.915+0000 7f48d13ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f48cc0b8510 0x7f48cc06b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:59.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.915+0000 7f48d13ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48cc0b88e0 con 0x7f48cc0cd460 2023-11-28T11:21:59.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.915+0000 7f48d13ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48cc0cd8f0 con 0x7f48cc0d0d60 2023-11-28T11:21:59.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.915+0000 7f48d13ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48cc0cdbd0 con 0x7f48cc0b8510 2023-11-28T11:21:59.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.915+0000 7f48c3fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f48cc0cd460 0x7f48cc06beb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55478/0 (socket says 172.21.15.92:55478) 2023-11-28T11:21:59.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.915+0000 7f48c3fff700 1 -- 172.21.15.92:0/514641016 learned_addr learned my addr 172.21.15.92:0/514641016 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:21:59.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.915+0000 7f48caffd700 1 --2- 172.21.15.92:0/514641016 >> v2:172.21.15.169:3300/0 conn(0x7f48cc0b8510 0x7f48cc06b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:59.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.916+0000 7f48ca7fc700 1 -- 172.21.15.92:0/514641016 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1015350610 0 0) 0x7f48cc0b88e0 con 0x7f48cc0cd460 2023-11-28T11:21:59.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.916+0000 7f48ca7fc700 1 -- 172.21.15.92:0/514641016 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48b80035b0 con 0x7f48cc0cd460 2023-11-28T11:21:59.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.916+0000 7f48ca7fc700 1 -- 172.21.15.92:0/514641016 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2773396783 0 0) 0x7f48cc0cd8f0 con 0x7f48cc0d0d60 2023-11-28T11:21:59.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.916+0000 7f48ca7fc700 1 -- 172.21.15.92:0/514641016 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48cc0b88e0 con 0x7f48cc0d0d60 2023-11-28T11:21:59.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.916+0000 7f48ca7fc700 1 -- 172.21.15.92:0/514641016 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 226387084 0 0) 0x7f48b80035b0 con 0x7f48cc0cd460 2023-11-28T11:21:59.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.916+0000 7f48ca7fc700 1 -- 172.21.15.92:0/514641016 >> v2:172.21.15.169:3300/0 conn(0x7f48cc0b8510 msgr2=0x7f48cc06b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:59.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.916+0000 7f48ca7fc700 1 --2- 172.21.15.92:0/514641016 >> v2:172.21.15.169:3300/0 conn(0x7f48cc0b8510 0x7f48cc06b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:59.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.916+0000 7f48ca7fc700 1 -- 172.21.15.92:0/514641016 >> v1:172.21.15.169:6789/0 conn(0x7f48cc0d0d60 legacy=0x7f48cc0d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:59.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.916+0000 7f48ca7fc700 1 -- 172.21.15.92:0/514641016 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48cc0cfd10 con 0x7f48cc0cd460 2023-11-28T11:21:59.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.916+0000 7f48ca7fc700 1 -- 172.21.15.92:0/514641016 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532161587 0 0) 0x7f48b4001670 con 0x7f48cc0cd460 2023-11-28T11:21:59.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.916+0000 7f48ca7fc700 1 -- 172.21.15.92:0/514641016 >> v1:172.21.15.92:6789/0 conn(0x7f48cc0cd460 legacy=0x7f48cc06beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:59.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.916+0000 7f48ca7fc700 1 --2- 172.21.15.92:0/514641016 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f48b8003be0 0x7f48b8003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:59.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.916+0000 7f48ca7fc700 1 --2- 172.21.15.92:0/514641016 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48b8004850 0x7f48b8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:59.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.916+0000 7f48ca7fc700 1 --2- 172.21.15.92:0/514641016 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48b8005200 0x7f48b800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:59.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.916+0000 7f48ca7fc700 1 -- 172.21.15.92:0/514641016 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48cc0cdbd0 con 0x7f48b8005200 2023-11-28T11:21:59.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.916+0000 7f48ca7fc700 1 -- 172.21.15.92:0/514641016 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b800c0e0 con 0x7f48b8004850 2023-11-28T11:21:59.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.916+0000 7f48ca7fc700 1 -- 172.21.15.92:0/514641016 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48b800c270 con 0x7f48b8003be0 2023-11-28T11:21:59.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.916+0000 7f48cb7fe700 1 --2- 172.21.15.92:0/514641016 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48b8005200 0x7f48b800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:59.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.916+0000 7f48c3fff700 1 --2- 172.21.15.92:0/514641016 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48b8004850 0x7f48b8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:59.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.916+0000 7f48caffd700 1 --2- 172.21.15.92:0/514641016 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f48b8003be0 0x7f48b8003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:59.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.917+0000 7f48cb7fe700 1 -- 172.21.15.92:0/514641016 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f48b8003be0 msgr2=0x7f48b8003fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:59.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.917+0000 7f48cb7fe700 1 --2- 172.21.15.92:0/514641016 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f48b8003be0 0x7f48b8003fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:59.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.917+0000 7f48cb7fe700 1 -- 172.21.15.92:0/514641016 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48b8004850 msgr2=0x7f48b8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:21:59.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.917+0000 7f48cb7fe700 1 --2- 172.21.15.92:0/514641016 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48b8004850 0x7f48b8004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:59.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.917+0000 7f48cb7fe700 1 -- 172.21.15.92:0/514641016 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b80035b0 con 0x7f48b8005200 2023-11-28T11:21:59.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.917+0000 7f48cb7fe700 1 -- 172.21.15.92:0/514641016 reap_dead start 2023-11-28T11:21:59.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.917+0000 7f48cb7fe700 1 --2- 172.21.15.92:0/514641016 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48b8005200 0x7f48b800ba10 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f48c4002230 tx=0x7f48b8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0310df1e854d7c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:59.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.917+0000 7f48ca7fc700 1 -- 172.21.15.92:0/514641016 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c4001f30 con 0x7f48b8005200 2023-11-28T11:21:59.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.917+0000 7f48ca7fc700 1 -- 172.21.15.92:0/514641016 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48c400ce60 con 0x7f48b8005200 2023-11-28T11:21:59.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.917+0000 7f48ca7fc700 1 -- 172.21.15.92:0/514641016 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c4003c40 con 0x7f48b8005200 2023-11-28T11:21:59.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.918+0000 7f48d13ff700 1 -- 172.21.15.92:0/514641016 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48b8005200 msgr2=0x7f48b800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:59.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.918+0000 7f48d13ff700 1 --2- 172.21.15.92:0/514641016 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48b8005200 0x7f48b800ba10 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f48c4002230 tx=0x7f48b8005880 comp rx=0 tx=0).stop 2023-11-28T11:21:59.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.918+0000 7f48d13ff700 1 -- 172.21.15.92:0/514641016 shutdown_connections 2023-11-28T11:21:59.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.918+0000 7f48d13ff700 1 --2- 172.21.15.92:0/514641016 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48b8005200 0x7f48b800ba10 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:59.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.918+0000 7f48d13ff700 1 -- 172.21.15.92:0/514641016 >> 172.21.15.92:0/514641016 conn(0x7f48cc0719a0 msgr2=0x7f48cc071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:21:59.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.918+0000 7f48d13ff700 1 -- 172.21.15.92:0/514641016 shutdown_connections 2023-11-28T11:21:59.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.918+0000 7f48d13ff700 1 -- 172.21.15.92:0/514641016 wait complete. 2023-11-28T11:21:59.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.920+0000 7f48d13ff700 1 Processor -- start 2023-11-28T11:21:59.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.920+0000 7f48d13ff700 1 -- start start 2023-11-28T11:21:59.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.920+0000 7f48d13ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48b8005200 0x7f48cc1e5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:59.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.921+0000 7f48d13ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f48cc1e4620 0x7f48cc1f1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:59.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.921+0000 7f48d13ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48cc0b6810 0x7f48cc0b6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:59.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.921+0000 7f48d13ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48cc1f1a60 con 0x7f48cc0b6810 2023-11-28T11:21:59.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.921+0000 7f48d13ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48cc0b71c0 con 0x7f48b8005200 2023-11-28T11:21:59.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.921+0000 7f48d13ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48cc1f1ce0 con 0x7f48cc1e4620 2023-11-28T11:21:59.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.921+0000 7f48cb7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48cc0b6810 0x7f48cc0b6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:59.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.921+0000 7f48cb7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48cc0b6810 0x7f48cc0b6c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56348/0 (socket says 172.21.15.92:56348) 2023-11-28T11:21:59.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.921+0000 7f48cb7fe700 1 -- 172.21.15.92:0/4099570664 learned_addr learned my addr 172.21.15.92:0/4099570664 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:21:59.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.921+0000 7f48caffd700 1 --2- 172.21.15.92:0/4099570664 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48b8005200 0x7f48cc1e5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:59.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.921+0000 7f48c3fff700 1 --2- 172.21.15.92:0/4099570664 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f48cc1e4620 0x7f48cc1f1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:59.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.921+0000 7f48cb7fe700 1 -- 172.21.15.92:0/4099570664 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f48cc1e4620 msgr2=0x7f48cc1f1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:59.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.921+0000 7f48cb7fe700 1 --2- 172.21.15.92:0/4099570664 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f48cc1e4620 0x7f48cc1f1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:59.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.921+0000 7f48cb7fe700 1 -- 172.21.15.92:0/4099570664 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48b8005200 msgr2=0x7f48cc1e5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:21:59.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.921+0000 7f48cb7fe700 1 --2- 172.21.15.92:0/4099570664 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48b8005200 0x7f48cc1e5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:21:59.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.921+0000 7f48cb7fe700 1 -- 172.21.15.92:0/4099570664 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48cc0d29c0 con 0x7f48cc0b6810 2023-11-28T11:21:59.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.921+0000 7f48c3fff700 1 --2- 172.21.15.92:0/4099570664 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f48cc1e4620 0x7f48cc1f1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:21:59.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.921+0000 7f48cb7fe700 1 --2- 172.21.15.92:0/4099570664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48cc0b6810 0x7f48cc0b6c80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f48c400c8b0 tx=0x7f48c4001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:59.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.922+0000 7f48c37fe700 1 -- 172.21.15.92:0/4099570664 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c4005280 con 0x7f48cc0b6810 2023-11-28T11:21:59.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.922+0000 7f48c37fe700 1 -- 172.21.15.92:0/4099570664 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48c40053e0 con 0x7f48cc0b6810 2023-11-28T11:21:59.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.922+0000 7f48c37fe700 1 -- 172.21.15.92:0/4099570664 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c4005790 con 0x7f48cc0b6810 2023-11-28T11:21:59.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.922+0000 7f48d13ff700 1 -- 172.21.15.92:0/4099570664 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48cc0cfff0 con 0x7f48cc0b6810 2023-11-28T11:21:59.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.922+0000 7f48d13ff700 1 -- 172.21.15.92:0/4099570664 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48cc0d2b50 con 0x7f48cc0b6810 2023-11-28T11:21:59.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.923+0000 7f48d13ff700 1 -- 172.21.15.92:0/4099570664 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48cc067880 con 0x7f48cc0b6810 2023-11-28T11:21:59.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.924+0000 7f48c37fe700 1 -- 172.21.15.92:0/4099570664 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f48c4010050 con 0x7f48cc0b6810 2023-11-28T11:21:59.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.924+0000 7f48c37fe700 1 --2- 172.21.15.92:0/4099570664 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f48ac0682f0 0x7f48ac06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:21:59.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.925+0000 7f48c37fe700 1 -- 172.21.15.92:0/4099570664 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f48c40870e0 con 0x7f48cc0b6810 2023-11-28T11:21:59.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.925+0000 7f48caffd700 1 --2- 172.21.15.92:0/4099570664 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f48ac0682f0 0x7f48ac06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:21:59.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.925+0000 7f48caffd700 1 --2- 172.21.15.92:0/4099570664 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f48ac0682f0 0x7f48ac06a7a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f48bc009be0 tx=0x7f48bc009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:21:59.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:21:59.929+0000 7f48c37fe700 1 -- 172.21.15.92:0/4099570664 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f48c4053520 con 0x7f48cc0b6810 2023-11-28T11:22:00.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.119+0000 7f48d13ff700 1 -- 172.21.15.92:0/4099570664 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f48cc1f1e20 con 0x7f48cc0b6810 2023-11-28T11:22:00.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.119+0000 7f48c37fe700 1 -- 172.21.15.92:0/4099570664 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f48c4056b40 con 0x7f48cc0b6810 2023-11-28T11:22:00.120 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-11-28T11:22:00.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.121+0000 7f48d13ff700 1 -- 172.21.15.92:0/4099570664 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f48ac0682f0 msgr2=0x7f48ac06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:00.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.121+0000 7f48d13ff700 1 --2- 172.21.15.92:0/4099570664 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f48ac0682f0 0x7f48ac06a7a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f48bc009be0 tx=0x7f48bc009210 comp rx=0 tx=0).stop 2023-11-28T11:22:00.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.122+0000 7f48d13ff700 1 -- 172.21.15.92:0/4099570664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48cc0b6810 msgr2=0x7f48cc0b6c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:00.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.122+0000 7f48d13ff700 1 --2- 172.21.15.92:0/4099570664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48cc0b6810 0x7f48cc0b6c80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f48c400c8b0 tx=0x7f48c4001650 comp rx=0 tx=0).stop 2023-11-28T11:22:00.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.122+0000 7f48d13ff700 1 -- 172.21.15.92:0/4099570664 shutdown_connections 2023-11-28T11:22:00.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.122+0000 7f48d13ff700 1 --2- 172.21.15.92:0/4099570664 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f48ac0682f0 0x7f48ac06a7a0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:00.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.122+0000 7f48d13ff700 1 --2- 172.21.15.92:0/4099570664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48cc0b6810 0x7f48cc0b6c80 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:00.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.122+0000 7f48d13ff700 1 --2- 172.21.15.92:0/4099570664 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f48b8005200 0x7f48cc1e5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:00.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.122+0000 7f48d13ff700 1 --2- 172.21.15.92:0/4099570664 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f48cc1e4620 0x7f48cc1f1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:00.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.122+0000 7f48d13ff700 1 -- 172.21.15.92:0/4099570664 >> 172.21.15.92:0/4099570664 conn(0x7f48cc0719a0 msgr2=0x7f48cc0d1160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:00.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.122+0000 7f48d13ff700 1 -- 172.21.15.92:0/4099570664 shutdown_connections 2023-11-28T11:22:00.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.122+0000 7f48d13ff700 1 -- 172.21.15.92:0/4099570664 wait complete. 2023-11-28T11:22:00.141 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:22:00.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.343+0000 7f3062dff700 1 Processor -- start 2023-11-28T11:22:00.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.343+0000 7f3062dff700 1 -- start start 2023-11-28T11:22:00.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.344+0000 7f3062dff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f305c0b4fc0 0x7f305c0cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:00.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.344+0000 7f3062dff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f305c0b1c60 0x7f305c0cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:00.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.344+0000 7f3062dff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f305c0d1950 con 0x7f305c0d2df0 2023-11-28T11:22:00.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.344+0000 7f3062dff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f305c0d3d30 con 0x7f305c0b4fc0 2023-11-28T11:22:00.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.344+0000 7f3062dff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f305c0b2500 con 0x7f305c0b1c60 2023-11-28T11:22:00.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.345+0000 7f305bfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f305c0b4fc0 0x7f305c0cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:00.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.345+0000 7f305bfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f305c0b4fc0 0x7f305c0cd620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56350/0 (socket says 172.21.15.92:56350) 2023-11-28T11:22:00.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.345+0000 7f305bfff700 1 -- 172.21.15.92:0/627495110 learned_addr learned my addr 172.21.15.92:0/627495110 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:00.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.345+0000 7f306139c700 1 --2- 172.21.15.92:0/627495110 >> v2:172.21.15.169:3300/0 conn(0x7f305c0b1c60 0x7f305c0cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:00.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.345+0000 7f305bfff700 1 -- 172.21.15.92:0/627495110 >> v2:172.21.15.169:3300/0 conn(0x7f305c0b1c60 msgr2=0x7f305c0cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:00.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.345+0000 7f305bfff700 1 --2- 172.21.15.92:0/627495110 >> v2:172.21.15.169:3300/0 conn(0x7f305c0b1c60 0x7f305c0cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:00.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.345+0000 7f305bfff700 1 -- 172.21.15.92:0/627495110 >> v1:172.21.15.169:6789/0 conn(0x7f305c0d2df0 legacy=0x7f305c0d31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:00.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.345+0000 7f305bfff700 1 -- 172.21.15.92:0/627495110 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f305c0d3a50 con 0x7f305c0b4fc0 2023-11-28T11:22:00.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.345+0000 7f305bfff700 1 --2- 172.21.15.92:0/627495110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f305c0b4fc0 0x7f305c0cd620 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f305c08c100 tx=0x7f3048009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b70fa586ec3cb6fd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:00.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.346+0000 7f305b7fe700 1 -- 172.21.15.92:0/627495110 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3048004230 con 0x7f305c0b4fc0 2023-11-28T11:22:00.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.346+0000 7f305b7fe700 1 -- 172.21.15.92:0/627495110 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3048004390 con 0x7f305c0b4fc0 2023-11-28T11:22:00.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.346+0000 7f305b7fe700 1 -- 172.21.15.92:0/627495110 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30480036a0 con 0x7f305c0b4fc0 2023-11-28T11:22:00.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.346+0000 7f3062dff700 1 -- 172.21.15.92:0/627495110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f305c0b4fc0 msgr2=0x7f305c0cd620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:00.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.346+0000 7f3062dff700 1 --2- 172.21.15.92:0/627495110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f305c0b4fc0 0x7f305c0cd620 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f305c08c100 tx=0x7f3048009bd0 comp rx=0 tx=0).stop 2023-11-28T11:22:00.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.346+0000 7f3062dff700 1 -- 172.21.15.92:0/627495110 shutdown_connections 2023-11-28T11:22:00.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.346+0000 7f3062dff700 1 --2- 172.21.15.92:0/627495110 >> v2:172.21.15.169:3300/0 conn(0x7f305c0b1c60 0x7f305c0cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:00.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.346+0000 7f3062dff700 1 --2- 172.21.15.92:0/627495110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f305c0b4fc0 0x7f305c0cd620 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:00.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.346+0000 7f3062dff700 1 -- 172.21.15.92:0/627495110 >> 172.21.15.92:0/627495110 conn(0x7f305c0719a0 msgr2=0x7f305c071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:00.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.346+0000 7f3062dff700 1 -- 172.21.15.92:0/627495110 shutdown_connections 2023-11-28T11:22:00.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.346+0000 7f3062dff700 1 -- 172.21.15.92:0/627495110 wait complete. 2023-11-28T11:22:00.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.348+0000 7f3062dff700 1 Processor -- start 2023-11-28T11:22:00.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.348+0000 7f3062dff700 1 -- start start 2023-11-28T11:22:00.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.349+0000 7f3062dff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f305c0b1c60 0x7f305c0c9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:00.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.349+0000 7f3062dff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f305c0b4fc0 0x7f305c156880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:00.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.349+0000 7f3062dff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f305c0d2df0 0x7f305c159da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:00.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.349+0000 7f3062dff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f305c1f6de0 con 0x7f305c0b4fc0 2023-11-28T11:22:00.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.349+0000 7f3062dff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f305c1f6260 con 0x7f305c0d2df0 2023-11-28T11:22:00.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.349+0000 7f3062dff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f305c1f6540 con 0x7f305c0b1c60 2023-11-28T11:22:00.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.349+0000 7f305bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f305c0b4fc0 0x7f305c156880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:00.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.349+0000 7f305bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f305c0b4fc0 0x7f305c156880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56362/0 (socket says 172.21.15.92:56362) 2023-11-28T11:22:00.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.349+0000 7f305bfff700 1 -- 172.21.15.92:0/2649406174 learned_addr learned my addr 172.21.15.92:0/2649406174 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:00.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.349+0000 7f3060b9b700 1 --2- 172.21.15.92:0/2649406174 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f305c0b1c60 0x7f305c0c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:00.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.349+0000 7f306139c700 1 --2- 172.21.15.92:0/2649406174 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f305c0d2df0 0x7f305c159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:00.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.350+0000 7f305bfff700 1 -- 172.21.15.92:0/2649406174 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f305c0b1c60 msgr2=0x7f305c0c9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:00.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.350+0000 7f305bfff700 1 --2- 172.21.15.92:0/2649406174 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f305c0b1c60 0x7f305c0c9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:00.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.350+0000 7f305bfff700 1 -- 172.21.15.92:0/2649406174 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f305c0d2df0 msgr2=0x7f305c159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:00.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.350+0000 7f305bfff700 1 --2- 172.21.15.92:0/2649406174 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f305c0d2df0 0x7f305c159da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:00.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.350+0000 7f305bfff700 1 -- 172.21.15.92:0/2649406174 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3048009550 con 0x7f305c0b4fc0 2023-11-28T11:22:00.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.350+0000 7f305bfff700 1 --2- 172.21.15.92:0/2649406174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f305c0b4fc0 0x7f305c156880 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3048009130 tx=0x7f3048004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:00.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.350+0000 7f30597fa700 1 -- 172.21.15.92:0/2649406174 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3048003d70 con 0x7f305c0b4fc0 2023-11-28T11:22:00.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.350+0000 7f30597fa700 1 -- 172.21.15.92:0/2649406174 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3048011630 con 0x7f305c0b4fc0 2023-11-28T11:22:00.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.350+0000 7f30597fa700 1 -- 172.21.15.92:0/2649406174 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30480119e0 con 0x7f305c0b4fc0 2023-11-28T11:22:00.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.350+0000 7f3062dff700 1 -- 172.21.15.92:0/2649406174 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f305c0b88f0 con 0x7f305c0b4fc0 2023-11-28T11:22:00.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.350+0000 7f3062dff700 1 -- 172.21.15.92:0/2649406174 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f305c0d15e0 con 0x7f305c0b4fc0 2023-11-28T11:22:00.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.351+0000 7f3062dff700 1 -- 172.21.15.92:0/2649406174 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f305c0b21b0 con 0x7f305c0b4fc0 2023-11-28T11:22:00.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.352+0000 7f30597fa700 1 -- 172.21.15.92:0/2649406174 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3048017020 con 0x7f305c0b4fc0 2023-11-28T11:22:00.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.353+0000 7f30597fa700 1 --2- 172.21.15.92:0/2649406174 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f30400682f0 0x7f304006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:00.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.353+0000 7f30597fa700 1 -- 172.21.15.92:0/2649406174 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f30480873a0 con 0x7f305c0b4fc0 2023-11-28T11:22:00.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.354+0000 7f3060b9b700 1 --2- 172.21.15.92:0/2649406174 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f30400682f0 0x7f304006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:00.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.354+0000 7f3060b9b700 1 --2- 172.21.15.92:0/2649406174 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f30400682f0 0x7f304006a7a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3050009960 tx=0x7f30500090e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:00.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.358+0000 7f30597fa700 1 -- 172.21.15.92:0/2649406174 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f30480537e0 con 0x7f305c0b4fc0 2023-11-28T11:22:00.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.549+0000 7f3062dff700 1 -- 172.21.15.92:0/2649406174 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f305c066c70 con 0x7f305c0b4fc0 2023-11-28T11:22:00.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.550+0000 7f30597fa700 1 -- 172.21.15.92:0/2649406174 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3048056e00 con 0x7f305c0b4fc0 2023-11-28T11:22:00.552 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-11-28T11:22:00.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.552+0000 7f3062dff700 1 -- 172.21.15.92:0/2649406174 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f30400682f0 msgr2=0x7f304006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:00.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.552+0000 7f3062dff700 1 --2- 172.21.15.92:0/2649406174 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f30400682f0 0x7f304006a7a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3050009960 tx=0x7f30500090e0 comp rx=0 tx=0).stop 2023-11-28T11:22:00.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.552+0000 7f3062dff700 1 -- 172.21.15.92:0/2649406174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f305c0b4fc0 msgr2=0x7f305c156880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:00.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.552+0000 7f3062dff700 1 --2- 172.21.15.92:0/2649406174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f305c0b4fc0 0x7f305c156880 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3048009130 tx=0x7f3048004750 comp rx=0 tx=0).stop 2023-11-28T11:22:00.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.552+0000 7f3062dff700 1 -- 172.21.15.92:0/2649406174 shutdown_connections 2023-11-28T11:22:00.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.552+0000 7f3062dff700 1 --2- 172.21.15.92:0/2649406174 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f305c0d2df0 0x7f305c159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:00.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.552+0000 7f3062dff700 1 --2- 172.21.15.92:0/2649406174 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f305c0b1c60 0x7f305c0c9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:00.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.552+0000 7f3062dff700 1 --2- 172.21.15.92:0/2649406174 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f30400682f0 0x7f304006a7a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:00.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.552+0000 7f3062dff700 1 --2- 172.21.15.92:0/2649406174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f305c0b4fc0 0x7f305c156880 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:00.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.552+0000 7f3062dff700 1 -- 172.21.15.92:0/2649406174 >> 172.21.15.92:0/2649406174 conn(0x7f305c0719a0 msgr2=0x7f305c0b7710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:00.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.553+0000 7f3062dff700 1 -- 172.21.15.92:0/2649406174 shutdown_connections 2023-11-28T11:22:00.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.553+0000 7f3062dff700 1 -- 172.21.15.92:0/2649406174 wait complete. 2023-11-28T11:22:00.572 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:22:00.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.773+0000 7fd3621ff700 1 Processor -- start 2023-11-28T11:22:00.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.773+0000 7fd3621ff700 1 -- start start 2023-11-28T11:22:00.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.775+0000 7fd3621ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd35c0b8510 0x7fd35c06b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:00.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.775+0000 7fd3621ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd35c0cd460 0x7fd35c06beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:00.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.775+0000 7fd3621ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd35c0b88e0 con 0x7fd35c06c3f0 2023-11-28T11:22:00.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.775+0000 7fd3621ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd35c0cdbd0 con 0x7fd35c0b8510 2023-11-28T11:22:00.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.775+0000 7fd3621ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd35c0d02d0 con 0x7fd35c0cd460 2023-11-28T11:22:00.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.775+0000 7fd35b7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd35c0b8510 0x7fd35c06b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:00.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.775+0000 7fd35b7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd35c0b8510 0x7fd35c06b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56378/0 (socket says 172.21.15.92:56378) 2023-11-28T11:22:00.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.775+0000 7fd35b7fe700 1 -- 172.21.15.92:0/400611441 learned_addr learned my addr 172.21.15.92:0/400611441 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:00.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.776+0000 7fd35affd700 1 --2- 172.21.15.92:0/400611441 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd35c0cd460 0x7fd35c06beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:00.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.776+0000 7fd35b7fe700 1 -- 172.21.15.92:0/400611441 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd35c0cd460 msgr2=0x7fd35c06beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:00.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.776+0000 7fd35b7fe700 1 --2- 172.21.15.92:0/400611441 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd35c0cd460 0x7fd35c06beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:00.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.776+0000 7fd35b7fe700 1 -- 172.21.15.92:0/400611441 >> v1:172.21.15.169:6789/0 conn(0x7fd35c06c3f0 legacy=0x7fd35c15cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:00.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.776+0000 7fd35b7fe700 1 -- 172.21.15.92:0/400611441 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd35c0cd8f0 con 0x7fd35c0b8510 2023-11-28T11:22:00.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.776+0000 7fd35affd700 1 --2- 172.21.15.92:0/400611441 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd35c0cd460 0x7fd35c06beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:22:00.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.776+0000 7fd35b7fe700 1 --2- 172.21.15.92:0/400611441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd35c0b8510 0x7fd35c06b970 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd34c009920 tx=0x7fd34c009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4484b2ae35cbfd2d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:00.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.776+0000 7fd35a7fc700 1 -- 172.21.15.92:0/400611441 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34c004230 con 0x7fd35c0b8510 2023-11-28T11:22:00.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.776+0000 7fd35a7fc700 1 -- 172.21.15.92:0/400611441 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd34c004390 con 0x7fd35c0b8510 2023-11-28T11:22:00.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.776+0000 7fd35a7fc700 1 -- 172.21.15.92:0/400611441 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34c009e70 con 0x7fd35c0b8510 2023-11-28T11:22:00.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.777+0000 7fd3621ff700 1 -- 172.21.15.92:0/400611441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd35c0b8510 msgr2=0x7fd35c06b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:00.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.777+0000 7fd3621ff700 1 --2- 172.21.15.92:0/400611441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd35c0b8510 0x7fd35c06b970 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd34c009920 tx=0x7fd34c009c30 comp rx=0 tx=0).stop 2023-11-28T11:22:00.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.777+0000 7fd3621ff700 1 -- 172.21.15.92:0/400611441 shutdown_connections 2023-11-28T11:22:00.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.777+0000 7fd3621ff700 1 --2- 172.21.15.92:0/400611441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd35c0b8510 0x7fd35c06b970 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:00.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.777+0000 7fd3621ff700 1 --2- 172.21.15.92:0/400611441 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd35c0cd460 0x7fd35c06beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:00.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.777+0000 7fd3621ff700 1 -- 172.21.15.92:0/400611441 >> 172.21.15.92:0/400611441 conn(0x7fd35c0719a0 msgr2=0x7fd35c071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:00.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.777+0000 7fd3621ff700 1 -- 172.21.15.92:0/400611441 shutdown_connections 2023-11-28T11:22:00.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.777+0000 7fd3621ff700 1 -- 172.21.15.92:0/400611441 wait complete. 2023-11-28T11:22:00.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.779+0000 7fd3621ff700 1 Processor -- start 2023-11-28T11:22:00.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.780+0000 7fd3621ff700 1 -- start start 2023-11-28T11:22:00.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.781+0000 7fd3621ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd35c06c3f0 0x7fd35c0c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:00.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.781+0000 7fd3621ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd35c0b8510 0x7fd35c0c96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:00.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.781+0000 7fd3621ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd35c0cd460 0x7fd35c0c73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:00.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.781+0000 7fd3621ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd35c0c7920 con 0x7fd35c0cd460 2023-11-28T11:22:00.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.781+0000 7fd3621ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd35c0c8650 con 0x7fd35c06c3f0 2023-11-28T11:22:00.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.781+0000 7fd3621ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd35c0c8930 con 0x7fd35c0b8510 2023-11-28T11:22:00.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.781+0000 7fd35bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd35c0cd460 0x7fd35c0c73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:00.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.781+0000 7fd35b7fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd35c06c3f0 0x7fd35c0c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:00.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.781+0000 7fd35b7fe700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd35c06c3f0 0x7fd35c0c7f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.92:33046/0 (socket says 172.21.15.92:33046) 2023-11-28T11:22:00.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.781+0000 7fd35b7fe700 1 -- 172.21.15.92:0/373221779 learned_addr learned my addr 172.21.15.92:0/373221779 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:00.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.781+0000 7fd35affd700 1 --2- 172.21.15.92:0/373221779 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd35c0b8510 0x7fd35c0c96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:00.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.781+0000 7fd35bfff700 1 -- 172.21.15.92:0/373221779 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd35c0b8510 msgr2=0x7fd35c0c96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:00.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.781+0000 7fd35bfff700 1 --2- 172.21.15.92:0/373221779 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd35c0b8510 0x7fd35c0c96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:00.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.781+0000 7fd35bfff700 1 -- 172.21.15.92:0/373221779 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd35c06c3f0 msgr2=0x7fd35c0c7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:00.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.781+0000 7fd35bfff700 1 --2- 172.21.15.92:0/373221779 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd35c06c3f0 0x7fd35c0c7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:00.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.781+0000 7fd35bfff700 1 -- 172.21.15.92:0/373221779 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd34c009550 con 0x7fd35c0cd460 2023-11-28T11:22:00.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.782+0000 7fd35bfff700 1 --2- 172.21.15.92:0/373221779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd35c0cd460 0x7fd35c0c73e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd35400cb10 tx=0x7fd35400ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:00.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.782+0000 7fd343fff700 1 -- 172.21.15.92:0/373221779 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35400a9e0 con 0x7fd35c0cd460 2023-11-28T11:22:00.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.782+0000 7fd343fff700 1 -- 172.21.15.92:0/373221779 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd35400ab40 con 0x7fd35c0cd460 2023-11-28T11:22:00.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.782+0000 7fd3621ff700 1 -- 172.21.15.92:0/373221779 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd35c0cfff0 con 0x7fd35c0cd460 2023-11-28T11:22:00.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.782+0000 7fd343fff700 1 -- 172.21.15.92:0/373221779 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd354012420 con 0x7fd35c0cd460 2023-11-28T11:22:00.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.782+0000 7fd3621ff700 1 -- 172.21.15.92:0/373221779 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd35c0cfd10 con 0x7fd35c0cd460 2023-11-28T11:22:00.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.783+0000 7fd3621ff700 1 -- 172.21.15.92:0/373221779 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd35c067880 con 0x7fd35c0cd460 2023-11-28T11:22:00.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.784+0000 7fd343fff700 1 -- 172.21.15.92:0/373221779 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd354022070 con 0x7fd35c0cd460 2023-11-28T11:22:00.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.785+0000 7fd343fff700 1 --2- 172.21.15.92:0/373221779 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd3480682f0 0x7fd34806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:00.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.785+0000 7fd343fff700 1 -- 172.21.15.92:0/373221779 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fd3540865c0 con 0x7fd35c0cd460 2023-11-28T11:22:00.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.785+0000 7fd35b7fe700 1 --2- 172.21.15.92:0/373221779 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd3480682f0 0x7fd34806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:00.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.786+0000 7fd35b7fe700 1 --2- 172.21.15.92:0/373221779 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd3480682f0 0x7fd34806a7a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd34c009c70 tx=0x7fd34c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:00.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.789+0000 7fd343fff700 1 -- 172.21.15.92:0/373221779 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd354052a50 con 0x7fd35c0cd460 2023-11-28T11:22:00.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.980+0000 7fd3621ff700 1 -- 172.21.15.92:0/373221779 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd35c0d2940 con 0x7fd35c0cd460 2023-11-28T11:22:00.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.980+0000 7fd343fff700 1 -- 172.21.15.92:0/373221779 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd354016020 con 0x7fd35c0cd460 2023-11-28T11:22:00.981 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-11-28T11:22:00.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.982+0000 7fd3621ff700 1 -- 172.21.15.92:0/373221779 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd3480682f0 msgr2=0x7fd34806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:00.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.982+0000 7fd3621ff700 1 --2- 172.21.15.92:0/373221779 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd3480682f0 0x7fd34806a7a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd34c009c70 tx=0x7fd34c008030 comp rx=0 tx=0).stop 2023-11-28T11:22:00.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.982+0000 7fd3621ff700 1 -- 172.21.15.92:0/373221779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd35c0cd460 msgr2=0x7fd35c0c73e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:00.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.982+0000 7fd3621ff700 1 --2- 172.21.15.92:0/373221779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd35c0cd460 0x7fd35c0c73e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd35400cb10 tx=0x7fd35400ce20 comp rx=0 tx=0).stop 2023-11-28T11:22:00.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.982+0000 7fd3621ff700 1 -- 172.21.15.92:0/373221779 shutdown_connections 2023-11-28T11:22:00.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.982+0000 7fd3621ff700 1 --2- 172.21.15.92:0/373221779 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd3480682f0 0x7fd34806a7a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:00.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.982+0000 7fd3621ff700 1 --2- 172.21.15.92:0/373221779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd35c0cd460 0x7fd35c0c73e0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:00.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.983+0000 7fd3621ff700 1 --2- 172.21.15.92:0/373221779 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd35c06c3f0 0x7fd35c0c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:00.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.983+0000 7fd3621ff700 1 --2- 172.21.15.92:0/373221779 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd35c0b8510 0x7fd35c0c96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:00.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.983+0000 7fd3621ff700 1 -- 172.21.15.92:0/373221779 >> 172.21.15.92:0/373221779 conn(0x7fd35c0719a0 msgr2=0x7fd35c0b3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:00.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.983+0000 7fd3621ff700 1 -- 172.21.15.92:0/373221779 shutdown_connections 2023-11-28T11:22:00.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:00.983+0000 7fd3621ff700 1 -- 172.21.15.92:0/373221779 wait complete. 2023-11-28T11:22:01.002 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:22:01.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.200+0000 7f4a20dff700 1 Processor -- start 2023-11-28T11:22:01.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.200+0000 7f4a20dff700 1 -- start start 2023-11-28T11:22:01.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.201+0000 7f4a20dff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4a1c0d1d70 0x7f4a1c167270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:01.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.201+0000 7f4a20dff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a1c06c790 con 0x7f4a1c0b8510 2023-11-28T11:22:01.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.201+0000 7f4a20dff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a1c0b88e0 con 0x7f4a1c0cd460 2023-11-28T11:22:01.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.201+0000 7f4a20dff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a1c0cdbd0 con 0x7f4a1c0d1d70 2023-11-28T11:22:01.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.201+0000 7f4a1ad9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4a1c0d1d70 0x7f4a1c167270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:01.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.202+0000 7f4a1a59c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f4a1c0b8510 0x7f4a1c06b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55488/0 (socket says 172.21.15.92:55488) 2023-11-28T11:22:01.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.202+0000 7f4a1a59c700 1 -- 172.21.15.92:0/3111464664 learned_addr learned my addr 172.21.15.92:0/3111464664 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:22:01.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.202+0000 7f4a1ad9d700 1 -- 172.21.15.92:0/3111464664 >> v1:172.21.15.169:6789/0 conn(0x7f4a1c0cd460 legacy=0x7f4a1c06c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:01.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.202+0000 7f4a1ad9d700 1 -- 172.21.15.92:0/3111464664 >> v1:172.21.15.92:6789/0 conn(0x7f4a1c0b8510 legacy=0x7f4a1c06b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:01.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.202+0000 7f4a1ad9d700 1 -- 172.21.15.92:0/3111464664 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a1c0cd8f0 con 0x7f4a1c0d1d70 2023-11-28T11:22:01.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.202+0000 7f4a1ad9d700 1 --2- 172.21.15.92:0/3111464664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a1c0d1d70 0x7f4a1c167270 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4a14009d90 tx=0x7f4a1400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc517c76224272b5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:01.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.202+0000 7f4a1959a700 1 -- 172.21.15.92:0/3111464664 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1400e070 con 0x7f4a1c0d1d70 2023-11-28T11:22:01.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.203+0000 7f4a1959a700 1 -- 172.21.15.92:0/3111464664 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a1400bad0 con 0x7f4a1c0d1d70 2023-11-28T11:22:01.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.203+0000 7f4a1959a700 1 -- 172.21.15.92:0/3111464664 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1400be80 con 0x7f4a1c0d1d70 2023-11-28T11:22:01.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.203+0000 7f4a20dff700 1 -- 172.21.15.92:0/3111464664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a1c0d1d70 msgr2=0x7f4a1c167270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:01.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.203+0000 7f4a20dff700 1 --2- 172.21.15.92:0/3111464664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a1c0d1d70 0x7f4a1c167270 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4a14009d90 tx=0x7f4a1400b330 comp rx=0 tx=0).stop 2023-11-28T11:22:01.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.203+0000 7f4a20dff700 1 -- 172.21.15.92:0/3111464664 shutdown_connections 2023-11-28T11:22:01.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.203+0000 7f4a20dff700 1 --2- 172.21.15.92:0/3111464664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a1c0d1d70 0x7f4a1c167270 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:01.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.203+0000 7f4a20dff700 1 -- 172.21.15.92:0/3111464664 >> 172.21.15.92:0/3111464664 conn(0x7f4a1c0719a0 msgr2=0x7f4a1c071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:01.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.203+0000 7f4a20dff700 1 -- 172.21.15.92:0/3111464664 shutdown_connections 2023-11-28T11:22:01.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.203+0000 7f4a20dff700 1 -- 172.21.15.92:0/3111464664 wait complete. 2023-11-28T11:22:01.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.205+0000 7f4a20dff700 1 Processor -- start 2023-11-28T11:22:01.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.205+0000 7f4a20dff700 1 -- start start 2023-11-28T11:22:01.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.206+0000 7f4a20dff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a1c0b8510 0x7f4a1c0c97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:01.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.206+0000 7f4a20dff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4a1c0cd460 0x7f4a1c0caf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:01.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.206+0000 7f4a20dff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a1c0d1d70 0x7f4a1c1e4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:01.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.206+0000 7f4a20dff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c0c9d30 con 0x7f4a1c0b8510 2023-11-28T11:22:01.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.206+0000 7f4a20dff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c0cc590 con 0x7f4a1c0d1d70 2023-11-28T11:22:01.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.206+0000 7f4a20dff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c0cb980 con 0x7f4a1c0cd460 2023-11-28T11:22:01.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.206+0000 7f4a1a59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a1c0b8510 0x7f4a1c0c97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:01.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.206+0000 7f4a1a59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a1c0b8510 0x7f4a1c0c97f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56406/0 (socket says 172.21.15.92:56406) 2023-11-28T11:22:01.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.206+0000 7f4a1a59c700 1 -- 172.21.15.92:0/3449075572 learned_addr learned my addr 172.21.15.92:0/3449075572 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:01.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.206+0000 7f4a19d9b700 1 --2- 172.21.15.92:0/3449075572 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4a1c0cd460 0x7f4a1c0caf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:01.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.206+0000 7f4a1ad9d700 1 --2- 172.21.15.92:0/3449075572 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a1c0d1d70 0x7f4a1c1e4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:01.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.207+0000 7f4a1a59c700 1 -- 172.21.15.92:0/3449075572 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4a1c0cd460 msgr2=0x7f4a1c0caf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:01.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.207+0000 7f4a1a59c700 1 --2- 172.21.15.92:0/3449075572 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4a1c0cd460 0x7f4a1c0caf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:01.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.207+0000 7f4a1a59c700 1 -- 172.21.15.92:0/3449075572 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a1c0d1d70 msgr2=0x7f4a1c1e4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:01.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.207+0000 7f4a1a59c700 1 --2- 172.21.15.92:0/3449075572 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a1c0d1d70 0x7f4a1c1e4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:01.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.207+0000 7f4a1a59c700 1 -- 172.21.15.92:0/3449075572 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a14009a70 con 0x7f4a1c0b8510 2023-11-28T11:22:01.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.207+0000 7f4a1a59c700 1 --2- 172.21.15.92:0/3449075572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a1c0b8510 0x7f4a1c0c97f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c00c7b0 tx=0x7f4a0c00cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:01.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.207+0000 7f4a0affd700 1 -- 172.21.15.92:0/3449075572 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c004c80 con 0x7f4a1c0b8510 2023-11-28T11:22:01.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.207+0000 7f4a0affd700 1 -- 172.21.15.92:0/3449075572 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a0c004de0 con 0x7f4a1c0b8510 2023-11-28T11:22:01.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.207+0000 7f4a20dff700 1 -- 172.21.15.92:0/3449075572 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a1c0cfff0 con 0x7f4a1c0b8510 2023-11-28T11:22:01.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.207+0000 7f4a0affd700 1 -- 172.21.15.92:0/3449075572 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c007310 con 0x7f4a1c0b8510 2023-11-28T11:22:01.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.207+0000 7f4a20dff700 1 -- 172.21.15.92:0/3449075572 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a1c0cfd10 con 0x7f4a1c0b8510 2023-11-28T11:22:01.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.208+0000 7f4a20dff700 1 -- 172.21.15.92:0/3449075572 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a1c0cc900 con 0x7f4a1c0b8510 2023-11-28T11:22:01.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.209+0000 7f4a0affd700 1 -- 172.21.15.92:0/3449075572 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4a0c01a020 con 0x7f4a1c0b8510 2023-11-28T11:22:01.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.210+0000 7f4a0affd700 1 --2- 172.21.15.92:0/3449075572 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f49fc0682f0 0x7f49fc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:01.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.211+0000 7f4a0affd700 1 -- 172.21.15.92:0/3449075572 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f4a0c085d00 con 0x7f4a1c0b8510 2023-11-28T11:22:01.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.211+0000 7f4a19d9b700 1 --2- 172.21.15.92:0/3449075572 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f49fc0682f0 0x7f49fc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:01.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.211+0000 7f4a19d9b700 1 --2- 172.21.15.92:0/3449075572 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f49fc0682f0 0x7f49fc06a7a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c0b7eb0 tx=0x7f4a10009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:01.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.215+0000 7f4a0affd700 1 -- 172.21.15.92:0/3449075572 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4a0c052140 con 0x7f4a1c0b8510 2023-11-28T11:22:01.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.402+0000 7f4a20dff700 1 -- 172.21.15.92:0/3449075572 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4a1c067880 con 0x7f4a1c0b8510 2023-11-28T11:22:01.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.403+0000 7f4a0affd700 1 -- 172.21.15.92:0/3449075572 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4a0c055760 con 0x7f4a1c0b8510 2023-11-28T11:22:01.404 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-11-28T11:22:01.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.405+0000 7f4a20dff700 1 -- 172.21.15.92:0/3449075572 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f49fc0682f0 msgr2=0x7f49fc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:01.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.405+0000 7f4a20dff700 1 --2- 172.21.15.92:0/3449075572 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f49fc0682f0 0x7f49fc06a7a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c0b7eb0 tx=0x7f4a10009040 comp rx=0 tx=0).stop 2023-11-28T11:22:01.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.405+0000 7f4a20dff700 1 -- 172.21.15.92:0/3449075572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a1c0b8510 msgr2=0x7f4a1c0c97f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:01.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.405+0000 7f4a20dff700 1 --2- 172.21.15.92:0/3449075572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a1c0b8510 0x7f4a1c0c97f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c00c7b0 tx=0x7f4a0c00cac0 comp rx=0 tx=0).stop 2023-11-28T11:22:01.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.405+0000 7f4a20dff700 1 -- 172.21.15.92:0/3449075572 shutdown_connections 2023-11-28T11:22:01.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.405+0000 7f4a20dff700 1 --2- 172.21.15.92:0/3449075572 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4a1c0d1d70 0x7f4a1c1e4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:01.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.405+0000 7f4a20dff700 1 --2- 172.21.15.92:0/3449075572 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f49fc0682f0 0x7f49fc06a7a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:01.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.405+0000 7f4a20dff700 1 --2- 172.21.15.92:0/3449075572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a1c0b8510 0x7f4a1c0c97f0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:01.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.405+0000 7f4a20dff700 1 --2- 172.21.15.92:0/3449075572 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4a1c0cd460 0x7f4a1c0caf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:01.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.405+0000 7f4a20dff700 1 -- 172.21.15.92:0/3449075572 >> 172.21.15.92:0/3449075572 conn(0x7f4a1c0719a0 msgr2=0x7f4a1c0b6b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:01.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.405+0000 7f4a20dff700 1 -- 172.21.15.92:0/3449075572 shutdown_connections 2023-11-28T11:22:01.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.405+0000 7f4a20dff700 1 -- 172.21.15.92:0/3449075572 wait complete. 2023-11-28T11:22:01.424 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T11:22:01.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.623+0000 7f74883ff700 1 Processor -- start 2023-11-28T11:22:01.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.623+0000 7f74883ff700 1 -- start start 2023-11-28T11:22:01.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.624+0000 7f74883ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f74800b4fc0 0x7f74800cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:01.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.624+0000 7f74883ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74800d1950 con 0x7f74800d2df0 2023-11-28T11:22:01.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.624+0000 7f74883ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74800b2500 con 0x7f74800b1c60 2023-11-28T11:22:01.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.624+0000 7f74883ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74800d3d30 con 0x7f74800b4fc0 2023-11-28T11:22:01.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.625+0000 7f748619b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f74800d2df0 0x7f74800d31c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55496/0 (socket says 172.21.15.92:55496) 2023-11-28T11:22:01.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.625+0000 7f748619b700 1 -- 172.21.15.92:0/109190523 learned_addr learned my addr 172.21.15.92:0/109190523 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:22:01.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.625+0000 7f748599a700 1 --2- 172.21.15.92:0/109190523 >> v2:172.21.15.169:3300/0 conn(0x7f74800b4fc0 0x7f74800cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:01.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.625+0000 7f7485199700 1 -- 172.21.15.92:0/109190523 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3048674355 0 0) 0x7f74800d1950 con 0x7f74800d2df0 2023-11-28T11:22:01.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.625+0000 7f7485199700 1 -- 172.21.15.92:0/109190523 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f746c0035b0 con 0x7f74800d2df0 2023-11-28T11:22:01.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.625+0000 7f7485199700 1 -- 172.21.15.92:0/109190523 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2714026180 0 0) 0x7f746c0035b0 con 0x7f74800d2df0 2023-11-28T11:22:01.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.625+0000 7f7485199700 1 -- 172.21.15.92:0/109190523 >> v2:172.21.15.169:3300/0 conn(0x7f74800b4fc0 msgr2=0x7f74800cd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:01.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.625+0000 7f7485199700 1 --2- 172.21.15.92:0/109190523 >> v2:172.21.15.169:3300/0 conn(0x7f74800b4fc0 0x7f74800cd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:01.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.625+0000 7f7485199700 1 -- 172.21.15.92:0/109190523 >> v1:172.21.15.169:6789/0 conn(0x7f74800b1c60 legacy=0x7f74800cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:01.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.625+0000 7f7485199700 1 -- 172.21.15.92:0/109190523 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74800d3a50 con 0x7f74800d2df0 2023-11-28T11:22:01.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.625+0000 7f7485199700 1 -- 172.21.15.92:0/109190523 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532161587 0 0) 0x7f747c001670 con 0x7f74800d2df0 2023-11-28T11:22:01.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.625+0000 7f7485199700 1 -- 172.21.15.92:0/109190523 >> v1:172.21.15.92:6789/0 conn(0x7f74800d2df0 legacy=0x7f74800d31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:01.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.625+0000 7f7485199700 1 --2- 172.21.15.92:0/109190523 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f746c003a00 0x7f746c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:01.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.626+0000 7f7485199700 1 --2- 172.21.15.92:0/109190523 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f746c0043f0 0x7f746c0048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:01.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.626+0000 7f7485199700 1 --2- 172.21.15.92:0/109190523 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f746c0050b0 0x7f746c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:01.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.626+0000 7f7485199700 1 -- 172.21.15.92:0/109190523 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74800d3d30 con 0x7f746c003a00 2023-11-28T11:22:01.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.626+0000 7f7485199700 1 -- 172.21.15.92:0/109190523 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746c00c000 con 0x7f746c0050b0 2023-11-28T11:22:01.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.626+0000 7f7485199700 1 -- 172.21.15.92:0/109190523 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f746c00c1c0 con 0x7f746c0043f0 2023-11-28T11:22:01.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.626+0000 7f748619b700 1 --2- 172.21.15.92:0/109190523 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f746c003a00 0x7f746c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:01.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.626+0000 7f748599a700 1 --2- 172.21.15.92:0/109190523 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f746c0043f0 0x7f746c0048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:01.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.626+0000 7f748699c700 1 --2- 172.21.15.92:0/109190523 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f746c0050b0 0x7f746c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:01.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.626+0000 7f748619b700 1 -- 172.21.15.92:0/109190523 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f746c0043f0 msgr2=0x7f746c0048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:01.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.626+0000 7f748619b700 1 --2- 172.21.15.92:0/109190523 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f746c0043f0 0x7f746c0048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:01.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.626+0000 7f748619b700 1 -- 172.21.15.92:0/109190523 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f746c0050b0 msgr2=0x7f746c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:01.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.626+0000 7f748619b700 1 --2- 172.21.15.92:0/109190523 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f746c0050b0 0x7f746c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:01.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.626+0000 7f748619b700 1 -- 172.21.15.92:0/109190523 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f746c0035b0 con 0x7f746c003a00 2023-11-28T11:22:01.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.626+0000 7f748699c700 1 --2- 172.21.15.92:0/109190523 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f746c0050b0 0x7f746c00ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:22:01.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.626+0000 7f748599a700 1 --2- 172.21.15.92:0/109190523 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f746c0043f0 0x7f746c0048b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:22:01.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.626+0000 7f748699c700 1 -- 172.21.15.92:0/109190523 reap_dead start 2023-11-28T11:22:01.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.626+0000 7f748619b700 1 --2- 172.21.15.92:0/109190523 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f746c003a00 0x7f746c003dd0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f747c00c2f0 tx=0x7f746c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9a27e76efb2d2b7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:01.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.627+0000 7f7485199700 1 -- 172.21.15.92:0/109190523 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747c00ce30 con 0x7f746c003a00 2023-11-28T11:22:01.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.627+0000 7f7485199700 1 -- 172.21.15.92:0/109190523 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f747c005960 con 0x7f746c003a00 2023-11-28T11:22:01.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.627+0000 7f7485199700 1 -- 172.21.15.92:0/109190523 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747c005cf0 con 0x7f746c003a00 2023-11-28T11:22:01.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.627+0000 7f74883ff700 1 -- 172.21.15.92:0/109190523 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f746c003a00 msgr2=0x7f746c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:01.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.627+0000 7f74883ff700 1 --2- 172.21.15.92:0/109190523 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f746c003a00 0x7f746c003dd0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f747c00c2f0 tx=0x7f746c005880 comp rx=0 tx=0).stop 2023-11-28T11:22:01.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.627+0000 7f74883ff700 1 -- 172.21.15.92:0/109190523 shutdown_connections 2023-11-28T11:22:01.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.627+0000 7f74883ff700 1 --2- 172.21.15.92:0/109190523 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f746c003a00 0x7f746c003dd0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:01.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.627+0000 7f74883ff700 1 -- 172.21.15.92:0/109190523 >> 172.21.15.92:0/109190523 conn(0x7f74800719a0 msgr2=0x7f7480071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:01.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.627+0000 7f74883ff700 1 -- 172.21.15.92:0/109190523 shutdown_connections 2023-11-28T11:22:01.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.627+0000 7f74883ff700 1 -- 172.21.15.92:0/109190523 wait complete. 2023-11-28T11:22:01.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.630+0000 7f74883ff700 1 Processor -- start 2023-11-28T11:22:01.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.630+0000 7f74883ff700 1 -- start start 2023-11-28T11:22:01.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.631+0000 7f74883ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f746c003a00 0x7f748014aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:01.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.631+0000 7f74883ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7480148040 0x7f74800b11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:01.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.631+0000 7f74883ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74801496a0 0x7f74800b3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:01.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.631+0000 7f74883ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f748014b020 con 0x7f74801496a0 2023-11-28T11:22:01.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.631+0000 7f74883ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7480148600 con 0x7f7480148040 2023-11-28T11:22:01.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.631+0000 7f74883ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74801488e0 con 0x7f746c003a00 2023-11-28T11:22:01.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.631+0000 7f748699c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74801496a0 0x7f74800b3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:01.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.631+0000 7f748699c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74801496a0 0x7f74800b3d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56428/0 (socket says 172.21.15.92:56428) 2023-11-28T11:22:01.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.631+0000 7f748699c700 1 -- 172.21.15.92:0/2067226931 learned_addr learned my addr 172.21.15.92:0/2067226931 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:01.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.632+0000 7f748619b700 1 --2- 172.21.15.92:0/2067226931 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f746c003a00 0x7f748014aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:01.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.632+0000 7f748599a700 1 --2- 172.21.15.92:0/2067226931 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7480148040 0x7f74800b11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:01.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.632+0000 7f748699c700 1 -- 172.21.15.92:0/2067226931 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f746c003a00 msgr2=0x7f748014aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:01.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.632+0000 7f748699c700 1 --2- 172.21.15.92:0/2067226931 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f746c003a00 0x7f748014aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:01.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.632+0000 7f748699c700 1 -- 172.21.15.92:0/2067226931 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7480148040 msgr2=0x7f74800b11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:01.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.632+0000 7f748699c700 1 --2- 172.21.15.92:0/2067226931 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7480148040 0x7f74800b11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:01.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.632+0000 7f748699c700 1 -- 172.21.15.92:0/2067226931 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74800b88f0 con 0x7f74801496a0 2023-11-28T11:22:01.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.632+0000 7f748699c700 1 --2- 172.21.15.92:0/2067226931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74801496a0 0x7f74800b3d60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f74800b8f30 tx=0x7f747800ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:01.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.632+0000 7f7476ffd700 1 -- 172.21.15.92:0/2067226931 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7478003c10 con 0x7f74801496a0 2023-11-28T11:22:01.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.632+0000 7f7476ffd700 1 -- 172.21.15.92:0/2067226931 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7478003d70 con 0x7f74801496a0 2023-11-28T11:22:01.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.632+0000 7f74883ff700 1 -- 172.21.15.92:0/2067226931 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74800d15e0 con 0x7f74801496a0 2023-11-28T11:22:01.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.632+0000 7f7476ffd700 1 -- 172.21.15.92:0/2067226931 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74780052f0 con 0x7f74801496a0 2023-11-28T11:22:01.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.633+0000 7f74883ff700 1 -- 172.21.15.92:0/2067226931 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74800b21b0 con 0x7f74801496a0 2023-11-28T11:22:01.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.633+0000 7f74883ff700 1 -- 172.21.15.92:0/2067226931 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74800ce170 con 0x7f74801496a0 2023-11-28T11:22:01.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.634+0000 7f7476ffd700 1 -- 172.21.15.92:0/2067226931 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7478005450 con 0x7f74801496a0 2023-11-28T11:22:01.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.635+0000 7f7476ffd700 1 --2- 172.21.15.92:0/2067226931 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f74640682f0 0x7f746406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:01.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.635+0000 7f7476ffd700 1 -- 172.21.15.92:0/2067226931 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f7478086230 con 0x7f74801496a0 2023-11-28T11:22:01.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.635+0000 7f748619b700 1 --2- 172.21.15.92:0/2067226931 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f74640682f0 0x7f746406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:01.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.636+0000 7f748619b700 1 --2- 172.21.15.92:0/2067226931 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f74640682f0 0x7f746406a7a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f747c000c00 tx=0x7f747c007000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:01.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.639+0000 7f7476ffd700 1 -- 172.21.15.92:0/2067226931 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7478052670 con 0x7f74801496a0 2023-11-28T11:22:01.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.829+0000 7f74883ff700 1 -- 172.21.15.92:0/2067226931 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f7480067170 con 0x7f74801496a0 2023-11-28T11:22:01.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.830+0000 7f7476ffd700 1 -- 172.21.15.92:0/2067226931 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7478055c90 con 0x7f74801496a0 2023-11-28T11:22:01.857 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-11-28T11:22:01.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.832+0000 7f74883ff700 1 -- 172.21.15.92:0/2067226931 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f74640682f0 msgr2=0x7f746406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:01.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.832+0000 7f74883ff700 1 --2- 172.21.15.92:0/2067226931 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f74640682f0 0x7f746406a7a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f747c000c00 tx=0x7f747c007000 comp rx=0 tx=0).stop 2023-11-28T11:22:01.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.832+0000 7f74883ff700 1 -- 172.21.15.92:0/2067226931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74801496a0 msgr2=0x7f74800b3d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:01.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.832+0000 7f74883ff700 1 --2- 172.21.15.92:0/2067226931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74801496a0 0x7f74800b3d60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f74800b8f30 tx=0x7f747800ee00 comp rx=0 tx=0).stop 2023-11-28T11:22:01.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.832+0000 7f74883ff700 1 -- 172.21.15.92:0/2067226931 shutdown_connections 2023-11-28T11:22:01.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.832+0000 7f74883ff700 1 --2- 172.21.15.92:0/2067226931 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f74640682f0 0x7f746406a7a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:01.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.832+0000 7f74883ff700 1 --2- 172.21.15.92:0/2067226931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74801496a0 0x7f74800b3d60 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:01.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.832+0000 7f74883ff700 1 --2- 172.21.15.92:0/2067226931 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f746c003a00 0x7f748014aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:01.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.832+0000 7f74883ff700 1 --2- 172.21.15.92:0/2067226931 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7480148040 0x7f74800b11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:01.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.832+0000 7f74883ff700 1 -- 172.21.15.92:0/2067226931 >> 172.21.15.92:0/2067226931 conn(0x7f74800719a0 msgr2=0x7f74800b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:01.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.833+0000 7f74883ff700 1 -- 172.21.15.92:0/2067226931 shutdown_connections 2023-11-28T11:22:01.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:01.833+0000 7f74883ff700 1 -- 172.21.15.92:0/2067226931 wait complete. 2023-11-28T11:22:01.859 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T11:22:02.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.059+0000 7feaddfff700 1 Processor -- start 2023-11-28T11:22:02.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.059+0000 7feaddfff700 1 -- start start 2023-11-28T11:22:02.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.060+0000 7feaddfff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fead815a7a0 0x7fead806c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:02.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.060+0000 7feaddfff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fead815eee0 0x7fead806a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:02.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.060+0000 7feaddfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fead806a7b0 0x7fead8070910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:02.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.060+0000 7feaddfff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fead806ac30 con 0x7fead806a7b0 2023-11-28T11:22:02.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.060+0000 7feaddfff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fead806cc10 con 0x7fead815eee0 2023-11-28T11:22:02.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.060+0000 7feaddfff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fead806cef0 con 0x7fead815a7a0 2023-11-28T11:22:02.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.060+0000 7fead7fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fead806a7b0 0x7fead8070910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:02.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.060+0000 7fead7fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fead806a7b0 0x7fead8070910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56432/0 (socket says 172.21.15.92:56432) 2023-11-28T11:22:02.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.060+0000 7fead7fff700 1 -- 172.21.15.92:0/3685179130 learned_addr learned my addr 172.21.15.92:0/3685179130 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:02.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.061+0000 7fead77fe700 1 --2- 172.21.15.92:0/3685179130 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fead815a7a0 0x7fead806c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:02.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.061+0000 7feacffff700 1 --2- 172.21.15.92:0/3685179130 >> v2:172.21.15.169:3300/0 conn(0x7fead815eee0 0x7fead806a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:02.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.061+0000 7fead7fff700 1 -- 172.21.15.92:0/3685179130 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fead815a7a0 msgr2=0x7fead806c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:02.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.061+0000 7fead7fff700 1 --2- 172.21.15.92:0/3685179130 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fead815a7a0 0x7fead806c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:02.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.061+0000 7fead7fff700 1 -- 172.21.15.92:0/3685179130 >> v2:172.21.15.169:3300/0 conn(0x7fead815eee0 msgr2=0x7fead806a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:02.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.061+0000 7fead7fff700 1 --2- 172.21.15.92:0/3685179130 >> v2:172.21.15.169:3300/0 conn(0x7fead815eee0 0x7fead806a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:02.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.061+0000 7fead7fff700 1 -- 172.21.15.92:0/3685179130 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fead80c9070 con 0x7fead806a7b0 2023-11-28T11:22:02.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.061+0000 7fead77fe700 1 --2- 172.21.15.92:0/3685179130 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fead815a7a0 0x7fead806c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:22:02.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.061+0000 7feacffff700 1 --2- 172.21.15.92:0/3685179130 >> v2:172.21.15.169:3300/0 conn(0x7fead815eee0 0x7fead806a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:22:02.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.061+0000 7fead7fff700 1 --2- 172.21.15.92:0/3685179130 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fead806a7b0 0x7fead8070910 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fead80cc500 tx=0x7fead000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4c66035c4973ed3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:02.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.061+0000 7fead6ffd700 1 -- 172.21.15.92:0/3685179130 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead000e070 con 0x7fead806a7b0 2023-11-28T11:22:02.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.061+0000 7fead6ffd700 1 -- 172.21.15.92:0/3685179130 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fead000b9c0 con 0x7fead806a7b0 2023-11-28T11:22:02.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.061+0000 7fead6ffd700 1 -- 172.21.15.92:0/3685179130 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead000bd70 con 0x7fead806a7b0 2023-11-28T11:22:02.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.062+0000 7feaddfff700 1 -- 172.21.15.92:0/3685179130 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fead806a7b0 msgr2=0x7fead8070910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:02.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.062+0000 7feaddfff700 1 --2- 172.21.15.92:0/3685179130 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fead806a7b0 0x7fead8070910 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fead80cc500 tx=0x7fead000b330 comp rx=0 tx=0).stop 2023-11-28T11:22:02.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.062+0000 7feaddfff700 1 -- 172.21.15.92:0/3685179130 shutdown_connections 2023-11-28T11:22:02.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.062+0000 7feaddfff700 1 --2- 172.21.15.92:0/3685179130 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fead806a7b0 0x7fead8070910 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:02.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.062+0000 7feaddfff700 1 --2- 172.21.15.92:0/3685179130 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fead815a7a0 0x7fead806c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:02.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.062+0000 7feaddfff700 1 --2- 172.21.15.92:0/3685179130 >> v2:172.21.15.169:3300/0 conn(0x7fead815eee0 0x7fead806a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:02.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.062+0000 7feaddfff700 1 -- 172.21.15.92:0/3685179130 >> 172.21.15.92:0/3685179130 conn(0x7fead80cb490 msgr2=0x7fead80cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:02.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.062+0000 7feaddfff700 1 -- 172.21.15.92:0/3685179130 shutdown_connections 2023-11-28T11:22:02.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.062+0000 7feaddfff700 1 -- 172.21.15.92:0/3685179130 wait complete. 2023-11-28T11:22:02.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.064+0000 7feaddfff700 1 Processor -- start 2023-11-28T11:22:02.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.065+0000 7feaddfff700 1 -- start start 2023-11-28T11:22:02.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.066+0000 7feaddfff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fead806a7b0 0x7fead8143da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:02.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.066+0000 7feaddfff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fead815a7a0 0x7fead80c3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:02.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.066+0000 7feaddfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fead815eee0 0x7fead81f5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:02.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.066+0000 7feaddfff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead80cd9c0 con 0x7fead815eee0 2023-11-28T11:22:02.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.066+0000 7feaddfff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead80ad500 con 0x7fead815a7a0 2023-11-28T11:22:02.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.066+0000 7feaddfff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fead80ad7e0 con 0x7fead806a7b0 2023-11-28T11:22:02.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.066+0000 7fead7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fead815eee0 0x7fead81f5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:02.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.066+0000 7fead77fe700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fead806a7b0 0x7fead8143da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:02.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.066+0000 7feacffff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fead815a7a0 0x7fead80c3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:02.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.066+0000 7fead7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fead815eee0 0x7fead81f5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56434/0 (socket says 172.21.15.92:56434) 2023-11-28T11:22:02.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.066+0000 7fead7fff700 1 -- 172.21.15.92:0/3442945166 learned_addr learned my addr 172.21.15.92:0/3442945166 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:02.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.066+0000 7fead7fff700 1 -- 172.21.15.92:0/3442945166 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fead806a7b0 msgr2=0x7fead8143da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:02.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.066+0000 7fead7fff700 1 --2- 172.21.15.92:0/3442945166 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fead806a7b0 0x7fead8143da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:02.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.066+0000 7fead7fff700 1 -- 172.21.15.92:0/3442945166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fead815a7a0 msgr2=0x7fead80c3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:02.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.066+0000 7fead7fff700 1 --2- 172.21.15.92:0/3442945166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fead815a7a0 0x7fead80c3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:02.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.066+0000 7fead7fff700 1 -- 172.21.15.92:0/3442945166 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fead0009b30 con 0x7fead815eee0 2023-11-28T11:22:02.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.067+0000 7fead7fff700 1 --2- 172.21.15.92:0/3442945166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fead815eee0 0x7fead81f5c60 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fead000b370 tx=0x7fead0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:02.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.067+0000 7fead4ff9700 1 -- 172.21.15.92:0/3442945166 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead0004250 con 0x7fead815eee0 2023-11-28T11:22:02.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.067+0000 7fead4ff9700 1 -- 172.21.15.92:0/3442945166 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fead00043f0 con 0x7fead815eee0 2023-11-28T11:22:02.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.067+0000 7feaddfff700 1 -- 172.21.15.92:0/3442945166 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fead80c9630 con 0x7fead815eee0 2023-11-28T11:22:02.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.067+0000 7fead4ff9700 1 -- 172.21.15.92:0/3442945166 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead00047a0 con 0x7fead815eee0 2023-11-28T11:22:02.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.067+0000 7feaddfff700 1 -- 172.21.15.92:0/3442945166 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fead80c9350 con 0x7fead815eee0 2023-11-28T11:22:02.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.068+0000 7feaddfff700 1 -- 172.21.15.92:0/3442945166 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fead815ab70 con 0x7fead815eee0 2023-11-28T11:22:02.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.069+0000 7fead4ff9700 1 -- 172.21.15.92:0/3442945166 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fead001b430 con 0x7fead815eee0 2023-11-28T11:22:02.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.070+0000 7fead4ff9700 1 --2- 172.21.15.92:0/3442945166 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7feab80682f0 0x7feab806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:02.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.070+0000 7fead4ff9700 1 -- 172.21.15.92:0/3442945166 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fead00873a0 con 0x7fead815eee0 2023-11-28T11:22:02.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.071+0000 7fead77fe700 1 --2- 172.21.15.92:0/3442945166 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7feab80682f0 0x7feab806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:02.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.071+0000 7fead77fe700 1 --2- 172.21.15.92:0/3442945166 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7feab80682f0 0x7feab806a7a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7feac8006fd0 tx=0x7feac8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:02.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.075+0000 7fead4ff9700 1 -- 172.21.15.92:0/3442945166 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fead0053780 con 0x7fead815eee0 2023-11-28T11:22:02.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.282+0000 7feaddfff700 1 -- 172.21.15.92:0/3442945166 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fead80ada00 con 0x7fead815eee0 2023-11-28T11:22:02.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.282+0000 7fead4ff9700 1 -- 172.21.15.92:0/3442945166 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) v1 ==== 80+0+0 (secure 0 0 0) 0x7fead0056da0 con 0x7fead815eee0 2023-11-28T11:22:02.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.284+0000 7feaddfff700 1 -- 172.21.15.92:0/3442945166 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7feab80682f0 msgr2=0x7feab806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:02.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.284+0000 7feaddfff700 1 --2- 172.21.15.92:0/3442945166 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7feab80682f0 0x7feab806a7a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7feac8006fd0 tx=0x7feac8008040 comp rx=0 tx=0).stop 2023-11-28T11:22:02.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.284+0000 7feaddfff700 1 -- 172.21.15.92:0/3442945166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fead815eee0 msgr2=0x7fead81f5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:02.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.284+0000 7feaddfff700 1 --2- 172.21.15.92:0/3442945166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fead815eee0 0x7fead81f5c60 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fead000b370 tx=0x7fead0004ad0 comp rx=0 tx=0).stop 2023-11-28T11:22:02.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.284+0000 7feaddfff700 1 -- 172.21.15.92:0/3442945166 shutdown_connections 2023-11-28T11:22:02.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.284+0000 7feaddfff700 1 --2- 172.21.15.92:0/3442945166 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7feab80682f0 0x7feab806a7a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:02.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.284+0000 7feaddfff700 1 --2- 172.21.15.92:0/3442945166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fead815eee0 0x7fead81f5c60 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:02.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.284+0000 7feaddfff700 1 --2- 172.21.15.92:0/3442945166 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fead806a7b0 0x7fead8143da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:02.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.284+0000 7feaddfff700 1 --2- 172.21.15.92:0/3442945166 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fead815a7a0 0x7fead80c3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:02.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.284+0000 7feaddfff700 1 -- 172.21.15.92:0/3442945166 >> 172.21.15.92:0/3442945166 conn(0x7fead80cb490 msgr2=0x7fead815e240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:02.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.285+0000 7feaddfff700 1 -- 172.21.15.92:0/3442945166 shutdown_connections 2023-11-28T11:22:02.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.285+0000 7feaddfff700 1 -- 172.21.15.92:0/3442945166 wait complete. 2023-11-28T11:22:02.287 INFO:teuthology.orchestra.run.smithi092.stderr:listed 0 entries 2023-11-28T11:22:02.303 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-11-28T11:22:02.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:22:02.303 DEBUG:teuthology.orchestra.run.smithi092:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-11-28T11:22:02.377 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2023-11-28T11:22:02.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:22:02.377 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-28T11:22:02.424 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-28T11:22:02.425 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:22:02.477 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-28T11:22:02.477 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:22:02.525 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-28T11:22:02.525 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:22:02.613 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:22:02.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.812+0000 7f4e8c3ff700 1 Processor -- start 2023-11-28T11:22:02.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.812+0000 7f4e8c3ff700 1 -- start start 2023-11-28T11:22:02.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.813+0000 7f4e8c3ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4e840b8510 0x7f4e8406b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:02.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.813+0000 7f4e8c3ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4e840cd460 0x7f4e8406beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:02.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.813+0000 7f4e8c3ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e840b88e0 con 0x7f4e8406c3f0 2023-11-28T11:22:02.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.813+0000 7f4e8c3ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e840cdbd0 con 0x7f4e840b8510 2023-11-28T11:22:02.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.813+0000 7f4e8c3ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e840d02d0 con 0x7f4e840cd460 2023-11-28T11:22:02.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.813+0000 7f4e8a19b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4e840b8510 0x7f4e8406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:02.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.813+0000 7f4e8a19b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4e840b8510 0x7f4e8406b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56448/0 (socket says 172.21.15.92:56448) 2023-11-28T11:22:02.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.813+0000 7f4e8a19b700 1 -- 172.21.15.92:0/2570360437 learned_addr learned my addr 172.21.15.92:0/2570360437 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:02.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.813+0000 7f4e8999a700 1 --2- 172.21.15.92:0/2570360437 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4e840cd460 0x7f4e8406beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:02.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.814+0000 7f4e8a19b700 1 -- 172.21.15.92:0/2570360437 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4e840cd460 msgr2=0x7f4e8406beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:02.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.814+0000 7f4e8a19b700 1 --2- 172.21.15.92:0/2570360437 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4e840cd460 0x7f4e8406beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:02.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.814+0000 7f4e8a19b700 1 -- 172.21.15.92:0/2570360437 >> v1:172.21.15.169:6789/0 conn(0x7f4e8406c3f0 legacy=0x7f4e8415cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:02.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.814+0000 7f4e8a19b700 1 -- 172.21.15.92:0/2570360437 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e840cd8f0 con 0x7f4e840b8510 2023-11-28T11:22:02.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.814+0000 7f4e8a19b700 1 --2- 172.21.15.92:0/2570360437 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e840b8510 0x7f4e8406b970 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4e80009920 tx=0x7f4e80009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40347e5debd7a7fb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:02.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.814+0000 7f4e89199700 1 -- 172.21.15.92:0/2570360437 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e80011830 con 0x7f4e840b8510 2023-11-28T11:22:02.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.814+0000 7f4e89199700 1 -- 172.21.15.92:0/2570360437 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e80011990 con 0x7f4e840b8510 2023-11-28T11:22:02.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.814+0000 7f4e89199700 1 -- 172.21.15.92:0/2570360437 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e80011d20 con 0x7f4e840b8510 2023-11-28T11:22:02.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.815+0000 7f4e8c3ff700 1 -- 172.21.15.92:0/2570360437 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e840b8510 msgr2=0x7f4e8406b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:02.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.815+0000 7f4e8c3ff700 1 --2- 172.21.15.92:0/2570360437 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e840b8510 0x7f4e8406b970 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4e80009920 tx=0x7f4e80009c30 comp rx=0 tx=0).stop 2023-11-28T11:22:02.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.815+0000 7f4e8c3ff700 1 -- 172.21.15.92:0/2570360437 shutdown_connections 2023-11-28T11:22:02.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.815+0000 7f4e8c3ff700 1 --2- 172.21.15.92:0/2570360437 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e840b8510 0x7f4e8406b970 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:02.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.815+0000 7f4e8c3ff700 1 --2- 172.21.15.92:0/2570360437 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4e840cd460 0x7f4e8406beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:02.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.815+0000 7f4e8c3ff700 1 -- 172.21.15.92:0/2570360437 >> 172.21.15.92:0/2570360437 conn(0x7f4e840719a0 msgr2=0x7f4e84071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:02.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.815+0000 7f4e8c3ff700 1 -- 172.21.15.92:0/2570360437 shutdown_connections 2023-11-28T11:22:02.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.815+0000 7f4e8c3ff700 1 -- 172.21.15.92:0/2570360437 wait complete. 2023-11-28T11:22:02.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.817+0000 7f4e8c3ff700 1 Processor -- start 2023-11-28T11:22:02.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.817+0000 7f4e8c3ff700 1 -- start start 2023-11-28T11:22:02.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.818+0000 7f4e8c3ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e8406c3f0 0x7f4e840c8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:02.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.818+0000 7f4e8c3ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4e840b8510 0x7f4e841f2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:02.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.818+0000 7f4e8c3ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e840cd460 0x7f4e84157810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:02.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.818+0000 7f4e8c3ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e84159220 con 0x7f4e8406c3f0 2023-11-28T11:22:02.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.818+0000 7f4e8c3ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e84158610 con 0x7f4e840cd460 2023-11-28T11:22:02.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.818+0000 7f4e8c3ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e841588f0 con 0x7f4e840b8510 2023-11-28T11:22:02.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.818+0000 7f4e8a19b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e8406c3f0 0x7f4e840c8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:02.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.818+0000 7f4e8a19b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e8406c3f0 0x7f4e840c8ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56462/0 (socket says 172.21.15.92:56462) 2023-11-28T11:22:02.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.818+0000 7f4e8a19b700 1 -- 172.21.15.92:0/232983839 learned_addr learned my addr 172.21.15.92:0/232983839 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:02.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.818+0000 7f4e8999a700 1 --2- 172.21.15.92:0/232983839 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4e840b8510 0x7f4e841f2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:02.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.818+0000 7f4e8a99c700 1 --2- 172.21.15.92:0/232983839 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e840cd460 0x7f4e84157810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:02.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.818+0000 7f4e8a19b700 1 -- 172.21.15.92:0/232983839 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4e840b8510 msgr2=0x7f4e841f2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:02.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.818+0000 7f4e8a19b700 1 --2- 172.21.15.92:0/232983839 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4e840b8510 0x7f4e841f2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:02.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.818+0000 7f4e8a19b700 1 -- 172.21.15.92:0/232983839 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e840cd460 msgr2=0x7f4e84157810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:02.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.818+0000 7f4e8a19b700 1 --2- 172.21.15.92:0/232983839 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e840cd460 0x7f4e84157810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:02.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.818+0000 7f4e8a19b700 1 -- 172.21.15.92:0/232983839 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e80009550 con 0x7f4e8406c3f0 2023-11-28T11:22:02.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.818+0000 7f4e8a99c700 1 --2- 172.21.15.92:0/232983839 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e840cd460 0x7f4e84157810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:22:02.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.818+0000 7f4e8a19b700 1 --2- 172.21.15.92:0/232983839 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e8406c3f0 0x7f4e840c8ca0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4e80000c00 tx=0x7f4e80004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:02.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.819+0000 7f4e7affd700 1 -- 172.21.15.92:0/232983839 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e80004380 con 0x7f4e8406c3f0 2023-11-28T11:22:02.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.819+0000 7f4e7affd700 1 -- 172.21.15.92:0/232983839 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e8001f070 con 0x7f4e8406c3f0 2023-11-28T11:22:02.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.819+0000 7f4e7affd700 1 -- 172.21.15.92:0/232983839 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e800036a0 con 0x7f4e8406c3f0 2023-11-28T11:22:02.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.819+0000 7f4e8c3ff700 1 -- 172.21.15.92:0/232983839 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e840cfff0 con 0x7f4e8406c3f0 2023-11-28T11:22:02.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.819+0000 7f4e8c3ff700 1 -- 172.21.15.92:0/232983839 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e840cfd10 con 0x7f4e8406c3f0 2023-11-28T11:22:02.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.820+0000 7f4e8c3ff700 1 -- 172.21.15.92:0/232983839 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e84067880 con 0x7f4e8406c3f0 2023-11-28T11:22:02.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.821+0000 7f4e7affd700 1 -- 172.21.15.92:0/232983839 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4e8002a020 con 0x7f4e8406c3f0 2023-11-28T11:22:02.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.821+0000 7f4e7affd700 1 --2- 172.21.15.92:0/232983839 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f4e680682f0 0x7f4e6806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:02.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.822+0000 7f4e7affd700 1 -- 172.21.15.92:0/232983839 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f4e80087550 con 0x7f4e8406c3f0 2023-11-28T11:22:02.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.822+0000 7f4e8999a700 1 --2- 172.21.15.92:0/232983839 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f4e680682f0 0x7f4e6806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:02.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.822+0000 7f4e8999a700 1 --2- 172.21.15.92:0/232983839 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f4e680682f0 0x7f4e6806a7a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4e840b41d0 tx=0x7f4e74009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:02.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:02.826+0000 7f4e7affd700 1 -- 172.21.15.92:0/232983839 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4e80053990 con 0x7f4e8406c3f0 2023-11-28T11:22:03.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.019+0000 7f4e8c3ff700 1 -- 172.21.15.92:0/232983839 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4e840d2940 con 0x7f4e8406c3f0 2023-11-28T11:22:03.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.021+0000 7f4e7affd700 1 -- 172.21.15.92:0/232983839 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+19844 (secure 0 0 0) 0x7f4e80056fb0 con 0x7f4e8406c3f0 2023-11-28T11:22:03.022 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:22:03.022 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":29,"fsid":"68956e0c-c16f-47cd-a220-f98364b0831d","created":"2023-11-28T11:20:28.697573+0000","modified":"2023-11-28T11:21:37.638578+0000","last_up_change":"2023-11-28T11:20:41.257011+0000","last_in_change":"2023-11-28T11:20:32.249767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:20:41.925237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:20:48.682497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:20:54.471398+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:20:55.204585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:20:58.297307+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6abe483f-0e03-4a26-b9cf-62041291747e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":115823},{"type":"v1","addr":"172.21.15.92:6809","nonce":115823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":115823},{"type":"v1","addr":"172.21.15.92:6813","nonce":115823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":115823},{"type":"v1","addr":"172.21.15.92:6821","nonce":115823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":115823},{"type":"v1","addr":"172.21.15.92:6817","nonce":115823}]},"public_addr":"172.21.15.92:6809/115823","cluster_addr":"172.21.15.92:6813/115823","heartbeat_back_addr":"172.21.15.92:6821/115823","heartbeat_front_addr":"172.21.15.92:6817/115823","state":["exists","up"]},{"osd":1,"uuid":"13c1de73-1a8c-4d7e-8a28-9dae52dfe089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":115824},{"type":"v1","addr":"172.21.15.92:6812","nonce":115824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":115824},{"type":"v1","addr":"172.21.15.92:6816","nonce":115824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":115824},{"type":"v1","addr":"172.21.15.92:6823","nonce":115824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":115824},{"type":"v1","addr":"172.21.15.92:6820","nonce":115824}]},"public_addr":"172.21.15.92:6812/115824","cluster_addr":"172.21.15.92:6816/115824","heartbeat_back_addr":"172.21.15.92:6823/115824","heartbeat_front_addr":"172.21.15.92:6820/115824","state":["exists","up"]},{"osd":2,"uuid":"25788a55-112f-4f00-810d-2bc37cf76778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":115821},{"type":"v1","addr":"172.21.15.92:6801","nonce":115821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":115821},{"type":"v1","addr":"172.21.15.92:6803","nonce":115821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":115821},{"type":"v1","addr":"172.21.15.92:6807","nonce":115821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":115821},{"type":"v1","addr":"172.21.15.92:6805","nonce":115821}]},"public_addr":"172.21.15.92:6801/115821","cluster_addr":"172.21.15.92:6803/115821","heartbeat_back_addr":"172.21.15.92:6807/115821","heartbeat_front_addr":"172.21.15.92:6805/115821","state":["exists","up"]},{"osd":3,"uuid":"adf54e24-33a9-41fe-9ce3-98a5ab6c158e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":115825},{"type":"v1","addr":"172.21.15.92:6825","nonce":115825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":115825},{"type":"v1","addr":"172.21.15.92:6827","nonce":115825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":115825},{"type":"v1","addr":"172.21.15.92:6831","nonce":115825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":115825},{"type":"v1","addr":"172.21.15.92:6829","nonce":115825}]},"public_addr":"172.21.15.92:6825/115825","cluster_addr":"172.21.15.92:6827/115825","heartbeat_back_addr":"172.21.15.92:6831/115825","heartbeat_front_addr":"172.21.15.92:6829/115825","state":["exists","up"]},{"osd":4,"uuid":"ccb9b747-f399-44ea-9a18-9db3152227e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":116035},{"type":"v1","addr":"172.21.15.169:6817","nonce":116035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":116035},{"type":"v1","addr":"172.21.15.169:6819","nonce":116035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":116035},{"type":"v1","addr":"172.21.15.169:6826","nonce":116035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":116035},{"type":"v1","addr":"172.21.15.169:6822","nonce":116035}]},"public_addr":"172.21.15.169:6817/116035","cluster_addr":"172.21.15.169:6819/116035","heartbeat_back_addr":"172.21.15.169:6826/116035","heartbeat_front_addr":"172.21.15.169:6822/116035","state":["exists","up"]},{"osd":5,"uuid":"308a1e63-75aa-4d7d-8a7c-cecc24714fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":116033},{"type":"v1","addr":"172.21.15.169:6801","nonce":116033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":116033},{"type":"v1","addr":"172.21.15.169:6803","nonce":116033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":116033},{"type":"v1","addr":"172.21.15.169:6807","nonce":116033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":116033},{"type":"v1","addr":"172.21.15.169:6805","nonce":116033}]},"public_addr":"172.21.15.169:6801/116033","cluster_addr":"172.21.15.169:6803/116033","heartbeat_back_addr":"172.21.15.169:6807/116033","heartbeat_front_addr":"172.21.15.169:6805/116033","state":["exists","up"]},{"osd":6,"uuid":"9c459b7b-b821-4b4b-b91e-6aaae271457c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":116034},{"type":"v1","addr":"172.21.15.169:6809","nonce":116034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":116034},{"type":"v1","addr":"172.21.15.169:6811","nonce":116034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":116034},{"type":"v1","addr":"172.21.15.169:6815","nonce":116034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":116034},{"type":"v1","addr":"172.21.15.169:6813","nonce":116034}]},"public_addr":"172.21.15.169:6809/116034","cluster_addr":"172.21.15.169:6811/116034","heartbeat_back_addr":"172.21.15.169:6815/116034","heartbeat_front_addr":"172.21.15.169:6813/116034","state":["exists","up"]},{"osd":7,"uuid":"1ca9aa4a-755d-4e28-a937-4d8dccab4df3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":116044},{"type":"v1","addr":"172.21.15.169:6823","nonce":116044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6825","nonce":116044},{"type":"v1","addr":"172.21.15.169:6827","nonce":116044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":116044},{"type":"v1","addr":"172.21.15.169:6831","nonce":116044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":116044},{"type":"v1","addr":"172.21.15.169:6829","nonce":116044}]},"public_addr":"172.21.15.169:6823/116044","cluster_addr":"172.21.15.169:6827/116044","heartbeat_back_addr":"172.21.15.169:6831/116044","heartbeat_front_addr":"172.21.15.169:6829/116044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.586781+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.869778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.508594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.860493+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.872267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.746933+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.611520+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.871446+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.1d","mappings":[{"from":3,"to":2}]},{"pgid":"4.21","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:22:03.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.023+0000 7f4e8c3ff700 1 -- 172.21.15.92:0/232983839 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f4e680682f0 msgr2=0x7f4e6806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:03.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.023+0000 7f4e8c3ff700 1 --2- 172.21.15.92:0/232983839 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f4e680682f0 0x7f4e6806a7a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4e840b41d0 tx=0x7f4e74009210 comp rx=0 tx=0).stop 2023-11-28T11:22:03.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.023+0000 7f4e8c3ff700 1 -- 172.21.15.92:0/232983839 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e8406c3f0 msgr2=0x7f4e840c8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:03.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.023+0000 7f4e8c3ff700 1 --2- 172.21.15.92:0/232983839 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e8406c3f0 0x7f4e840c8ca0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4e80000c00 tx=0x7f4e80004750 comp rx=0 tx=0).stop 2023-11-28T11:22:03.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.023+0000 7f4e8c3ff700 1 -- 172.21.15.92:0/232983839 shutdown_connections 2023-11-28T11:22:03.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.023+0000 7f4e8c3ff700 1 --2- 172.21.15.92:0/232983839 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e840cd460 0x7f4e84157810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:03.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.023+0000 7f4e8c3ff700 1 --2- 172.21.15.92:0/232983839 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f4e680682f0 0x7f4e6806a7a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:03.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.023+0000 7f4e8c3ff700 1 --2- 172.21.15.92:0/232983839 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e8406c3f0 0x7f4e840c8ca0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:03.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.023+0000 7f4e8c3ff700 1 --2- 172.21.15.92:0/232983839 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4e840b8510 0x7f4e841f2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:03.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.023+0000 7f4e8c3ff700 1 -- 172.21.15.92:0/232983839 >> 172.21.15.92:0/232983839 conn(0x7f4e840719a0 msgr2=0x7f4e840b5d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:03.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.023+0000 7f4e8c3ff700 1 -- 172.21.15.92:0/232983839 shutdown_connections 2023-11-28T11:22:03.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.023+0000 7f4e8c3ff700 1 -- 172.21.15.92:0/232983839 wait complete. 2023-11-28T11:22:03.042 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:20:41.925237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:20:48.682497+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:20:54.471398+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:20:55.204585+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T11:20:58.297307+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T11:22:03.042 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:22:03.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.240+0000 7f1df97ff700 1 Processor -- start 2023-11-28T11:22:03.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.240+0000 7f1df97ff700 1 -- start start 2023-11-28T11:22:03.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.241+0000 7f1df97ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1df40b8510 0x7f1df406b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:03.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.241+0000 7f1df97ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1df40cd460 0x7f1df406beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:03.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.242+0000 7f1df97ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1df40b88e0 con 0x7f1df406c3f0 2023-11-28T11:22:03.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.242+0000 7f1df97ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1df40cdbd0 con 0x7f1df40b8510 2023-11-28T11:22:03.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.242+0000 7f1df97ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1df40d02d0 con 0x7f1df40cd460 2023-11-28T11:22:03.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.242+0000 7f1df27fc700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1df40cd460 0x7f1df406beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:03.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.242+0000 7f1df2ffd700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1df40b8510 0x7f1df406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:03.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.242+0000 7f1df37fe700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f1df406c3f0 0x7f1df415cc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.92:43910/0 (socket says 172.21.15.92:43910) 2023-11-28T11:22:03.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.242+0000 7f1df37fe700 1 -- 172.21.15.92:0/1495543357 learned_addr learned my addr 172.21.15.92:0/1495543357 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:22:03.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.243+0000 7f1df1ffb700 1 -- 172.21.15.92:0/1495543357 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2950450335 0 0) 0x7f1df40b88e0 con 0x7f1df406c3f0 2023-11-28T11:22:03.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.243+0000 7f1df1ffb700 1 -- 172.21.15.92:0/1495543357 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ddc0035b0 con 0x7f1df406c3f0 2023-11-28T11:22:03.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.243+0000 7f1df2ffd700 1 -- 172.21.15.92:0/1495543357 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1df40cd460 msgr2=0x7f1df406beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:03.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.243+0000 7f1df2ffd700 1 --2- 172.21.15.92:0/1495543357 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1df40cd460 0x7f1df406beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:03.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.243+0000 7f1df2ffd700 1 -- 172.21.15.92:0/1495543357 >> v1:172.21.15.169:6789/0 conn(0x7f1df406c3f0 legacy=0x7f1df415cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:03.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.243+0000 7f1df2ffd700 1 -- 172.21.15.92:0/1495543357 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df40cd8f0 con 0x7f1df40b8510 2023-11-28T11:22:03.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.243+0000 7f1df27fc700 1 --2- 172.21.15.92:0/1495543357 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1df40cd460 0x7f1df406beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:22:03.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.243+0000 7f1df2ffd700 1 --2- 172.21.15.92:0/1495543357 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1df40b8510 0x7f1df406b970 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1de4009870 tx=0x7f1de4009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1080e874bc3721c1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:03.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.243+0000 7f1df1ffb700 1 -- 172.21.15.92:0/1495543357 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de4011830 con 0x7f1df40b8510 2023-11-28T11:22:03.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.243+0000 7f1df1ffb700 1 -- 172.21.15.92:0/1495543357 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1de4011990 con 0x7f1df40b8510 2023-11-28T11:22:03.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.243+0000 7f1df1ffb700 1 -- 172.21.15.92:0/1495543357 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de4011d00 con 0x7f1df40b8510 2023-11-28T11:22:03.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.244+0000 7f1df97ff700 1 -- 172.21.15.92:0/1495543357 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1df40b8510 msgr2=0x7f1df406b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:03.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.244+0000 7f1df97ff700 1 --2- 172.21.15.92:0/1495543357 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1df40b8510 0x7f1df406b970 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1de4009870 tx=0x7f1de4009b80 comp rx=0 tx=0).stop 2023-11-28T11:22:03.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.244+0000 7f1df97ff700 1 -- 172.21.15.92:0/1495543357 shutdown_connections 2023-11-28T11:22:03.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.244+0000 7f1df97ff700 1 --2- 172.21.15.92:0/1495543357 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1df40b8510 0x7f1df406b970 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:03.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.244+0000 7f1df97ff700 1 --2- 172.21.15.92:0/1495543357 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1df40cd460 0x7f1df406beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:03.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.244+0000 7f1df97ff700 1 -- 172.21.15.92:0/1495543357 >> 172.21.15.92:0/1495543357 conn(0x7f1df40719a0 msgr2=0x7f1df4071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:03.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.244+0000 7f1df97ff700 1 -- 172.21.15.92:0/1495543357 shutdown_connections 2023-11-28T11:22:03.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.244+0000 7f1df97ff700 1 -- 172.21.15.92:0/1495543357 wait complete. 2023-11-28T11:22:03.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.246+0000 7f1df97ff700 1 Processor -- start 2023-11-28T11:22:03.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.246+0000 7f1df97ff700 1 -- start start 2023-11-28T11:22:03.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.247+0000 7f1df97ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1df406c3f0 0x7f1df4148c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:03.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.247+0000 7f1df97ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1df40b8510 0x7f1df40c8e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:03.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.247+0000 7f1df97ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1df40cd460 0x7f1df41e3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:03.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.247+0000 7f1df97ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df40c8a60 con 0x7f1df406c3f0 2023-11-28T11:22:03.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.247+0000 7f1df97ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df40cd170 con 0x7f1df40b8510 2023-11-28T11:22:03.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.247+0000 7f1df97ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1df41e6b10 con 0x7f1df40cd460 2023-11-28T11:22:03.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.247+0000 7f1df2ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1df406c3f0 0x7f1df4148c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:03.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.247+0000 7f1df2ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1df406c3f0 0x7f1df4148c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56476/0 (socket says 172.21.15.92:56476) 2023-11-28T11:22:03.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.247+0000 7f1df2ffd700 1 -- 172.21.15.92:0/3889024290 learned_addr learned my addr 172.21.15.92:0/3889024290 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:03.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.248+0000 7f1df27fc700 1 --2- 172.21.15.92:0/3889024290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1df40b8510 0x7f1df40c8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:03.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.248+0000 7f1df2ffd700 1 -- 172.21.15.92:0/3889024290 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1df40cd460 msgr2=0x7f1df41e3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:03.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.248+0000 7f1df2ffd700 1 --2- 172.21.15.92:0/3889024290 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1df40cd460 0x7f1df41e3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:03.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.248+0000 7f1df2ffd700 1 -- 172.21.15.92:0/3889024290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1df40b8510 msgr2=0x7f1df40c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:03.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.248+0000 7f1df2ffd700 1 --2- 172.21.15.92:0/3889024290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1df40b8510 0x7f1df40c8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:03.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.248+0000 7f1df2ffd700 1 -- 172.21.15.92:0/3889024290 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1de80094a0 con 0x7f1df406c3f0 2023-11-28T11:22:03.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.248+0000 7f1df27fc700 1 --2- 172.21.15.92:0/3889024290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1df40b8510 0x7f1df40c8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:22:03.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.248+0000 7f1df2ffd700 1 --2- 172.21.15.92:0/3889024290 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1df406c3f0 0x7f1df4148c20 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1de4000c00 tx=0x7f1de4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:03.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.248+0000 7f1ddb7fe700 1 -- 172.21.15.92:0/3889024290 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de40045c0 con 0x7f1df406c3f0 2023-11-28T11:22:03.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.248+0000 7f1ddb7fe700 1 -- 172.21.15.92:0/3889024290 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1de401f070 con 0x7f1df406c3f0 2023-11-28T11:22:03.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.248+0000 7f1ddb7fe700 1 -- 172.21.15.92:0/3889024290 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de40038c0 con 0x7f1df406c3f0 2023-11-28T11:22:03.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.248+0000 7f1df97ff700 1 -- 172.21.15.92:0/3889024290 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1de40094a0 con 0x7f1df406c3f0 2023-11-28T11:22:03.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.248+0000 7f1df97ff700 1 -- 172.21.15.92:0/3889024290 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1df40cfff0 con 0x7f1df406c3f0 2023-11-28T11:22:03.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.249+0000 7f1df97ff700 1 -- 172.21.15.92:0/3889024290 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1df40cfd10 con 0x7f1df406c3f0 2023-11-28T11:22:03.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.250+0000 7f1ddb7fe700 1 -- 172.21.15.92:0/3889024290 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1de400f050 con 0x7f1df406c3f0 2023-11-28T11:22:03.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.251+0000 7f1ddb7fe700 1 --2- 172.21.15.92:0/3889024290 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f1de00682f0 0x7f1de006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:03.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.252+0000 7f1ddb7fe700 1 -- 172.21.15.92:0/3889024290 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f1de4021070 con 0x7f1df406c3f0 2023-11-28T11:22:03.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.252+0000 7f1df27fc700 1 --2- 172.21.15.92:0/3889024290 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f1de00682f0 0x7f1de006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:03.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.252+0000 7f1df27fc700 1 --2- 172.21.15.92:0/3889024290 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f1de00682f0 0x7f1de006a7a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1df40c96f0 tx=0x7f1de8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:03.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.256+0000 7f1ddb7fe700 1 -- 172.21.15.92:0/3889024290 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1de4053dc0 con 0x7f1df406c3f0 2023-11-28T11:22:03.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.445+0000 7f1df97ff700 1 -- 172.21.15.92:0/3889024290 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1df4067880 con 0x7f1df406c3f0 2023-11-28T11:22:03.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.446+0000 7f1ddb7fe700 1 -- 172.21.15.92:0/3889024290 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1de402b020 con 0x7f1df406c3f0 2023-11-28T11:22:03.447 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-11-28T11:22:03.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.448+0000 7f1df97ff700 1 -- 172.21.15.92:0/3889024290 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f1de00682f0 msgr2=0x7f1de006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:03.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.448+0000 7f1df97ff700 1 --2- 172.21.15.92:0/3889024290 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f1de00682f0 0x7f1de006a7a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1df40c96f0 tx=0x7f1de8008040 comp rx=0 tx=0).stop 2023-11-28T11:22:03.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.448+0000 7f1df97ff700 1 -- 172.21.15.92:0/3889024290 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1df406c3f0 msgr2=0x7f1df4148c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:03.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.448+0000 7f1df97ff700 1 --2- 172.21.15.92:0/3889024290 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1df406c3f0 0x7f1df4148c20 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1de4000c00 tx=0x7f1de4004750 comp rx=0 tx=0).stop 2023-11-28T11:22:03.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.448+0000 7f1df97ff700 1 -- 172.21.15.92:0/3889024290 shutdown_connections 2023-11-28T11:22:03.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.448+0000 7f1df97ff700 1 --2- 172.21.15.92:0/3889024290 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1df40cd460 0x7f1df41e3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:03.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.448+0000 7f1df97ff700 1 --2- 172.21.15.92:0/3889024290 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f1de00682f0 0x7f1de006a7a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:03.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.448+0000 7f1df97ff700 1 --2- 172.21.15.92:0/3889024290 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1df406c3f0 0x7f1df4148c20 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:03.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.448+0000 7f1df97ff700 1 --2- 172.21.15.92:0/3889024290 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1df40b8510 0x7f1df40c8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:03.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.448+0000 7f1df97ff700 1 -- 172.21.15.92:0/3889024290 >> 172.21.15.92:0/3889024290 conn(0x7f1df40719a0 msgr2=0x7f1df40d1ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:03.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.448+0000 7f1df97ff700 1 -- 172.21.15.92:0/3889024290 shutdown_connections 2023-11-28T11:22:03.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.448+0000 7f1df97ff700 1 -- 172.21.15.92:0/3889024290 wait complete. 2023-11-28T11:22:03.467 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:22:03.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.667+0000 7f1dee1ff700 1 Processor -- start 2023-11-28T11:22:03.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.667+0000 7f1dee1ff700 1 -- start start 2023-11-28T11:22:03.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.668+0000 7f1dee1ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f1de80b8510 0x7f1de806b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:03.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.668+0000 7f1dee1ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1de80d0d60 0x7f1de80d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:03.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.668+0000 7f1dee1ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1de80b88e0 con 0x7f1de80cd460 2023-11-28T11:22:03.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.668+0000 7f1dee1ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1de80cdbd0 con 0x7f1de80b8510 2023-11-28T11:22:03.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.668+0000 7f1dee1ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1de80d02d0 con 0x7f1de80d0d60 2023-11-28T11:22:03.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.668+0000 7f1ddffff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f1de80cd460 0x7f1de806beb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55500/0 (socket says 172.21.15.92:55500) 2023-11-28T11:22:03.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.668+0000 7f1ddffff700 1 -- 172.21.15.92:0/36531828 learned_addr learned my addr 172.21.15.92:0/36531828 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:22:03.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.668+0000 7f1de77fe700 1 --2- 172.21.15.92:0/36531828 >> v2:172.21.15.169:3300/0 conn(0x7f1de80b8510 0x7f1de806b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:03.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.668+0000 7f1de7fff700 1 --2- 172.21.15.92:0/36531828 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1de80d0d60 0x7f1de80d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:03.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.669+0000 7f1de6ffd700 1 -- 172.21.15.92:0/36531828 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1044171701 0 0) 0x7f1de80b88e0 con 0x7f1de80cd460 2023-11-28T11:22:03.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.669+0000 7f1de6ffd700 1 -- 172.21.15.92:0/36531828 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1dd40035b0 con 0x7f1de80cd460 2023-11-28T11:22:03.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.669+0000 7f1de77fe700 1 -- 172.21.15.92:0/36531828 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1de80d0d60 msgr2=0x7f1de80d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:03.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.669+0000 7f1de77fe700 1 --2- 172.21.15.92:0/36531828 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1de80d0d60 0x7f1de80d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:03.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.669+0000 7f1de77fe700 1 -- 172.21.15.92:0/36531828 >> v1:172.21.15.92:6789/0 conn(0x7f1de80cd460 legacy=0x7f1de806beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:03.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.669+0000 7f1de77fe700 1 -- 172.21.15.92:0/36531828 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1de80cd8f0 con 0x7f1de80b8510 2023-11-28T11:22:03.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.669+0000 7f1de77fe700 1 --2- 172.21.15.92:0/36531828 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1de80b8510 0x7f1de806b970 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1dd8009870 tx=0x7f1dd8009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=432ab7abc6572591 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:03.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.669+0000 7f1de6ffd700 1 -- 172.21.15.92:0/36531828 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd8011830 con 0x7f1de80b8510 2023-11-28T11:22:03.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.670+0000 7f1de6ffd700 1 -- 172.21.15.92:0/36531828 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dd8011990 con 0x7f1de80b8510 2023-11-28T11:22:03.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.670+0000 7f1de6ffd700 1 -- 172.21.15.92:0/36531828 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd8011d00 con 0x7f1de80b8510 2023-11-28T11:22:03.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.670+0000 7f1dee1ff700 1 -- 172.21.15.92:0/36531828 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1de80b8510 msgr2=0x7f1de806b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:03.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.670+0000 7f1dee1ff700 1 --2- 172.21.15.92:0/36531828 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1de80b8510 0x7f1de806b970 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1dd8009870 tx=0x7f1dd8009b80 comp rx=0 tx=0).stop 2023-11-28T11:22:03.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.670+0000 7f1dee1ff700 1 -- 172.21.15.92:0/36531828 shutdown_connections 2023-11-28T11:22:03.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.670+0000 7f1dee1ff700 1 --2- 172.21.15.92:0/36531828 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1de80d0d60 0x7f1de80d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:03.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.670+0000 7f1dee1ff700 1 --2- 172.21.15.92:0/36531828 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1de80b8510 0x7f1de806b970 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:03.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.670+0000 7f1dee1ff700 1 -- 172.21.15.92:0/36531828 >> 172.21.15.92:0/36531828 conn(0x7f1de80719a0 msgr2=0x7f1de8071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:03.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.670+0000 7f1dee1ff700 1 -- 172.21.15.92:0/36531828 shutdown_connections 2023-11-28T11:22:03.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.670+0000 7f1dee1ff700 1 -- 172.21.15.92:0/36531828 wait complete. 2023-11-28T11:22:03.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.672+0000 7f1dee1ff700 1 Processor -- start 2023-11-28T11:22:03.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.672+0000 7f1dee1ff700 1 -- start start 2023-11-28T11:22:03.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.673+0000 7f1dee1ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1de80b8510 0x7f1de81f1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:03.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.673+0000 7f1dee1ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1de80cd460 0x7f1de815a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:03.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.673+0000 7f1dee1ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1de80d0d60 0x7f1de815a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:03.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.673+0000 7f1dee1ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de81e5420 con 0x7f1de80d0d60 2023-11-28T11:22:03.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.673+0000 7f1dee1ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de81f51c0 con 0x7f1de80b8510 2023-11-28T11:22:03.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.673+0000 7f1dee1ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1de81e45d0 con 0x7f1de80cd460 2023-11-28T11:22:03.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.673+0000 7f1de7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1de80d0d60 0x7f1de815a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:03.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.673+0000 7f1de7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1de80d0d60 0x7f1de815a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56486/0 (socket says 172.21.15.92:56486) 2023-11-28T11:22:03.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.673+0000 7f1de7fff700 1 -- 172.21.15.92:0/1859432298 learned_addr learned my addr 172.21.15.92:0/1859432298 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:03.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.673+0000 7f1ddffff700 1 --2- 172.21.15.92:0/1859432298 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1de80cd460 0x7f1de815a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:03.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.673+0000 7f1de77fe700 1 --2- 172.21.15.92:0/1859432298 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1de80b8510 0x7f1de81f1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:03.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.674+0000 7f1de7fff700 1 -- 172.21.15.92:0/1859432298 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1de80cd460 msgr2=0x7f1de815a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:03.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.674+0000 7f1de7fff700 1 --2- 172.21.15.92:0/1859432298 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1de80cd460 0x7f1de815a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:03.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.674+0000 7f1de7fff700 1 -- 172.21.15.92:0/1859432298 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1de80b8510 msgr2=0x7f1de81f1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:03.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.674+0000 7f1de7fff700 1 --2- 172.21.15.92:0/1859432298 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1de80b8510 0x7f1de81f1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:03.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.674+0000 7f1de7fff700 1 -- 172.21.15.92:0/1859432298 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd80094a0 con 0x7f1de80d0d60 2023-11-28T11:22:03.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.674+0000 7f1de77fe700 1 --2- 172.21.15.92:0/1859432298 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1de80b8510 0x7f1de81f1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:22:03.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.674+0000 7f1de7fff700 1 --2- 172.21.15.92:0/1859432298 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1de80d0d60 0x7f1de815a550 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1de0009d30 tx=0x7f1de000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:03.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.674+0000 7f1de4ff9700 1 -- 172.21.15.92:0/1859432298 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de0009ba0 con 0x7f1de80d0d60 2023-11-28T11:22:03.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.674+0000 7f1de4ff9700 1 -- 172.21.15.92:0/1859432298 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1de000f040 con 0x7f1de80d0d60 2023-11-28T11:22:03.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.674+0000 7f1de4ff9700 1 -- 172.21.15.92:0/1859432298 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de0014480 con 0x7f1de80d0d60 2023-11-28T11:22:03.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.674+0000 7f1dee1ff700 1 -- 172.21.15.92:0/1859432298 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1de80cfff0 con 0x7f1de80d0d60 2023-11-28T11:22:03.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.674+0000 7f1dee1ff700 1 -- 172.21.15.92:0/1859432298 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1de80cfd10 con 0x7f1de80d0d60 2023-11-28T11:22:03.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.675+0000 7f1dee1ff700 1 -- 172.21.15.92:0/1859432298 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1de8067880 con 0x7f1de80d0d60 2023-11-28T11:22:03.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.676+0000 7f1de4ff9700 1 -- 172.21.15.92:0/1859432298 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1de00145e0 con 0x7f1de80d0d60 2023-11-28T11:22:03.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.677+0000 7f1de4ff9700 1 --2- 172.21.15.92:0/1859432298 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f1dc80682f0 0x7f1dc806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:03.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.677+0000 7f1de4ff9700 1 -- 172.21.15.92:0/1859432298 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f1de0086b80 con 0x7f1de80d0d60 2023-11-28T11:22:03.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.677+0000 7f1de77fe700 1 --2- 172.21.15.92:0/1859432298 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f1dc80682f0 0x7f1dc806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:03.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.678+0000 7f1de77fe700 1 --2- 172.21.15.92:0/1859432298 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f1dc80682f0 0x7f1dc806a7a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1dd80090b0 tx=0x7f1dd8008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:03.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.681+0000 7f1de4ff9700 1 -- 172.21.15.92:0/1859432298 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1de0052fc0 con 0x7f1de80d0d60 2023-11-28T11:22:03.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.873+0000 7f1dee1ff700 1 -- 172.21.15.92:0/1859432298 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1de81e5590 con 0x7f1de80d0d60 2023-11-28T11:22:03.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.873+0000 7f1de4ff9700 1 -- 172.21.15.92:0/1859432298 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1de00565e0 con 0x7f1de80d0d60 2023-11-28T11:22:03.875 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-11-28T11:22:03.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.875+0000 7f1dee1ff700 1 -- 172.21.15.92:0/1859432298 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f1dc80682f0 msgr2=0x7f1dc806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:03.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.875+0000 7f1dee1ff700 1 --2- 172.21.15.92:0/1859432298 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f1dc80682f0 0x7f1dc806a7a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1dd80090b0 tx=0x7f1dd8008030 comp rx=0 tx=0).stop 2023-11-28T11:22:03.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.875+0000 7f1dee1ff700 1 -- 172.21.15.92:0/1859432298 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1de80d0d60 msgr2=0x7f1de815a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:03.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.875+0000 7f1dee1ff700 1 --2- 172.21.15.92:0/1859432298 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1de80d0d60 0x7f1de815a550 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1de0009d30 tx=0x7f1de000c3a0 comp rx=0 tx=0).stop 2023-11-28T11:22:03.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.875+0000 7f1dee1ff700 1 -- 172.21.15.92:0/1859432298 shutdown_connections 2023-11-28T11:22:03.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.875+0000 7f1dee1ff700 1 --2- 172.21.15.92:0/1859432298 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f1dc80682f0 0x7f1dc806a7a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:03.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.875+0000 7f1dee1ff700 1 --2- 172.21.15.92:0/1859432298 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1de80d0d60 0x7f1de815a550 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:03.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.875+0000 7f1dee1ff700 1 --2- 172.21.15.92:0/1859432298 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1de80b8510 0x7f1de81f1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:03.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.875+0000 7f1dee1ff700 1 --2- 172.21.15.92:0/1859432298 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f1de80cd460 0x7f1de815a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:03.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.875+0000 7f1dee1ff700 1 -- 172.21.15.92:0/1859432298 >> 172.21.15.92:0/1859432298 conn(0x7f1de80719a0 msgr2=0x7f1de80b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:03.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.875+0000 7f1dee1ff700 1 -- 172.21.15.92:0/1859432298 shutdown_connections 2023-11-28T11:22:03.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:03.876+0000 7f1dee1ff700 1 -- 172.21.15.92:0/1859432298 wait complete. 2023-11-28T11:22:03.894 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:22:04.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.091+0000 7efc5abff700 1 Processor -- start 2023-11-28T11:22:04.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.091+0000 7efc5abff700 1 -- start start 2023-11-28T11:22:04.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.092+0000 7efc5abff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc540cd460 0x7efc5406c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:04.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.092+0000 7efc5abff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7efc540d0d60 0x7efc540d11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:04.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.092+0000 7efc5abff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc540b88e0 con 0x7efc540b8510 2023-11-28T11:22:04.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.092+0000 7efc5abff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc540cdbd0 con 0x7efc540d0d60 2023-11-28T11:22:04.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.092+0000 7efc5abff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc540d02d0 con 0x7efc540cd460 2023-11-28T11:22:04.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.093+0000 7efc5899b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7efc540b8510 0x7efc5406b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55514/0 (socket says 172.21.15.92:55514) 2023-11-28T11:22:04.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.093+0000 7efc5899b700 1 -- 172.21.15.92:0/3378492387 learned_addr learned my addr 172.21.15.92:0/3378492387 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:22:04.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.093+0000 7efc53fff700 1 --2- 172.21.15.92:0/3378492387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc540cd460 0x7efc5406c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:04.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.093+0000 7efc537fe700 1 -- 172.21.15.92:0/3378492387 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 134807616 0 0) 0x7efc540b88e0 con 0x7efc540b8510 2023-11-28T11:22:04.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.093+0000 7efc5919c700 1 --2- 172.21.15.92:0/3378492387 >> v2:172.21.15.169:3300/0 conn(0x7efc540d0d60 0x7efc540d11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:04.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.093+0000 7efc537fe700 1 -- 172.21.15.92:0/3378492387 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc440035b0 con 0x7efc540b8510 2023-11-28T11:22:04.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.093+0000 7efc537fe700 1 -- 172.21.15.92:0/3378492387 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1437847126 0 0) 0x7efc440035b0 con 0x7efc540b8510 2023-11-28T11:22:04.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.093+0000 7efc537fe700 1 -- 172.21.15.92:0/3378492387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc540cd460 msgr2=0x7efc5406c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:04.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.093+0000 7efc537fe700 1 --2- 172.21.15.92:0/3378492387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc540cd460 0x7efc5406c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:04.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.093+0000 7efc537fe700 1 -- 172.21.15.92:0/3378492387 >> v2:172.21.15.169:3300/0 conn(0x7efc540d0d60 msgr2=0x7efc540d11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:04.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.093+0000 7efc537fe700 1 --2- 172.21.15.92:0/3378492387 >> v2:172.21.15.169:3300/0 conn(0x7efc540d0d60 0x7efc540d11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:04.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.093+0000 7efc537fe700 1 -- 172.21.15.92:0/3378492387 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc540cd8f0 con 0x7efc540b8510 2023-11-28T11:22:04.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.093+0000 7efc537fe700 1 -- 172.21.15.92:0/3378492387 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532161587 0 0) 0x7efc48001670 con 0x7efc540b8510 2023-11-28T11:22:04.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.094+0000 7efc537fe700 1 -- 172.21.15.92:0/3378492387 >> v1:172.21.15.92:6789/0 conn(0x7efc540b8510 legacy=0x7efc5406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:04.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.094+0000 7efc537fe700 1 --2- 172.21.15.92:0/3378492387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc44003a30 0x7efc44003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:04.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.094+0000 7efc537fe700 1 --2- 172.21.15.92:0/3378492387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc44004850 0x7efc44004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:04.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.094+0000 7efc537fe700 1 --2- 172.21.15.92:0/3378492387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc540cd460 0x7efc44005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:04.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.094+0000 7efc537fe700 1 -- 172.21.15.92:0/3378492387 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc540cdbd0 con 0x7efc44003a30 2023-11-28T11:22:04.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.094+0000 7efc537fe700 1 -- 172.21.15.92:0/3378492387 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc540d02d0 con 0x7efc44004850 2023-11-28T11:22:04.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.094+0000 7efc537fe700 1 -- 172.21.15.92:0/3378492387 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc4400ba10 con 0x7efc540cd460 2023-11-28T11:22:04.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.094+0000 7efc5899b700 1 --2- 172.21.15.92:0/3378492387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc44003a30 0x7efc44003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:04.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.094+0000 7efc5919c700 1 --2- 172.21.15.92:0/3378492387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc540cd460 0x7efc44005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:04.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.094+0000 7efc53fff700 1 --2- 172.21.15.92:0/3378492387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc44004850 0x7efc44004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:04.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.094+0000 7efc5899b700 1 -- 172.21.15.92:0/3378492387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc540cd460 msgr2=0x7efc44005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:04.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.094+0000 7efc5899b700 1 --2- 172.21.15.92:0/3378492387 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc540cd460 0x7efc44005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:04.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.094+0000 7efc5899b700 1 -- 172.21.15.92:0/3378492387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc44004850 msgr2=0x7efc44004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:04.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.094+0000 7efc5899b700 1 --2- 172.21.15.92:0/3378492387 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc44004850 0x7efc44004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:04.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.094+0000 7efc5899b700 1 -- 172.21.15.92:0/3378492387 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc440035b0 con 0x7efc44003a30 2023-11-28T11:22:04.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.095+0000 7efc5899b700 1 --2- 172.21.15.92:0/3378492387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc44003a30 0x7efc44003e40 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7efc4800c240 tx=0x7efc44005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=874143a4fe7d52fa server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:04.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.095+0000 7efc537fe700 1 -- 172.21.15.92:0/3378492387 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc48007c10 con 0x7efc44003a30 2023-11-28T11:22:04.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.095+0000 7efc537fe700 1 -- 172.21.15.92:0/3378492387 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc4800ce00 con 0x7efc44003a30 2023-11-28T11:22:04.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.095+0000 7efc537fe700 1 -- 172.21.15.92:0/3378492387 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc48005ba0 con 0x7efc44003a30 2023-11-28T11:22:04.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.095+0000 7efc5abff700 1 -- 172.21.15.92:0/3378492387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc44003a30 msgr2=0x7efc44003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:04.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.095+0000 7efc5abff700 1 --2- 172.21.15.92:0/3378492387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc44003a30 0x7efc44003e40 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7efc4800c240 tx=0x7efc44005880 comp rx=0 tx=0).stop 2023-11-28T11:22:04.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.095+0000 7efc5919c700 1 -- 172.21.15.92:0/3378492387 reap_dead start 2023-11-28T11:22:04.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.095+0000 7efc5abff700 1 -- 172.21.15.92:0/3378492387 shutdown_connections 2023-11-28T11:22:04.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.095+0000 7efc5abff700 1 -- 172.21.15.92:0/3378492387 >> 172.21.15.92:0/3378492387 conn(0x7efc540719a0 msgr2=0x7efc54071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:04.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.095+0000 7efc5abff700 1 -- 172.21.15.92:0/3378492387 shutdown_connections 2023-11-28T11:22:04.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.096+0000 7efc5abff700 1 -- 172.21.15.92:0/3378492387 wait complete. 2023-11-28T11:22:04.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.097+0000 7efc5abff700 1 Processor -- start 2023-11-28T11:22:04.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.097+0000 7efc5abff700 1 -- start start 2023-11-28T11:22:04.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.098+0000 7efc5abff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc44003a30 0x7efc540cb640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:04.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.098+0000 7efc5abff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc54153eb0 0x7efc5415b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:04.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.098+0000 7efc5abff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc54156eb0 0x7efc54157320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:04.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.098+0000 7efc5abff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc54157860 con 0x7efc54156eb0 2023-11-28T11:22:04.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.098+0000 7efc5abff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc540cbb80 con 0x7efc54153eb0 2023-11-28T11:22:04.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.098+0000 7efc5abff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc5415a3d0 con 0x7efc44003a30 2023-11-28T11:22:04.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.098+0000 7efc5919c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc54156eb0 0x7efc54157320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:04.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.099+0000 7efc5919c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc54156eb0 0x7efc54157320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56518/0 (socket says 172.21.15.92:56518) 2023-11-28T11:22:04.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.099+0000 7efc5919c700 1 -- 172.21.15.92:0/2890053260 learned_addr learned my addr 172.21.15.92:0/2890053260 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:04.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.099+0000 7efc5899b700 1 --2- 172.21.15.92:0/2890053260 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc44003a30 0x7efc540cb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:04.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.099+0000 7efc53fff700 1 --2- 172.21.15.92:0/2890053260 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc54153eb0 0x7efc5415b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:04.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.099+0000 7efc5919c700 1 -- 172.21.15.92:0/2890053260 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc44003a30 msgr2=0x7efc540cb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:04.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.099+0000 7efc5919c700 1 --2- 172.21.15.92:0/2890053260 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc44003a30 0x7efc540cb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:04.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.099+0000 7efc5919c700 1 -- 172.21.15.92:0/2890053260 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc54153eb0 msgr2=0x7efc5415b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:04.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.099+0000 7efc5919c700 1 --2- 172.21.15.92:0/2890053260 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc54153eb0 0x7efc5415b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:04.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.099+0000 7efc5919c700 1 -- 172.21.15.92:0/2890053260 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc540cfff0 con 0x7efc54156eb0 2023-11-28T11:22:04.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.099+0000 7efc5919c700 1 --2- 172.21.15.92:0/2890053260 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc54156eb0 0x7efc54157320 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7efc540cdd70 tx=0x7efc4c011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:04.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.099+0000 7efc517fa700 1 -- 172.21.15.92:0/2890053260 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4c00e040 con 0x7efc54156eb0 2023-11-28T11:22:04.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.099+0000 7efc517fa700 1 -- 172.21.15.92:0/2890053260 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc4c00f040 con 0x7efc54156eb0 2023-11-28T11:22:04.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.100+0000 7efc5abff700 1 -- 172.21.15.92:0/2890053260 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc540cfd10 con 0x7efc54156eb0 2023-11-28T11:22:04.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.100+0000 7efc517fa700 1 -- 172.21.15.92:0/2890053260 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4c00fc40 con 0x7efc54156eb0 2023-11-28T11:22:04.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.100+0000 7efc5abff700 1 -- 172.21.15.92:0/2890053260 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc540d29c0 con 0x7efc54156eb0 2023-11-28T11:22:04.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.101+0000 7efc5abff700 1 -- 172.21.15.92:0/2890053260 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc54067880 con 0x7efc54156eb0 2023-11-28T11:22:04.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.102+0000 7efc517fa700 1 -- 172.21.15.92:0/2890053260 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7efc4c00fda0 con 0x7efc54156eb0 2023-11-28T11:22:04.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.103+0000 7efc517fa700 1 --2- 172.21.15.92:0/2890053260 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7efc38068320 0x7efc3806a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:04.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.103+0000 7efc517fa700 1 -- 172.21.15.92:0/2890053260 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7efc4c086470 con 0x7efc54156eb0 2023-11-28T11:22:04.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.103+0000 7efc5899b700 1 --2- 172.21.15.92:0/2890053260 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7efc38068320 0x7efc3806a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:04.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.104+0000 7efc5899b700 1 --2- 172.21.15.92:0/2890053260 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7efc38068320 0x7efc3806a7d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7efc48015040 tx=0x7efc48007000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:04.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.107+0000 7efc517fa700 1 -- 172.21.15.92:0/2890053260 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7efc4c0529a0 con 0x7efc54156eb0 2023-11-28T11:22:04.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.293+0000 7efc5abff700 1 -- 172.21.15.92:0/2890053260 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efc54072240 con 0x7efc54156eb0 2023-11-28T11:22:04.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.293+0000 7efc517fa700 1 -- 172.21.15.92:0/2890053260 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7efc4c055fc0 con 0x7efc54156eb0 2023-11-28T11:22:04.295 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-11-28T11:22:04.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.296+0000 7efc5abff700 1 -- 172.21.15.92:0/2890053260 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7efc38068320 msgr2=0x7efc3806a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:04.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.296+0000 7efc5abff700 1 --2- 172.21.15.92:0/2890053260 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7efc38068320 0x7efc3806a7d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7efc48015040 tx=0x7efc48007000 comp rx=0 tx=0).stop 2023-11-28T11:22:04.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.296+0000 7efc5abff700 1 -- 172.21.15.92:0/2890053260 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc54156eb0 msgr2=0x7efc54157320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:04.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.296+0000 7efc5abff700 1 --2- 172.21.15.92:0/2890053260 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc54156eb0 0x7efc54157320 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7efc540cdd70 tx=0x7efc4c011ed0 comp rx=0 tx=0).stop 2023-11-28T11:22:04.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.296+0000 7efc5abff700 1 -- 172.21.15.92:0/2890053260 shutdown_connections 2023-11-28T11:22:04.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.296+0000 7efc5abff700 1 --2- 172.21.15.92:0/2890053260 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7efc38068320 0x7efc3806a7d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:04.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.296+0000 7efc5abff700 1 --2- 172.21.15.92:0/2890053260 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc54156eb0 0x7efc54157320 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:04.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.296+0000 7efc5abff700 1 --2- 172.21.15.92:0/2890053260 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efc44003a30 0x7efc540cb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:04.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.296+0000 7efc5abff700 1 --2- 172.21.15.92:0/2890053260 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efc54153eb0 0x7efc5415b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:04.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.296+0000 7efc5abff700 1 -- 172.21.15.92:0/2890053260 >> 172.21.15.92:0/2890053260 conn(0x7efc540719a0 msgr2=0x7efc5415c680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:04.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.296+0000 7efc5abff700 1 -- 172.21.15.92:0/2890053260 shutdown_connections 2023-11-28T11:22:04.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.296+0000 7efc5abff700 1 -- 172.21.15.92:0/2890053260 wait complete. 2023-11-28T11:22:04.317 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:22:04.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.515+0000 7f2da6fff700 1 Processor -- start 2023-11-28T11:22:04.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.516+0000 7f2da6fff700 1 -- start start 2023-11-28T11:22:04.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.517+0000 7f2da6fff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2da00cd460 0x7f2da006c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:04.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.517+0000 7f2da6fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2da00d0d60 0x7f2da00d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:04.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.517+0000 7f2da6fff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2da00b88e0 con 0x7f2da00b8510 2023-11-28T11:22:04.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.517+0000 7f2da6fff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2da00cdbd0 con 0x7f2da00d0d60 2023-11-28T11:22:04.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.517+0000 7f2da6fff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2da00d02d0 con 0x7f2da00cd460 2023-11-28T11:22:04.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.517+0000 7f2da4d9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f2da00b8510 0x7f2da006b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55518/0 (socket says 172.21.15.92:55518) 2023-11-28T11:22:04.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.517+0000 7f2da4d9b700 1 -- 172.21.15.92:0/2031547954 learned_addr learned my addr 172.21.15.92:0/2031547954 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:22:04.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.517+0000 7f2da559c700 1 --2- 172.21.15.92:0/2031547954 >> v2:172.21.15.92:3300/0 conn(0x7f2da00d0d60 0x7f2da00d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:04.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.517+0000 7f2d9ffff700 1 --2- 172.21.15.92:0/2031547954 >> v2:172.21.15.169:3300/0 conn(0x7f2da00cd460 0x7f2da006c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:04.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.517+0000 7f2da559c700 1 -- 172.21.15.92:0/2031547954 >> v2:172.21.15.169:3300/0 conn(0x7f2da00cd460 msgr2=0x7f2da006c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:04.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.517+0000 7f2da559c700 1 --2- 172.21.15.92:0/2031547954 >> v2:172.21.15.169:3300/0 conn(0x7f2da00cd460 0x7f2da006c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:04.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.517+0000 7f2da559c700 1 -- 172.21.15.92:0/2031547954 >> v1:172.21.15.92:6789/0 conn(0x7f2da00b8510 legacy=0x7f2da006b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:04.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.517+0000 7f2da559c700 1 -- 172.21.15.92:0/2031547954 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da00cd8f0 con 0x7f2da00d0d60 2023-11-28T11:22:04.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.518+0000 7f2da559c700 1 --2- 172.21.15.92:0/2031547954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2da00d0d60 0x7f2da00d1200 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2da00ceba0 tx=0x7f2d9800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6be3b0e8b359d39e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:04.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.518+0000 7f2d9f7fe700 1 -- 172.21.15.92:0/2031547954 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9800e070 con 0x7f2da00d0d60 2023-11-28T11:22:04.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.518+0000 7f2d9f7fe700 1 -- 172.21.15.92:0/2031547954 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d9800bb90 con 0x7f2da00d0d60 2023-11-28T11:22:04.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.518+0000 7f2d9f7fe700 1 -- 172.21.15.92:0/2031547954 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d98012430 con 0x7f2da00d0d60 2023-11-28T11:22:04.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.518+0000 7f2da6fff700 1 -- 172.21.15.92:0/2031547954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2da00d0d60 msgr2=0x7f2da00d1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:04.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.518+0000 7f2da6fff700 1 --2- 172.21.15.92:0/2031547954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2da00d0d60 0x7f2da00d1200 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2da00ceba0 tx=0x7f2d9800b330 comp rx=0 tx=0).stop 2023-11-28T11:22:04.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.518+0000 7f2da6fff700 1 -- 172.21.15.92:0/2031547954 shutdown_connections 2023-11-28T11:22:04.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.518+0000 7f2da6fff700 1 --2- 172.21.15.92:0/2031547954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2da00d0d60 0x7f2da00d1200 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:04.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.518+0000 7f2da6fff700 1 --2- 172.21.15.92:0/2031547954 >> v2:172.21.15.169:3300/0 conn(0x7f2da00cd460 0x7f2da006c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:04.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.518+0000 7f2da6fff700 1 -- 172.21.15.92:0/2031547954 >> 172.21.15.92:0/2031547954 conn(0x7f2da00719a0 msgr2=0x7f2da0071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:04.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.518+0000 7f2da6fff700 1 -- 172.21.15.92:0/2031547954 shutdown_connections 2023-11-28T11:22:04.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.519+0000 7f2da6fff700 1 -- 172.21.15.92:0/2031547954 wait complete. 2023-11-28T11:22:04.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.520+0000 7f2da6fff700 1 Processor -- start 2023-11-28T11:22:04.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.520+0000 7f2da6fff700 1 -- start start 2023-11-28T11:22:04.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.521+0000 7f2da6fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2da00b8510 0x7f2da00ccb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:04.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.521+0000 7f2da6fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2da00cd460 0x7f2da00aaee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:04.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.521+0000 7f2da6fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2da00d0d60 0x7f2da0159160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:04.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.521+0000 7f2da6fff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da00aa850 con 0x7f2da00b8510 2023-11-28T11:22:04.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.521+0000 7f2da6fff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da00cc6f0 con 0x7f2da00d0d60 2023-11-28T11:22:04.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.521+0000 7f2da6fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2da00cbdc0 con 0x7f2da00cd460 2023-11-28T11:22:04.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.522+0000 7f2da4d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2da00b8510 0x7f2da00ccb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:04.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.522+0000 7f2da4d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2da00b8510 0x7f2da00ccb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56534/0 (socket says 172.21.15.92:56534) 2023-11-28T11:22:04.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.522+0000 7f2da4d9b700 1 -- 172.21.15.92:0/2153390752 learned_addr learned my addr 172.21.15.92:0/2153390752 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:04.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.522+0000 7f2da4d9b700 1 -- 172.21.15.92:0/2153390752 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2da00cd460 msgr2=0x7f2da00aaee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:04.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.522+0000 7f2da4d9b700 1 --2- 172.21.15.92:0/2153390752 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2da00cd460 0x7f2da00aaee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:04.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.522+0000 7f2da4d9b700 1 -- 172.21.15.92:0/2153390752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2da00d0d60 msgr2=0x7f2da0159160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:04.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.522+0000 7f2da4d9b700 1 --2- 172.21.15.92:0/2153390752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2da00d0d60 0x7f2da0159160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:04.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.522+0000 7f2da4d9b700 1 -- 172.21.15.92:0/2153390752 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d98009a70 con 0x7f2da00b8510 2023-11-28T11:22:04.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.522+0000 7f2da4d9b700 1 --2- 172.21.15.92:0/2153390752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2da00b8510 0x7f2da00ccb80 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2d9400c520 tx=0x7f2d9400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:04.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.522+0000 7f2d9d7fa700 1 -- 172.21.15.92:0/2153390752 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d94005d70 con 0x7f2da00b8510 2023-11-28T11:22:04.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.522+0000 7f2d9d7fa700 1 -- 172.21.15.92:0/2153390752 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d940070d0 con 0x7f2da00b8510 2023-11-28T11:22:04.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.522+0000 7f2da6fff700 1 -- 172.21.15.92:0/2153390752 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2da00cfff0 con 0x7f2da00b8510 2023-11-28T11:22:04.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.522+0000 7f2d9d7fa700 1 -- 172.21.15.92:0/2153390752 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d94007440 con 0x7f2da00b8510 2023-11-28T11:22:04.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.523+0000 7f2da6fff700 1 -- 172.21.15.92:0/2153390752 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2da00cfd10 con 0x7f2da00b8510 2023-11-28T11:22:04.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.524+0000 7f2da6fff700 1 -- 172.21.15.92:0/2153390752 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2da0067880 con 0x7f2da00b8510 2023-11-28T11:22:04.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.524+0000 7f2d9d7fa700 1 -- 172.21.15.92:0/2153390752 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2d94019030 con 0x7f2da00b8510 2023-11-28T11:22:04.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.525+0000 7f2d9d7fa700 1 --2- 172.21.15.92:0/2153390752 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f2d840682f0 0x7f2d8406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:04.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.526+0000 7f2d9d7fa700 1 -- 172.21.15.92:0/2153390752 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f2d94085340 con 0x7f2da00b8510 2023-11-28T11:22:04.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.526+0000 7f2d9ffff700 1 --2- 172.21.15.92:0/2153390752 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f2d840682f0 0x7f2d8406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:04.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.526+0000 7f2d9ffff700 1 --2- 172.21.15.92:0/2153390752 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f2d840682f0 0x7f2d8406a7a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2da00cd3f0 tx=0x7f2d8c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:04.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.529+0000 7f2d9d7fa700 1 -- 172.21.15.92:0/2153390752 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2d9404b5e0 con 0x7f2da00b8510 2023-11-28T11:22:04.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.720+0000 7f2da6fff700 1 -- 172.21.15.92:0/2153390752 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2da00d2940 con 0x7f2da00b8510 2023-11-28T11:22:04.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.721+0000 7f2d9d7fa700 1 -- 172.21.15.92:0/2153390752 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2d94051830 con 0x7f2da00b8510 2023-11-28T11:22:04.722 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-11-28T11:22:04.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.723+0000 7f2da6fff700 1 -- 172.21.15.92:0/2153390752 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f2d840682f0 msgr2=0x7f2d8406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:04.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.723+0000 7f2da6fff700 1 --2- 172.21.15.92:0/2153390752 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f2d840682f0 0x7f2d8406a7a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2da00cd3f0 tx=0x7f2d8c008040 comp rx=0 tx=0).stop 2023-11-28T11:22:04.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.723+0000 7f2da6fff700 1 -- 172.21.15.92:0/2153390752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2da00b8510 msgr2=0x7f2da00ccb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:04.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.723+0000 7f2da6fff700 1 --2- 172.21.15.92:0/2153390752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2da00b8510 0x7f2da00ccb80 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2d9400c520 tx=0x7f2d9400c8e0 comp rx=0 tx=0).stop 2023-11-28T11:22:04.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.723+0000 7f2da6fff700 1 -- 172.21.15.92:0/2153390752 shutdown_connections 2023-11-28T11:22:04.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.723+0000 7f2da6fff700 1 --2- 172.21.15.92:0/2153390752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2da00d0d60 0x7f2da0159160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:04.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.723+0000 7f2da6fff700 1 --2- 172.21.15.92:0/2153390752 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f2d840682f0 0x7f2d8406a7a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:04.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.723+0000 7f2da6fff700 1 --2- 172.21.15.92:0/2153390752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2da00b8510 0x7f2da00ccb80 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:04.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.723+0000 7f2da6fff700 1 --2- 172.21.15.92:0/2153390752 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2da00cd460 0x7f2da00aaee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:04.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.723+0000 7f2da6fff700 1 -- 172.21.15.92:0/2153390752 >> 172.21.15.92:0/2153390752 conn(0x7f2da00719a0 msgr2=0x7f2da00b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:04.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.723+0000 7f2da6fff700 1 -- 172.21.15.92:0/2153390752 shutdown_connections 2023-11-28T11:22:04.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.723+0000 7f2da6fff700 1 -- 172.21.15.92:0/2153390752 wait complete. 2023-11-28T11:22:04.742 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T11:22:04.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.940+0000 7f42ee5ff700 1 Processor -- start 2023-11-28T11:22:04.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.940+0000 7f42ee5ff700 1 -- start start 2023-11-28T11:22:04.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.941+0000 7f42ee5ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f42e80cd460 0x7f42e806c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:04.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.941+0000 7f42ee5ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42e80d0d60 0x7f42e80d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:04.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.941+0000 7f42ee5ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42e80b88e0 con 0x7f42e80b8510 2023-11-28T11:22:04.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.941+0000 7f42ee5ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42e80cdbd0 con 0x7f42e80cd460 2023-11-28T11:22:04.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.941+0000 7f42ee5ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42e80d02d0 con 0x7f42e80d0d60 2023-11-28T11:22:04.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.942+0000 7f42e77fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f42e80cd460 0x7f42e806c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:04.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.942+0000 7f42e77fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f42e80cd460 0x7f42e806c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56542/0 (socket says 172.21.15.92:56542) 2023-11-28T11:22:04.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.942+0000 7f42e77fe700 1 -- 172.21.15.92:0/2866830868 learned_addr learned my addr 172.21.15.92:0/2866830868 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:04.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.942+0000 7f42ecb9c700 1 --2- 172.21.15.92:0/2866830868 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42e80d0d60 0x7f42e80d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:04.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.942+0000 7f42e77fe700 1 -- 172.21.15.92:0/2866830868 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42e80d0d60 msgr2=0x7f42e80d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:04.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.942+0000 7f42e77fe700 1 --2- 172.21.15.92:0/2866830868 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42e80d0d60 0x7f42e80d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:04.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.942+0000 7f42e77fe700 1 -- 172.21.15.92:0/2866830868 >> v1:172.21.15.169:6789/0 conn(0x7f42e80b8510 legacy=0x7f42e806b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:04.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.942+0000 7f42e77fe700 1 -- 172.21.15.92:0/2866830868 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42e80cd8f0 con 0x7f42e80cd460 2023-11-28T11:22:04.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.942+0000 7f42e77fe700 1 --2- 172.21.15.92:0/2866830868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42e80cd460 0x7f42e806c080 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f42dc009920 tx=0x7f42dc009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de459202d9d7325a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:04.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.943+0000 7f42e6ffd700 1 -- 172.21.15.92:0/2866830868 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42dc00ba20 con 0x7f42e80cd460 2023-11-28T11:22:04.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.943+0000 7f42e6ffd700 1 -- 172.21.15.92:0/2866830868 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42dc00bb80 con 0x7f42e80cd460 2023-11-28T11:22:04.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.943+0000 7f42e6ffd700 1 -- 172.21.15.92:0/2866830868 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42dc009e70 con 0x7f42e80cd460 2023-11-28T11:22:04.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.943+0000 7f42ee5ff700 1 -- 172.21.15.92:0/2866830868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42e80cd460 msgr2=0x7f42e806c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:04.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.943+0000 7f42ee5ff700 1 --2- 172.21.15.92:0/2866830868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42e80cd460 0x7f42e806c080 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f42dc009920 tx=0x7f42dc009c30 comp rx=0 tx=0).stop 2023-11-28T11:22:04.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.943+0000 7f42ee5ff700 1 -- 172.21.15.92:0/2866830868 shutdown_connections 2023-11-28T11:22:04.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.943+0000 7f42ee5ff700 1 --2- 172.21.15.92:0/2866830868 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42e80d0d60 0x7f42e80d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:04.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.943+0000 7f42ee5ff700 1 --2- 172.21.15.92:0/2866830868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42e80cd460 0x7f42e806c080 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:04.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.943+0000 7f42ee5ff700 1 -- 172.21.15.92:0/2866830868 >> 172.21.15.92:0/2866830868 conn(0x7f42e80719a0 msgr2=0x7f42e8071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:04.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.943+0000 7f42ee5ff700 1 -- 172.21.15.92:0/2866830868 shutdown_connections 2023-11-28T11:22:04.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.943+0000 7f42ee5ff700 1 -- 172.21.15.92:0/2866830868 wait complete. 2023-11-28T11:22:04.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.945+0000 7f42ee5ff700 1 Processor -- start 2023-11-28T11:22:04.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.945+0000 7f42ee5ff700 1 -- start start 2023-11-28T11:22:04.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.946+0000 7f42ee5ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42e80b8510 0x7f42e81e4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:04.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.946+0000 7f42ee5ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42e80cd460 0x7f42e80cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:04.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.946+0000 7f42ee5ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42e80d0d60 0x7f42e81f2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:04.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.946+0000 7f42ee5ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e80cbdc0 con 0x7f42e80d0d60 2023-11-28T11:22:04.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.946+0000 7f42ee5ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e80caf30 con 0x7f42e80cd460 2023-11-28T11:22:04.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.946+0000 7f42ee5ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42e80cb140 con 0x7f42e80b8510 2023-11-28T11:22:04.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.946+0000 7f42ecb9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42e80d0d60 0x7f42e81f2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:04.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.946+0000 7f42ecb9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42e80d0d60 0x7f42e81f2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56544/0 (socket says 172.21.15.92:56544) 2023-11-28T11:22:04.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.946+0000 7f42ecb9c700 1 -- 172.21.15.92:0/295823509 learned_addr learned my addr 172.21.15.92:0/295823509 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:04.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.946+0000 7f42e7fff700 1 --2- 172.21.15.92:0/295823509 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42e80b8510 0x7f42e81e4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:04.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.946+0000 7f42e77fe700 1 --2- 172.21.15.92:0/295823509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42e80cd460 0x7f42e80cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:04.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.947+0000 7f42ecb9c700 1 -- 172.21.15.92:0/295823509 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42e80b8510 msgr2=0x7f42e81e4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:04.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.947+0000 7f42ecb9c700 1 --2- 172.21.15.92:0/295823509 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42e80b8510 0x7f42e81e4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:04.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.947+0000 7f42ecb9c700 1 -- 172.21.15.92:0/295823509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42e80cd460 msgr2=0x7f42e80cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:04.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.947+0000 7f42ecb9c700 1 --2- 172.21.15.92:0/295823509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42e80cd460 0x7f42e80cc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:04.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.947+0000 7f42ecb9c700 1 -- 172.21.15.92:0/295823509 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42dc009550 con 0x7f42e80d0d60 2023-11-28T11:22:04.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.947+0000 7f42ecb9c700 1 --2- 172.21.15.92:0/295823509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42e80d0d60 0x7f42e81f2ed0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f42e0007e60 tx=0x7f42e000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:04.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.947+0000 7f42e4ff9700 1 -- 172.21.15.92:0/295823509 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e000ede0 con 0x7f42e80d0d60 2023-11-28T11:22:04.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.947+0000 7f42e4ff9700 1 -- 172.21.15.92:0/295823509 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42e0014420 con 0x7f42e80d0d60 2023-11-28T11:22:04.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.947+0000 7f42e4ff9700 1 -- 172.21.15.92:0/295823509 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e00147d0 con 0x7f42e80d0d60 2023-11-28T11:22:04.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.947+0000 7f42ee5ff700 1 -- 172.21.15.92:0/295823509 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42e80cfff0 con 0x7f42e80d0d60 2023-11-28T11:22:04.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.947+0000 7f42ee5ff700 1 -- 172.21.15.92:0/295823509 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42e80cfd10 con 0x7f42e80d0d60 2023-11-28T11:22:04.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.948+0000 7f42ee5ff700 1 -- 172.21.15.92:0/295823509 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42e80cbf00 con 0x7f42e80d0d60 2023-11-28T11:22:04.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.949+0000 7f42e4ff9700 1 -- 172.21.15.92:0/295823509 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f42e0012050 con 0x7f42e80d0d60 2023-11-28T11:22:04.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.950+0000 7f42e4ff9700 1 --2- 172.21.15.92:0/295823509 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f42d40682f0 0x7f42d406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:04.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.950+0000 7f42e4ff9700 1 -- 172.21.15.92:0/295823509 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f42e00866e0 con 0x7f42e80d0d60 2023-11-28T11:22:04.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.950+0000 7f42e7fff700 1 --2- 172.21.15.92:0/295823509 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f42d40682f0 0x7f42d406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:04.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.951+0000 7f42e7fff700 1 --2- 172.21.15.92:0/295823509 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f42d40682f0 0x7f42d406a7a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f42d8000f60 tx=0x7f42d8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:04.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:04.954+0000 7f42e4ff9700 1 -- 172.21.15.92:0/295823509 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f42e80cbf00 con 0x7f42e80d0d60 2023-11-28T11:22:05.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.146+0000 7f42ee5ff700 1 -- 172.21.15.92:0/295823509 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f42e80cbf00 con 0x7f42e80d0d60 2023-11-28T11:22:05.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.146+0000 7f42e4ff9700 1 -- 172.21.15.92:0/295823509 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7f42e80cbf00 con 0x7f42e80d0d60 2023-11-28T11:22:05.148 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-11-28T11:22:05.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.148+0000 7f42ee5ff700 1 -- 172.21.15.92:0/295823509 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f42d40682f0 msgr2=0x7f42d406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:05.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.148+0000 7f42ee5ff700 1 --2- 172.21.15.92:0/295823509 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f42d40682f0 0x7f42d406a7a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f42d8000f60 tx=0x7f42d8009040 comp rx=0 tx=0).stop 2023-11-28T11:22:05.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.148+0000 7f42ee5ff700 1 -- 172.21.15.92:0/295823509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42e80d0d60 msgr2=0x7f42e81f2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:05.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.148+0000 7f42ee5ff700 1 --2- 172.21.15.92:0/295823509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42e80d0d60 0x7f42e81f2ed0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f42e0007e60 tx=0x7f42e000e3c0 comp rx=0 tx=0).stop 2023-11-28T11:22:05.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.149+0000 7f42ee5ff700 1 -- 172.21.15.92:0/295823509 shutdown_connections 2023-11-28T11:22:05.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.149+0000 7f42ee5ff700 1 --2- 172.21.15.92:0/295823509 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f42d40682f0 0x7f42d406a7a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:05.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.149+0000 7f42ee5ff700 1 --2- 172.21.15.92:0/295823509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42e80d0d60 0x7f42e81f2ed0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:05.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.149+0000 7f42ee5ff700 1 --2- 172.21.15.92:0/295823509 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f42e80b8510 0x7f42e81e4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:05.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.149+0000 7f42ee5ff700 1 --2- 172.21.15.92:0/295823509 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f42e80cd460 0x7f42e80cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:05.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.149+0000 7f42ee5ff700 1 -- 172.21.15.92:0/295823509 >> 172.21.15.92:0/295823509 conn(0x7f42e80719a0 msgr2=0x7f42e80b3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:05.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.149+0000 7f42ee5ff700 1 -- 172.21.15.92:0/295823509 shutdown_connections 2023-11-28T11:22:05.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.149+0000 7f42ee5ff700 1 -- 172.21.15.92:0/295823509 wait complete. 2023-11-28T11:22:05.168 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T11:22:05.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.364+0000 7f7b093ff700 1 Processor -- start 2023-11-28T11:22:05.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.364+0000 7f7b093ff700 1 -- start start 2023-11-28T11:22:05.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.365+0000 7f7b093ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b040b8510 0x7f7b0406b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:05.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.365+0000 7f7b093ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b040b88e0 con 0x7f7b040d0d60 2023-11-28T11:22:05.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.365+0000 7f7b093ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b040cd8f0 con 0x7f7b040cd460 2023-11-28T11:22:05.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.366+0000 7f7b093ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b040cdbd0 con 0x7f7b040b8510 2023-11-28T11:22:05.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.366+0000 7f7b037fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f7b040d0d60 0x7f7b040d11a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55522/0 (socket says 172.21.15.92:55522) 2023-11-28T11:22:05.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.366+0000 7f7b037fe700 1 -- 172.21.15.92:0/1579951226 learned_addr learned my addr 172.21.15.92:0/1579951226 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:22:05.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.366+0000 7f7b02ffd700 1 --2- 172.21.15.92:0/1579951226 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b040b8510 0x7f7b0406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:05.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.366+0000 7f7b01ffb700 1 -- 172.21.15.92:0/1579951226 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3412759150 0 0) 0x7f7b040b88e0 con 0x7f7b040d0d60 2023-11-28T11:22:05.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.366+0000 7f7b01ffb700 1 -- 172.21.15.92:0/1579951226 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7aec0035b0 con 0x7f7b040d0d60 2023-11-28T11:22:05.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.366+0000 7f7b01ffb700 1 -- 172.21.15.92:0/1579951226 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1782549368 0 0) 0x7f7b040cd8f0 con 0x7f7b040cd460 2023-11-28T11:22:05.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.366+0000 7f7b01ffb700 1 -- 172.21.15.92:0/1579951226 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b040b88e0 con 0x7f7b040cd460 2023-11-28T11:22:05.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.366+0000 7f7b02ffd700 1 -- 172.21.15.92:0/1579951226 >> v1:172.21.15.169:6789/0 conn(0x7f7b040cd460 legacy=0x7f7b0406beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:05.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.366+0000 7f7b02ffd700 1 -- 172.21.15.92:0/1579951226 >> v1:172.21.15.92:6789/0 conn(0x7f7b040d0d60 legacy=0x7f7b040d11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:05.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.366+0000 7f7b01ffb700 1 -- 172.21.15.92:0/1579951226 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3250738837 0 0) 0x7f7aec0035b0 con 0x7f7b040d0d60 2023-11-28T11:22:05.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.367+0000 7f7b02ffd700 1 -- 172.21.15.92:0/1579951226 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b040cfd10 con 0x7f7b040b8510 2023-11-28T11:22:05.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.367+0000 7f7b02ffd700 1 --2- 172.21.15.92:0/1579951226 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b040b8510 0x7f7b0406b970 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7af4009870 tx=0x7f7af4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42b8cacd5932d14b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:05.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.367+0000 7f7b01ffb700 1 -- 172.21.15.92:0/1579951226 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc0019f0 con 0x7f7b040b8510 2023-11-28T11:22:05.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.367+0000 7f7b01ffb700 1 -- 172.21.15.92:0/1579951226 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7af4004030 con 0x7f7b040b8510 2023-11-28T11:22:05.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.367+0000 7f7b01ffb700 1 -- 172.21.15.92:0/1579951226 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af40043c0 con 0x7f7b040b8510 2023-11-28T11:22:05.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.368+0000 7f7b093ff700 1 -- 172.21.15.92:0/1579951226 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b040b8510 msgr2=0x7f7b0406b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:05.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.368+0000 7f7b093ff700 1 --2- 172.21.15.92:0/1579951226 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b040b8510 0x7f7b0406b970 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7af4009870 tx=0x7f7af4009b80 comp rx=0 tx=0).stop 2023-11-28T11:22:05.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.368+0000 7f7b093ff700 1 -- 172.21.15.92:0/1579951226 shutdown_connections 2023-11-28T11:22:05.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.368+0000 7f7b093ff700 1 --2- 172.21.15.92:0/1579951226 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b040b8510 0x7f7b0406b970 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:05.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.368+0000 7f7b093ff700 1 -- 172.21.15.92:0/1579951226 >> 172.21.15.92:0/1579951226 conn(0x7f7b040719a0 msgr2=0x7f7b04071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:05.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.368+0000 7f7b093ff700 1 -- 172.21.15.92:0/1579951226 shutdown_connections 2023-11-28T11:22:05.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.368+0000 7f7b093ff700 1 -- 172.21.15.92:0/1579951226 wait complete. 2023-11-28T11:22:05.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.370+0000 7f7b093ff700 1 Processor -- start 2023-11-28T11:22:05.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.370+0000 7f7b093ff700 1 -- start start 2023-11-28T11:22:05.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.371+0000 7f7b093ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b040b8510 0x7f7b0414d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:05.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.371+0000 7f7b093ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b040cd460 0x7f7b040b6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:05.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.371+0000 7f7b093ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b040d0d60 0x7f7b040b7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:05.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.371+0000 7f7b093ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b040c9f80 con 0x7f7b040d0d60 2023-11-28T11:22:05.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.371+0000 7f7b093ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0414d7b0 con 0x7f7b040b8510 2023-11-28T11:22:05.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.371+0000 7f7b093ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b040ca220 con 0x7f7b040cd460 2023-11-28T11:22:05.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.371+0000 7f7b037fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b040d0d60 0x7f7b040b7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:05.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.371+0000 7f7b037fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b040d0d60 0x7f7b040b7410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56554/0 (socket says 172.21.15.92:56554) 2023-11-28T11:22:05.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.371+0000 7f7b037fe700 1 -- 172.21.15.92:0/3046339638 learned_addr learned my addr 172.21.15.92:0/3046339638 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:05.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.371+0000 7f7b02ffd700 1 --2- 172.21.15.92:0/3046339638 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b040b8510 0x7f7b0414d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:05.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.371+0000 7f7b027fc700 1 --2- 172.21.15.92:0/3046339638 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b040cd460 0x7f7b040b6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:05.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.371+0000 7f7b037fe700 1 -- 172.21.15.92:0/3046339638 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b040cd460 msgr2=0x7f7b040b6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:05.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.371+0000 7f7b037fe700 1 --2- 172.21.15.92:0/3046339638 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b040cd460 0x7f7b040b6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:05.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.371+0000 7f7b037fe700 1 -- 172.21.15.92:0/3046339638 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b040b8510 msgr2=0x7f7b0414d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:05.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.371+0000 7f7b037fe700 1 --2- 172.21.15.92:0/3046339638 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b040b8510 0x7f7b0414d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:05.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.371+0000 7f7b037fe700 1 -- 172.21.15.92:0/3046339638 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7af40094a0 con 0x7f7b040d0d60 2023-11-28T11:22:05.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.371+0000 7f7b027fc700 1 --2- 172.21.15.92:0/3046339638 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b040cd460 0x7f7b040b6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:22:05.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.372+0000 7f7b037fe700 1 --2- 172.21.15.92:0/3046339638 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b040d0d60 0x7f7b040b7410 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7afc00cc00 tx=0x7f7afc00cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:05.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.372+0000 7f7aeb7fe700 1 -- 172.21.15.92:0/3046339638 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc00ab30 con 0x7f7b040d0d60 2023-11-28T11:22:05.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.372+0000 7f7aeb7fe700 1 -- 172.21.15.92:0/3046339638 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7afc00ac90 con 0x7f7b040d0d60 2023-11-28T11:22:05.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.372+0000 7f7b093ff700 1 -- 172.21.15.92:0/3046339638 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b040d29c0 con 0x7f7b040d0d60 2023-11-28T11:22:05.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.372+0000 7f7aeb7fe700 1 -- 172.21.15.92:0/3046339638 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc0070d0 con 0x7f7b040d0d60 2023-11-28T11:22:05.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.372+0000 7f7b093ff700 1 -- 172.21.15.92:0/3046339638 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b040cfff0 con 0x7f7b040d0d60 2023-11-28T11:22:05.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.373+0000 7f7b093ff700 1 -- 172.21.15.92:0/3046339638 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b040d2b50 con 0x7f7b040d0d60 2023-11-28T11:22:05.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.374+0000 7f7aeb7fe700 1 -- 172.21.15.92:0/3046339638 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7afc010070 con 0x7f7b040d0d60 2023-11-28T11:22:05.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.375+0000 7f7aeb7fe700 1 --2- 172.21.15.92:0/3046339638 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7af00682f0 0x7f7af006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:05.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.375+0000 7f7aeb7fe700 1 -- 172.21.15.92:0/3046339638 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f7afc0859c0 con 0x7f7b040d0d60 2023-11-28T11:22:05.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.375+0000 7f7b02ffd700 1 --2- 172.21.15.92:0/3046339638 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7af00682f0 0x7f7af006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:05.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.376+0000 7f7b02ffd700 1 --2- 172.21.15.92:0/3046339638 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7af00682f0 0x7f7af006a7a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7af40097c0 tx=0x7f7af4008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:05.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.379+0000 7f7aeb7fe700 1 -- 172.21.15.92:0/3046339638 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7afc051ef0 con 0x7f7b040d0d60 2023-11-28T11:22:05.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.583+0000 7f7b093ff700 1 -- 172.21.15.92:0/3046339638 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7b04067880 con 0x7f7b040d0d60 2023-11-28T11:22:05.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.584+0000 7f7aeb7fe700 1 -- 172.21.15.92:0/3046339638 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7afc055510 con 0x7f7b040d0d60 2023-11-28T11:22:05.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.586+0000 7f7b093ff700 1 -- 172.21.15.92:0/3046339638 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7af00682f0 msgr2=0x7f7af006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:05.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.586+0000 7f7b093ff700 1 --2- 172.21.15.92:0/3046339638 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7af00682f0 0x7f7af006a7a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7af40097c0 tx=0x7f7af4008030 comp rx=0 tx=0).stop 2023-11-28T11:22:05.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.586+0000 7f7b093ff700 1 -- 172.21.15.92:0/3046339638 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b040d0d60 msgr2=0x7f7b040b7410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:05.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.586+0000 7f7b093ff700 1 --2- 172.21.15.92:0/3046339638 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b040d0d60 0x7f7b040b7410 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7afc00cc00 tx=0x7f7afc00cf10 comp rx=0 tx=0).stop 2023-11-28T11:22:05.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.586+0000 7f7b093ff700 1 -- 172.21.15.92:0/3046339638 shutdown_connections 2023-11-28T11:22:05.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.586+0000 7f7b093ff700 1 --2- 172.21.15.92:0/3046339638 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7af00682f0 0x7f7af006a7a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:05.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.586+0000 7f7b093ff700 1 --2- 172.21.15.92:0/3046339638 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b040d0d60 0x7f7b040b7410 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:05.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.586+0000 7f7b093ff700 1 --2- 172.21.15.92:0/3046339638 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7b040b8510 0x7f7b0414d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:05.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.586+0000 7f7b093ff700 1 --2- 172.21.15.92:0/3046339638 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7b040cd460 0x7f7b040b6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:05.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.586+0000 7f7b093ff700 1 -- 172.21.15.92:0/3046339638 >> 172.21.15.92:0/3046339638 conn(0x7f7b040719a0 msgr2=0x7f7b040b39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:05.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.586+0000 7f7b093ff700 1 -- 172.21.15.92:0/3046339638 shutdown_connections 2023-11-28T11:22:05.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.586+0000 7f7b093ff700 1 -- 172.21.15.92:0/3046339638 wait complete. 2023-11-28T11:22:05.588 INFO:teuthology.orchestra.run.smithi092.stderr:listed 0 entries 2023-11-28T11:22:05.605 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-11-28T11:22:05.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:22:05.605 DEBUG:teuthology.orchestra.run.smithi169:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-11-28T11:22:05.687 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi169.front.sepia.ceph.com 2023-11-28T11:22:05.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:22:05.688 DEBUG:teuthology.orchestra.run.smithi169:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-28T11:22:05.737 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-28T11:22:05.737 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:22:05.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.933+0000 7f8b015ff700 1 Processor -- start 2023-11-28T11:22:05.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.934+0000 7f8b015ff700 1 -- start start 2023-11-28T11:22:05.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.935+0000 7f8b015ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8afc0b8510 0x7f8afc06b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:05.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.935+0000 7f8b015ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8afc0d0d60 0x7f8afc0d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:05.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.935+0000 7f8b015ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8afc0b88e0 con 0x7f8afc0cd460 2023-11-28T11:22:05.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.935+0000 7f8b015ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8afc0cdbd0 con 0x7f8afc0b8510 2023-11-28T11:22:05.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.935+0000 7f8b015ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8afc0d02d0 con 0x7f8afc0d0d60 2023-11-28T11:22:05.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.935+0000 7f8afaffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8afc0b8510 0x7f8afc06b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:05.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.935+0000 7f8afaffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8afc0b8510 0x7f8afc06b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56556/0 (socket says 172.21.15.92:56556) 2023-11-28T11:22:05.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.935+0000 7f8afaffd700 1 -- 172.21.15.92:0/520260043 learned_addr learned my addr 172.21.15.92:0/520260043 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:05.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.935+0000 7f8afb7fe700 1 --2- 172.21.15.92:0/520260043 >> v2:172.21.15.169:3300/0 conn(0x7f8afc0d0d60 0x7f8afc0d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:05.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.935+0000 7f8afaffd700 1 -- 172.21.15.92:0/520260043 >> v2:172.21.15.169:3300/0 conn(0x7f8afc0d0d60 msgr2=0x7f8afc0d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:05.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.935+0000 7f8afaffd700 1 --2- 172.21.15.92:0/520260043 >> v2:172.21.15.169:3300/0 conn(0x7f8afc0d0d60 0x7f8afc0d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:05.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.935+0000 7f8afaffd700 1 -- 172.21.15.92:0/520260043 >> v1:172.21.15.92:6789/0 conn(0x7f8afc0cd460 legacy=0x7f8afc06beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:05.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.935+0000 7f8afaffd700 1 -- 172.21.15.92:0/520260043 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8afc0cd8f0 con 0x7f8afc0b8510 2023-11-28T11:22:05.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.936+0000 7f8afaffd700 1 --2- 172.21.15.92:0/520260043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8afc0b8510 0x7f8afc06b970 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8afc0ce8c0 tx=0x7f8aec009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=281b16fff49a5270 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:05.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.936+0000 7f8afa7fc700 1 -- 172.21.15.92:0/520260043 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aec011830 con 0x7f8afc0b8510 2023-11-28T11:22:05.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.936+0000 7f8afa7fc700 1 -- 172.21.15.92:0/520260043 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aec011990 con 0x7f8afc0b8510 2023-11-28T11:22:05.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.936+0000 7f8afa7fc700 1 -- 172.21.15.92:0/520260043 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aec011d20 con 0x7f8afc0b8510 2023-11-28T11:22:05.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.936+0000 7f8b015ff700 1 -- 172.21.15.92:0/520260043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8afc0b8510 msgr2=0x7f8afc06b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:05.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.936+0000 7f8b015ff700 1 --2- 172.21.15.92:0/520260043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8afc0b8510 0x7f8afc06b970 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8afc0ce8c0 tx=0x7f8aec009c00 comp rx=0 tx=0).stop 2023-11-28T11:22:05.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.937+0000 7f8b015ff700 1 -- 172.21.15.92:0/520260043 shutdown_connections 2023-11-28T11:22:05.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.937+0000 7f8b015ff700 1 --2- 172.21.15.92:0/520260043 >> v2:172.21.15.169:3300/0 conn(0x7f8afc0d0d60 0x7f8afc0d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:05.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.937+0000 7f8b015ff700 1 --2- 172.21.15.92:0/520260043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8afc0b8510 0x7f8afc06b970 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:05.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.937+0000 7f8b015ff700 1 -- 172.21.15.92:0/520260043 >> 172.21.15.92:0/520260043 conn(0x7f8afc0719a0 msgr2=0x7f8afc071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:05.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.937+0000 7f8b015ff700 1 -- 172.21.15.92:0/520260043 shutdown_connections 2023-11-28T11:22:05.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.937+0000 7f8b015ff700 1 -- 172.21.15.92:0/520260043 wait complete. 2023-11-28T11:22:05.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.939+0000 7f8b015ff700 1 Processor -- start 2023-11-28T11:22:05.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.939+0000 7f8b015ff700 1 -- start start 2023-11-28T11:22:05.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.940+0000 7f8b015ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8afc0b8510 0x7f8afc15b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:05.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.940+0000 7f8b015ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8afc0cd460 0x7f8afc15a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:05.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.940+0000 7f8b015ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8afc0d0d60 0x7f8afc1e5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:05.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.940+0000 7f8b015ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8afc1e4bb0 con 0x7f8afc0cd460 2023-11-28T11:22:05.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.940+0000 7f8b015ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8afc1e4e90 con 0x7f8afc0b8510 2023-11-28T11:22:05.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.940+0000 7f8b015ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8afc1e5170 con 0x7f8afc0d0d60 2023-11-28T11:22:05.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.941+0000 7f8af3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8afc0cd460 0x7f8afc15a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:05.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.941+0000 7f8af3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8afc0cd460 0x7f8afc15a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56562/0 (socket says 172.21.15.92:56562) 2023-11-28T11:22:05.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.941+0000 7f8af3fff700 1 -- 172.21.15.92:0/1895336663 learned_addr learned my addr 172.21.15.92:0/1895336663 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:05.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.941+0000 7f8afaffd700 1 --2- 172.21.15.92:0/1895336663 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8afc0b8510 0x7f8afc15b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:05.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.941+0000 7f8afb7fe700 1 --2- 172.21.15.92:0/1895336663 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8afc0d0d60 0x7f8afc1e5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:05.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.941+0000 7f8af3fff700 1 -- 172.21.15.92:0/1895336663 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8afc0d0d60 msgr2=0x7f8afc1e5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:05.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.941+0000 7f8af3fff700 1 --2- 172.21.15.92:0/1895336663 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8afc0d0d60 0x7f8afc1e5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:05.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.941+0000 7f8af3fff700 1 -- 172.21.15.92:0/1895336663 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8afc0b8510 msgr2=0x7f8afc15b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:05.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.941+0000 7f8af3fff700 1 --2- 172.21.15.92:0/1895336663 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8afc0b8510 0x7f8afc15b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:05.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.941+0000 7f8af3fff700 1 -- 172.21.15.92:0/1895336663 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aec009550 con 0x7f8afc0cd460 2023-11-28T11:22:05.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.941+0000 7f8af3fff700 1 --2- 172.21.15.92:0/1895336663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8afc0cd460 0x7f8afc15a8e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8ae400c480 tx=0x7f8ae400c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:05.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.942+0000 7f8af37fe700 1 -- 172.21.15.92:0/1895336663 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae40052d0 con 0x7f8afc0cd460 2023-11-28T11:22:05.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.942+0000 7f8af37fe700 1 -- 172.21.15.92:0/1895336663 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ae4005430 con 0x7f8afc0cd460 2023-11-28T11:22:05.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.942+0000 7f8af37fe700 1 -- 172.21.15.92:0/1895336663 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae40038c0 con 0x7f8afc0cd460 2023-11-28T11:22:05.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.942+0000 7f8b015ff700 1 -- 172.21.15.92:0/1895336663 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8afc0cfff0 con 0x7f8afc0cd460 2023-11-28T11:22:05.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.942+0000 7f8b015ff700 1 -- 172.21.15.92:0/1895336663 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8afc0cfd10 con 0x7f8afc0cd460 2023-11-28T11:22:05.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.943+0000 7f8b015ff700 1 -- 172.21.15.92:0/1895336663 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8afc067880 con 0x7f8afc0cd460 2023-11-28T11:22:05.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.944+0000 7f8af37fe700 1 -- 172.21.15.92:0/1895336663 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8ae4019030 con 0x7f8afc0cd460 2023-11-28T11:22:05.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.944+0000 7f8af37fe700 1 --2- 172.21.15.92:0/1895336663 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8adc0682f0 0x7f8adc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:05.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.944+0000 7f8af37fe700 1 -- 172.21.15.92:0/1895336663 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f8ae40852e0 con 0x7f8afc0cd460 2023-11-28T11:22:05.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.945+0000 7f8afaffd700 1 --2- 172.21.15.92:0/1895336663 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8adc0682f0 0x7f8adc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:05.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.945+0000 7f8afaffd700 1 --2- 172.21.15.92:0/1895336663 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8adc0682f0 0x7f8adc06a7a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8aec009870 tx=0x7f8aec008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:05.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:05.948+0000 7f8af37fe700 1 -- 172.21.15.92:0/1895336663 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8ae40517e0 con 0x7f8afc0cd460 2023-11-28T11:22:06.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.178+0000 7f8b015ff700 1 -- 172.21.15.92:0/1895336663 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8afc0d2940 con 0x7f8afc0cd460 2023-11-28T11:22:06.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.178+0000 7f8af37fe700 1 -- 172.21.15.92:0/1895336663 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4647 (secure 0 0 0) 0x7f8ae4054e00 con 0x7f8afc0cd460 2023-11-28T11:22:06.179 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:22:06.179 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/3257209990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3257209990},{"type":"v1","addr":"172.21.15.92:6836","nonce":3257209990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2193338949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2193338949},{"type":"v1","addr":"172.21.15.169:6834","nonce":2193338949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4308,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/471035619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":471035619},{"type":"v1","addr":"172.21.15.92:6837","nonce":471035619}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:20:56.234970+0000","modified":"2023-11-28T11:21:16.544352+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4326},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4326":{"gid":4326,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/2067251916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2067251916},{"type":"v1","addr":"172.21.15.169:6835","nonce":2067251916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T11:22:06.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.180+0000 7f8b015ff700 1 -- 172.21.15.92:0/1895336663 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8adc0682f0 msgr2=0x7f8adc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:06.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.180+0000 7f8b015ff700 1 --2- 172.21.15.92:0/1895336663 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8adc0682f0 0x7f8adc06a7a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8aec009870 tx=0x7f8aec008030 comp rx=0 tx=0).stop 2023-11-28T11:22:06.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.180+0000 7f8b015ff700 1 -- 172.21.15.92:0/1895336663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8afc0cd460 msgr2=0x7f8afc15a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:06.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.180+0000 7f8b015ff700 1 --2- 172.21.15.92:0/1895336663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8afc0cd460 0x7f8afc15a8e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8ae400c480 tx=0x7f8ae400c790 comp rx=0 tx=0).stop 2023-11-28T11:22:06.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.181+0000 7f8b015ff700 1 -- 172.21.15.92:0/1895336663 shutdown_connections 2023-11-28T11:22:06.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.181+0000 7f8b015ff700 1 --2- 172.21.15.92:0/1895336663 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8afc0d0d60 0x7f8afc1e5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:06.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.181+0000 7f8b015ff700 1 --2- 172.21.15.92:0/1895336663 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8afc0b8510 0x7f8afc15b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:06.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.181+0000 7f8b015ff700 1 --2- 172.21.15.92:0/1895336663 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8adc0682f0 0x7f8adc06a7a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:06.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.181+0000 7f8b015ff700 1 --2- 172.21.15.92:0/1895336663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8afc0cd460 0x7f8afc15a8e0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:06.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.181+0000 7f8b015ff700 1 -- 172.21.15.92:0/1895336663 >> 172.21.15.92:0/1895336663 conn(0x7f8afc0719a0 msgr2=0x7f8afc0b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:06.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.181+0000 7f8b015ff700 1 -- 172.21.15.92:0/1895336663 shutdown_connections 2023-11-28T11:22:06.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.181+0000 7f8b015ff700 1 -- 172.21.15.92:0/1895336663 wait complete. 2023-11-28T11:22:06.183 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 9 2023-11-28T11:22:06.200 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:22:06.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.394+0000 7fe7cb9ff700 1 Processor -- start 2023-11-28T11:22:06.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.395+0000 7fe7cb9ff700 1 -- start start 2023-11-28T11:22:06.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.396+0000 7fe7cb9ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe7c4142e80 0x7fe7c406a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:06.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.396+0000 7fe7cb9ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7c406a7b0 0x7fe7c40c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:06.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.396+0000 7fe7cb9ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe7c40ca430 0x7fe7c406d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:06.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.396+0000 7fe7cb9ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7c40c9380 con 0x7fe7c40ca430 2023-11-28T11:22:06.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.396+0000 7fe7cb9ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7c406ac00 con 0x7fe7c4142e80 2023-11-28T11:22:06.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.396+0000 7fe7cb9ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c4159b10 con 0x7fe7c406a7b0 2023-11-28T11:22:06.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.396+0000 7fe7c9f9c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe7c40ca430 0x7fe7c406d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:06.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.396+0000 7fe7c9f9c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe7c40ca430 0x7fe7c406d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56566/0 (socket says 172.21.15.92:56566) 2023-11-28T11:22:06.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.396+0000 7fe7c9f9c700 1 -- 172.21.15.92:0/798809536 learned_addr learned my addr 172.21.15.92:0/798809536 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:06.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.396+0000 7fe7c8f9a700 1 --2- 172.21.15.92:0/798809536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7c406a7b0 0x7fe7c40c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:06.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.396+0000 7fe7c979b700 1 --2- 172.21.15.92:0/798809536 >> v2:172.21.15.169:3300/0 conn(0x7fe7c4142e80 0x7fe7c406a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:06.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.397+0000 7fe7c9f9c700 1 -- 172.21.15.92:0/798809536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7c406a7b0 msgr2=0x7fe7c40c9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:06.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.397+0000 7fe7c9f9c700 1 --2- 172.21.15.92:0/798809536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7c406a7b0 0x7fe7c40c9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:06.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.397+0000 7fe7c9f9c700 1 -- 172.21.15.92:0/798809536 >> v2:172.21.15.169:3300/0 conn(0x7fe7c4142e80 msgr2=0x7fe7c406a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:06.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.397+0000 7fe7c9f9c700 1 --2- 172.21.15.92:0/798809536 >> v2:172.21.15.169:3300/0 conn(0x7fe7c4142e80 0x7fe7c406a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:06.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.397+0000 7fe7c9f9c700 1 -- 172.21.15.92:0/798809536 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7c40ca890 con 0x7fe7c40ca430 2023-11-28T11:22:06.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.397+0000 7fe7c9f9c700 1 --2- 172.21.15.92:0/798809536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7c40ca430 0x7fe7c406d2d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe7c40cb460 tx=0x7fe7bc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c71d2ce37c6ac0fe server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:06.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.397+0000 7fe7bbfff700 1 -- 172.21.15.92:0/798809536 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc00e070 con 0x7fe7c40ca430 2023-11-28T11:22:06.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.397+0000 7fe7bbfff700 1 -- 172.21.15.92:0/798809536 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7bc00b9c0 con 0x7fe7c40ca430 2023-11-28T11:22:06.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.397+0000 7fe7bbfff700 1 -- 172.21.15.92:0/798809536 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc00bd70 con 0x7fe7c40ca430 2023-11-28T11:22:06.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.398+0000 7fe7cb9ff700 1 -- 172.21.15.92:0/798809536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7c40ca430 msgr2=0x7fe7c406d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:06.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.398+0000 7fe7cb9ff700 1 --2- 172.21.15.92:0/798809536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7c40ca430 0x7fe7c406d2d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe7c40cb460 tx=0x7fe7bc00b330 comp rx=0 tx=0).stop 2023-11-28T11:22:06.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.398+0000 7fe7cb9ff700 1 -- 172.21.15.92:0/798809536 shutdown_connections 2023-11-28T11:22:06.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.398+0000 7fe7cb9ff700 1 --2- 172.21.15.92:0/798809536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7c40ca430 0x7fe7c406d2d0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:06.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.398+0000 7fe7cb9ff700 1 --2- 172.21.15.92:0/798809536 >> v2:172.21.15.169:3300/0 conn(0x7fe7c4142e80 0x7fe7c406a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:06.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.398+0000 7fe7cb9ff700 1 --2- 172.21.15.92:0/798809536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7c406a7b0 0x7fe7c40c9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:06.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.398+0000 7fe7cb9ff700 1 -- 172.21.15.92:0/798809536 >> 172.21.15.92:0/798809536 conn(0x7fe7c40cb490 msgr2=0x7fe7c40cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:06.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.398+0000 7fe7cb9ff700 1 -- 172.21.15.92:0/798809536 shutdown_connections 2023-11-28T11:22:06.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.398+0000 7fe7cb9ff700 1 -- 172.21.15.92:0/798809536 wait complete. 2023-11-28T11:22:06.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.400+0000 7fe7cb9ff700 1 Processor -- start 2023-11-28T11:22:06.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.400+0000 7fe7cb9ff700 1 -- start start 2023-11-28T11:22:06.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.401+0000 7fe7cb9ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7c406a7b0 0x7fe7c41efbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:06.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.401+0000 7fe7cb9ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7c40ca430 0x7fe7c40c2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:06.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.401+0000 7fe7cb9ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7c4142e80 0x7fe7c40c29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:06.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.401+0000 7fe7cb9ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c415a2b0 con 0x7fe7c40ca430 2023-11-28T11:22:06.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.401+0000 7fe7cb9ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c415ab50 con 0x7fe7c406a7b0 2023-11-28T11:22:06.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.401+0000 7fe7cb9ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c40c8330 con 0x7fe7c4142e80 2023-11-28T11:22:06.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.401+0000 7fe7c8f9a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7c40ca430 0x7fe7c40c2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:06.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.401+0000 7fe7c8f9a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7c40ca430 0x7fe7c40c2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56570/0 (socket says 172.21.15.92:56570) 2023-11-28T11:22:06.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.401+0000 7fe7c8f9a700 1 -- 172.21.15.92:0/2814351094 learned_addr learned my addr 172.21.15.92:0/2814351094 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:06.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.401+0000 7fe7c979b700 1 --2- 172.21.15.92:0/2814351094 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7c406a7b0 0x7fe7c41efbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:06.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.401+0000 7fe7c9f9c700 1 --2- 172.21.15.92:0/2814351094 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7c4142e80 0x7fe7c40c29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:06.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.401+0000 7fe7c8f9a700 1 -- 172.21.15.92:0/2814351094 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7c4142e80 msgr2=0x7fe7c40c29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:06.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.401+0000 7fe7c8f9a700 1 --2- 172.21.15.92:0/2814351094 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7c4142e80 0x7fe7c40c29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:06.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.401+0000 7fe7c8f9a700 1 -- 172.21.15.92:0/2814351094 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7c406a7b0 msgr2=0x7fe7c41efbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:06.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.401+0000 7fe7c8f9a700 1 --2- 172.21.15.92:0/2814351094 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7c406a7b0 0x7fe7c41efbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:06.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.401+0000 7fe7c8f9a700 1 -- 172.21.15.92:0/2814351094 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7bc009b30 con 0x7fe7c40ca430 2023-11-28T11:22:06.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.402+0000 7fe7c8f9a700 1 --2- 172.21.15.92:0/2814351094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7c40ca430 0x7fe7c40c2490 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe7b400e810 tx=0x7fe7b400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:06.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.402+0000 7fe7b9ffb700 1 -- 172.21.15.92:0/2814351094 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b400c070 con 0x7fe7c40ca430 2023-11-28T11:22:06.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.402+0000 7fe7b9ffb700 1 -- 172.21.15.92:0/2814351094 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7b4013610 con 0x7fe7c40ca430 2023-11-28T11:22:06.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.402+0000 7fe7b9ffb700 1 -- 172.21.15.92:0/2814351094 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b40139c0 con 0x7fe7c40ca430 2023-11-28T11:22:06.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.402+0000 7fe7cb9ff700 1 -- 172.21.15.92:0/2814351094 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7c40cab00 con 0x7fe7c40ca430 2023-11-28T11:22:06.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.402+0000 7fe7cb9ff700 1 -- 172.21.15.92:0/2814351094 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7c4143250 con 0x7fe7c40ca430 2023-11-28T11:22:06.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.403+0000 7fe7cb9ff700 1 -- 172.21.15.92:0/2814351094 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7c40cc5f0 con 0x7fe7c40ca430 2023-11-28T11:22:06.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.404+0000 7fe7b9ffb700 1 -- 172.21.15.92:0/2814351094 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe7b4023070 con 0x7fe7c40ca430 2023-11-28T11:22:06.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.405+0000 7fe7b9ffb700 1 --2- 172.21.15.92:0/2814351094 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe7a80682f0 0x7fe7a806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:06.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.405+0000 7fe7b9ffb700 1 -- 172.21.15.92:0/2814351094 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fe7b4085df0 con 0x7fe7c40ca430 2023-11-28T11:22:06.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.405+0000 7fe7c979b700 1 --2- 172.21.15.92:0/2814351094 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe7a80682f0 0x7fe7a806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:06.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.406+0000 7fe7c979b700 1 --2- 172.21.15.92:0/2814351094 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe7a80682f0 0x7fe7a806a7a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0006b20 tx=0x7fe7c0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:06.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.409+0000 7fe7b9ffb700 1 -- 172.21.15.92:0/2814351094 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe7b4052240 con 0x7fe7c40ca430 2023-11-28T11:22:06.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.602+0000 7fe7cb9ff700 1 -- 172.21.15.92:0/2814351094 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe7c415acc0 con 0x7fe7c40ca430 2023-11-28T11:22:06.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.604+0000 7fe7b9ffb700 1 -- 172.21.15.92:0/2814351094 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+19844 (secure 0 0 0) 0x7fe7b4059e60 con 0x7fe7c40ca430 2023-11-28T11:22:06.605 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:22:06.605 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":29,"fsid":"68956e0c-c16f-47cd-a220-f98364b0831d","created":"2023-11-28T11:20:28.697573+0000","modified":"2023-11-28T11:21:37.638578+0000","last_up_change":"2023-11-28T11:20:41.257011+0000","last_in_change":"2023-11-28T11:20:32.249767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:20:41.925237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:20:48.682497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:20:54.471398+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:20:55.204585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:20:58.297307+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6abe483f-0e03-4a26-b9cf-62041291747e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":115823},{"type":"v1","addr":"172.21.15.92:6809","nonce":115823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":115823},{"type":"v1","addr":"172.21.15.92:6813","nonce":115823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":115823},{"type":"v1","addr":"172.21.15.92:6821","nonce":115823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":115823},{"type":"v1","addr":"172.21.15.92:6817","nonce":115823}]},"public_addr":"172.21.15.92:6809/115823","cluster_addr":"172.21.15.92:6813/115823","heartbeat_back_addr":"172.21.15.92:6821/115823","heartbeat_front_addr":"172.21.15.92:6817/115823","state":["exists","up"]},{"osd":1,"uuid":"13c1de73-1a8c-4d7e-8a28-9dae52dfe089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":115824},{"type":"v1","addr":"172.21.15.92:6812","nonce":115824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":115824},{"type":"v1","addr":"172.21.15.92:6816","nonce":115824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":115824},{"type":"v1","addr":"172.21.15.92:6823","nonce":115824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":115824},{"type":"v1","addr":"172.21.15.92:6820","nonce":115824}]},"public_addr":"172.21.15.92:6812/115824","cluster_addr":"172.21.15.92:6816/115824","heartbeat_back_addr":"172.21.15.92:6823/115824","heartbeat_front_addr":"172.21.15.92:6820/115824","state":["exists","up"]},{"osd":2,"uuid":"25788a55-112f-4f00-810d-2bc37cf76778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":115821},{"type":"v1","addr":"172.21.15.92:6801","nonce":115821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":115821},{"type":"v1","addr":"172.21.15.92:6803","nonce":115821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":115821},{"type":"v1","addr":"172.21.15.92:6807","nonce":115821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":115821},{"type":"v1","addr":"172.21.15.92:6805","nonce":115821}]},"public_addr":"172.21.15.92:6801/115821","cluster_addr":"172.21.15.92:6803/115821","heartbeat_back_addr":"172.21.15.92:6807/115821","heartbeat_front_addr":"172.21.15.92:6805/115821","state":["exists","up"]},{"osd":3,"uuid":"adf54e24-33a9-41fe-9ce3-98a5ab6c158e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":115825},{"type":"v1","addr":"172.21.15.92:6825","nonce":115825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":115825},{"type":"v1","addr":"172.21.15.92:6827","nonce":115825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":115825},{"type":"v1","addr":"172.21.15.92:6831","nonce":115825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":115825},{"type":"v1","addr":"172.21.15.92:6829","nonce":115825}]},"public_addr":"172.21.15.92:6825/115825","cluster_addr":"172.21.15.92:6827/115825","heartbeat_back_addr":"172.21.15.92:6831/115825","heartbeat_front_addr":"172.21.15.92:6829/115825","state":["exists","up"]},{"osd":4,"uuid":"ccb9b747-f399-44ea-9a18-9db3152227e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":116035},{"type":"v1","addr":"172.21.15.169:6817","nonce":116035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":116035},{"type":"v1","addr":"172.21.15.169:6819","nonce":116035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":116035},{"type":"v1","addr":"172.21.15.169:6826","nonce":116035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":116035},{"type":"v1","addr":"172.21.15.169:6822","nonce":116035}]},"public_addr":"172.21.15.169:6817/116035","cluster_addr":"172.21.15.169:6819/116035","heartbeat_back_addr":"172.21.15.169:6826/116035","heartbeat_front_addr":"172.21.15.169:6822/116035","state":["exists","up"]},{"osd":5,"uuid":"308a1e63-75aa-4d7d-8a7c-cecc24714fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":116033},{"type":"v1","addr":"172.21.15.169:6801","nonce":116033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":116033},{"type":"v1","addr":"172.21.15.169:6803","nonce":116033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":116033},{"type":"v1","addr":"172.21.15.169:6807","nonce":116033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":116033},{"type":"v1","addr":"172.21.15.169:6805","nonce":116033}]},"public_addr":"172.21.15.169:6801/116033","cluster_addr":"172.21.15.169:6803/116033","heartbeat_back_addr":"172.21.15.169:6807/116033","heartbeat_front_addr":"172.21.15.169:6805/116033","state":["exists","up"]},{"osd":6,"uuid":"9c459b7b-b821-4b4b-b91e-6aaae271457c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":116034},{"type":"v1","addr":"172.21.15.169:6809","nonce":116034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":116034},{"type":"v1","addr":"172.21.15.169:6811","nonce":116034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":116034},{"type":"v1","addr":"172.21.15.169:6815","nonce":116034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":116034},{"type":"v1","addr":"172.21.15.169:6813","nonce":116034}]},"public_addr":"172.21.15.169:6809/116034","cluster_addr":"172.21.15.169:6811/116034","heartbeat_back_addr":"172.21.15.169:6815/116034","heartbeat_front_addr":"172.21.15.169:6813/116034","state":["exists","up"]},{"osd":7,"uuid":"1ca9aa4a-755d-4e28-a937-4d8dccab4df3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":116044},{"type":"v1","addr":"172.21.15.169:6823","nonce":116044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6825","nonce":116044},{"type":"v1","addr":"172.21.15.169:6827","nonce":116044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":116044},{"type":"v1","addr":"172.21.15.169:6831","nonce":116044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":116044},{"type":"v1","addr":"172.21.15.169:6829","nonce":116044}]},"public_addr":"172.21.15.169:6823/116044","cluster_addr":"172.21.15.169:6827/116044","heartbeat_back_addr":"172.21.15.169:6831/116044","heartbeat_front_addr":"172.21.15.169:6829/116044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.586781+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.869778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.508594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.860493+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.872267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.746933+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.611520+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.871446+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.1d","mappings":[{"from":3,"to":2}]},{"pgid":"4.21","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:22:06.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.606+0000 7fe7cb9ff700 1 -- 172.21.15.92:0/2814351094 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe7a80682f0 msgr2=0x7fe7a806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:06.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.606+0000 7fe7cb9ff700 1 --2- 172.21.15.92:0/2814351094 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe7a80682f0 0x7fe7a806a7a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0006b20 tx=0x7fe7c0008040 comp rx=0 tx=0).stop 2023-11-28T11:22:06.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.606+0000 7fe7cb9ff700 1 -- 172.21.15.92:0/2814351094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7c40ca430 msgr2=0x7fe7c40c2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:06.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.606+0000 7fe7cb9ff700 1 --2- 172.21.15.92:0/2814351094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7c40ca430 0x7fe7c40c2490 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe7b400e810 tx=0x7fe7b400eb20 comp rx=0 tx=0).stop 2023-11-28T11:22:06.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.606+0000 7fe7cb9ff700 1 -- 172.21.15.92:0/2814351094 shutdown_connections 2023-11-28T11:22:06.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.606+0000 7fe7cb9ff700 1 --2- 172.21.15.92:0/2814351094 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7c4142e80 0x7fe7c40c29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:06.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.606+0000 7fe7cb9ff700 1 --2- 172.21.15.92:0/2814351094 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7c406a7b0 0x7fe7c41efbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:06.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.606+0000 7fe7cb9ff700 1 --2- 172.21.15.92:0/2814351094 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe7a80682f0 0x7fe7a806a7a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:06.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.606+0000 7fe7cb9ff700 1 --2- 172.21.15.92:0/2814351094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7c40ca430 0x7fe7c40c2490 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:06.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.606+0000 7fe7cb9ff700 1 -- 172.21.15.92:0/2814351094 >> 172.21.15.92:0/2814351094 conn(0x7fe7c40cb490 msgr2=0x7fe7c41590b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:06.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.606+0000 7fe7cb9ff700 1 -- 172.21.15.92:0/2814351094 shutdown_connections 2023-11-28T11:22:06.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.606+0000 7fe7cb9ff700 1 -- 172.21.15.92:0/2814351094 wait complete. 2023-11-28T11:22:06.625 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:20:41.925237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:20:48.682497+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:20:54.471398+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:20:55.204585+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T11:20:58.297307+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T11:22:06.626 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:22:06.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.826+0000 7fc2615ff700 1 Processor -- start 2023-11-28T11:22:06.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.826+0000 7fc2615ff700 1 -- start start 2023-11-28T11:22:06.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.828+0000 7fc2615ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc25c0cd460 0x7fc25c06c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:06.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.828+0000 7fc2615ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc25c0d0d60 0x7fc25c0d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:06.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.828+0000 7fc2615ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc25c0b88e0 con 0x7fc25c0b8510 2023-11-28T11:22:06.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.828+0000 7fc2615ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc25c0cdbd0 con 0x7fc25c0cd460 2023-11-28T11:22:06.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.828+0000 7fc2615ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc25c0d02d0 con 0x7fc25c0d0d60 2023-11-28T11:22:06.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.828+0000 7fc25b7fe700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc25c0d0d60 0x7fc25c0d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:06.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.828+0000 7fc253fff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc25c0cd460 0x7fc25c06c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:06.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.828+0000 7fc25affd700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fc25c0b8510 0x7fc25c06b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.92:43924/0 (socket says 172.21.15.92:43924) 2023-11-28T11:22:06.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.828+0000 7fc25b7fe700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc25c0d0d60 0x7fc25c0d1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.92:47252/0 (socket says 172.21.15.92:47252) 2023-11-28T11:22:06.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.828+0000 7fc25affd700 1 -- 172.21.15.92:0/1558732121 learned_addr learned my addr 172.21.15.92:0/1558732121 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:22:06.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.829+0000 7fc25a7fc700 1 -- 172.21.15.92:0/1558732121 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2624601635 0 0) 0x7fc25c0b88e0 con 0x7fc25c0b8510 2023-11-28T11:22:06.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.829+0000 7fc25a7fc700 1 -- 172.21.15.92:0/1558732121 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2480035b0 con 0x7fc25c0b8510 2023-11-28T11:22:06.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.829+0000 7fc253fff700 1 -- 172.21.15.92:0/1558732121 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc25c0d0d60 msgr2=0x7fc25c0d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:06.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.829+0000 7fc253fff700 1 --2- 172.21.15.92:0/1558732121 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc25c0d0d60 0x7fc25c0d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:06.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.829+0000 7fc253fff700 1 -- 172.21.15.92:0/1558732121 >> v1:172.21.15.169:6789/0 conn(0x7fc25c0b8510 legacy=0x7fc25c06b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:06.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.829+0000 7fc253fff700 1 -- 172.21.15.92:0/1558732121 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc25c0cd8f0 con 0x7fc25c0cd460 2023-11-28T11:22:06.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.829+0000 7fc25b7fe700 1 --2- 172.21.15.92:0/1558732121 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc25c0d0d60 0x7fc25c0d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:22:06.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.829+0000 7fc253fff700 1 --2- 172.21.15.92:0/1558732121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc25c0cd460 0x7fc25c06c080 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc244009870 tx=0x7fc244009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a890c626cf4aae7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:06.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.829+0000 7fc25a7fc700 1 -- 172.21.15.92:0/1558732121 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc244011830 con 0x7fc25c0cd460 2023-11-28T11:22:06.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.829+0000 7fc25a7fc700 1 -- 172.21.15.92:0/1558732121 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc244011990 con 0x7fc25c0cd460 2023-11-28T11:22:06.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.829+0000 7fc25a7fc700 1 -- 172.21.15.92:0/1558732121 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc244011d20 con 0x7fc25c0cd460 2023-11-28T11:22:06.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.830+0000 7fc2615ff700 1 -- 172.21.15.92:0/1558732121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc25c0cd460 msgr2=0x7fc25c06c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:06.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.830+0000 7fc2615ff700 1 --2- 172.21.15.92:0/1558732121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc25c0cd460 0x7fc25c06c080 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc244009870 tx=0x7fc244009b80 comp rx=0 tx=0).stop 2023-11-28T11:22:06.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.830+0000 7fc2615ff700 1 -- 172.21.15.92:0/1558732121 shutdown_connections 2023-11-28T11:22:06.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.830+0000 7fc2615ff700 1 --2- 172.21.15.92:0/1558732121 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc25c0d0d60 0x7fc25c0d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:06.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.830+0000 7fc2615ff700 1 --2- 172.21.15.92:0/1558732121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc25c0cd460 0x7fc25c06c080 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:06.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.830+0000 7fc2615ff700 1 -- 172.21.15.92:0/1558732121 >> 172.21.15.92:0/1558732121 conn(0x7fc25c0719a0 msgr2=0x7fc25c071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:06.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.830+0000 7fc2615ff700 1 -- 172.21.15.92:0/1558732121 shutdown_connections 2023-11-28T11:22:06.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.830+0000 7fc2615ff700 1 -- 172.21.15.92:0/1558732121 wait complete. 2023-11-28T11:22:06.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.832+0000 7fc2615ff700 1 Processor -- start 2023-11-28T11:22:06.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.832+0000 7fc2615ff700 1 -- start start 2023-11-28T11:22:06.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.833+0000 7fc2615ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc25c0b8510 0x7fc25c0ca480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:06.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.833+0000 7fc2615ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc25c0cd460 0x7fc25c0ca9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:06.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.833+0000 7fc2615ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc25c0d0d60 0x7fc25c15b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:06.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.833+0000 7fc2615ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25c1f3110 con 0x7fc25c0cd460 2023-11-28T11:22:06.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.833+0000 7fc2615ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25c1f2500 con 0x7fc25c0b8510 2023-11-28T11:22:06.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.833+0000 7fc2615ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc25c1f27e0 con 0x7fc25c0d0d60 2023-11-28T11:22:06.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.833+0000 7fc253fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc25c0cd460 0x7fc25c0ca9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:06.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.833+0000 7fc253fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc25c0cd460 0x7fc25c0ca9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56576/0 (socket says 172.21.15.92:56576) 2023-11-28T11:22:06.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.833+0000 7fc253fff700 1 -- 172.21.15.92:0/2608581177 learned_addr learned my addr 172.21.15.92:0/2608581177 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:06.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.833+0000 7fc25b7fe700 1 --2- 172.21.15.92:0/2608581177 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc25c0d0d60 0x7fc25c15b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:06.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.833+0000 7fc25affd700 1 --2- 172.21.15.92:0/2608581177 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc25c0b8510 0x7fc25c0ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:06.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.833+0000 7fc253fff700 1 -- 172.21.15.92:0/2608581177 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc25c0d0d60 msgr2=0x7fc25c15b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:06.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.833+0000 7fc253fff700 1 --2- 172.21.15.92:0/2608581177 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc25c0d0d60 0x7fc25c15b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:06.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.833+0000 7fc253fff700 1 -- 172.21.15.92:0/2608581177 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc25c0b8510 msgr2=0x7fc25c0ca480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:06.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.833+0000 7fc253fff700 1 --2- 172.21.15.92:0/2608581177 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc25c0b8510 0x7fc25c0ca480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:06.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.833+0000 7fc253fff700 1 -- 172.21.15.92:0/2608581177 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc254009a70 con 0x7fc25c0cd460 2023-11-28T11:22:06.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.833+0000 7fc25b7fe700 1 --2- 172.21.15.92:0/2608581177 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc25c0d0d60 0x7fc25c15b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:22:06.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.833+0000 7fc25affd700 1 --2- 172.21.15.92:0/2608581177 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc25c0b8510 0x7fc25c0ca480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:22:06.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.834+0000 7fc253fff700 1 --2- 172.21.15.92:0/2608581177 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc25c0cd460 0x7fc25c0ca9c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc244000c00 tx=0x7fc244004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:06.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.834+0000 7fc2537fe700 1 -- 172.21.15.92:0/2608581177 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2440036a0 con 0x7fc25c0cd460 2023-11-28T11:22:06.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.834+0000 7fc2537fe700 1 -- 172.21.15.92:0/2608581177 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc24401f070 con 0x7fc25c0cd460 2023-11-28T11:22:06.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.834+0000 7fc2537fe700 1 -- 172.21.15.92:0/2608581177 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc244003a40 con 0x7fc25c0cd460 2023-11-28T11:22:06.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.834+0000 7fc2615ff700 1 -- 172.21.15.92:0/2608581177 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2440094a0 con 0x7fc25c0cd460 2023-11-28T11:22:06.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.834+0000 7fc2615ff700 1 -- 172.21.15.92:0/2608581177 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc25c0cfff0 con 0x7fc25c0cd460 2023-11-28T11:22:06.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.835+0000 7fc2615ff700 1 -- 172.21.15.92:0/2608581177 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc25c1e4750 con 0x7fc25c0cd460 2023-11-28T11:22:06.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.836+0000 7fc2537fe700 1 -- 172.21.15.92:0/2608581177 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc24400f050 con 0x7fc25c0cd460 2023-11-28T11:22:06.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.837+0000 7fc2537fe700 1 --2- 172.21.15.92:0/2608581177 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc23c0682f0 0x7fc23c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:06.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.837+0000 7fc2537fe700 1 -- 172.21.15.92:0/2608581177 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fc244021070 con 0x7fc25c0cd460 2023-11-28T11:22:06.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.837+0000 7fc25affd700 1 --2- 172.21.15.92:0/2608581177 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc23c0682f0 0x7fc23c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:06.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.838+0000 7fc25affd700 1 --2- 172.21.15.92:0/2608581177 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc23c0682f0 0x7fc23c06a7a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc24c0011e0 tx=0x7fc24c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:06.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:06.841+0000 7fc2537fe700 1 -- 172.21.15.92:0/2608581177 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc25c1e4750 con 0x7fc25c0cd460 2023-11-28T11:22:07.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.032+0000 7fc2615ff700 1 -- 172.21.15.92:0/2608581177 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc25c1e4750 con 0x7fc25c0cd460 2023-11-28T11:22:07.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.033+0000 7fc2537fe700 1 -- 172.21.15.92:0/2608581177 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc24402b020 con 0x7fc25c0cd460 2023-11-28T11:22:07.034 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-11-28T11:22:07.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.035+0000 7fc2615ff700 1 -- 172.21.15.92:0/2608581177 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc23c0682f0 msgr2=0x7fc23c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:07.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.035+0000 7fc2615ff700 1 --2- 172.21.15.92:0/2608581177 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc23c0682f0 0x7fc23c06a7a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc24c0011e0 tx=0x7fc24c009040 comp rx=0 tx=0).stop 2023-11-28T11:22:07.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.035+0000 7fc2615ff700 1 -- 172.21.15.92:0/2608581177 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc25c0cd460 msgr2=0x7fc25c0ca9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:07.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.035+0000 7fc2615ff700 1 --2- 172.21.15.92:0/2608581177 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc25c0cd460 0x7fc25c0ca9c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc244000c00 tx=0x7fc244004750 comp rx=0 tx=0).stop 2023-11-28T11:22:07.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.035+0000 7fc2615ff700 1 -- 172.21.15.92:0/2608581177 shutdown_connections 2023-11-28T11:22:07.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.035+0000 7fc2615ff700 1 --2- 172.21.15.92:0/2608581177 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc25c0d0d60 0x7fc25c15b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:07.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.035+0000 7fc2615ff700 1 --2- 172.21.15.92:0/2608581177 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc25c0b8510 0x7fc25c0ca480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:07.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.035+0000 7fc2615ff700 1 --2- 172.21.15.92:0/2608581177 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc23c0682f0 0x7fc23c06a7a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:07.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.035+0000 7fc2615ff700 1 --2- 172.21.15.92:0/2608581177 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc25c0cd460 0x7fc25c0ca9c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:07.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.035+0000 7fc2615ff700 1 -- 172.21.15.92:0/2608581177 >> 172.21.15.92:0/2608581177 conn(0x7fc25c0719a0 msgr2=0x7fc25c0b3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:07.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.035+0000 7fc2615ff700 1 -- 172.21.15.92:0/2608581177 shutdown_connections 2023-11-28T11:22:07.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.035+0000 7fc2615ff700 1 -- 172.21.15.92:0/2608581177 wait complete. 2023-11-28T11:22:07.054 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:22:07.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.253+0000 7ff6c3fff700 1 Processor -- start 2023-11-28T11:22:07.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.253+0000 7ff6c3fff700 1 -- start start 2023-11-28T11:22:07.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.254+0000 7ff6c3fff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff6bc0cd460 0x7ff6bc06c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:07.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.254+0000 7ff6c3fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff6bc0d0d60 0x7ff6bc0d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:07.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.254+0000 7ff6c3fff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6bc0b88e0 con 0x7ff6bc0b8510 2023-11-28T11:22:07.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.254+0000 7ff6c3fff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6bc0cdbd0 con 0x7ff6bc0cd460 2023-11-28T11:22:07.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.254+0000 7ff6c3fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6bc0d02d0 con 0x7ff6bc0d0d60 2023-11-28T11:22:07.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.255+0000 7ff6c259c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff6bc0d0d60 0x7ff6bc0d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:07.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.255+0000 7ff6c159a700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff6bc0cd460 0x7ff6bc06c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:07.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.255+0000 7ff6c1d9b700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7ff6bc0b8510 0x7ff6bc06b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.92:43928/0 (socket says 172.21.15.92:43928) 2023-11-28T11:22:07.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.255+0000 7ff6c1d9b700 1 -- 172.21.15.92:0/1144468368 learned_addr learned my addr 172.21.15.92:0/1144468368 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:22:07.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.255+0000 7ff6c0d99700 1 -- 172.21.15.92:0/1144468368 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2759891178 0 0) 0x7ff6bc0b88e0 con 0x7ff6bc0b8510 2023-11-28T11:22:07.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.255+0000 7ff6c0d99700 1 -- 172.21.15.92:0/1144468368 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6a80035b0 con 0x7ff6bc0b8510 2023-11-28T11:22:07.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.255+0000 7ff6c259c700 1 -- 172.21.15.92:0/1144468368 >> v2:172.21.15.169:3300/0 conn(0x7ff6bc0cd460 msgr2=0x7ff6bc06c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:07.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.255+0000 7ff6c259c700 1 --2- 172.21.15.92:0/1144468368 >> v2:172.21.15.169:3300/0 conn(0x7ff6bc0cd460 0x7ff6bc06c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:07.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.255+0000 7ff6c259c700 1 -- 172.21.15.92:0/1144468368 >> v1:172.21.15.169:6789/0 conn(0x7ff6bc0b8510 legacy=0x7ff6bc06b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:07.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.255+0000 7ff6c259c700 1 -- 172.21.15.92:0/1144468368 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6bc0cd8f0 con 0x7ff6bc0d0d60 2023-11-28T11:22:07.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.255+0000 7ff6c159a700 1 --2- 172.21.15.92:0/1144468368 >> v2:172.21.15.169:3300/0 conn(0x7ff6bc0cd460 0x7ff6bc06c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:22:07.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.256+0000 7ff6c259c700 1 --2- 172.21.15.92:0/1144468368 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff6bc0d0d60 0x7ff6bc0d1200 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff6b4009d90 tx=0x7ff6b400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46d727a5137c33f5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:07.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.256+0000 7ff6c0d99700 1 -- 172.21.15.92:0/1144468368 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b400e070 con 0x7ff6bc0d0d60 2023-11-28T11:22:07.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.256+0000 7ff6c0d99700 1 -- 172.21.15.92:0/1144468368 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6b400bb20 con 0x7ff6bc0d0d60 2023-11-28T11:22:07.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.256+0000 7ff6c0d99700 1 -- 172.21.15.92:0/1144468368 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b4012430 con 0x7ff6bc0d0d60 2023-11-28T11:22:07.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.256+0000 7ff6c3fff700 1 -- 172.21.15.92:0/1144468368 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff6bc0d0d60 msgr2=0x7ff6bc0d1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:07.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.256+0000 7ff6c3fff700 1 --2- 172.21.15.92:0/1144468368 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff6bc0d0d60 0x7ff6bc0d1200 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff6b4009d90 tx=0x7ff6b400b330 comp rx=0 tx=0).stop 2023-11-28T11:22:07.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.257+0000 7ff6c3fff700 1 -- 172.21.15.92:0/1144468368 shutdown_connections 2023-11-28T11:22:07.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.257+0000 7ff6c3fff700 1 --2- 172.21.15.92:0/1144468368 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff6bc0d0d60 0x7ff6bc0d1200 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:07.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.257+0000 7ff6c3fff700 1 --2- 172.21.15.92:0/1144468368 >> v2:172.21.15.169:3300/0 conn(0x7ff6bc0cd460 0x7ff6bc06c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:07.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.257+0000 7ff6c3fff700 1 -- 172.21.15.92:0/1144468368 >> 172.21.15.92:0/1144468368 conn(0x7ff6bc0719a0 msgr2=0x7ff6bc071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:07.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.257+0000 7ff6c3fff700 1 -- 172.21.15.92:0/1144468368 shutdown_connections 2023-11-28T11:22:07.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.257+0000 7ff6c3fff700 1 -- 172.21.15.92:0/1144468368 wait complete. 2023-11-28T11:22:07.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.258+0000 7ff6c3fff700 1 Processor -- start 2023-11-28T11:22:07.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.258+0000 7ff6c3fff700 1 -- start start 2023-11-28T11:22:07.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.259+0000 7ff6c3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6bc0b8510 0x7ff6bc1e3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:07.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.259+0000 7ff6c3fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6bc0cd460 0x7ff6bc15a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:07.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.259+0000 7ff6c3fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff6bc0d0d60 0x7ff6bc149750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:07.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.259+0000 7ff6c3fff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6bc1e4150 con 0x7ff6bc0b8510 2023-11-28T11:22:07.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.259+0000 7ff6c3fff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6bc148f70 con 0x7ff6bc0cd460 2023-11-28T11:22:07.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.259+0000 7ff6c3fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6bc1e43d0 con 0x7ff6bc0d0d60 2023-11-28T11:22:07.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.260+0000 7ff6c1d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6bc0b8510 0x7ff6bc1e3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:07.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.260+0000 7ff6c1d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6bc0b8510 0x7ff6bc1e3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56592/0 (socket says 172.21.15.92:56592) 2023-11-28T11:22:07.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.260+0000 7ff6c1d9b700 1 -- 172.21.15.92:0/3458702857 learned_addr learned my addr 172.21.15.92:0/3458702857 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:07.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.260+0000 7ff6c159a700 1 --2- 172.21.15.92:0/3458702857 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6bc0cd460 0x7ff6bc15a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:07.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.260+0000 7ff6c259c700 1 --2- 172.21.15.92:0/3458702857 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff6bc0d0d60 0x7ff6bc149750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:07.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.260+0000 7ff6c1d9b700 1 -- 172.21.15.92:0/3458702857 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff6bc0d0d60 msgr2=0x7ff6bc149750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:07.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.260+0000 7ff6c1d9b700 1 --2- 172.21.15.92:0/3458702857 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff6bc0d0d60 0x7ff6bc149750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:07.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.260+0000 7ff6c1d9b700 1 -- 172.21.15.92:0/3458702857 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6bc0cd460 msgr2=0x7ff6bc15a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:07.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.260+0000 7ff6c1d9b700 1 --2- 172.21.15.92:0/3458702857 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6bc0cd460 0x7ff6bc15a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:07.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.260+0000 7ff6c1d9b700 1 -- 172.21.15.92:0/3458702857 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6ac0094a0 con 0x7ff6bc0b8510 2023-11-28T11:22:07.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.260+0000 7ff6c1d9b700 1 --2- 172.21.15.92:0/3458702857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6bc0b8510 0x7ff6bc1e3c10 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff6b800c700 tx=0x7ff6b800ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:07.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.261+0000 7ff6b27fc700 1 -- 172.21.15.92:0/3458702857 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b8005d70 con 0x7ff6bc0b8510 2023-11-28T11:22:07.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.261+0000 7ff6b27fc700 1 -- 172.21.15.92:0/3458702857 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6b80070d0 con 0x7ff6bc0b8510 2023-11-28T11:22:07.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.261+0000 7ff6c3fff700 1 -- 172.21.15.92:0/3458702857 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6b4009a70 con 0x7ff6bc0b8510 2023-11-28T11:22:07.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.261+0000 7ff6b27fc700 1 -- 172.21.15.92:0/3458702857 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b8007460 con 0x7ff6bc0b8510 2023-11-28T11:22:07.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.261+0000 7ff6c3fff700 1 -- 172.21.15.92:0/3458702857 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6bc0cfff0 con 0x7ff6bc0b8510 2023-11-28T11:22:07.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.262+0000 7ff6c3fff700 1 -- 172.21.15.92:0/3458702857 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6bc0cfd10 con 0x7ff6bc0b8510 2023-11-28T11:22:07.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.263+0000 7ff6b27fc700 1 -- 172.21.15.92:0/3458702857 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff6b8019030 con 0x7ff6bc0b8510 2023-11-28T11:22:07.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.264+0000 7ff6b27fc700 1 --2- 172.21.15.92:0/3458702857 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff6a0068320 0x7ff6a006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:07.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.264+0000 7ff6b27fc700 1 -- 172.21.15.92:0/3458702857 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7ff6b8085310 con 0x7ff6bc0b8510 2023-11-28T11:22:07.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.264+0000 7ff6c159a700 1 --2- 172.21.15.92:0/3458702857 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff6a0068320 0x7ff6a006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:07.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.264+0000 7ff6c159a700 1 --2- 172.21.15.92:0/3458702857 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff6a0068320 0x7ff6a006a7d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc146db0 tx=0x7ff6ac009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:07.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.268+0000 7ff6b27fc700 1 -- 172.21.15.92:0/3458702857 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff6b80517b0 con 0x7ff6bc0b8510 2023-11-28T11:22:07.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.457+0000 7ff6c3fff700 1 -- 172.21.15.92:0/3458702857 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff6bc067880 con 0x7ff6bc0b8510 2023-11-28T11:22:07.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.457+0000 7ff6b27fc700 1 -- 172.21.15.92:0/3458702857 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff6b8017070 con 0x7ff6bc0b8510 2023-11-28T11:22:07.458 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-11-28T11:22:07.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.460+0000 7ff6c3fff700 1 -- 172.21.15.92:0/3458702857 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff6a0068320 msgr2=0x7ff6a006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:07.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.460+0000 7ff6c3fff700 1 --2- 172.21.15.92:0/3458702857 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff6a0068320 0x7ff6a006a7d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc146db0 tx=0x7ff6ac009210 comp rx=0 tx=0).stop 2023-11-28T11:22:07.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.460+0000 7ff6c3fff700 1 -- 172.21.15.92:0/3458702857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6bc0b8510 msgr2=0x7ff6bc1e3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:07.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.460+0000 7ff6c3fff700 1 --2- 172.21.15.92:0/3458702857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6bc0b8510 0x7ff6bc1e3c10 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff6b800c700 tx=0x7ff6b800ca10 comp rx=0 tx=0).stop 2023-11-28T11:22:07.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.460+0000 7ff6c3fff700 1 -- 172.21.15.92:0/3458702857 shutdown_connections 2023-11-28T11:22:07.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.460+0000 7ff6c3fff700 1 --2- 172.21.15.92:0/3458702857 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff6bc0d0d60 0x7ff6bc149750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:07.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.460+0000 7ff6c3fff700 1 --2- 172.21.15.92:0/3458702857 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff6a0068320 0x7ff6a006a7d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:07.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.460+0000 7ff6c3fff700 1 --2- 172.21.15.92:0/3458702857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6bc0b8510 0x7ff6bc1e3c10 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:07.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.460+0000 7ff6c3fff700 1 --2- 172.21.15.92:0/3458702857 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6bc0cd460 0x7ff6bc15a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:07.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.460+0000 7ff6c3fff700 1 -- 172.21.15.92:0/3458702857 >> 172.21.15.92:0/3458702857 conn(0x7ff6bc0719a0 msgr2=0x7ff6bc0d1160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:07.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.460+0000 7ff6c3fff700 1 -- 172.21.15.92:0/3458702857 shutdown_connections 2023-11-28T11:22:07.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.460+0000 7ff6c3fff700 1 -- 172.21.15.92:0/3458702857 wait complete. 2023-11-28T11:22:07.479 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:22:07.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.678+0000 7fde559ff700 1 Processor -- start 2023-11-28T11:22:07.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.678+0000 7fde559ff700 1 -- start start 2023-11-28T11:22:07.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.679+0000 7fde559ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fde500cd460 0x7fde5006c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:07.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.679+0000 7fde559ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fde500d0d60 0x7fde500d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:07.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.679+0000 7fde559ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde500b88e0 con 0x7fde500b8510 2023-11-28T11:22:07.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.679+0000 7fde559ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde500cdbd0 con 0x7fde500cd460 2023-11-28T11:22:07.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.679+0000 7fde559ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde500d02d0 con 0x7fde500d0d60 2023-11-28T11:22:07.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.679+0000 7fde4effd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fde500b8510 0x7fde5006b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55540/0 (socket says 172.21.15.92:55540) 2023-11-28T11:22:07.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.679+0000 7fde4effd700 1 -- 172.21.15.92:0/2715804210 learned_addr learned my addr 172.21.15.92:0/2715804210 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:22:07.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.679+0000 7fde4f7fe700 1 --2- 172.21.15.92:0/2715804210 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fde500d0d60 0x7fde500d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:07.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.680+0000 7fde47fff700 1 --2- 172.21.15.92:0/2715804210 >> v2:172.21.15.169:3300/0 conn(0x7fde500cd460 0x7fde5006c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:07.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.680+0000 7fde4e7fc700 1 -- 172.21.15.92:0/2715804210 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 829187620 0 0) 0x7fde500b88e0 con 0x7fde500b8510 2023-11-28T11:22:07.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.680+0000 7fde4e7fc700 1 -- 172.21.15.92:0/2715804210 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde3c0035b0 con 0x7fde500b8510 2023-11-28T11:22:07.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.680+0000 7fde4e7fc700 1 -- 172.21.15.92:0/2715804210 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2990211477 0 0) 0x7fde3c0035b0 con 0x7fde500b8510 2023-11-28T11:22:07.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.680+0000 7fde4e7fc700 1 -- 172.21.15.92:0/2715804210 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fde500d0d60 msgr2=0x7fde500d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:07.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.680+0000 7fde4e7fc700 1 --2- 172.21.15.92:0/2715804210 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fde500d0d60 0x7fde500d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:07.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.680+0000 7fde4e7fc700 1 -- 172.21.15.92:0/2715804210 >> v2:172.21.15.169:3300/0 conn(0x7fde500cd460 msgr2=0x7fde5006c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:07.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.680+0000 7fde4e7fc700 1 --2- 172.21.15.92:0/2715804210 >> v2:172.21.15.169:3300/0 conn(0x7fde500cd460 0x7fde5006c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:07.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.680+0000 7fde4e7fc700 1 -- 172.21.15.92:0/2715804210 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde500cd8f0 con 0x7fde500b8510 2023-11-28T11:22:07.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.680+0000 7fde4e7fc700 1 -- 172.21.15.92:0/2715804210 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532161587 0 0) 0x7fde40001670 con 0x7fde500b8510 2023-11-28T11:22:07.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.680+0000 7fde47fff700 1 --2- 172.21.15.92:0/2715804210 >> v2:172.21.15.169:3300/0 conn(0x7fde500cd460 0x7fde5006c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:22:07.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.680+0000 7fde4e7fc700 1 -- 172.21.15.92:0/2715804210 >> v1:172.21.15.92:6789/0 conn(0x7fde500b8510 legacy=0x7fde5006b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:07.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.680+0000 7fde4e7fc700 1 --2- 172.21.15.92:0/2715804210 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde3c003a30 0x7fde3c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:07.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.680+0000 7fde4e7fc700 1 --2- 172.21.15.92:0/2715804210 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde3c004850 0x7fde3c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:07.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.680+0000 7fde4e7fc700 1 --2- 172.21.15.92:0/2715804210 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fde500d0d60 0x7fde3c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:07.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.680+0000 7fde4e7fc700 1 -- 172.21.15.92:0/2715804210 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde500cdbd0 con 0x7fde3c004850 2023-11-28T11:22:07.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.680+0000 7fde4e7fc700 1 -- 172.21.15.92:0/2715804210 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde500d02d0 con 0x7fde3c003a30 2023-11-28T11:22:07.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.680+0000 7fde4e7fc700 1 -- 172.21.15.92:0/2715804210 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde3c00ba10 con 0x7fde500d0d60 2023-11-28T11:22:07.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.680+0000 7fde47fff700 1 --2- 172.21.15.92:0/2715804210 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde3c004850 0x7fde3c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:07.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.681+0000 7fde4effd700 1 --2- 172.21.15.92:0/2715804210 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde3c003a30 0x7fde3c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:07.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.681+0000 7fde4f7fe700 1 --2- 172.21.15.92:0/2715804210 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fde500d0d60 0x7fde3c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:07.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.681+0000 7fde47fff700 1 -- 172.21.15.92:0/2715804210 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fde500d0d60 msgr2=0x7fde3c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:07.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.681+0000 7fde47fff700 1 --2- 172.21.15.92:0/2715804210 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fde500d0d60 0x7fde3c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:07.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.681+0000 7fde47fff700 1 -- 172.21.15.92:0/2715804210 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde3c003a30 msgr2=0x7fde3c003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:07.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.681+0000 7fde47fff700 1 --2- 172.21.15.92:0/2715804210 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde3c003a30 0x7fde3c003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:07.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.681+0000 7fde47fff700 1 -- 172.21.15.92:0/2715804210 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde3c0035b0 con 0x7fde3c004850 2023-11-28T11:22:07.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.681+0000 7fde4effd700 1 --2- 172.21.15.92:0/2715804210 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde3c003a30 0x7fde3c003e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:22:07.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.681+0000 7fde47fff700 1 --2- 172.21.15.92:0/2715804210 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde3c004850 0x7fde3c004cc0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fde500b1de0 tx=0x7fde3c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3503ccffc500fbae server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:07.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.681+0000 7fde4e7fc700 1 -- 172.21.15.92:0/2715804210 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3800e780 con 0x7fde3c004850 2023-11-28T11:22:07.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.681+0000 7fde4e7fc700 1 -- 172.21.15.92:0/2715804210 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde3800e8e0 con 0x7fde3c004850 2023-11-28T11:22:07.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.681+0000 7fde4e7fc700 1 -- 172.21.15.92:0/2715804210 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3800ec90 con 0x7fde3c004850 2023-11-28T11:22:07.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.682+0000 7fde559ff700 1 -- 172.21.15.92:0/2715804210 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde3c004850 msgr2=0x7fde3c004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:07.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.682+0000 7fde559ff700 1 --2- 172.21.15.92:0/2715804210 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde3c004850 0x7fde3c004cc0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fde500b1de0 tx=0x7fde3c005880 comp rx=0 tx=0).stop 2023-11-28T11:22:07.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.682+0000 7fde4f7fe700 1 -- 172.21.15.92:0/2715804210 reap_dead start 2023-11-28T11:22:07.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.682+0000 7fde559ff700 1 -- 172.21.15.92:0/2715804210 shutdown_connections 2023-11-28T11:22:07.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.682+0000 7fde559ff700 1 -- 172.21.15.92:0/2715804210 >> 172.21.15.92:0/2715804210 conn(0x7fde500719a0 msgr2=0x7fde50071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:07.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.682+0000 7fde559ff700 1 -- 172.21.15.92:0/2715804210 shutdown_connections 2023-11-28T11:22:07.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.682+0000 7fde559ff700 1 -- 172.21.15.92:0/2715804210 wait complete. 2023-11-28T11:22:07.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.684+0000 7fde559ff700 1 Processor -- start 2023-11-28T11:22:07.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.685+0000 7fde559ff700 1 -- start start 2023-11-28T11:22:07.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.686+0000 7fde559ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde3c004850 0x7fde501563e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:07.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.686+0000 7fde559ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde501f61c0 0x7fde501536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:07.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.686+0000 7fde559ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fde500b6d20 0x7fde500b71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:07.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.686+0000 7fde559ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde50156920 con 0x7fde3c004850 2023-11-28T11:22:07.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.686+0000 7fde559ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde500b7700 con 0x7fde501f61c0 2023-11-28T11:22:07.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.686+0000 7fde559ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde50153c10 con 0x7fde500b6d20 2023-11-28T11:22:07.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.686+0000 7fde4effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde3c004850 0x7fde501563e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:07.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.686+0000 7fde4effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde3c004850 0x7fde501563e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56614/0 (socket says 172.21.15.92:56614) 2023-11-28T11:22:07.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.686+0000 7fde4effd700 1 -- 172.21.15.92:0/4124954946 learned_addr learned my addr 172.21.15.92:0/4124954946 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:07.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.686+0000 7fde47fff700 1 --2- 172.21.15.92:0/4124954946 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde501f61c0 0x7fde501536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:07.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.686+0000 7fde4f7fe700 1 --2- 172.21.15.92:0/4124954946 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fde500b6d20 0x7fde500b71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:07.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.686+0000 7fde4effd700 1 -- 172.21.15.92:0/4124954946 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fde500b6d20 msgr2=0x7fde500b71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:07.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.686+0000 7fde4effd700 1 --2- 172.21.15.92:0/4124954946 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fde500b6d20 0x7fde500b71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:07.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.686+0000 7fde4effd700 1 -- 172.21.15.92:0/4124954946 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde501f61c0 msgr2=0x7fde501536d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:07.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.686+0000 7fde4effd700 1 --2- 172.21.15.92:0/4124954946 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde501f61c0 0x7fde501536d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:07.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.686+0000 7fde4effd700 1 -- 172.21.15.92:0/4124954946 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde500cfff0 con 0x7fde3c004850 2023-11-28T11:22:07.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.687+0000 7fde4effd700 1 --2- 172.21.15.92:0/4124954946 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde3c004850 0x7fde501563e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fde4000eb50 tx=0x7fde4000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:07.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.687+0000 7fde4cff9700 1 -- 172.21.15.92:0/4124954946 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4000c700 con 0x7fde3c004850 2023-11-28T11:22:07.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.687+0000 7fde4cff9700 1 -- 172.21.15.92:0/4124954946 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde4000c860 con 0x7fde3c004850 2023-11-28T11:22:07.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.687+0000 7fde559ff700 1 -- 172.21.15.92:0/4124954946 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde500cfd10 con 0x7fde3c004850 2023-11-28T11:22:07.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.687+0000 7fde4cff9700 1 -- 172.21.15.92:0/4124954946 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4000c9d0 con 0x7fde3c004850 2023-11-28T11:22:07.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.687+0000 7fde559ff700 1 -- 172.21.15.92:0/4124954946 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde500d29c0 con 0x7fde3c004850 2023-11-28T11:22:07.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.688+0000 7fde559ff700 1 -- 172.21.15.92:0/4124954946 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde501584b0 con 0x7fde3c004850 2023-11-28T11:22:07.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.689+0000 7fde4cff9700 1 -- 172.21.15.92:0/4124954946 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fde4001b030 con 0x7fde3c004850 2023-11-28T11:22:07.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.690+0000 7fde4cff9700 1 --2- 172.21.15.92:0/4124954946 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fde300682f0 0x7fde3006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:07.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.690+0000 7fde4cff9700 1 -- 172.21.15.92:0/4124954946 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fde40085680 con 0x7fde3c004850 2023-11-28T11:22:07.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.690+0000 7fde47fff700 1 --2- 172.21.15.92:0/4124954946 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fde300682f0 0x7fde3006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:07.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.691+0000 7fde47fff700 1 --2- 172.21.15.92:0/4124954946 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fde300682f0 0x7fde3006a7a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fde380090b0 tx=0x7fde38008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:07.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.694+0000 7fde4cff9700 1 -- 172.21.15.92:0/4124954946 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fde4004d5e0 con 0x7fde3c004850 2023-11-28T11:22:07.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.891+0000 7fde559ff700 1 -- 172.21.15.92:0/4124954946 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fde50067880 con 0x7fde3c004850 2023-11-28T11:22:07.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.891+0000 7fde4cff9700 1 -- 172.21.15.92:0/4124954946 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7fde40019020 con 0x7fde3c004850 2023-11-28T11:22:07.892 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-11-28T11:22:07.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.893+0000 7fde559ff700 1 -- 172.21.15.92:0/4124954946 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fde300682f0 msgr2=0x7fde3006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:07.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.893+0000 7fde559ff700 1 --2- 172.21.15.92:0/4124954946 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fde300682f0 0x7fde3006a7a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fde380090b0 tx=0x7fde38008030 comp rx=0 tx=0).stop 2023-11-28T11:22:07.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.893+0000 7fde559ff700 1 -- 172.21.15.92:0/4124954946 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde3c004850 msgr2=0x7fde501563e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:07.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.893+0000 7fde559ff700 1 --2- 172.21.15.92:0/4124954946 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde3c004850 0x7fde501563e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fde4000eb50 tx=0x7fde4000ef10 comp rx=0 tx=0).stop 2023-11-28T11:22:07.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.893+0000 7fde559ff700 1 -- 172.21.15.92:0/4124954946 shutdown_connections 2023-11-28T11:22:07.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.893+0000 7fde559ff700 1 --2- 172.21.15.92:0/4124954946 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fde500b6d20 0x7fde500b71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:07.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.893+0000 7fde559ff700 1 --2- 172.21.15.92:0/4124954946 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fde300682f0 0x7fde3006a7a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:07.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.893+0000 7fde559ff700 1 --2- 172.21.15.92:0/4124954946 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde3c004850 0x7fde501563e0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:07.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.893+0000 7fde559ff700 1 --2- 172.21.15.92:0/4124954946 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fde501f61c0 0x7fde501536d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:07.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.893+0000 7fde559ff700 1 -- 172.21.15.92:0/4124954946 >> 172.21.15.92:0/4124954946 conn(0x7fde500719a0 msgr2=0x7fde500cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:07.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.894+0000 7fde559ff700 1 -- 172.21.15.92:0/4124954946 shutdown_connections 2023-11-28T11:22:07.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:07.894+0000 7fde559ff700 1 -- 172.21.15.92:0/4124954946 wait complete. 2023-11-28T11:22:07.913 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:22:08.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.112+0000 7f918cdff700 1 Processor -- start 2023-11-28T11:22:08.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.112+0000 7f918cdff700 1 -- start start 2023-11-28T11:22:08.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.113+0000 7f918cdff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f91880b8510 0x7f918806b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:08.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.113+0000 7f918cdff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f91880cd460 0x7f918806beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:08.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.113+0000 7f918cdff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f918806c3f0 0x7f918815cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:08.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.113+0000 7f918cdff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91880cdbd0 con 0x7f91880b8510 2023-11-28T11:22:08.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.113+0000 7f918cdff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91880d02d0 con 0x7f91880cd460 2023-11-28T11:22:08.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.113+0000 7f918cdff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91880b88e0 con 0x7f918806c3f0 2023-11-28T11:22:08.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.113+0000 7f918659c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f91880b8510 0x7f918806b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:08.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.113+0000 7f918659c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f91880b8510 0x7f918806b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56626/0 (socket says 172.21.15.92:56626) 2023-11-28T11:22:08.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.113+0000 7f918659c700 1 -- 172.21.15.92:0/2253388927 learned_addr learned my addr 172.21.15.92:0/2253388927 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:08.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.114+0000 7f9185d9b700 1 --2- 172.21.15.92:0/2253388927 >> v2:172.21.15.169:3300/0 conn(0x7f91880cd460 0x7f918806beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:08.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.114+0000 7f9186d9d700 1 --2- 172.21.15.92:0/2253388927 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f918806c3f0 0x7f918815cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:08.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.114+0000 7f918659c700 1 -- 172.21.15.92:0/2253388927 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f918806c3f0 msgr2=0x7f918815cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:08.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.114+0000 7f918659c700 1 --2- 172.21.15.92:0/2253388927 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f918806c3f0 0x7f918815cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:08.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.114+0000 7f918659c700 1 -- 172.21.15.92:0/2253388927 >> v2:172.21.15.169:3300/0 conn(0x7f91880cd460 msgr2=0x7f918806beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:08.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.114+0000 7f918659c700 1 --2- 172.21.15.92:0/2253388927 >> v2:172.21.15.169:3300/0 conn(0x7f91880cd460 0x7f918806beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:08.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.114+0000 7f918659c700 1 -- 172.21.15.92:0/2253388927 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91880cd8f0 con 0x7f91880b8510 2023-11-28T11:22:08.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.114+0000 7f918659c700 1 --2- 172.21.15.92:0/2253388927 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91880b8510 0x7f918806b970 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9178009920 tx=0x7f9178009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aab4fb19f4d19fcf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:08.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.114+0000 7f918559a700 1 -- 172.21.15.92:0/2253388927 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917800b9e0 con 0x7f91880b8510 2023-11-28T11:22:08.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.114+0000 7f918559a700 1 -- 172.21.15.92:0/2253388927 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f917800bb40 con 0x7f91880b8510 2023-11-28T11:22:08.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.114+0000 7f918559a700 1 -- 172.21.15.92:0/2253388927 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9178009e70 con 0x7f91880b8510 2023-11-28T11:22:08.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.115+0000 7f918cdff700 1 -- 172.21.15.92:0/2253388927 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91880b8510 msgr2=0x7f918806b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:08.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.115+0000 7f918cdff700 1 --2- 172.21.15.92:0/2253388927 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91880b8510 0x7f918806b970 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9178009920 tx=0x7f9178009c30 comp rx=0 tx=0).stop 2023-11-28T11:22:08.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.115+0000 7f918cdff700 1 -- 172.21.15.92:0/2253388927 shutdown_connections 2023-11-28T11:22:08.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.115+0000 7f918cdff700 1 --2- 172.21.15.92:0/2253388927 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f918806c3f0 0x7f918815cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:08.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.115+0000 7f918cdff700 1 --2- 172.21.15.92:0/2253388927 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91880b8510 0x7f918806b970 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:08.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.115+0000 7f918cdff700 1 --2- 172.21.15.92:0/2253388927 >> v2:172.21.15.169:3300/0 conn(0x7f91880cd460 0x7f918806beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:08.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.115+0000 7f918cdff700 1 -- 172.21.15.92:0/2253388927 >> 172.21.15.92:0/2253388927 conn(0x7f91880719a0 msgr2=0x7f9188071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:08.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.115+0000 7f918cdff700 1 -- 172.21.15.92:0/2253388927 shutdown_connections 2023-11-28T11:22:08.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.115+0000 7f918cdff700 1 -- 172.21.15.92:0/2253388927 wait complete. 2023-11-28T11:22:08.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.117+0000 7f918cdff700 1 Processor -- start 2023-11-28T11:22:08.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.117+0000 7f918cdff700 1 -- start start 2023-11-28T11:22:08.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.118+0000 7f918cdff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f918806c3f0 0x7f91880cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:08.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.118+0000 7f918cdff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91880b8510 0x7f9188155a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:08.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.118+0000 7f918cdff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91880cd460 0x7f91880ca780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:08.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.118+0000 7f918cdff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91880ccbd0 con 0x7f918806c3f0 2023-11-28T11:22:08.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.118+0000 7f918cdff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91880cceb0 con 0x7f91880cd460 2023-11-28T11:22:08.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.118+0000 7f918cdff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91880c9f80 con 0x7f91880b8510 2023-11-28T11:22:08.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.118+0000 7f918659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f918806c3f0 0x7f91880cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:08.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.118+0000 7f918659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f918806c3f0 0x7f91880cbf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56632/0 (socket says 172.21.15.92:56632) 2023-11-28T11:22:08.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.118+0000 7f918659c700 1 -- 172.21.15.92:0/831994836 learned_addr learned my addr 172.21.15.92:0/831994836 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:08.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.118+0000 7f918659c700 1 -- 172.21.15.92:0/831994836 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91880b8510 msgr2=0x7f9188155a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:08.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.118+0000 7f918659c700 1 --2- 172.21.15.92:0/831994836 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91880b8510 0x7f9188155a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:08.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.118+0000 7f918659c700 1 -- 172.21.15.92:0/831994836 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91880cd460 msgr2=0x7f91880ca780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:08.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.118+0000 7f918659c700 1 --2- 172.21.15.92:0/831994836 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91880cd460 0x7f91880ca780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:08.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.118+0000 7f918659c700 1 -- 172.21.15.92:0/831994836 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9178009550 con 0x7f918806c3f0 2023-11-28T11:22:08.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.119+0000 7f918659c700 1 --2- 172.21.15.92:0/831994836 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f918806c3f0 0x7f91880cbf40 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9178009c70 tx=0x7f9178004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:08.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.119+0000 7f9176ffd700 1 -- 172.21.15.92:0/831994836 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9178004400 con 0x7f918806c3f0 2023-11-28T11:22:08.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.119+0000 7f9176ffd700 1 -- 172.21.15.92:0/831994836 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f917801f070 con 0x7f918806c3f0 2023-11-28T11:22:08.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.119+0000 7f9176ffd700 1 -- 172.21.15.92:0/831994836 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91780038c0 con 0x7f918806c3f0 2023-11-28T11:22:08.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.119+0000 7f918cdff700 1 -- 172.21.15.92:0/831994836 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91880cfff0 con 0x7f918806c3f0 2023-11-28T11:22:08.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.119+0000 7f918cdff700 1 -- 172.21.15.92:0/831994836 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91880cfd10 con 0x7f918806c3f0 2023-11-28T11:22:08.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.120+0000 7f918cdff700 1 -- 172.21.15.92:0/831994836 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91880ca4a0 con 0x7f918806c3f0 2023-11-28T11:22:08.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.121+0000 7f9176ffd700 1 -- 172.21.15.92:0/831994836 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f917802a020 con 0x7f918806c3f0 2023-11-28T11:22:08.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.122+0000 7f9176ffd700 1 --2- 172.21.15.92:0/831994836 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f91680682f0 0x7f916806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:08.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.122+0000 7f9176ffd700 1 -- 172.21.15.92:0/831994836 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f9178087690 con 0x7f918806c3f0 2023-11-28T11:22:08.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.122+0000 7f9185d9b700 1 --2- 172.21.15.92:0/831994836 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f91680682f0 0x7f916806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:08.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.122+0000 7f9185d9b700 1 --2- 172.21.15.92:0/831994836 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f91680682f0 0x7f916806a7a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f91881f5640 tx=0x7f917c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:08.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.126+0000 7f9176ffd700 1 -- 172.21.15.92:0/831994836 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f91880ca4a0 con 0x7f918806c3f0 2023-11-28T11:22:08.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.322+0000 7f918cdff700 1 -- 172.21.15.92:0/831994836 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f91880ca4a0 con 0x7f918806c3f0 2023-11-28T11:22:08.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.322+0000 7f9176ffd700 1 -- 172.21.15.92:0/831994836 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7f91880ca4a0 con 0x7f918806c3f0 2023-11-28T11:22:08.324 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-11-28T11:22:08.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.325+0000 7f918cdff700 1 -- 172.21.15.92:0/831994836 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f91680682f0 msgr2=0x7f916806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:08.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.325+0000 7f918cdff700 1 --2- 172.21.15.92:0/831994836 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f91680682f0 0x7f916806a7a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f91881f5640 tx=0x7f917c008040 comp rx=0 tx=0).stop 2023-11-28T11:22:08.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.325+0000 7f918cdff700 1 -- 172.21.15.92:0/831994836 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f918806c3f0 msgr2=0x7f91880cbf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:08.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.325+0000 7f918cdff700 1 --2- 172.21.15.92:0/831994836 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f918806c3f0 0x7f91880cbf40 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9178009c70 tx=0x7f9178004750 comp rx=0 tx=0).stop 2023-11-28T11:22:08.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.325+0000 7f918cdff700 1 -- 172.21.15.92:0/831994836 shutdown_connections 2023-11-28T11:22:08.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.325+0000 7f918cdff700 1 --2- 172.21.15.92:0/831994836 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91880cd460 0x7f91880ca780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:08.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.325+0000 7f918cdff700 1 --2- 172.21.15.92:0/831994836 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f91680682f0 0x7f916806a7a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:08.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.325+0000 7f918cdff700 1 --2- 172.21.15.92:0/831994836 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f918806c3f0 0x7f91880cbf40 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:08.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.325+0000 7f918cdff700 1 --2- 172.21.15.92:0/831994836 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91880b8510 0x7f9188155a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:08.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.325+0000 7f918cdff700 1 -- 172.21.15.92:0/831994836 >> 172.21.15.92:0/831994836 conn(0x7f91880719a0 msgr2=0x7f91880b7a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:08.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.325+0000 7f918cdff700 1 -- 172.21.15.92:0/831994836 shutdown_connections 2023-11-28T11:22:08.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.325+0000 7f918cdff700 1 -- 172.21.15.92:0/831994836 wait complete. 2023-11-28T11:22:08.344 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T11:22:08.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.543+0000 7f44b47ff700 1 Processor -- start 2023-11-28T11:22:08.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.544+0000 7f44b47ff700 1 -- start start 2023-11-28T11:22:08.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.545+0000 7f44b47ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f44ac0ca600 0x7f44ac164220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:08.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.545+0000 7f44b47ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44ac159b10 con 0x7f44ac142e80 2023-11-28T11:22:08.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.545+0000 7f44b47ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44ac143250 con 0x7f44ac06a980 2023-11-28T11:22:08.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.545+0000 7f44b47ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44ac0c9380 con 0x7f44ac0ca600 2023-11-28T11:22:08.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.545+0000 7f44b259b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f44ac142e80 0x7f44ac06a270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55556/0 (socket says 172.21.15.92:55556) 2023-11-28T11:22:08.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.545+0000 7f44b259b700 1 -- 172.21.15.92:0/1158967754 learned_addr learned my addr 172.21.15.92:0/1158967754 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:22:08.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.545+0000 7f44b2d9c700 1 --2- 172.21.15.92:0/1158967754 >> v2:172.21.15.169:3300/0 conn(0x7f44ac0ca600 0x7f44ac164220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:08.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.545+0000 7f44b1599700 1 -- 172.21.15.92:0/1158967754 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1167366319 0 0) 0x7f44ac159b10 con 0x7f44ac142e80 2023-11-28T11:22:08.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.546+0000 7f44b1599700 1 -- 172.21.15.92:0/1158967754 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44980035b0 con 0x7f44ac142e80 2023-11-28T11:22:08.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.546+0000 7f44b2d9c700 1 -- 172.21.15.92:0/1158967754 >> v1:172.21.15.169:6789/0 conn(0x7f44ac06a980 legacy=0x7f44ac0c9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:08.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.546+0000 7f44b2d9c700 1 -- 172.21.15.92:0/1158967754 >> v1:172.21.15.92:6789/0 conn(0x7f44ac142e80 legacy=0x7f44ac06a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:08.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.546+0000 7f44b2d9c700 1 -- 172.21.15.92:0/1158967754 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44ac0cab00 con 0x7f44ac0ca600 2023-11-28T11:22:08.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.546+0000 7f44b2d9c700 1 --2- 172.21.15.92:0/1158967754 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44ac0ca600 0x7f44ac164220 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f44a4009d90 tx=0x7f44a400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=187bce5d6e63da4b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:08.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.546+0000 7f44b1599700 1 -- 172.21.15.92:0/1158967754 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a400e070 con 0x7f44ac0ca600 2023-11-28T11:22:08.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.547+0000 7f44b1599700 1 -- 172.21.15.92:0/1158967754 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44a4003a20 con 0x7f44ac0ca600 2023-11-28T11:22:08.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.547+0000 7f44b1599700 1 -- 172.21.15.92:0/1158967754 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a8001670 con 0x7f44ac0ca600 2023-11-28T11:22:08.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.547+0000 7f44b47ff700 1 -- 172.21.15.92:0/1158967754 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44ac0ca600 msgr2=0x7f44ac164220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:08.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.547+0000 7f44b47ff700 1 --2- 172.21.15.92:0/1158967754 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44ac0ca600 0x7f44ac164220 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f44a4009d90 tx=0x7f44a400b330 comp rx=0 tx=0).stop 2023-11-28T11:22:08.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.547+0000 7f44b47ff700 1 -- 172.21.15.92:0/1158967754 shutdown_connections 2023-11-28T11:22:08.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.547+0000 7f44b47ff700 1 --2- 172.21.15.92:0/1158967754 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44ac0ca600 0x7f44ac164220 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:08.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.547+0000 7f44b47ff700 1 -- 172.21.15.92:0/1158967754 >> 172.21.15.92:0/1158967754 conn(0x7f44ac0cb490 msgr2=0x7f44ac0cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:08.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.547+0000 7f44b47ff700 1 -- 172.21.15.92:0/1158967754 shutdown_connections 2023-11-28T11:22:08.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.547+0000 7f44b47ff700 1 -- 172.21.15.92:0/1158967754 wait complete. 2023-11-28T11:22:08.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.549+0000 7f44b47ff700 1 Processor -- start 2023-11-28T11:22:08.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.549+0000 7f44b47ff700 1 -- start start 2023-11-28T11:22:08.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.550+0000 7f44b47ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44ac06a980 0x7f44ac0c4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:08.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.550+0000 7f44b47ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44ac0ca600 0x7f44ac0aca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:08.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.550+0000 7f44b47ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44ac142e80 0x7f44ac15beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:08.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.550+0000 7f44b47ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44ac154550 con 0x7f44ac0ca600 2023-11-28T11:22:08.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.550+0000 7f44b47ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44ac0b28d0 con 0x7f44ac06a980 2023-11-28T11:22:08.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.550+0000 7f44b47ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44ac0b2bb0 con 0x7f44ac142e80 2023-11-28T11:22:08.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.550+0000 7f44b1d9a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44ac0ca600 0x7f44ac0aca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:08.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.550+0000 7f44b1d9a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44ac0ca600 0x7f44ac0aca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56648/0 (socket says 172.21.15.92:56648) 2023-11-28T11:22:08.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.550+0000 7f44b1d9a700 1 -- 172.21.15.92:0/3906263904 learned_addr learned my addr 172.21.15.92:0/3906263904 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:08.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.551+0000 7f44b259b700 1 --2- 172.21.15.92:0/3906263904 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44ac06a980 0x7f44ac0c4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:08.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.551+0000 7f44b2d9c700 1 --2- 172.21.15.92:0/3906263904 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44ac142e80 0x7f44ac15beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:08.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.551+0000 7f44b1d9a700 1 -- 172.21.15.92:0/3906263904 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44ac142e80 msgr2=0x7f44ac15beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:08.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.551+0000 7f44b1d9a700 1 --2- 172.21.15.92:0/3906263904 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44ac142e80 0x7f44ac15beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:08.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.551+0000 7f44b1d9a700 1 -- 172.21.15.92:0/3906263904 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44ac06a980 msgr2=0x7f44ac0c4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:08.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.551+0000 7f44b1d9a700 1 --2- 172.21.15.92:0/3906263904 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44ac06a980 0x7f44ac0c4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:08.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.551+0000 7f44b1d9a700 1 -- 172.21.15.92:0/3906263904 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a4009a70 con 0x7f44ac0ca600 2023-11-28T11:22:08.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.551+0000 7f44b1d9a700 1 --2- 172.21.15.92:0/3906263904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44ac0ca600 0x7f44ac0aca00 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f449c00c4e0 tx=0x7f449c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:08.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.551+0000 7f44a2ffd700 1 -- 172.21.15.92:0/3906263904 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449c0052d0 con 0x7f44ac0ca600 2023-11-28T11:22:08.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.551+0000 7f44a2ffd700 1 -- 172.21.15.92:0/3906263904 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f449c005430 con 0x7f44ac0ca600 2023-11-28T11:22:08.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.551+0000 7f44a2ffd700 1 -- 172.21.15.92:0/3906263904 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449c0038c0 con 0x7f44ac0ca600 2023-11-28T11:22:08.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.551+0000 7f44b47ff700 1 -- 172.21.15.92:0/3906263904 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44ac0cc8d0 con 0x7f44ac0ca600 2023-11-28T11:22:08.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.551+0000 7f44b47ff700 1 -- 172.21.15.92:0/3906263904 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44ac0cc5f0 con 0x7f44ac0ca600 2023-11-28T11:22:08.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.552+0000 7f44b47ff700 1 -- 172.21.15.92:0/3906263904 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44ac0ccbb0 con 0x7f44ac0ca600 2023-11-28T11:22:08.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.553+0000 7f44a2ffd700 1 -- 172.21.15.92:0/3906263904 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f449c019030 con 0x7f44ac0ca600 2023-11-28T11:22:08.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.554+0000 7f44a2ffd700 1 --2- 172.21.15.92:0/3906263904 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f44900682f0 0x7f449006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:08.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.554+0000 7f44a2ffd700 1 -- 172.21.15.92:0/3906263904 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f449c085410 con 0x7f44ac0ca600 2023-11-28T11:22:08.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.554+0000 7f44b259b700 1 --2- 172.21.15.92:0/3906263904 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f44900682f0 0x7f449006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:08.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.555+0000 7f44b259b700 1 --2- 172.21.15.92:0/3906263904 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f44900682f0 0x7f449006a7a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f44a80017a0 tx=0x7f44a8001630 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:08.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.558+0000 7f44a2ffd700 1 -- 172.21.15.92:0/3906263904 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f449c051850 con 0x7f44ac0ca600 2023-11-28T11:22:08.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.749+0000 7f44b47ff700 1 -- 172.21.15.92:0/3906263904 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f44ac067880 con 0x7f44ac0ca600 2023-11-28T11:22:08.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.749+0000 7f44a2ffd700 1 -- 172.21.15.92:0/3906263904 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7f449c054e70 con 0x7f44ac0ca600 2023-11-28T11:22:08.750 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-11-28T11:22:08.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.751+0000 7f44b47ff700 1 -- 172.21.15.92:0/3906263904 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f44900682f0 msgr2=0x7f449006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:08.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.751+0000 7f44b47ff700 1 --2- 172.21.15.92:0/3906263904 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f44900682f0 0x7f449006a7a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f44a80017a0 tx=0x7f44a8001630 comp rx=0 tx=0).stop 2023-11-28T11:22:08.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.751+0000 7f44b47ff700 1 -- 172.21.15.92:0/3906263904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44ac0ca600 msgr2=0x7f44ac0aca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:08.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.751+0000 7f44b47ff700 1 --2- 172.21.15.92:0/3906263904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44ac0ca600 0x7f44ac0aca00 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f449c00c4e0 tx=0x7f449c00c8a0 comp rx=0 tx=0).stop 2023-11-28T11:22:08.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.751+0000 7f44b47ff700 1 -- 172.21.15.92:0/3906263904 shutdown_connections 2023-11-28T11:22:08.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.751+0000 7f44b47ff700 1 --2- 172.21.15.92:0/3906263904 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44ac142e80 0x7f44ac15beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:08.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.751+0000 7f44b47ff700 1 --2- 172.21.15.92:0/3906263904 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44ac06a980 0x7f44ac0c4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:08.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.751+0000 7f44b47ff700 1 --2- 172.21.15.92:0/3906263904 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f44900682f0 0x7f449006a7a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:08.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.751+0000 7f44b47ff700 1 --2- 172.21.15.92:0/3906263904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44ac0ca600 0x7f44ac0aca00 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:08.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.751+0000 7f44b47ff700 1 -- 172.21.15.92:0/3906263904 >> 172.21.15.92:0/3906263904 conn(0x7f44ac0cb490 msgr2=0x7f44ac0cda50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:08.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.751+0000 7f44b47ff700 1 -- 172.21.15.92:0/3906263904 shutdown_connections 2023-11-28T11:22:08.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.752+0000 7f44b47ff700 1 -- 172.21.15.92:0/3906263904 wait complete. 2023-11-28T11:22:08.771 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:22:08.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.965+0000 7fe0d11ff700 1 Processor -- start 2023-11-28T11:22:08.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.965+0000 7fe0d11ff700 1 -- start start 2023-11-28T11:22:08.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.966+0000 7fe0d11ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe0cc0b8510 0x7fe0cc06b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:08.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.966+0000 7fe0d11ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0cc0d0d60 0x7fe0cc0d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:08.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.966+0000 7fe0d11ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0cc0b88e0 con 0x7fe0cc0cd460 2023-11-28T11:22:08.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.966+0000 7fe0d11ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0cc0cdbd0 con 0x7fe0cc0b8510 2023-11-28T11:22:08.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.966+0000 7fe0d11ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0cc0d02d0 con 0x7fe0cc0d0d60 2023-11-28T11:22:08.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.966+0000 7fe0cad9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe0cc0b8510 0x7fe0cc06b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:08.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.967+0000 7fe0cad9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe0cc0b8510 0x7fe0cc06b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56658/0 (socket says 172.21.15.92:56658) 2023-11-28T11:22:08.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.967+0000 7fe0cad9d700 1 -- 172.21.15.92:0/3053817644 learned_addr learned my addr 172.21.15.92:0/3053817644 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:08.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.967+0000 7fe0cb59e700 1 --2- 172.21.15.92:0/3053817644 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0cc0d0d60 0x7fe0cc0d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:08.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.967+0000 7fe0cad9d700 1 -- 172.21.15.92:0/3053817644 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0cc0d0d60 msgr2=0x7fe0cc0d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:08.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.967+0000 7fe0cad9d700 1 --2- 172.21.15.92:0/3053817644 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0cc0d0d60 0x7fe0cc0d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:08.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.967+0000 7fe0cad9d700 1 -- 172.21.15.92:0/3053817644 >> v1:172.21.15.92:6789/0 conn(0x7fe0cc0cd460 legacy=0x7fe0cc06beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:08.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.967+0000 7fe0cad9d700 1 -- 172.21.15.92:0/3053817644 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0cc0cd8f0 con 0x7fe0cc0b8510 2023-11-28T11:22:08.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.967+0000 7fe0cad9d700 1 --2- 172.21.15.92:0/3053817644 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0cc0b8510 0x7fe0cc06b970 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc009920 tx=0x7fe0bc009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d5a347624ed341c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:08.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.967+0000 7fe0c9d9b700 1 -- 172.21.15.92:0/3053817644 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc00ba20 con 0x7fe0cc0b8510 2023-11-28T11:22:08.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.968+0000 7fe0c9d9b700 1 -- 172.21.15.92:0/3053817644 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0bc00bb80 con 0x7fe0cc0b8510 2023-11-28T11:22:08.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.968+0000 7fe0c9d9b700 1 -- 172.21.15.92:0/3053817644 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc009e70 con 0x7fe0cc0b8510 2023-11-28T11:22:08.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.968+0000 7fe0d11ff700 1 -- 172.21.15.92:0/3053817644 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0cc0b8510 msgr2=0x7fe0cc06b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:08.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.968+0000 7fe0d11ff700 1 --2- 172.21.15.92:0/3053817644 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0cc0b8510 0x7fe0cc06b970 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc009920 tx=0x7fe0bc009c30 comp rx=0 tx=0).stop 2023-11-28T11:22:08.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.968+0000 7fe0d11ff700 1 -- 172.21.15.92:0/3053817644 shutdown_connections 2023-11-28T11:22:08.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.968+0000 7fe0d11ff700 1 --2- 172.21.15.92:0/3053817644 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0cc0d0d60 0x7fe0cc0d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:08.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.968+0000 7fe0d11ff700 1 --2- 172.21.15.92:0/3053817644 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0cc0b8510 0x7fe0cc06b970 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:08.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.968+0000 7fe0d11ff700 1 -- 172.21.15.92:0/3053817644 >> 172.21.15.92:0/3053817644 conn(0x7fe0cc0719a0 msgr2=0x7fe0cc071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:08.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.968+0000 7fe0d11ff700 1 -- 172.21.15.92:0/3053817644 shutdown_connections 2023-11-28T11:22:08.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.968+0000 7fe0d11ff700 1 -- 172.21.15.92:0/3053817644 wait complete. 2023-11-28T11:22:08.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.971+0000 7fe0d11ff700 1 Processor -- start 2023-11-28T11:22:08.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.971+0000 7fe0d11ff700 1 -- start start 2023-11-28T11:22:08.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.972+0000 7fe0d11ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0cc0b8510 0x7fe0cc15a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:08.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.972+0000 7fe0d11ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0cc0cd460 0x7fe0cc0c8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:08.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.972+0000 7fe0d11ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0cc0d0d60 0x7fe0cc0c8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:08.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.972+0000 7fe0d11ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0cc158910 con 0x7fe0cc0b8510 2023-11-28T11:22:08.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.972+0000 7fe0d11ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0cc0c9280 con 0x7fe0cc0d0d60 2023-11-28T11:22:08.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.972+0000 7fe0d11ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0cc1f66f0 con 0x7fe0cc0cd460 2023-11-28T11:22:08.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.972+0000 7fe0cad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0cc0b8510 0x7fe0cc15a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:08.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.972+0000 7fe0cad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0cc0b8510 0x7fe0cc15a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48588/0 (socket says 172.21.15.92:48588) 2023-11-28T11:22:08.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.972+0000 7fe0cad9d700 1 -- 172.21.15.92:0/662040514 learned_addr learned my addr 172.21.15.92:0/662040514 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:08.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.972+0000 7fe0ca59c700 1 --2- 172.21.15.92:0/662040514 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0cc0cd460 0x7fe0cc0c8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:08.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.972+0000 7fe0cb59e700 1 --2- 172.21.15.92:0/662040514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0cc0d0d60 0x7fe0cc0c8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:08.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.973+0000 7fe0cad9d700 1 -- 172.21.15.92:0/662040514 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0cc0cd460 msgr2=0x7fe0cc0c8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:08.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.973+0000 7fe0cad9d700 1 --2- 172.21.15.92:0/662040514 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0cc0cd460 0x7fe0cc0c8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:08.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.973+0000 7fe0cad9d700 1 -- 172.21.15.92:0/662040514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0cc0d0d60 msgr2=0x7fe0cc0c8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:08.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.973+0000 7fe0cad9d700 1 --2- 172.21.15.92:0/662040514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0cc0d0d60 0x7fe0cc0c8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:08.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.973+0000 7fe0cad9d700 1 -- 172.21.15.92:0/662040514 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0bc009550 con 0x7fe0cc0b8510 2023-11-28T11:22:08.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.973+0000 7fe0cb59e700 1 --2- 172.21.15.92:0/662040514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0cc0d0d60 0x7fe0cc0c8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:22:08.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.973+0000 7fe0cad9d700 1 --2- 172.21.15.92:0/662040514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0cc0b8510 0x7fe0cc15a4a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc00c010 tx=0x7fe0bc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:08.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.973+0000 7fe0b37fe700 1 -- 172.21.15.92:0/662040514 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc004300 con 0x7fe0cc0b8510 2023-11-28T11:22:08.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.973+0000 7fe0b37fe700 1 -- 172.21.15.92:0/662040514 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0bc01f070 con 0x7fe0cc0b8510 2023-11-28T11:22:08.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.973+0000 7fe0b37fe700 1 -- 172.21.15.92:0/662040514 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc0036a0 con 0x7fe0cc0b8510 2023-11-28T11:22:08.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.973+0000 7fe0d11ff700 1 -- 172.21.15.92:0/662040514 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0cc0cfff0 con 0x7fe0cc0b8510 2023-11-28T11:22:08.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.973+0000 7fe0d11ff700 1 -- 172.21.15.92:0/662040514 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0cc0cfd10 con 0x7fe0cc0b8510 2023-11-28T11:22:08.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.974+0000 7fe0d11ff700 1 -- 172.21.15.92:0/662040514 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0cc067880 con 0x7fe0cc0b8510 2023-11-28T11:22:08.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.975+0000 7fe0b37fe700 1 -- 172.21.15.92:0/662040514 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe0bc02a020 con 0x7fe0cc0b8510 2023-11-28T11:22:08.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.976+0000 7fe0b37fe700 1 --2- 172.21.15.92:0/662040514 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe0b80682f0 0x7fe0b806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:08.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.976+0000 7fe0b37fe700 1 -- 172.21.15.92:0/662040514 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fe0bc087610 con 0x7fe0cc0b8510 2023-11-28T11:22:08.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.976+0000 7fe0ca59c700 1 --2- 172.21.15.92:0/662040514 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe0b80682f0 0x7fe0b806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:08.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.977+0000 7fe0ca59c700 1 --2- 172.21.15.92:0/662040514 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe0b80682f0 0x7fe0b806a7a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc156350 tx=0x7fe0c0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:08.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:08.980+0000 7fe0b37fe700 1 -- 172.21.15.92:0/662040514 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe0bc053a00 con 0x7fe0cc0b8510 2023-11-28T11:22:09.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.214+0000 7fe0d11ff700 1 -- 172.21.15.92:0/662040514 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe0cc0d2940 con 0x7fe0cc0b8510 2023-11-28T11:22:09.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.214+0000 7fe0b37fe700 1 -- 172.21.15.92:0/662040514 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4647 (secure 0 0 0) 0x7fe0bc057020 con 0x7fe0cc0b8510 2023-11-28T11:22:09.215 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:22:09.216 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/3257209990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3257209990},{"type":"v1","addr":"172.21.15.92:6836","nonce":3257209990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2193338949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2193338949},{"type":"v1","addr":"172.21.15.169:6834","nonce":2193338949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4308,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/471035619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":471035619},{"type":"v1","addr":"172.21.15.92:6837","nonce":471035619}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:20:56.234970+0000","modified":"2023-11-28T11:21:16.544352+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4326},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4326":{"gid":4326,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/2067251916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2067251916},{"type":"v1","addr":"172.21.15.169:6835","nonce":2067251916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T11:22:09.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.216+0000 7fe0d11ff700 1 -- 172.21.15.92:0/662040514 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe0b80682f0 msgr2=0x7fe0b806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:09.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.216+0000 7fe0d11ff700 1 --2- 172.21.15.92:0/662040514 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe0b80682f0 0x7fe0b806a7a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc156350 tx=0x7fe0c0009040 comp rx=0 tx=0).stop 2023-11-28T11:22:09.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.217+0000 7fe0d11ff700 1 -- 172.21.15.92:0/662040514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0cc0b8510 msgr2=0x7fe0cc15a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:09.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.217+0000 7fe0d11ff700 1 --2- 172.21.15.92:0/662040514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0cc0b8510 0x7fe0cc15a4a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc00c010 tx=0x7fe0bc004750 comp rx=0 tx=0).stop 2023-11-28T11:22:09.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.217+0000 7fe0d11ff700 1 -- 172.21.15.92:0/662040514 shutdown_connections 2023-11-28T11:22:09.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.217+0000 7fe0d11ff700 1 --2- 172.21.15.92:0/662040514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0cc0d0d60 0x7fe0cc0c8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:09.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.217+0000 7fe0d11ff700 1 --2- 172.21.15.92:0/662040514 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe0b80682f0 0x7fe0b806a7a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:09.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.217+0000 7fe0d11ff700 1 --2- 172.21.15.92:0/662040514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0cc0b8510 0x7fe0cc15a4a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:09.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.217+0000 7fe0d11ff700 1 --2- 172.21.15.92:0/662040514 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0cc0cd460 0x7fe0cc0c8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:09.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.217+0000 7fe0d11ff700 1 -- 172.21.15.92:0/662040514 >> 172.21.15.92:0/662040514 conn(0x7fe0cc0719a0 msgr2=0x7fe0cc0b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:09.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.217+0000 7fe0d11ff700 1 -- 172.21.15.92:0/662040514 shutdown_connections 2023-11-28T11:22:09.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.217+0000 7fe0d11ff700 1 -- 172.21.15.92:0/662040514 wait complete. 2023-11-28T11:22:09.219 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 9 2023-11-28T11:22:09.235 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:22:09.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.430+0000 7f7eaafff700 1 Processor -- start 2023-11-28T11:22:09.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.430+0000 7f7eaafff700 1 -- start start 2023-11-28T11:22:09.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.431+0000 7f7eaafff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7ea40b8510 0x7f7ea406b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:09.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.431+0000 7f7eaafff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f7ea40cd460 0x7f7ea406beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:09.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.431+0000 7f7eaafff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ea406c3f0 0x7f7ea415cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:09.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.431+0000 7f7eaafff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ea40cdbd0 con 0x7f7ea40b8510 2023-11-28T11:22:09.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.431+0000 7f7eaafff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ea40d02d0 con 0x7f7ea40cd460 2023-11-28T11:22:09.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.431+0000 7f7eaafff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea40b88e0 con 0x7f7ea406c3f0 2023-11-28T11:22:09.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.431+0000 7f7ea8d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7ea40b8510 0x7f7ea406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:09.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.431+0000 7f7ea8d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7ea40b8510 0x7f7ea406b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48600/0 (socket says 172.21.15.92:48600) 2023-11-28T11:22:09.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.431+0000 7f7ea8d9b700 1 -- 172.21.15.92:0/2391540881 learned_addr learned my addr 172.21.15.92:0/2391540881 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:09.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.431+0000 7f7ea959c700 1 --2- 172.21.15.92:0/2391540881 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ea406c3f0 0x7f7ea415cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:09.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.431+0000 7f7ea3fff700 1 --2- 172.21.15.92:0/2391540881 >> v2:172.21.15.169:3300/0 conn(0x7f7ea40cd460 0x7f7ea406beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:09.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.432+0000 7f7ea8d9b700 1 -- 172.21.15.92:0/2391540881 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ea406c3f0 msgr2=0x7f7ea415cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:09.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.432+0000 7f7ea8d9b700 1 --2- 172.21.15.92:0/2391540881 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ea406c3f0 0x7f7ea415cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:09.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.432+0000 7f7ea8d9b700 1 -- 172.21.15.92:0/2391540881 >> v2:172.21.15.169:3300/0 conn(0x7f7ea40cd460 msgr2=0x7f7ea406beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:09.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.432+0000 7f7ea8d9b700 1 --2- 172.21.15.92:0/2391540881 >> v2:172.21.15.169:3300/0 conn(0x7f7ea40cd460 0x7f7ea406beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:09.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.432+0000 7f7ea8d9b700 1 -- 172.21.15.92:0/2391540881 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea40cd8f0 con 0x7f7ea40b8510 2023-11-28T11:22:09.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.432+0000 7f7ea8d9b700 1 --2- 172.21.15.92:0/2391540881 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ea40b8510 0x7f7ea406b970 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f7e98009920 tx=0x7f7e98009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f12947f7d5c8cf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:09.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.432+0000 7f7ea37fe700 1 -- 172.21.15.92:0/2391540881 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e98011830 con 0x7f7ea40b8510 2023-11-28T11:22:09.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.432+0000 7f7ea37fe700 1 -- 172.21.15.92:0/2391540881 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e98011990 con 0x7f7ea40b8510 2023-11-28T11:22:09.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.432+0000 7f7ea37fe700 1 -- 172.21.15.92:0/2391540881 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e98011d20 con 0x7f7ea40b8510 2023-11-28T11:22:09.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.433+0000 7f7eaafff700 1 -- 172.21.15.92:0/2391540881 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ea40b8510 msgr2=0x7f7ea406b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:09.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.433+0000 7f7eaafff700 1 --2- 172.21.15.92:0/2391540881 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ea40b8510 0x7f7ea406b970 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f7e98009920 tx=0x7f7e98009c30 comp rx=0 tx=0).stop 2023-11-28T11:22:09.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.433+0000 7f7eaafff700 1 -- 172.21.15.92:0/2391540881 shutdown_connections 2023-11-28T11:22:09.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.433+0000 7f7eaafff700 1 --2- 172.21.15.92:0/2391540881 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ea406c3f0 0x7f7ea415cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:09.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.433+0000 7f7eaafff700 1 --2- 172.21.15.92:0/2391540881 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ea40b8510 0x7f7ea406b970 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:09.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.433+0000 7f7eaafff700 1 --2- 172.21.15.92:0/2391540881 >> v2:172.21.15.169:3300/0 conn(0x7f7ea40cd460 0x7f7ea406beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:09.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.433+0000 7f7eaafff700 1 -- 172.21.15.92:0/2391540881 >> 172.21.15.92:0/2391540881 conn(0x7f7ea40719a0 msgr2=0x7f7ea4071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:09.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.433+0000 7f7eaafff700 1 -- 172.21.15.92:0/2391540881 shutdown_connections 2023-11-28T11:22:09.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.433+0000 7f7eaafff700 1 -- 172.21.15.92:0/2391540881 wait complete. 2023-11-28T11:22:09.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.435+0000 7f7eaafff700 1 Processor -- start 2023-11-28T11:22:09.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.435+0000 7f7eaafff700 1 -- start start 2023-11-28T11:22:09.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.436+0000 7f7eaafff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ea406c3f0 0x7f7ea40cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:09.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.436+0000 7f7eaafff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ea40b8510 0x7f7ea4155a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:09.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.436+0000 7f7eaafff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ea40cd460 0x7f7ea40ca780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:09.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.436+0000 7f7eaafff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea40ccbd0 con 0x7f7ea406c3f0 2023-11-28T11:22:09.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.436+0000 7f7eaafff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea40cceb0 con 0x7f7ea40b8510 2023-11-28T11:22:09.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.436+0000 7f7eaafff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea40c9f80 con 0x7f7ea40cd460 2023-11-28T11:22:09.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.436+0000 7f7ea8d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ea406c3f0 0x7f7ea40cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:09.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.436+0000 7f7ea959c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ea40cd460 0x7f7ea40ca780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:09.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.436+0000 7f7ea3fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ea40b8510 0x7f7ea4155a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:09.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.436+0000 7f7ea959c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ea40cd460 0x7f7ea40ca780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.92:45684/0 (socket says 172.21.15.92:45684) 2023-11-28T11:22:09.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.436+0000 7f7ea959c700 1 -- 172.21.15.92:0/3476647342 learned_addr learned my addr 172.21.15.92:0/3476647342 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:09.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.436+0000 7f7ea3fff700 1 --2- 172.21.15.92:0/3476647342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ea40b8510 0x7f7ea4155a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.92:44832/0 (socket says 172.21.15.92:44832) 2023-11-28T11:22:09.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.437+0000 7f7ea8d9b700 1 -- 172.21.15.92:0/3476647342 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ea40cd460 msgr2=0x7f7ea40ca780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:09.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.437+0000 7f7ea8d9b700 1 --2- 172.21.15.92:0/3476647342 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ea40cd460 0x7f7ea40ca780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:09.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.437+0000 7f7ea8d9b700 1 -- 172.21.15.92:0/3476647342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ea40b8510 msgr2=0x7f7ea4155a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:09.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.437+0000 7f7ea8d9b700 1 --2- 172.21.15.92:0/3476647342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ea40b8510 0x7f7ea4155a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:09.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.437+0000 7f7ea8d9b700 1 -- 172.21.15.92:0/3476647342 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e98009550 con 0x7f7ea406c3f0 2023-11-28T11:22:09.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.437+0000 7f7ea8d9b700 1 --2- 172.21.15.92:0/3476647342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ea406c3f0 0x7f7ea40cbf40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f7e98016000 tx=0x7f7e98004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:09.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.437+0000 7f7ea17fa700 1 -- 172.21.15.92:0/3476647342 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e98004300 con 0x7f7ea406c3f0 2023-11-28T11:22:09.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.437+0000 7f7ea17fa700 1 -- 172.21.15.92:0/3476647342 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e9801f070 con 0x7f7ea406c3f0 2023-11-28T11:22:09.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.437+0000 7f7ea17fa700 1 -- 172.21.15.92:0/3476647342 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e980036a0 con 0x7f7ea406c3f0 2023-11-28T11:22:09.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.437+0000 7f7eaafff700 1 -- 172.21.15.92:0/3476647342 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ea40cfff0 con 0x7f7ea406c3f0 2023-11-28T11:22:09.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.437+0000 7f7eaafff700 1 -- 172.21.15.92:0/3476647342 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ea40cfd10 con 0x7f7ea406c3f0 2023-11-28T11:22:09.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.438+0000 7f7eaafff700 1 -- 172.21.15.92:0/3476647342 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ea4067880 con 0x7f7ea406c3f0 2023-11-28T11:22:09.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.439+0000 7f7ea17fa700 1 -- 172.21.15.92:0/3476647342 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7e9802a020 con 0x7f7ea406c3f0 2023-11-28T11:22:09.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.440+0000 7f7ea17fa700 1 --2- 172.21.15.92:0/3476647342 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7e880682f0 0x7f7e8806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:09.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.440+0000 7f7ea17fa700 1 -- 172.21.15.92:0/3476647342 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f7e98087350 con 0x7f7ea406c3f0 2023-11-28T11:22:09.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.441+0000 7f7ea3fff700 1 --2- 172.21.15.92:0/3476647342 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7e880682f0 0x7f7e8806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:09.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.441+0000 7f7ea3fff700 1 --2- 172.21.15.92:0/3476647342 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7e880682f0 0x7f7e8806a7a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7ea41f5640 tx=0x7f7e90008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:09.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.445+0000 7f7ea17fa700 1 -- 172.21.15.92:0/3476647342 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7e98053880 con 0x7f7ea406c3f0 2023-11-28T11:22:09.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.635+0000 7f7eaafff700 1 -- 172.21.15.92:0/3476647342 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ea40d2940 con 0x7f7ea406c3f0 2023-11-28T11:22:09.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.637+0000 7f7ea17fa700 1 -- 172.21.15.92:0/3476647342 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+19844 (secure 0 0 0) 0x7f7e98056ea0 con 0x7f7ea406c3f0 2023-11-28T11:22:09.638 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:22:09.638 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":29,"fsid":"68956e0c-c16f-47cd-a220-f98364b0831d","created":"2023-11-28T11:20:28.697573+0000","modified":"2023-11-28T11:21:37.638578+0000","last_up_change":"2023-11-28T11:20:41.257011+0000","last_in_change":"2023-11-28T11:20:32.249767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:20:41.925237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:20:48.682497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:20:54.471398+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:20:55.204585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:20:58.297307+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6abe483f-0e03-4a26-b9cf-62041291747e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":115823},{"type":"v1","addr":"172.21.15.92:6809","nonce":115823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":115823},{"type":"v1","addr":"172.21.15.92:6813","nonce":115823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":115823},{"type":"v1","addr":"172.21.15.92:6821","nonce":115823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":115823},{"type":"v1","addr":"172.21.15.92:6817","nonce":115823}]},"public_addr":"172.21.15.92:6809/115823","cluster_addr":"172.21.15.92:6813/115823","heartbeat_back_addr":"172.21.15.92:6821/115823","heartbeat_front_addr":"172.21.15.92:6817/115823","state":["exists","up"]},{"osd":1,"uuid":"13c1de73-1a8c-4d7e-8a28-9dae52dfe089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":115824},{"type":"v1","addr":"172.21.15.92:6812","nonce":115824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":115824},{"type":"v1","addr":"172.21.15.92:6816","nonce":115824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":115824},{"type":"v1","addr":"172.21.15.92:6823","nonce":115824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":115824},{"type":"v1","addr":"172.21.15.92:6820","nonce":115824}]},"public_addr":"172.21.15.92:6812/115824","cluster_addr":"172.21.15.92:6816/115824","heartbeat_back_addr":"172.21.15.92:6823/115824","heartbeat_front_addr":"172.21.15.92:6820/115824","state":["exists","up"]},{"osd":2,"uuid":"25788a55-112f-4f00-810d-2bc37cf76778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":115821},{"type":"v1","addr":"172.21.15.92:6801","nonce":115821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":115821},{"type":"v1","addr":"172.21.15.92:6803","nonce":115821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":115821},{"type":"v1","addr":"172.21.15.92:6807","nonce":115821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":115821},{"type":"v1","addr":"172.21.15.92:6805","nonce":115821}]},"public_addr":"172.21.15.92:6801/115821","cluster_addr":"172.21.15.92:6803/115821","heartbeat_back_addr":"172.21.15.92:6807/115821","heartbeat_front_addr":"172.21.15.92:6805/115821","state":["exists","up"]},{"osd":3,"uuid":"adf54e24-33a9-41fe-9ce3-98a5ab6c158e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":115825},{"type":"v1","addr":"172.21.15.92:6825","nonce":115825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":115825},{"type":"v1","addr":"172.21.15.92:6827","nonce":115825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":115825},{"type":"v1","addr":"172.21.15.92:6831","nonce":115825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":115825},{"type":"v1","addr":"172.21.15.92:6829","nonce":115825}]},"public_addr":"172.21.15.92:6825/115825","cluster_addr":"172.21.15.92:6827/115825","heartbeat_back_addr":"172.21.15.92:6831/115825","heartbeat_front_addr":"172.21.15.92:6829/115825","state":["exists","up"]},{"osd":4,"uuid":"ccb9b747-f399-44ea-9a18-9db3152227e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":116035},{"type":"v1","addr":"172.21.15.169:6817","nonce":116035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":116035},{"type":"v1","addr":"172.21.15.169:6819","nonce":116035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":116035},{"type":"v1","addr":"172.21.15.169:6826","nonce":116035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":116035},{"type":"v1","addr":"172.21.15.169:6822","nonce":116035}]},"public_addr":"172.21.15.169:6817/116035","cluster_addr":"172.21.15.169:6819/116035","heartbeat_back_addr":"172.21.15.169:6826/116035","heartbeat_front_addr":"172.21.15.169:6822/116035","state":["exists","up"]},{"osd":5,"uuid":"308a1e63-75aa-4d7d-8a7c-cecc24714fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":116033},{"type":"v1","addr":"172.21.15.169:6801","nonce":116033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":116033},{"type":"v1","addr":"172.21.15.169:6803","nonce":116033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":116033},{"type":"v1","addr":"172.21.15.169:6807","nonce":116033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":116033},{"type":"v1","addr":"172.21.15.169:6805","nonce":116033}]},"public_addr":"172.21.15.169:6801/116033","cluster_addr":"172.21.15.169:6803/116033","heartbeat_back_addr":"172.21.15.169:6807/116033","heartbeat_front_addr":"172.21.15.169:6805/116033","state":["exists","up"]},{"osd":6,"uuid":"9c459b7b-b821-4b4b-b91e-6aaae271457c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":116034},{"type":"v1","addr":"172.21.15.169:6809","nonce":116034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":116034},{"type":"v1","addr":"172.21.15.169:6811","nonce":116034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":116034},{"type":"v1","addr":"172.21.15.169:6815","nonce":116034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":116034},{"type":"v1","addr":"172.21.15.169:6813","nonce":116034}]},"public_addr":"172.21.15.169:6809/116034","cluster_addr":"172.21.15.169:6811/116034","heartbeat_back_addr":"172.21.15.169:6815/116034","heartbeat_front_addr":"172.21.15.169:6813/116034","state":["exists","up"]},{"osd":7,"uuid":"1ca9aa4a-755d-4e28-a937-4d8dccab4df3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":116044},{"type":"v1","addr":"172.21.15.169:6823","nonce":116044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6825","nonce":116044},{"type":"v1","addr":"172.21.15.169:6827","nonce":116044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":116044},{"type":"v1","addr":"172.21.15.169:6831","nonce":116044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":116044},{"type":"v1","addr":"172.21.15.169:6829","nonce":116044}]},"public_addr":"172.21.15.169:6823/116044","cluster_addr":"172.21.15.169:6827/116044","heartbeat_back_addr":"172.21.15.169:6831/116044","heartbeat_front_addr":"172.21.15.169:6829/116044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.586781+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.869778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.508594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.860493+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.872267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.746933+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.611520+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.871446+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.1d","mappings":[{"from":3,"to":2}]},{"pgid":"4.21","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:22:09.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.639+0000 7f7eaafff700 1 -- 172.21.15.92:0/3476647342 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7e880682f0 msgr2=0x7f7e8806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:09.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.639+0000 7f7eaafff700 1 --2- 172.21.15.92:0/3476647342 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7e880682f0 0x7f7e8806a7a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7ea41f5640 tx=0x7f7e90008040 comp rx=0 tx=0).stop 2023-11-28T11:22:09.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.639+0000 7f7eaafff700 1 -- 172.21.15.92:0/3476647342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ea406c3f0 msgr2=0x7f7ea40cbf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:09.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.639+0000 7f7eaafff700 1 --2- 172.21.15.92:0/3476647342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ea406c3f0 0x7f7ea40cbf40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f7e98016000 tx=0x7f7e98004750 comp rx=0 tx=0).stop 2023-11-28T11:22:09.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.639+0000 7f7eaafff700 1 -- 172.21.15.92:0/3476647342 shutdown_connections 2023-11-28T11:22:09.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.639+0000 7f7eaafff700 1 --2- 172.21.15.92:0/3476647342 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7ea40cd460 0x7f7ea40ca780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:09.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.639+0000 7f7eaafff700 1 --2- 172.21.15.92:0/3476647342 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7e880682f0 0x7f7e8806a7a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:09.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.639+0000 7f7eaafff700 1 --2- 172.21.15.92:0/3476647342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ea406c3f0 0x7f7ea40cbf40 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:09.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.639+0000 7f7eaafff700 1 --2- 172.21.15.92:0/3476647342 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7ea40b8510 0x7f7ea4155a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:09.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.639+0000 7f7eaafff700 1 -- 172.21.15.92:0/3476647342 >> 172.21.15.92:0/3476647342 conn(0x7f7ea40719a0 msgr2=0x7f7ea40b7a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:09.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.639+0000 7f7eaafff700 1 -- 172.21.15.92:0/3476647342 shutdown_connections 2023-11-28T11:22:09.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.639+0000 7f7eaafff700 1 -- 172.21.15.92:0/3476647342 wait complete. 2023-11-28T11:22:09.660 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T11:22:09.660 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:22:09.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.857+0000 7f8f9f3ff700 1 Processor -- start 2023-11-28T11:22:09.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.858+0000 7f8f9f3ff700 1 -- start start 2023-11-28T11:22:09.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.859+0000 7f8f9f3ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f980d1d70 0x7f8f98167270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:09.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.859+0000 7f8f9f3ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f9806c790 con 0x7f8f980b8510 2023-11-28T11:22:09.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.859+0000 7f8f9f3ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f980b88e0 con 0x7f8f980cd460 2023-11-28T11:22:09.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.859+0000 7f8f9f3ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f980cdbd0 con 0x7f8f980d1d70 2023-11-28T11:22:09.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.859+0000 7f8f9d19b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f8f980b8510 0x7f8f9806b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56130/0 (socket says 172.21.15.92:56130) 2023-11-28T11:22:09.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.859+0000 7f8f9d19b700 1 -- 172.21.15.92:0/1279596375 learned_addr learned my addr 172.21.15.92:0/1279596375 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:22:09.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.859+0000 7f8f9d99c700 1 --2- 172.21.15.92:0/1279596375 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f980d1d70 0x7f8f98167270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:09.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.859+0000 7f8f8ffff700 1 -- 172.21.15.92:0/1279596375 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1930510447 0 0) 0x7f8f9806c790 con 0x7f8f980b8510 2023-11-28T11:22:09.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.859+0000 7f8f8ffff700 1 -- 172.21.15.92:0/1279596375 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f840035b0 con 0x7f8f980b8510 2023-11-28T11:22:09.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.860+0000 7f8f8ffff700 1 -- 172.21.15.92:0/1279596375 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 867608904 0 0) 0x7f8f840035b0 con 0x7f8f980b8510 2023-11-28T11:22:09.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.860+0000 7f8f8ffff700 1 -- 172.21.15.92:0/1279596375 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f980d1d70 msgr2=0x7f8f98167270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:09.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.860+0000 7f8f8ffff700 1 --2- 172.21.15.92:0/1279596375 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f980d1d70 0x7f8f98167270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:09.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.860+0000 7f8f8ffff700 1 -- 172.21.15.92:0/1279596375 >> v1:172.21.15.169:6789/0 conn(0x7f8f980cd460 legacy=0x7f8f9806c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:09.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.860+0000 7f8f8ffff700 1 -- 172.21.15.92:0/1279596375 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f980cd8f0 con 0x7f8f980b8510 2023-11-28T11:22:09.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.860+0000 7f8f8ffff700 1 -- 172.21.15.92:0/1279596375 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532161587 0 0) 0x7f8f94001670 con 0x7f8f980b8510 2023-11-28T11:22:09.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.860+0000 7f8f8ffff700 1 -- 172.21.15.92:0/1279596375 >> v1:172.21.15.92:6789/0 conn(0x7f8f980b8510 legacy=0x7f8f9806b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:09.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.860+0000 7f8f8ffff700 1 --2- 172.21.15.92:0/1279596375 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f980d1d70 0x7f8f84003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:09.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.860+0000 7f8f8ffff700 1 --2- 172.21.15.92:0/1279596375 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f84003f70 0x7f8f84004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:09.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.860+0000 7f8f8ffff700 1 --2- 172.21.15.92:0/1279596375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f84004d90 0x7f8f84005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:09.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.860+0000 7f8f8ffff700 1 -- 172.21.15.92:0/1279596375 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f980cdbd0 con 0x7f8f84004d90 2023-11-28T11:22:09.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.860+0000 7f8f8ffff700 1 -- 172.21.15.92:0/1279596375 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8400bac0 con 0x7f8f84003f70 2023-11-28T11:22:09.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.860+0000 7f8f8ffff700 1 -- 172.21.15.92:0/1279596375 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8400bc80 con 0x7f8f980d1d70 2023-11-28T11:22:09.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.860+0000 7f8f9d99c700 1 --2- 172.21.15.92:0/1279596375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f84004d90 0x7f8f84005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:09.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.860+0000 7f8f9d19b700 1 --2- 172.21.15.92:0/1279596375 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f980d1d70 0x7f8f84003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:09.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.860+0000 7f8f9c99a700 1 --2- 172.21.15.92:0/1279596375 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f84003f70 0x7f8f84004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:09.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.861+0000 7f8f9d99c700 1 -- 172.21.15.92:0/1279596375 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f980d1d70 msgr2=0x7f8f84003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:09.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.861+0000 7f8f9d99c700 1 --2- 172.21.15.92:0/1279596375 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f980d1d70 0x7f8f84003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:09.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.861+0000 7f8f9d99c700 1 -- 172.21.15.92:0/1279596375 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f84003f70 msgr2=0x7f8f84004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:09.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.861+0000 7f8f9d99c700 1 --2- 172.21.15.92:0/1279596375 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f84003f70 0x7f8f84004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:09.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.861+0000 7f8f9d99c700 1 -- 172.21.15.92:0/1279596375 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f840035b0 con 0x7f8f84004d90 2023-11-28T11:22:09.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.861+0000 7f8f9c99a700 1 --2- 172.21.15.92:0/1279596375 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f84003f70 0x7f8f84004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:22:09.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.861+0000 7f8f9d99c700 1 --2- 172.21.15.92:0/1279596375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f84004d90 0x7f8f84005200 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8f90009f90 tx=0x7f8f84005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d03fcc8fd82237ba server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:09.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.861+0000 7f8f8ffff700 1 -- 172.21.15.92:0/1279596375 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9000ec30 con 0x7f8f84004d90 2023-11-28T11:22:09.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.861+0000 7f8f8ffff700 1 -- 172.21.15.92:0/1279596375 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f9000f040 con 0x7f8f84004d90 2023-11-28T11:22:09.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.861+0000 7f8f8ffff700 1 -- 172.21.15.92:0/1279596375 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f90014480 con 0x7f8f84004d90 2023-11-28T11:22:09.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.862+0000 7f8f9f3ff700 1 -- 172.21.15.92:0/1279596375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f84004d90 msgr2=0x7f8f84005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:09.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.862+0000 7f8f9f3ff700 1 --2- 172.21.15.92:0/1279596375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f84004d90 0x7f8f84005200 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8f90009f90 tx=0x7f8f84005880 comp rx=0 tx=0).stop 2023-11-28T11:22:09.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.862+0000 7f8f9d99c700 1 -- 172.21.15.92:0/1279596375 reap_dead start 2023-11-28T11:22:09.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.862+0000 7f8f9f3ff700 1 -- 172.21.15.92:0/1279596375 shutdown_connections 2023-11-28T11:22:09.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.862+0000 7f8f9f3ff700 1 -- 172.21.15.92:0/1279596375 >> 172.21.15.92:0/1279596375 conn(0x7f8f980719a0 msgr2=0x7f8f98071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:09.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.862+0000 7f8f9f3ff700 1 -- 172.21.15.92:0/1279596375 shutdown_connections 2023-11-28T11:22:09.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.862+0000 7f8f9f3ff700 1 -- 172.21.15.92:0/1279596375 wait complete. 2023-11-28T11:22:09.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.864+0000 7f8f9f3ff700 1 Processor -- start 2023-11-28T11:22:09.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.864+0000 7f8f9f3ff700 1 -- start start 2023-11-28T11:22:09.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.865+0000 7f8f9f3ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f84004d90 0x7f8f9814d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:09.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.866+0000 7f8f9f3ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f980ca760 0x7f8f980cabd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:09.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.866+0000 7f8f9f3ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f9815b120 0x7f8f9815b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:09.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.866+0000 7f8f9f3ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9815ad00 con 0x7f8f9815b120 2023-11-28T11:22:09.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.866+0000 7f8f9f3ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f980cb1e0 con 0x7f8f980ca760 2023-11-28T11:22:09.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.866+0000 7f8f9f3ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9815a360 con 0x7f8f84004d90 2023-11-28T11:22:09.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.866+0000 7f8f9d99c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f9815b120 0x7f8f9815b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:09.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.866+0000 7f8f9d99c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f9815b120 0x7f8f9815b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48630/0 (socket says 172.21.15.92:48630) 2023-11-28T11:22:09.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.866+0000 7f8f9d19b700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f84004d90 0x7f8f9814d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:09.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.866+0000 7f8f9d99c700 1 -- 172.21.15.92:0/1386934369 learned_addr learned my addr 172.21.15.92:0/1386934369 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:09.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.866+0000 7f8f9c99a700 1 --2- 172.21.15.92:0/1386934369 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f980ca760 0x7f8f980cabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:09.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.866+0000 7f8f9d99c700 1 -- 172.21.15.92:0/1386934369 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f84004d90 msgr2=0x7f8f9814d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:09.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.866+0000 7f8f9d99c700 1 --2- 172.21.15.92:0/1386934369 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f84004d90 0x7f8f9814d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:09.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.866+0000 7f8f9d99c700 1 -- 172.21.15.92:0/1386934369 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f980ca760 msgr2=0x7f8f980cabd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:09.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.866+0000 7f8f9d99c700 1 --2- 172.21.15.92:0/1386934369 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f980ca760 0x7f8f980cabd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:09.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.866+0000 7f8f9d99c700 1 -- 172.21.15.92:0/1386934369 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f980cfff0 con 0x7f8f9815b120 2023-11-28T11:22:09.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.867+0000 7f8f9d99c700 1 --2- 172.21.15.92:0/1386934369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f9815b120 0x7f8f9815b590 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8f840047b0 tx=0x7f8f90009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:09.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.867+0000 7f8f8dffb700 1 -- 172.21.15.92:0/1386934369 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f90009410 con 0x7f8f9815b120 2023-11-28T11:22:09.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.867+0000 7f8f8dffb700 1 -- 172.21.15.92:0/1386934369 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f90003a20 con 0x7f8f9815b120 2023-11-28T11:22:09.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.867+0000 7f8f9f3ff700 1 -- 172.21.15.92:0/1386934369 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f980cfd10 con 0x7f8f9815b120 2023-11-28T11:22:09.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.867+0000 7f8f8dffb700 1 -- 172.21.15.92:0/1386934369 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f90003dd0 con 0x7f8f9815b120 2023-11-28T11:22:09.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.867+0000 7f8f9f3ff700 1 -- 172.21.15.92:0/1386934369 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f980d29c0 con 0x7f8f9815b120 2023-11-28T11:22:09.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.868+0000 7f8f9f3ff700 1 -- 172.21.15.92:0/1386934369 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f98067880 con 0x7f8f9815b120 2023-11-28T11:22:09.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.869+0000 7f8f8dffb700 1 -- 172.21.15.92:0/1386934369 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8f9002d070 con 0x7f8f9815b120 2023-11-28T11:22:09.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.870+0000 7f8f8dffb700 1 --2- 172.21.15.92:0/1386934369 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8f7c0682f0 0x7f8f7c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:09.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.870+0000 7f8f8dffb700 1 -- 172.21.15.92:0/1386934369 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f8f90087470 con 0x7f8f9815b120 2023-11-28T11:22:09.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.870+0000 7f8f9d19b700 1 --2- 172.21.15.92:0/1386934369 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8f7c0682f0 0x7f8f7c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:09.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.871+0000 7f8f9d19b700 1 --2- 172.21.15.92:0/1386934369 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8f7c0682f0 0x7f8f7c06a7a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8f94007f30 tx=0x7f8f94001690 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:09.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:09.874+0000 7f8f8dffb700 1 -- 172.21.15.92:0/1386934369 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8f900538b0 con 0x7f8f9815b120 2023-11-28T11:22:10.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.106+0000 7f8f9f3ff700 1 -- 172.21.15.92:0/1386934369 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8f9815a140 con 0x7f8f9815b120 2023-11-28T11:22:10.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.107+0000 7f8f8dffb700 1 -- 172.21.15.92:0/1386934369 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4647 (secure 0 0 0) 0x7f8f90056ed0 con 0x7f8f9815b120 2023-11-28T11:22:10.109 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:22:10.109 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/3257209990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3257209990},{"type":"v1","addr":"172.21.15.92:6836","nonce":3257209990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2193338949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2193338949},{"type":"v1","addr":"172.21.15.169:6834","nonce":2193338949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4308,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/471035619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":471035619},{"type":"v1","addr":"172.21.15.92:6837","nonce":471035619}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:20:56.234970+0000","modified":"2023-11-28T11:21:16.544352+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4326},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4326":{"gid":4326,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/2067251916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2067251916},{"type":"v1","addr":"172.21.15.169:6835","nonce":2067251916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T11:22:10.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.109+0000 7f8f9f3ff700 1 -- 172.21.15.92:0/1386934369 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8f7c0682f0 msgr2=0x7f8f7c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:10.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.109+0000 7f8f9f3ff700 1 --2- 172.21.15.92:0/1386934369 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8f7c0682f0 0x7f8f7c06a7a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8f94007f30 tx=0x7f8f94001690 comp rx=0 tx=0).stop 2023-11-28T11:22:10.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.109+0000 7f8f9f3ff700 1 -- 172.21.15.92:0/1386934369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f9815b120 msgr2=0x7f8f9815b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:10.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.109+0000 7f8f9f3ff700 1 --2- 172.21.15.92:0/1386934369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f9815b120 0x7f8f9815b590 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8f840047b0 tx=0x7f8f90009750 comp rx=0 tx=0).stop 2023-11-28T11:22:10.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.109+0000 7f8f9f3ff700 1 -- 172.21.15.92:0/1386934369 shutdown_connections 2023-11-28T11:22:10.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.109+0000 7f8f9f3ff700 1 --2- 172.21.15.92:0/1386934369 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8f7c0682f0 0x7f8f7c06a7a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:10.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.109+0000 7f8f9f3ff700 1 --2- 172.21.15.92:0/1386934369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f9815b120 0x7f8f9815b590 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:10.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.109+0000 7f8f9f3ff700 1 --2- 172.21.15.92:0/1386934369 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8f84004d90 0x7f8f9814d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:10.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.109+0000 7f8f9f3ff700 1 --2- 172.21.15.92:0/1386934369 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8f980ca760 0x7f8f980cabd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:10.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.109+0000 7f8f9f3ff700 1 -- 172.21.15.92:0/1386934369 >> 172.21.15.92:0/1386934369 conn(0x7f8f980719a0 msgr2=0x7f8f9815c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:10.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.109+0000 7f8f9f3ff700 1 -- 172.21.15.92:0/1386934369 shutdown_connections 2023-11-28T11:22:10.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.109+0000 7f8f9f3ff700 1 -- 172.21.15.92:0/1386934369 wait complete. 2023-11-28T11:22:10.112 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 9 2023-11-28T11:22:10.129 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:22:10.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.326+0000 7f3b2f1ff700 1 Processor -- start 2023-11-28T11:22:10.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.326+0000 7f3b2f1ff700 1 -- start start 2023-11-28T11:22:10.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.327+0000 7f3b2f1ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b280d1d70 0x7f3b28167270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:10.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.327+0000 7f3b2f1ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b2806c790 con 0x7f3b280cd460 2023-11-28T11:22:10.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.327+0000 7f3b2f1ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b280b88e0 con 0x7f3b280b8510 2023-11-28T11:22:10.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.327+0000 7f3b2f1ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b280cdbd0 con 0x7f3b280d1d70 2023-11-28T11:22:10.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.327+0000 7f3b27fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f3b280cd460 0x7f3b2806c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56142/0 (socket says 172.21.15.92:56142) 2023-11-28T11:22:10.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.327+0000 7f3b27fff700 1 -- 172.21.15.92:0/2720712735 learned_addr learned my addr 172.21.15.92:0/2720712735 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:22:10.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.328+0000 7f3b2d79c700 1 --2- 172.21.15.92:0/2720712735 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b280d1d70 0x7f3b28167270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:10.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.327+0000 7f3b277fe700 1 -- 172.21.15.92:0/2720712735 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1144241795 0 0) 0x7f3b2806c790 con 0x7f3b280cd460 2023-11-28T11:22:10.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.328+0000 7f3b277fe700 1 -- 172.21.15.92:0/2720712735 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b180035b0 con 0x7f3b280cd460 2023-11-28T11:22:10.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.328+0000 7f3b277fe700 1 -- 172.21.15.92:0/2720712735 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3695231728 0 0) 0x7f3b180035b0 con 0x7f3b280cd460 2023-11-28T11:22:10.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.328+0000 7f3b277fe700 1 -- 172.21.15.92:0/2720712735 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b280d1d70 msgr2=0x7f3b28167270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:10.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.328+0000 7f3b277fe700 1 --2- 172.21.15.92:0/2720712735 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b280d1d70 0x7f3b28167270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:10.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.328+0000 7f3b277fe700 1 -- 172.21.15.92:0/2720712735 >> v1:172.21.15.169:6789/0 conn(0x7f3b280b8510 legacy=0x7f3b2806b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:10.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.328+0000 7f3b277fe700 1 -- 172.21.15.92:0/2720712735 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b280cd8f0 con 0x7f3b280cd460 2023-11-28T11:22:10.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.328+0000 7f3b277fe700 1 -- 172.21.15.92:0/2720712735 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532161587 0 0) 0x7f3b14001670 con 0x7f3b280cd460 2023-11-28T11:22:10.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.328+0000 7f3b277fe700 1 -- 172.21.15.92:0/2720712735 >> v1:172.21.15.92:6789/0 conn(0x7f3b280cd460 legacy=0x7f3b2806c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:10.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.328+0000 7f3b277fe700 1 --2- 172.21.15.92:0/2720712735 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b280d1d70 0x7f3b18003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:10.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.328+0000 7f3b277fe700 1 --2- 172.21.15.92:0/2720712735 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b18003f70 0x7f3b18004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:10.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.328+0000 7f3b277fe700 1 --2- 172.21.15.92:0/2720712735 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b18004d90 0x7f3b18005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:10.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.328+0000 7f3b277fe700 1 -- 172.21.15.92:0/2720712735 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b280cdbd0 con 0x7f3b18003f70 2023-11-28T11:22:10.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.328+0000 7f3b277fe700 1 -- 172.21.15.92:0/2720712735 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1800ba40 con 0x7f3b18004d90 2023-11-28T11:22:10.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.328+0000 7f3b277fe700 1 -- 172.21.15.92:0/2720712735 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1800bc00 con 0x7f3b280d1d70 2023-11-28T11:22:10.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.328+0000 7f3b27fff700 1 --2- 172.21.15.92:0/2720712735 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b18003f70 0x7f3b18004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:10.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.328+0000 7f3b2cf9b700 1 --2- 172.21.15.92:0/2720712735 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b280d1d70 0x7f3b18003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:10.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.329+0000 7f3b2d79c700 1 --2- 172.21.15.92:0/2720712735 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b18004d90 0x7f3b18005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:10.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.329+0000 7f3b27fff700 1 -- 172.21.15.92:0/2720712735 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b280d1d70 msgr2=0x7f3b18003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:10.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.329+0000 7f3b27fff700 1 --2- 172.21.15.92:0/2720712735 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b280d1d70 0x7f3b18003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:10.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.329+0000 7f3b27fff700 1 -- 172.21.15.92:0/2720712735 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b18004d90 msgr2=0x7f3b18005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:10.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.329+0000 7f3b27fff700 1 --2- 172.21.15.92:0/2720712735 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b18004d90 0x7f3b18005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:10.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.329+0000 7f3b27fff700 1 -- 172.21.15.92:0/2720712735 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b180035b0 con 0x7f3b18003f70 2023-11-28T11:22:10.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.329+0000 7f3b27fff700 1 --2- 172.21.15.92:0/2720712735 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b18003f70 0x7f3b18004850 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3b280b1de0 tx=0x7f3b18005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e8c711772787ac0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:10.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.329+0000 7f3b277fe700 1 -- 172.21.15.92:0/2720712735 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1400cd20 con 0x7f3b18003f70 2023-11-28T11:22:10.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.329+0000 7f3b277fe700 1 -- 172.21.15.92:0/2720712735 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b1400ce80 con 0x7f3b18003f70 2023-11-28T11:22:10.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.329+0000 7f3b277fe700 1 -- 172.21.15.92:0/2720712735 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b14005ba0 con 0x7f3b18003f70 2023-11-28T11:22:10.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.330+0000 7f3b2f1ff700 1 -- 172.21.15.92:0/2720712735 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b18003f70 msgr2=0x7f3b18004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:10.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.330+0000 7f3b2f1ff700 1 --2- 172.21.15.92:0/2720712735 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b18003f70 0x7f3b18004850 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3b280b1de0 tx=0x7f3b18005880 comp rx=0 tx=0).stop 2023-11-28T11:22:10.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.330+0000 7f3b2d79c700 1 -- 172.21.15.92:0/2720712735 reap_dead start 2023-11-28T11:22:10.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.330+0000 7f3b2f1ff700 1 -- 172.21.15.92:0/2720712735 shutdown_connections 2023-11-28T11:22:10.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.330+0000 7f3b2f1ff700 1 -- 172.21.15.92:0/2720712735 >> 172.21.15.92:0/2720712735 conn(0x7f3b280719a0 msgr2=0x7f3b28071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:10.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.330+0000 7f3b2f1ff700 1 -- 172.21.15.92:0/2720712735 shutdown_connections 2023-11-28T11:22:10.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.330+0000 7f3b2f1ff700 1 -- 172.21.15.92:0/2720712735 wait complete. 2023-11-28T11:22:10.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.332+0000 7f3b2f1ff700 1 Processor -- start 2023-11-28T11:22:10.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.332+0000 7f3b2f1ff700 1 -- start start 2023-11-28T11:22:10.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.333+0000 7f3b2f1ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b18003f70 0x7f3b2815a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:10.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.333+0000 7f3b2f1ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b2814d350 0x7f3b280b5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:10.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.333+0000 7f3b2f1ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b280b6210 0x7f3b280ca930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:10.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.333+0000 7f3b2f1ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b280b66b0 con 0x7f3b2814d350 2023-11-28T11:22:10.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.333+0000 7f3b2f1ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b281f5f50 con 0x7f3b18003f70 2023-11-28T11:22:10.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.333+0000 7f3b2f1ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2814d7c0 con 0x7f3b280b6210 2023-11-28T11:22:10.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.333+0000 7f3b27fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b2814d350 0x7f3b280b5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:10.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.333+0000 7f3b27fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b2814d350 0x7f3b280b5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48656/0 (socket says 172.21.15.92:48656) 2023-11-28T11:22:10.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.333+0000 7f3b27fff700 1 -- 172.21.15.92:0/1470945733 learned_addr learned my addr 172.21.15.92:0/1470945733 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:10.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.333+0000 7f3b2cf9b700 1 --2- 172.21.15.92:0/1470945733 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b18003f70 0x7f3b2815a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:10.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.333+0000 7f3b2d79c700 1 --2- 172.21.15.92:0/1470945733 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b280b6210 0x7f3b280ca930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:10.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.333+0000 7f3b27fff700 1 -- 172.21.15.92:0/1470945733 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b280b6210 msgr2=0x7f3b280ca930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:10.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.333+0000 7f3b27fff700 1 --2- 172.21.15.92:0/1470945733 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b280b6210 0x7f3b280ca930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:10.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.333+0000 7f3b27fff700 1 -- 172.21.15.92:0/1470945733 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b18003f70 msgr2=0x7f3b2815a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:10.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.333+0000 7f3b27fff700 1 --2- 172.21.15.92:0/1470945733 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b18003f70 0x7f3b2815a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:10.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.333+0000 7f3b27fff700 1 -- 172.21.15.92:0/1470945733 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b280cfff0 con 0x7f3b2814d350 2023-11-28T11:22:10.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.333+0000 7f3b2d79c700 1 --2- 172.21.15.92:0/1470945733 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b280b6210 0x7f3b280ca930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:22:10.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.334+0000 7f3b27fff700 1 --2- 172.21.15.92:0/1470945733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b2814d350 0x7f3b280b5cd0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f3b1400cdd0 tx=0x7f3b14001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:10.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.334+0000 7f3b257fa700 1 -- 172.21.15.92:0/1470945733 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b140072d0 con 0x7f3b2814d350 2023-11-28T11:22:10.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.334+0000 7f3b257fa700 1 -- 172.21.15.92:0/1470945733 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b14007430 con 0x7f3b2814d350 2023-11-28T11:22:10.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.334+0000 7f3b257fa700 1 -- 172.21.15.92:0/1470945733 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b140077e0 con 0x7f3b2814d350 2023-11-28T11:22:10.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.334+0000 7f3b2f1ff700 1 -- 172.21.15.92:0/1470945733 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b280cfd10 con 0x7f3b2814d350 2023-11-28T11:22:10.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.334+0000 7f3b2f1ff700 1 -- 172.21.15.92:0/1470945733 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b280d29c0 con 0x7f3b2814d350 2023-11-28T11:22:10.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.335+0000 7f3b2f1ff700 1 -- 172.21.15.92:0/1470945733 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b28067880 con 0x7f3b2814d350 2023-11-28T11:22:10.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.336+0000 7f3b257fa700 1 -- 172.21.15.92:0/1470945733 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3b1402d070 con 0x7f3b2814d350 2023-11-28T11:22:10.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.337+0000 7f3b257fa700 1 --2- 172.21.15.92:0/1470945733 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3b0c0682f0 0x7f3b0c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:10.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.337+0000 7f3b257fa700 1 -- 172.21.15.92:0/1470945733 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f3b14087330 con 0x7f3b2814d350 2023-11-28T11:22:10.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.337+0000 7f3b2cf9b700 1 --2- 172.21.15.92:0/1470945733 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3b0c0682f0 0x7f3b0c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:10.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.338+0000 7f3b2cf9b700 1 --2- 172.21.15.92:0/1470945733 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3b0c0682f0 0x7f3b0c06a7a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3b280b7650 tx=0x7f3b1c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:10.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.341+0000 7f3b257fa700 1 -- 172.21.15.92:0/1470945733 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3b140536f0 con 0x7f3b2814d350 2023-11-28T11:22:10.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.575+0000 7f3b2f1ff700 1 -- 172.21.15.92:0/1470945733 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3b2815a340 con 0x7f3b2814d350 2023-11-28T11:22:10.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.575+0000 7f3b257fa700 1 -- 172.21.15.92:0/1470945733 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4647 (secure 0 0 0) 0x7f3b14056d10 con 0x7f3b2814d350 2023-11-28T11:22:10.576 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:22:10.577 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/3257209990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3257209990},{"type":"v1","addr":"172.21.15.92:6836","nonce":3257209990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2193338949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2193338949},{"type":"v1","addr":"172.21.15.169:6834","nonce":2193338949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4308,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/471035619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":471035619},{"type":"v1","addr":"172.21.15.92:6837","nonce":471035619}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:20:56.234970+0000","modified":"2023-11-28T11:21:16.544352+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4326},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4326":{"gid":4326,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.169:6835/2067251916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2067251916},{"type":"v1","addr":"172.21.15.169:6835","nonce":2067251916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T11:22:10.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.577+0000 7f3b2f1ff700 1 -- 172.21.15.92:0/1470945733 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3b0c0682f0 msgr2=0x7f3b0c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:10.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.577+0000 7f3b2f1ff700 1 --2- 172.21.15.92:0/1470945733 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3b0c0682f0 0x7f3b0c06a7a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3b280b7650 tx=0x7f3b1c009040 comp rx=0 tx=0).stop 2023-11-28T11:22:10.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.577+0000 7f3b2f1ff700 1 -- 172.21.15.92:0/1470945733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b2814d350 msgr2=0x7f3b280b5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:10.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.577+0000 7f3b2f1ff700 1 --2- 172.21.15.92:0/1470945733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b2814d350 0x7f3b280b5cd0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f3b1400cdd0 tx=0x7f3b14001980 comp rx=0 tx=0).stop 2023-11-28T11:22:10.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.577+0000 7f3b2f1ff700 1 -- 172.21.15.92:0/1470945733 shutdown_connections 2023-11-28T11:22:10.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.578+0000 7f3b2f1ff700 1 --2- 172.21.15.92:0/1470945733 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3b280b6210 0x7f3b280ca930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:10.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.578+0000 7f3b2f1ff700 1 --2- 172.21.15.92:0/1470945733 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3b18003f70 0x7f3b2815a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:10.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.578+0000 7f3b2f1ff700 1 --2- 172.21.15.92:0/1470945733 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3b0c0682f0 0x7f3b0c06a7a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:10.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.578+0000 7f3b2f1ff700 1 --2- 172.21.15.92:0/1470945733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b2814d350 0x7f3b280b5cd0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:10.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.578+0000 7f3b2f1ff700 1 -- 172.21.15.92:0/1470945733 >> 172.21.15.92:0/1470945733 conn(0x7f3b280719a0 msgr2=0x7f3b280b3730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:10.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.578+0000 7f3b2f1ff700 1 -- 172.21.15.92:0/1470945733 shutdown_connections 2023-11-28T11:22:10.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.578+0000 7f3b2f1ff700 1 -- 172.21.15.92:0/1470945733 wait complete. 2023-11-28T11:22:10.580 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 9 2023-11-28T11:22:10.597 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:22:10.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.792+0000 7fce7bfff700 1 Processor -- start 2023-11-28T11:22:10.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.792+0000 7fce7bfff700 1 -- start start 2023-11-28T11:22:10.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.793+0000 7fce7bfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fce740b8510 0x7fce7406b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:10.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.793+0000 7fce7bfff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fce740cd460 0x7fce7406beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:10.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.793+0000 7fce7bfff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce740b88e0 con 0x7fce7406c3f0 2023-11-28T11:22:10.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.793+0000 7fce7bfff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce740cdbd0 con 0x7fce740b8510 2023-11-28T11:22:10.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.793+0000 7fce7bfff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce740d02d0 con 0x7fce740cd460 2023-11-28T11:22:10.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.793+0000 7fce79d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fce740b8510 0x7fce7406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:10.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.793+0000 7fce79d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fce740b8510 0x7fce7406b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48660/0 (socket says 172.21.15.92:48660) 2023-11-28T11:22:10.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.793+0000 7fce79d9b700 1 -- 172.21.15.92:0/760977491 learned_addr learned my addr 172.21.15.92:0/760977491 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:10.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.794+0000 7fce79d9b700 1 -- 172.21.15.92:0/760977491 >> v2:172.21.15.169:3300/0 conn(0x7fce740cd460 msgr2=0x7fce7406beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:10.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.794+0000 7fce79d9b700 1 --2- 172.21.15.92:0/760977491 >> v2:172.21.15.169:3300/0 conn(0x7fce740cd460 0x7fce7406beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:10.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.794+0000 7fce79d9b700 1 -- 172.21.15.92:0/760977491 >> v1:172.21.15.169:6789/0 conn(0x7fce7406c3f0 legacy=0x7fce7415cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:10.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.794+0000 7fce79d9b700 1 -- 172.21.15.92:0/760977491 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce740cd8f0 con 0x7fce740b8510 2023-11-28T11:22:10.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.794+0000 7fce79d9b700 1 --2- 172.21.15.92:0/760977491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce740b8510 0x7fce7406b970 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fce70009920 tx=0x7fce70009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=921644a59f92e9bb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:10.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.794+0000 7fce78d99700 1 -- 172.21.15.92:0/760977491 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce70004230 con 0x7fce740b8510 2023-11-28T11:22:10.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.794+0000 7fce78d99700 1 -- 172.21.15.92:0/760977491 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce70004390 con 0x7fce740b8510 2023-11-28T11:22:10.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.794+0000 7fce78d99700 1 -- 172.21.15.92:0/760977491 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce70009e70 con 0x7fce740b8510 2023-11-28T11:22:10.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.795+0000 7fce7bfff700 1 -- 172.21.15.92:0/760977491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce740b8510 msgr2=0x7fce7406b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:10.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.795+0000 7fce7bfff700 1 --2- 172.21.15.92:0/760977491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce740b8510 0x7fce7406b970 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fce70009920 tx=0x7fce70009c30 comp rx=0 tx=0).stop 2023-11-28T11:22:10.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.795+0000 7fce7bfff700 1 -- 172.21.15.92:0/760977491 shutdown_connections 2023-11-28T11:22:10.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.795+0000 7fce7bfff700 1 --2- 172.21.15.92:0/760977491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce740b8510 0x7fce7406b970 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:10.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.795+0000 7fce7bfff700 1 --2- 172.21.15.92:0/760977491 >> v2:172.21.15.169:3300/0 conn(0x7fce740cd460 0x7fce7406beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:10.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.795+0000 7fce7bfff700 1 -- 172.21.15.92:0/760977491 >> 172.21.15.92:0/760977491 conn(0x7fce740719a0 msgr2=0x7fce74071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:10.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.795+0000 7fce7bfff700 1 -- 172.21.15.92:0/760977491 shutdown_connections 2023-11-28T11:22:10.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.795+0000 7fce7bfff700 1 -- 172.21.15.92:0/760977491 wait complete. 2023-11-28T11:22:10.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.797+0000 7fce7bfff700 1 Processor -- start 2023-11-28T11:22:10.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.797+0000 7fce7bfff700 1 -- start start 2023-11-28T11:22:10.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.798+0000 7fce7bfff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce7406c3f0 0x7fce7414e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:10.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.798+0000 7fce7bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce740b8510 0x7fce740d1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:10.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.798+0000 7fce7bfff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce740cd460 0x7fce740d2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:10.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.798+0000 7fce7bfff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce741e3ad0 con 0x7fce740b8510 2023-11-28T11:22:10.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.798+0000 7fce7bfff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce741e36b0 con 0x7fce7406c3f0 2023-11-28T11:22:10.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.798+0000 7fce7bfff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce741e2a30 con 0x7fce740cd460 2023-11-28T11:22:10.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.798+0000 7fce7959a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce740b8510 0x7fce740d1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:10.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.798+0000 7fce7959a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce740b8510 0x7fce740d1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48676/0 (socket says 172.21.15.92:48676) 2023-11-28T11:22:10.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.798+0000 7fce7959a700 1 -- 172.21.15.92:0/3156318002 learned_addr learned my addr 172.21.15.92:0/3156318002 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:10.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.798+0000 7fce79d9b700 1 --2- 172.21.15.92:0/3156318002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce7406c3f0 0x7fce7414e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:10.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.798+0000 7fce7959a700 1 -- 172.21.15.92:0/3156318002 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce740cd460 msgr2=0x7fce740d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:10.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.798+0000 7fce7959a700 1 --2- 172.21.15.92:0/3156318002 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce740cd460 0x7fce740d2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:10.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.798+0000 7fce7959a700 1 -- 172.21.15.92:0/3156318002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce7406c3f0 msgr2=0x7fce7414e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:10.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.798+0000 7fce7959a700 1 --2- 172.21.15.92:0/3156318002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce7406c3f0 0x7fce7414e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:10.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.798+0000 7fce7959a700 1 -- 172.21.15.92:0/3156318002 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce70009550 con 0x7fce740b8510 2023-11-28T11:22:10.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.798+0000 7fce7959a700 1 --2- 172.21.15.92:0/3156318002 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce740b8510 0x7fce740d1ce0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fce6400b430 tx=0x7fce6400b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:10.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.799+0000 7fce6a7fc700 1 -- 172.21.15.92:0/3156318002 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce64011830 con 0x7fce740b8510 2023-11-28T11:22:10.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.799+0000 7fce6a7fc700 1 -- 172.21.15.92:0/3156318002 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce64011990 con 0x7fce740b8510 2023-11-28T11:22:10.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.799+0000 7fce7bfff700 1 -- 172.21.15.92:0/3156318002 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce740cfff0 con 0x7fce740b8510 2023-11-28T11:22:10.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.799+0000 7fce6a7fc700 1 -- 172.21.15.92:0/3156318002 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce64011d40 con 0x7fce740b8510 2023-11-28T11:22:10.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.799+0000 7fce7bfff700 1 -- 172.21.15.92:0/3156318002 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce740cfd10 con 0x7fce740b8510 2023-11-28T11:22:10.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.800+0000 7fce7bfff700 1 -- 172.21.15.92:0/3156318002 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce741e37f0 con 0x7fce740b8510 2023-11-28T11:22:10.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.801+0000 7fce6a7fc700 1 -- 172.21.15.92:0/3156318002 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fce64021070 con 0x7fce740b8510 2023-11-28T11:22:10.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.802+0000 7fce6a7fc700 1 --2- 172.21.15.92:0/3156318002 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fce580682f0 0x7fce5806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:10.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.802+0000 7fce6a7fc700 1 -- 172.21.15.92:0/3156318002 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fce64085f90 con 0x7fce740b8510 2023-11-28T11:22:10.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.802+0000 7fce79d9b700 1 --2- 172.21.15.92:0/3156318002 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fce580682f0 0x7fce5806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:10.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.802+0000 7fce79d9b700 1 --2- 172.21.15.92:0/3156318002 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fce580682f0 0x7fce5806a7a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fce70009c70 tx=0x7fce70008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:10.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.806+0000 7fce6a7fc700 1 -- 172.21.15.92:0/3156318002 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fce741e37f0 con 0x7fce740b8510 2023-11-28T11:22:11.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:10.999+0000 7fce7bfff700 1 -- 172.21.15.92:0/3156318002 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fce741e37f0 con 0x7fce740b8510 2023-11-28T11:22:11.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.001+0000 7fce6a7fc700 1 -- 172.21.15.92:0/3156318002 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+19844 (secure 0 0 0) 0x7fce741e37f0 con 0x7fce740b8510 2023-11-28T11:22:11.002 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:22:11.002 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":29,"fsid":"68956e0c-c16f-47cd-a220-f98364b0831d","created":"2023-11-28T11:20:28.697573+0000","modified":"2023-11-28T11:21:37.638578+0000","last_up_change":"2023-11-28T11:20:41.257011+0000","last_in_change":"2023-11-28T11:20:32.249767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:20:41.925237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:20:48.682497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:20:54.471398+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:20:55.204585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:20:58.297307+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6abe483f-0e03-4a26-b9cf-62041291747e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":115823},{"type":"v1","addr":"172.21.15.92:6809","nonce":115823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":115823},{"type":"v1","addr":"172.21.15.92:6813","nonce":115823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":115823},{"type":"v1","addr":"172.21.15.92:6821","nonce":115823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":115823},{"type":"v1","addr":"172.21.15.92:6817","nonce":115823}]},"public_addr":"172.21.15.92:6809/115823","cluster_addr":"172.21.15.92:6813/115823","heartbeat_back_addr":"172.21.15.92:6821/115823","heartbeat_front_addr":"172.21.15.92:6817/115823","state":["exists","up"]},{"osd":1,"uuid":"13c1de73-1a8c-4d7e-8a28-9dae52dfe089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":115824},{"type":"v1","addr":"172.21.15.92:6812","nonce":115824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":115824},{"type":"v1","addr":"172.21.15.92:6816","nonce":115824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":115824},{"type":"v1","addr":"172.21.15.92:6823","nonce":115824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":115824},{"type":"v1","addr":"172.21.15.92:6820","nonce":115824}]},"public_addr":"172.21.15.92:6812/115824","cluster_addr":"172.21.15.92:6816/115824","heartbeat_back_addr":"172.21.15.92:6823/115824","heartbeat_front_addr":"172.21.15.92:6820/115824","state":["exists","up"]},{"osd":2,"uuid":"25788a55-112f-4f00-810d-2bc37cf76778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":115821},{"type":"v1","addr":"172.21.15.92:6801","nonce":115821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":115821},{"type":"v1","addr":"172.21.15.92:6803","nonce":115821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":115821},{"type":"v1","addr":"172.21.15.92:6807","nonce":115821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":115821},{"type":"v1","addr":"172.21.15.92:6805","nonce":115821}]},"public_addr":"172.21.15.92:6801/115821","cluster_addr":"172.21.15.92:6803/115821","heartbeat_back_addr":"172.21.15.92:6807/115821","heartbeat_front_addr":"172.21.15.92:6805/115821","state":["exists","up"]},{"osd":3,"uuid":"adf54e24-33a9-41fe-9ce3-98a5ab6c158e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":115825},{"type":"v1","addr":"172.21.15.92:6825","nonce":115825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":115825},{"type":"v1","addr":"172.21.15.92:6827","nonce":115825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":115825},{"type":"v1","addr":"172.21.15.92:6831","nonce":115825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":115825},{"type":"v1","addr":"172.21.15.92:6829","nonce":115825}]},"public_addr":"172.21.15.92:6825/115825","cluster_addr":"172.21.15.92:6827/115825","heartbeat_back_addr":"172.21.15.92:6831/115825","heartbeat_front_addr":"172.21.15.92:6829/115825","state":["exists","up"]},{"osd":4,"uuid":"ccb9b747-f399-44ea-9a18-9db3152227e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":116035},{"type":"v1","addr":"172.21.15.169:6817","nonce":116035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":116035},{"type":"v1","addr":"172.21.15.169:6819","nonce":116035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":116035},{"type":"v1","addr":"172.21.15.169:6826","nonce":116035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":116035},{"type":"v1","addr":"172.21.15.169:6822","nonce":116035}]},"public_addr":"172.21.15.169:6817/116035","cluster_addr":"172.21.15.169:6819/116035","heartbeat_back_addr":"172.21.15.169:6826/116035","heartbeat_front_addr":"172.21.15.169:6822/116035","state":["exists","up"]},{"osd":5,"uuid":"308a1e63-75aa-4d7d-8a7c-cecc24714fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":116033},{"type":"v1","addr":"172.21.15.169:6801","nonce":116033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":116033},{"type":"v1","addr":"172.21.15.169:6803","nonce":116033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":116033},{"type":"v1","addr":"172.21.15.169:6807","nonce":116033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":116033},{"type":"v1","addr":"172.21.15.169:6805","nonce":116033}]},"public_addr":"172.21.15.169:6801/116033","cluster_addr":"172.21.15.169:6803/116033","heartbeat_back_addr":"172.21.15.169:6807/116033","heartbeat_front_addr":"172.21.15.169:6805/116033","state":["exists","up"]},{"osd":6,"uuid":"9c459b7b-b821-4b4b-b91e-6aaae271457c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":116034},{"type":"v1","addr":"172.21.15.169:6809","nonce":116034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":116034},{"type":"v1","addr":"172.21.15.169:6811","nonce":116034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":116034},{"type":"v1","addr":"172.21.15.169:6815","nonce":116034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":116034},{"type":"v1","addr":"172.21.15.169:6813","nonce":116034}]},"public_addr":"172.21.15.169:6809/116034","cluster_addr":"172.21.15.169:6811/116034","heartbeat_back_addr":"172.21.15.169:6815/116034","heartbeat_front_addr":"172.21.15.169:6813/116034","state":["exists","up"]},{"osd":7,"uuid":"1ca9aa4a-755d-4e28-a937-4d8dccab4df3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":116044},{"type":"v1","addr":"172.21.15.169:6823","nonce":116044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6825","nonce":116044},{"type":"v1","addr":"172.21.15.169:6827","nonce":116044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":116044},{"type":"v1","addr":"172.21.15.169:6831","nonce":116044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":116044},{"type":"v1","addr":"172.21.15.169:6829","nonce":116044}]},"public_addr":"172.21.15.169:6823/116044","cluster_addr":"172.21.15.169:6827/116044","heartbeat_back_addr":"172.21.15.169:6831/116044","heartbeat_front_addr":"172.21.15.169:6829/116044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.586781+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.869778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.508594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.860493+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.872267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.746933+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.611520+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.871446+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.1d","mappings":[{"from":3,"to":2}]},{"pgid":"4.21","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:22:11.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.003+0000 7fce7bfff700 1 -- 172.21.15.92:0/3156318002 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fce580682f0 msgr2=0x7fce5806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:11.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.003+0000 7fce7bfff700 1 --2- 172.21.15.92:0/3156318002 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fce580682f0 0x7fce5806a7a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fce70009c70 tx=0x7fce70008030 comp rx=0 tx=0).stop 2023-11-28T11:22:11.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.003+0000 7fce7bfff700 1 -- 172.21.15.92:0/3156318002 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce740b8510 msgr2=0x7fce740d1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:11.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.003+0000 7fce7bfff700 1 --2- 172.21.15.92:0/3156318002 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce740b8510 0x7fce740d1ce0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fce6400b430 tx=0x7fce6400b7f0 comp rx=0 tx=0).stop 2023-11-28T11:22:11.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.003+0000 7fce7bfff700 1 -- 172.21.15.92:0/3156318002 shutdown_connections 2023-11-28T11:22:11.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.003+0000 7fce7bfff700 1 --2- 172.21.15.92:0/3156318002 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fce740cd460 0x7fce740d2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:11.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.003+0000 7fce7bfff700 1 --2- 172.21.15.92:0/3156318002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fce7406c3f0 0x7fce7414e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:11.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.003+0000 7fce7bfff700 1 --2- 172.21.15.92:0/3156318002 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fce580682f0 0x7fce5806a7a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:11.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.003+0000 7fce7bfff700 1 --2- 172.21.15.92:0/3156318002 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce740b8510 0x7fce740d1ce0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:11.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.003+0000 7fce7bfff700 1 -- 172.21.15.92:0/3156318002 >> 172.21.15.92:0/3156318002 conn(0x7fce740719a0 msgr2=0x7fce740cca80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:11.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.003+0000 7fce7bfff700 1 -- 172.21.15.92:0/3156318002 shutdown_connections 2023-11-28T11:22:11.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.003+0000 7fce7bfff700 1 -- 172.21.15.92:0/3156318002 wait complete. 2023-11-28T11:22:11.022 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T11:22:11.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.219+0000 7fe7279ff700 1 Processor -- start 2023-11-28T11:22:11.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.219+0000 7fe7279ff700 1 -- start start 2023-11-28T11:22:11.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.221+0000 7fe7279ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe7200b8510 0x7fe72006b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:11.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.221+0000 7fe7279ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe7200d0d60 0x7fe7200d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:11.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.221+0000 7fe7279ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7200b88e0 con 0x7fe7200cd460 2023-11-28T11:22:11.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.221+0000 7fe7279ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7200cdbd0 con 0x7fe7200b8510 2023-11-28T11:22:11.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.221+0000 7fe7279ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7200d02d0 con 0x7fe7200d0d60 2023-11-28T11:22:11.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.221+0000 7fe72579b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe7200b8510 0x7fe72006b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:11.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.221+0000 7fe72579b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe7200b8510 0x7fe72006b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48694/0 (socket says 172.21.15.92:48694) 2023-11-28T11:22:11.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.221+0000 7fe72579b700 1 -- 172.21.15.92:0/1401167361 learned_addr learned my addr 172.21.15.92:0/1401167361 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:11.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.222+0000 7fe725f9c700 1 --2- 172.21.15.92:0/1401167361 >> v2:172.21.15.169:3300/0 conn(0x7fe7200d0d60 0x7fe7200d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:11.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.222+0000 7fe72579b700 1 -- 172.21.15.92:0/1401167361 >> v2:172.21.15.169:3300/0 conn(0x7fe7200d0d60 msgr2=0x7fe7200d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:11.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.222+0000 7fe72579b700 1 --2- 172.21.15.92:0/1401167361 >> v2:172.21.15.169:3300/0 conn(0x7fe7200d0d60 0x7fe7200d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:11.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.222+0000 7fe72579b700 1 -- 172.21.15.92:0/1401167361 >> v1:172.21.15.169:6789/0 conn(0x7fe7200cd460 legacy=0x7fe72006beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:11.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.222+0000 7fe72579b700 1 -- 172.21.15.92:0/1401167361 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7200cd8f0 con 0x7fe7200b8510 2023-11-28T11:22:11.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.222+0000 7fe72579b700 1 --2- 172.21.15.92:0/1401167361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7200b8510 0x7fe72006b970 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe7200ce8c0 tx=0x7fe71c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7779823436c24379 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:11.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.222+0000 7fe717fff700 1 -- 172.21.15.92:0/1401167361 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71c00b860 con 0x7fe7200b8510 2023-11-28T11:22:11.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.222+0000 7fe717fff700 1 -- 172.21.15.92:0/1401167361 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe71c00b9c0 con 0x7fe7200b8510 2023-11-28T11:22:11.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.222+0000 7fe717fff700 1 -- 172.21.15.92:0/1401167361 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71c00bd50 con 0x7fe7200b8510 2023-11-28T11:22:11.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.223+0000 7fe7279ff700 1 -- 172.21.15.92:0/1401167361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7200b8510 msgr2=0x7fe72006b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:11.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.223+0000 7fe7279ff700 1 --2- 172.21.15.92:0/1401167361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7200b8510 0x7fe72006b970 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe7200ce8c0 tx=0x7fe71c009c00 comp rx=0 tx=0).stop 2023-11-28T11:22:11.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.223+0000 7fe7279ff700 1 -- 172.21.15.92:0/1401167361 shutdown_connections 2023-11-28T11:22:11.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.223+0000 7fe7279ff700 1 --2- 172.21.15.92:0/1401167361 >> v2:172.21.15.169:3300/0 conn(0x7fe7200d0d60 0x7fe7200d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:11.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.223+0000 7fe7279ff700 1 --2- 172.21.15.92:0/1401167361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7200b8510 0x7fe72006b970 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:11.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.223+0000 7fe7279ff700 1 -- 172.21.15.92:0/1401167361 >> 172.21.15.92:0/1401167361 conn(0x7fe7200719a0 msgr2=0x7fe720071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:11.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.223+0000 7fe7279ff700 1 -- 172.21.15.92:0/1401167361 shutdown_connections 2023-11-28T11:22:11.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.223+0000 7fe7279ff700 1 -- 172.21.15.92:0/1401167361 wait complete. 2023-11-28T11:22:11.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.225+0000 7fe7279ff700 1 Processor -- start 2023-11-28T11:22:11.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.226+0000 7fe7279ff700 1 -- start start 2023-11-28T11:22:11.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.227+0000 7fe7279ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7200b8510 0x7fe720149a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:11.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.227+0000 7fe7279ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7200cd460 0x7fe7200c97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:11.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.227+0000 7fe7279ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7200d0d60 0x7fe720147ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:11.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.227+0000 7fe7279ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7200c9d30 con 0x7fe7200cd460 2023-11-28T11:22:11.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.227+0000 7fe7279ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe720149570 con 0x7fe7200b8510 2023-11-28T11:22:11.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.227+0000 7fe7279ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe720148960 con 0x7fe7200d0d60 2023-11-28T11:22:11.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.227+0000 7fe724f9a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7200cd460 0x7fe7200c97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:11.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.227+0000 7fe72579b700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7200b8510 0x7fe720149a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:11.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.227+0000 7fe724f9a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7200cd460 0x7fe7200c97f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48700/0 (socket says 172.21.15.92:48700) 2023-11-28T11:22:11.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.227+0000 7fe724f9a700 1 -- 172.21.15.92:0/3653978248 learned_addr learned my addr 172.21.15.92:0/3653978248 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:11.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.227+0000 7fe72579b700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7200b8510 0x7fe720149a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.92:44910/0 (socket says 172.21.15.92:44910) 2023-11-28T11:22:11.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.227+0000 7fe725f9c700 1 --2- 172.21.15.92:0/3653978248 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7200d0d60 0x7fe720147ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:11.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.228+0000 7fe724f9a700 1 -- 172.21.15.92:0/3653978248 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7200d0d60 msgr2=0x7fe720147ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:11.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.228+0000 7fe724f9a700 1 --2- 172.21.15.92:0/3653978248 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7200d0d60 0x7fe720147ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:11.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.228+0000 7fe724f9a700 1 -- 172.21.15.92:0/3653978248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7200b8510 msgr2=0x7fe720149a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:11.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.228+0000 7fe724f9a700 1 --2- 172.21.15.92:0/3653978248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7200b8510 0x7fe720149a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:11.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.228+0000 7fe724f9a700 1 -- 172.21.15.92:0/3653978248 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe71c009550 con 0x7fe7200cd460 2023-11-28T11:22:11.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.228+0000 7fe724f9a700 1 --2- 172.21.15.92:0/3653978248 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7200cd460 0x7fe7200c97f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe71000c480 tx=0x7fe71000c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:11.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.228+0000 7fe715ffb700 1 -- 172.21.15.92:0/3653978248 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7100052d0 con 0x7fe7200cd460 2023-11-28T11:22:11.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.228+0000 7fe715ffb700 1 -- 172.21.15.92:0/3653978248 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe71000ce90 con 0x7fe7200cd460 2023-11-28T11:22:11.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.228+0000 7fe715ffb700 1 -- 172.21.15.92:0/3653978248 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe710003680 con 0x7fe7200cd460 2023-11-28T11:22:11.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.228+0000 7fe7279ff700 1 -- 172.21.15.92:0/3653978248 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7200cfff0 con 0x7fe7200cd460 2023-11-28T11:22:11.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.228+0000 7fe7279ff700 1 -- 172.21.15.92:0/3653978248 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7200cfd10 con 0x7fe7200cd460 2023-11-28T11:22:11.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.229+0000 7fe7279ff700 1 -- 172.21.15.92:0/3653978248 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe720067880 con 0x7fe7200cd460 2023-11-28T11:22:11.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.230+0000 7fe715ffb700 1 -- 172.21.15.92:0/3653978248 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe710019030 con 0x7fe7200cd460 2023-11-28T11:22:11.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.231+0000 7fe715ffb700 1 --2- 172.21.15.92:0/3653978248 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe7040682f0 0x7fe70406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:11.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.231+0000 7fe715ffb700 1 -- 172.21.15.92:0/3653978248 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fe710085240 con 0x7fe7200cd460 2023-11-28T11:22:11.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.231+0000 7fe72579b700 1 --2- 172.21.15.92:0/3653978248 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe7040682f0 0x7fe70406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:11.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.232+0000 7fe72579b700 1 --2- 172.21.15.92:0/3653978248 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe7040682f0 0x7fe70406a7a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe71c000c00 tx=0x7fe71c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:11.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.235+0000 7fe715ffb700 1 -- 172.21.15.92:0/3653978248 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe710051680 con 0x7fe7200cd460 2023-11-28T11:22:11.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.466+0000 7fe7279ff700 1 -- 172.21.15.92:0/3653978248 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe7200d2940 con 0x7fe7200cd460 2023-11-28T11:22:11.779 INFO:tasks.ceph.mon.a.smithi092.stderr:2023-11-28T11:22:11.778+0000 7f086abf9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T11:22:11.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.783+0000 7fe715ffb700 1 -- 172.21.15.92:0/3653978248 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe710054ca0 con 0x7fe7200cd460 2023-11-28T11:22:11.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.786+0000 7fe7279ff700 1 -- 172.21.15.92:0/3653978248 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe7040682f0 msgr2=0x7fe70406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:11.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.786+0000 7fe7279ff700 1 --2- 172.21.15.92:0/3653978248 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe7040682f0 0x7fe70406a7a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe71c000c00 tx=0x7fe71c008030 comp rx=0 tx=0).stop 2023-11-28T11:22:11.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.786+0000 7fe7279ff700 1 -- 172.21.15.92:0/3653978248 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7200cd460 msgr2=0x7fe7200c97f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:11.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.786+0000 7fe7279ff700 1 --2- 172.21.15.92:0/3653978248 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7200cd460 0x7fe7200c97f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe71000c480 tx=0x7fe71000c790 comp rx=0 tx=0).stop 2023-11-28T11:22:11.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.786+0000 7fe7279ff700 1 -- 172.21.15.92:0/3653978248 shutdown_connections 2023-11-28T11:22:11.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.786+0000 7fe7279ff700 1 --2- 172.21.15.92:0/3653978248 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe7200d0d60 0x7fe720147ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:11.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.786+0000 7fe7279ff700 1 --2- 172.21.15.92:0/3653978248 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe7200b8510 0x7fe720149a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:11.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.786+0000 7fe7279ff700 1 --2- 172.21.15.92:0/3653978248 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe7040682f0 0x7fe70406a7a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:11.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.786+0000 7fe7279ff700 1 --2- 172.21.15.92:0/3653978248 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7200cd460 0x7fe7200c97f0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:11.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.786+0000 7fe7279ff700 1 -- 172.21.15.92:0/3653978248 >> 172.21.15.92:0/3653978248 conn(0x7fe7200719a0 msgr2=0x7fe7200d1200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:11.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.786+0000 7fe7279ff700 1 -- 172.21.15.92:0/3653978248 shutdown_connections 2023-11-28T11:22:11.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:11.786+0000 7fe7279ff700 1 -- 172.21.15.92:0/3653978248 wait complete. 2023-11-28T11:22:11.788 INFO:teuthology.orchestra.run.smithi092.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T11:22:11.801 INFO:tasks.ceph.mds.b.smithi169.stderr: -6045> 2023-11-28T11:21:01.727+0000 7ff7b7ff1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T11:22:11.806 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T11:22:11.855 INFO:tasks.ceph.mds.b.smithi169.stdout:starting mds.b at 2023-11-28T11:22:12.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.005+0000 7f6bf73ff700 1 Processor -- start 2023-11-28T11:22:12.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.006+0000 7f6bf73ff700 1 -- start start 2023-11-28T11:22:12.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.007+0000 7f6bf73ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6bf00cd460 0x7f6bf006c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:12.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.007+0000 7f6bf73ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6bf00d0d60 0x7f6bf00d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:12.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.007+0000 7f6bf73ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bf00b88e0 con 0x7f6bf00b8510 2023-11-28T11:22:12.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.007+0000 7f6bf73ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bf00cdbd0 con 0x7f6bf00cd460 2023-11-28T11:22:12.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.007+0000 7f6bf73ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf00d02d0 con 0x7f6bf00d0d60 2023-11-28T11:22:12.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.007+0000 7f6bf499a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6bf00cd460 0x7f6bf006c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:12.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.007+0000 7f6bf499a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6bf00cd460 0x7f6bf006c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48716/0 (socket says 172.21.15.92:48716) 2023-11-28T11:22:12.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.007+0000 7f6bf499a700 1 -- 172.21.15.92:0/3652126900 learned_addr learned my addr 172.21.15.92:0/3652126900 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:12.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.007+0000 7f6bf599c700 1 --2- 172.21.15.92:0/3652126900 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6bf00d0d60 0x7f6bf00d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:12.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.008+0000 7f6bf499a700 1 -- 172.21.15.92:0/3652126900 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6bf00d0d60 msgr2=0x7f6bf00d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:12.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.008+0000 7f6bf499a700 1 --2- 172.21.15.92:0/3652126900 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6bf00d0d60 0x7f6bf00d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:12.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.008+0000 7f6bf499a700 1 -- 172.21.15.92:0/3652126900 >> v1:172.21.15.169:6789/0 conn(0x7f6bf00b8510 legacy=0x7f6bf006b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:12.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.008+0000 7f6bf499a700 1 -- 172.21.15.92:0/3652126900 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bf00cd8f0 con 0x7f6bf00cd460 2023-11-28T11:22:12.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.008+0000 7f6bf499a700 1 --2- 172.21.15.92:0/3652126900 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bf00cd460 0x7f6bf006c080 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6be0009920 tx=0x7f6be0009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4da7528c119cd6b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:12.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.008+0000 7f6be7fff700 1 -- 172.21.15.92:0/3652126900 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be000ba20 con 0x7f6bf00cd460 2023-11-28T11:22:12.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.008+0000 7f6be7fff700 1 -- 172.21.15.92:0/3652126900 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6be000bb80 con 0x7f6bf00cd460 2023-11-28T11:22:12.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.008+0000 7f6be7fff700 1 -- 172.21.15.92:0/3652126900 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be0009e70 con 0x7f6bf00cd460 2023-11-28T11:22:12.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.009+0000 7f6bf73ff700 1 -- 172.21.15.92:0/3652126900 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bf00cd460 msgr2=0x7f6bf006c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:12.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.009+0000 7f6bf73ff700 1 --2- 172.21.15.92:0/3652126900 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bf00cd460 0x7f6bf006c080 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6be0009920 tx=0x7f6be0009c30 comp rx=0 tx=0).stop 2023-11-28T11:22:12.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.009+0000 7f6bf73ff700 1 -- 172.21.15.92:0/3652126900 shutdown_connections 2023-11-28T11:22:12.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.009+0000 7f6bf73ff700 1 --2- 172.21.15.92:0/3652126900 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6bf00d0d60 0x7f6bf00d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:12.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.009+0000 7f6bf73ff700 1 --2- 172.21.15.92:0/3652126900 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bf00cd460 0x7f6bf006c080 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:12.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.009+0000 7f6bf73ff700 1 -- 172.21.15.92:0/3652126900 >> 172.21.15.92:0/3652126900 conn(0x7f6bf00719a0 msgr2=0x7f6bf0071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:12.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.009+0000 7f6bf73ff700 1 -- 172.21.15.92:0/3652126900 shutdown_connections 2023-11-28T11:22:12.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.009+0000 7f6bf73ff700 1 -- 172.21.15.92:0/3652126900 wait complete. 2023-11-28T11:22:12.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.010+0000 7f6bf73ff700 1 Processor -- start 2023-11-28T11:22:12.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.011+0000 7f6bf73ff700 1 -- start start 2023-11-28T11:22:12.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.011+0000 7f6bf73ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6bf00b8510 0x7f6bf01e2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:12.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.011+0000 7f6bf73ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bf00cd460 0x7f6bf01e6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:12.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.011+0000 7f6bf73ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bf00d0d60 0x7f6bf015a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:12.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.011+0000 7f6bf73ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf01e1e70 con 0x7f6bf00d0d60 2023-11-28T11:22:12.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.011+0000 7f6bf73ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf01e2150 con 0x7f6bf00cd460 2023-11-28T11:22:12.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.011+0000 7f6bf73ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf01e2430 con 0x7f6bf00b8510 2023-11-28T11:22:12.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.012+0000 7f6bf599c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bf00d0d60 0x7f6bf015a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:12.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.012+0000 7f6bf599c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bf00d0d60 0x7f6bf015a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48732/0 (socket says 172.21.15.92:48732) 2023-11-28T11:22:12.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.012+0000 7f6bf599c700 1 -- 172.21.15.92:0/3690399812 learned_addr learned my addr 172.21.15.92:0/3690399812 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:12.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.012+0000 7f6bf499a700 1 --2- 172.21.15.92:0/3690399812 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bf00cd460 0x7f6bf01e6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:12.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.012+0000 7f6bf519b700 1 --2- 172.21.15.92:0/3690399812 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6bf00b8510 0x7f6bf01e2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:12.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.012+0000 7f6bf599c700 1 -- 172.21.15.92:0/3690399812 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6bf00b8510 msgr2=0x7f6bf01e2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:12.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.012+0000 7f6bf599c700 1 --2- 172.21.15.92:0/3690399812 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6bf00b8510 0x7f6bf01e2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:12.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.012+0000 7f6bf599c700 1 -- 172.21.15.92:0/3690399812 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bf00cd460 msgr2=0x7f6bf01e6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:12.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.012+0000 7f6bf599c700 1 --2- 172.21.15.92:0/3690399812 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bf00cd460 0x7f6bf01e6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:12.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.012+0000 7f6bf599c700 1 -- 172.21.15.92:0/3690399812 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be0009550 con 0x7f6bf00d0d60 2023-11-28T11:22:12.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.012+0000 7f6bf599c700 1 --2- 172.21.15.92:0/3690399812 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bf00d0d60 0x7f6bf015a3d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6be8007e60 tx=0x7f6be800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:12.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.013+0000 7f6be5ffb700 1 -- 172.21.15.92:0/3690399812 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be800ede0 con 0x7f6bf00d0d60 2023-11-28T11:22:12.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.013+0000 7f6be5ffb700 1 -- 172.21.15.92:0/3690399812 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6be8014420 con 0x7f6bf00d0d60 2023-11-28T11:22:12.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.013+0000 7f6be5ffb700 1 -- 172.21.15.92:0/3690399812 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be80147d0 con 0x7f6bf00d0d60 2023-11-28T11:22:12.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.013+0000 7f6bf73ff700 1 -- 172.21.15.92:0/3690399812 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bf00cfff0 con 0x7f6bf00d0d60 2023-11-28T11:22:12.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.013+0000 7f6bf73ff700 1 -- 172.21.15.92:0/3690399812 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bf00cfd10 con 0x7f6bf00d0d60 2023-11-28T11:22:12.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.014+0000 7f6bf73ff700 1 -- 172.21.15.92:0/3690399812 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bf0067880 con 0x7f6bf00d0d60 2023-11-28T11:22:12.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.015+0000 7f6be5ffb700 1 -- 172.21.15.92:0/3690399812 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6be8012050 con 0x7f6bf00d0d60 2023-11-28T11:22:12.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.015+0000 7f6be5ffb700 1 --2- 172.21.15.92:0/3690399812 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6bd40682f0 0x7f6bd406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:12.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.016+0000 7f6be5ffb700 1 -- 172.21.15.92:0/3690399812 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6957+0+0 (secure 0 0 0) 0x7f6be80866a0 con 0x7f6bf00d0d60 2023-11-28T11:22:12.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.016+0000 7f6bf519b700 1 --2- 172.21.15.92:0/3690399812 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6bd40682f0 0x7f6bd406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:12.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.016+0000 7f6bf519b700 1 --2- 172.21.15.92:0/3690399812 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6bd40682f0 0x7f6bd406a7a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6bf00b2e00 tx=0x7f6bec009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:12.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.019+0000 7f6be5ffb700 1 -- 172.21.15.92:0/3690399812 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6be8052a30 con 0x7f6bf00d0d60 2023-11-28T11:22:12.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.256+0000 7f6bf73ff700 1 -- 172.21.15.92:0/3690399812 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f6bf00d2940 con 0x7f6bf00d0d60 2023-11-28T11:22:12.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.791+0000 7f6be5ffb700 1 -- 172.21.15.92:0/3690399812 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6be8056050 con 0x7f6bf00d0d60 2023-11-28T11:22:12.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.793+0000 7f6bf73ff700 1 -- 172.21.15.92:0/3690399812 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6bd40682f0 msgr2=0x7f6bd406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:12.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.793+0000 7f6bf73ff700 1 --2- 172.21.15.92:0/3690399812 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6bd40682f0 0x7f6bd406a7a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6bf00b2e00 tx=0x7f6bec009040 comp rx=0 tx=0).stop 2023-11-28T11:22:12.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.793+0000 7f6bf73ff700 1 -- 172.21.15.92:0/3690399812 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bf00d0d60 msgr2=0x7f6bf015a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:12.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.793+0000 7f6bf73ff700 1 --2- 172.21.15.92:0/3690399812 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bf00d0d60 0x7f6bf015a3d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6be8007e60 tx=0x7f6be800e3c0 comp rx=0 tx=0).stop 2023-11-28T11:22:12.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.793+0000 7f6bf73ff700 1 -- 172.21.15.92:0/3690399812 shutdown_connections 2023-11-28T11:22:12.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.793+0000 7f6bf73ff700 1 --2- 172.21.15.92:0/3690399812 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6bd40682f0 0x7f6bd406a7a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:12.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.793+0000 7f6bf73ff700 1 --2- 172.21.15.92:0/3690399812 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bf00d0d60 0x7f6bf015a3d0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:12.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.793+0000 7f6bf73ff700 1 --2- 172.21.15.92:0/3690399812 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6bf00b8510 0x7f6bf01e2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:12.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.793+0000 7f6bf73ff700 1 --2- 172.21.15.92:0/3690399812 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6bf00cd460 0x7f6bf01e6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:12.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.794+0000 7f6bf73ff700 1 -- 172.21.15.92:0/3690399812 >> 172.21.15.92:0/3690399812 conn(0x7f6bf00719a0 msgr2=0x7f6bf00b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:12.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.794+0000 7f6bf73ff700 1 -- 172.21.15.92:0/3690399812 shutdown_connections 2023-11-28T11:22:12.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:12.794+0000 7f6bf73ff700 1 -- 172.21.15.92:0/3690399812 wait complete. 2023-11-28T11:22:12.814 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T11:22:13.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.014+0000 7fad93bff700 1 Processor -- start 2023-11-28T11:22:13.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.015+0000 7fad93bff700 1 -- start start 2023-11-28T11:22:13.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.016+0000 7fad93bff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fad8c0cd460 0x7fad8c06b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:13.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.016+0000 7fad93bff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad8c06be80 0x7fad8c0d0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:13.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.016+0000 7fad93bff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad8c0b5240 con 0x7fad8c0b8790 2023-11-28T11:22:13.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.016+0000 7fad93bff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad8c06c300 con 0x7fad8c0cd460 2023-11-28T11:22:13.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.016+0000 7fad93bff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad8c0b5520 con 0x7fad8c06be80 2023-11-28T11:22:13.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.016+0000 7fad9199b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fad8c0b8790 0x7fad8c0cfb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56146/0 (socket says 172.21.15.92:56146) 2023-11-28T11:22:13.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.016+0000 7fad9199b700 1 -- 172.21.15.92:0/3272443458 learned_addr learned my addr 172.21.15.92:0/3272443458 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:22:13.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.016+0000 7fad9119a700 1 --2- 172.21.15.92:0/3272443458 >> v2:172.21.15.169:3300/0 conn(0x7fad8c0cd460 0x7fad8c06b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:13.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.016+0000 7fad9219c700 1 --2- 172.21.15.92:0/3272443458 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad8c06be80 0x7fad8c0d0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:13.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.017+0000 7fad90999700 1 -- 172.21.15.92:0/3272443458 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2071342179 0 0) 0x7fad8c0b5240 con 0x7fad8c0b8790 2023-11-28T11:22:13.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.017+0000 7fad90999700 1 -- 172.21.15.92:0/3272443458 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad780035b0 con 0x7fad8c0b8790 2023-11-28T11:22:13.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.017+0000 7fad9119a700 1 -- 172.21.15.92:0/3272443458 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad8c06be80 msgr2=0x7fad8c0d0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:13.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.017+0000 7fad9119a700 1 --2- 172.21.15.92:0/3272443458 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad8c06be80 0x7fad8c0d0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:13.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.017+0000 7fad9119a700 1 -- 172.21.15.92:0/3272443458 >> v1:172.21.15.92:6789/0 conn(0x7fad8c0b8790 legacy=0x7fad8c0cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:13.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.017+0000 7fad9119a700 1 -- 172.21.15.92:0/3272443458 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad8c0d0a80 con 0x7fad8c0cd460 2023-11-28T11:22:13.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.017+0000 7fad9219c700 1 --2- 172.21.15.92:0/3272443458 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad8c06be80 0x7fad8c0d0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:22:13.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.017+0000 7fad9119a700 1 --2- 172.21.15.92:0/3272443458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad8c0cd460 0x7fad8c06b940 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fad7c009870 tx=0x7fad7c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=996c90efe78f1e4b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:13.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.017+0000 7fad90999700 1 -- 172.21.15.92:0/3272443458 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7c011830 con 0x7fad8c0cd460 2023-11-28T11:22:13.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.017+0000 7fad90999700 1 -- 172.21.15.92:0/3272443458 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad7c011990 con 0x7fad8c0cd460 2023-11-28T11:22:13.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.017+0000 7fad90999700 1 -- 172.21.15.92:0/3272443458 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7c011d20 con 0x7fad8c0cd460 2023-11-28T11:22:13.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.018+0000 7fad93bff700 1 -- 172.21.15.92:0/3272443458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad8c0cd460 msgr2=0x7fad8c06b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:13.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.018+0000 7fad93bff700 1 --2- 172.21.15.92:0/3272443458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad8c0cd460 0x7fad8c06b940 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fad7c009870 tx=0x7fad7c009b80 comp rx=0 tx=0).stop 2023-11-28T11:22:13.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.018+0000 7fad93bff700 1 -- 172.21.15.92:0/3272443458 shutdown_connections 2023-11-28T11:22:13.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.018+0000 7fad93bff700 1 --2- 172.21.15.92:0/3272443458 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad8c06be80 0x7fad8c0d0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:13.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.018+0000 7fad93bff700 1 --2- 172.21.15.92:0/3272443458 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad8c0cd460 0x7fad8c06b940 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:13.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.018+0000 7fad93bff700 1 -- 172.21.15.92:0/3272443458 >> 172.21.15.92:0/3272443458 conn(0x7fad8c0719a0 msgr2=0x7fad8c071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:13.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.018+0000 7fad93bff700 1 -- 172.21.15.92:0/3272443458 shutdown_connections 2023-11-28T11:22:13.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.018+0000 7fad93bff700 1 -- 172.21.15.92:0/3272443458 wait complete. 2023-11-28T11:22:13.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.020+0000 7fad93bff700 1 Processor -- start 2023-11-28T11:22:13.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.020+0000 7fad93bff700 1 -- start start 2023-11-28T11:22:13.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.021+0000 7fad93bff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad8c06be80 0x7fad8c1493a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:13.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.021+0000 7fad93bff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad8c0b8790 0x7fad8c0c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:13.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.021+0000 7fad93bff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad8c0cd460 0x7fad8c1f52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:13.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.021+0000 7fad93bff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8c14b9f0 con 0x7fad8c06be80 2023-11-28T11:22:13.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.021+0000 7fad93bff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8c1498e0 con 0x7fad8c0cd460 2023-11-28T11:22:13.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.021+0000 7fad93bff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad8c14b0c0 con 0x7fad8c0b8790 2023-11-28T11:22:13.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.021+0000 7fad9199b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad8c06be80 0x7fad8c1493a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:13.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.021+0000 7fad9199b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad8c06be80 0x7fad8c1493a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48742/0 (socket says 172.21.15.92:48742) 2023-11-28T11:22:13.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.021+0000 7fad9199b700 1 -- 172.21.15.92:0/1865399163 learned_addr learned my addr 172.21.15.92:0/1865399163 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:13.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.021+0000 7fad9119a700 1 --2- 172.21.15.92:0/1865399163 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad8c0b8790 0x7fad8c0c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:13.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.021+0000 7fad9219c700 1 --2- 172.21.15.92:0/1865399163 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad8c0cd460 0x7fad8c1f52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:13.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.021+0000 7fad9199b700 1 -- 172.21.15.92:0/1865399163 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad8c0b8790 msgr2=0x7fad8c0c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:13.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.021+0000 7fad9199b700 1 --2- 172.21.15.92:0/1865399163 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad8c0b8790 0x7fad8c0c88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:13.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.021+0000 7fad9199b700 1 -- 172.21.15.92:0/1865399163 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad8c0cd460 msgr2=0x7fad8c1f52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:13.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.021+0000 7fad9199b700 1 --2- 172.21.15.92:0/1865399163 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad8c0cd460 0x7fad8c1f52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:13.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.021+0000 7fad9199b700 1 -- 172.21.15.92:0/1865399163 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad84009a70 con 0x7fad8c06be80 2023-11-28T11:22:13.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.021+0000 7fad9219c700 1 --2- 172.21.15.92:0/1865399163 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad8c0cd460 0x7fad8c1f52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:22:13.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.021+0000 7fad9119a700 1 --2- 172.21.15.92:0/1865399163 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad8c0b8790 0x7fad8c0c88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:22:13.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.022+0000 7fad9199b700 1 --2- 172.21.15.92:0/1865399163 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad8c06be80 0x7fad8c1493a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fad8800c4d0 tx=0x7fad8800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:13.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.022+0000 7fad827fc700 1 -- 172.21.15.92:0/1865399163 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad88005d70 con 0x7fad8c06be80 2023-11-28T11:22:13.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.022+0000 7fad827fc700 1 -- 172.21.15.92:0/1865399163 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad880070d0 con 0x7fad8c06be80 2023-11-28T11:22:13.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.022+0000 7fad93bff700 1 -- 172.21.15.92:0/1865399163 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad7c0094a0 con 0x7fad8c06be80 2023-11-28T11:22:13.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.022+0000 7fad827fc700 1 -- 172.21.15.92:0/1865399163 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad88007440 con 0x7fad8c06be80 2023-11-28T11:22:13.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.022+0000 7fad93bff700 1 -- 172.21.15.92:0/1865399163 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad8c0d14e0 con 0x7fad8c06be80 2023-11-28T11:22:13.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.023+0000 7fad93bff700 1 -- 172.21.15.92:0/1865399163 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad8c0b5800 con 0x7fad8c06be80 2023-11-28T11:22:13.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.024+0000 7fad827fc700 1 -- 172.21.15.92:0/1865399163 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fad88019030 con 0x7fad8c06be80 2023-11-28T11:22:13.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.025+0000 7fad827fc700 1 --2- 172.21.15.92:0/1865399163 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fad700682f0 0x7fad7006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:13.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.025+0000 7fad827fc700 1 -- 172.21.15.92:0/1865399163 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6957+0+0 (secure 0 0 0) 0x7fad880853a0 con 0x7fad8c06be80 2023-11-28T11:22:13.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.025+0000 7fad9119a700 1 --2- 172.21.15.92:0/1865399163 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fad700682f0 0x7fad7006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:13.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.025+0000 7fad9119a700 1 --2- 172.21.15.92:0/1865399163 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fad700682f0 0x7fad7006a7a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fad7c0090b0 tx=0x7fad7c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:13.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.029+0000 7fad827fc700 1 -- 172.21.15.92:0/1865399163 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fad8804b5e0 con 0x7fad8c06be80 2023-11-28T11:22:13.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.219+0000 7fad93bff700 1 -- 172.21.15.92:0/1865399163 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fad8c0d1670 con 0x7fad8c06be80 2023-11-28T11:22:13.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.876+0000 7fad827fc700 1 -- 172.21.15.92:0/1865399163 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v31) v1 ==== 181+0+0 (secure 0 0 0) 0x7fad880170d0 con 0x7fad8c06be80 2023-11-28T11:22:13.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.879+0000 7fad93bff700 1 -- 172.21.15.92:0/1865399163 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fad700682f0 msgr2=0x7fad7006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:13.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.879+0000 7fad93bff700 1 --2- 172.21.15.92:0/1865399163 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fad700682f0 0x7fad7006a7a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fad7c0090b0 tx=0x7fad7c008030 comp rx=0 tx=0).stop 2023-11-28T11:22:13.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.880+0000 7fad93bff700 1 -- 172.21.15.92:0/1865399163 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad8c06be80 msgr2=0x7fad8c1493a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:13.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.880+0000 7fad93bff700 1 --2- 172.21.15.92:0/1865399163 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad8c06be80 0x7fad8c1493a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fad8800c4d0 tx=0x7fad8800c890 comp rx=0 tx=0).stop 2023-11-28T11:22:13.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.880+0000 7fad93bff700 1 -- 172.21.15.92:0/1865399163 shutdown_connections 2023-11-28T11:22:13.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.880+0000 7fad93bff700 1 --2- 172.21.15.92:0/1865399163 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fad8c0cd460 0x7fad8c1f52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:13.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.880+0000 7fad93bff700 1 --2- 172.21.15.92:0/1865399163 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fad700682f0 0x7fad7006a7a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:13.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.880+0000 7fad93bff700 1 --2- 172.21.15.92:0/1865399163 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad8c06be80 0x7fad8c1493a0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:13.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.880+0000 7fad93bff700 1 --2- 172.21.15.92:0/1865399163 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fad8c0b8790 0x7fad8c0c88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:13.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.880+0000 7fad93bff700 1 -- 172.21.15.92:0/1865399163 >> 172.21.15.92:0/1865399163 conn(0x7fad8c0719a0 msgr2=0x7fad8c15c010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:13.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.880+0000 7fad93bff700 1 -- 172.21.15.92:0/1865399163 shutdown_connections 2023-11-28T11:22:13.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:13.880+0000 7fad93bff700 1 -- 172.21.15.92:0/1865399163 wait complete. 2023-11-28T11:22:13.882 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' removed 2023-11-28T11:22:13.901 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T11:22:14.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.103+0000 7f5bfc3ff700 1 Processor -- start 2023-11-28T11:22:14.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.103+0000 7f5bfc3ff700 1 -- start start 2023-11-28T11:22:14.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.104+0000 7f5bfc3ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5bf40b8790 0x7f5bf40cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:14.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.104+0000 7f5bfc3ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5bf406c050 0x7f5bf40d0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:14.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.104+0000 7f5bfc3ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bf40d00b0 con 0x7f5bf40cd460 2023-11-28T11:22:14.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.104+0000 7f5bfc3ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bf40b5240 con 0x7f5bf406c050 2023-11-28T11:22:14.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.104+0000 7f5bfc3ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf40b5520 con 0x7f5bf40b8790 2023-11-28T11:22:14.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.104+0000 7f5bfa99c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5bf406c050 0x7f5bf40d0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:14.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.104+0000 7f5bfa99c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5bf406c050 0x7f5bf40d0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48758/0 (socket says 172.21.15.92:48758) 2023-11-28T11:22:14.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.104+0000 7f5bfa99c700 1 -- 172.21.15.92:0/758715327 learned_addr learned my addr 172.21.15.92:0/758715327 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:14.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.105+0000 7f5bfa19b700 1 --2- 172.21.15.92:0/758715327 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5bf40b8790 0x7f5bf40cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:14.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.105+0000 7f5bfa99c700 1 -- 172.21.15.92:0/758715327 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5bf40b8790 msgr2=0x7f5bf40cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:14.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.105+0000 7f5bfa99c700 1 --2- 172.21.15.92:0/758715327 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5bf40b8790 0x7f5bf40cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:14.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.105+0000 7f5bfa99c700 1 -- 172.21.15.92:0/758715327 >> v1:172.21.15.169:6789/0 conn(0x7f5bf40cd460 legacy=0x7f5bf406b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:14.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.105+0000 7f5bfa99c700 1 -- 172.21.15.92:0/758715327 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bf40d0a80 con 0x7f5bf406c050 2023-11-28T11:22:14.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.105+0000 7f5bfa99c700 1 --2- 172.21.15.92:0/758715327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bf406c050 0x7f5bf40d0fa0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5bec009e50 tx=0x7f5bec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59681cc59ed3058b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:14.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.105+0000 7f5bf9199700 1 -- 172.21.15.92:0/758715327 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bec00e070 con 0x7f5bf406c050 2023-11-28T11:22:14.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.105+0000 7f5bf9199700 1 -- 172.21.15.92:0/758715327 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bec00b9c0 con 0x7f5bf406c050 2023-11-28T11:22:14.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.105+0000 7f5bf9199700 1 -- 172.21.15.92:0/758715327 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bec00bd70 con 0x7f5bf406c050 2023-11-28T11:22:14.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.106+0000 7f5bfc3ff700 1 -- 172.21.15.92:0/758715327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bf406c050 msgr2=0x7f5bf40d0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:14.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.106+0000 7f5bfc3ff700 1 --2- 172.21.15.92:0/758715327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bf406c050 0x7f5bf40d0fa0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5bec009e50 tx=0x7f5bec00b330 comp rx=0 tx=0).stop 2023-11-28T11:22:14.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.106+0000 7f5bfc3ff700 1 -- 172.21.15.92:0/758715327 shutdown_connections 2023-11-28T11:22:14.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.106+0000 7f5bfc3ff700 1 --2- 172.21.15.92:0/758715327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bf406c050 0x7f5bf40d0fa0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:14.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.106+0000 7f5bfc3ff700 1 --2- 172.21.15.92:0/758715327 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5bf40b8790 0x7f5bf40cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:14.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.106+0000 7f5bfc3ff700 1 -- 172.21.15.92:0/758715327 >> 172.21.15.92:0/758715327 conn(0x7f5bf40719a0 msgr2=0x7f5bf4071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:14.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.106+0000 7f5bfc3ff700 1 -- 172.21.15.92:0/758715327 shutdown_connections 2023-11-28T11:22:14.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.106+0000 7f5bfc3ff700 1 -- 172.21.15.92:0/758715327 wait complete. 2023-11-28T11:22:14.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.108+0000 7f5bfc3ff700 1 Processor -- start 2023-11-28T11:22:14.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.108+0000 7f5bfc3ff700 1 -- start start 2023-11-28T11:22:14.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.109+0000 7f5bfc3ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bf406c050 0x7f5bf40c9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:14.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.109+0000 7f5bfc3ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5bf40b8790 0x7f5bf41fa430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:14.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.109+0000 7f5bfc3ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5bf40cd460 0x7f5bf4158660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:14.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.109+0000 7f5bfc3ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf40c9760 con 0x7f5bf406c050 2023-11-28T11:22:14.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.109+0000 7f5bfc3ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf41f6540 con 0x7f5bf40b8790 2023-11-28T11:22:14.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.109+0000 7f5bfc3ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf41fa970 con 0x7f5bf40cd460 2023-11-28T11:22:14.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.109+0000 7f5bfa19b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bf406c050 0x7f5bf40c9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:14.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.109+0000 7f5bfa19b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bf406c050 0x7f5bf40c9220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48774/0 (socket says 172.21.15.92:48774) 2023-11-28T11:22:14.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.109+0000 7f5bfa19b700 1 -- 172.21.15.92:0/3914664559 learned_addr learned my addr 172.21.15.92:0/3914664559 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:14.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.109+0000 7f5bf999a700 1 --2- 172.21.15.92:0/3914664559 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5bf40b8790 0x7f5bf41fa430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:14.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.109+0000 7f5bfa99c700 1 --2- 172.21.15.92:0/3914664559 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5bf40cd460 0x7f5bf4158660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:14.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.110+0000 7f5bfa19b700 1 -- 172.21.15.92:0/3914664559 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5bf40cd460 msgr2=0x7f5bf4158660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:14.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.110+0000 7f5bfa19b700 1 --2- 172.21.15.92:0/3914664559 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5bf40cd460 0x7f5bf4158660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:14.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.110+0000 7f5bfa19b700 1 -- 172.21.15.92:0/3914664559 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5bf40b8790 msgr2=0x7f5bf41fa430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:14.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.110+0000 7f5bfa19b700 1 --2- 172.21.15.92:0/3914664559 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5bf40b8790 0x7f5bf41fa430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:14.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.110+0000 7f5bfa19b700 1 -- 172.21.15.92:0/3914664559 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bec009b30 con 0x7f5bf406c050 2023-11-28T11:22:14.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.110+0000 7f5bfa19b700 1 --2- 172.21.15.92:0/3914664559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bf406c050 0x7f5bf40c9220 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5bf000e840 tx=0x7f5bf000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:14.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.110+0000 7f5beaffd700 1 -- 172.21.15.92:0/3914664559 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf0013610 con 0x7f5bf406c050 2023-11-28T11:22:14.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.110+0000 7f5beaffd700 1 -- 172.21.15.92:0/3914664559 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bf0013770 con 0x7f5bf406c050 2023-11-28T11:22:14.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.110+0000 7f5bfc3ff700 1 -- 172.21.15.92:0/3914664559 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bf40d14e0 con 0x7f5bf406c050 2023-11-28T11:22:14.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.110+0000 7f5beaffd700 1 -- 172.21.15.92:0/3914664559 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf0013b20 con 0x7f5bf406c050 2023-11-28T11:22:14.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.110+0000 7f5bfc3ff700 1 -- 172.21.15.92:0/3914664559 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bf40b5800 con 0x7f5bf406c050 2023-11-28T11:22:14.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.111+0000 7f5bfc3ff700 1 -- 172.21.15.92:0/3914664559 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bf40d1670 con 0x7f5bf406c050 2023-11-28T11:22:14.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.112+0000 7f5beaffd700 1 -- 172.21.15.92:0/3914664559 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5bf0023070 con 0x7f5bf406c050 2023-11-28T11:22:14.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.113+0000 7f5beaffd700 1 --2- 172.21.15.92:0/3914664559 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5bd80682f0 0x7f5bd806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:14.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.113+0000 7f5beaffd700 1 -- 172.21.15.92:0/3914664559 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6519+0+0 (secure 0 0 0) 0x7f5bf0085b30 con 0x7f5bf406c050 2023-11-28T11:22:14.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.113+0000 7f5bf999a700 1 --2- 172.21.15.92:0/3914664559 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5bd80682f0 0x7f5bd806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:14.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.114+0000 7f5bf999a700 1 --2- 172.21.15.92:0/3914664559 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5bd80682f0 0x7f5bd806a7a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5be4000f60 tx=0x7f5be4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:14.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.118+0000 7f5beaffd700 1 -- 172.21.15.92:0/3914664559 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5bf00521c0 con 0x7f5bf406c050 2023-11-28T11:22:14.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.320+0000 7f5bfc3ff700 1 -- 172.21.15.92:0/3914664559 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5bf4154bd0 con 0x7f5bf406c050 2023-11-28T11:22:14.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.879+0000 7f5beaffd700 1 -- 172.21.15.92:0/3914664559 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v32) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5bf00557e0 con 0x7f5bf406c050 2023-11-28T11:22:14.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.883+0000 7f5bfc3ff700 1 -- 172.21.15.92:0/3914664559 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5bd80682f0 msgr2=0x7f5bd806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:14.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.883+0000 7f5bfc3ff700 1 --2- 172.21.15.92:0/3914664559 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5bd80682f0 0x7f5bd806a7a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5be4000f60 tx=0x7f5be4009040 comp rx=0 tx=0).stop 2023-11-28T11:22:14.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.883+0000 7f5bfc3ff700 1 -- 172.21.15.92:0/3914664559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bf406c050 msgr2=0x7f5bf40c9220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:14.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.883+0000 7f5bfc3ff700 1 --2- 172.21.15.92:0/3914664559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bf406c050 0x7f5bf40c9220 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5bf000e840 tx=0x7f5bf000ec00 comp rx=0 tx=0).stop 2023-11-28T11:22:14.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.883+0000 7f5bfc3ff700 1 -- 172.21.15.92:0/3914664559 shutdown_connections 2023-11-28T11:22:14.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.883+0000 7f5bfc3ff700 1 --2- 172.21.15.92:0/3914664559 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5bf40cd460 0x7f5bf4158660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:14.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.883+0000 7f5bfc3ff700 1 --2- 172.21.15.92:0/3914664559 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5bd80682f0 0x7f5bd806a7a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:14.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.883+0000 7f5bfc3ff700 1 --2- 172.21.15.92:0/3914664559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bf406c050 0x7f5bf40c9220 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:14.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.883+0000 7f5bfc3ff700 1 --2- 172.21.15.92:0/3914664559 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5bf40b8790 0x7f5bf41fa430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:14.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.883+0000 7f5bfc3ff700 1 -- 172.21.15.92:0/3914664559 >> 172.21.15.92:0/3914664559 conn(0x7f5bf40719a0 msgr2=0x7f5bf406b940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:14.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.884+0000 7f5bfc3ff700 1 -- 172.21.15.92:0/3914664559 shutdown_connections 2023-11-28T11:22:14.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:14.884+0000 7f5bfc3ff700 1 -- 172.21.15.92:0/3914664559 wait complete. 2023-11-28T11:22:14.886 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' removed 2023-11-28T11:22:14.905 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-11-28T11:22:15.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.106+0000 7f5bfa9ff700 1 Processor -- start 2023-11-28T11:22:15.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.106+0000 7f5bfa9ff700 1 -- start start 2023-11-28T11:22:15.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.107+0000 7f5bfa9ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5bf406a980 0x7f5bf40c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:15.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.107+0000 7f5bfa9ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5bf40ca430 0x7f5bf40cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:15.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.107+0000 7f5bfa9ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bf4159b10 con 0x7f5bf4142e80 2023-11-28T11:22:15.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.107+0000 7f5bfa9ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bf40c9380 con 0x7f5bf406a980 2023-11-28T11:22:15.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.107+0000 7f5bfa9ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf40ca890 con 0x7f5bf40ca430 2023-11-28T11:22:15.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.108+0000 7f5bf37fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5bf406a980 0x7f5bf40c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:15.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.108+0000 7f5bf37fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5bf406a980 0x7f5bf40c9ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48790/0 (socket says 172.21.15.92:48790) 2023-11-28T11:22:15.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.108+0000 7f5bf3fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f5bf4142e80 0x7f5bf406a270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56162/0 (socket says 172.21.15.92:56162) 2023-11-28T11:22:15.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.108+0000 7f5bf37fe700 1 -- 172.21.15.92:0/3121156286 learned_addr learned my addr 172.21.15.92:0/3121156286 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:15.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.108+0000 7f5bf37fe700 1 -- 172.21.15.92:0/3121156286 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5bf40ca430 msgr2=0x7f5bf40cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:15.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.108+0000 7f5bf37fe700 1 --2- 172.21.15.92:0/3121156286 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5bf40ca430 0x7f5bf40cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:15.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.108+0000 7f5bf37fe700 1 -- 172.21.15.92:0/3121156286 >> v1:172.21.15.92:6789/0 conn(0x7f5bf4142e80 legacy=0x7f5bf406a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:15.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.108+0000 7f5bf2ffd700 1 -- 172.21.15.92:0/3121156286 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3776624317 0 0) 0x7f5bf4159b10 con 0x7f5bf4142e80 2023-11-28T11:22:15.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.108+0000 7f5bf37fe700 1 -- 172.21.15.92:0/3121156286 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bf4143250 con 0x7f5bf406a980 2023-11-28T11:22:15.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.108+0000 7f5bf37fe700 1 --2- 172.21.15.92:0/3121156286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bf406a980 0x7f5bf40c9ef0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f5be800bb90 tx=0x7f5be800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdbaf8d22563499 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:15.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.109+0000 7f5bf2ffd700 1 -- 172.21.15.92:0/3121156286 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be800dd50 con 0x7f5bf406a980 2023-11-28T11:22:15.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.109+0000 7f5bf2ffd700 1 -- 172.21.15.92:0/3121156286 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5be8010470 con 0x7f5bf406a980 2023-11-28T11:22:15.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.109+0000 7f5bf2ffd700 1 -- 172.21.15.92:0/3121156286 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be8010800 con 0x7f5bf406a980 2023-11-28T11:22:15.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.109+0000 7f5bfa9ff700 1 -- 172.21.15.92:0/3121156286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bf406a980 msgr2=0x7f5bf40c9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:15.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.109+0000 7f5bfa9ff700 1 --2- 172.21.15.92:0/3121156286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bf406a980 0x7f5bf40c9ef0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f5be800bb90 tx=0x7f5be800bea0 comp rx=0 tx=0).stop 2023-11-28T11:22:15.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.109+0000 7f5bfa9ff700 1 -- 172.21.15.92:0/3121156286 shutdown_connections 2023-11-28T11:22:15.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.109+0000 7f5bfa9ff700 1 --2- 172.21.15.92:0/3121156286 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5bf40ca430 0x7f5bf40cd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:15.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.109+0000 7f5bfa9ff700 1 --2- 172.21.15.92:0/3121156286 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bf406a980 0x7f5bf40c9ef0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:15.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.109+0000 7f5bfa9ff700 1 -- 172.21.15.92:0/3121156286 >> 172.21.15.92:0/3121156286 conn(0x7f5bf40cb490 msgr2=0x7f5bf40cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:15.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.109+0000 7f5bfa9ff700 1 -- 172.21.15.92:0/3121156286 shutdown_connections 2023-11-28T11:22:15.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.109+0000 7f5bfa9ff700 1 -- 172.21.15.92:0/3121156286 wait complete. 2023-11-28T11:22:15.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.112+0000 7f5bfa9ff700 1 Processor -- start 2023-11-28T11:22:15.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.112+0000 7f5bfa9ff700 1 -- start start 2023-11-28T11:22:15.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.113+0000 7f5bfa9ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bf406a980 0x7f5bf40c4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:15.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.113+0000 7f5bfa9ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5bf40ca430 0x7f5bf41edf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:15.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.113+0000 7f5bfa9ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5bf4142e80 0x7f5bf4070910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:15.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.113+0000 7f5bfa9ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf40c5350 con 0x7f5bf406a980 2023-11-28T11:22:15.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.113+0000 7f5bfa9ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf40b0180 con 0x7f5bf40ca430 2023-11-28T11:22:15.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.113+0000 7f5bfa9ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf40af5b0 con 0x7f5bf4142e80 2023-11-28T11:22:15.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.113+0000 7f5bf3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bf406a980 0x7f5bf40c4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:15.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.113+0000 7f5bf3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bf406a980 0x7f5bf40c4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48806/0 (socket says 172.21.15.92:48806) 2023-11-28T11:22:15.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.113+0000 7f5bf3fff700 1 -- 172.21.15.92:0/175367793 learned_addr learned my addr 172.21.15.92:0/175367793 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:15.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.114+0000 7f5bf37fe700 1 --2- 172.21.15.92:0/175367793 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5bf40ca430 0x7f5bf41edf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:15.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.114+0000 7f5bf8f9c700 1 --2- 172.21.15.92:0/175367793 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5bf4142e80 0x7f5bf4070910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:15.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.114+0000 7f5bf3fff700 1 -- 172.21.15.92:0/175367793 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5bf4142e80 msgr2=0x7f5bf4070910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:15.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.114+0000 7f5bf3fff700 1 --2- 172.21.15.92:0/175367793 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5bf4142e80 0x7f5bf4070910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:15.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.114+0000 7f5bf3fff700 1 -- 172.21.15.92:0/175367793 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5bf40ca430 msgr2=0x7f5bf41edf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:15.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.114+0000 7f5bf3fff700 1 --2- 172.21.15.92:0/175367793 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5bf40ca430 0x7f5bf41edf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:15.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.114+0000 7f5bf3fff700 1 -- 172.21.15.92:0/175367793 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be800b7f0 con 0x7f5bf406a980 2023-11-28T11:22:15.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.114+0000 7f5bf37fe700 1 --2- 172.21.15.92:0/175367793 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5bf40ca430 0x7f5bf41edf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:22:15.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.114+0000 7f5bf3fff700 1 --2- 172.21.15.92:0/175367793 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bf406a980 0x7f5bf40c4e10 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5be400c550 tx=0x7f5be400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:15.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.114+0000 7f5bf0ff9700 1 -- 172.21.15.92:0/175367793 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be4004c80 con 0x7f5bf406a980 2023-11-28T11:22:15.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.114+0000 7f5bf0ff9700 1 -- 172.21.15.92:0/175367793 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5be4004e00 con 0x7f5bf406a980 2023-11-28T11:22:15.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.114+0000 7f5bfa9ff700 1 -- 172.21.15.92:0/175367793 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bf40cc5f0 con 0x7f5bf406a980 2023-11-28T11:22:15.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.114+0000 7f5bf0ff9700 1 -- 172.21.15.92:0/175367793 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be4007310 con 0x7f5bf406a980 2023-11-28T11:22:15.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.114+0000 7f5bfa9ff700 1 -- 172.21.15.92:0/175367793 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bf40cab00 con 0x7f5bf406a980 2023-11-28T11:22:15.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.115+0000 7f5bfa9ff700 1 -- 172.21.15.92:0/175367793 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bf40cc8d0 con 0x7f5bf406a980 2023-11-28T11:22:15.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.116+0000 7f5bf0ff9700 1 -- 172.21.15.92:0/175367793 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5be401a020 con 0x7f5bf406a980 2023-11-28T11:22:15.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.118+0000 7f5bf0ff9700 1 --2- 172.21.15.92:0/175367793 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5be00682f0 0x7f5be006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:15.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.118+0000 7f5bf0ff9700 1 -- 172.21.15.92:0/175367793 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6047+0+0 (secure 0 0 0) 0x7f5be4085920 con 0x7f5bf406a980 2023-11-28T11:22:15.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.118+0000 7f5bf37fe700 1 --2- 172.21.15.92:0/175367793 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5be00682f0 0x7f5be006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:15.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.118+0000 7f5bf37fe700 1 --2- 172.21.15.92:0/175367793 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5be00682f0 0x7f5be006a7a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5bf40b48e0 tx=0x7f5be8013000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:15.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.122+0000 7f5bf0ff9700 1 -- 172.21.15.92:0/175367793 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5be4052150 con 0x7f5bf406a980 2023-11-28T11:22:15.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.311+0000 7f5bfa9ff700 1 -- 172.21.15.92:0/175367793 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5bf406b400 con 0x7f5bf406a980 2023-11-28T11:22:15.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.891+0000 7f5bf0ff9700 1 -- 172.21.15.92:0/175367793 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v33) v1 ==== 178+0+0 (secure 0 0 0) 0x7f5be4055770 con 0x7f5bf406a980 2023-11-28T11:22:15.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.896+0000 7f5bfa9ff700 1 -- 172.21.15.92:0/175367793 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5be00682f0 msgr2=0x7f5be006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:15.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.896+0000 7f5bfa9ff700 1 --2- 172.21.15.92:0/175367793 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5be00682f0 0x7f5be006a7a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5bf40b48e0 tx=0x7f5be8013000 comp rx=0 tx=0).stop 2023-11-28T11:22:15.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.896+0000 7f5bfa9ff700 1 -- 172.21.15.92:0/175367793 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bf406a980 msgr2=0x7f5bf40c4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:15.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.896+0000 7f5bfa9ff700 1 --2- 172.21.15.92:0/175367793 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bf406a980 0x7f5bf40c4e10 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5be400c550 tx=0x7f5be400c910 comp rx=0 tx=0).stop 2023-11-28T11:22:15.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.896+0000 7f5bfa9ff700 1 -- 172.21.15.92:0/175367793 shutdown_connections 2023-11-28T11:22:15.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.896+0000 7f5bfa9ff700 1 --2- 172.21.15.92:0/175367793 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5bf4142e80 0x7f5bf4070910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:15.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.896+0000 7f5bfa9ff700 1 --2- 172.21.15.92:0/175367793 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5be00682f0 0x7f5be006a7a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:15.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.896+0000 7f5bfa9ff700 1 --2- 172.21.15.92:0/175367793 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bf406a980 0x7f5bf40c4e10 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:15.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.896+0000 7f5bfa9ff700 1 --2- 172.21.15.92:0/175367793 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5bf40ca430 0x7f5bf41edf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:15.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.896+0000 7f5bfa9ff700 1 -- 172.21.15.92:0/175367793 >> 172.21.15.92:0/175367793 conn(0x7f5bf40cb490 msgr2=0x7f5bf40b20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:15.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.896+0000 7f5bfa9ff700 1 -- 172.21.15.92:0/175367793 shutdown_connections 2023-11-28T11:22:15.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:15.896+0000 7f5bfa9ff700 1 -- 172.21.15.92:0/175367793 wait complete. 2023-11-28T11:22:15.899 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data_ec' removed 2023-11-28T11:22:15.928 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T11:22:15.928 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-28T11:22:15.928 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T11:22:15.929 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:22:15.929 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T11:22:15.929 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-28T11:22:15.929 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T11:22:15.929 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:22:15.929 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T11:22:15.929 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-28T11:22:15.930 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T11:22:15.930 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:22:15.930 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T11:22:15.930 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-28T11:22:15.930 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T11:22:15.930 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:22:15.946 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T11:22:15.946 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T11:22:15.948 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T11:22:15.948 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T11:22:15.949 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T11:22:15.949 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T11:22:15.950 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T11:22:15.950 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T11:22:15.991 INFO:tasks.ceph.mds.a:Started 2023-11-28T11:22:15.992 DEBUG:teuthology.parallel:result is None 2023-11-28T11:22:15.994 INFO:tasks.ceph.mds.b:Started 2023-11-28T11:22:15.994 DEBUG:teuthology.parallel:result is None 2023-11-28T11:22:15.995 INFO:tasks.ceph.mds.c:Started 2023-11-28T11:22:15.995 DEBUG:teuthology.parallel:result is None 2023-11-28T11:22:15.997 INFO:tasks.ceph.mds.d:Started 2023-11-28T11:22:15.997 DEBUG:teuthology.parallel:result is None 2023-11-28T11:22:15.997 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-28T11:22:16.142 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-11-28T11:22:16.145 INFO:tasks.ceph.mds.c.smithi092.stdout:starting mds.c at 2023-11-28T11:22:16.147 INFO:tasks.ceph.mds.d.smithi169.stdout:starting mds.d at 2023-11-28T11:22:16.147 INFO:tasks.ceph.mds.b.smithi169.stdout:starting mds.b at 2023-11-28T11:22:16.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:16.213+0000 7f36507ff700 1 Processor -- start 2023-11-28T11:22:16.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:16.213+0000 7f36507ff700 1 -- start start 2023-11-28T11:22:16.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:16.214+0000 7f36507ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f364815c710 0x7f3648160cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:16.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:16.214+0000 7f36507ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f364806a950 con 0x7f364806bdf0 2023-11-28T11:22:16.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:16.215+0000 7f36507ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f364806ffc0 con 0x7f36480cd680 2023-11-28T11:22:16.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:16.215+0000 7f36507ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3648161200 con 0x7f364815c710 2023-11-28T11:22:16.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:16.215+0000 7f364e59b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f364815c710 0x7f3648160cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:16.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:16.215+0000 7f364ed9c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f364806bdf0 0x7f364806f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56180/0 (socket says 172.21.15.92:56180) 2023-11-28T11:22:16.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:16.215+0000 7f364ed9c700 1 -- 172.21.15.92:0/3813244418 learned_addr learned my addr 172.21.15.92:0/3813244418 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:22:16.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:16.215+0000 7f364e59b700 1 -- 172.21.15.92:0/3813244418 >> v1:172.21.15.169:6789/0 conn(0x7f36480cd680 legacy=0x7f364806f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:16.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:16.215+0000 7f364e59b700 1 -- 172.21.15.92:0/3813244418 >> v1:172.21.15.92:6789/0 conn(0x7f364806bdf0 legacy=0x7f364806f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:16.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:16.215+0000 7f364e59b700 1 -- 172.21.15.92:0/3813244418 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36480b1980 con 0x7f364815c710 2023-11-28T11:22:16.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:16.215+0000 7f364d599700 1 -- 172.21.15.92:0/3813244418 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2339945039 0 0) 0x7f364806ffc0 con 0x7f36480cd680 2023-11-28T11:22:16.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:16.216+0000 7f364e59b700 1 --2- 172.21.15.92:0/3813244418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f364815c710 0x7f3648160cc0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3644009840 tx=0x7f3644009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f5b6d8e797e4a91 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:16.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:16.216+0000 7f364d599700 1 -- 172.21.15.92:0/3813244418 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3644004230 con 0x7f364815c710 2023-11-28T11:22:16.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:16.216+0000 7f364d599700 1 -- 172.21.15.92:0/3813244418 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3644004390 con 0x7f364815c710 2023-11-28T11:22:16.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:16.216+0000 7f364d599700 1 -- 172.21.15.92:0/3813244418 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36440036a0 con 0x7f364815c710 2023-11-28T11:22:16.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:16.216+0000 7f36507ff700 1 -- 172.21.15.92:0/3813244418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f364815c710 msgr2=0x7f3648160cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:16.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:16.216+0000 7f36507ff700 1 --2- 172.21.15.92:0/3813244418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f364815c710 0x7f3648160cc0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3644009840 tx=0x7f3644009b50 comp rx=0 tx=0).stop 2023-11-28T11:22:16.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:16.216+0000 7f36507ff700 1 -- 172.21.15.92:0/3813244418 shutdown_connections 2023-11-28T11:22:16.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:16.216+0000 7f36507ff700 1 --2- 172.21.15.92:0/3813244418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f364815c710 0x7f3648160cc0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:16.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:16.216+0000 7f36507ff700 1 -- 172.21.15.92:0/3813244418 >> 172.21.15.92:0/3813244418 conn(0x7f36480d07d0 msgr2=0x7f36480cc5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:16.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:16.216+0000 7f36507ff700 1 -- 172.21.15.92:0/3813244418 shutdown_connections 2023-11-28T11:22:16.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:16.217+0000 7f36507ff700 1 -- 172.21.15.92:0/3813244418 wait complete. 2023-11-28T11:22:16.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:16.218+0000 7f36507ff700 1 Processor -- start 2023-11-28T11:22:16.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:16.218+0000 7f36507ff700 1 -- start start 2023-11-28T11:22:16.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:16.219+0000 7f36507ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f364806bdf0 0x7f3648153110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:16.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:16.219+0000 7f36507ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36480cd680 0x7f364806ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:16.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:16.219+0000 7f36507ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f364815c710 0x7f364806b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:16.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:16.219+0000 7f36507ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36480b2ed0 con 0x7f36480cd680 2023-11-28T11:22:16.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:16.219+0000 7f36507ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364815a8f0 con 0x7f364806bdf0 2023-11-28T11:22:16.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:16.219+0000 7f36507ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f364815abd0 con 0x7f364815c710 2023-11-28T11:22:16.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:16.219+0000 7f364dd9a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36480cd680 0x7f364806ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:16.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:16.220+0000 7f364dd9a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36480cd680 0x7f364806ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48838/0 (socket says 172.21.15.92:48838) 2023-11-28T11:22:16.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:16.220+0000 7f364dd9a700 1 -- 172.21.15.92:0/2252454093 learned_addr learned my addr 172.21.15.92:0/2252454093 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:16.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:16.220+0000 7f364e59b700 1 --2- 172.21.15.92:0/2252454093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f364806bdf0 0x7f3648153110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:16.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:16.220+0000 7f364ed9c700 1 --2- 172.21.15.92:0/2252454093 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f364815c710 0x7f364806b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:16.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:16.220+0000 7f364dd9a700 1 -- 172.21.15.92:0/2252454093 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f364815c710 msgr2=0x7f364806b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:16.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:16.220+0000 7f364dd9a700 1 --2- 172.21.15.92:0/2252454093 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f364815c710 0x7f364806b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:16.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:16.220+0000 7f364dd9a700 1 -- 172.21.15.92:0/2252454093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f364806bdf0 msgr2=0x7f3648153110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:16.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:16.220+0000 7f364dd9a700 1 --2- 172.21.15.92:0/2252454093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f364806bdf0 0x7f3648153110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:16.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:16.220+0000 7f364dd9a700 1 -- 172.21.15.92:0/2252454093 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36440094a0 con 0x7f36480cd680 2023-11-28T11:22:16.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:16.220+0000 7f364dd9a700 1 --2- 172.21.15.92:0/2252454093 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36480cd680 0x7f364806ade0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f36380011e0 tx=0x7f363800ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:16.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:16.220+0000 7f363effd700 1 -- 172.21.15.92:0/2252454093 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36380054e0 con 0x7f36480cd680 2023-11-28T11:22:16.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:16.220+0000 7f363effd700 1 -- 172.21.15.92:0/2252454093 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3638003680 con 0x7f36480cd680 2023-11-28T11:22:16.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:16.220+0000 7f363effd700 1 -- 172.21.15.92:0/2252454093 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3638003a30 con 0x7f36480cd680 2023-11-28T11:22:16.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:16.220+0000 7f36507ff700 1 -- 172.21.15.92:0/2252454093 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36480b1f40 con 0x7f36480cd680 2023-11-28T11:22:16.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:16.221+0000 7f36507ff700 1 -- 172.21.15.92:0/2252454093 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36480b1c60 con 0x7f36480cd680 2023-11-28T11:22:16.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:16.222+0000 7f36507ff700 1 -- 172.21.15.92:0/2252454093 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f364815cae0 con 0x7f36480cd680 2023-11-28T11:22:16.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:16.222+0000 7f363effd700 1 -- 172.21.15.92:0/2252454093 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3638019030 con 0x7f36480cd680 2023-11-28T11:22:16.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:16.223+0000 7f363effd700 1 --2- 172.21.15.92:0/2252454093 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f362c0682f0 0x7f362c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:16.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:16.223+0000 7f363effd700 1 -- 172.21.15.92:0/2252454093 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f3638084dc0 con 0x7f36480cd680 2023-11-28T11:22:16.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:16.223+0000 7f364e59b700 1 --2- 172.21.15.92:0/2252454093 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f362c0682f0 0x7f362c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:16.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:16.224+0000 7f364e59b700 1 --2- 172.21.15.92:0/2252454093 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f362c0682f0 0x7f362c06a7a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f36440090b0 tx=0x7f3644008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:16.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:16.227+0000 7f363effd700 1 -- 172.21.15.92:0/2252454093 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f36380517a0 con 0x7f36480cd680 2023-11-28T11:22:16.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:16.433+0000 7f36507ff700 1 -- 172.21.15.92:0/2252454093 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f36480b3010 con 0x7f36480cd680 2023-11-28T11:22:17.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.191+0000 7f363effd700 1 -- 172.21.15.92:0/2252454093 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v38) v1 ==== 97+0+0 (secure 0 0 0) 0x7f36480b3010 con 0x7f36480cd680 2023-11-28T11:22:17.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.194+0000 7f36507ff700 1 -- 172.21.15.92:0/2252454093 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f362c0682f0 msgr2=0x7f362c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:17.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.194+0000 7f36507ff700 1 --2- 172.21.15.92:0/2252454093 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f362c0682f0 0x7f362c06a7a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f36440090b0 tx=0x7f3644008030 comp rx=0 tx=0).stop 2023-11-28T11:22:17.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.194+0000 7f36507ff700 1 -- 172.21.15.92:0/2252454093 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36480cd680 msgr2=0x7f364806ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:17.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.194+0000 7f36507ff700 1 --2- 172.21.15.92:0/2252454093 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36480cd680 0x7f364806ade0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f36380011e0 tx=0x7f363800ca30 comp rx=0 tx=0).stop 2023-11-28T11:22:17.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.194+0000 7f36507ff700 1 -- 172.21.15.92:0/2252454093 shutdown_connections 2023-11-28T11:22:17.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.194+0000 7f36507ff700 1 --2- 172.21.15.92:0/2252454093 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f364815c710 0x7f364806b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:17.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.194+0000 7f36507ff700 1 --2- 172.21.15.92:0/2252454093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f364806bdf0 0x7f3648153110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:17.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.194+0000 7f36507ff700 1 --2- 172.21.15.92:0/2252454093 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f362c0682f0 0x7f362c06a7a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:17.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.194+0000 7f36507ff700 1 --2- 172.21.15.92:0/2252454093 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36480cd680 0x7f364806ade0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:17.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.194+0000 7f36507ff700 1 -- 172.21.15.92:0/2252454093 >> 172.21.15.92:0/2252454093 conn(0x7f36480d07d0 msgr2=0x7f36480ca1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:17.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.194+0000 7f36507ff700 1 -- 172.21.15.92:0/2252454093 shutdown_connections 2023-11-28T11:22:17.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.194+0000 7f36507ff700 1 -- 172.21.15.92:0/2252454093 wait complete. 2023-11-28T11:22:17.197 INFO:teuthology.orchestra.run.smithi092.stderr: removed all blocklist entries 2023-11-28T11:22:17.215 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-28T11:22:17.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.412+0000 7f8769bff700 1 Processor -- start 2023-11-28T11:22:17.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.412+0000 7f8769bff700 1 -- start start 2023-11-28T11:22:17.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.413+0000 7f8769bff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f87640b8510 0x7f876406b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:17.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.413+0000 7f8769bff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87640b88e0 con 0x7f87640d0d60 2023-11-28T11:22:17.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.413+0000 7f8769bff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87640cd8f0 con 0x7f87640cd460 2023-11-28T11:22:17.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.413+0000 7f8769bff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87640cdbd0 con 0x7f87640b8510 2023-11-28T11:22:17.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.413+0000 7f87637fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f87640b8510 0x7f876406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:17.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.413+0000 7f8763fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f87640d0d60 0x7f87640d1200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56194/0 (socket says 172.21.15.92:56194) 2023-11-28T11:22:17.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.413+0000 7f8763fff700 1 -- 172.21.15.92:0/512078966 learned_addr learned my addr 172.21.15.92:0/512078966 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:22:17.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.414+0000 7f87637fe700 1 -- 172.21.15.92:0/512078966 >> v1:172.21.15.169:6789/0 conn(0x7f87640cd460 legacy=0x7f876406beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:17.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.414+0000 7f87637fe700 1 -- 172.21.15.92:0/512078966 >> v1:172.21.15.92:6789/0 conn(0x7f87640d0d60 legacy=0x7f87640d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:17.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.414+0000 7f87637fe700 1 -- 172.21.15.92:0/512078966 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87640cfd10 con 0x7f87640b8510 2023-11-28T11:22:17.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.414+0000 7f87627fc700 1 -- 172.21.15.92:0/512078966 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3884003142 0 0) 0x7f87640b88e0 con 0x7f87640d0d60 2023-11-28T11:22:17.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.414+0000 7f87637fe700 1 --2- 172.21.15.92:0/512078966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87640b8510 0x7f876406b970 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8754009840 tx=0x7f8754009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4ac534b0872cee8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:17.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.414+0000 7f87627fc700 1 -- 172.21.15.92:0/512078966 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8754004230 con 0x7f87640b8510 2023-11-28T11:22:17.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.414+0000 7f87627fc700 1 -- 172.21.15.92:0/512078966 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8754004390 con 0x7f87640b8510 2023-11-28T11:22:17.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.414+0000 7f87627fc700 1 -- 172.21.15.92:0/512078966 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87540036a0 con 0x7f87640b8510 2023-11-28T11:22:17.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.415+0000 7f8769bff700 1 -- 172.21.15.92:0/512078966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87640b8510 msgr2=0x7f876406b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:17.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.415+0000 7f8769bff700 1 --2- 172.21.15.92:0/512078966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87640b8510 0x7f876406b970 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8754009840 tx=0x7f8754009b50 comp rx=0 tx=0).stop 2023-11-28T11:22:17.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.415+0000 7f8769bff700 1 -- 172.21.15.92:0/512078966 shutdown_connections 2023-11-28T11:22:17.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.415+0000 7f8769bff700 1 --2- 172.21.15.92:0/512078966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87640b8510 0x7f876406b970 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:17.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.415+0000 7f8769bff700 1 -- 172.21.15.92:0/512078966 >> 172.21.15.92:0/512078966 conn(0x7f87640719a0 msgr2=0x7f8764071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:17.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.415+0000 7f8769bff700 1 -- 172.21.15.92:0/512078966 shutdown_connections 2023-11-28T11:22:17.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.415+0000 7f8769bff700 1 -- 172.21.15.92:0/512078966 wait complete. 2023-11-28T11:22:17.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.417+0000 7f8769bff700 1 Processor -- start 2023-11-28T11:22:17.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.417+0000 7f8769bff700 1 -- start start 2023-11-28T11:22:17.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.418+0000 7f8769bff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87640b8510 0x7f8764155c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:17.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.418+0000 7f8769bff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f87640cd460 0x7f87640cb810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:17.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.418+0000 7f8769bff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87640d0d60 0x7f8764159360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:17.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.418+0000 7f8769bff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87640ccea0 con 0x7f87640d0d60 2023-11-28T11:22:17.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.418+0000 7f8769bff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87640cc2d0 con 0x7f87640b8510 2023-11-28T11:22:17.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.418+0000 7f8769bff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87640cc570 con 0x7f87640cd460 2023-11-28T11:22:17.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.418+0000 7f8763fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87640d0d60 0x7f8764159360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:17.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.418+0000 7f8763fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87640d0d60 0x7f8764159360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48862/0 (socket says 172.21.15.92:48862) 2023-11-28T11:22:17.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.418+0000 7f8763fff700 1 -- 172.21.15.92:0/1154650954 learned_addr learned my addr 172.21.15.92:0/1154650954 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:17.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.418+0000 7f87637fe700 1 --2- 172.21.15.92:0/1154650954 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87640b8510 0x7f8764155c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:17.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.418+0000 7f8762ffd700 1 --2- 172.21.15.92:0/1154650954 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f87640cd460 0x7f87640cb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:17.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.418+0000 7f8763fff700 1 -- 172.21.15.92:0/1154650954 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f87640cd460 msgr2=0x7f87640cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:17.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.419+0000 7f8763fff700 1 --2- 172.21.15.92:0/1154650954 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f87640cd460 0x7f87640cb810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:17.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.419+0000 7f8763fff700 1 -- 172.21.15.92:0/1154650954 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87640b8510 msgr2=0x7f8764155c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:17.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.419+0000 7f8763fff700 1 --2- 172.21.15.92:0/1154650954 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87640b8510 0x7f8764155c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:17.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.419+0000 7f8763fff700 1 -- 172.21.15.92:0/1154650954 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87540094a0 con 0x7f87640d0d60 2023-11-28T11:22:17.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.419+0000 7f87637fe700 1 --2- 172.21.15.92:0/1154650954 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87640b8510 0x7f8764155c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:22:17.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.419+0000 7f8762ffd700 1 --2- 172.21.15.92:0/1154650954 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f87640cd460 0x7f87640cb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:22:17.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.419+0000 7f8763fff700 1 --2- 172.21.15.92:0/1154650954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87640d0d60 0x7f8764159360 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f875c00cb60 tx=0x7f875c00cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:17.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.419+0000 7f874bfff700 1 -- 172.21.15.92:0/1154650954 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875c00ab50 con 0x7f87640d0d60 2023-11-28T11:22:17.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.419+0000 7f874bfff700 1 -- 172.21.15.92:0/1154650954 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f875c00acb0 con 0x7f87640d0d60 2023-11-28T11:22:17.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.419+0000 7f8769bff700 1 -- 172.21.15.92:0/1154650954 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87640d29c0 con 0x7f87640d0d60 2023-11-28T11:22:17.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.419+0000 7f874bfff700 1 -- 172.21.15.92:0/1154650954 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875c007310 con 0x7f87640d0d60 2023-11-28T11:22:17.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.419+0000 7f8769bff700 1 -- 172.21.15.92:0/1154650954 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87640cfff0 con 0x7f87640d0d60 2023-11-28T11:22:17.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.420+0000 7f8769bff700 1 -- 172.21.15.92:0/1154650954 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87640d2b50 con 0x7f87640d0d60 2023-11-28T11:22:17.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.421+0000 7f874bfff700 1 -- 172.21.15.92:0/1154650954 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f875c01a030 con 0x7f87640d0d60 2023-11-28T11:22:17.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.422+0000 7f874bfff700 1 --2- 172.21.15.92:0/1154650954 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f87500682f0 0x7f875006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:17.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.422+0000 7f874bfff700 1 -- 172.21.15.92:0/1154650954 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f875c085880 con 0x7f87640d0d60 2023-11-28T11:22:17.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.422+0000 7f87637fe700 1 --2- 172.21.15.92:0/1154650954 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f87500682f0 0x7f875006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:17.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.423+0000 7f87637fe700 1 --2- 172.21.15.92:0/1154650954 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f87500682f0 0x7f875006a7a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8754000c00 tx=0x7f8754008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:17.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.426+0000 7f874bfff700 1 -- 172.21.15.92:0/1154650954 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f875c0521a0 con 0x7f87640d0d60 2023-11-28T11:22:17.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.688+0000 7f8769bff700 1 -- 172.21.15.92:0/1154650954 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f8764067880 con 0x7f87640d0d60 2023-11-28T11:22:17.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.688+0000 7f874bfff700 1 -- 172.21.15.92:0/1154650954 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v8) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f875c018090 con 0x7f87640d0d60 2023-11-28T11:22:17.689 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:22:17.689 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-28T11:22:17.689 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_dump": [ 2023-11-28T11:22:17.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:22:17.690 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.a", 2023-11-28T11:22:17.690 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDXzGVlH/PGCBAAor2D9Zz+LcAPQsi1BhIuQA==", 2023-11-28T11:22:17.690 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:22:17.690 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-11-28T11:22:17.690 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-11-28T11:22:17.690 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-11-28T11:22:17.690 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-28T11:22:17.690 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:22:17.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:22:17.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:22:17.690 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.b", 2023-11-28T11:22:17.690 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDXzGVlamQ0IBAAbJVb+gWu1RqSykFqIM7Hhg==", 2023-11-28T11:22:17.691 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:22:17.691 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-11-28T11:22:17.691 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-11-28T11:22:17.691 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-11-28T11:22:17.691 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-28T11:22:17.691 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:22:17.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:22:17.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:22:17.691 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.c", 2023-11-28T11:22:17.691 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDXzGVlpfloFBAAVSQxpfstP6SpWbWUMhB/hg==", 2023-11-28T11:22:17.691 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:22:17.691 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-11-28T11:22:17.692 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-11-28T11:22:17.692 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-11-28T11:22:17.692 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-28T11:22:17.692 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:22:17.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:22:17.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:22:17.692 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.d", 2023-11-28T11:22:17.692 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDXzGVlqQ8ALBAAESaHrVIhfnF5ocn44xdS5A==", 2023-11-28T11:22:17.692 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:22:17.692 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-11-28T11:22:17.692 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-11-28T11:22:17.692 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-11-28T11:22:17.692 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-28T11:22:17.693 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:22:17.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:22:17.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:22:17.693 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.0", 2023-11-28T11:22:17.693 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDbzGVl9MVvKxAA1Pd/AN+XdW47auUMbazqdw==", 2023-11-28T11:22:17.693 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:22:17.693 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-11-28T11:22:17.693 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-11-28T11:22:17.693 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-28T11:22:17.693 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:22:17.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:22:17.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:22:17.693 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.1", 2023-11-28T11:22:17.694 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDezGVldH9fLxAASVpF1+s0RIj4ObUNZoLDKg==", 2023-11-28T11:22:17.694 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:22:17.694 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-11-28T11:22:17.694 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-11-28T11:22:17.694 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-28T11:22:17.694 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:22:17.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:22:17.694 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:22:17.694 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.2", 2023-11-28T11:22:17.694 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDhzGVl+hnoMxAAXG0K3wm2UYF3BUeGjOkHhw==", 2023-11-28T11:22:17.694 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:22:17.694 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-11-28T11:22:17.695 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-11-28T11:22:17.695 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-28T11:22:17.695 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:22:17.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:22:17.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:22:17.695 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.3", 2023-11-28T11:22:17.695 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDkzGVlBB0EORAAQ3HJyXBI2AFfb/I5REmALA==", 2023-11-28T11:22:17.695 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:22:17.695 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-11-28T11:22:17.695 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-11-28T11:22:17.695 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-28T11:22:17.695 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:22:17.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:22:17.696 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:22:17.696 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.4", 2023-11-28T11:22:17.696 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDrzGVlgOjDGBAAqwSISduMc7UF8VWdiSNJXQ==", 2023-11-28T11:22:17.696 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:22:17.696 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-11-28T11:22:17.696 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-11-28T11:22:17.696 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-28T11:22:17.696 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:22:17.696 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:22:17.696 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:22:17.696 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.5", 2023-11-28T11:22:17.696 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDuzGVl9FMMHhAAohYzehTD1xpWI+/zxipY4Q==", 2023-11-28T11:22:17.696 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:22:17.697 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-11-28T11:22:17.697 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-11-28T11:22:17.697 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-28T11:22:17.697 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:22:17.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:22:17.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:22:17.697 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.6", 2023-11-28T11:22:17.697 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDxzGVl+gOMIhAA0FwY/JqNT12aDRet8sSTYw==", 2023-11-28T11:22:17.697 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:22:17.697 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-11-28T11:22:17.697 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-11-28T11:22:17.697 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-28T11:22:17.697 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:22:17.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:22:17.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:22:17.698 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.7", 2023-11-28T11:22:17.698 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQD0zGVlFPIiJhAATxpSqw+VG3u46b/IIb+yqA==", 2023-11-28T11:22:17.698 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:22:17.698 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-11-28T11:22:17.698 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-11-28T11:22:17.698 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-28T11:22:17.698 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:22:17.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:22:17.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:22:17.698 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.0", 2023-11-28T11:22:17.699 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDXzGVl8nA7NhAAGB5c8Mp6PgErGowSH4BBdw==", 2023-11-28T11:22:17.699 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:22:17.699 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-11-28T11:22:17.699 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-11-28T11:22:17.699 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-11-28T11:22:17.699 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-11-28T11:22:17.699 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:22:17.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:22:17.699 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:22:17.699 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.1", 2023-11-28T11:22:17.699 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDYzGVlbl/KCRAA23f+Q5tU6l1e3gNiomDzqQ==", 2023-11-28T11:22:17.699 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:22:17.700 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-11-28T11:22:17.700 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-11-28T11:22:17.700 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-11-28T11:22:17.700 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-11-28T11:22:17.700 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:22:17.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:22:17.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:22:17.700 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.2", 2023-11-28T11:22:17.700 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDYzGVl1oM3AhAAdrqZFbX0CRVpZ1q531/+Dg==", 2023-11-28T11:22:17.700 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:22:17.700 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-11-28T11:22:17.700 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-11-28T11:22:17.700 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-11-28T11:22:17.701 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-11-28T11:22:17.701 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:22:17.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:22:17.701 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:22:17.701 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.3", 2023-11-28T11:22:17.701 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDYzGVlGZJjERAA6iq24nyDkkyLjaWb9NkrtA==", 2023-11-28T11:22:17.701 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:22:17.701 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-11-28T11:22:17.701 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-11-28T11:22:17.701 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-11-28T11:22:17.701 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-11-28T11:22:17.701 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:22:17.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:22:17.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:22:17.702 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.admin", 2023-11-28T11:22:17.702 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDWzGVlmiaaDBAAcA6vmjgBJaurjIyTEGXdLg==", 2023-11-28T11:22:17.702 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:22:17.702 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-11-28T11:22:17.702 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-11-28T11:22:17.702 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-11-28T11:22:17.702 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-28T11:22:17.702 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:22:17.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:22:17.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:22:17.702 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.bootstrap-mds", 2023-11-28T11:22:17.703 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQD8zGVlFK6bKRAAcW4ZQ+V4f5BJfxv7/DCK/w==", 2023-11-28T11:22:17.703 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:22:17.703 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile bootstrap-mds" 2023-11-28T11:22:17.703 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:22:17.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:22:17.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:22:17.703 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.bootstrap-mgr", 2023-11-28T11:22:17.703 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQD8zGVl3LebKRAAjdVBrAZHeL3mkq0cV11OUg==", 2023-11-28T11:22:17.703 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:22:17.703 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile bootstrap-mgr" 2023-11-28T11:22:17.703 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:22:17.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:22:17.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:22:17.704 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.bootstrap-osd", 2023-11-28T11:22:17.704 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQD8zGVlbcKbKRAA3j6n3JB9OUW+t1LNPCSMHw==", 2023-11-28T11:22:17.704 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:22:17.704 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile bootstrap-osd" 2023-11-28T11:22:17.704 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:22:17.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:22:17.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:22:17.704 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.bootstrap-rbd", 2023-11-28T11:22:17.704 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQD8zGVlKMybKRAAqNUnHOX0I1veemudO5OLcA==", 2023-11-28T11:22:17.704 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:22:17.704 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile bootstrap-rbd" 2023-11-28T11:22:17.704 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:22:17.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:22:17.705 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:22:17.705 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-11-28T11:22:17.705 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQD8zGVlAtWbKRAAHWcFhWrzyXRdIxsw2j31zQ==", 2023-11-28T11:22:17.705 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:22:17.705 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-11-28T11:22:17.705 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:22:17.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:22:17.705 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:22:17.705 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.bootstrap-rgw", 2023-11-28T11:22:17.705 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQD8zGVlxd2bKRAAOf+S6aSrcLGAbEJOv4LhTg==", 2023-11-28T11:22:17.705 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:22:17.706 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile bootstrap-rgw" 2023-11-28T11:22:17.706 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:22:17.706 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:22:17.706 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:22:17.706 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.x", 2023-11-28T11:22:17.706 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDXzGVlb4AkARAAxs1XCTncmt28cyu+vL2AYA==", 2023-11-28T11:22:17.706 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:22:17.706 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-11-28T11:22:17.706 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-11-28T11:22:17.706 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-28T11:22:17.706 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:22:17.706 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:22:17.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:22:17.707 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.y", 2023-11-28T11:22:17.707 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDWzGVldLQpNBAAhdEG2ocL6T0ByZkLnwic2g==", 2023-11-28T11:22:17.707 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:22:17.707 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-11-28T11:22:17.707 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-11-28T11:22:17.707 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-28T11:22:17.707 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:22:17.707 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:22:17.707 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-28T11:22:17.707 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-28T11:22:17.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.690+0000 7f8769bff700 1 -- 172.21.15.92:0/1154650954 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f87500682f0 msgr2=0x7f875006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:17.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.690+0000 7f8769bff700 1 --2- 172.21.15.92:0/1154650954 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f87500682f0 0x7f875006a7a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8754000c00 tx=0x7f8754008030 comp rx=0 tx=0).stop 2023-11-28T11:22:17.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.690+0000 7f8769bff700 1 -- 172.21.15.92:0/1154650954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87640d0d60 msgr2=0x7f8764159360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:17.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.690+0000 7f8769bff700 1 --2- 172.21.15.92:0/1154650954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87640d0d60 0x7f8764159360 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f875c00cb60 tx=0x7f875c00cf20 comp rx=0 tx=0).stop 2023-11-28T11:22:17.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.690+0000 7f8769bff700 1 -- 172.21.15.92:0/1154650954 shutdown_connections 2023-11-28T11:22:17.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.690+0000 7f8769bff700 1 --2- 172.21.15.92:0/1154650954 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f87500682f0 0x7f875006a7a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:17.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.690+0000 7f8769bff700 1 --2- 172.21.15.92:0/1154650954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87640d0d60 0x7f8764159360 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:17.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.691+0000 7f8769bff700 1 --2- 172.21.15.92:0/1154650954 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f87640b8510 0x7f8764155c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:17.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.691+0000 7f8769bff700 1 --2- 172.21.15.92:0/1154650954 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f87640cd460 0x7f87640cb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:17.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.691+0000 7f8769bff700 1 -- 172.21.15.92:0/1154650954 >> 172.21.15.92:0/1154650954 conn(0x7f87640719a0 msgr2=0x7f87640b5d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:17.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.691+0000 7f8769bff700 1 -- 172.21.15.92:0/1154650954 shutdown_connections 2023-11-28T11:22:17.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.691+0000 7f8769bff700 1 -- 172.21.15.92:0/1154650954 wait complete. 2023-11-28T11:22:17.710 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-11-28T11:22:17.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.908+0000 7fe0895ff700 1 Processor -- start 2023-11-28T11:22:17.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.908+0000 7fe0895ff700 1 -- start start 2023-11-28T11:22:17.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.909+0000 7fe0895ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe0840b8510 0x7fe08406b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:17.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.909+0000 7fe0895ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0840cd460 0x7fe08406beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:17.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.910+0000 7fe0895ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0840b88e0 con 0x7fe08406c3f0 2023-11-28T11:22:17.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.910+0000 7fe0895ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0840cdbd0 con 0x7fe0840b8510 2023-11-28T11:22:17.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.910+0000 7fe0895ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0840d02d0 con 0x7fe0840cd460 2023-11-28T11:22:17.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.910+0000 7fe082ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe0840b8510 0x7fe08406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:17.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.910+0000 7fe082ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe0840b8510 0x7fe08406b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48874/0 (socket says 172.21.15.92:48874) 2023-11-28T11:22:17.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.910+0000 7fe082ffd700 1 -- 172.21.15.92:0/3284786819 learned_addr learned my addr 172.21.15.92:0/3284786819 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:17.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.910+0000 7fe07bfff700 1 --2- 172.21.15.92:0/3284786819 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0840cd460 0x7fe08406beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:17.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.910+0000 7fe082ffd700 1 -- 172.21.15.92:0/3284786819 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0840cd460 msgr2=0x7fe08406beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:17.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.910+0000 7fe082ffd700 1 --2- 172.21.15.92:0/3284786819 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0840cd460 0x7fe08406beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:17.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.910+0000 7fe082ffd700 1 -- 172.21.15.92:0/3284786819 >> v1:172.21.15.169:6789/0 conn(0x7fe08406c3f0 legacy=0x7fe08415cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:17.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.910+0000 7fe082ffd700 1 -- 172.21.15.92:0/3284786819 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0840cd8f0 con 0x7fe0840b8510 2023-11-28T11:22:17.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.910+0000 7fe07bfff700 1 --2- 172.21.15.92:0/3284786819 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0840cd460 0x7fe08406beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:22:17.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.911+0000 7fe082ffd700 1 --2- 172.21.15.92:0/3284786819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0840b8510 0x7fe08406b970 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe074009920 tx=0x7fe074009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a08c965c59a2850 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:17.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.911+0000 7fe0827fc700 1 -- 172.21.15.92:0/3284786819 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07400ba20 con 0x7fe0840b8510 2023-11-28T11:22:17.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.911+0000 7fe0827fc700 1 -- 172.21.15.92:0/3284786819 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe07400bb80 con 0x7fe0840b8510 2023-11-28T11:22:17.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.911+0000 7fe0827fc700 1 -- 172.21.15.92:0/3284786819 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe074009e70 con 0x7fe0840b8510 2023-11-28T11:22:17.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.911+0000 7fe0895ff700 1 -- 172.21.15.92:0/3284786819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0840b8510 msgr2=0x7fe08406b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:17.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.911+0000 7fe0895ff700 1 --2- 172.21.15.92:0/3284786819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0840b8510 0x7fe08406b970 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe074009920 tx=0x7fe074009c30 comp rx=0 tx=0).stop 2023-11-28T11:22:17.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.911+0000 7fe0895ff700 1 -- 172.21.15.92:0/3284786819 shutdown_connections 2023-11-28T11:22:17.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.911+0000 7fe0895ff700 1 --2- 172.21.15.92:0/3284786819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0840b8510 0x7fe08406b970 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:17.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.911+0000 7fe0895ff700 1 --2- 172.21.15.92:0/3284786819 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe0840cd460 0x7fe08406beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:17.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.911+0000 7fe0895ff700 1 -- 172.21.15.92:0/3284786819 >> 172.21.15.92:0/3284786819 conn(0x7fe0840719a0 msgr2=0x7fe084071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:17.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.911+0000 7fe0895ff700 1 -- 172.21.15.92:0/3284786819 shutdown_connections 2023-11-28T11:22:17.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.912+0000 7fe0895ff700 1 -- 172.21.15.92:0/3284786819 wait complete. 2023-11-28T11:22:17.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.913+0000 7fe0895ff700 1 Processor -- start 2023-11-28T11:22:17.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.913+0000 7fe0895ff700 1 -- start start 2023-11-28T11:22:17.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.914+0000 7fe0895ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe08406c3f0 0x7fe084156c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:17.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.914+0000 7fe0895ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0840b8510 0x7fe0840c81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:17.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.914+0000 7fe0895ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0840cd460 0x7fe0840c8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:17.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.914+0000 7fe0895ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe084157170 con 0x7fe0840cd460 2023-11-28T11:22:17.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.914+0000 7fe0895ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe084157d50 con 0x7fe0840b8510 2023-11-28T11:22:17.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.914+0000 7fe0895ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe084158030 con 0x7fe08406c3f0 2023-11-28T11:22:17.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.914+0000 7fe0837fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0840cd460 0x7fe0840c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:17.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.915+0000 7fe0837fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0840cd460 0x7fe0840c8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48886/0 (socket says 172.21.15.92:48886) 2023-11-28T11:22:17.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.915+0000 7fe0837fe700 1 -- 172.21.15.92:0/1965952953 learned_addr learned my addr 172.21.15.92:0/1965952953 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:17.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.915+0000 7fe082ffd700 1 --2- 172.21.15.92:0/1965952953 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe08406c3f0 0x7fe084156c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:17.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.915+0000 7fe07bfff700 1 --2- 172.21.15.92:0/1965952953 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0840b8510 0x7fe0840c81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:17.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.915+0000 7fe0837fe700 1 -- 172.21.15.92:0/1965952953 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe08406c3f0 msgr2=0x7fe084156c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:17.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.915+0000 7fe0837fe700 1 --2- 172.21.15.92:0/1965952953 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe08406c3f0 0x7fe084156c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:17.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.915+0000 7fe0837fe700 1 -- 172.21.15.92:0/1965952953 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0840b8510 msgr2=0x7fe0840c81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:17.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.915+0000 7fe0837fe700 1 --2- 172.21.15.92:0/1965952953 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0840b8510 0x7fe0840c81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:17.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.915+0000 7fe0837fe700 1 -- 172.21.15.92:0/1965952953 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe074009550 con 0x7fe0840cd460 2023-11-28T11:22:17.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.915+0000 7fe082ffd700 1 --2- 172.21.15.92:0/1965952953 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe08406c3f0 0x7fe084156c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:22:17.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.915+0000 7fe07bfff700 1 --2- 172.21.15.92:0/1965952953 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0840b8510 0x7fe0840c81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:22:17.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.915+0000 7fe0837fe700 1 --2- 172.21.15.92:0/1965952953 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0840cd460 0x7fe0840c8710 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe07c00cc50 tx=0x7fe07c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:17.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.915+0000 7fe07b7fe700 1 -- 172.21.15.92:0/1965952953 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07c00ad80 con 0x7fe0840cd460 2023-11-28T11:22:17.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.915+0000 7fe07b7fe700 1 -- 172.21.15.92:0/1965952953 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe07c00ce50 con 0x7fe0840cd460 2023-11-28T11:22:17.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.915+0000 7fe07b7fe700 1 -- 172.21.15.92:0/1965952953 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07c003c40 con 0x7fe0840cd460 2023-11-28T11:22:17.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.915+0000 7fe0895ff700 1 -- 172.21.15.92:0/1965952953 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0840cfff0 con 0x7fe0840cd460 2023-11-28T11:22:17.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.916+0000 7fe0895ff700 1 -- 172.21.15.92:0/1965952953 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0840cfd10 con 0x7fe0840cd460 2023-11-28T11:22:17.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.917+0000 7fe0895ff700 1 -- 172.21.15.92:0/1965952953 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe084067880 con 0x7fe0840cd460 2023-11-28T11:22:17.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.917+0000 7fe07b7fe700 1 -- 172.21.15.92:0/1965952953 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe07c010050 con 0x7fe0840cd460 2023-11-28T11:22:17.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.919+0000 7fe07b7fe700 1 --2- 172.21.15.92:0/1965952953 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe0640682f0 0x7fe06406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:17.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.919+0000 7fe07b7fe700 1 -- 172.21.15.92:0/1965952953 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fe07c085b00 con 0x7fe0840cd460 2023-11-28T11:22:17.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.919+0000 7fe082ffd700 1 --2- 172.21.15.92:0/1965952953 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe0640682f0 0x7fe06406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:17.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.920+0000 7fe082ffd700 1 --2- 172.21.15.92:0/1965952953 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe0640682f0 0x7fe06406a7a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe074009920 tx=0x7fe074008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:17.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:17.923+0000 7fe07b7fe700 1 -- 172.21.15.92:0/1965952953 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe07c087e60 con 0x7fe0840cd460 2023-11-28T11:22:18.070 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:22:18.075 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:22:18.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.198+0000 7fe0895ff700 1 -- 172.21.15.92:0/1965952953 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fe084157e90 con 0x7fe0840cd460 2023-11-28T11:22:18.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.201+0000 7fe07b7fe700 1 -- 172.21.15.92:0/1965952953 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7fe084157e90 con 0x7fe0840cd460 2023-11-28T11:22:18.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.203+0000 7fe0895ff700 1 -- 172.21.15.92:0/1965952953 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe0640682f0 msgr2=0x7fe06406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:18.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.203+0000 7fe0895ff700 1 --2- 172.21.15.92:0/1965952953 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe0640682f0 0x7fe06406a7a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe074009920 tx=0x7fe074008030 comp rx=0 tx=0).stop 2023-11-28T11:22:18.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.203+0000 7fe0895ff700 1 -- 172.21.15.92:0/1965952953 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0840cd460 msgr2=0x7fe0840c8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:18.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.203+0000 7fe0895ff700 1 --2- 172.21.15.92:0/1965952953 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0840cd460 0x7fe0840c8710 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe07c00cc50 tx=0x7fe07c00a340 comp rx=0 tx=0).stop 2023-11-28T11:22:18.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.204+0000 7fe0895ff700 1 -- 172.21.15.92:0/1965952953 shutdown_connections 2023-11-28T11:22:18.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.204+0000 7fe0895ff700 1 --2- 172.21.15.92:0/1965952953 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe0640682f0 0x7fe06406a7a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:18.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.204+0000 7fe0895ff700 1 --2- 172.21.15.92:0/1965952953 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0840cd460 0x7fe0840c8710 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:18.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.204+0000 7fe0895ff700 1 --2- 172.21.15.92:0/1965952953 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe08406c3f0 0x7fe084156c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:18.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.204+0000 7fe0895ff700 1 --2- 172.21.15.92:0/1965952953 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe0840b8510 0x7fe0840c81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:18.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.204+0000 7fe0895ff700 1 -- 172.21.15.92:0/1965952953 >> 172.21.15.92:0/1965952953 conn(0x7fe0840719a0 msgr2=0x7fe0840d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:18.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.204+0000 7fe0895ff700 1 -- 172.21.15.92:0/1965952953 shutdown_connections 2023-11-28T11:22:18.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.204+0000 7fe0895ff700 1 -- 172.21.15.92:0/1965952953 wait complete. 2023-11-28T11:22:18.223 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-11-28T11:22:18.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.458+0000 7fc45c7ff700 1 Processor -- start 2023-11-28T11:22:18.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.458+0000 7fc45c7ff700 1 -- start start 2023-11-28T11:22:18.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.459+0000 7fc45c7ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4540cd460 0x7fc45406c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:18.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.460+0000 7fc45c7ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4540b88e0 con 0x7fc4540d0d60 2023-11-28T11:22:18.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.460+0000 7fc45c7ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4540cd8f0 con 0x7fc4540b8510 2023-11-28T11:22:18.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.460+0000 7fc45c7ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4540cdbd0 con 0x7fc4540cd460 2023-11-28T11:22:18.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.460+0000 7fc45ad9c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc4540d0d60 0x7fc4540d11e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56204/0 (socket says 172.21.15.92:56204) 2023-11-28T11:22:18.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.460+0000 7fc45ad9c700 1 -- 172.21.15.92:0/2919706828 learned_addr learned my addr 172.21.15.92:0/2919706828 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:22:18.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.460+0000 7fc459d9a700 1 --2- 172.21.15.92:0/2919706828 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4540cd460 0x7fc45406c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:18.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.460+0000 7fc459599700 1 -- 172.21.15.92:0/2919706828 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 964992508 0 0) 0x7fc4540b88e0 con 0x7fc4540d0d60 2023-11-28T11:22:18.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.460+0000 7fc459599700 1 -- 172.21.15.92:0/2919706828 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4400035b0 con 0x7fc4540d0d60 2023-11-28T11:22:18.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.461+0000 7fc459599700 1 -- 172.21.15.92:0/2919706828 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2100321198 0 0) 0x7fc4540cd8f0 con 0x7fc4540b8510 2023-11-28T11:22:18.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.461+0000 7fc459599700 1 -- 172.21.15.92:0/2919706828 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4540b88e0 con 0x7fc4540b8510 2023-11-28T11:22:18.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.461+0000 7fc459599700 1 -- 172.21.15.92:0/2919706828 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2831736959 0 0) 0x7fc4400035b0 con 0x7fc4540d0d60 2023-11-28T11:22:18.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.461+0000 7fc459d9a700 1 -- 172.21.15.92:0/2919706828 >> v1:172.21.15.169:6789/0 conn(0x7fc4540b8510 legacy=0x7fc45406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:18.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.461+0000 7fc459d9a700 1 -- 172.21.15.92:0/2919706828 >> v1:172.21.15.92:6789/0 conn(0x7fc4540d0d60 legacy=0x7fc4540d11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:18.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.461+0000 7fc459d9a700 1 -- 172.21.15.92:0/2919706828 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4540cfd10 con 0x7fc4540cd460 2023-11-28T11:22:18.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.461+0000 7fc459d9a700 1 --2- 172.21.15.92:0/2919706828 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4540cd460 0x7fc45406c080 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc444009870 tx=0x7fc444009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c33199a2ce08946 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:18.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.461+0000 7fc459599700 1 -- 172.21.15.92:0/2919706828 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44c0019f0 con 0x7fc4540cd460 2023-11-28T11:22:18.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.461+0000 7fc459599700 1 -- 172.21.15.92:0/2919706828 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc444004030 con 0x7fc4540cd460 2023-11-28T11:22:18.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.461+0000 7fc459599700 1 -- 172.21.15.92:0/2919706828 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4440043c0 con 0x7fc4540cd460 2023-11-28T11:22:18.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.462+0000 7fc45c7ff700 1 -- 172.21.15.92:0/2919706828 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4540cd460 msgr2=0x7fc45406c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:18.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.462+0000 7fc45c7ff700 1 --2- 172.21.15.92:0/2919706828 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4540cd460 0x7fc45406c080 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc444009870 tx=0x7fc444009b80 comp rx=0 tx=0).stop 2023-11-28T11:22:18.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.462+0000 7fc45c7ff700 1 -- 172.21.15.92:0/2919706828 shutdown_connections 2023-11-28T11:22:18.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.462+0000 7fc45c7ff700 1 --2- 172.21.15.92:0/2919706828 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4540cd460 0x7fc45406c080 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:18.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.462+0000 7fc45c7ff700 1 -- 172.21.15.92:0/2919706828 >> 172.21.15.92:0/2919706828 conn(0x7fc4540719a0 msgr2=0x7fc454071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:18.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.462+0000 7fc45c7ff700 1 -- 172.21.15.92:0/2919706828 shutdown_connections 2023-11-28T11:22:18.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.462+0000 7fc45c7ff700 1 -- 172.21.15.92:0/2919706828 wait complete. 2023-11-28T11:22:18.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.463+0000 7fc45c7ff700 1 Processor -- start 2023-11-28T11:22:18.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.464+0000 7fc45c7ff700 1 -- start start 2023-11-28T11:22:18.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.465+0000 7fc45c7ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4540b8510 0x7fc45414d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:18.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.465+0000 7fc45c7ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4540cd460 0x7fc45415b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:18.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.465+0000 7fc45c7ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4540d0d60 0x7fc4541f1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:18.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.465+0000 7fc45c7ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc454157420 con 0x7fc4540b8510 2023-11-28T11:22:18.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.465+0000 7fc45c7ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc454156810 con 0x7fc4540d0d60 2023-11-28T11:22:18.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.465+0000 7fc45c7ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc454156af0 con 0x7fc4540cd460 2023-11-28T11:22:18.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.465+0000 7fc45a59b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4540b8510 0x7fc45414d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:18.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.465+0000 7fc45a59b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4540b8510 0x7fc45414d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48890/0 (socket says 172.21.15.92:48890) 2023-11-28T11:22:18.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.465+0000 7fc45a59b700 1 -- 172.21.15.92:0/374349174 learned_addr learned my addr 172.21.15.92:0/374349174 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:18.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.465+0000 7fc459d9a700 1 --2- 172.21.15.92:0/374349174 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4540cd460 0x7fc45415b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:18.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.465+0000 7fc45ad9c700 1 --2- 172.21.15.92:0/374349174 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4540d0d60 0x7fc4541f1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:18.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.465+0000 7fc45a59b700 1 -- 172.21.15.92:0/374349174 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4540cd460 msgr2=0x7fc45415b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:18.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.465+0000 7fc45a59b700 1 --2- 172.21.15.92:0/374349174 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4540cd460 0x7fc45415b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:18.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.465+0000 7fc45a59b700 1 -- 172.21.15.92:0/374349174 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4540d0d60 msgr2=0x7fc4541f1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:18.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.465+0000 7fc45a59b700 1 --2- 172.21.15.92:0/374349174 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4540d0d60 0x7fc4541f1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:18.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.465+0000 7fc45a59b700 1 -- 172.21.15.92:0/374349174 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4440094a0 con 0x7fc4540b8510 2023-11-28T11:22:18.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.465+0000 7fc45a59b700 1 --2- 172.21.15.92:0/374349174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4540b8510 0x7fc45414d120 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc45000c520 tx=0x7fc45000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:18.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.466+0000 7fc44affd700 1 -- 172.21.15.92:0/374349174 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4500052d0 con 0x7fc4540b8510 2023-11-28T11:22:18.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.466+0000 7fc44affd700 1 -- 172.21.15.92:0/374349174 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc450005430 con 0x7fc4540b8510 2023-11-28T11:22:18.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.466+0000 7fc45c7ff700 1 -- 172.21.15.92:0/374349174 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4540d29c0 con 0x7fc4540b8510 2023-11-28T11:22:18.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.466+0000 7fc44affd700 1 -- 172.21.15.92:0/374349174 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4500038c0 con 0x7fc4540b8510 2023-11-28T11:22:18.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.466+0000 7fc45c7ff700 1 -- 172.21.15.92:0/374349174 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4540cfff0 con 0x7fc4540b8510 2023-11-28T11:22:18.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.467+0000 7fc45c7ff700 1 -- 172.21.15.92:0/374349174 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4540d2b50 con 0x7fc4540b8510 2023-11-28T11:22:18.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.468+0000 7fc44affd700 1 -- 172.21.15.92:0/374349174 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc450019030 con 0x7fc4540b8510 2023-11-28T11:22:18.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.469+0000 7fc44affd700 1 --2- 172.21.15.92:0/374349174 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc4380682f0 0x7fc43806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:18.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.469+0000 7fc44affd700 1 -- 172.21.15.92:0/374349174 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fc450084fa0 con 0x7fc4540b8510 2023-11-28T11:22:18.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.469+0000 7fc459d9a700 1 --2- 172.21.15.92:0/374349174 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc4380682f0 0x7fc43806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:18.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.470+0000 7fc459d9a700 1 --2- 172.21.15.92:0/374349174 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc4380682f0 0x7fc43806a7a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc4440097c0 tx=0x7fc444008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:18.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.473+0000 7fc44affd700 1 -- 172.21.15.92:0/374349174 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc450051960 con 0x7fc4540b8510 2023-11-28T11:22:18.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.746+0000 7fc45c7ff700 1 -- 172.21.15.92:0/374349174 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7fc454067880 con 0x7fc4540b8510 2023-11-28T11:22:18.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.750+0000 7fc44affd700 1 -- 172.21.15.92:0/374349174 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7fc450054f80 con 0x7fc4540b8510 2023-11-28T11:22:18.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.752+0000 7fc45c7ff700 1 -- 172.21.15.92:0/374349174 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc4380682f0 msgr2=0x7fc43806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:18.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.752+0000 7fc45c7ff700 1 --2- 172.21.15.92:0/374349174 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc4380682f0 0x7fc43806a7a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc4440097c0 tx=0x7fc444008030 comp rx=0 tx=0).stop 2023-11-28T11:22:18.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.752+0000 7fc45c7ff700 1 -- 172.21.15.92:0/374349174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4540b8510 msgr2=0x7fc45414d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:18.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.752+0000 7fc45c7ff700 1 --2- 172.21.15.92:0/374349174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4540b8510 0x7fc45414d120 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc45000c520 tx=0x7fc45000c8e0 comp rx=0 tx=0).stop 2023-11-28T11:22:18.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.753+0000 7fc45c7ff700 1 -- 172.21.15.92:0/374349174 shutdown_connections 2023-11-28T11:22:18.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.753+0000 7fc45c7ff700 1 --2- 172.21.15.92:0/374349174 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4540d0d60 0x7fc4541f1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:18.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.753+0000 7fc45c7ff700 1 --2- 172.21.15.92:0/374349174 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc4380682f0 0x7fc43806a7a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:18.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.753+0000 7fc45c7ff700 1 --2- 172.21.15.92:0/374349174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4540b8510 0x7fc45414d120 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:18.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.753+0000 7fc45c7ff700 1 --2- 172.21.15.92:0/374349174 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4540cd460 0x7fc45415b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:18.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.753+0000 7fc45c7ff700 1 -- 172.21.15.92:0/374349174 >> 172.21.15.92:0/374349174 conn(0x7fc4540719a0 msgr2=0x7fc4540b28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:18.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.753+0000 7fc45c7ff700 1 -- 172.21.15.92:0/374349174 shutdown_connections 2023-11-28T11:22:18.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.753+0000 7fc45c7ff700 1 -- 172.21.15.92:0/374349174 wait complete. 2023-11-28T11:22:18.772 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-11-28T11:22:18.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.968+0000 7f11da5ff700 1 Processor -- start 2023-11-28T11:22:18.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.968+0000 7f11da5ff700 1 -- start start 2023-11-28T11:22:18.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.969+0000 7f11da5ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f11d40d1d70 0x7f11d4167270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:18.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.969+0000 7f11da5ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11d406c790 con 0x7f11d40cd460 2023-11-28T11:22:18.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.970+0000 7f11da5ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11d40b88e0 con 0x7f11d40b8510 2023-11-28T11:22:18.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.970+0000 7f11da5ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11d40cdbd0 con 0x7f11d40d1d70 2023-11-28T11:22:18.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.970+0000 7f11d37fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f11d40cd460 0x7f11d406c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56216/0 (socket says 172.21.15.92:56216) 2023-11-28T11:22:18.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.970+0000 7f11d37fe700 1 -- 172.21.15.92:0/294650038 learned_addr learned my addr 172.21.15.92:0/294650038 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:22:18.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.970+0000 7f11d2ffd700 1 -- 172.21.15.92:0/294650038 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3587166820 0 0) 0x7f11d406c790 con 0x7f11d40cd460 2023-11-28T11:22:18.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.970+0000 7f11d2ffd700 1 -- 172.21.15.92:0/294650038 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11bc0035b0 con 0x7f11d40cd460 2023-11-28T11:22:18.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.970+0000 7f11d8b9c700 1 --2- 172.21.15.92:0/294650038 >> v2:172.21.15.169:3300/0 conn(0x7f11d40d1d70 0x7f11d4167270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:18.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.971+0000 7f11d2ffd700 1 -- 172.21.15.92:0/294650038 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3110768333 0 0) 0x7f11bc0035b0 con 0x7f11d40cd460 2023-11-28T11:22:18.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.971+0000 7f11d2ffd700 1 -- 172.21.15.92:0/294650038 >> v2:172.21.15.169:3300/0 conn(0x7f11d40d1d70 msgr2=0x7f11d4167270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:18.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.971+0000 7f11d2ffd700 1 --2- 172.21.15.92:0/294650038 >> v2:172.21.15.169:3300/0 conn(0x7f11d40d1d70 0x7f11d4167270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:18.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.971+0000 7f11d2ffd700 1 -- 172.21.15.92:0/294650038 >> v1:172.21.15.169:6789/0 conn(0x7f11d40b8510 legacy=0x7f11d406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:18.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.971+0000 7f11d2ffd700 1 -- 172.21.15.92:0/294650038 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11d40cd8f0 con 0x7f11d40cd460 2023-11-28T11:22:18.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.971+0000 7f11d2ffd700 1 -- 172.21.15.92:0/294650038 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532161587 0 0) 0x7f11c8001670 con 0x7f11d40cd460 2023-11-28T11:22:18.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.971+0000 7f11d2ffd700 1 -- 172.21.15.92:0/294650038 >> v1:172.21.15.92:6789/0 conn(0x7f11d40cd460 legacy=0x7f11d406c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:18.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.971+0000 7f11d2ffd700 1 --2- 172.21.15.92:0/294650038 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11bc003a00 0x7f11bc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:18.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.971+0000 7f11d2ffd700 1 --2- 172.21.15.92:0/294650038 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11bc004850 0x7f11bc004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:18.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.971+0000 7f11d2ffd700 1 --2- 172.21.15.92:0/294650038 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11bc005200 0x7f11bc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:18.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.971+0000 7f11d2ffd700 1 -- 172.21.15.92:0/294650038 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d40cdbd0 con 0x7f11bc003a00 2023-11-28T11:22:18.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.971+0000 7f11d2ffd700 1 -- 172.21.15.92:0/294650038 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11bc00c080 con 0x7f11bc005200 2023-11-28T11:22:18.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.971+0000 7f11d2ffd700 1 -- 172.21.15.92:0/294650038 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11bc00c240 con 0x7f11bc004850 2023-11-28T11:22:18.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.971+0000 7f11d3fff700 1 --2- 172.21.15.92:0/294650038 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11bc003a00 0x7f11bc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:18.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.971+0000 7f11d37fe700 1 --2- 172.21.15.92:0/294650038 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11bc004850 0x7f11bc004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:18.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.971+0000 7f11d8b9c700 1 --2- 172.21.15.92:0/294650038 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11bc005200 0x7f11bc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:18.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.971+0000 7f11d3fff700 1 -- 172.21.15.92:0/294650038 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11bc004850 msgr2=0x7f11bc004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:18.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.971+0000 7f11d3fff700 1 --2- 172.21.15.92:0/294650038 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11bc004850 0x7f11bc004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:18.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.971+0000 7f11d3fff700 1 -- 172.21.15.92:0/294650038 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11bc005200 msgr2=0x7f11bc00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:18.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.971+0000 7f11d3fff700 1 --2- 172.21.15.92:0/294650038 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11bc005200 0x7f11bc00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:18.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.971+0000 7f11d3fff700 1 -- 172.21.15.92:0/294650038 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11bc0035b0 con 0x7f11bc003a00 2023-11-28T11:22:18.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.971+0000 7f11d8b9c700 1 -- 172.21.15.92:0/294650038 reap_dead start 2023-11-28T11:22:18.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.972+0000 7f11d3fff700 1 --2- 172.21.15.92:0/294650038 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11bc003a00 0x7f11bc003e10 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f11c40017c0 tx=0x7f11bc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ff3b23790a877d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:18.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.972+0000 7f11d2ffd700 1 -- 172.21.15.92:0/294650038 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c40052d0 con 0x7f11bc003a00 2023-11-28T11:22:18.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.972+0000 7f11d2ffd700 1 -- 172.21.15.92:0/294650038 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11c4005450 con 0x7f11bc003a00 2023-11-28T11:22:18.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.972+0000 7f11d2ffd700 1 -- 172.21.15.92:0/294650038 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c40038c0 con 0x7f11bc003a00 2023-11-28T11:22:18.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.972+0000 7f11da5ff700 1 -- 172.21.15.92:0/294650038 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11bc003a00 msgr2=0x7f11bc003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:18.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.972+0000 7f11da5ff700 1 --2- 172.21.15.92:0/294650038 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11bc003a00 0x7f11bc003e10 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f11c40017c0 tx=0x7f11bc005880 comp rx=0 tx=0).stop 2023-11-28T11:22:18.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.972+0000 7f11da5ff700 1 -- 172.21.15.92:0/294650038 shutdown_connections 2023-11-28T11:22:18.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.972+0000 7f11da5ff700 1 --2- 172.21.15.92:0/294650038 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11bc003a00 0x7f11bc003e10 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:18.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.973+0000 7f11da5ff700 1 -- 172.21.15.92:0/294650038 >> 172.21.15.92:0/294650038 conn(0x7f11d40719a0 msgr2=0x7f11d4071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:18.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.973+0000 7f11da5ff700 1 -- 172.21.15.92:0/294650038 shutdown_connections 2023-11-28T11:22:18.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.973+0000 7f11da5ff700 1 -- 172.21.15.92:0/294650038 wait complete. 2023-11-28T11:22:18.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.974+0000 7f11da5ff700 1 Processor -- start 2023-11-28T11:22:18.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.974+0000 7f11da5ff700 1 -- start start 2023-11-28T11:22:18.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.975+0000 7f11da5ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11bc003a00 0x7f11d414e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:18.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.975+0000 7f11da5ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11d41e76d0 0x7f11d41e7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:18.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.975+0000 7f11da5ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11d41e3470 0x7f11d41e38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:18.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.975+0000 7f11da5ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d41e8080 con 0x7f11d41e76d0 2023-11-28T11:22:18.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.975+0000 7f11da5ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d41e44a0 con 0x7f11bc003a00 2023-11-28T11:22:18.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.975+0000 7f11da5ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11d41e41c0 con 0x7f11d41e3470 2023-11-28T11:22:18.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.976+0000 7f11d37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11d41e76d0 0x7f11d41e7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:18.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.976+0000 7f11d3fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11bc003a00 0x7f11d414e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:18.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.976+0000 7f11d3fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11bc003a00 0x7f11d414e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.92:56426/0 (socket says 172.21.15.92:56426) 2023-11-28T11:22:18.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.976+0000 7f11d37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11d41e76d0 0x7f11d41e7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51082/0 (socket says 172.21.15.92:51082) 2023-11-28T11:22:18.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.976+0000 7f11d3fff700 1 -- 172.21.15.92:0/2977610932 learned_addr learned my addr 172.21.15.92:0/2977610932 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:18.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.976+0000 7f11d8b9c700 1 --2- 172.21.15.92:0/2977610932 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11d41e3470 0x7f11d41e38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:18.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.976+0000 7f11d37fe700 1 -- 172.21.15.92:0/2977610932 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11d41e3470 msgr2=0x7f11d41e38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:18.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.976+0000 7f11d37fe700 1 --2- 172.21.15.92:0/2977610932 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11d41e3470 0x7f11d41e38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:18.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.976+0000 7f11d37fe700 1 -- 172.21.15.92:0/2977610932 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11bc003a00 msgr2=0x7f11d414e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:18.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.976+0000 7f11d37fe700 1 --2- 172.21.15.92:0/2977610932 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11bc003a00 0x7f11d414e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:18.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.976+0000 7f11d37fe700 1 -- 172.21.15.92:0/2977610932 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11d40cfff0 con 0x7f11d41e76d0 2023-11-28T11:22:18.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.976+0000 7f11d37fe700 1 --2- 172.21.15.92:0/2977610932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11d41e76d0 0x7f11d41e7b40 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f11d40688a0 tx=0x7f11c800ea60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:18.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.977+0000 7f11d0ff9700 1 -- 172.21.15.92:0/2977610932 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c8014480 con 0x7f11d41e76d0 2023-11-28T11:22:18.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.977+0000 7f11d0ff9700 1 -- 172.21.15.92:0/2977610932 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11c80145e0 con 0x7f11d41e76d0 2023-11-28T11:22:18.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.977+0000 7f11d0ff9700 1 -- 172.21.15.92:0/2977610932 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c8014990 con 0x7f11d41e76d0 2023-11-28T11:22:18.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.977+0000 7f11da5ff700 1 -- 172.21.15.92:0/2977610932 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11d40cfd10 con 0x7f11d41e76d0 2023-11-28T11:22:18.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.977+0000 7f11da5ff700 1 -- 172.21.15.92:0/2977610932 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11d40d29c0 con 0x7f11d41e76d0 2023-11-28T11:22:18.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.978+0000 7f11da5ff700 1 -- 172.21.15.92:0/2977610932 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11d4067880 con 0x7f11d41e76d0 2023-11-28T11:22:18.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.979+0000 7f11d0ff9700 1 -- 172.21.15.92:0/2977610932 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f11c801b030 con 0x7f11d41e76d0 2023-11-28T11:22:18.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.980+0000 7f11d0ff9700 1 --2- 172.21.15.92:0/2977610932 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f11c00682f0 0x7f11c006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:18.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.980+0000 7f11d0ff9700 1 -- 172.21.15.92:0/2977610932 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f11c8085080 con 0x7f11d41e76d0 2023-11-28T11:22:18.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.980+0000 7f11d3fff700 1 --2- 172.21.15.92:0/2977610932 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f11c00682f0 0x7f11c006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:18.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.981+0000 7f11d3fff700 1 --2- 172.21.15.92:0/2977610932 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f11c00682f0 0x7f11c006a7a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f11c4015040 tx=0x7f11c4009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:18.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:18.984+0000 7f11d0ff9700 1 -- 172.21.15.92:0/2977610932 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f11c8051a00 con 0x7f11d41e76d0 2023-11-28T11:22:19.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.250+0000 7f11da5ff700 1 -- 172.21.15.92:0/2977610932 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f11d40d0210 con 0x7f11d41e76d0 2023-11-28T11:22:19.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.254+0000 7f11d0ff9700 1 -- 172.21.15.92:0/2977610932 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v11) v1 ==== 90+0+0 (secure 0 0 0) 0x7f11c8019020 con 0x7f11d41e76d0 2023-11-28T11:22:19.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.256+0000 7f11da5ff700 1 -- 172.21.15.92:0/2977610932 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f11c00682f0 msgr2=0x7f11c006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:19.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.256+0000 7f11da5ff700 1 --2- 172.21.15.92:0/2977610932 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f11c00682f0 0x7f11c006a7a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f11c4015040 tx=0x7f11c4009030 comp rx=0 tx=0).stop 2023-11-28T11:22:19.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.256+0000 7f11da5ff700 1 -- 172.21.15.92:0/2977610932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11d41e76d0 msgr2=0x7f11d41e7b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:19.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.256+0000 7f11da5ff700 1 --2- 172.21.15.92:0/2977610932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11d41e76d0 0x7f11d41e7b40 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f11d40688a0 tx=0x7f11c800ea60 comp rx=0 tx=0).stop 2023-11-28T11:22:19.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.256+0000 7f11da5ff700 1 -- 172.21.15.92:0/2977610932 shutdown_connections 2023-11-28T11:22:19.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.256+0000 7f11da5ff700 1 --2- 172.21.15.92:0/2977610932 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11d41e3470 0x7f11d41e38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:19.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.256+0000 7f11da5ff700 1 --2- 172.21.15.92:0/2977610932 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11bc003a00 0x7f11d414e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:19.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.256+0000 7f11da5ff700 1 --2- 172.21.15.92:0/2977610932 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f11c00682f0 0x7f11c006a7a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:19.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.256+0000 7f11da5ff700 1 --2- 172.21.15.92:0/2977610932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11d41e76d0 0x7f11d41e7b40 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:19.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.256+0000 7f11da5ff700 1 -- 172.21.15.92:0/2977610932 >> 172.21.15.92:0/2977610932 conn(0x7f11d40719a0 msgr2=0x7f11d406b970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:19.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.256+0000 7f11da5ff700 1 -- 172.21.15.92:0/2977610932 shutdown_connections 2023-11-28T11:22:19.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.256+0000 7f11da5ff700 1 -- 172.21.15.92:0/2977610932 wait complete. 2023-11-28T11:22:19.275 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-11-28T11:22:19.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.471+0000 7fe3cf1ff700 1 Processor -- start 2023-11-28T11:22:19.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.471+0000 7fe3cf1ff700 1 -- start start 2023-11-28T11:22:19.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.472+0000 7fe3cf1ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe3c80b8510 0x7fe3c806b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:19.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.472+0000 7fe3cf1ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3c80b88e0 con 0x7fe3c80cd460 2023-11-28T11:22:19.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.472+0000 7fe3cf1ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3c80cd8f0 con 0x7fe3c80d0d60 2023-11-28T11:22:19.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.472+0000 7fe3cf1ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3c80cdbd0 con 0x7fe3c80b8510 2023-11-28T11:22:19.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.473+0000 7fe3c7fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe3c80cd460 0x7fe3c806beb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33726/0 (socket says 172.21.15.92:33726) 2023-11-28T11:22:19.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.473+0000 7fe3ccf9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe3c80b8510 0x7fe3c806b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:19.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.473+0000 7fe3c7fff700 1 -- 172.21.15.92:0/4289682850 learned_addr learned my addr 172.21.15.92:0/4289682850 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:22:19.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.473+0000 7fe3c77fe700 1 -- 172.21.15.92:0/4289682850 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4258692184 0 0) 0x7fe3c80b88e0 con 0x7fe3c80cd460 2023-11-28T11:22:19.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.473+0000 7fe3c77fe700 1 -- 172.21.15.92:0/4289682850 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3b80035b0 con 0x7fe3c80cd460 2023-11-28T11:22:19.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.473+0000 7fe3ccf9b700 1 -- 172.21.15.92:0/4289682850 >> v1:172.21.15.169:6789/0 conn(0x7fe3c80d0d60 legacy=0x7fe3c80d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:19.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.473+0000 7fe3ccf9b700 1 -- 172.21.15.92:0/4289682850 >> v1:172.21.15.92:6789/0 conn(0x7fe3c80cd460 legacy=0x7fe3c806beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:19.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.473+0000 7fe3ccf9b700 1 -- 172.21.15.92:0/4289682850 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c80cfd10 con 0x7fe3c80b8510 2023-11-28T11:22:19.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.473+0000 7fe3ccf9b700 1 --2- 172.21.15.92:0/4289682850 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3c80b8510 0x7fe3c806b970 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc009840 tx=0x7fe3bc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=991830f6a20ef4fb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:19.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.473+0000 7fe3c77fe700 1 -- 172.21.15.92:0/4289682850 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc011830 con 0x7fe3c80b8510 2023-11-28T11:22:19.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.474+0000 7fe3c77fe700 1 -- 172.21.15.92:0/4289682850 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3bc011990 con 0x7fe3c80b8510 2023-11-28T11:22:19.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.474+0000 7fe3c77fe700 1 -- 172.21.15.92:0/4289682850 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc011d20 con 0x7fe3c80b8510 2023-11-28T11:22:19.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.474+0000 7fe3cf1ff700 1 -- 172.21.15.92:0/4289682850 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3c80b8510 msgr2=0x7fe3c806b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:19.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.474+0000 7fe3cf1ff700 1 --2- 172.21.15.92:0/4289682850 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3c80b8510 0x7fe3c806b970 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc009840 tx=0x7fe3bc009b50 comp rx=0 tx=0).stop 2023-11-28T11:22:19.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.474+0000 7fe3cf1ff700 1 -- 172.21.15.92:0/4289682850 shutdown_connections 2023-11-28T11:22:19.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.474+0000 7fe3cf1ff700 1 --2- 172.21.15.92:0/4289682850 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3c80b8510 0x7fe3c806b970 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:19.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.474+0000 7fe3cf1ff700 1 -- 172.21.15.92:0/4289682850 >> 172.21.15.92:0/4289682850 conn(0x7fe3c80719a0 msgr2=0x7fe3c8071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:19.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.474+0000 7fe3cf1ff700 1 -- 172.21.15.92:0/4289682850 shutdown_connections 2023-11-28T11:22:19.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.474+0000 7fe3cf1ff700 1 -- 172.21.15.92:0/4289682850 wait complete. 2023-11-28T11:22:19.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.476+0000 7fe3cf1ff700 1 Processor -- start 2023-11-28T11:22:19.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.477+0000 7fe3cf1ff700 1 -- start start 2023-11-28T11:22:19.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.478+0000 7fe3cf1ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3c80b8510 0x7fe3c81f25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:19.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.478+0000 7fe3cf1ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3c80cd460 0x7fe3c80b3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:19.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.478+0000 7fe3cf1ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3c80d0d60 0x7fe3c80b3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:19.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.478+0000 7fe3cf1ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c81f2fc0 con 0x7fe3c80d0d60 2023-11-28T11:22:19.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.478+0000 7fe3cf1ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c81f32a0 con 0x7fe3c80b8510 2023-11-28T11:22:19.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.478+0000 7fe3cf1ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c81f3580 con 0x7fe3c80cd460 2023-11-28T11:22:19.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.478+0000 7fe3cd79c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3c80d0d60 0x7fe3c80b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:19.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.478+0000 7fe3cd79c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3c80d0d60 0x7fe3c80b3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51100/0 (socket says 172.21.15.92:51100) 2023-11-28T11:22:19.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.478+0000 7fe3cd79c700 1 -- 172.21.15.92:0/1550359559 learned_addr learned my addr 172.21.15.92:0/1550359559 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:19.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.478+0000 7fe3ccf9b700 1 --2- 172.21.15.92:0/1550359559 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3c80b8510 0x7fe3c81f25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:19.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.478+0000 7fe3c7fff700 1 --2- 172.21.15.92:0/1550359559 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3c80cd460 0x7fe3c80b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:19.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.479+0000 7fe3cd79c700 1 -- 172.21.15.92:0/1550359559 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3c80cd460 msgr2=0x7fe3c80b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:19.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.479+0000 7fe3cd79c700 1 --2- 172.21.15.92:0/1550359559 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3c80cd460 0x7fe3c80b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:19.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.479+0000 7fe3cd79c700 1 -- 172.21.15.92:0/1550359559 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3c80b8510 msgr2=0x7fe3c81f25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:19.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.479+0000 7fe3cd79c700 1 --2- 172.21.15.92:0/1550359559 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3c80b8510 0x7fe3c81f25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:19.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.479+0000 7fe3cd79c700 1 -- 172.21.15.92:0/1550359559 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3bc0094a0 con 0x7fe3c80d0d60 2023-11-28T11:22:19.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.479+0000 7fe3cd79c700 1 --2- 172.21.15.92:0/1550359559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3c80d0d60 0x7fe3c80b3950 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe3c000cb50 tx=0x7fe3c000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:19.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.479+0000 7fe3c57fa700 1 -- 172.21.15.92:0/1550359559 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c0003c00 con 0x7fe3c80d0d60 2023-11-28T11:22:19.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.479+0000 7fe3c57fa700 1 -- 172.21.15.92:0/1550359559 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3c0003d60 con 0x7fe3c80d0d60 2023-11-28T11:22:19.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.479+0000 7fe3c57fa700 1 -- 172.21.15.92:0/1550359559 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c000aa20 con 0x7fe3c80d0d60 2023-11-28T11:22:19.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.479+0000 7fe3cf1ff700 1 -- 172.21.15.92:0/1550359559 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3c80d29c0 con 0x7fe3c80d0d60 2023-11-28T11:22:19.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.479+0000 7fe3cf1ff700 1 -- 172.21.15.92:0/1550359559 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3c80cfff0 con 0x7fe3c80d0d60 2023-11-28T11:22:19.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.480+0000 7fe3cf1ff700 1 -- 172.21.15.92:0/1550359559 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3c80d2b50 con 0x7fe3c80d0d60 2023-11-28T11:22:19.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.481+0000 7fe3c57fa700 1 -- 172.21.15.92:0/1550359559 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe3c0010050 con 0x7fe3c80d0d60 2023-11-28T11:22:19.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.482+0000 7fe3c57fa700 1 --2- 172.21.15.92:0/1550359559 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe3ac0682f0 0x7fe3ac06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:19.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.482+0000 7fe3c57fa700 1 -- 172.21.15.92:0/1550359559 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fe3c0085a00 con 0x7fe3c80d0d60 2023-11-28T11:22:19.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.483+0000 7fe3ccf9b700 1 --2- 172.21.15.92:0/1550359559 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe3ac0682f0 0x7fe3ac06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:19.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.483+0000 7fe3ccf9b700 1 --2- 172.21.15.92:0/1550359559 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe3ac0682f0 0x7fe3ac06a7a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc0090b0 tx=0x7fe3bc008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:19.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.486+0000 7fe3c57fa700 1 -- 172.21.15.92:0/1550359559 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe3c0052380 con 0x7fe3c80d0d60 2023-11-28T11:22:19.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.754+0000 7fe3cf1ff700 1 -- 172.21.15.92:0/1550359559 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fe3c8067880 con 0x7fe3c80d0d60 2023-11-28T11:22:19.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.758+0000 7fe3c57fa700 1 -- 172.21.15.92:0/1550359559 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7fe3c00559a0 con 0x7fe3c80d0d60 2023-11-28T11:22:19.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.760+0000 7fe3cf1ff700 1 -- 172.21.15.92:0/1550359559 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe3ac0682f0 msgr2=0x7fe3ac06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:19.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.760+0000 7fe3cf1ff700 1 --2- 172.21.15.92:0/1550359559 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe3ac0682f0 0x7fe3ac06a7a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc0090b0 tx=0x7fe3bc008030 comp rx=0 tx=0).stop 2023-11-28T11:22:19.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.760+0000 7fe3cf1ff700 1 -- 172.21.15.92:0/1550359559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3c80d0d60 msgr2=0x7fe3c80b3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:19.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.760+0000 7fe3cf1ff700 1 --2- 172.21.15.92:0/1550359559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3c80d0d60 0x7fe3c80b3950 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe3c000cb50 tx=0x7fe3c000ce60 comp rx=0 tx=0).stop 2023-11-28T11:22:19.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.760+0000 7fe3cf1ff700 1 -- 172.21.15.92:0/1550359559 shutdown_connections 2023-11-28T11:22:19.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.760+0000 7fe3cf1ff700 1 --2- 172.21.15.92:0/1550359559 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe3ac0682f0 0x7fe3ac06a7a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:19.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.760+0000 7fe3cf1ff700 1 --2- 172.21.15.92:0/1550359559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3c80d0d60 0x7fe3c80b3950 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:19.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.760+0000 7fe3cf1ff700 1 --2- 172.21.15.92:0/1550359559 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe3c80b8510 0x7fe3c81f25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:19.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.760+0000 7fe3cf1ff700 1 --2- 172.21.15.92:0/1550359559 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe3c80cd460 0x7fe3c80b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:19.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.760+0000 7fe3cf1ff700 1 -- 172.21.15.92:0/1550359559 >> 172.21.15.92:0/1550359559 conn(0x7fe3c80719a0 msgr2=0x7fe3c80b6c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:19.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.760+0000 7fe3cf1ff700 1 -- 172.21.15.92:0/1550359559 shutdown_connections 2023-11-28T11:22:19.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.760+0000 7fe3cf1ff700 1 -- 172.21.15.92:0/1550359559 wait complete. 2023-11-28T11:22:19.779 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-11-28T11:22:19.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.975+0000 7ff2541ff700 1 Processor -- start 2023-11-28T11:22:19.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.975+0000 7ff2541ff700 1 -- start start 2023-11-28T11:22:19.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.976+0000 7ff2541ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff24c0cd460 0x7ff24c06c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:19.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.976+0000 7ff2541ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff24c0d0d60 0x7ff24c0d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:19.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.976+0000 7ff2541ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff24c0b88e0 con 0x7ff24c0b8510 2023-11-28T11:22:19.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.976+0000 7ff2541ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff24c0cdbd0 con 0x7ff24c0cd460 2023-11-28T11:22:19.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.976+0000 7ff2541ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff24c0d02d0 con 0x7ff24c0d0d60 2023-11-28T11:22:19.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.976+0000 7ff251f9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff24c0b8510 0x7ff24c06b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33738/0 (socket says 172.21.15.92:33738) 2023-11-28T11:22:19.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.976+0000 7ff25179a700 1 --2- 172.21.15.92:0/3202759127 >> v2:172.21.15.92:3300/0 conn(0x7ff24c0cd460 0x7ff24c06c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:19.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.976+0000 7ff251f9b700 1 -- 172.21.15.92:0/3202759127 learned_addr learned my addr 172.21.15.92:0/3202759127 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:22:19.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.977+0000 7ff25279c700 1 --2- 172.21.15.92:0/3202759127 >> v2:172.21.15.169:3300/0 conn(0x7ff24c0d0d60 0x7ff24c0d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:19.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.977+0000 7ff25179a700 1 -- 172.21.15.92:0/3202759127 >> v2:172.21.15.169:3300/0 conn(0x7ff24c0d0d60 msgr2=0x7ff24c0d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:19.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.977+0000 7ff25179a700 1 --2- 172.21.15.92:0/3202759127 >> v2:172.21.15.169:3300/0 conn(0x7ff24c0d0d60 0x7ff24c0d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:19.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.977+0000 7ff25179a700 1 -- 172.21.15.92:0/3202759127 >> v1:172.21.15.92:6789/0 conn(0x7ff24c0b8510 legacy=0x7ff24c06b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:19.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.977+0000 7ff25179a700 1 -- 172.21.15.92:0/3202759127 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff24c0cd8f0 con 0x7ff24c0cd460 2023-11-28T11:22:19.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.977+0000 7ff25179a700 1 --2- 172.21.15.92:0/3202759127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff24c0cd460 0x7ff24c06c080 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff24c0ce8c0 tx=0x7ff23c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b227264707ea2974 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:19.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.977+0000 7ff250f99700 1 -- 172.21.15.92:0/3202759127 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23c011830 con 0x7ff24c0cd460 2023-11-28T11:22:19.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.977+0000 7ff250f99700 1 -- 172.21.15.92:0/3202759127 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff23c011990 con 0x7ff24c0cd460 2023-11-28T11:22:19.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.977+0000 7ff250f99700 1 -- 172.21.15.92:0/3202759127 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23c011d20 con 0x7ff24c0cd460 2023-11-28T11:22:19.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.978+0000 7ff2541ff700 1 -- 172.21.15.92:0/3202759127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff24c0cd460 msgr2=0x7ff24c06c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:19.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.978+0000 7ff2541ff700 1 --2- 172.21.15.92:0/3202759127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff24c0cd460 0x7ff24c06c080 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff24c0ce8c0 tx=0x7ff23c009b50 comp rx=0 tx=0).stop 2023-11-28T11:22:19.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.978+0000 7ff2541ff700 1 -- 172.21.15.92:0/3202759127 shutdown_connections 2023-11-28T11:22:19.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.978+0000 7ff2541ff700 1 --2- 172.21.15.92:0/3202759127 >> v2:172.21.15.169:3300/0 conn(0x7ff24c0d0d60 0x7ff24c0d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:19.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.978+0000 7ff2541ff700 1 --2- 172.21.15.92:0/3202759127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff24c0cd460 0x7ff24c06c080 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:19.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.978+0000 7ff2541ff700 1 -- 172.21.15.92:0/3202759127 >> 172.21.15.92:0/3202759127 conn(0x7ff24c0719a0 msgr2=0x7ff24c071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:19.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.978+0000 7ff2541ff700 1 -- 172.21.15.92:0/3202759127 shutdown_connections 2023-11-28T11:22:19.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.978+0000 7ff2541ff700 1 -- 172.21.15.92:0/3202759127 wait complete. 2023-11-28T11:22:19.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.980+0000 7ff2541ff700 1 Processor -- start 2023-11-28T11:22:19.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.980+0000 7ff2541ff700 1 -- start start 2023-11-28T11:22:19.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.981+0000 7ff2541ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff24c0b8510 0x7ff24c1e1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:19.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.981+0000 7ff2541ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff24c0cd460 0x7ff24c1e4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:19.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.981+0000 7ff2541ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff24c0d0d60 0x7ff24c14de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:19.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.981+0000 7ff2541ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff24c1e5120 con 0x7ff24c0b8510 2023-11-28T11:22:19.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.981+0000 7ff2541ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff24c1e2320 con 0x7ff24c0d0d60 2023-11-28T11:22:19.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.981+0000 7ff2541ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff24c1e2600 con 0x7ff24c0cd460 2023-11-28T11:22:19.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.981+0000 7ff251f9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff24c0b8510 0x7ff24c1e1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:19.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.981+0000 7ff251f9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff24c0b8510 0x7ff24c1e1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51116/0 (socket says 172.21.15.92:51116) 2023-11-28T11:22:19.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.981+0000 7ff251f9b700 1 -- 172.21.15.92:0/1724649233 learned_addr learned my addr 172.21.15.92:0/1724649233 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:19.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.981+0000 7ff25279c700 1 --2- 172.21.15.92:0/1724649233 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff24c0d0d60 0x7ff24c14de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:19.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.981+0000 7ff25179a700 1 --2- 172.21.15.92:0/1724649233 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff24c0cd460 0x7ff24c1e4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:19.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.982+0000 7ff251f9b700 1 -- 172.21.15.92:0/1724649233 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff24c0cd460 msgr2=0x7ff24c1e4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:19.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.982+0000 7ff251f9b700 1 --2- 172.21.15.92:0/1724649233 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff24c0cd460 0x7ff24c1e4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:19.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.982+0000 7ff251f9b700 1 -- 172.21.15.92:0/1724649233 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff24c0d0d60 msgr2=0x7ff24c14de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:19.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.982+0000 7ff251f9b700 1 --2- 172.21.15.92:0/1724649233 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff24c0d0d60 0x7ff24c14de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:19.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.982+0000 7ff251f9b700 1 -- 172.21.15.92:0/1724649233 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff23c0094a0 con 0x7ff24c0b8510 2023-11-28T11:22:19.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.982+0000 7ff25179a700 1 --2- 172.21.15.92:0/1724649233 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff24c0cd460 0x7ff24c1e4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:22:19.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.982+0000 7ff251f9b700 1 --2- 172.21.15.92:0/1724649233 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff24c0b8510 0x7ff24c1e1ad0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff24800c520 tx=0x7ff24800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:19.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.982+0000 7ff2427fc700 1 -- 172.21.15.92:0/1724649233 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff248005d70 con 0x7ff24c0b8510 2023-11-28T11:22:19.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.982+0000 7ff2427fc700 1 -- 172.21.15.92:0/1724649233 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2480070d0 con 0x7ff24c0b8510 2023-11-28T11:22:19.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.982+0000 7ff2427fc700 1 -- 172.21.15.92:0/1724649233 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff248007440 con 0x7ff24c0b8510 2023-11-28T11:22:19.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.982+0000 7ff2541ff700 1 -- 172.21.15.92:0/1724649233 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff24c0cfff0 con 0x7ff24c0b8510 2023-11-28T11:22:19.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.982+0000 7ff2541ff700 1 -- 172.21.15.92:0/1724649233 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff24c0cfd10 con 0x7ff24c0b8510 2023-11-28T11:22:19.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.983+0000 7ff2541ff700 1 -- 172.21.15.92:0/1724649233 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff24c067880 con 0x7ff24c0b8510 2023-11-28T11:22:19.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.984+0000 7ff2427fc700 1 -- 172.21.15.92:0/1724649233 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff248019030 con 0x7ff24c0b8510 2023-11-28T11:22:19.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.985+0000 7ff2427fc700 1 --2- 172.21.15.92:0/1724649233 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff2300682f0 0x7ff23006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:19.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.985+0000 7ff2427fc700 1 -- 172.21.15.92:0/1724649233 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7ff248084da0 con 0x7ff24c0b8510 2023-11-28T11:22:19.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.985+0000 7ff25179a700 1 --2- 172.21.15.92:0/1724649233 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff2300682f0 0x7ff23006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:19.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.986+0000 7ff25179a700 1 --2- 172.21.15.92:0/1724649233 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff2300682f0 0x7ff23006a7a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff24c0b1de0 tx=0x7ff23c01c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:19.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:19.989+0000 7ff2427fc700 1 -- 172.21.15.92:0/1724649233 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff24804b5e0 con 0x7ff24c0b8510 2023-11-28T11:22:20.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.258+0000 7ff2541ff700 1 -- 172.21.15.92:0/1724649233 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7ff24c0d2940 con 0x7ff24c0b8510 2023-11-28T11:22:20.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.262+0000 7ff2427fc700 1 -- 172.21.15.92:0/1724649233 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v13) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff2480170a0 con 0x7ff24c0b8510 2023-11-28T11:22:20.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.264+0000 7ff2541ff700 1 -- 172.21.15.92:0/1724649233 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff2300682f0 msgr2=0x7ff23006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:20.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.264+0000 7ff2541ff700 1 --2- 172.21.15.92:0/1724649233 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff2300682f0 0x7ff23006a7a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff24c0b1de0 tx=0x7ff23c01c040 comp rx=0 tx=0).stop 2023-11-28T11:22:20.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.264+0000 7ff2541ff700 1 -- 172.21.15.92:0/1724649233 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff24c0b8510 msgr2=0x7ff24c1e1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:20.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.264+0000 7ff2541ff700 1 --2- 172.21.15.92:0/1724649233 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff24c0b8510 0x7ff24c1e1ad0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff24800c520 tx=0x7ff24800c830 comp rx=0 tx=0).stop 2023-11-28T11:22:20.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.264+0000 7ff2541ff700 1 -- 172.21.15.92:0/1724649233 shutdown_connections 2023-11-28T11:22:20.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.264+0000 7ff2541ff700 1 --2- 172.21.15.92:0/1724649233 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff24c0d0d60 0x7ff24c14de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:20.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.264+0000 7ff2541ff700 1 --2- 172.21.15.92:0/1724649233 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff2300682f0 0x7ff23006a7a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:20.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.264+0000 7ff2541ff700 1 --2- 172.21.15.92:0/1724649233 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff24c0b8510 0x7ff24c1e1ad0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:20.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.264+0000 7ff2541ff700 1 --2- 172.21.15.92:0/1724649233 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff24c0cd460 0x7ff24c1e4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:20.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.264+0000 7ff2541ff700 1 -- 172.21.15.92:0/1724649233 >> 172.21.15.92:0/1724649233 conn(0x7ff24c0719a0 msgr2=0x7ff24c0b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:20.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.264+0000 7ff2541ff700 1 -- 172.21.15.92:0/1724649233 shutdown_connections 2023-11-28T11:22:20.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.264+0000 7ff2541ff700 1 -- 172.21.15.92:0/1724649233 wait complete. 2023-11-28T11:22:20.282 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-11-28T11:22:20.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.479+0000 7f248e9ff700 1 Processor -- start 2023-11-28T11:22:20.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.479+0000 7f248e9ff700 1 -- start start 2023-11-28T11:22:20.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.480+0000 7f248e9ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f24880cd460 0x7f248806c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:20.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.480+0000 7f248e9ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24880d0d60 0x7f24880d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:20.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.480+0000 7f248e9ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24880b88e0 con 0x7f24880b8510 2023-11-28T11:22:20.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.480+0000 7f248e9ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24880cdbd0 con 0x7f24880cd460 2023-11-28T11:22:20.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.480+0000 7f248e9ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24880d02d0 con 0x7f24880d0d60 2023-11-28T11:22:20.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.480+0000 7f24877fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f24880cd460 0x7f248806c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:20.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.480+0000 7f2487fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f24880b8510 0x7f248806b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33754/0 (socket says 172.21.15.92:33754) 2023-11-28T11:22:20.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.480+0000 7f2487fff700 1 -- 172.21.15.92:0/2905784784 learned_addr learned my addr 172.21.15.92:0/2905784784 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:22:20.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.481+0000 7f24877fe700 1 -- 172.21.15.92:0/2905784784 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24880d0d60 msgr2=0x7f24880d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:20.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.481+0000 7f248cf9c700 1 --2- 172.21.15.92:0/2905784784 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24880d0d60 0x7f24880d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:20.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.481+0000 7f24877fe700 1 --2- 172.21.15.92:0/2905784784 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24880d0d60 0x7f24880d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:20.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.481+0000 7f24877fe700 1 -- 172.21.15.92:0/2905784784 >> v1:172.21.15.92:6789/0 conn(0x7f24880b8510 legacy=0x7f248806b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:20.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.481+0000 7f2486ffd700 1 -- 172.21.15.92:0/2905784784 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 330611162 0 0) 0x7f24880b88e0 con 0x7f24880b8510 2023-11-28T11:22:20.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.481+0000 7f24877fe700 1 -- 172.21.15.92:0/2905784784 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24880cd8f0 con 0x7f24880cd460 2023-11-28T11:22:20.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.481+0000 7f248cf9c700 1 --2- 172.21.15.92:0/2905784784 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24880d0d60 0x7f24880d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:22:20.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.481+0000 7f24877fe700 1 --2- 172.21.15.92:0/2905784784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24880cd460 0x7f248806c080 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f247c009870 tx=0x7f247c009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5664332b3f276c4a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:20.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.482+0000 7f2486ffd700 1 -- 172.21.15.92:0/2905784784 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247c011830 con 0x7f24880cd460 2023-11-28T11:22:20.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.482+0000 7f2486ffd700 1 -- 172.21.15.92:0/2905784784 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f247c011990 con 0x7f24880cd460 2023-11-28T11:22:20.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.482+0000 7f2486ffd700 1 -- 172.21.15.92:0/2905784784 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247c011d20 con 0x7f24880cd460 2023-11-28T11:22:20.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.482+0000 7f248e9ff700 1 -- 172.21.15.92:0/2905784784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24880cd460 msgr2=0x7f248806c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:20.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.482+0000 7f248e9ff700 1 --2- 172.21.15.92:0/2905784784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24880cd460 0x7f248806c080 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f247c009870 tx=0x7f247c009b80 comp rx=0 tx=0).stop 2023-11-28T11:22:20.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.482+0000 7f248e9ff700 1 -- 172.21.15.92:0/2905784784 shutdown_connections 2023-11-28T11:22:20.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.482+0000 7f248e9ff700 1 --2- 172.21.15.92:0/2905784784 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24880d0d60 0x7f24880d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:20.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.482+0000 7f248e9ff700 1 --2- 172.21.15.92:0/2905784784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24880cd460 0x7f248806c080 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:20.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.482+0000 7f248e9ff700 1 -- 172.21.15.92:0/2905784784 >> 172.21.15.92:0/2905784784 conn(0x7f24880719a0 msgr2=0x7f2488071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:20.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.482+0000 7f248e9ff700 1 -- 172.21.15.92:0/2905784784 shutdown_connections 2023-11-28T11:22:20.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.482+0000 7f248e9ff700 1 -- 172.21.15.92:0/2905784784 wait complete. 2023-11-28T11:22:20.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.484+0000 7f248e9ff700 1 Processor -- start 2023-11-28T11:22:20.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.484+0000 7f248e9ff700 1 -- start start 2023-11-28T11:22:20.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.485+0000 7f248e9ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24880b8510 0x7f24881e5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:20.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.485+0000 7f248e9ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24880cd460 0x7f24880a9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:20.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.485+0000 7f248e9ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24880d0d60 0x7f248814af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:20.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.485+0000 7f248e9ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24880a9d80 con 0x7f24880cd460 2023-11-28T11:22:20.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.485+0000 7f248e9ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24881e5cc0 con 0x7f24880b8510 2023-11-28T11:22:20.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.485+0000 7f248e9ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24881e5fa0 con 0x7f24880d0d60 2023-11-28T11:22:20.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.485+0000 7f24877fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24880cd460 0x7f24880a9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:20.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.485+0000 7f24877fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24880cd460 0x7f24880a9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51128/0 (socket says 172.21.15.92:51128) 2023-11-28T11:22:20.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.485+0000 7f24877fe700 1 -- 172.21.15.92:0/784412481 learned_addr learned my addr 172.21.15.92:0/784412481 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:20.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.486+0000 7f2487fff700 1 --2- 172.21.15.92:0/784412481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24880b8510 0x7f24881e5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:20.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.486+0000 7f248cf9c700 1 --2- 172.21.15.92:0/784412481 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24880d0d60 0x7f248814af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:20.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.486+0000 7f24877fe700 1 -- 172.21.15.92:0/784412481 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24880d0d60 msgr2=0x7f248814af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:20.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.486+0000 7f24877fe700 1 --2- 172.21.15.92:0/784412481 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24880d0d60 0x7f248814af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:20.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.486+0000 7f24877fe700 1 -- 172.21.15.92:0/784412481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24880b8510 msgr2=0x7f24881e5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:20.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.486+0000 7f24877fe700 1 --2- 172.21.15.92:0/784412481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24880b8510 0x7f24881e5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:20.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.486+0000 7f24877fe700 1 -- 172.21.15.92:0/784412481 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f247c0094a0 con 0x7f24880cd460 2023-11-28T11:22:20.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.486+0000 7f24877fe700 1 --2- 172.21.15.92:0/784412481 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24880cd460 0x7f24880a9840 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f247c0090b0 tx=0x7f247c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:20.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.486+0000 7f2484ff9700 1 -- 172.21.15.92:0/784412481 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247c0038a0 con 0x7f24880cd460 2023-11-28T11:22:20.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.486+0000 7f2484ff9700 1 -- 172.21.15.92:0/784412481 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f247c01f070 con 0x7f24880cd460 2023-11-28T11:22:20.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.486+0000 7f2484ff9700 1 -- 172.21.15.92:0/784412481 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247c003c60 con 0x7f24880cd460 2023-11-28T11:22:20.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.486+0000 7f248e9ff700 1 -- 172.21.15.92:0/784412481 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24880cfff0 con 0x7f24880cd460 2023-11-28T11:22:20.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.486+0000 7f248e9ff700 1 -- 172.21.15.92:0/784412481 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24880cfd10 con 0x7f24880cd460 2023-11-28T11:22:20.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.487+0000 7f248e9ff700 1 -- 172.21.15.92:0/784412481 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2488067880 con 0x7f24880cd460 2023-11-28T11:22:20.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.488+0000 7f2484ff9700 1 -- 172.21.15.92:0/784412481 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f247c00f050 con 0x7f24880cd460 2023-11-28T11:22:20.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.489+0000 7f2484ff9700 1 --2- 172.21.15.92:0/784412481 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f24740682f0 0x7f247406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:20.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.489+0000 7f2484ff9700 1 -- 172.21.15.92:0/784412481 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f247c021070 con 0x7f24880cd460 2023-11-28T11:22:20.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.489+0000 7f2487fff700 1 --2- 172.21.15.92:0/784412481 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f24740682f0 0x7f247406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:20.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.490+0000 7f2487fff700 1 --2- 172.21.15.92:0/784412481 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f24740682f0 0x7f247406a7a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2478000f60 tx=0x7f2478007be0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:20.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.493+0000 7f2484ff9700 1 -- 172.21.15.92:0/784412481 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f247c054050 con 0x7f24880cd460 2023-11-28T11:22:20.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.760+0000 7f248e9ff700 1 -- 172.21.15.92:0/784412481 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f24880d2940 con 0x7f24880cd460 2023-11-28T11:22:20.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.763+0000 7f2484ff9700 1 -- 172.21.15.92:0/784412481 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v14) v1 ==== 90+0+0 (secure 0 0 0) 0x7f247c02b0a0 con 0x7f24880cd460 2023-11-28T11:22:20.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.765+0000 7f248e9ff700 1 -- 172.21.15.92:0/784412481 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f24740682f0 msgr2=0x7f247406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:20.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.765+0000 7f248e9ff700 1 --2- 172.21.15.92:0/784412481 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f24740682f0 0x7f247406a7a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2478000f60 tx=0x7f2478007be0 comp rx=0 tx=0).stop 2023-11-28T11:22:20.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.765+0000 7f248e9ff700 1 -- 172.21.15.92:0/784412481 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24880cd460 msgr2=0x7f24880a9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:20.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.765+0000 7f248e9ff700 1 --2- 172.21.15.92:0/784412481 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24880cd460 0x7f24880a9840 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f247c0090b0 tx=0x7f247c004750 comp rx=0 tx=0).stop 2023-11-28T11:22:20.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.766+0000 7f248e9ff700 1 -- 172.21.15.92:0/784412481 shutdown_connections 2023-11-28T11:22:20.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.766+0000 7f248e9ff700 1 --2- 172.21.15.92:0/784412481 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24880d0d60 0x7f248814af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:20.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.766+0000 7f248e9ff700 1 --2- 172.21.15.92:0/784412481 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24880b8510 0x7f24881e5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:20.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.766+0000 7f248e9ff700 1 --2- 172.21.15.92:0/784412481 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f24740682f0 0x7f247406a7a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:20.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.766+0000 7f248e9ff700 1 --2- 172.21.15.92:0/784412481 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24880cd460 0x7f24880a9840 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:20.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.766+0000 7f248e9ff700 1 -- 172.21.15.92:0/784412481 >> 172.21.15.92:0/784412481 conn(0x7f24880719a0 msgr2=0x7f248806b970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:20.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.766+0000 7f248e9ff700 1 -- 172.21.15.92:0/784412481 shutdown_connections 2023-11-28T11:22:20.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.766+0000 7f248e9ff700 1 -- 172.21.15.92:0/784412481 wait complete. 2023-11-28T11:22:20.785 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:22:20.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.981+0000 7f358d7ff700 1 Processor -- start 2023-11-28T11:22:20.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.981+0000 7f358d7ff700 1 -- start start 2023-11-28T11:22:20.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.983+0000 7f358d7ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f35880b8510 0x7f358806b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:20.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.983+0000 7f358d7ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35880cd460 0x7f358806beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:20.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.983+0000 7f358d7ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f358806c3f0 0x7f358815cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:20.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.983+0000 7f358d7ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35880cdbd0 con 0x7f35880b8510 2023-11-28T11:22:20.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.983+0000 7f358d7ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35880d02d0 con 0x7f358806c3f0 2023-11-28T11:22:20.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.983+0000 7f358d7ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35880b88e0 con 0x7f35880cd460 2023-11-28T11:22:20.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.983+0000 7f3586ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f35880b8510 0x7f358806b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:20.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.984+0000 7f3586ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f35880b8510 0x7f358806b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51132/0 (socket says 172.21.15.92:51132) 2023-11-28T11:22:20.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.984+0000 7f3586ffd700 1 -- 172.21.15.92:0/2805871527 learned_addr learned my addr 172.21.15.92:0/2805871527 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:20.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.984+0000 7f35877fe700 1 --2- 172.21.15.92:0/2805871527 >> v2:172.21.15.169:3300/0 conn(0x7f358806c3f0 0x7f358815cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:20.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.984+0000 7f357ffff700 1 --2- 172.21.15.92:0/2805871527 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35880cd460 0x7f358806beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:20.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.984+0000 7f3586ffd700 1 -- 172.21.15.92:0/2805871527 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35880cd460 msgr2=0x7f358806beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:20.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.984+0000 7f3586ffd700 1 --2- 172.21.15.92:0/2805871527 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35880cd460 0x7f358806beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:20.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.984+0000 7f3586ffd700 1 -- 172.21.15.92:0/2805871527 >> v2:172.21.15.169:3300/0 conn(0x7f358806c3f0 msgr2=0x7f358815cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:20.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.984+0000 7f3586ffd700 1 --2- 172.21.15.92:0/2805871527 >> v2:172.21.15.169:3300/0 conn(0x7f358806c3f0 0x7f358815cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:20.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.984+0000 7f3586ffd700 1 -- 172.21.15.92:0/2805871527 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35880cd8f0 con 0x7f35880b8510 2023-11-28T11:22:20.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.984+0000 7f3586ffd700 1 --2- 172.21.15.92:0/2805871527 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35880b8510 0x7f358806b970 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f35880b4c80 tx=0x7f3578009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1012707c1512311 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:20.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.984+0000 7f35867fc700 1 -- 172.21.15.92:0/2805871527 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3578011830 con 0x7f35880b8510 2023-11-28T11:22:20.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.985+0000 7f35867fc700 1 -- 172.21.15.92:0/2805871527 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3578011990 con 0x7f35880b8510 2023-11-28T11:22:20.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.985+0000 7f35867fc700 1 -- 172.21.15.92:0/2805871527 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3578011d20 con 0x7f35880b8510 2023-11-28T11:22:20.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.985+0000 7f358d7ff700 1 -- 172.21.15.92:0/2805871527 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35880b8510 msgr2=0x7f358806b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:20.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.985+0000 7f358d7ff700 1 --2- 172.21.15.92:0/2805871527 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35880b8510 0x7f358806b970 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f35880b4c80 tx=0x7f3578009c00 comp rx=0 tx=0).stop 2023-11-28T11:22:20.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.985+0000 7f358d7ff700 1 -- 172.21.15.92:0/2805871527 shutdown_connections 2023-11-28T11:22:20.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.985+0000 7f358d7ff700 1 --2- 172.21.15.92:0/2805871527 >> v2:172.21.15.169:3300/0 conn(0x7f358806c3f0 0x7f358815cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:20.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.985+0000 7f358d7ff700 1 --2- 172.21.15.92:0/2805871527 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35880b8510 0x7f358806b970 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:20.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.985+0000 7f358d7ff700 1 --2- 172.21.15.92:0/2805871527 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35880cd460 0x7f358806beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:20.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.985+0000 7f358d7ff700 1 -- 172.21.15.92:0/2805871527 >> 172.21.15.92:0/2805871527 conn(0x7f35880719a0 msgr2=0x7f3588071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:20.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.985+0000 7f358d7ff700 1 -- 172.21.15.92:0/2805871527 shutdown_connections 2023-11-28T11:22:20.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.985+0000 7f358d7ff700 1 -- 172.21.15.92:0/2805871527 wait complete. 2023-11-28T11:22:20.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.987+0000 7f358d7ff700 1 Processor -- start 2023-11-28T11:22:20.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.987+0000 7f358d7ff700 1 -- start start 2023-11-28T11:22:20.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.988+0000 7f358d7ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f358806c3f0 0x7f35880cc990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:20.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.988+0000 7f358d7ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35880b8510 0x7f35880c8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:20.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.988+0000 7f358d7ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35880cd460 0x7f3588157a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:20.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.988+0000 7f358d7ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35880cced0 con 0x7f35880cd460 2023-11-28T11:22:20.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.988+0000 7f358d7ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35880c91e0 con 0x7f358806c3f0 2023-11-28T11:22:20.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.988+0000 7f358d7ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3588157fa0 con 0x7f35880b8510 2023-11-28T11:22:20.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.988+0000 7f35877fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35880cd460 0x7f3588157a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:20.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.988+0000 7f35877fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35880cd460 0x7f3588157a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51140/0 (socket says 172.21.15.92:51140) 2023-11-28T11:22:20.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.988+0000 7f35877fe700 1 -- 172.21.15.92:0/1327639606 learned_addr learned my addr 172.21.15.92:0/1327639606 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:20.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.988+0000 7f3586ffd700 1 --2- 172.21.15.92:0/1327639606 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f358806c3f0 0x7f35880cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:20.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.988+0000 7f357ffff700 1 --2- 172.21.15.92:0/1327639606 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35880b8510 0x7f35880c8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:20.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.989+0000 7f35877fe700 1 -- 172.21.15.92:0/1327639606 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35880b8510 msgr2=0x7f35880c8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:20.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.989+0000 7f35877fe700 1 --2- 172.21.15.92:0/1327639606 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35880b8510 0x7f35880c8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:20.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.989+0000 7f35877fe700 1 -- 172.21.15.92:0/1327639606 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f358806c3f0 msgr2=0x7f35880cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:20.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.989+0000 7f35877fe700 1 --2- 172.21.15.92:0/1327639606 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f358806c3f0 0x7f35880cc990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:20.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.989+0000 7f35877fe700 1 -- 172.21.15.92:0/1327639606 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3578009550 con 0x7f35880cd460 2023-11-28T11:22:20.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.989+0000 7f35877fe700 1 --2- 172.21.15.92:0/1327639606 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35880cd460 0x7f3588157a60 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f358000eed0 tx=0x7f358000b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:20.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.989+0000 7f357f7fe700 1 -- 172.21.15.92:0/1327639606 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3580010040 con 0x7f35880cd460 2023-11-28T11:22:20.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.989+0000 7f357f7fe700 1 -- 172.21.15.92:0/1327639606 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f358000bc80 con 0x7f35880cd460 2023-11-28T11:22:20.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.989+0000 7f357f7fe700 1 -- 172.21.15.92:0/1327639606 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3580009e40 con 0x7f35880cd460 2023-11-28T11:22:20.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.989+0000 7f358d7ff700 1 -- 172.21.15.92:0/1327639606 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35880cfff0 con 0x7f35880cd460 2023-11-28T11:22:20.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.989+0000 7f358d7ff700 1 -- 172.21.15.92:0/1327639606 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35880cfd10 con 0x7f35880cd460 2023-11-28T11:22:20.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.990+0000 7f358d7ff700 1 -- 172.21.15.92:0/1327639606 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3588067880 con 0x7f35880cd460 2023-11-28T11:22:20.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.992+0000 7f357f7fe700 1 -- 172.21.15.92:0/1327639606 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f358001d400 con 0x7f35880cd460 2023-11-28T11:22:20.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.992+0000 7f357f7fe700 1 --2- 172.21.15.92:0/1327639606 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f35680682f0 0x7f356806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:20.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.992+0000 7f357f7fe700 1 -- 172.21.15.92:0/1327639606 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f3580086220 con 0x7f35880cd460 2023-11-28T11:22:20.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.993+0000 7f3586ffd700 1 --2- 172.21.15.92:0/1327639606 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f35680682f0 0x7f356806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:20.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.993+0000 7f3586ffd700 1 --2- 172.21.15.92:0/1327639606 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f35680682f0 0x7f356806a7a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f35780098f0 tx=0x7f3578008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:20.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:20.996+0000 7f357f7fe700 1 -- 172.21.15.92:0/1327639606 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3580052ce0 con 0x7f35880cd460 2023-11-28T11:22:21.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.187+0000 7f358d7ff700 1 -- 172.21.15.92:0/1327639606 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f35880d2940 con 0x7f35880cd460 2023-11-28T11:22:21.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.188+0000 7f357f7fe700 1 -- 172.21.15.92:0/1327639606 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+14472 (secure 0 0 0) 0x7f3580056300 con 0x7f35880cd460 2023-11-28T11:22:21.189 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:22:21.189 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":38,"fsid":"68956e0c-c16f-47cd-a220-f98364b0831d","created":"2023-11-28T11:20:28.697573+0000","modified":"2023-11-28T11:22:17.185656+0000","last_up_change":"2023-11-28T11:20:41.257011+0000","last_in_change":"2023-11-28T11:20:32.249767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:20:41.925237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:20:48.682497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6abe483f-0e03-4a26-b9cf-62041291747e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":115823},{"type":"v1","addr":"172.21.15.92:6809","nonce":115823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":115823},{"type":"v1","addr":"172.21.15.92:6813","nonce":115823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":115823},{"type":"v1","addr":"172.21.15.92:6821","nonce":115823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":115823},{"type":"v1","addr":"172.21.15.92:6817","nonce":115823}]},"public_addr":"172.21.15.92:6809/115823","cluster_addr":"172.21.15.92:6813/115823","heartbeat_back_addr":"172.21.15.92:6821/115823","heartbeat_front_addr":"172.21.15.92:6817/115823","state":["exists","up"]},{"osd":1,"uuid":"13c1de73-1a8c-4d7e-8a28-9dae52dfe089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":115824},{"type":"v1","addr":"172.21.15.92:6812","nonce":115824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":115824},{"type":"v1","addr":"172.21.15.92:6816","nonce":115824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":115824},{"type":"v1","addr":"172.21.15.92:6823","nonce":115824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":115824},{"type":"v1","addr":"172.21.15.92:6820","nonce":115824}]},"public_addr":"172.21.15.92:6812/115824","cluster_addr":"172.21.15.92:6816/115824","heartbeat_back_addr":"172.21.15.92:6823/115824","heartbeat_front_addr":"172.21.15.92:6820/115824","state":["exists","up"]},{"osd":2,"uuid":"25788a55-112f-4f00-810d-2bc37cf76778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":115821},{"type":"v1","addr":"172.21.15.92:6801","nonce":115821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":115821},{"type":"v1","addr":"172.21.15.92:6803","nonce":115821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":115821},{"type":"v1","addr":"172.21.15.92:6807","nonce":115821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":115821},{"type":"v1","addr":"172.21.15.92:6805","nonce":115821}]},"public_addr":"172.21.15.92:6801/115821","cluster_addr":"172.21.15.92:6803/115821","heartbeat_back_addr":"172.21.15.92:6807/115821","heartbeat_front_addr":"172.21.15.92:6805/115821","state":["exists","up"]},{"osd":3,"uuid":"adf54e24-33a9-41fe-9ce3-98a5ab6c158e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":115825},{"type":"v1","addr":"172.21.15.92:6825","nonce":115825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":115825},{"type":"v1","addr":"172.21.15.92:6827","nonce":115825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":115825},{"type":"v1","addr":"172.21.15.92:6831","nonce":115825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":115825},{"type":"v1","addr":"172.21.15.92:6829","nonce":115825}]},"public_addr":"172.21.15.92:6825/115825","cluster_addr":"172.21.15.92:6827/115825","heartbeat_back_addr":"172.21.15.92:6831/115825","heartbeat_front_addr":"172.21.15.92:6829/115825","state":["exists","up"]},{"osd":4,"uuid":"ccb9b747-f399-44ea-9a18-9db3152227e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":116035},{"type":"v1","addr":"172.21.15.169:6817","nonce":116035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":116035},{"type":"v1","addr":"172.21.15.169:6819","nonce":116035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":116035},{"type":"v1","addr":"172.21.15.169:6826","nonce":116035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":116035},{"type":"v1","addr":"172.21.15.169:6822","nonce":116035}]},"public_addr":"172.21.15.169:6817/116035","cluster_addr":"172.21.15.169:6819/116035","heartbeat_back_addr":"172.21.15.169:6826/116035","heartbeat_front_addr":"172.21.15.169:6822/116035","state":["exists","up"]},{"osd":5,"uuid":"308a1e63-75aa-4d7d-8a7c-cecc24714fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":116033},{"type":"v1","addr":"172.21.15.169:6801","nonce":116033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":116033},{"type":"v1","addr":"172.21.15.169:6803","nonce":116033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":116033},{"type":"v1","addr":"172.21.15.169:6807","nonce":116033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":116033},{"type":"v1","addr":"172.21.15.169:6805","nonce":116033}]},"public_addr":"172.21.15.169:6801/116033","cluster_addr":"172.21.15.169:6803/116033","heartbeat_back_addr":"172.21.15.169:6807/116033","heartbeat_front_addr":"172.21.15.169:6805/116033","state":["exists","up"]},{"osd":6,"uuid":"9c459b7b-b821-4b4b-b91e-6aaae271457c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":116034},{"type":"v1","addr":"172.21.15.169:6809","nonce":116034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":116034},{"type":"v1","addr":"172.21.15.169:6811","nonce":116034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":116034},{"type":"v1","addr":"172.21.15.169:6815","nonce":116034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":116034},{"type":"v1","addr":"172.21.15.169:6813","nonce":116034}]},"public_addr":"172.21.15.169:6809/116034","cluster_addr":"172.21.15.169:6811/116034","heartbeat_back_addr":"172.21.15.169:6815/116034","heartbeat_front_addr":"172.21.15.169:6813/116034","state":["exists","up"]},{"osd":7,"uuid":"1ca9aa4a-755d-4e28-a937-4d8dccab4df3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":116044},{"type":"v1","addr":"172.21.15.169:6823","nonce":116044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6825","nonce":116044},{"type":"v1","addr":"172.21.15.169:6827","nonce":116044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":116044},{"type":"v1","addr":"172.21.15.169:6831","nonce":116044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":116044},{"type":"v1","addr":"172.21.15.169:6829","nonce":116044}]},"public_addr":"172.21.15.169:6823/116044","cluster_addr":"172.21.15.169:6827/116044","heartbeat_back_addr":"172.21.15.169:6831/116044","heartbeat_front_addr":"172.21.15.169:6829/116044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.586781+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.869778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.508594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.860493+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.872267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.746933+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.611520+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.871446+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:22:21.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.190+0000 7f358d7ff700 1 -- 172.21.15.92:0/1327639606 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f35680682f0 msgr2=0x7f356806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:21.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.190+0000 7f358d7ff700 1 --2- 172.21.15.92:0/1327639606 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f35680682f0 0x7f356806a7a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f35780098f0 tx=0x7f3578008030 comp rx=0 tx=0).stop 2023-11-28T11:22:21.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.190+0000 7f358d7ff700 1 -- 172.21.15.92:0/1327639606 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35880cd460 msgr2=0x7f3588157a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:21.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.190+0000 7f358d7ff700 1 --2- 172.21.15.92:0/1327639606 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35880cd460 0x7f3588157a60 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f358000eed0 tx=0x7f358000b390 comp rx=0 tx=0).stop 2023-11-28T11:22:21.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.190+0000 7f358d7ff700 1 -- 172.21.15.92:0/1327639606 shutdown_connections 2023-11-28T11:22:21.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.190+0000 7f358d7ff700 1 --2- 172.21.15.92:0/1327639606 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f35680682f0 0x7f356806a7a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:21.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.190+0000 7f358d7ff700 1 --2- 172.21.15.92:0/1327639606 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35880cd460 0x7f3588157a60 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:21.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.190+0000 7f358d7ff700 1 --2- 172.21.15.92:0/1327639606 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f358806c3f0 0x7f35880cc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:21.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.190+0000 7f358d7ff700 1 --2- 172.21.15.92:0/1327639606 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35880b8510 0x7f35880c8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:21.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.190+0000 7f358d7ff700 1 -- 172.21.15.92:0/1327639606 >> 172.21.15.92:0/1327639606 conn(0x7f35880719a0 msgr2=0x7f35880b4150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:21.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.190+0000 7f358d7ff700 1 -- 172.21.15.92:0/1327639606 shutdown_connections 2023-11-28T11:22:21.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.190+0000 7f358d7ff700 1 -- 172.21.15.92:0/1327639606 wait complete. 2023-11-28T11:22:21.211 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:20:41.925237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:20:48.682497+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:22:21.211 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:22:21.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.409+0000 7faf18bff700 1 Processor -- start 2023-11-28T11:22:21.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.409+0000 7faf18bff700 1 -- start start 2023-11-28T11:22:21.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.410+0000 7faf18bff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7faf140b8510 0x7faf1406b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:21.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.410+0000 7faf18bff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf140cd460 0x7faf1406beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:21.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.410+0000 7faf18bff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf140b88e0 con 0x7faf1406c3f0 2023-11-28T11:22:21.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.410+0000 7faf18bff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf140cdbd0 con 0x7faf140b8510 2023-11-28T11:22:21.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.410+0000 7faf18bff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf140d02d0 con 0x7faf140cd460 2023-11-28T11:22:21.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.411+0000 7faf11d9b700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf140cd460 0x7faf1406beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:21.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.411+0000 7faf11d9b700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf140cd460 0x7faf1406beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.92:59988/0 (socket says 172.21.15.92:59988) 2023-11-28T11:22:21.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.411+0000 7faf11d9b700 1 -- 172.21.15.92:0/1101850710 learned_addr learned my addr 172.21.15.92:0/1101850710 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:21.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.411+0000 7faf1259c700 1 --2- 172.21.15.92:0/1101850710 >> v2:172.21.15.169:3300/0 conn(0x7faf140b8510 0x7faf1406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:21.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.411+0000 7faf1159a700 1 -- 172.21.15.92:0/1101850710 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3895325794 0 0) 0x7faf140b88e0 con 0x7faf1406c3f0 2023-11-28T11:22:21.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.411+0000 7faf1159a700 1 -- 172.21.15.92:0/1101850710 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faefc0035b0 con 0x7faf1406c3f0 2023-11-28T11:22:21.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.411+0000 7faf11d9b700 1 -- 172.21.15.92:0/1101850710 >> v2:172.21.15.169:3300/0 conn(0x7faf140b8510 msgr2=0x7faf1406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:21.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.411+0000 7faf11d9b700 1 --2- 172.21.15.92:0/1101850710 >> v2:172.21.15.169:3300/0 conn(0x7faf140b8510 0x7faf1406b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:21.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.411+0000 7faf11d9b700 1 -- 172.21.15.92:0/1101850710 >> v1:172.21.15.169:6789/0 conn(0x7faf1406c3f0 legacy=0x7faf1415cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:21.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.411+0000 7faf11d9b700 1 -- 172.21.15.92:0/1101850710 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf140cd8f0 con 0x7faf140cd460 2023-11-28T11:22:21.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.411+0000 7faf1259c700 1 --2- 172.21.15.92:0/1101850710 >> v2:172.21.15.169:3300/0 conn(0x7faf140b8510 0x7faf1406b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:22:21.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.412+0000 7faf11d9b700 1 --2- 172.21.15.92:0/1101850710 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf140cd460 0x7faf1406beb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7faf080098f0 tx=0x7faf08009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eabe9c57e51dfc5d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:21.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.412+0000 7faf1159a700 1 -- 172.21.15.92:0/1101850710 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf08011830 con 0x7faf140cd460 2023-11-28T11:22:21.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.412+0000 7faf1159a700 1 -- 172.21.15.92:0/1101850710 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf08011990 con 0x7faf140cd460 2023-11-28T11:22:21.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.412+0000 7faf1159a700 1 -- 172.21.15.92:0/1101850710 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf08011d20 con 0x7faf140cd460 2023-11-28T11:22:21.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.412+0000 7faf18bff700 1 -- 172.21.15.92:0/1101850710 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf140cd460 msgr2=0x7faf1406beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:21.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.412+0000 7faf18bff700 1 --2- 172.21.15.92:0/1101850710 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf140cd460 0x7faf1406beb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7faf080098f0 tx=0x7faf08009c00 comp rx=0 tx=0).stop 2023-11-28T11:22:21.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.413+0000 7faf18bff700 1 -- 172.21.15.92:0/1101850710 shutdown_connections 2023-11-28T11:22:21.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.413+0000 7faf18bff700 1 --2- 172.21.15.92:0/1101850710 >> v2:172.21.15.169:3300/0 conn(0x7faf140b8510 0x7faf1406b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:21.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.413+0000 7faf18bff700 1 --2- 172.21.15.92:0/1101850710 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf140cd460 0x7faf1406beb0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:21.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.413+0000 7faf18bff700 1 -- 172.21.15.92:0/1101850710 >> 172.21.15.92:0/1101850710 conn(0x7faf140719a0 msgr2=0x7faf14071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:21.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.413+0000 7faf18bff700 1 -- 172.21.15.92:0/1101850710 shutdown_connections 2023-11-28T11:22:21.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.413+0000 7faf18bff700 1 -- 172.21.15.92:0/1101850710 wait complete. 2023-11-28T11:22:21.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.414+0000 7faf18bff700 1 Processor -- start 2023-11-28T11:22:21.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.414+0000 7faf18bff700 1 -- start start 2023-11-28T11:22:21.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.415+0000 7faf18bff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf1406c3f0 0x7faf141f19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:21.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.415+0000 7faf18bff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf140b8510 0x7faf141f5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:21.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.415+0000 7faf18bff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf140cd460 0x7faf141f5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:21.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.415+0000 7faf18bff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf141f2f70 con 0x7faf1406c3f0 2023-11-28T11:22:21.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.415+0000 7faf18bff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf141f2310 con 0x7faf140b8510 2023-11-28T11:22:21.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.415+0000 7faf18bff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf141f25f0 con 0x7faf140cd460 2023-11-28T11:22:21.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.416+0000 7faf1259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf1406c3f0 0x7faf141f19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:21.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.416+0000 7faf1259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf1406c3f0 0x7faf141f19a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51156/0 (socket says 172.21.15.92:51156) 2023-11-28T11:22:21.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.416+0000 7faf1259c700 1 -- 172.21.15.92:0/3019998287 learned_addr learned my addr 172.21.15.92:0/3019998287 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:21.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.416+0000 7faf11d9b700 1 --2- 172.21.15.92:0/3019998287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf140b8510 0x7faf141f5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:21.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.416+0000 7faf12d9d700 1 --2- 172.21.15.92:0/3019998287 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf140cd460 0x7faf141f5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:21.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.416+0000 7faf1259c700 1 -- 172.21.15.92:0/3019998287 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf140cd460 msgr2=0x7faf141f5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:21.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.416+0000 7faf1259c700 1 --2- 172.21.15.92:0/3019998287 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf140cd460 0x7faf141f5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:21.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.416+0000 7faf1259c700 1 -- 172.21.15.92:0/3019998287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf140b8510 msgr2=0x7faf141f5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:21.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.416+0000 7faf1259c700 1 --2- 172.21.15.92:0/3019998287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf140b8510 0x7faf141f5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:21.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.416+0000 7faf1259c700 1 -- 172.21.15.92:0/3019998287 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf040094a0 con 0x7faf1406c3f0 2023-11-28T11:22:21.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.416+0000 7faf12d9d700 1 --2- 172.21.15.92:0/3019998287 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf140cd460 0x7faf141f5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:22:21.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.416+0000 7faf11d9b700 1 --2- 172.21.15.92:0/3019998287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf140b8510 0x7faf141f5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:22:21.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.416+0000 7faf1259c700 1 --2- 172.21.15.92:0/3019998287 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf1406c3f0 0x7faf141f19a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7faf0400ebe0 tx=0x7faf04009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:21.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.417+0000 7faf02ffd700 1 -- 172.21.15.92:0/3019998287 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0400c910 con 0x7faf1406c3f0 2023-11-28T11:22:21.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.417+0000 7faf02ffd700 1 -- 172.21.15.92:0/3019998287 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf0400ca70 con 0x7faf1406c3f0 2023-11-28T11:22:21.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.417+0000 7faf18bff700 1 -- 172.21.15.92:0/3019998287 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf08009550 con 0x7faf1406c3f0 2023-11-28T11:22:21.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.417+0000 7faf02ffd700 1 -- 172.21.15.92:0/3019998287 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0400cde0 con 0x7faf1406c3f0 2023-11-28T11:22:21.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.417+0000 7faf18bff700 1 -- 172.21.15.92:0/3019998287 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf140cfff0 con 0x7faf1406c3f0 2023-11-28T11:22:21.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.418+0000 7faf18bff700 1 -- 172.21.15.92:0/3019998287 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf140cfcf0 con 0x7faf1406c3f0 2023-11-28T11:22:21.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.419+0000 7faf02ffd700 1 -- 172.21.15.92:0/3019998287 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7faf04019030 con 0x7faf1406c3f0 2023-11-28T11:22:21.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.420+0000 7faf02ffd700 1 --2- 172.21.15.92:0/3019998287 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7faef40682f0 0x7faef406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:21.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.420+0000 7faf02ffd700 1 -- 172.21.15.92:0/3019998287 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7faf04085be0 con 0x7faf1406c3f0 2023-11-28T11:22:21.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.420+0000 7faf11d9b700 1 --2- 172.21.15.92:0/3019998287 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7faef40682f0 0x7faef406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:21.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.421+0000 7faf11d9b700 1 --2- 172.21.15.92:0/3019998287 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7faef40682f0 0x7faef406a7a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7faf08000c00 tx=0x7faf08008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:21.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.424+0000 7faf02ffd700 1 -- 172.21.15.92:0/3019998287 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7faf040525d0 con 0x7faf1406c3f0 2023-11-28T11:22:21.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.615+0000 7faf18bff700 1 -- 172.21.15.92:0/3019998287 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faf14067880 con 0x7faf1406c3f0 2023-11-28T11:22:21.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.616+0000 7faf02ffd700 1 -- 172.21.15.92:0/3019998287 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) v1 ==== 93+0+10 (secure 0 0 0) 0x7faf04055bf0 con 0x7faf1406c3f0 2023-11-28T11:22:21.617 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-11-28T11:22:21.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.618+0000 7faf18bff700 1 -- 172.21.15.92:0/3019998287 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7faef40682f0 msgr2=0x7faef406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:21.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.618+0000 7faf18bff700 1 --2- 172.21.15.92:0/3019998287 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7faef40682f0 0x7faef406a7a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7faf08000c00 tx=0x7faf08008030 comp rx=0 tx=0).stop 2023-11-28T11:22:21.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.618+0000 7faf18bff700 1 -- 172.21.15.92:0/3019998287 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf1406c3f0 msgr2=0x7faf141f19a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:21.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.618+0000 7faf18bff700 1 --2- 172.21.15.92:0/3019998287 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf1406c3f0 0x7faf141f19a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7faf0400ebe0 tx=0x7faf04009ec0 comp rx=0 tx=0).stop 2023-11-28T11:22:21.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.618+0000 7faf18bff700 1 -- 172.21.15.92:0/3019998287 shutdown_connections 2023-11-28T11:22:21.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.618+0000 7faf18bff700 1 --2- 172.21.15.92:0/3019998287 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf140cd460 0x7faf141f5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:21.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.618+0000 7faf18bff700 1 --2- 172.21.15.92:0/3019998287 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7faef40682f0 0x7faef406a7a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:21.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.618+0000 7faf18bff700 1 --2- 172.21.15.92:0/3019998287 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf1406c3f0 0x7faf141f19a0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:21.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.618+0000 7faf18bff700 1 --2- 172.21.15.92:0/3019998287 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf140b8510 0x7faf141f5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:21.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.618+0000 7faf18bff700 1 -- 172.21.15.92:0/3019998287 >> 172.21.15.92:0/3019998287 conn(0x7faf140719a0 msgr2=0x7faf140b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:21.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.618+0000 7faf18bff700 1 -- 172.21.15.92:0/3019998287 shutdown_connections 2023-11-28T11:22:21.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.618+0000 7faf18bff700 1 -- 172.21.15.92:0/3019998287 wait complete. 2023-11-28T11:22:21.637 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:22:21.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.837+0000 7fd03f7ff700 1 Processor -- start 2023-11-28T11:22:21.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.837+0000 7fd03f7ff700 1 -- start start 2023-11-28T11:22:21.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.838+0000 7fd03f7ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd0380b4fc0 0x7fd0380cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:21.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.838+0000 7fd03f7ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0380d1950 con 0x7fd0380b1c60 2023-11-28T11:22:21.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.838+0000 7fd03f7ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0380b2500 con 0x7fd0380d2df0 2023-11-28T11:22:21.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.838+0000 7fd03f7ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0380d3d30 con 0x7fd0380b4fc0 2023-11-28T11:22:21.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.838+0000 7fd03dd9c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd0380b1c60 0x7fd0380cf010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33756/0 (socket says 172.21.15.92:33756) 2023-11-28T11:22:21.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.838+0000 7fd03dd9c700 1 -- 172.21.15.92:0/2636123649 learned_addr learned my addr 172.21.15.92:0/2636123649 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:22:21.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.839+0000 7fd03cd9a700 1 --2- 172.21.15.92:0/2636123649 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd0380b4fc0 0x7fd0380cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:21.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.839+0000 7fd02ffff700 1 -- 172.21.15.92:0/2636123649 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1486196535 0 0) 0x7fd0380d1950 con 0x7fd0380b1c60 2023-11-28T11:22:21.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.839+0000 7fd02ffff700 1 -- 172.21.15.92:0/2636123649 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0240035b0 con 0x7fd0380b1c60 2023-11-28T11:22:21.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.839+0000 7fd02ffff700 1 -- 172.21.15.92:0/2636123649 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3312640655 0 0) 0x7fd0380b2500 con 0x7fd0380d2df0 2023-11-28T11:22:21.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.839+0000 7fd02ffff700 1 -- 172.21.15.92:0/2636123649 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0380d1950 con 0x7fd0380d2df0 2023-11-28T11:22:21.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.839+0000 7fd02ffff700 1 -- 172.21.15.92:0/2636123649 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3040939727 0 0) 0x7fd0240035b0 con 0x7fd0380b1c60 2023-11-28T11:22:21.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.839+0000 7fd02ffff700 1 -- 172.21.15.92:0/2636123649 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd0380b4fc0 msgr2=0x7fd0380cd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:21.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.839+0000 7fd02ffff700 1 --2- 172.21.15.92:0/2636123649 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd0380b4fc0 0x7fd0380cd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:21.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.839+0000 7fd02ffff700 1 -- 172.21.15.92:0/2636123649 >> v1:172.21.15.169:6789/0 conn(0x7fd0380d2df0 legacy=0x7fd0380d31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:21.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.839+0000 7fd02ffff700 1 -- 172.21.15.92:0/2636123649 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0380d3a50 con 0x7fd0380b1c60 2023-11-28T11:22:21.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.839+0000 7fd02ffff700 1 -- 172.21.15.92:0/2636123649 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532161587 0 0) 0x7fd0300019f0 con 0x7fd0380b1c60 2023-11-28T11:22:21.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.839+0000 7fd03cd9a700 1 --2- 172.21.15.92:0/2636123649 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd0380b4fc0 0x7fd0380cd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:22:21.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.839+0000 7fd02ffff700 1 -- 172.21.15.92:0/2636123649 >> v1:172.21.15.92:6789/0 conn(0x7fd0380b1c60 legacy=0x7fd0380cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:21.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.839+0000 7fd02ffff700 1 --2- 172.21.15.92:0/2636123649 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd024003bc0 0x7fd024003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:21.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.839+0000 7fd02ffff700 1 --2- 172.21.15.92:0/2636123649 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd024004850 0x7fd024004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:21.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.839+0000 7fd02ffff700 1 --2- 172.21.15.92:0/2636123649 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd0380b4fc0 0x7fd024005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:21.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.839+0000 7fd02ffff700 1 -- 172.21.15.92:0/2636123649 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0380d3d30 con 0x7fd024003bc0 2023-11-28T11:22:21.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.839+0000 7fd02ffff700 1 -- 172.21.15.92:0/2636123649 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd02400ba40 con 0x7fd024004850 2023-11-28T11:22:21.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.839+0000 7fd02ffff700 1 -- 172.21.15.92:0/2636123649 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd02400bc00 con 0x7fd0380b4fc0 2023-11-28T11:22:21.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.840+0000 7fd03d59b700 1 --2- 172.21.15.92:0/2636123649 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd024003bc0 0x7fd024003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:21.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.840+0000 7fd03cd9a700 1 --2- 172.21.15.92:0/2636123649 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd024004850 0x7fd024004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:21.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.840+0000 7fd03dd9c700 1 --2- 172.21.15.92:0/2636123649 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd0380b4fc0 0x7fd024005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:21.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.840+0000 7fd03d59b700 1 -- 172.21.15.92:0/2636123649 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd0380b4fc0 msgr2=0x7fd024005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:21.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.840+0000 7fd03d59b700 1 --2- 172.21.15.92:0/2636123649 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd0380b4fc0 0x7fd024005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:21.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.840+0000 7fd03d59b700 1 -- 172.21.15.92:0/2636123649 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd024004850 msgr2=0x7fd024004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:21.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.840+0000 7fd03d59b700 1 --2- 172.21.15.92:0/2636123649 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd024004850 0x7fd024004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:21.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.840+0000 7fd03d59b700 1 -- 172.21.15.92:0/2636123649 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0280094a0 con 0x7fd024003bc0 2023-11-28T11:22:21.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.840+0000 7fd03d59b700 1 --2- 172.21.15.92:0/2636123649 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd024003bc0 0x7fd024003fb0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd034001940 tx=0x7fd024005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3650150badebf47b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:21.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.841+0000 7fd02ffff700 1 -- 172.21.15.92:0/2636123649 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd034001430 con 0x7fd024003bc0 2023-11-28T11:22:21.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.841+0000 7fd02ffff700 1 -- 172.21.15.92:0/2636123649 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0340050d0 con 0x7fd024003bc0 2023-11-28T11:22:21.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.841+0000 7fd02ffff700 1 -- 172.21.15.92:0/2636123649 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd034005480 con 0x7fd024003bc0 2023-11-28T11:22:21.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.841+0000 7fd03f7ff700 1 -- 172.21.15.92:0/2636123649 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd024003bc0 msgr2=0x7fd024003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:21.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.841+0000 7fd03f7ff700 1 --2- 172.21.15.92:0/2636123649 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd024003bc0 0x7fd024003fb0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd034001940 tx=0x7fd024005880 comp rx=0 tx=0).stop 2023-11-28T11:22:21.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.841+0000 7fd03dd9c700 1 -- 172.21.15.92:0/2636123649 reap_dead start 2023-11-28T11:22:21.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.841+0000 7fd03f7ff700 1 -- 172.21.15.92:0/2636123649 shutdown_connections 2023-11-28T11:22:21.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.841+0000 7fd03f7ff700 1 -- 172.21.15.92:0/2636123649 >> 172.21.15.92:0/2636123649 conn(0x7fd0380719a0 msgr2=0x7fd038071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:21.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.841+0000 7fd03f7ff700 1 -- 172.21.15.92:0/2636123649 shutdown_connections 2023-11-28T11:22:21.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.841+0000 7fd03f7ff700 1 -- 172.21.15.92:0/2636123649 wait complete. 2023-11-28T11:22:21.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.843+0000 7fd03f7ff700 1 Processor -- start 2023-11-28T11:22:21.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.843+0000 7fd03f7ff700 1 -- start start 2023-11-28T11:22:21.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.844+0000 7fd03f7ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd024003bc0 0x7fd038157030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:21.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.844+0000 7fd03f7ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd038153410 0x7fd0380c8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:21.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.844+0000 7fd03f7ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd0380ccb00 0x7fd03815ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:21.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.844+0000 7fd03f7ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd038157570 con 0x7fd024003bc0 2023-11-28T11:22:21.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.844+0000 7fd03f7ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0380cb810 con 0x7fd038153410 2023-11-28T11:22:21.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.844+0000 7fd03f7ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0380cba60 con 0x7fd0380ccb00 2023-11-28T11:22:21.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.844+0000 7fd03d59b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd024003bc0 0x7fd038157030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:21.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.844+0000 7fd03d59b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd024003bc0 0x7fd038157030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51186/0 (socket says 172.21.15.92:51186) 2023-11-28T11:22:21.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.844+0000 7fd03d59b700 1 -- 172.21.15.92:0/3516249006 learned_addr learned my addr 172.21.15.92:0/3516249006 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:21.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.845+0000 7fd03d59b700 1 -- 172.21.15.92:0/3516249006 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd0380ccb00 msgr2=0x7fd03815ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:21.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.845+0000 7fd03d59b700 1 --2- 172.21.15.92:0/3516249006 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd0380ccb00 0x7fd03815ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:21.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.845+0000 7fd03cd9a700 1 --2- 172.21.15.92:0/3516249006 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd038153410 0x7fd0380c8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:21.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.845+0000 7fd03d59b700 1 -- 172.21.15.92:0/3516249006 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd038153410 msgr2=0x7fd0380c8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:21.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.845+0000 7fd03d59b700 1 --2- 172.21.15.92:0/3516249006 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd038153410 0x7fd0380c8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:21.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.845+0000 7fd03d59b700 1 -- 172.21.15.92:0/3516249006 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0380b88f0 con 0x7fd024003bc0 2023-11-28T11:22:21.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.845+0000 7fd03cd9a700 1 --2- 172.21.15.92:0/3516249006 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd038153410 0x7fd0380c8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:22:21.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.845+0000 7fd03d59b700 1 --2- 172.21.15.92:0/3516249006 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd024003bc0 0x7fd038157030 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd034005690 tx=0x7fd0340012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:21.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.845+0000 7fd02dffb700 1 -- 172.21.15.92:0/3516249006 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd034006c70 con 0x7fd024003bc0 2023-11-28T11:22:21.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.845+0000 7fd02dffb700 1 -- 172.21.15.92:0/3516249006 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd034006dd0 con 0x7fd024003bc0 2023-11-28T11:22:21.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.845+0000 7fd03f7ff700 1 -- 172.21.15.92:0/3516249006 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0380d15e0 con 0x7fd024003bc0 2023-11-28T11:22:21.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.845+0000 7fd02dffb700 1 -- 172.21.15.92:0/3516249006 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd034007280 con 0x7fd024003bc0 2023-11-28T11:22:21.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.845+0000 7fd03f7ff700 1 -- 172.21.15.92:0/3516249006 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0380b21b0 con 0x7fd024003bc0 2023-11-28T11:22:21.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.846+0000 7fd03f7ff700 1 -- 172.21.15.92:0/3516249006 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0380cc950 con 0x7fd024003bc0 2023-11-28T11:22:21.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.847+0000 7fd02dffb700 1 -- 172.21.15.92:0/3516249006 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd03402e070 con 0x7fd024003bc0 2023-11-28T11:22:21.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.848+0000 7fd02dffb700 1 --2- 172.21.15.92:0/3516249006 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd01c0682f0 0x7fd01c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:21.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.848+0000 7fd02dffb700 1 -- 172.21.15.92:0/3516249006 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fd034086b20 con 0x7fd024003bc0 2023-11-28T11:22:21.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.848+0000 7fd03cd9a700 1 --2- 172.21.15.92:0/3516249006 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd01c0682f0 0x7fd01c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:21.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.849+0000 7fd03cd9a700 1 --2- 172.21.15.92:0/3516249006 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd01c0682f0 0x7fd01c06a7a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd0380cb580 tx=0x7fd028008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:21.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:21.852+0000 7fd02dffb700 1 -- 172.21.15.92:0/3516249006 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd0340534a0 con 0x7fd024003bc0 2023-11-28T11:22:22.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.046+0000 7fd03f7ff700 1 -- 172.21.15.92:0/3516249006 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd0380b4a80 con 0x7fd024003bc0 2023-11-28T11:22:22.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.047+0000 7fd02dffb700 1 -- 172.21.15.92:0/3516249006 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v38) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd0340170a0 con 0x7fd024003bc0 2023-11-28T11:22:22.048 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-11-28T11:22:22.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.049+0000 7fd03f7ff700 1 -- 172.21.15.92:0/3516249006 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd01c0682f0 msgr2=0x7fd01c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:22.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.049+0000 7fd03f7ff700 1 --2- 172.21.15.92:0/3516249006 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd01c0682f0 0x7fd01c06a7a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd0380cb580 tx=0x7fd028008040 comp rx=0 tx=0).stop 2023-11-28T11:22:22.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.049+0000 7fd03f7ff700 1 -- 172.21.15.92:0/3516249006 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd024003bc0 msgr2=0x7fd038157030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:22.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.049+0000 7fd03f7ff700 1 --2- 172.21.15.92:0/3516249006 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd024003bc0 0x7fd038157030 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd034005690 tx=0x7fd0340012d0 comp rx=0 tx=0).stop 2023-11-28T11:22:22.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.049+0000 7fd03f7ff700 1 -- 172.21.15.92:0/3516249006 shutdown_connections 2023-11-28T11:22:22.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.049+0000 7fd03f7ff700 1 --2- 172.21.15.92:0/3516249006 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd0380ccb00 0x7fd03815ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:22.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.049+0000 7fd03f7ff700 1 --2- 172.21.15.92:0/3516249006 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd01c0682f0 0x7fd01c06a7a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:22.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.049+0000 7fd03f7ff700 1 --2- 172.21.15.92:0/3516249006 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd024003bc0 0x7fd038157030 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:22.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.049+0000 7fd03f7ff700 1 --2- 172.21.15.92:0/3516249006 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd038153410 0x7fd0380c8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:22.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.049+0000 7fd03f7ff700 1 -- 172.21.15.92:0/3516249006 >> 172.21.15.92:0/3516249006 conn(0x7fd0380719a0 msgr2=0x7fd0380b4130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:22.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.049+0000 7fd03f7ff700 1 -- 172.21.15.92:0/3516249006 shutdown_connections 2023-11-28T11:22:22.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.049+0000 7fd03f7ff700 1 -- 172.21.15.92:0/3516249006 wait complete. 2023-11-28T11:22:22.068 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T11:22:22.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.265+0000 7f8db05ff700 1 Processor -- start 2023-11-28T11:22:22.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.265+0000 7f8db05ff700 1 -- start start 2023-11-28T11:22:22.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.267+0000 7f8db05ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8da80d1d70 0x7f8da8167270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:22.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.267+0000 7f8db05ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8da806c790 con 0x7f8da80b8510 2023-11-28T11:22:22.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.267+0000 7f8db05ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8da80b88e0 con 0x7f8da80cd460 2023-11-28T11:22:22.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.267+0000 7f8db05ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8da80cdbd0 con 0x7f8da80d1d70 2023-11-28T11:22:22.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.267+0000 7f8daeb9c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8da80d1d70 0x7f8da8167270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:22.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.267+0000 7f8dae39b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f8da80b8510 0x7f8da806b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33766/0 (socket says 172.21.15.92:33766) 2023-11-28T11:22:22.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.267+0000 7f8dae39b700 1 -- 172.21.15.92:0/1621795698 learned_addr learned my addr 172.21.15.92:0/1621795698 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:22:22.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.267+0000 7f8dad399700 1 -- 172.21.15.92:0/1621795698 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 912634733 0 0) 0x7f8da806c790 con 0x7f8da80b8510 2023-11-28T11:22:22.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.267+0000 7f8dad399700 1 -- 172.21.15.92:0/1621795698 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d940035b0 con 0x7f8da80b8510 2023-11-28T11:22:22.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.267+0000 7f8daeb9c700 1 -- 172.21.15.92:0/1621795698 >> v1:172.21.15.169:6789/0 conn(0x7f8da80cd460 legacy=0x7f8da806c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:22.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.267+0000 7f8daeb9c700 1 -- 172.21.15.92:0/1621795698 >> v1:172.21.15.92:6789/0 conn(0x7f8da80b8510 legacy=0x7f8da806b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:22.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.267+0000 7f8daeb9c700 1 -- 172.21.15.92:0/1621795698 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da80cd8f0 con 0x7f8da80d1d70 2023-11-28T11:22:22.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.268+0000 7f8daeb9c700 1 --2- 172.21.15.92:0/1621795698 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8da80d1d70 0x7f8da8167270 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8da0009d90 tx=0x7f8da000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab10c1c486024cf6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:22.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.268+0000 7f8dad399700 1 -- 172.21.15.92:0/1621795698 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da000e070 con 0x7f8da80d1d70 2023-11-28T11:22:22.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.268+0000 7f8dad399700 1 -- 172.21.15.92:0/1621795698 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8da000bae0 con 0x7f8da80d1d70 2023-11-28T11:22:22.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.268+0000 7f8dad399700 1 -- 172.21.15.92:0/1621795698 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da000be90 con 0x7f8da80d1d70 2023-11-28T11:22:22.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.268+0000 7f8db05ff700 1 -- 172.21.15.92:0/1621795698 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8da80d1d70 msgr2=0x7f8da8167270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:22.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.268+0000 7f8db05ff700 1 --2- 172.21.15.92:0/1621795698 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8da80d1d70 0x7f8da8167270 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8da0009d90 tx=0x7f8da000b330 comp rx=0 tx=0).stop 2023-11-28T11:22:22.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.268+0000 7f8db05ff700 1 -- 172.21.15.92:0/1621795698 shutdown_connections 2023-11-28T11:22:22.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.268+0000 7f8db05ff700 1 --2- 172.21.15.92:0/1621795698 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8da80d1d70 0x7f8da8167270 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:22.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.268+0000 7f8db05ff700 1 -- 172.21.15.92:0/1621795698 >> 172.21.15.92:0/1621795698 conn(0x7f8da80719a0 msgr2=0x7f8da8071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:22.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.268+0000 7f8db05ff700 1 -- 172.21.15.92:0/1621795698 shutdown_connections 2023-11-28T11:22:22.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.269+0000 7f8db05ff700 1 -- 172.21.15.92:0/1621795698 wait complete. 2023-11-28T11:22:22.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.270+0000 7f8db05ff700 1 Processor -- start 2023-11-28T11:22:22.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.270+0000 7f8db05ff700 1 -- start start 2023-11-28T11:22:22.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.271+0000 7f8db05ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8da80b8510 0x7f8da81e27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:22.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.271+0000 7f8db05ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8da80cd460 0x7f8da81e7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:22.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.271+0000 7f8db05ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8da80d1d70 0x7f8da815a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:22.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.271+0000 7f8db05ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da81e4be0 con 0x7f8da80d1d70 2023-11-28T11:22:22.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.271+0000 7f8db05ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da81e4ec0 con 0x7f8da80b8510 2023-11-28T11:22:22.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.271+0000 7f8db05ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8da81e30b0 con 0x7f8da80cd460 2023-11-28T11:22:22.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.271+0000 7f8daeb9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8da80d1d70 0x7f8da815a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:22.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.271+0000 7f8daeb9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8da80d1d70 0x7f8da815a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51202/0 (socket says 172.21.15.92:51202) 2023-11-28T11:22:22.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.271+0000 7f8daeb9c700 1 -- 172.21.15.92:0/454535040 learned_addr learned my addr 172.21.15.92:0/454535040 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:22.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.272+0000 7f8daeb9c700 1 -- 172.21.15.92:0/454535040 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8da80cd460 msgr2=0x7f8da81e7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:22.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.272+0000 7f8daeb9c700 1 --2- 172.21.15.92:0/454535040 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8da80cd460 0x7f8da81e7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:22.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.272+0000 7f8daeb9c700 1 -- 172.21.15.92:0/454535040 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8da80b8510 msgr2=0x7f8da81e27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:22.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.272+0000 7f8dae39b700 1 --2- 172.21.15.92:0/454535040 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8da80b8510 0x7f8da81e27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:22.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.272+0000 7f8daeb9c700 1 --2- 172.21.15.92:0/454535040 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8da80b8510 0x7f8da81e27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:22.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.272+0000 7f8daeb9c700 1 -- 172.21.15.92:0/454535040 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da0009a70 con 0x7f8da80d1d70 2023-11-28T11:22:22.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.272+0000 7f8dae39b700 1 --2- 172.21.15.92:0/454535040 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8da80b8510 0x7f8da81e27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:22:22.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.272+0000 7f8daeb9c700 1 --2- 172.21.15.92:0/454535040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8da80d1d70 0x7f8da815a360 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f8da000b680 tx=0x7f8da0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:22.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.272+0000 7f8d9effd700 1 -- 172.21.15.92:0/454535040 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da0004750 con 0x7f8da80d1d70 2023-11-28T11:22:22.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.272+0000 7f8d9effd700 1 -- 172.21.15.92:0/454535040 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8da00048d0 con 0x7f8da80d1d70 2023-11-28T11:22:22.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.272+0000 7f8db05ff700 1 -- 172.21.15.92:0/454535040 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8da80cfff0 con 0x7f8da80d1d70 2023-11-28T11:22:22.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.272+0000 7f8d9effd700 1 -- 172.21.15.92:0/454535040 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da0012670 con 0x7f8da80d1d70 2023-11-28T11:22:22.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.273+0000 7f8db05ff700 1 -- 172.21.15.92:0/454535040 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8da80cfd10 con 0x7f8da80d1d70 2023-11-28T11:22:22.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.273+0000 7f8db05ff700 1 -- 172.21.15.92:0/454535040 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8da8067880 con 0x7f8da80d1d70 2023-11-28T11:22:22.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.274+0000 7f8d9effd700 1 -- 172.21.15.92:0/454535040 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8da00127d0 con 0x7f8da80d1d70 2023-11-28T11:22:22.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.275+0000 7f8d9effd700 1 --2- 172.21.15.92:0/454535040 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8d8c0682f0 0x7f8d8c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:22.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.275+0000 7f8d9effd700 1 -- 172.21.15.92:0/454535040 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f8da0087920 con 0x7f8da80d1d70 2023-11-28T11:22:22.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.276+0000 7f8dae39b700 1 --2- 172.21.15.92:0/454535040 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8d8c0682f0 0x7f8d8c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:22.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.276+0000 7f8dae39b700 1 --2- 172.21.15.92:0/454535040 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8d8c0682f0 0x7f8d8c06a7a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8da4005910 tx=0x7f8da4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:22.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.280+0000 7f8d9effd700 1 -- 172.21.15.92:0/454535040 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8da0054240 con 0x7f8da80d1d70 2023-11-28T11:22:22.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.473+0000 7f8db05ff700 1 -- 172.21.15.92:0/454535040 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f8da80d2940 con 0x7f8da80d1d70 2023-11-28T11:22:22.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.474+0000 7f8d9effd700 1 -- 172.21.15.92:0/454535040 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v38) v1 ==== 84+0+126 (secure 0 0 0) 0x7f8da0057860 con 0x7f8da80d1d70 2023-11-28T11:22:22.475 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:22:22.475 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-11-28T11:22:22.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:22:22.475 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 1, 2023-11-28T11:22:22.475 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": ".mgr" 2023-11-28T11:22:22.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:22:22.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:22:22.475 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 2, 2023-11-28T11:22:22.475 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": "rbd" 2023-11-28T11:22:22.475 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:22:22.475 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-11-28T11:22:22.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.476+0000 7f8db05ff700 1 -- 172.21.15.92:0/454535040 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8d8c0682f0 msgr2=0x7f8d8c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:22.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.476+0000 7f8db05ff700 1 --2- 172.21.15.92:0/454535040 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8d8c0682f0 0x7f8d8c06a7a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8da4005910 tx=0x7f8da4009040 comp rx=0 tx=0).stop 2023-11-28T11:22:22.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.476+0000 7f8db05ff700 1 -- 172.21.15.92:0/454535040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8da80d1d70 msgr2=0x7f8da815a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:22.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.476+0000 7f8db05ff700 1 --2- 172.21.15.92:0/454535040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8da80d1d70 0x7f8da815a360 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f8da000b680 tx=0x7f8da0004ad0 comp rx=0 tx=0).stop 2023-11-28T11:22:22.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.476+0000 7f8db05ff700 1 -- 172.21.15.92:0/454535040 shutdown_connections 2023-11-28T11:22:22.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.476+0000 7f8db05ff700 1 --2- 172.21.15.92:0/454535040 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8d8c0682f0 0x7f8d8c06a7a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:22.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.476+0000 7f8db05ff700 1 --2- 172.21.15.92:0/454535040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8da80d1d70 0x7f8da815a360 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:22.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.476+0000 7f8db05ff700 1 --2- 172.21.15.92:0/454535040 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8da80b8510 0x7f8da81e27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:22.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.476+0000 7f8db05ff700 1 --2- 172.21.15.92:0/454535040 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8da80cd460 0x7f8da81e7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:22.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.476+0000 7f8db05ff700 1 -- 172.21.15.92:0/454535040 >> 172.21.15.92:0/454535040 conn(0x7f8da80719a0 msgr2=0x7f8da80b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:22.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.476+0000 7f8db05ff700 1 -- 172.21.15.92:0/454535040 shutdown_connections 2023-11-28T11:22:22.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.476+0000 7f8db05ff700 1 -- 172.21.15.92:0/454535040 wait complete. 2023-11-28T11:22:22.495 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T11:22:22.495 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-28T11:22:22.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.694+0000 7f7d6cfff700 1 Processor -- start 2023-11-28T11:22:22.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.695+0000 7f7d6cfff700 1 -- start start 2023-11-28T11:22:22.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.696+0000 7f7d6cfff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d6806c050 0x7f7d68167410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:22.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.696+0000 7f7d6cfff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d680b5240 con 0x7f7d680b8790 2023-11-28T11:22:22.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.696+0000 7f7d6cfff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d680b5520 con 0x7f7d680cd460 2023-11-28T11:22:22.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.696+0000 7f7d6cfff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d680d0a80 con 0x7f7d6806c050 2023-11-28T11:22:22.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.696+0000 7f7d6659c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f7d680b8790 0x7f7d680cfb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33772/0 (socket says 172.21.15.92:33772) 2023-11-28T11:22:22.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.696+0000 7f7d6659c700 1 -- 172.21.15.92:0/3118017334 learned_addr learned my addr 172.21.15.92:0/3118017334 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:22:22.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.696+0000 7f7d66d9d700 1 --2- 172.21.15.92:0/3118017334 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d6806c050 0x7f7d68167410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:22.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.696+0000 7f7d65d9b700 1 -- 172.21.15.92:0/3118017334 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3698540404 0 0) 0x7f7d680b5240 con 0x7f7d680b8790 2023-11-28T11:22:22.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.696+0000 7f7d65d9b700 1 -- 172.21.15.92:0/3118017334 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d540035b0 con 0x7f7d680b8790 2023-11-28T11:22:22.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.697+0000 7f7d65d9b700 1 -- 172.21.15.92:0/3118017334 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2321496320 0 0) 0x7f7d540035b0 con 0x7f7d680b8790 2023-11-28T11:22:22.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.697+0000 7f7d65d9b700 1 -- 172.21.15.92:0/3118017334 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d6806c050 msgr2=0x7f7d68167410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:22.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.697+0000 7f7d65d9b700 1 --2- 172.21.15.92:0/3118017334 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d6806c050 0x7f7d68167410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:22.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.697+0000 7f7d65d9b700 1 -- 172.21.15.92:0/3118017334 >> v1:172.21.15.169:6789/0 conn(0x7f7d680cd460 legacy=0x7f7d6806b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:22.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.697+0000 7f7d65d9b700 1 -- 172.21.15.92:0/3118017334 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d680b5800 con 0x7f7d680b8790 2023-11-28T11:22:22.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.697+0000 7f7d65d9b700 1 -- 172.21.15.92:0/3118017334 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532161587 0 0) 0x7f7d58001670 con 0x7f7d680b8790 2023-11-28T11:22:22.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.697+0000 7f7d65d9b700 1 -- 172.21.15.92:0/3118017334 >> v1:172.21.15.92:6789/0 conn(0x7f7d680b8790 legacy=0x7f7d680cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:22.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.697+0000 7f7d65d9b700 1 --2- 172.21.15.92:0/3118017334 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d6806c050 0x7f7d54003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:22.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.697+0000 7f7d65d9b700 1 --2- 172.21.15.92:0/3118017334 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d54003f70 0x7f7d54004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:22.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.697+0000 7f7d65d9b700 1 --2- 172.21.15.92:0/3118017334 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d54004d90 0x7f7d54005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:22.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.697+0000 7f7d65d9b700 1 -- 172.21.15.92:0/3118017334 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d680d0a80 con 0x7f7d54003f70 2023-11-28T11:22:22.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.697+0000 7f7d65d9b700 1 -- 172.21.15.92:0/3118017334 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5400ba40 con 0x7f7d54004d90 2023-11-28T11:22:22.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.697+0000 7f7d65d9b700 1 -- 172.21.15.92:0/3118017334 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5400bc00 con 0x7f7d6806c050 2023-11-28T11:22:22.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.697+0000 7f7d5ffff700 1 --2- 172.21.15.92:0/3118017334 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d54003f70 0x7f7d54004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:22.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.697+0000 7f7d6659c700 1 --2- 172.21.15.92:0/3118017334 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d6806c050 0x7f7d54003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:22.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.697+0000 7f7d66d9d700 1 --2- 172.21.15.92:0/3118017334 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d54004d90 0x7f7d54005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:22.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.698+0000 7f7d6659c700 1 -- 172.21.15.92:0/3118017334 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d54004d90 msgr2=0x7f7d54005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:22.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.698+0000 7f7d6659c700 1 --2- 172.21.15.92:0/3118017334 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d54004d90 0x7f7d54005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:22.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.698+0000 7f7d6659c700 1 -- 172.21.15.92:0/3118017334 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d54003f70 msgr2=0x7f7d54004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:22.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.698+0000 7f7d6659c700 1 --2- 172.21.15.92:0/3118017334 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d54003f70 0x7f7d54004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:22.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.698+0000 7f7d6659c700 1 -- 172.21.15.92:0/3118017334 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d540035b0 con 0x7f7d6806c050 2023-11-28T11:22:22.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.698+0000 7f7d5ffff700 1 --2- 172.21.15.92:0/3118017334 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d54003f70 0x7f7d54004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:22:22.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.698+0000 7f7d6659c700 1 --2- 172.21.15.92:0/3118017334 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d6806c050 0x7f7d54003a30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7d5800c240 tx=0x7f7d54005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e0480a6fb240c4f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:22.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.698+0000 7f7d65d9b700 1 -- 172.21.15.92:0/3118017334 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5800cd80 con 0x7f7d6806c050 2023-11-28T11:22:22.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.698+0000 7f7d65d9b700 1 -- 172.21.15.92:0/3118017334 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d58005960 con 0x7f7d6806c050 2023-11-28T11:22:22.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.698+0000 7f7d65d9b700 1 -- 172.21.15.92:0/3118017334 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d58005d10 con 0x7f7d6806c050 2023-11-28T11:22:22.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.699+0000 7f7d6cfff700 1 -- 172.21.15.92:0/3118017334 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d6806c050 msgr2=0x7f7d54003a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:22.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.699+0000 7f7d6cfff700 1 --2- 172.21.15.92:0/3118017334 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d6806c050 0x7f7d54003a30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7d5800c240 tx=0x7f7d54005880 comp rx=0 tx=0).stop 2023-11-28T11:22:22.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.699+0000 7f7d66d9d700 1 -- 172.21.15.92:0/3118017334 reap_dead start 2023-11-28T11:22:22.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.699+0000 7f7d6cfff700 1 -- 172.21.15.92:0/3118017334 shutdown_connections 2023-11-28T11:22:22.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.699+0000 7f7d6cfff700 1 -- 172.21.15.92:0/3118017334 >> 172.21.15.92:0/3118017334 conn(0x7f7d680719a0 msgr2=0x7f7d68071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:22.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.699+0000 7f7d6cfff700 1 -- 172.21.15.92:0/3118017334 shutdown_connections 2023-11-28T11:22:22.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.699+0000 7f7d6cfff700 1 -- 172.21.15.92:0/3118017334 wait complete. 2023-11-28T11:22:22.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.701+0000 7f7d6cfff700 1 Processor -- start 2023-11-28T11:22:22.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.701+0000 7f7d6cfff700 1 -- start start 2023-11-28T11:22:22.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.702+0000 7f7d6cfff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d6806c050 0x7f7d6814fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:22.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.702+0000 7f7d6cfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d680d2e30 0x7f7d680d32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:22.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.702+0000 7f7d6cfff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d6815e710 0x7f7d6815ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:22.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.702+0000 7f7d6cfff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d680d37e0 con 0x7f7d680d2e30 2023-11-28T11:22:22.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.702+0000 7f7d6cfff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6815f0f0 con 0x7f7d6815e710 2023-11-28T11:22:22.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.702+0000 7f7d6cfff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d68150770 con 0x7f7d6806c050 2023-11-28T11:22:22.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.702+0000 7f7d5ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d680d2e30 0x7f7d680d32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:22.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.702+0000 7f7d5ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d680d2e30 0x7f7d680d32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51218/0 (socket says 172.21.15.92:51218) 2023-11-28T11:22:22.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.702+0000 7f7d5ffff700 1 -- 172.21.15.92:0/1478100163 learned_addr learned my addr 172.21.15.92:0/1478100163 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:22.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.702+0000 7f7d6659c700 1 --2- 172.21.15.92:0/1478100163 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d6806c050 0x7f7d6814fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:22.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.702+0000 7f7d66d9d700 1 --2- 172.21.15.92:0/1478100163 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d6815e710 0x7f7d6815ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:22.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.702+0000 7f7d5ffff700 1 -- 172.21.15.92:0/1478100163 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d6806c050 msgr2=0x7f7d6814fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:22.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.702+0000 7f7d5ffff700 1 --2- 172.21.15.92:0/1478100163 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d6806c050 0x7f7d6814fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:22.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.702+0000 7f7d5ffff700 1 -- 172.21.15.92:0/1478100163 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d6815e710 msgr2=0x7f7d6815ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:22.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.702+0000 7f7d5ffff700 1 --2- 172.21.15.92:0/1478100163 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d6815e710 0x7f7d6815ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:22.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.702+0000 7f7d5ffff700 1 -- 172.21.15.92:0/1478100163 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d680d2140 con 0x7f7d680d2e30 2023-11-28T11:22:22.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.702+0000 7f7d66d9d700 1 --2- 172.21.15.92:0/1478100163 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d6815e710 0x7f7d6815ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:22:22.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.703+0000 7f7d5ffff700 1 --2- 172.21.15.92:0/1478100163 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d680d2e30 0x7f7d680d32a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7d5000fb30 tx=0x7f7d5000fe40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:22.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.703+0000 7f7d5effd700 1 -- 172.21.15.92:0/1478100163 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d500052e0 con 0x7f7d680d2e30 2023-11-28T11:22:22.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.703+0000 7f7d5effd700 1 -- 172.21.15.92:0/1478100163 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d50005440 con 0x7f7d680d2e30 2023-11-28T11:22:22.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.703+0000 7f7d6cfff700 1 -- 172.21.15.92:0/1478100163 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d680d1fb0 con 0x7f7d680d2e30 2023-11-28T11:22:22.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.703+0000 7f7d5effd700 1 -- 172.21.15.92:0/1478100163 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d50003880 con 0x7f7d680d2e30 2023-11-28T11:22:22.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.703+0000 7f7d6cfff700 1 -- 172.21.15.92:0/1478100163 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d681501d0 con 0x7f7d680d2e30 2023-11-28T11:22:22.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.704+0000 7f7d6cfff700 1 -- 172.21.15.92:0/1478100163 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d680d2aa0 con 0x7f7d680d2e30 2023-11-28T11:22:22.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.705+0000 7f7d5effd700 1 -- 172.21.15.92:0/1478100163 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7d500039e0 con 0x7f7d680d2e30 2023-11-28T11:22:22.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.706+0000 7f7d5effd700 1 --2- 172.21.15.92:0/1478100163 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7d480682f0 0x7f7d4806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:22.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.706+0000 7f7d5effd700 1 -- 172.21.15.92:0/1478100163 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f7d50085ba0 con 0x7f7d680d2e30 2023-11-28T11:22:22.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.706+0000 7f7d6659c700 1 --2- 172.21.15.92:0/1478100163 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7d480682f0 0x7f7d4806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:22.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.706+0000 7f7d6659c700 1 --2- 172.21.15.92:0/1478100163 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7d480682f0 0x7f7d4806a7a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7d58015040 tx=0x7f7d5800a000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:22.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.710+0000 7f7d5effd700 1 -- 172.21.15.92:0/1478100163 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7d680d2aa0 con 0x7f7d680d2e30 2023-11-28T11:22:22.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:22.902+0000 7f7d6cfff700 1 -- 172.21.15.92:0/1478100163 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f7d680d2aa0 con 0x7f7d680d2e30 2023-11-28T11:22:23.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.210+0000 7f7d5effd700 1 -- 172.21.15.92:0/1478100163 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v39) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7d680d2aa0 con 0x7f7d680d2e30 2023-11-28T11:22:23.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.214+0000 7f7d6cfff700 1 -- 172.21.15.92:0/1478100163 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7d480682f0 msgr2=0x7f7d4806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:23.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.214+0000 7f7d6cfff700 1 --2- 172.21.15.92:0/1478100163 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7d480682f0 0x7f7d4806a7a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7d58015040 tx=0x7f7d5800a000 comp rx=0 tx=0).stop 2023-11-28T11:22:23.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.214+0000 7f7d6cfff700 1 -- 172.21.15.92:0/1478100163 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d680d2e30 msgr2=0x7f7d680d32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:23.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.214+0000 7f7d6cfff700 1 --2- 172.21.15.92:0/1478100163 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d680d2e30 0x7f7d680d32a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7d5000fb30 tx=0x7f7d5000fe40 comp rx=0 tx=0).stop 2023-11-28T11:22:23.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.214+0000 7f7d6cfff700 1 -- 172.21.15.92:0/1478100163 shutdown_connections 2023-11-28T11:22:23.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.214+0000 7f7d6cfff700 1 --2- 172.21.15.92:0/1478100163 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d6815e710 0x7f7d6815ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:23.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.214+0000 7f7d6cfff700 1 --2- 172.21.15.92:0/1478100163 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d6806c050 0x7f7d6814fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:23.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.214+0000 7f7d6cfff700 1 --2- 172.21.15.92:0/1478100163 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7d480682f0 0x7f7d4806a7a0 secure :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7d58015040 tx=0x7f7d5800a000 comp rx=0 tx=0).stop 2023-11-28T11:22:23.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.214+0000 7f7d6cfff700 1 --2- 172.21.15.92:0/1478100163 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d680d2e30 0x7f7d680d32a0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:23.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.214+0000 7f7d6cfff700 1 -- 172.21.15.92:0/1478100163 >> 172.21.15.92:0/1478100163 conn(0x7f7d680719a0 msgr2=0x7f7d680b2120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:23.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.216+0000 7f7d6cfff700 1 -- 172.21.15.92:0/1478100163 shutdown_connections 2023-11-28T11:22:23.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.216+0000 7f7d6cfff700 1 -- 172.21.15.92:0/1478100163 wait complete. 2023-11-28T11:22:23.220 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' created 2023-11-28T11:22:23.237 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T11:22:23.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.439+0000 7efeb2bff700 1 Processor -- start 2023-11-28T11:22:23.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.439+0000 7efeb2bff700 1 -- start start 2023-11-28T11:22:23.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.440+0000 7efeb2bff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7efeac0b2280 0x7efeac0cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:23.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.440+0000 7efeb2bff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efeac0cd4d0 con 0x7efeac15d590 2023-11-28T11:22:23.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.440+0000 7efeb2bff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efeac0cf470 con 0x7efeac0d2e10 2023-11-28T11:22:23.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.440+0000 7efeb2bff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efeac0cdce0 con 0x7efeac0b2280 2023-11-28T11:22:23.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.441+0000 7efeabfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7efeac15d590 0x7efeac15dee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33784/0 (socket says 172.21.15.92:33784) 2023-11-28T11:22:23.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.441+0000 7efeabfff700 1 -- 172.21.15.92:0/3161754832 learned_addr learned my addr 172.21.15.92:0/3161754832 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:22:23.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.441+0000 7efeb119c700 1 --2- 172.21.15.92:0/3161754832 >> v2:172.21.15.92:3300/0 conn(0x7efeac0b2280 0x7efeac0cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:23.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.441+0000 7efeab7fe700 1 -- 172.21.15.92:0/3161754832 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2783558822 0 0) 0x7efeac0cd4d0 con 0x7efeac15d590 2023-11-28T11:22:23.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.441+0000 7efeab7fe700 1 -- 172.21.15.92:0/3161754832 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe9c0035b0 con 0x7efeac15d590 2023-11-28T11:22:23.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.441+0000 7efeb119c700 1 -- 172.21.15.92:0/3161754832 >> v1:172.21.15.169:6789/0 conn(0x7efeac0d2e10 legacy=0x7efeac0d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:23.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.441+0000 7efeab7fe700 1 -- 172.21.15.92:0/3161754832 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 59648513 0 0) 0x7efeac0cf470 con 0x7efeac0d2e10 2023-11-28T11:22:23.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.441+0000 7efeb119c700 1 -- 172.21.15.92:0/3161754832 >> v1:172.21.15.92:6789/0 conn(0x7efeac15d590 legacy=0x7efeac15dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:23.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.441+0000 7efeb119c700 1 -- 172.21.15.92:0/3161754832 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeac156fa0 con 0x7efeac0b2280 2023-11-28T11:22:23.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.442+0000 7efeb119c700 1 --2- 172.21.15.92:0/3161754832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efeac0b2280 0x7efeac0cf900 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7efea4009d90 tx=0x7efea400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ab854832c8a74e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:23.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.442+0000 7efeab7fe700 1 -- 172.21.15.92:0/3161754832 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea400e070 con 0x7efeac0b2280 2023-11-28T11:22:23.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.442+0000 7efeab7fe700 1 -- 172.21.15.92:0/3161754832 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efea400bb20 con 0x7efeac0b2280 2023-11-28T11:22:23.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.442+0000 7efeab7fe700 1 -- 172.21.15.92:0/3161754832 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea400bed0 con 0x7efeac0b2280 2023-11-28T11:22:23.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.442+0000 7efeb2bff700 1 -- 172.21.15.92:0/3161754832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efeac0b2280 msgr2=0x7efeac0cf900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:23.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.442+0000 7efeb2bff700 1 --2- 172.21.15.92:0/3161754832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efeac0b2280 0x7efeac0cf900 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7efea4009d90 tx=0x7efea400b330 comp rx=0 tx=0).stop 2023-11-28T11:22:23.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.443+0000 7efeb2bff700 1 -- 172.21.15.92:0/3161754832 shutdown_connections 2023-11-28T11:22:23.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.443+0000 7efeb2bff700 1 --2- 172.21.15.92:0/3161754832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efeac0b2280 0x7efeac0cf900 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:23.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.443+0000 7efeb2bff700 1 -- 172.21.15.92:0/3161754832 >> 172.21.15.92:0/3161754832 conn(0x7efeac0719a0 msgr2=0x7efeac071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:23.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.443+0000 7efeb2bff700 1 -- 172.21.15.92:0/3161754832 shutdown_connections 2023-11-28T11:22:23.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.443+0000 7efeb2bff700 1 -- 172.21.15.92:0/3161754832 wait complete. 2023-11-28T11:22:23.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.444+0000 7efeb2bff700 1 Processor -- start 2023-11-28T11:22:23.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.444+0000 7efeb2bff700 1 -- start start 2023-11-28T11:22:23.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.445+0000 7efeb2bff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efeac0b2280 0x7efeac0c6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:23.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.445+0000 7efeb2bff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efeac0d2e10 0x7efeac152b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:23.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.445+0000 7efeb2bff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efeac15d590 0x7efeac158fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:23.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.445+0000 7efeb2bff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeac0b87b0 con 0x7efeac15d590 2023-11-28T11:22:23.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.445+0000 7efeb2bff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeac0b8a90 con 0x7efeac0d2e10 2023-11-28T11:22:23.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.445+0000 7efeb2bff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeac1587e0 con 0x7efeac0b2280 2023-11-28T11:22:23.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.446+0000 7efeb119c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efeac15d590 0x7efeac158fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:23.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.446+0000 7efeb119c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efeac15d590 0x7efeac158fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51234/0 (socket says 172.21.15.92:51234) 2023-11-28T11:22:23.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.446+0000 7efeb119c700 1 -- 172.21.15.92:0/2579092054 learned_addr learned my addr 172.21.15.92:0/2579092054 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:23.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.446+0000 7efeb119c700 1 -- 172.21.15.92:0/2579092054 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efeac0b2280 msgr2=0x7efeac0c6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:23.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.446+0000 7efeb119c700 1 --2- 172.21.15.92:0/2579092054 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efeac0b2280 0x7efeac0c6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:23.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.446+0000 7efeb119c700 1 -- 172.21.15.92:0/2579092054 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efeac0d2e10 msgr2=0x7efeac152b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:23.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.446+0000 7efeb119c700 1 --2- 172.21.15.92:0/2579092054 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efeac0d2e10 0x7efeac152b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:23.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.446+0000 7efeb119c700 1 -- 172.21.15.92:0/2579092054 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea4009a70 con 0x7efeac15d590 2023-11-28T11:22:23.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.446+0000 7efeb119c700 1 --2- 172.21.15.92:0/2579092054 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efeac15d590 0x7efeac158fe0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7efea4014040 tx=0x7efea4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:23.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.446+0000 7efea97fa700 1 -- 172.21.15.92:0/2579092054 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea401b810 con 0x7efeac15d590 2023-11-28T11:22:23.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.447+0000 7efea97fa700 1 -- 172.21.15.92:0/2579092054 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efea4020070 con 0x7efeac15d590 2023-11-28T11:22:23.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.447+0000 7efeb2bff700 1 -- 172.21.15.92:0/2579092054 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeac0cd720 con 0x7efeac15d590 2023-11-28T11:22:23.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.447+0000 7efea97fa700 1 -- 172.21.15.92:0/2579092054 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea401bbb0 con 0x7efeac15d590 2023-11-28T11:22:23.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.447+0000 7efeb2bff700 1 -- 172.21.15.92:0/2579092054 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeac0b2680 con 0x7efeac15d590 2023-11-28T11:22:23.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.448+0000 7efeb2bff700 1 -- 172.21.15.92:0/2579092054 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efeac0cda00 con 0x7efeac15d590 2023-11-28T11:22:23.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.449+0000 7efea97fa700 1 -- 172.21.15.92:0/2579092054 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7efea401bd10 con 0x7efeac15d590 2023-11-28T11:22:23.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.450+0000 7efea97fa700 1 --2- 172.21.15.92:0/2579092054 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7efe900682f0 0x7efe9006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:23.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.450+0000 7efea97fa700 1 -- 172.21.15.92:0/2579092054 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5900+0+0 (secure 0 0 0) 0x7efea4022070 con 0x7efeac15d590 2023-11-28T11:22:23.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.450+0000 7efeb099b700 1 --2- 172.21.15.92:0/2579092054 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7efe900682f0 0x7efe9006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:23.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.451+0000 7efeb099b700 1 --2- 172.21.15.92:0/2579092054 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7efe900682f0 0x7efe9006a7a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7efea0007fd0 tx=0x7efea00079a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:23.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.454+0000 7efea97fa700 1 -- 172.21.15.92:0/2579092054 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7efea4058870 con 0x7efeac15d590 2023-11-28T11:22:23.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:23.644+0000 7efeb2bff700 1 -- 172.21.15.92:0/2579092054 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7efeac158cd0 con 0x7efeac15d590 2023-11-28T11:22:24.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.217+0000 7efea97fa700 1 -- 172.21.15.92:0/2579092054 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v40) v1 ==== 170+0+0 (secure 0 0 0) 0x7efea4058460 con 0x7efeac15d590 2023-11-28T11:22:24.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.223+0000 7efeb2bff700 1 -- 172.21.15.92:0/2579092054 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7efe900682f0 msgr2=0x7efe9006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:24.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.223+0000 7efeb2bff700 1 --2- 172.21.15.92:0/2579092054 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7efe900682f0 0x7efe9006a7a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7efea0007fd0 tx=0x7efea00079a0 comp rx=0 tx=0).stop 2023-11-28T11:22:24.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.223+0000 7efeb2bff700 1 -- 172.21.15.92:0/2579092054 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efeac15d590 msgr2=0x7efeac158fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:24.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.223+0000 7efeb2bff700 1 --2- 172.21.15.92:0/2579092054 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efeac15d590 0x7efeac158fe0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7efea4014040 tx=0x7efea4004ad0 comp rx=0 tx=0).stop 2023-11-28T11:22:24.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.224+0000 7efeb2bff700 1 -- 172.21.15.92:0/2579092054 shutdown_connections 2023-11-28T11:22:24.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.224+0000 7efeb2bff700 1 --2- 172.21.15.92:0/2579092054 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7efe900682f0 0x7efe9006a7a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:24.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.224+0000 7efeb2bff700 1 --2- 172.21.15.92:0/2579092054 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efeac15d590 0x7efeac158fe0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:24.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.224+0000 7efeb2bff700 1 --2- 172.21.15.92:0/2579092054 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efeac0b2280 0x7efeac0c6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:24.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.224+0000 7efeb2bff700 1 --2- 172.21.15.92:0/2579092054 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efeac0d2e10 0x7efeac152b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:24.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.224+0000 7efeb2bff700 1 -- 172.21.15.92:0/2579092054 >> 172.21.15.92:0/2579092054 conn(0x7efeac0719a0 msgr2=0x7efeac072cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:24.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.228+0000 7efeb2bff700 1 -- 172.21.15.92:0/2579092054 shutdown_connections 2023-11-28T11:22:24.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.229+0000 7efeb2bff700 1 -- 172.21.15.92:0/2579092054 wait complete. 2023-11-28T11:22:24.237 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' created 2023-11-28T11:22:24.261 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T11:22:24.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.460+0000 7f20eb5ff700 1 Processor -- start 2023-11-28T11:22:24.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.460+0000 7f20eb5ff700 1 -- start start 2023-11-28T11:22:24.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.461+0000 7f20eb5ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f20e40b8510 0x7f20e406b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:24.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.461+0000 7f20eb5ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f20e40cd460 0x7f20e406beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:24.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.461+0000 7f20eb5ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20e40b88e0 con 0x7f20e406c3f0 2023-11-28T11:22:24.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.462+0000 7f20eb5ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20e40cdbd0 con 0x7f20e40cd460 2023-11-28T11:22:24.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.462+0000 7f20eb5ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20e40d02d0 con 0x7f20e40b8510 2023-11-28T11:22:24.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.462+0000 7f20e8b9a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f20e40cd460 0x7f20e406beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:24.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.462+0000 7f20e8b9a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f20e40cd460 0x7f20e406beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51238/0 (socket says 172.21.15.92:51238) 2023-11-28T11:22:24.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.462+0000 7f20e8b9a700 1 -- 172.21.15.92:0/2589224098 learned_addr learned my addr 172.21.15.92:0/2589224098 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:24.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.462+0000 7f20e939b700 1 --2- 172.21.15.92:0/2589224098 >> v2:172.21.15.169:3300/0 conn(0x7f20e40b8510 0x7f20e406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:24.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.462+0000 7f20e8b9a700 1 -- 172.21.15.92:0/2589224098 >> v2:172.21.15.169:3300/0 conn(0x7f20e40b8510 msgr2=0x7f20e406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:24.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.462+0000 7f20e8b9a700 1 --2- 172.21.15.92:0/2589224098 >> v2:172.21.15.169:3300/0 conn(0x7f20e40b8510 0x7f20e406b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:24.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.462+0000 7f20e8b9a700 1 -- 172.21.15.92:0/2589224098 >> v1:172.21.15.169:6789/0 conn(0x7f20e406c3f0 legacy=0x7f20e415cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:24.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.462+0000 7f20e8b9a700 1 -- 172.21.15.92:0/2589224098 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e40cd8f0 con 0x7f20e40cd460 2023-11-28T11:22:24.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.463+0000 7f20e8b9a700 1 --2- 172.21.15.92:0/2589224098 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20e40cd460 0x7f20e406beb0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f20d4009920 tx=0x7f20d4009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b73916672d1d5b7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:24.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.463+0000 7f20dbfff700 1 -- 172.21.15.92:0/2589224098 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d400b9e0 con 0x7f20e40cd460 2023-11-28T11:22:24.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.463+0000 7f20dbfff700 1 -- 172.21.15.92:0/2589224098 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20d400bb40 con 0x7f20e40cd460 2023-11-28T11:22:24.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.463+0000 7f20dbfff700 1 -- 172.21.15.92:0/2589224098 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d4009e70 con 0x7f20e40cd460 2023-11-28T11:22:24.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.463+0000 7f20eb5ff700 1 -- 172.21.15.92:0/2589224098 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20e40cd460 msgr2=0x7f20e406beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:24.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.463+0000 7f20eb5ff700 1 --2- 172.21.15.92:0/2589224098 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20e40cd460 0x7f20e406beb0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f20d4009920 tx=0x7f20d4009c30 comp rx=0 tx=0).stop 2023-11-28T11:22:24.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.464+0000 7f20eb5ff700 1 -- 172.21.15.92:0/2589224098 shutdown_connections 2023-11-28T11:22:24.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.464+0000 7f20eb5ff700 1 --2- 172.21.15.92:0/2589224098 >> v2:172.21.15.169:3300/0 conn(0x7f20e40b8510 0x7f20e406b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:24.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.464+0000 7f20eb5ff700 1 --2- 172.21.15.92:0/2589224098 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20e40cd460 0x7f20e406beb0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:24.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.464+0000 7f20eb5ff700 1 -- 172.21.15.92:0/2589224098 >> 172.21.15.92:0/2589224098 conn(0x7f20e40719a0 msgr2=0x7f20e4071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:24.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.464+0000 7f20eb5ff700 1 -- 172.21.15.92:0/2589224098 shutdown_connections 2023-11-28T11:22:24.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.464+0000 7f20eb5ff700 1 -- 172.21.15.92:0/2589224098 wait complete. 2023-11-28T11:22:24.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.465+0000 7f20eb5ff700 1 Processor -- start 2023-11-28T11:22:24.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.465+0000 7f20eb5ff700 1 -- start start 2023-11-28T11:22:24.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.466+0000 7f20eb5ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20e406c3f0 0x7f20e40cb470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:24.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.466+0000 7f20eb5ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20e40b8510 0x7f20e41e2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:24.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.466+0000 7f20eb5ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20e40cd460 0x7f20e40c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:24.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.466+0000 7f20eb5ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e40c9490 con 0x7f20e40cd460 2023-11-28T11:22:24.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.466+0000 7f20eb5ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e40cbb50 con 0x7f20e40b8510 2023-11-28T11:22:24.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.466+0000 7f20eb5ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20e40cbe30 con 0x7f20e406c3f0 2023-11-28T11:22:24.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.467+0000 7f20e9b9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20e40cd460 0x7f20e40c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:24.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.467+0000 7f20e9b9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20e40cd460 0x7f20e40c8f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51246/0 (socket says 172.21.15.92:51246) 2023-11-28T11:22:24.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.467+0000 7f20e9b9c700 1 -- 172.21.15.92:0/1010078506 learned_addr learned my addr 172.21.15.92:0/1010078506 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:24.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.467+0000 7f20e8b9a700 1 --2- 172.21.15.92:0/1010078506 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20e40b8510 0x7f20e41e2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:24.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.467+0000 7f20e939b700 1 --2- 172.21.15.92:0/1010078506 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20e406c3f0 0x7f20e40cb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:24.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.467+0000 7f20e9b9c700 1 -- 172.21.15.92:0/1010078506 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20e406c3f0 msgr2=0x7f20e40cb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:24.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.467+0000 7f20e9b9c700 1 --2- 172.21.15.92:0/1010078506 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20e406c3f0 0x7f20e40cb470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:24.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.467+0000 7f20e9b9c700 1 -- 172.21.15.92:0/1010078506 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20e40b8510 msgr2=0x7f20e41e2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:24.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.467+0000 7f20e9b9c700 1 --2- 172.21.15.92:0/1010078506 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20e40b8510 0x7f20e41e2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:24.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.467+0000 7f20e9b9c700 1 -- 172.21.15.92:0/1010078506 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d4009550 con 0x7f20e40cd460 2023-11-28T11:22:24.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.467+0000 7f20e939b700 1 --2- 172.21.15.92:0/1010078506 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20e406c3f0 0x7f20e40cb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:22:24.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.467+0000 7f20e9b9c700 1 --2- 172.21.15.92:0/1010078506 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20e40cd460 0x7f20e40c8f50 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f20dc00cc50 tx=0x7f20dc00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:24.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.468+0000 7f20d9ffb700 1 -- 172.21.15.92:0/1010078506 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20dc00ad80 con 0x7f20e40cd460 2023-11-28T11:22:24.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.468+0000 7f20d9ffb700 1 -- 172.21.15.92:0/1010078506 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20dc00ce50 con 0x7f20e40cd460 2023-11-28T11:22:24.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.468+0000 7f20eb5ff700 1 -- 172.21.15.92:0/1010078506 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20e40cfff0 con 0x7f20e40cd460 2023-11-28T11:22:24.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.468+0000 7f20d9ffb700 1 -- 172.21.15.92:0/1010078506 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20dc003c40 con 0x7f20e40cd460 2023-11-28T11:22:24.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.468+0000 7f20eb5ff700 1 -- 172.21.15.92:0/1010078506 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20e40cfd10 con 0x7f20e40cd460 2023-11-28T11:22:24.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.469+0000 7f20eb5ff700 1 -- 172.21.15.92:0/1010078506 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20e4067880 con 0x7f20e40cd460 2023-11-28T11:22:24.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.470+0000 7f20d9ffb700 1 -- 172.21.15.92:0/1010078506 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f20dc010050 con 0x7f20e40cd460 2023-11-28T11:22:24.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.470+0000 7f20d9ffb700 1 --2- 172.21.15.92:0/1010078506 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f20c80682f0 0x7f20c806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:24.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.471+0000 7f20d9ffb700 1 -- 172.21.15.92:0/1010078506 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6282+0+0 (secure 0 0 0) 0x7f20dc085d60 con 0x7f20e40cd460 2023-11-28T11:22:24.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.471+0000 7f20e939b700 1 --2- 172.21.15.92:0/1010078506 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f20c80682f0 0x7f20c806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:24.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.471+0000 7f20e939b700 1 --2- 172.21.15.92:0/1010078506 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f20c80682f0 0x7f20c806a7a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f20e0009a00 tx=0x7f20e0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:24.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.475+0000 7f20d9ffb700 1 -- 172.21.15.92:0/1010078506 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f20dc052480 con 0x7f20e40cd460 2023-11-28T11:22:24.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.713+0000 7f20eb5ff700 1 -- 172.21.15.92:0/1010078506 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f20e40d2940 con 0x7f20e40cd460 2023-11-28T11:22:24.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.723+0000 7f20d9ffb700 1 -- 172.21.15.92:0/1010078506 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7f20dc055aa0 con 0x7f20e40cd460 2023-11-28T11:22:24.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.729+0000 7f20eb5ff700 1 -- 172.21.15.92:0/1010078506 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f20c80682f0 msgr2=0x7f20c806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:24.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.729+0000 7f20eb5ff700 1 --2- 172.21.15.92:0/1010078506 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f20c80682f0 0x7f20c806a7a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f20e0009a00 tx=0x7f20e0008040 comp rx=0 tx=0).stop 2023-11-28T11:22:24.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.729+0000 7f20eb5ff700 1 -- 172.21.15.92:0/1010078506 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20e40cd460 msgr2=0x7f20e40c8f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:24.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.729+0000 7f20eb5ff700 1 --2- 172.21.15.92:0/1010078506 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20e40cd460 0x7f20e40c8f50 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f20dc00cc50 tx=0x7f20dc00a340 comp rx=0 tx=0).stop 2023-11-28T11:22:24.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.736+0000 7f20eb5ff700 1 -- 172.21.15.92:0/1010078506 shutdown_connections 2023-11-28T11:22:24.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.736+0000 7f20eb5ff700 1 --2- 172.21.15.92:0/1010078506 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f20c80682f0 0x7f20c806a7a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:24.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.736+0000 7f20eb5ff700 1 --2- 172.21.15.92:0/1010078506 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20e40cd460 0x7f20e40c8f50 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:24.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.736+0000 7f20eb5ff700 1 --2- 172.21.15.92:0/1010078506 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20e406c3f0 0x7f20e40cb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:24.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.736+0000 7f20eb5ff700 1 --2- 172.21.15.92:0/1010078506 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20e40b8510 0x7f20e41e2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:24.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.736+0000 7f20eb5ff700 1 -- 172.21.15.92:0/1010078506 >> 172.21.15.92:0/1010078506 conn(0x7f20e40719a0 msgr2=0x7f20e40b5d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:24.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.736+0000 7f20eb5ff700 1 -- 172.21.15.92:0/1010078506 shutdown_connections 2023-11-28T11:22:24.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.737+0000 7f20eb5ff700 1 -- 172.21.15.92:0/1010078506 wait complete. 2023-11-28T11:22:24.741 INFO:teuthology.orchestra.run.smithi092.stderr:new fs with metadata pool 6 and data pool 7 2023-11-28T11:22:24.769 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:22:24.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.964+0000 7f47bf3ff700 1 Processor -- start 2023-11-28T11:22:24.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.964+0000 7f47bf3ff700 1 -- start start 2023-11-28T11:22:24.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.965+0000 7f47bf3ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f47b80cd460 0x7f47b806c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:24.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.965+0000 7f47bf3ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47b80d0d60 0x7f47b80d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:24.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.966+0000 7f47bf3ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47b80b88e0 con 0x7f47b80b8510 2023-11-28T11:22:24.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.966+0000 7f47bf3ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47b80cdbd0 con 0x7f47b80cd460 2023-11-28T11:22:24.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.966+0000 7f47bf3ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47b80d02d0 con 0x7f47b80d0d60 2023-11-28T11:22:24.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.966+0000 7f47bd19b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f47b80b8510 0x7f47b806b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33794/0 (socket says 172.21.15.92:33794) 2023-11-28T11:22:24.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.966+0000 7f47bd19b700 1 -- 172.21.15.92:0/365845003 learned_addr learned my addr 172.21.15.92:0/365845003 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:22:24.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.966+0000 7f47bc99a700 1 --2- 172.21.15.92:0/365845003 >> v2:172.21.15.92:3300/0 conn(0x7f47b80cd460 0x7f47b806c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:24.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.966+0000 7f47bd99c700 1 --2- 172.21.15.92:0/365845003 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47b80d0d60 0x7f47b80d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:24.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.966+0000 7f47affff700 1 -- 172.21.15.92:0/365845003 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2385106744 0 0) 0x7f47b80b88e0 con 0x7f47b80b8510 2023-11-28T11:22:24.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.966+0000 7f47affff700 1 -- 172.21.15.92:0/365845003 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47a40035b0 con 0x7f47b80b8510 2023-11-28T11:22:24.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.966+0000 7f47bc99a700 1 -- 172.21.15.92:0/365845003 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47b80d0d60 msgr2=0x7f47b80d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:24.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.966+0000 7f47bc99a700 1 --2- 172.21.15.92:0/365845003 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47b80d0d60 0x7f47b80d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:24.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.966+0000 7f47bc99a700 1 -- 172.21.15.92:0/365845003 >> v1:172.21.15.92:6789/0 conn(0x7f47b80b8510 legacy=0x7f47b806b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:24.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.966+0000 7f47bc99a700 1 -- 172.21.15.92:0/365845003 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b80cd8f0 con 0x7f47b80cd460 2023-11-28T11:22:24.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.967+0000 7f47bc99a700 1 --2- 172.21.15.92:0/365845003 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47b80cd460 0x7f47b806c080 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f47a8009870 tx=0x7f47a8009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c6126706ac4e59 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:24.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.967+0000 7f47affff700 1 -- 172.21.15.92:0/365845003 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a8011830 con 0x7f47b80cd460 2023-11-28T11:22:24.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.967+0000 7f47affff700 1 -- 172.21.15.92:0/365845003 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47a8011990 con 0x7f47b80cd460 2023-11-28T11:22:24.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.967+0000 7f47affff700 1 -- 172.21.15.92:0/365845003 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a8011d20 con 0x7f47b80cd460 2023-11-28T11:22:24.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.967+0000 7f47bf3ff700 1 -- 172.21.15.92:0/365845003 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47b80cd460 msgr2=0x7f47b806c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:24.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.967+0000 7f47bf3ff700 1 --2- 172.21.15.92:0/365845003 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47b80cd460 0x7f47b806c080 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f47a8009870 tx=0x7f47a8009b80 comp rx=0 tx=0).stop 2023-11-28T11:22:24.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.968+0000 7f47bf3ff700 1 -- 172.21.15.92:0/365845003 shutdown_connections 2023-11-28T11:22:24.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.968+0000 7f47bf3ff700 1 --2- 172.21.15.92:0/365845003 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47b80d0d60 0x7f47b80d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:24.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.968+0000 7f47bf3ff700 1 --2- 172.21.15.92:0/365845003 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47b80cd460 0x7f47b806c080 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:24.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.968+0000 7f47bf3ff700 1 -- 172.21.15.92:0/365845003 >> 172.21.15.92:0/365845003 conn(0x7f47b80719a0 msgr2=0x7f47b8071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:24.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.968+0000 7f47bf3ff700 1 -- 172.21.15.92:0/365845003 shutdown_connections 2023-11-28T11:22:24.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.968+0000 7f47bf3ff700 1 -- 172.21.15.92:0/365845003 wait complete. 2023-11-28T11:22:24.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.969+0000 7f47bf3ff700 1 Processor -- start 2023-11-28T11:22:24.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.969+0000 7f47bf3ff700 1 -- start start 2023-11-28T11:22:24.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.970+0000 7f47bf3ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47b80b8510 0x7f47b80c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:24.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.970+0000 7f47bf3ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47b80cd460 0x7f47b8157e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:24.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.970+0000 7f47bf3ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47b80d0d60 0x7f47b8157250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:24.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.970+0000 7f47bf3ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b80cb1d0 con 0x7f47b80cd460 2023-11-28T11:22:24.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.970+0000 7f47bf3ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b80ca5c0 con 0x7f47b80b8510 2023-11-28T11:22:24.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.970+0000 7f47bf3ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47b80ca8a0 con 0x7f47b80d0d60 2023-11-28T11:22:24.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.971+0000 7f47bc99a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47b80cd460 0x7f47b8157e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:24.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.971+0000 7f47bc99a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47b80cd460 0x7f47b8157e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51270/0 (socket says 172.21.15.92:51270) 2023-11-28T11:22:24.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.971+0000 7f47bc99a700 1 -- 172.21.15.92:0/1261069598 learned_addr learned my addr 172.21.15.92:0/1261069598 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:24.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.971+0000 7f47bd99c700 1 --2- 172.21.15.92:0/1261069598 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47b80d0d60 0x7f47b8157250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:24.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.971+0000 7f47bc99a700 1 -- 172.21.15.92:0/1261069598 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47b80d0d60 msgr2=0x7f47b8157250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:24.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.971+0000 7f47bc99a700 1 --2- 172.21.15.92:0/1261069598 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47b80d0d60 0x7f47b8157250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:24.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.971+0000 7f47bc99a700 1 -- 172.21.15.92:0/1261069598 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47b80b8510 msgr2=0x7f47b80c9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:24.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.971+0000 7f47bc99a700 1 --2- 172.21.15.92:0/1261069598 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47b80b8510 0x7f47b80c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:24.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.971+0000 7f47bc99a700 1 -- 172.21.15.92:0/1261069598 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a80094a0 con 0x7f47b80cd460 2023-11-28T11:22:24.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.971+0000 7f47bc99a700 1 --2- 172.21.15.92:0/1261069598 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47b80cd460 0x7f47b8157e60 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f47a8000c00 tx=0x7f47a8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:24.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.971+0000 7f47adffb700 1 -- 172.21.15.92:0/1261069598 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a80036a0 con 0x7f47b80cd460 2023-11-28T11:22:24.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.971+0000 7f47adffb700 1 -- 172.21.15.92:0/1261069598 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47a801f070 con 0x7f47b80cd460 2023-11-28T11:22:24.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.971+0000 7f47adffb700 1 -- 172.21.15.92:0/1261069598 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a8003a40 con 0x7f47b80cd460 2023-11-28T11:22:24.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.972+0000 7f47bf3ff700 1 -- 172.21.15.92:0/1261069598 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47b80cfff0 con 0x7f47b80cd460 2023-11-28T11:22:24.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.972+0000 7f47bf3ff700 1 -- 172.21.15.92:0/1261069598 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47b80cfd10 con 0x7f47b80cd460 2023-11-28T11:22:24.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.973+0000 7f47bf3ff700 1 -- 172.21.15.92:0/1261069598 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47b8067880 con 0x7f47b80cd460 2023-11-28T11:22:24.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.974+0000 7f47adffb700 1 -- 172.21.15.92:0/1261069598 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f47a800f050 con 0x7f47b80cd460 2023-11-28T11:22:24.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.974+0000 7f47adffb700 1 --2- 172.21.15.92:0/1261069598 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f479c0682f0 0x7f479c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:24.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.974+0000 7f47adffb700 1 -- 172.21.15.92:0/1261069598 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f47a8021070 con 0x7f47b80cd460 2023-11-28T11:22:24.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.975+0000 7f47bd19b700 1 --2- 172.21.15.92:0/1261069598 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f479c0682f0 0x7f479c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:24.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.975+0000 7f47bd19b700 1 --2- 172.21.15.92:0/1261069598 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f479c0682f0 0x7f479c06a7a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f47b40016b0 tx=0x7f47b40079c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:24.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:24.979+0000 7f47adffb700 1 -- 172.21.15.92:0/1261069598 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f47a8053da0 con 0x7f47b80cd460 2023-11-28T11:22:25.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.171+0000 7f47bf3ff700 1 -- 172.21.15.92:0/1261069598 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f47b80cb310 con 0x7f47b80cd460 2023-11-28T11:22:25.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.173+0000 7f47adffb700 1 -- 172.21.15.92:0/1261069598 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+18141 (secure 0 0 0) 0x7f47b80cb310 con 0x7f47b80cd460 2023-11-28T11:22:25.175 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:22:25.175 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":41,"fsid":"68956e0c-c16f-47cd-a220-f98364b0831d","created":"2023-11-28T11:20:28.697573+0000","modified":"2023-11-28T11:22:24.715083+0000","last_up_change":"2023-11-28T11:20:41.257011+0000","last_in_change":"2023-11-28T11:20:32.249767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:20:41.925237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:20:48.682497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:22:22.942125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-28T11:22:23.684528+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6abe483f-0e03-4a26-b9cf-62041291747e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":115823},{"type":"v1","addr":"172.21.15.92:6809","nonce":115823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":115823},{"type":"v1","addr":"172.21.15.92:6813","nonce":115823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":115823},{"type":"v1","addr":"172.21.15.92:6821","nonce":115823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":115823},{"type":"v1","addr":"172.21.15.92:6817","nonce":115823}]},"public_addr":"172.21.15.92:6809/115823","cluster_addr":"172.21.15.92:6813/115823","heartbeat_back_addr":"172.21.15.92:6821/115823","heartbeat_front_addr":"172.21.15.92:6817/115823","state":["exists","up"]},{"osd":1,"uuid":"13c1de73-1a8c-4d7e-8a28-9dae52dfe089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":115824},{"type":"v1","addr":"172.21.15.92:6812","nonce":115824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":115824},{"type":"v1","addr":"172.21.15.92:6816","nonce":115824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":115824},{"type":"v1","addr":"172.21.15.92:6823","nonce":115824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":115824},{"type":"v1","addr":"172.21.15.92:6820","nonce":115824}]},"public_addr":"172.21.15.92:6812/115824","cluster_addr":"172.21.15.92:6816/115824","heartbeat_back_addr":"172.21.15.92:6823/115824","heartbeat_front_addr":"172.21.15.92:6820/115824","state":["exists","up"]},{"osd":2,"uuid":"25788a55-112f-4f00-810d-2bc37cf76778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":115821},{"type":"v1","addr":"172.21.15.92:6801","nonce":115821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":115821},{"type":"v1","addr":"172.21.15.92:6803","nonce":115821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":115821},{"type":"v1","addr":"172.21.15.92:6807","nonce":115821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":115821},{"type":"v1","addr":"172.21.15.92:6805","nonce":115821}]},"public_addr":"172.21.15.92:6801/115821","cluster_addr":"172.21.15.92:6803/115821","heartbeat_back_addr":"172.21.15.92:6807/115821","heartbeat_front_addr":"172.21.15.92:6805/115821","state":["exists","up"]},{"osd":3,"uuid":"adf54e24-33a9-41fe-9ce3-98a5ab6c158e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":115825},{"type":"v1","addr":"172.21.15.92:6825","nonce":115825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":115825},{"type":"v1","addr":"172.21.15.92:6827","nonce":115825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":115825},{"type":"v1","addr":"172.21.15.92:6831","nonce":115825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":115825},{"type":"v1","addr":"172.21.15.92:6829","nonce":115825}]},"public_addr":"172.21.15.92:6825/115825","cluster_addr":"172.21.15.92:6827/115825","heartbeat_back_addr":"172.21.15.92:6831/115825","heartbeat_front_addr":"172.21.15.92:6829/115825","state":["exists","up"]},{"osd":4,"uuid":"ccb9b747-f399-44ea-9a18-9db3152227e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":116035},{"type":"v1","addr":"172.21.15.169:6817","nonce":116035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":116035},{"type":"v1","addr":"172.21.15.169:6819","nonce":116035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":116035},{"type":"v1","addr":"172.21.15.169:6826","nonce":116035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":116035},{"type":"v1","addr":"172.21.15.169:6822","nonce":116035}]},"public_addr":"172.21.15.169:6817/116035","cluster_addr":"172.21.15.169:6819/116035","heartbeat_back_addr":"172.21.15.169:6826/116035","heartbeat_front_addr":"172.21.15.169:6822/116035","state":["exists","up"]},{"osd":5,"uuid":"308a1e63-75aa-4d7d-8a7c-cecc24714fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":116033},{"type":"v1","addr":"172.21.15.169:6801","nonce":116033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":116033},{"type":"v1","addr":"172.21.15.169:6803","nonce":116033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":116033},{"type":"v1","addr":"172.21.15.169:6807","nonce":116033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":116033},{"type":"v1","addr":"172.21.15.169:6805","nonce":116033}]},"public_addr":"172.21.15.169:6801/116033","cluster_addr":"172.21.15.169:6803/116033","heartbeat_back_addr":"172.21.15.169:6807/116033","heartbeat_front_addr":"172.21.15.169:6805/116033","state":["exists","up"]},{"osd":6,"uuid":"9c459b7b-b821-4b4b-b91e-6aaae271457c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":116034},{"type":"v1","addr":"172.21.15.169:6809","nonce":116034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":116034},{"type":"v1","addr":"172.21.15.169:6811","nonce":116034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":116034},{"type":"v1","addr":"172.21.15.169:6815","nonce":116034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":116034},{"type":"v1","addr":"172.21.15.169:6813","nonce":116034}]},"public_addr":"172.21.15.169:6809/116034","cluster_addr":"172.21.15.169:6811/116034","heartbeat_back_addr":"172.21.15.169:6815/116034","heartbeat_front_addr":"172.21.15.169:6813/116034","state":["exists","up"]},{"osd":7,"uuid":"1ca9aa4a-755d-4e28-a937-4d8dccab4df3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":116044},{"type":"v1","addr":"172.21.15.169:6823","nonce":116044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6825","nonce":116044},{"type":"v1","addr":"172.21.15.169:6827","nonce":116044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":116044},{"type":"v1","addr":"172.21.15.169:6831","nonce":116044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":116044},{"type":"v1","addr":"172.21.15.169:6829","nonce":116044}]},"public_addr":"172.21.15.169:6823/116044","cluster_addr":"172.21.15.169:6827/116044","heartbeat_back_addr":"172.21.15.169:6831/116044","heartbeat_front_addr":"172.21.15.169:6829/116044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.586781+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.869778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.508594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.860493+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.872267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.746933+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.611520+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.871446+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:22:25.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.175+0000 7f47bf3ff700 1 -- 172.21.15.92:0/1261069598 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f479c0682f0 msgr2=0x7f479c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:25.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.175+0000 7f47bf3ff700 1 --2- 172.21.15.92:0/1261069598 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f479c0682f0 0x7f479c06a7a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f47b40016b0 tx=0x7f47b40079c0 comp rx=0 tx=0).stop 2023-11-28T11:22:25.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.175+0000 7f47bf3ff700 1 -- 172.21.15.92:0/1261069598 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47b80cd460 msgr2=0x7f47b8157e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:25.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.175+0000 7f47bf3ff700 1 --2- 172.21.15.92:0/1261069598 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47b80cd460 0x7f47b8157e60 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f47a8000c00 tx=0x7f47a8004750 comp rx=0 tx=0).stop 2023-11-28T11:22:25.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.175+0000 7f47bf3ff700 1 -- 172.21.15.92:0/1261069598 shutdown_connections 2023-11-28T11:22:25.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.175+0000 7f47bf3ff700 1 --2- 172.21.15.92:0/1261069598 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47b80d0d60 0x7f47b8157250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:25.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.175+0000 7f47bf3ff700 1 --2- 172.21.15.92:0/1261069598 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47b80b8510 0x7f47b80c9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:25.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.175+0000 7f47bf3ff700 1 --2- 172.21.15.92:0/1261069598 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f479c0682f0 0x7f479c06a7a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:25.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.175+0000 7f47bf3ff700 1 --2- 172.21.15.92:0/1261069598 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47b80cd460 0x7f47b8157e60 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:25.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.175+0000 7f47bf3ff700 1 -- 172.21.15.92:0/1261069598 >> 172.21.15.92:0/1261069598 conn(0x7f47b80719a0 msgr2=0x7f47b80d1200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:25.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.175+0000 7f47bf3ff700 1 -- 172.21.15.92:0/1261069598 shutdown_connections 2023-11-28T11:22:25.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.175+0000 7f47bf3ff700 1 -- 172.21.15.92:0/1261069598 wait complete. 2023-11-28T11:22:25.196 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:22:25.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.392+0000 7f6ee15ff700 1 Processor -- start 2023-11-28T11:22:25.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.392+0000 7f6ee15ff700 1 -- start start 2023-11-28T11:22:25.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.393+0000 7f6ee15ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6edc0b8510 0x7f6edc06b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:25.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.393+0000 7f6ee15ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6edc0cd460 0x7f6edc06beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:25.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.393+0000 7f6ee15ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6edc06c3f0 0x7f6edc15cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:25.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.393+0000 7f6ee15ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6edc0cdbd0 con 0x7f6edc06c3f0 2023-11-28T11:22:25.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.393+0000 7f6ee15ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6edc0d02d0 con 0x7f6edc0b8510 2023-11-28T11:22:25.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.393+0000 7f6ee15ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6edc0b88e0 con 0x7f6edc0cd460 2023-11-28T11:22:25.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.393+0000 7f6edb7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6edc06c3f0 0x7f6edc15cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:25.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.394+0000 7f6edb7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6edc06c3f0 0x7f6edc15cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51284/0 (socket says 172.21.15.92:51284) 2023-11-28T11:22:25.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.394+0000 7f6edb7fe700 1 -- 172.21.15.92:0/2846956406 learned_addr learned my addr 172.21.15.92:0/2846956406 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:25.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.394+0000 7f6edaffd700 1 --2- 172.21.15.92:0/2846956406 >> v2:172.21.15.169:3300/0 conn(0x7f6edc0b8510 0x7f6edc06b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:25.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.394+0000 7f6eda7fc700 1 --2- 172.21.15.92:0/2846956406 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6edc0cd460 0x7f6edc06beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:25.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.394+0000 7f6edb7fe700 1 -- 172.21.15.92:0/2846956406 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6edc0cd460 msgr2=0x7f6edc06beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:25.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.394+0000 7f6edb7fe700 1 --2- 172.21.15.92:0/2846956406 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6edc0cd460 0x7f6edc06beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:25.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.394+0000 7f6edb7fe700 1 -- 172.21.15.92:0/2846956406 >> v2:172.21.15.169:3300/0 conn(0x7f6edc0b8510 msgr2=0x7f6edc06b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:25.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.394+0000 7f6edb7fe700 1 --2- 172.21.15.92:0/2846956406 >> v2:172.21.15.169:3300/0 conn(0x7f6edc0b8510 0x7f6edc06b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:25.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.394+0000 7f6edb7fe700 1 -- 172.21.15.92:0/2846956406 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6edc0cd8f0 con 0x7f6edc06c3f0 2023-11-28T11:22:25.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.394+0000 7f6edb7fe700 1 --2- 172.21.15.92:0/2846956406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6edc06c3f0 0x7f6edc15cc60 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f6edc0b5a50 tx=0x7f6ed400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3241da6eb9d0a7f0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:25.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.394+0000 7f6ed9ffb700 1 -- 172.21.15.92:0/2846956406 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed400e070 con 0x7f6edc06c3f0 2023-11-28T11:22:25.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.395+0000 7f6ed9ffb700 1 -- 172.21.15.92:0/2846956406 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ed400b980 con 0x7f6edc06c3f0 2023-11-28T11:22:25.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.395+0000 7f6ed9ffb700 1 -- 172.21.15.92:0/2846956406 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed400bd30 con 0x7f6edc06c3f0 2023-11-28T11:22:25.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.395+0000 7f6ee15ff700 1 -- 172.21.15.92:0/2846956406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6edc06c3f0 msgr2=0x7f6edc15cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:25.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.395+0000 7f6ee15ff700 1 --2- 172.21.15.92:0/2846956406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6edc06c3f0 0x7f6edc15cc60 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f6edc0b5a50 tx=0x7f6ed400b330 comp rx=0 tx=0).stop 2023-11-28T11:22:25.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.395+0000 7f6ee15ff700 1 -- 172.21.15.92:0/2846956406 shutdown_connections 2023-11-28T11:22:25.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.395+0000 7f6ee15ff700 1 --2- 172.21.15.92:0/2846956406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6edc06c3f0 0x7f6edc15cc60 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:25.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.395+0000 7f6ee15ff700 1 --2- 172.21.15.92:0/2846956406 >> v2:172.21.15.169:3300/0 conn(0x7f6edc0b8510 0x7f6edc06b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:25.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.395+0000 7f6ee15ff700 1 --2- 172.21.15.92:0/2846956406 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6edc0cd460 0x7f6edc06beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:25.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.395+0000 7f6ee15ff700 1 -- 172.21.15.92:0/2846956406 >> 172.21.15.92:0/2846956406 conn(0x7f6edc0719a0 msgr2=0x7f6edc071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:25.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.395+0000 7f6ee15ff700 1 -- 172.21.15.92:0/2846956406 shutdown_connections 2023-11-28T11:22:25.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.395+0000 7f6ee15ff700 1 -- 172.21.15.92:0/2846956406 wait complete. 2023-11-28T11:22:25.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.397+0000 7f6ee15ff700 1 Processor -- start 2023-11-28T11:22:25.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.397+0000 7f6ee15ff700 1 -- start start 2023-11-28T11:22:25.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.398+0000 7f6ee15ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6edc06c3f0 0x7f6edc0c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:25.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.398+0000 7f6ee15ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6edc0b8510 0x7f6edc0cc750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:25.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.398+0000 7f6ee15ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6edc0cd460 0x7f6edc0d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:25.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.398+0000 7f6ee15ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6edc0c9300 con 0x7f6edc0b8510 2023-11-28T11:22:25.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.398+0000 7f6ee15ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6edc1e5240 con 0x7f6edc06c3f0 2023-11-28T11:22:25.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.398+0000 7f6ee15ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6edc1e4630 con 0x7f6edc0cd460 2023-11-28T11:22:25.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.398+0000 7f6eda7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6edc0b8510 0x7f6edc0cc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:25.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.398+0000 7f6eda7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6edc0b8510 0x7f6edc0cc750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51300/0 (socket says 172.21.15.92:51300) 2023-11-28T11:22:25.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.398+0000 7f6eda7fc700 1 -- 172.21.15.92:0/2403432096 learned_addr learned my addr 172.21.15.92:0/2403432096 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:25.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.399+0000 7f6edaffd700 1 --2- 172.21.15.92:0/2403432096 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6edc06c3f0 0x7f6edc0c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:25.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.399+0000 7f6edb7fe700 1 --2- 172.21.15.92:0/2403432096 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6edc0cd460 0x7f6edc0d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:25.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.399+0000 7f6eda7fc700 1 -- 172.21.15.92:0/2403432096 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6edc0cd460 msgr2=0x7f6edc0d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:25.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.399+0000 7f6eda7fc700 1 --2- 172.21.15.92:0/2403432096 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6edc0cd460 0x7f6edc0d0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:25.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.399+0000 7f6eda7fc700 1 -- 172.21.15.92:0/2403432096 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6edc06c3f0 msgr2=0x7f6edc0c8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:25.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.399+0000 7f6eda7fc700 1 --2- 172.21.15.92:0/2403432096 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6edc06c3f0 0x7f6edc0c8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:25.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.399+0000 7f6eda7fc700 1 -- 172.21.15.92:0/2403432096 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ed4009b30 con 0x7f6edc0b8510 2023-11-28T11:22:25.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.399+0000 7f6edaffd700 1 --2- 172.21.15.92:0/2403432096 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6edc06c3f0 0x7f6edc0c8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:22:25.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.399+0000 7f6eda7fc700 1 --2- 172.21.15.92:0/2403432096 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6edc0b8510 0x7f6edc0cc750 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f6ed000d800 tx=0x7f6ed000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:25.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.399+0000 7f6ec37fe700 1 -- 172.21.15.92:0/2403432096 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed00125f0 con 0x7f6edc0b8510 2023-11-28T11:22:25.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.399+0000 7f6ec37fe700 1 -- 172.21.15.92:0/2403432096 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ed0012750 con 0x7f6edc0b8510 2023-11-28T11:22:25.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.399+0000 7f6ec37fe700 1 -- 172.21.15.92:0/2403432096 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed0012b00 con 0x7f6edc0b8510 2023-11-28T11:22:25.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.399+0000 7f6ee15ff700 1 -- 172.21.15.92:0/2403432096 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6edc0cfff0 con 0x7f6edc0b8510 2023-11-28T11:22:25.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.400+0000 7f6ee15ff700 1 -- 172.21.15.92:0/2403432096 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6edc0cfd10 con 0x7f6edc0b8510 2023-11-28T11:22:25.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.400+0000 7f6ee15ff700 1 -- 172.21.15.92:0/2403432096 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6edc067880 con 0x7f6edc0b8510 2023-11-28T11:22:25.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.401+0000 7f6ec37fe700 1 -- 172.21.15.92:0/2403432096 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6ed0012c60 con 0x7f6edc0b8510 2023-11-28T11:22:25.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.402+0000 7f6ec37fe700 1 --2- 172.21.15.92:0/2403432096 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6ec80682f0 0x7f6ec806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:25.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.402+0000 7f6ec37fe700 1 -- 172.21.15.92:0/2403432096 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f6ed0086380 con 0x7f6edc0b8510 2023-11-28T11:22:25.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.402+0000 7f6edaffd700 1 --2- 172.21.15.92:0/2403432096 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6ec80682f0 0x7f6ec806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:25.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.403+0000 7f6edaffd700 1 --2- 172.21.15.92:0/2403432096 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6ec80682f0 0x7f6ec806a7a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc009970 tx=0x7f6ecc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:25.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.406+0000 7f6ec37fe700 1 -- 172.21.15.92:0/2403432096 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6ed00529b0 con 0x7f6edc0b8510 2023-11-28T11:22:25.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.601+0000 7f6ee15ff700 1 -- 172.21.15.92:0/2403432096 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6edc0d2940 con 0x7f6edc0b8510 2023-11-28T11:22:25.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.602+0000 7f6ec37fe700 1 -- 172.21.15.92:0/2403432096 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+18141 (secure 0 0 0) 0x7f6ed0055fd0 con 0x7f6edc0b8510 2023-11-28T11:22:25.604 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:22:25.605 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":41,"fsid":"68956e0c-c16f-47cd-a220-f98364b0831d","created":"2023-11-28T11:20:28.697573+0000","modified":"2023-11-28T11:22:24.715083+0000","last_up_change":"2023-11-28T11:20:41.257011+0000","last_in_change":"2023-11-28T11:20:32.249767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:20:41.925237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:20:48.682497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:22:22.942125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-28T11:22:23.684528+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6abe483f-0e03-4a26-b9cf-62041291747e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":115823},{"type":"v1","addr":"172.21.15.92:6809","nonce":115823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":115823},{"type":"v1","addr":"172.21.15.92:6813","nonce":115823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":115823},{"type":"v1","addr":"172.21.15.92:6821","nonce":115823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":115823},{"type":"v1","addr":"172.21.15.92:6817","nonce":115823}]},"public_addr":"172.21.15.92:6809/115823","cluster_addr":"172.21.15.92:6813/115823","heartbeat_back_addr":"172.21.15.92:6821/115823","heartbeat_front_addr":"172.21.15.92:6817/115823","state":["exists","up"]},{"osd":1,"uuid":"13c1de73-1a8c-4d7e-8a28-9dae52dfe089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":115824},{"type":"v1","addr":"172.21.15.92:6812","nonce":115824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":115824},{"type":"v1","addr":"172.21.15.92:6816","nonce":115824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":115824},{"type":"v1","addr":"172.21.15.92:6823","nonce":115824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":115824},{"type":"v1","addr":"172.21.15.92:6820","nonce":115824}]},"public_addr":"172.21.15.92:6812/115824","cluster_addr":"172.21.15.92:6816/115824","heartbeat_back_addr":"172.21.15.92:6823/115824","heartbeat_front_addr":"172.21.15.92:6820/115824","state":["exists","up"]},{"osd":2,"uuid":"25788a55-112f-4f00-810d-2bc37cf76778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":115821},{"type":"v1","addr":"172.21.15.92:6801","nonce":115821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":115821},{"type":"v1","addr":"172.21.15.92:6803","nonce":115821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":115821},{"type":"v1","addr":"172.21.15.92:6807","nonce":115821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":115821},{"type":"v1","addr":"172.21.15.92:6805","nonce":115821}]},"public_addr":"172.21.15.92:6801/115821","cluster_addr":"172.21.15.92:6803/115821","heartbeat_back_addr":"172.21.15.92:6807/115821","heartbeat_front_addr":"172.21.15.92:6805/115821","state":["exists","up"]},{"osd":3,"uuid":"adf54e24-33a9-41fe-9ce3-98a5ab6c158e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":115825},{"type":"v1","addr":"172.21.15.92:6825","nonce":115825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":115825},{"type":"v1","addr":"172.21.15.92:6827","nonce":115825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":115825},{"type":"v1","addr":"172.21.15.92:6831","nonce":115825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":115825},{"type":"v1","addr":"172.21.15.92:6829","nonce":115825}]},"public_addr":"172.21.15.92:6825/115825","cluster_addr":"172.21.15.92:6827/115825","heartbeat_back_addr":"172.21.15.92:6831/115825","heartbeat_front_addr":"172.21.15.92:6829/115825","state":["exists","up"]},{"osd":4,"uuid":"ccb9b747-f399-44ea-9a18-9db3152227e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":116035},{"type":"v1","addr":"172.21.15.169:6817","nonce":116035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":116035},{"type":"v1","addr":"172.21.15.169:6819","nonce":116035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":116035},{"type":"v1","addr":"172.21.15.169:6826","nonce":116035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":116035},{"type":"v1","addr":"172.21.15.169:6822","nonce":116035}]},"public_addr":"172.21.15.169:6817/116035","cluster_addr":"172.21.15.169:6819/116035","heartbeat_back_addr":"172.21.15.169:6826/116035","heartbeat_front_addr":"172.21.15.169:6822/116035","state":["exists","up"]},{"osd":5,"uuid":"308a1e63-75aa-4d7d-8a7c-cecc24714fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":116033},{"type":"v1","addr":"172.21.15.169:6801","nonce":116033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":116033},{"type":"v1","addr":"172.21.15.169:6803","nonce":116033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":116033},{"type":"v1","addr":"172.21.15.169:6807","nonce":116033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":116033},{"type":"v1","addr":"172.21.15.169:6805","nonce":116033}]},"public_addr":"172.21.15.169:6801/116033","cluster_addr":"172.21.15.169:6803/116033","heartbeat_back_addr":"172.21.15.169:6807/116033","heartbeat_front_addr":"172.21.15.169:6805/116033","state":["exists","up"]},{"osd":6,"uuid":"9c459b7b-b821-4b4b-b91e-6aaae271457c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":116034},{"type":"v1","addr":"172.21.15.169:6809","nonce":116034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":116034},{"type":"v1","addr":"172.21.15.169:6811","nonce":116034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":116034},{"type":"v1","addr":"172.21.15.169:6815","nonce":116034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":116034},{"type":"v1","addr":"172.21.15.169:6813","nonce":116034}]},"public_addr":"172.21.15.169:6809/116034","cluster_addr":"172.21.15.169:6811/116034","heartbeat_back_addr":"172.21.15.169:6815/116034","heartbeat_front_addr":"172.21.15.169:6813/116034","state":["exists","up"]},{"osd":7,"uuid":"1ca9aa4a-755d-4e28-a937-4d8dccab4df3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":116044},{"type":"v1","addr":"172.21.15.169:6823","nonce":116044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6825","nonce":116044},{"type":"v1","addr":"172.21.15.169:6827","nonce":116044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":116044},{"type":"v1","addr":"172.21.15.169:6831","nonce":116044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":116044},{"type":"v1","addr":"172.21.15.169:6829","nonce":116044}]},"public_addr":"172.21.15.169:6823/116044","cluster_addr":"172.21.15.169:6827/116044","heartbeat_back_addr":"172.21.15.169:6831/116044","heartbeat_front_addr":"172.21.15.169:6829/116044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.586781+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.869778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.508594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.860493+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.872267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.746933+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.611520+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.871446+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:22:25.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.605+0000 7f6ee15ff700 1 -- 172.21.15.92:0/2403432096 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6ec80682f0 msgr2=0x7f6ec806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:25.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.605+0000 7f6ee15ff700 1 --2- 172.21.15.92:0/2403432096 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6ec80682f0 0x7f6ec806a7a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc009970 tx=0x7f6ecc008040 comp rx=0 tx=0).stop 2023-11-28T11:22:25.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.605+0000 7f6ee15ff700 1 -- 172.21.15.92:0/2403432096 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6edc0b8510 msgr2=0x7f6edc0cc750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:25.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.605+0000 7f6ee15ff700 1 --2- 172.21.15.92:0/2403432096 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6edc0b8510 0x7f6edc0cc750 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f6ed000d800 tx=0x7f6ed000dbc0 comp rx=0 tx=0).stop 2023-11-28T11:22:25.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.605+0000 7f6ee15ff700 1 -- 172.21.15.92:0/2403432096 shutdown_connections 2023-11-28T11:22:25.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.605+0000 7f6ee15ff700 1 --2- 172.21.15.92:0/2403432096 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6edc0cd460 0x7f6edc0d0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:25.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.605+0000 7f6ee15ff700 1 --2- 172.21.15.92:0/2403432096 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6edc06c3f0 0x7f6edc0c8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:25.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.605+0000 7f6ee15ff700 1 --2- 172.21.15.92:0/2403432096 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6ec80682f0 0x7f6ec806a7a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:25.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.605+0000 7f6ee15ff700 1 --2- 172.21.15.92:0/2403432096 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6edc0b8510 0x7f6edc0cc750 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:25.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.605+0000 7f6ee15ff700 1 -- 172.21.15.92:0/2403432096 >> 172.21.15.92:0/2403432096 conn(0x7f6edc0719a0 msgr2=0x7f6edc0b3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:25.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.605+0000 7f6ee15ff700 1 -- 172.21.15.92:0/2403432096 shutdown_connections 2023-11-28T11:22:25.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.605+0000 7f6ee15ff700 1 -- 172.21.15.92:0/2403432096 wait complete. 2023-11-28T11:22:25.625 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T11:22:25.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.825+0000 7f965f5ff700 1 Processor -- start 2023-11-28T11:22:25.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.825+0000 7f965f5ff700 1 -- start start 2023-11-28T11:22:25.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.826+0000 7f965f5ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96580cd460 0x7f965806c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:25.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.826+0000 7f965f5ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f96580d0d60 0x7f96580d11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:25.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.826+0000 7f965f5ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96580b88e0 con 0x7f96580b8510 2023-11-28T11:22:25.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.826+0000 7f965f5ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96580cdbd0 con 0x7f96580d0d60 2023-11-28T11:22:25.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.826+0000 7f965f5ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96580d02d0 con 0x7f96580cd460 2023-11-28T11:22:25.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.826+0000 7f965d39b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f96580b8510 0x7f965806b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33802/0 (socket says 172.21.15.92:33802) 2023-11-28T11:22:25.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.826+0000 7f965d39b700 1 -- 172.21.15.92:0/928850801 learned_addr learned my addr 172.21.15.92:0/928850801 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:22:25.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.827+0000 7f965cb9a700 1 --2- 172.21.15.92:0/928850801 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96580cd460 0x7f965806c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:25.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.827+0000 7f965db9c700 1 --2- 172.21.15.92:0/928850801 >> v2:172.21.15.169:3300/0 conn(0x7f96580d0d60 0x7f96580d11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:25.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.827+0000 7f964ffff700 1 -- 172.21.15.92:0/928850801 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2337976436 0 0) 0x7f96580b88e0 con 0x7f96580b8510 2023-11-28T11:22:25.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.827+0000 7f964ffff700 1 -- 172.21.15.92:0/928850801 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96440035b0 con 0x7f96580b8510 2023-11-28T11:22:25.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.827+0000 7f964ffff700 1 -- 172.21.15.92:0/928850801 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3086754091 0 0) 0x7f96440035b0 con 0x7f96580b8510 2023-11-28T11:22:25.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.827+0000 7f964ffff700 1 -- 172.21.15.92:0/928850801 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96580cd460 msgr2=0x7f965806c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:25.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.827+0000 7f964ffff700 1 --2- 172.21.15.92:0/928850801 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96580cd460 0x7f965806c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:25.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.827+0000 7f964ffff700 1 -- 172.21.15.92:0/928850801 >> v2:172.21.15.169:3300/0 conn(0x7f96580d0d60 msgr2=0x7f96580d11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:25.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.827+0000 7f964ffff700 1 --2- 172.21.15.92:0/928850801 >> v2:172.21.15.169:3300/0 conn(0x7f96580d0d60 0x7f96580d11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:25.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.827+0000 7f964ffff700 1 -- 172.21.15.92:0/928850801 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96580cd8f0 con 0x7f96580b8510 2023-11-28T11:22:25.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.827+0000 7f964ffff700 1 -- 172.21.15.92:0/928850801 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532161587 0 0) 0x7f9654001670 con 0x7f96580b8510 2023-11-28T11:22:25.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.827+0000 7f964ffff700 1 -- 172.21.15.92:0/928850801 >> v1:172.21.15.92:6789/0 conn(0x7f96580b8510 legacy=0x7f965806b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:25.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.827+0000 7f964ffff700 1 --2- 172.21.15.92:0/928850801 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96580cd460 0x7f9644003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:25.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.827+0000 7f964ffff700 1 --2- 172.21.15.92:0/928850801 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9644003fb0 0x7f9644004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:25.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.827+0000 7f964ffff700 1 --2- 172.21.15.92:0/928850801 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9644004d90 0x7f9644005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:25.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.827+0000 7f964ffff700 1 -- 172.21.15.92:0/928850801 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96580cdbd0 con 0x7f9644003fb0 2023-11-28T11:22:25.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.827+0000 7f964ffff700 1 -- 172.21.15.92:0/928850801 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96580d02d0 con 0x7f9644004d90 2023-11-28T11:22:25.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.827+0000 7f964ffff700 1 -- 172.21.15.92:0/928850801 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f964400ba10 con 0x7f96580cd460 2023-11-28T11:22:25.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.828+0000 7f965cb9a700 1 --2- 172.21.15.92:0/928850801 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9644003fb0 0x7f9644004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:25.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.828+0000 7f965d39b700 1 --2- 172.21.15.92:0/928850801 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96580cd460 0x7f9644003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:25.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.828+0000 7f965db9c700 1 --2- 172.21.15.92:0/928850801 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9644004d90 0x7f9644005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:25.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.828+0000 7f965cb9a700 1 -- 172.21.15.92:0/928850801 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96580cd460 msgr2=0x7f9644003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:25.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.828+0000 7f965cb9a700 1 --2- 172.21.15.92:0/928850801 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96580cd460 0x7f9644003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:25.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.828+0000 7f965cb9a700 1 -- 172.21.15.92:0/928850801 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9644004d90 msgr2=0x7f9644005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:25.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.828+0000 7f965cb9a700 1 --2- 172.21.15.92:0/928850801 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9644004d90 0x7f9644005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:25.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.828+0000 7f965cb9a700 1 -- 172.21.15.92:0/928850801 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96440035b0 con 0x7f9644003fb0 2023-11-28T11:22:25.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.828+0000 7f965cb9a700 1 --2- 172.21.15.92:0/928850801 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9644003fb0 0x7f9644004850 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f9648009b60 tx=0x7f9644005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfde6720e5131840 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:25.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.828+0000 7f964ffff700 1 -- 172.21.15.92:0/928850801 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964800e6c0 con 0x7f9644003fb0 2023-11-28T11:22:25.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.828+0000 7f964ffff700 1 -- 172.21.15.92:0/928850801 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f964800e820 con 0x7f9644003fb0 2023-11-28T11:22:25.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.828+0000 7f964ffff700 1 -- 172.21.15.92:0/928850801 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964800eb90 con 0x7f9644003fb0 2023-11-28T11:22:25.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.829+0000 7f965f5ff700 1 -- 172.21.15.92:0/928850801 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9644003fb0 msgr2=0x7f9644004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:25.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.829+0000 7f965f5ff700 1 --2- 172.21.15.92:0/928850801 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9644003fb0 0x7f9644004850 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f9648009b60 tx=0x7f9644005880 comp rx=0 tx=0).stop 2023-11-28T11:22:25.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.829+0000 7f965db9c700 1 -- 172.21.15.92:0/928850801 reap_dead start 2023-11-28T11:22:25.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.829+0000 7f965f5ff700 1 -- 172.21.15.92:0/928850801 shutdown_connections 2023-11-28T11:22:25.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.829+0000 7f965f5ff700 1 -- 172.21.15.92:0/928850801 >> 172.21.15.92:0/928850801 conn(0x7f96580719a0 msgr2=0x7f9658071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:25.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.829+0000 7f965f5ff700 1 -- 172.21.15.92:0/928850801 shutdown_connections 2023-11-28T11:22:25.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.829+0000 7f965f5ff700 1 -- 172.21.15.92:0/928850801 wait complete. 2023-11-28T11:22:25.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.831+0000 7f965f5ff700 1 Processor -- start 2023-11-28T11:22:25.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.831+0000 7f965f5ff700 1 -- start start 2023-11-28T11:22:25.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.832+0000 7f965f5ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9644003fb0 0x7f96581e86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:25.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.832+0000 7f965f5ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96581e8be0 0x7f96580cca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:25.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.832+0000 7f965f5ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96580ccfb0 0x7f9658158db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:25.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.832+0000 7f965f5ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96580cb730 con 0x7f96581e8be0 2023-11-28T11:22:25.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.832+0000 7f965f5ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96580cb9d0 con 0x7f9644003fb0 2023-11-28T11:22:25.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.832+0000 7f965f5ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96580cbcb0 con 0x7f96580ccfb0 2023-11-28T11:22:25.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.832+0000 7f965cb9a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96581e8be0 0x7f96580cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:25.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.832+0000 7f965cb9a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96581e8be0 0x7f96580cca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51308/0 (socket says 172.21.15.92:51308) 2023-11-28T11:22:25.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.832+0000 7f965cb9a700 1 -- 172.21.15.92:0/2683300256 learned_addr learned my addr 172.21.15.92:0/2683300256 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:25.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.833+0000 7f965cb9a700 1 -- 172.21.15.92:0/2683300256 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96580ccfb0 msgr2=0x7f9658158db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:25.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.833+0000 7f965cb9a700 1 --2- 172.21.15.92:0/2683300256 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96580ccfb0 0x7f9658158db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:25.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.833+0000 7f965cb9a700 1 -- 172.21.15.92:0/2683300256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9644003fb0 msgr2=0x7f96581e86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:25.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.833+0000 7f965cb9a700 1 --2- 172.21.15.92:0/2683300256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9644003fb0 0x7f96581e86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:25.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.833+0000 7f965cb9a700 1 -- 172.21.15.92:0/2683300256 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96580cfff0 con 0x7f96581e8be0 2023-11-28T11:22:25.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.833+0000 7f965cb9a700 1 --2- 172.21.15.92:0/2683300256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96581e8be0 0x7f96580cca70 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f9644004700 tx=0x7f96480047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:25.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.833+0000 7f964dffb700 1 -- 172.21.15.92:0/2683300256 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9648021070 con 0x7f96581e8be0 2023-11-28T11:22:25.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.833+0000 7f964dffb700 1 -- 172.21.15.92:0/2683300256 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9648004550 con 0x7f96581e8be0 2023-11-28T11:22:25.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.833+0000 7f965f5ff700 1 -- 172.21.15.92:0/2683300256 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96580cfd10 con 0x7f96581e8be0 2023-11-28T11:22:25.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.833+0000 7f964dffb700 1 -- 172.21.15.92:0/2683300256 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964800e6c0 con 0x7f96581e8be0 2023-11-28T11:22:25.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.833+0000 7f965f5ff700 1 -- 172.21.15.92:0/2683300256 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96580d29c0 con 0x7f96581e8be0 2023-11-28T11:22:25.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.834+0000 7f965f5ff700 1 -- 172.21.15.92:0/2683300256 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9658067880 con 0x7f96581e8be0 2023-11-28T11:22:25.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.835+0000 7f964dffb700 1 -- 172.21.15.92:0/2683300256 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f964802e030 con 0x7f96581e8be0 2023-11-28T11:22:25.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.836+0000 7f964dffb700 1 --2- 172.21.15.92:0/2683300256 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f963c0682f0 0x7f963c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:25.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.836+0000 7f964dffb700 1 -- 172.21.15.92:0/2683300256 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f9648086fc0 con 0x7f96581e8be0 2023-11-28T11:22:25.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.836+0000 7f965d39b700 1 --2- 172.21.15.92:0/2683300256 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f963c0682f0 0x7f963c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:25.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.837+0000 7f965d39b700 1 --2- 172.21.15.92:0/2683300256 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f963c0682f0 0x7f963c06a7a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f96581e2ec0 tx=0x7f9654000f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:25.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:25.840+0000 7f964dffb700 1 -- 172.21.15.92:0/2683300256 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f96480536e0 con 0x7f96581e8be0 2023-11-28T11:22:26.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.079+0000 7f965f5ff700 1 -- 172.21.15.92:0/2683300256 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f96580b6430 con 0x7f96581e8be0 2023-11-28T11:22:26.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.733+0000 7f964dffb700 1 -- 172.21.15.92:0/2683300256 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7f96480170b0 con 0x7f96581e8be0 2023-11-28T11:22:26.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.735+0000 7f965f5ff700 1 -- 172.21.15.92:0/2683300256 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f963c0682f0 msgr2=0x7f963c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:26.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.735+0000 7f965f5ff700 1 --2- 172.21.15.92:0/2683300256 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f963c0682f0 0x7f963c06a7a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f96581e2ec0 tx=0x7f9654000f90 comp rx=0 tx=0).stop 2023-11-28T11:22:26.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.735+0000 7f965f5ff700 1 -- 172.21.15.92:0/2683300256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96581e8be0 msgr2=0x7f96580cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:26.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.735+0000 7f965f5ff700 1 --2- 172.21.15.92:0/2683300256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96581e8be0 0x7f96580cca70 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f9644004700 tx=0x7f96480047c0 comp rx=0 tx=0).stop 2023-11-28T11:22:26.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.735+0000 7f965f5ff700 1 -- 172.21.15.92:0/2683300256 shutdown_connections 2023-11-28T11:22:26.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.735+0000 7f965f5ff700 1 --2- 172.21.15.92:0/2683300256 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96580ccfb0 0x7f9658158db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:26.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.735+0000 7f965f5ff700 1 --2- 172.21.15.92:0/2683300256 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9644003fb0 0x7f96581e86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:26.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.735+0000 7f965f5ff700 1 --2- 172.21.15.92:0/2683300256 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f963c0682f0 0x7f963c06a7a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:26.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.735+0000 7f965f5ff700 1 --2- 172.21.15.92:0/2683300256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96581e8be0 0x7f96580cca70 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:26.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.735+0000 7f965f5ff700 1 -- 172.21.15.92:0/2683300256 >> 172.21.15.92:0/2683300256 conn(0x7f96580719a0 msgr2=0x7f96580d2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:26.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.735+0000 7f965f5ff700 1 -- 172.21.15.92:0/2683300256 shutdown_connections 2023-11-28T11:22:26.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.735+0000 7f965f5ff700 1 -- 172.21.15.92:0/2683300256 wait complete. 2023-11-28T11:22:26.758 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-28T11:22:26.760 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T11:22:26.898 INFO:tasks.ceph.mds.a.smithi092.stderr:2023-11-28T11:22:26.897+0000 7fa25f9f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T11:22:26.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.959+0000 7fd528f9d700 1 Processor -- start 2023-11-28T11:22:26.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.959+0000 7fd528f9d700 1 -- start start 2023-11-28T11:22:26.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.960+0000 7fd528f9d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd52415f960 0x7fd52415d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:26.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.960+0000 7fd528f9d700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd5240d15c0 0x7fd5240b7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:26.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.960+0000 7fd528f9d700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5240cdb10 con 0x7fd5240d1b30 2023-11-28T11:22:26.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.960+0000 7fd528f9d700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd52415daf0 con 0x7fd5240d15c0 2023-11-28T11:22:26.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.960+0000 7fd528f9d700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5240cd500 con 0x7fd52415f960 2023-11-28T11:22:26.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.961+0000 7fd5237fe700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd52415f960 0x7fd52415d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:26.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.961+0000 7fd5237fe700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd52415f960 0x7fd52415d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.92:60170/0 (socket says 172.21.15.92:60170) 2023-11-28T11:22:26.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.961+0000 7fd5237fe700 1 -- 172.21.15.92:0/3140265887 learned_addr learned my addr 172.21.15.92:0/3140265887 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:26.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.961+0000 7fd523fff700 1 --2- 172.21.15.92:0/3140265887 >> v2:172.21.15.169:3300/0 conn(0x7fd5240d15c0 0x7fd5240b7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:26.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.961+0000 7fd522ffd700 1 --1- 172.21.15.92:0/3140265887 >> v1:172.21.15.169:6789/0 conn(0x7fd5240d1b30 0x7fd5240d0eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.92:59444/0 (socket says 172.21.15.92:59444) 2023-11-28T11:22:26.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.961+0000 7fd5227fc700 1 -- 172.21.15.92:0/3140265887 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1555883067 0 0) 0x7fd5240cdb10 con 0x7fd5240d1b30 2023-11-28T11:22:26.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.961+0000 7fd5227fc700 1 -- 172.21.15.92:0/3140265887 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd50c0035b0 con 0x7fd5240d1b30 2023-11-28T11:22:26.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.961+0000 7fd5237fe700 1 -- 172.21.15.92:0/3140265887 >> v2:172.21.15.169:3300/0 conn(0x7fd5240d15c0 msgr2=0x7fd5240b7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:26.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.961+0000 7fd5237fe700 1 --2- 172.21.15.92:0/3140265887 >> v2:172.21.15.169:3300/0 conn(0x7fd5240d15c0 0x7fd5240b7c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:26.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.961+0000 7fd5237fe700 1 -- 172.21.15.92:0/3140265887 >> v1:172.21.15.169:6789/0 conn(0x7fd5240d1b30 legacy=0x7fd5240d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:26.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.961+0000 7fd5237fe700 1 -- 172.21.15.92:0/3140265887 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5240d25f0 con 0x7fd52415f960 2023-11-28T11:22:26.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.961+0000 7fd523fff700 1 --2- 172.21.15.92:0/3140265887 >> v2:172.21.15.169:3300/0 conn(0x7fd5240d15c0 0x7fd5240b7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:22:26.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.962+0000 7fd5237fe700 1 --2- 172.21.15.92:0/3140265887 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd52415f960 0x7fd52415d5b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd5140098f0 tx=0x7fd514009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=677bf4e9d829fa63 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:26.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.962+0000 7fd5227fc700 1 -- 172.21.15.92:0/3140265887 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd514011830 con 0x7fd52415f960 2023-11-28T11:22:26.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.962+0000 7fd5227fc700 1 -- 172.21.15.92:0/3140265887 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd514011990 con 0x7fd52415f960 2023-11-28T11:22:26.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.962+0000 7fd5227fc700 1 -- 172.21.15.92:0/3140265887 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd514011d00 con 0x7fd52415f960 2023-11-28T11:22:26.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.963+0000 7fd528f9d700 1 -- 172.21.15.92:0/3140265887 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd52415f960 msgr2=0x7fd52415d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:26.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.963+0000 7fd528f9d700 1 --2- 172.21.15.92:0/3140265887 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd52415f960 0x7fd52415d5b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd5140098f0 tx=0x7fd514009c00 comp rx=0 tx=0).stop 2023-11-28T11:22:26.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.963+0000 7fd528f9d700 1 -- 172.21.15.92:0/3140265887 shutdown_connections 2023-11-28T11:22:26.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.963+0000 7fd528f9d700 1 --2- 172.21.15.92:0/3140265887 >> v2:172.21.15.169:3300/0 conn(0x7fd5240d15c0 0x7fd5240b7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:26.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.963+0000 7fd528f9d700 1 --2- 172.21.15.92:0/3140265887 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd52415f960 0x7fd52415d5b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:26.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.963+0000 7fd528f9d700 1 -- 172.21.15.92:0/3140265887 >> 172.21.15.92:0/3140265887 conn(0x7fd5240719a0 msgr2=0x7fd524071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:26.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.963+0000 7fd528f9d700 1 -- 172.21.15.92:0/3140265887 shutdown_connections 2023-11-28T11:22:26.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.963+0000 7fd528f9d700 1 -- 172.21.15.92:0/3140265887 wait complete. 2023-11-28T11:22:26.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.964+0000 7fd528f9d700 1 Processor -- start 2023-11-28T11:22:26.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.965+0000 7fd528f9d700 1 -- start start 2023-11-28T11:22:26.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.965+0000 7fd528f9d700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5240d15c0 0x7fd5240b7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:26.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.965+0000 7fd528f9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5240d1b30 0x7fd52414ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:26.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.965+0000 7fd528f9d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd52415f960 0x7fd5240cbb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:26.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.965+0000 7fd528f9d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5240b6650 con 0x7fd5240d1b30 2023-11-28T11:22:26.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.965+0000 7fd528f9d700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52414d540 con 0x7fd5240d15c0 2023-11-28T11:22:26.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.965+0000 7fd528f9d700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5240b6860 con 0x7fd52415f960 2023-11-28T11:22:26.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.966+0000 7fd522ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5240d1b30 0x7fd52414ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:26.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.966+0000 7fd522ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5240d1b30 0x7fd52414ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51320/0 (socket says 172.21.15.92:51320) 2023-11-28T11:22:26.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.966+0000 7fd522ffd700 1 -- 172.21.15.92:0/2748487501 learned_addr learned my addr 172.21.15.92:0/2748487501 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:26.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.966+0000 7fd523fff700 1 --2- 172.21.15.92:0/2748487501 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd52415f960 0x7fd5240cbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:26.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.966+0000 7fd5237fe700 1 --2- 172.21.15.92:0/2748487501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5240d15c0 0x7fd5240b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:26.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.966+0000 7fd522ffd700 1 -- 172.21.15.92:0/2748487501 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd52415f960 msgr2=0x7fd5240cbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:26.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.966+0000 7fd522ffd700 1 --2- 172.21.15.92:0/2748487501 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd52415f960 0x7fd5240cbb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:26.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.966+0000 7fd522ffd700 1 -- 172.21.15.92:0/2748487501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5240d15c0 msgr2=0x7fd5240b7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:26.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.966+0000 7fd522ffd700 1 --2- 172.21.15.92:0/2748487501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5240d15c0 0x7fd5240b7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:26.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.966+0000 7fd522ffd700 1 -- 172.21.15.92:0/2748487501 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd51c009a70 con 0x7fd5240d1b30 2023-11-28T11:22:26.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.966+0000 7fd5237fe700 1 --2- 172.21.15.92:0/2748487501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5240d15c0 0x7fd5240b7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:22:26.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.966+0000 7fd522ffd700 1 --2- 172.21.15.92:0/2748487501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5240d1b30 0x7fd52414ca20 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd51800c500 tx=0x7fd51800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:26.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.967+0000 7fd50bfff700 1 -- 172.21.15.92:0/2748487501 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd518005cb0 con 0x7fd5240d1b30 2023-11-28T11:22:26.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.967+0000 7fd50bfff700 1 -- 172.21.15.92:0/2748487501 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd518005e10 con 0x7fd5240d1b30 2023-11-28T11:22:26.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.967+0000 7fd50bfff700 1 -- 172.21.15.92:0/2748487501 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd518007310 con 0x7fd5240d1b30 2023-11-28T11:22:26.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.967+0000 7fd528f9d700 1 -- 172.21.15.92:0/2748487501 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd514009550 con 0x7fd5240d1b30 2023-11-28T11:22:26.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.967+0000 7fd528f9d700 1 -- 172.21.15.92:0/2748487501 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5240cd7a0 con 0x7fd5240d1b30 2023-11-28T11:22:26.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.968+0000 7fd528f9d700 1 -- 172.21.15.92:0/2748487501 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5240d28d0 con 0x7fd5240d1b30 2023-11-28T11:22:26.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.969+0000 7fd50bfff700 1 -- 172.21.15.92:0/2748487501 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd518019030 con 0x7fd5240d1b30 2023-11-28T11:22:26.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.970+0000 7fd50bfff700 1 --2- 172.21.15.92:0/2748487501 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd5100682f0 0x7fd51006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:26.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.970+0000 7fd50bfff700 1 -- 172.21.15.92:0/2748487501 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fd5180851a0 con 0x7fd5240d1b30 2023-11-28T11:22:26.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.970+0000 7fd5237fe700 1 --2- 172.21.15.92:0/2748487501 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd5100682f0 0x7fd51006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:26.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.971+0000 7fd5237fe700 1 --2- 172.21.15.92:0/2748487501 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd5100682f0 0x7fd51006a7a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd514009c40 tx=0x7fd514008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:26.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:26.974+0000 7fd50bfff700 1 -- 172.21.15.92:0/2748487501 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd518051920 con 0x7fd5240d1b30 2023-11-28T11:22:27.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.207+0000 7fd528f9d700 1 -- 172.21.15.92:0/2748487501 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fd5240d2bb0 con 0x7fd5240d1b30 2023-11-28T11:22:27.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.735+0000 7fd50bfff700 1 -- 172.21.15.92:0/2748487501 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd518054f40 con 0x7fd5240d1b30 2023-11-28T11:22:27.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.738+0000 7fd528f9d700 1 -- 172.21.15.92:0/2748487501 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd5100682f0 msgr2=0x7fd51006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:27.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.738+0000 7fd528f9d700 1 --2- 172.21.15.92:0/2748487501 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd5100682f0 0x7fd51006a7a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd514009c40 tx=0x7fd514008030 comp rx=0 tx=0).stop 2023-11-28T11:22:27.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.738+0000 7fd528f9d700 1 -- 172.21.15.92:0/2748487501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5240d1b30 msgr2=0x7fd52414ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:27.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.738+0000 7fd528f9d700 1 --2- 172.21.15.92:0/2748487501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5240d1b30 0x7fd52414ca20 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd51800c500 tx=0x7fd51800c810 comp rx=0 tx=0).stop 2023-11-28T11:22:27.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.738+0000 7fd528f9d700 1 -- 172.21.15.92:0/2748487501 shutdown_connections 2023-11-28T11:22:27.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.738+0000 7fd528f9d700 1 --2- 172.21.15.92:0/2748487501 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd52415f960 0x7fd5240cbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:27.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.738+0000 7fd528f9d700 1 --2- 172.21.15.92:0/2748487501 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd5240d15c0 0x7fd5240b7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:27.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.738+0000 7fd528f9d700 1 --2- 172.21.15.92:0/2748487501 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd5100682f0 0x7fd51006a7a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:27.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.738+0000 7fd528f9d700 1 --2- 172.21.15.92:0/2748487501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5240d1b30 0x7fd52414ca20 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:27.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.738+0000 7fd528f9d700 1 -- 172.21.15.92:0/2748487501 >> 172.21.15.92:0/2748487501 conn(0x7fd5240719a0 msgr2=0x7fd5240cf2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:27.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.738+0000 7fd528f9d700 1 -- 172.21.15.92:0/2748487501 shutdown_connections 2023-11-28T11:22:27.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.738+0000 7fd528f9d700 1 -- 172.21.15.92:0/2748487501 wait complete. 2023-11-28T11:22:27.759 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T11:22:27.760 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:22:27.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.954+0000 7f28e91ff700 1 Processor -- start 2023-11-28T11:22:27.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.955+0000 7f28e91ff700 1 -- start start 2023-11-28T11:22:27.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.956+0000 7f28e91ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28e40b8510 0x7f28e406b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:27.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.956+0000 7f28e91ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f28e40d0d60 0x7f28e40d11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:27.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.956+0000 7f28e91ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28e40b88e0 con 0x7f28e40cd460 2023-11-28T11:22:27.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.956+0000 7f28e91ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28e40cdbd0 con 0x7f28e40d0d60 2023-11-28T11:22:27.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.956+0000 7f28e91ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28e40d02d0 con 0x7f28e40b8510 2023-11-28T11:22:27.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.956+0000 7f28dbfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f28e40cd460 0x7f28e406beb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33818/0 (socket says 172.21.15.92:33818) 2023-11-28T11:22:27.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.956+0000 7f28dbfff700 1 -- 172.21.15.92:0/3745717767 learned_addr learned my addr 172.21.15.92:0/3745717767 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:22:27.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.956+0000 7f28e259c700 1 -- 172.21.15.92:0/3745717767 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1448022561 0 0) 0x7f28e40b88e0 con 0x7f28e40cd460 2023-11-28T11:22:27.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.956+0000 7f28e259c700 1 -- 172.21.15.92:0/3745717767 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28d00035b0 con 0x7f28e40cd460 2023-11-28T11:22:27.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.956+0000 7f28e2d9d700 1 --2- 172.21.15.92:0/3745717767 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28e40b8510 0x7f28e406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:27.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.957+0000 7f28e359e700 1 --2- 172.21.15.92:0/3745717767 >> v2:172.21.15.169:3300/0 conn(0x7f28e40d0d60 0x7f28e40d11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:27.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.957+0000 7f28e259c700 1 -- 172.21.15.92:0/3745717767 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2664821091 0 0) 0x7f28d00035b0 con 0x7f28e40cd460 2023-11-28T11:22:27.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.957+0000 7f28e259c700 1 -- 172.21.15.92:0/3745717767 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28e40b8510 msgr2=0x7f28e406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:27.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.957+0000 7f28e259c700 1 --2- 172.21.15.92:0/3745717767 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28e40b8510 0x7f28e406b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:27.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.957+0000 7f28e259c700 1 -- 172.21.15.92:0/3745717767 >> v2:172.21.15.169:3300/0 conn(0x7f28e40d0d60 msgr2=0x7f28e40d11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:27.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.957+0000 7f28e259c700 1 --2- 172.21.15.92:0/3745717767 >> v2:172.21.15.169:3300/0 conn(0x7f28e40d0d60 0x7f28e40d11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:27.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.957+0000 7f28e259c700 1 -- 172.21.15.92:0/3745717767 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e40cd8f0 con 0x7f28e40cd460 2023-11-28T11:22:27.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.957+0000 7f28e259c700 1 -- 172.21.15.92:0/3745717767 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532161587 0 0) 0x7f28cc001670 con 0x7f28e40cd460 2023-11-28T11:22:27.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.957+0000 7f28e359e700 1 --2- 172.21.15.92:0/3745717767 >> v2:172.21.15.169:3300/0 conn(0x7f28e40d0d60 0x7f28e40d11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:22:27.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.957+0000 7f28e259c700 1 -- 172.21.15.92:0/3745717767 >> v1:172.21.15.92:6789/0 conn(0x7f28e40cd460 legacy=0x7f28e406beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:27.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.957+0000 7f28e259c700 1 --2- 172.21.15.92:0/3745717767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28d0003a30 0x7f28d0003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:27.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.957+0000 7f28e259c700 1 --2- 172.21.15.92:0/3745717767 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28d0004850 0x7f28d0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:27.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.957+0000 7f28e259c700 1 --2- 172.21.15.92:0/3745717767 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28e40b8510 0x7f28d0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:27.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.957+0000 7f28e259c700 1 -- 172.21.15.92:0/3745717767 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e40cdbd0 con 0x7f28d0003a30 2023-11-28T11:22:27.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.957+0000 7f28e259c700 1 -- 172.21.15.92:0/3745717767 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e40d02d0 con 0x7f28d0004850 2023-11-28T11:22:27.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.957+0000 7f28e259c700 1 -- 172.21.15.92:0/3745717767 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28d000ba10 con 0x7f28e40b8510 2023-11-28T11:22:27.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.957+0000 7f28e2d9d700 1 --2- 172.21.15.92:0/3745717767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28d0003a30 0x7f28d0003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:27.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.957+0000 7f28dbfff700 1 --2- 172.21.15.92:0/3745717767 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28d0004850 0x7f28d0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:27.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.957+0000 7f28e359e700 1 --2- 172.21.15.92:0/3745717767 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28e40b8510 0x7f28d0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:27.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.958+0000 7f28e2d9d700 1 -- 172.21.15.92:0/3745717767 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28e40b8510 msgr2=0x7f28d0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:27.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.958+0000 7f28e2d9d700 1 --2- 172.21.15.92:0/3745717767 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28e40b8510 0x7f28d0005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:27.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.958+0000 7f28e2d9d700 1 -- 172.21.15.92:0/3745717767 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28d0004850 msgr2=0x7f28d0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:27.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.958+0000 7f28e2d9d700 1 --2- 172.21.15.92:0/3745717767 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28d0004850 0x7f28d0004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:27.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.958+0000 7f28e2d9d700 1 -- 172.21.15.92:0/3745717767 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28d00035b0 con 0x7f28d0003a30 2023-11-28T11:22:27.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.958+0000 7f28dbfff700 1 --2- 172.21.15.92:0/3745717767 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28d0004850 0x7f28d0004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:22:27.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.958+0000 7f28e2d9d700 1 --2- 172.21.15.92:0/3745717767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28d0003a30 0x7f28d0003e40 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f28d4009b60 tx=0x7f28d0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c97ac9fc1322f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:27.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.958+0000 7f28e259c700 1 -- 172.21.15.92:0/3745717767 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d400e6c0 con 0x7f28d0003a30 2023-11-28T11:22:27.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.958+0000 7f28e259c700 1 -- 172.21.15.92:0/3745717767 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28d400e820 con 0x7f28d0003a30 2023-11-28T11:22:27.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.958+0000 7f28e259c700 1 -- 172.21.15.92:0/3745717767 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d400eb90 con 0x7f28d0003a30 2023-11-28T11:22:27.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.959+0000 7f28e91ff700 1 -- 172.21.15.92:0/3745717767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28d0003a30 msgr2=0x7f28d0003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:27.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.959+0000 7f28e91ff700 1 --2- 172.21.15.92:0/3745717767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28d0003a30 0x7f28d0003e40 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f28d4009b60 tx=0x7f28d0005880 comp rx=0 tx=0).stop 2023-11-28T11:22:27.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.959+0000 7f28e359e700 1 -- 172.21.15.92:0/3745717767 reap_dead start 2023-11-28T11:22:27.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.959+0000 7f28e91ff700 1 -- 172.21.15.92:0/3745717767 shutdown_connections 2023-11-28T11:22:27.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.959+0000 7f28e91ff700 1 -- 172.21.15.92:0/3745717767 >> 172.21.15.92:0/3745717767 conn(0x7f28e40719a0 msgr2=0x7f28e4071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:27.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.959+0000 7f28e91ff700 1 -- 172.21.15.92:0/3745717767 shutdown_connections 2023-11-28T11:22:27.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.959+0000 7f28e91ff700 1 -- 172.21.15.92:0/3745717767 wait complete. 2023-11-28T11:22:27.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.961+0000 7f28e91ff700 1 Processor -- start 2023-11-28T11:22:27.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.961+0000 7f28e91ff700 1 -- start start 2023-11-28T11:22:27.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.963+0000 7f28e91ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28d0003a30 0x7f28e414d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:27.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.963+0000 7f28e91ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28e4158660 0x7f28e415b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:27.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.963+0000 7f28e91ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28e4159200 0x7f28e4159670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:27.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.963+0000 7f28e91ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e41547a0 con 0x7f28e4159200 2023-11-28T11:22:27.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.963+0000 7f28e91ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e4158ad0 con 0x7f28e4158660 2023-11-28T11:22:27.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.963+0000 7f28e91ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28e4159bb0 con 0x7f28d0003a30 2023-11-28T11:22:27.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.963+0000 7f28e359e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28e4159200 0x7f28e4159670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:27.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.963+0000 7f28e359e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28e4159200 0x7f28e4159670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51350/0 (socket says 172.21.15.92:51350) 2023-11-28T11:22:27.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.963+0000 7f28e359e700 1 -- 172.21.15.92:0/3691954764 learned_addr learned my addr 172.21.15.92:0/3691954764 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:27.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.963+0000 7f28e2d9d700 1 --2- 172.21.15.92:0/3691954764 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28d0003a30 0x7f28e414d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:27.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.963+0000 7f28dbfff700 1 --2- 172.21.15.92:0/3691954764 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28e4158660 0x7f28e415b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:27.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.963+0000 7f28e359e700 1 -- 172.21.15.92:0/3691954764 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28d0003a30 msgr2=0x7f28e414d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:27.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.963+0000 7f28e359e700 1 --2- 172.21.15.92:0/3691954764 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28d0003a30 0x7f28e414d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:27.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.963+0000 7f28e359e700 1 -- 172.21.15.92:0/3691954764 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28e4158660 msgr2=0x7f28e415b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:27.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.963+0000 7f28e359e700 1 --2- 172.21.15.92:0/3691954764 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28e4158660 0x7f28e415b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:27.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.963+0000 7f28e359e700 1 -- 172.21.15.92:0/3691954764 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e40cfff0 con 0x7f28e4159200 2023-11-28T11:22:27.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.964+0000 7f28e359e700 1 --2- 172.21.15.92:0/3691954764 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28e4159200 0x7f28e4159670 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f28e40b4c50 tx=0x7f28dc0048c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:27.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.964+0000 7f28db7fe700 1 -- 172.21.15.92:0/3691954764 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28dc007940 con 0x7f28e4159200 2023-11-28T11:22:27.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.964+0000 7f28db7fe700 1 -- 172.21.15.92:0/3691954764 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28dc00f050 con 0x7f28e4159200 2023-11-28T11:22:27.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.964+0000 7f28e91ff700 1 -- 172.21.15.92:0/3691954764 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28e40cfd10 con 0x7f28e4159200 2023-11-28T11:22:27.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.964+0000 7f28db7fe700 1 -- 172.21.15.92:0/3691954764 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28dc007cc0 con 0x7f28e4159200 2023-11-28T11:22:27.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.964+0000 7f28e91ff700 1 -- 172.21.15.92:0/3691954764 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28e40d29c0 con 0x7f28e4159200 2023-11-28T11:22:27.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.965+0000 7f28e91ff700 1 -- 172.21.15.92:0/3691954764 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28e4067880 con 0x7f28e4159200 2023-11-28T11:22:27.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.966+0000 7f28db7fe700 1 -- 172.21.15.92:0/3691954764 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f28dc01e460 con 0x7f28e4159200 2023-11-28T11:22:27.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.967+0000 7f28db7fe700 1 --2- 172.21.15.92:0/3691954764 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f28c40682f0 0x7f28c406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:27.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.967+0000 7f28db7fe700 1 -- 172.21.15.92:0/3691954764 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f28dc0860d0 con 0x7f28e4159200 2023-11-28T11:22:27.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.967+0000 7f28e2d9d700 1 --2- 172.21.15.92:0/3691954764 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f28c40682f0 0x7f28c406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:27.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.968+0000 7f28e2d9d700 1 --2- 172.21.15.92:0/3691954764 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f28c40682f0 0x7f28c406a7a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f28d40090b0 tx=0x7f28d401f040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:27.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:27.971+0000 7f28db7fe700 1 -- 172.21.15.92:0/3691954764 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f28dc0526a0 con 0x7f28e4159200 2023-11-28T11:22:28.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.204+0000 7f28e91ff700 1 -- 172.21.15.92:0/3691954764 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f28e4158420 con 0x7f28e4159200 2023-11-28T11:22:28.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.205+0000 7f28db7fe700 1 -- 172.21.15.92:0/3691954764 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4645 (secure 0 0 0) 0x7f28dc055cc0 con 0x7f28e4159200 2023-11-28T11:22:28.207 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:22:28.207 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4976,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/3102603064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3102603064},{"type":"v1","addr":"172.21.15.169:6834","nonce":3102603064}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4979,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/79247121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":79247121},{"type":"v1","addr":"172.21.15.92:6837","nonce":79247121}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5221,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/219797192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":219797192},{"type":"v1","addr":"172.21.15.169:6835","nonce":219797192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:22:24.715447+0000","modified":"2023-11-28T11:22:27.733117+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5298},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5298":{"gid":5298,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.92:6835/1577294649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1577294649},{"type":"v1","addr":"172.21.15.92:6835","nonce":1577294649}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T11:22:28.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.207+0000 7f28e91ff700 1 -- 172.21.15.92:0/3691954764 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f28c40682f0 msgr2=0x7f28c406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:28.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.207+0000 7f28e91ff700 1 --2- 172.21.15.92:0/3691954764 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f28c40682f0 0x7f28c406a7a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f28d40090b0 tx=0x7f28d401f040 comp rx=0 tx=0).stop 2023-11-28T11:22:28.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.207+0000 7f28e91ff700 1 -- 172.21.15.92:0/3691954764 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28e4159200 msgr2=0x7f28e4159670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:28.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.207+0000 7f28e91ff700 1 --2- 172.21.15.92:0/3691954764 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28e4159200 0x7f28e4159670 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f28e40b4c50 tx=0x7f28dc0048c0 comp rx=0 tx=0).stop 2023-11-28T11:22:28.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.207+0000 7f28e91ff700 1 -- 172.21.15.92:0/3691954764 shutdown_connections 2023-11-28T11:22:28.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.207+0000 7f28e91ff700 1 --2- 172.21.15.92:0/3691954764 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f28c40682f0 0x7f28c406a7a0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:28.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.207+0000 7f28e91ff700 1 --2- 172.21.15.92:0/3691954764 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28e4159200 0x7f28e4159670 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:28.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.207+0000 7f28e91ff700 1 --2- 172.21.15.92:0/3691954764 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28d0003a30 0x7f28e414d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:28.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.207+0000 7f28e91ff700 1 --2- 172.21.15.92:0/3691954764 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28e4158660 0x7f28e415b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:28.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.207+0000 7f28e91ff700 1 -- 172.21.15.92:0/3691954764 >> 172.21.15.92:0/3691954764 conn(0x7f28e40719a0 msgr2=0x7f28e40b27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:28.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.207+0000 7f28e91ff700 1 -- 172.21.15.92:0/3691954764 shutdown_connections 2023-11-28T11:22:28.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.207+0000 7f28e91ff700 1 -- 172.21.15.92:0/3691954764 wait complete. 2023-11-28T11:22:28.210 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 17 2023-11-28T11:22:28.227 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:22:28.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.422+0000 7fd34b9ff700 1 Processor -- start 2023-11-28T11:22:28.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.422+0000 7fd34b9ff700 1 -- start start 2023-11-28T11:22:28.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.424+0000 7fd34b9ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd344159080 0x7fd3440ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:28.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.424+0000 7fd34b9ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd34406d870 con 0x7fd344071020 2023-11-28T11:22:28.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.424+0000 7fd34b9ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd34406cc60 con 0x7fd344158cb0 2023-11-28T11:22:28.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.424+0000 7fd34b9ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd34406cf40 con 0x7fd344159080 2023-11-28T11:22:28.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.424+0000 7fd349f9c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd344071020 0x7fd3440cc550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33828/0 (socket says 172.21.15.92:33828) 2023-11-28T11:22:28.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.424+0000 7fd349f9c700 1 -- 172.21.15.92:0/3375675344 learned_addr learned my addr 172.21.15.92:0/3375675344 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:22:28.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.424+0000 7fd348f9a700 1 --2- 172.21.15.92:0/3375675344 >> v2:172.21.15.169:3300/0 conn(0x7fd344159080 0x7fd3440ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:28.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.424+0000 7fd33bfff700 1 -- 172.21.15.92:0/3375675344 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3348181782 0 0) 0x7fd34406d870 con 0x7fd344071020 2023-11-28T11:22:28.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.424+0000 7fd33bfff700 1 -- 172.21.15.92:0/3375675344 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3300035b0 con 0x7fd344071020 2023-11-28T11:22:28.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.425+0000 7fd33bfff700 1 -- 172.21.15.92:0/3375675344 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3696975720 0 0) 0x7fd3300035b0 con 0x7fd344071020 2023-11-28T11:22:28.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.425+0000 7fd33bfff700 1 -- 172.21.15.92:0/3375675344 >> v2:172.21.15.169:3300/0 conn(0x7fd344159080 msgr2=0x7fd3440ca1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:28.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.425+0000 7fd33bfff700 1 --2- 172.21.15.92:0/3375675344 >> v2:172.21.15.169:3300/0 conn(0x7fd344159080 0x7fd3440ca1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:28.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.425+0000 7fd33bfff700 1 -- 172.21.15.92:0/3375675344 >> v1:172.21.15.169:6789/0 conn(0x7fd344158cb0 legacy=0x7fd344070910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:28.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.425+0000 7fd33bfff700 1 -- 172.21.15.92:0/3375675344 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd34406d220 con 0x7fd344071020 2023-11-28T11:22:28.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.425+0000 7fd33bfff700 1 -- 172.21.15.92:0/3375675344 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532161587 0 0) 0x7fd33c0019f0 con 0x7fd344071020 2023-11-28T11:22:28.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.425+0000 7fd33bfff700 1 -- 172.21.15.92:0/3375675344 >> v1:172.21.15.92:6789/0 conn(0x7fd344071020 legacy=0x7fd3440cc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:28.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.425+0000 7fd33bfff700 1 --2- 172.21.15.92:0/3375675344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd330003a50 0x7fd330003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:28.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.425+0000 7fd33bfff700 1 --2- 172.21.15.92:0/3375675344 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd330004850 0x7fd330004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:28.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.425+0000 7fd33bfff700 1 --2- 172.21.15.92:0/3375675344 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd330005200 0x7fd33000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:28.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.425+0000 7fd33bfff700 1 -- 172.21.15.92:0/3375675344 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd34406cf40 con 0x7fd330004850 2023-11-28T11:22:28.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.425+0000 7fd33bfff700 1 -- 172.21.15.92:0/3375675344 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd33000e2d0 con 0x7fd330003a50 2023-11-28T11:22:28.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.425+0000 7fd33bfff700 1 -- 172.21.15.92:0/3375675344 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd33000e490 con 0x7fd330005200 2023-11-28T11:22:28.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.425+0000 7fd348f9a700 1 --2- 172.21.15.92:0/3375675344 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd330004850 0x7fd330004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:28.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.425+0000 7fd34979b700 1 --2- 172.21.15.92:0/3375675344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd330003a50 0x7fd330003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:28.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.425+0000 7fd349f9c700 1 --2- 172.21.15.92:0/3375675344 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd330005200 0x7fd33000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:28.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.425+0000 7fd348f9a700 1 -- 172.21.15.92:0/3375675344 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd330005200 msgr2=0x7fd33000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:28.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.425+0000 7fd348f9a700 1 --2- 172.21.15.92:0/3375675344 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd330005200 0x7fd33000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:28.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.425+0000 7fd348f9a700 1 -- 172.21.15.92:0/3375675344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd330003a50 msgr2=0x7fd330003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:28.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.425+0000 7fd348f9a700 1 --2- 172.21.15.92:0/3375675344 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd330003a50 0x7fd330003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:28.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.425+0000 7fd348f9a700 1 -- 172.21.15.92:0/3375675344 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3300035b0 con 0x7fd330004850 2023-11-28T11:22:28.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.425+0000 7fd349f9c700 1 -- 172.21.15.92:0/3375675344 reap_dead start 2023-11-28T11:22:28.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.426+0000 7fd348f9a700 1 --2- 172.21.15.92:0/3375675344 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd330004850 0x7fd330004cc0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fd33400aae0 tx=0x7fd330005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ee0583dba03aa6f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:28.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.426+0000 7fd33bfff700 1 -- 172.21.15.92:0/3375675344 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33400e960 con 0x7fd330004850 2023-11-28T11:22:28.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.426+0000 7fd33bfff700 1 -- 172.21.15.92:0/3375675344 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd33400eac0 con 0x7fd330004850 2023-11-28T11:22:28.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.426+0000 7fd33bfff700 1 -- 172.21.15.92:0/3375675344 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33400ee70 con 0x7fd330004850 2023-11-28T11:22:28.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.426+0000 7fd34b9ff700 1 -- 172.21.15.92:0/3375675344 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd330004850 msgr2=0x7fd330004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:28.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.426+0000 7fd34b9ff700 1 --2- 172.21.15.92:0/3375675344 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd330004850 0x7fd330004cc0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fd33400aae0 tx=0x7fd330005880 comp rx=0 tx=0).stop 2023-11-28T11:22:28.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.427+0000 7fd34b9ff700 1 -- 172.21.15.92:0/3375675344 shutdown_connections 2023-11-28T11:22:28.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.427+0000 7fd34b9ff700 1 --2- 172.21.15.92:0/3375675344 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd330004850 0x7fd330004cc0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:28.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.427+0000 7fd34b9ff700 1 -- 172.21.15.92:0/3375675344 >> 172.21.15.92:0/3375675344 conn(0x7fd3440cb490 msgr2=0x7fd3440cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:28.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.427+0000 7fd34b9ff700 1 -- 172.21.15.92:0/3375675344 shutdown_connections 2023-11-28T11:22:28.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.427+0000 7fd34b9ff700 1 -- 172.21.15.92:0/3375675344 wait complete. 2023-11-28T11:22:28.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.429+0000 7fd34b9ff700 1 Processor -- start 2023-11-28T11:22:28.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.429+0000 7fd34b9ff700 1 -- start start 2023-11-28T11:22:28.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.430+0000 7fd34b9ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd330004850 0x7fd3440b2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:28.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.430+0000 7fd34b9ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3441ee0f0 0x7fd344150fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:28.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.430+0000 7fd34b9ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd3440cb860 0x7fd3440cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:28.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.431+0000 7fd34b9ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3440b2790 con 0x7fd330004850 2023-11-28T11:22:28.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.431+0000 7fd34b9ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd344151510 con 0x7fd3440cb860 2023-11-28T11:22:28.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.431+0000 7fd34b9ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3440cc210 con 0x7fd3441ee0f0 2023-11-28T11:22:28.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.431+0000 7fd34979b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd330004850 0x7fd3440b2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:28.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.431+0000 7fd34979b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd330004850 0x7fd3440b2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51372/0 (socket says 172.21.15.92:51372) 2023-11-28T11:22:28.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.431+0000 7fd34979b700 1 -- 172.21.15.92:0/1325622506 learned_addr learned my addr 172.21.15.92:0/1325622506 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:28.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.431+0000 7fd348f9a700 1 --2- 172.21.15.92:0/1325622506 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3441ee0f0 0x7fd344150fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:28.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.431+0000 7fd349f9c700 1 --2- 172.21.15.92:0/1325622506 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd3440cb860 0x7fd3440cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:28.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.431+0000 7fd34979b700 1 -- 172.21.15.92:0/1325622506 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3441ee0f0 msgr2=0x7fd344150fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:28.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.431+0000 7fd34979b700 1 --2- 172.21.15.92:0/1325622506 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3441ee0f0 0x7fd344150fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:28.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.431+0000 7fd34979b700 1 -- 172.21.15.92:0/1325622506 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd3440cb860 msgr2=0x7fd3440cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:28.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.431+0000 7fd34979b700 1 --2- 172.21.15.92:0/1325622506 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd3440cb860 0x7fd3440cbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:28.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.431+0000 7fd34979b700 1 -- 172.21.15.92:0/1325622506 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd344071b20 con 0x7fd330004850 2023-11-28T11:22:28.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.431+0000 7fd34979b700 1 --2- 172.21.15.92:0/1325622506 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd330004850 0x7fd3440b2250 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd34000e580 tx=0x7fd34000e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:28.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.432+0000 7fd339ffb700 1 -- 172.21.15.92:0/1325622506 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3400052e0 con 0x7fd330004850 2023-11-28T11:22:28.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.432+0000 7fd339ffb700 1 -- 172.21.15.92:0/1325622506 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd340005460 con 0x7fd330004850 2023-11-28T11:22:28.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.432+0000 7fd339ffb700 1 -- 172.21.15.92:0/1325622506 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3400038c0 con 0x7fd330004850 2023-11-28T11:22:28.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.432+0000 7fd34b9ff700 1 -- 172.21.15.92:0/1325622506 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd34406d500 con 0x7fd330004850 2023-11-28T11:22:28.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.432+0000 7fd34b9ff700 1 -- 172.21.15.92:0/1325622506 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3440c9300 con 0x7fd330004850 2023-11-28T11:22:28.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.433+0000 7fd34b9ff700 1 -- 172.21.15.92:0/1325622506 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd344067880 con 0x7fd330004850 2023-11-28T11:22:28.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.434+0000 7fd339ffb700 1 -- 172.21.15.92:0/1325622506 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd34001b030 con 0x7fd330004850 2023-11-28T11:22:28.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.435+0000 7fd339ffb700 1 --2- 172.21.15.92:0/1325622506 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd3280682f0 0x7fd32806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:28.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.435+0000 7fd339ffb700 1 -- 172.21.15.92:0/1325622506 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fd340085280 con 0x7fd330004850 2023-11-28T11:22:28.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.435+0000 7fd348f9a700 1 --2- 172.21.15.92:0/1325622506 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd3280682f0 0x7fd32806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:28.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.436+0000 7fd348f9a700 1 --2- 172.21.15.92:0/1325622506 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd3280682f0 0x7fd32806a7a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd3440c8ce0 tx=0x7fd33401d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:28.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.439+0000 7fd339ffb700 1 -- 172.21.15.92:0/1325622506 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd34004d5e0 con 0x7fd330004850 2023-11-28T11:22:28.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.631+0000 7fd34b9ff700 1 -- 172.21.15.92:0/1325622506 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd3440716d0 con 0x7fd330004850 2023-11-28T11:22:28.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.632+0000 7fd339ffb700 1 -- 172.21.15.92:0/1325622506 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+18128 (secure 0 0 0) 0x7fd340051a50 con 0x7fd330004850 2023-11-28T11:22:28.635 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:22:28.635 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":42,"fsid":"68956e0c-c16f-47cd-a220-f98364b0831d","created":"2023-11-28T11:20:28.697573+0000","modified":"2023-11-28T11:22:25.719298+0000","last_up_change":"2023-11-28T11:20:41.257011+0000","last_in_change":"2023-11-28T11:20:32.249767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:20:41.925237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:20:48.682497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:22:22.942125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-28T11:22:23.684528+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6abe483f-0e03-4a26-b9cf-62041291747e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":115823},{"type":"v1","addr":"172.21.15.92:6809","nonce":115823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":115823},{"type":"v1","addr":"172.21.15.92:6813","nonce":115823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":115823},{"type":"v1","addr":"172.21.15.92:6821","nonce":115823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":115823},{"type":"v1","addr":"172.21.15.92:6817","nonce":115823}]},"public_addr":"172.21.15.92:6809/115823","cluster_addr":"172.21.15.92:6813/115823","heartbeat_back_addr":"172.21.15.92:6821/115823","heartbeat_front_addr":"172.21.15.92:6817/115823","state":["exists","up"]},{"osd":1,"uuid":"13c1de73-1a8c-4d7e-8a28-9dae52dfe089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":115824},{"type":"v1","addr":"172.21.15.92:6812","nonce":115824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":115824},{"type":"v1","addr":"172.21.15.92:6816","nonce":115824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":115824},{"type":"v1","addr":"172.21.15.92:6823","nonce":115824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":115824},{"type":"v1","addr":"172.21.15.92:6820","nonce":115824}]},"public_addr":"172.21.15.92:6812/115824","cluster_addr":"172.21.15.92:6816/115824","heartbeat_back_addr":"172.21.15.92:6823/115824","heartbeat_front_addr":"172.21.15.92:6820/115824","state":["exists","up"]},{"osd":2,"uuid":"25788a55-112f-4f00-810d-2bc37cf76778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":115821},{"type":"v1","addr":"172.21.15.92:6801","nonce":115821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":115821},{"type":"v1","addr":"172.21.15.92:6803","nonce":115821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":115821},{"type":"v1","addr":"172.21.15.92:6807","nonce":115821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":115821},{"type":"v1","addr":"172.21.15.92:6805","nonce":115821}]},"public_addr":"172.21.15.92:6801/115821","cluster_addr":"172.21.15.92:6803/115821","heartbeat_back_addr":"172.21.15.92:6807/115821","heartbeat_front_addr":"172.21.15.92:6805/115821","state":["exists","up"]},{"osd":3,"uuid":"adf54e24-33a9-41fe-9ce3-98a5ab6c158e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":115825},{"type":"v1","addr":"172.21.15.92:6825","nonce":115825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":115825},{"type":"v1","addr":"172.21.15.92:6827","nonce":115825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":115825},{"type":"v1","addr":"172.21.15.92:6831","nonce":115825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":115825},{"type":"v1","addr":"172.21.15.92:6829","nonce":115825}]},"public_addr":"172.21.15.92:6825/115825","cluster_addr":"172.21.15.92:6827/115825","heartbeat_back_addr":"172.21.15.92:6831/115825","heartbeat_front_addr":"172.21.15.92:6829/115825","state":["exists","up"]},{"osd":4,"uuid":"ccb9b747-f399-44ea-9a18-9db3152227e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":116035},{"type":"v1","addr":"172.21.15.169:6817","nonce":116035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":116035},{"type":"v1","addr":"172.21.15.169:6819","nonce":116035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":116035},{"type":"v1","addr":"172.21.15.169:6826","nonce":116035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":116035},{"type":"v1","addr":"172.21.15.169:6822","nonce":116035}]},"public_addr":"172.21.15.169:6817/116035","cluster_addr":"172.21.15.169:6819/116035","heartbeat_back_addr":"172.21.15.169:6826/116035","heartbeat_front_addr":"172.21.15.169:6822/116035","state":["exists","up"]},{"osd":5,"uuid":"308a1e63-75aa-4d7d-8a7c-cecc24714fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":116033},{"type":"v1","addr":"172.21.15.169:6801","nonce":116033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":116033},{"type":"v1","addr":"172.21.15.169:6803","nonce":116033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":116033},{"type":"v1","addr":"172.21.15.169:6807","nonce":116033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":116033},{"type":"v1","addr":"172.21.15.169:6805","nonce":116033}]},"public_addr":"172.21.15.169:6801/116033","cluster_addr":"172.21.15.169:6803/116033","heartbeat_back_addr":"172.21.15.169:6807/116033","heartbeat_front_addr":"172.21.15.169:6805/116033","state":["exists","up"]},{"osd":6,"uuid":"9c459b7b-b821-4b4b-b91e-6aaae271457c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":116034},{"type":"v1","addr":"172.21.15.169:6809","nonce":116034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":116034},{"type":"v1","addr":"172.21.15.169:6811","nonce":116034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":116034},{"type":"v1","addr":"172.21.15.169:6815","nonce":116034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":116034},{"type":"v1","addr":"172.21.15.169:6813","nonce":116034}]},"public_addr":"172.21.15.169:6809/116034","cluster_addr":"172.21.15.169:6811/116034","heartbeat_back_addr":"172.21.15.169:6815/116034","heartbeat_front_addr":"172.21.15.169:6813/116034","state":["exists","up"]},{"osd":7,"uuid":"1ca9aa4a-755d-4e28-a937-4d8dccab4df3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":116044},{"type":"v1","addr":"172.21.15.169:6823","nonce":116044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6825","nonce":116044},{"type":"v1","addr":"172.21.15.169:6827","nonce":116044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":116044},{"type":"v1","addr":"172.21.15.169:6831","nonce":116044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":116044},{"type":"v1","addr":"172.21.15.169:6829","nonce":116044}]},"public_addr":"172.21.15.169:6823/116044","cluster_addr":"172.21.15.169:6827/116044","heartbeat_back_addr":"172.21.15.169:6831/116044","heartbeat_front_addr":"172.21.15.169:6829/116044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.586781+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.869778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.508594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.860493+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.872267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.746933+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.611520+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.871446+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:22:28.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.635+0000 7fd34b9ff700 1 -- 172.21.15.92:0/1325622506 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd3280682f0 msgr2=0x7fd32806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:28.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.635+0000 7fd34b9ff700 1 --2- 172.21.15.92:0/1325622506 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd3280682f0 0x7fd32806a7a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd3440c8ce0 tx=0x7fd33401d040 comp rx=0 tx=0).stop 2023-11-28T11:22:28.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.635+0000 7fd34b9ff700 1 -- 172.21.15.92:0/1325622506 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd330004850 msgr2=0x7fd3440b2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:28.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.635+0000 7fd34b9ff700 1 --2- 172.21.15.92:0/1325622506 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd330004850 0x7fd3440b2250 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd34000e580 tx=0x7fd34000e940 comp rx=0 tx=0).stop 2023-11-28T11:22:28.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.635+0000 7fd34b9ff700 1 -- 172.21.15.92:0/1325622506 shutdown_connections 2023-11-28T11:22:28.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.635+0000 7fd34b9ff700 1 --2- 172.21.15.92:0/1325622506 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd3440cb860 0x7fd3440cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:28.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.635+0000 7fd34b9ff700 1 --2- 172.21.15.92:0/1325622506 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd3280682f0 0x7fd32806a7a0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:28.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.635+0000 7fd34b9ff700 1 --2- 172.21.15.92:0/1325622506 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd330004850 0x7fd3440b2250 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:28.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.635+0000 7fd34b9ff700 1 --2- 172.21.15.92:0/1325622506 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd3441ee0f0 0x7fd344150fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:28.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.635+0000 7fd34b9ff700 1 -- 172.21.15.92:0/1325622506 >> 172.21.15.92:0/1325622506 conn(0x7fd3440cb490 msgr2=0x7fd3440afd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:28.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.635+0000 7fd34b9ff700 1 -- 172.21.15.92:0/1325622506 shutdown_connections 2023-11-28T11:22:28.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.635+0000 7fd34b9ff700 1 -- 172.21.15.92:0/1325622506 wait complete. 2023-11-28T11:22:28.655 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T11:22:28.656 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:22:28.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.851+0000 7f4c651ff700 1 Processor -- start 2023-11-28T11:22:28.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.851+0000 7f4c651ff700 1 -- start start 2023-11-28T11:22:28.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.852+0000 7f4c651ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c600b8510 0x7f4c6006b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:28.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.852+0000 7f4c651ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4c600d0d60 0x7f4c600d11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:28.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.852+0000 7f4c651ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c600b88e0 con 0x7f4c600cd460 2023-11-28T11:22:28.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.852+0000 7f4c651ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c600cdbd0 con 0x7f4c600d0d60 2023-11-28T11:22:28.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.852+0000 7f4c651ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c600d02d0 con 0x7f4c600b8510 2023-11-28T11:22:28.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.853+0000 7f4c5ed9d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c600b8510 0x7f4c6006b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:28.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.853+0000 7f4c5f59e700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4c600d0d60 0x7f4c600d11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:28.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.853+0000 7f4c5e59c700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f4c600cd460 0x7f4c6006beb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.92:59458/0 (socket says 172.21.15.92:59458) 2023-11-28T11:22:28.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.853+0000 7f4c5e59c700 1 -- 172.21.15.92:0/1199079231 learned_addr learned my addr 172.21.15.92:0/1199079231 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:22:28.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.853+0000 7f4c5dd9b700 1 -- 172.21.15.92:0/1199079231 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4290387002 0 0) 0x7f4c600b88e0 con 0x7f4c600cd460 2023-11-28T11:22:28.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.853+0000 7f4c5dd9b700 1 -- 172.21.15.92:0/1199079231 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c480035b0 con 0x7f4c600cd460 2023-11-28T11:22:28.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.853+0000 7f4c5f59e700 1 -- 172.21.15.92:0/1199079231 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c600b8510 msgr2=0x7f4c6006b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:28.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.853+0000 7f4c5f59e700 1 --2- 172.21.15.92:0/1199079231 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c600b8510 0x7f4c6006b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:28.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.853+0000 7f4c5f59e700 1 -- 172.21.15.92:0/1199079231 >> v1:172.21.15.169:6789/0 conn(0x7f4c600cd460 legacy=0x7f4c6006beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:28.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.853+0000 7f4c5f59e700 1 -- 172.21.15.92:0/1199079231 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c600cd8f0 con 0x7f4c600d0d60 2023-11-28T11:22:28.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.853+0000 7f4c5ed9d700 1 --2- 172.21.15.92:0/1199079231 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c600b8510 0x7f4c6006b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:22:28.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.854+0000 7f4c5f59e700 1 --2- 172.21.15.92:0/1199079231 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c600d0d60 0x7f4c600d11a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4c58009dc0 tx=0x7f4c5800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4405730d737476c6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:28.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.854+0000 7f4c5dd9b700 1 -- 172.21.15.92:0/1199079231 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5800e070 con 0x7f4c600d0d60 2023-11-28T11:22:28.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.854+0000 7f4c5dd9b700 1 -- 172.21.15.92:0/1199079231 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c5800bae0 con 0x7f4c600d0d60 2023-11-28T11:22:28.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.854+0000 7f4c5dd9b700 1 -- 172.21.15.92:0/1199079231 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5800be90 con 0x7f4c600d0d60 2023-11-28T11:22:28.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.855+0000 7f4c651ff700 1 -- 172.21.15.92:0/1199079231 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c600d0d60 msgr2=0x7f4c600d11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:28.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.855+0000 7f4c651ff700 1 --2- 172.21.15.92:0/1199079231 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c600d0d60 0x7f4c600d11a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4c58009dc0 tx=0x7f4c5800b330 comp rx=0 tx=0).stop 2023-11-28T11:22:28.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.855+0000 7f4c651ff700 1 -- 172.21.15.92:0/1199079231 shutdown_connections 2023-11-28T11:22:28.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.855+0000 7f4c651ff700 1 --2- 172.21.15.92:0/1199079231 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c600d0d60 0x7f4c600d11a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:28.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.855+0000 7f4c651ff700 1 --2- 172.21.15.92:0/1199079231 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c600b8510 0x7f4c6006b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:28.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.855+0000 7f4c651ff700 1 -- 172.21.15.92:0/1199079231 >> 172.21.15.92:0/1199079231 conn(0x7f4c600719a0 msgr2=0x7f4c60071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:28.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.855+0000 7f4c651ff700 1 -- 172.21.15.92:0/1199079231 shutdown_connections 2023-11-28T11:22:28.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.855+0000 7f4c651ff700 1 -- 172.21.15.92:0/1199079231 wait complete. 2023-11-28T11:22:28.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.857+0000 7f4c651ff700 1 Processor -- start 2023-11-28T11:22:28.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.857+0000 7f4c651ff700 1 -- start start 2023-11-28T11:22:28.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.858+0000 7f4c651ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c600b8510 0x7f4c600c7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:28.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.858+0000 7f4c651ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c600cd460 0x7f4c601f1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:28.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.858+0000 7f4c651ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c600d0d60 0x7f4c601f1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:28.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.858+0000 7f4c651ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c600acc90 con 0x7f4c600cd460 2023-11-28T11:22:28.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.858+0000 7f4c651ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c600c83b0 con 0x7f4c600d0d60 2023-11-28T11:22:28.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.858+0000 7f4c651ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c600c80d0 con 0x7f4c600b8510 2023-11-28T11:22:28.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.858+0000 7f4c5e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c600cd460 0x7f4c601f1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:28.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.858+0000 7f4c5e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c600cd460 0x7f4c601f1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51382/0 (socket says 172.21.15.92:51382) 2023-11-28T11:22:28.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.858+0000 7f4c5e59c700 1 -- 172.21.15.92:0/2884173278 learned_addr learned my addr 172.21.15.92:0/2884173278 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:28.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.859+0000 7f4c5f59e700 1 --2- 172.21.15.92:0/2884173278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c600d0d60 0x7f4c601f1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:28.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.859+0000 7f4c5ed9d700 1 --2- 172.21.15.92:0/2884173278 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c600b8510 0x7f4c600c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:28.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.859+0000 7f4c5e59c700 1 -- 172.21.15.92:0/2884173278 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c600b8510 msgr2=0x7f4c600c7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:28.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.859+0000 7f4c5e59c700 1 --2- 172.21.15.92:0/2884173278 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c600b8510 0x7f4c600c7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:28.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.859+0000 7f4c5e59c700 1 -- 172.21.15.92:0/2884173278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c600d0d60 msgr2=0x7f4c601f1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:28.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.859+0000 7f4c5e59c700 1 --2- 172.21.15.92:0/2884173278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c600d0d60 0x7f4c601f1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:28.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.859+0000 7f4c5e59c700 1 -- 172.21.15.92:0/2884173278 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c500094a0 con 0x7f4c600cd460 2023-11-28T11:22:28.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.859+0000 7f4c5f59e700 1 --2- 172.21.15.92:0/2884173278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c600d0d60 0x7f4c601f1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:22:28.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.859+0000 7f4c5e59c700 1 --2- 172.21.15.92:0/2884173278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c600cd460 0x7f4c601f1750 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f4c5400c700 tx=0x7f4c5400cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:28.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.859+0000 7f4c477fe700 1 -- 172.21.15.92:0/2884173278 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c54005d70 con 0x7f4c600cd460 2023-11-28T11:22:28.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.859+0000 7f4c477fe700 1 -- 172.21.15.92:0/2884173278 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c540070d0 con 0x7f4c600cd460 2023-11-28T11:22:28.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.859+0000 7f4c651ff700 1 -- 172.21.15.92:0/2884173278 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c58009a70 con 0x7f4c600cd460 2023-11-28T11:22:28.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.859+0000 7f4c477fe700 1 -- 172.21.15.92:0/2884173278 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c54007460 con 0x7f4c600cd460 2023-11-28T11:22:28.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.860+0000 7f4c651ff700 1 -- 172.21.15.92:0/2884173278 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c600cfff0 con 0x7f4c600cd460 2023-11-28T11:22:28.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.861+0000 7f4c651ff700 1 -- 172.21.15.92:0/2884173278 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c600cfd10 con 0x7f4c600cd460 2023-11-28T11:22:28.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.861+0000 7f4c477fe700 1 -- 172.21.15.92:0/2884173278 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4c54019030 con 0x7f4c600cd460 2023-11-28T11:22:28.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.862+0000 7f4c477fe700 1 --2- 172.21.15.92:0/2884173278 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f4c4c0682f0 0x7f4c4c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:28.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.862+0000 7f4c477fe700 1 -- 172.21.15.92:0/2884173278 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f4c54085400 con 0x7f4c600cd460 2023-11-28T11:22:28.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.862+0000 7f4c5ed9d700 1 --2- 172.21.15.92:0/2884173278 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f4c4c0682f0 0x7f4c4c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:28.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.863+0000 7f4c5ed9d700 1 --2- 172.21.15.92:0/2884173278 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f4c4c0682f0 0x7f4c4c06a7a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4c6015c850 tx=0x7f4c50008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:28.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:28.867+0000 7f4c477fe700 1 -- 172.21.15.92:0/2884173278 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4c540519b0 con 0x7f4c600cd460 2023-11-28T11:22:29.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.055+0000 7f4c651ff700 1 -- 172.21.15.92:0/2884173278 --> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4c60067880 con 0x7f4c4c0682f0 2023-11-28T11:22:29.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.065+0000 7f4c477fe700 1 -- 172.21.15.92:0/2884173278 <== mgr.4103 v2:172.21.15.92:6832/115482 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311324 (secure 0 0 0) 0x7f4c60067880 con 0x7f4c4c0682f0 2023-11-28T11:22:29.072 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:22:29.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.069+0000 7f4c651ff700 1 -- 172.21.15.92:0/2884173278 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f4c4c0682f0 msgr2=0x7f4c4c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:29.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.069+0000 7f4c651ff700 1 --2- 172.21.15.92:0/2884173278 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f4c4c0682f0 0x7f4c4c06a7a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4c6015c850 tx=0x7f4c50008040 comp rx=0 tx=0).stop 2023-11-28T11:22:29.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.069+0000 7f4c651ff700 1 -- 172.21.15.92:0/2884173278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c600cd460 msgr2=0x7f4c601f1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:29.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.069+0000 7f4c651ff700 1 --2- 172.21.15.92:0/2884173278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c600cd460 0x7f4c601f1750 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f4c5400c700 tx=0x7f4c5400cac0 comp rx=0 tx=0).stop 2023-11-28T11:22:29.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.070+0000 7f4c651ff700 1 -- 172.21.15.92:0/2884173278 shutdown_connections 2023-11-28T11:22:29.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.070+0000 7f4c651ff700 1 --2- 172.21.15.92:0/2884173278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4c600d0d60 0x7f4c601f1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:29.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.070+0000 7f4c651ff700 1 --2- 172.21.15.92:0/2884173278 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4c600b8510 0x7f4c600c7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:29.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.070+0000 7f4c651ff700 1 --2- 172.21.15.92:0/2884173278 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f4c4c0682f0 0x7f4c4c06a7a0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:29.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.070+0000 7f4c651ff700 1 --2- 172.21.15.92:0/2884173278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c600cd460 0x7f4c601f1750 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:29.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.070+0000 7f4c651ff700 1 -- 172.21.15.92:0/2884173278 >> 172.21.15.92:0/2884173278 conn(0x7f4c600719a0 msgr2=0x7f4c600b71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:29.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.070+0000 7f4c651ff700 1 -- 172.21.15.92:0/2884173278 shutdown_connections 2023-11-28T11:22:29.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.070+0000 7f4c651ff700 1 -- 172.21.15.92:0/2884173278 wait complete. 2023-11-28T11:22:29.077 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-11-28T11:22:29.090 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":88,"stamp":"2023-11-28T11:22:27.880247+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":157,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":71832,"kb_used_data":5016,"kb_used_omap":0,"kb_used_meta":66752,"kb_avail":754902888,"statfs":{"total":773094113280,"available":773020557312,"internally_reserved":0,"allocated":5136384,"data_stored":2928009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68354048},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.669966"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.106893+0000","last_change":"2023-11-28T11:22:24.743559+0000","last_active":"2023-11-28T11:22:26.106893+0000","last_peered":"2023-11-28T11:22:26.106893+0000","last_clean":"2023-11-28T11:22:26.106893+0000","last_became_active":"2023-11-28T11:22:24.743278+0000","last_became_peered":"2023-11-28T11:22:24.743278+0000","last_unstale":"2023-11-28T11:22:26.106893+0000","last_undegraded":"2023-11-28T11:22:26.106893+0000","last_fullsized":"2023-11-28T11:22:26.106893+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:52:15.093133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.735233+0000","last_change":"2023-11-28T11:22:24.763133+0000","last_active":"2023-11-28T11:22:25.735233+0000","last_peered":"2023-11-28T11:22:25.735233+0000","last_clean":"2023-11-28T11:22:25.735233+0000","last_became_active":"2023-11-28T11:22:24.762707+0000","last_became_peered":"2023-11-28T11:22:24.762707+0000","last_unstale":"2023-11-28T11:22:25.735233+0000","last_undegraded":"2023-11-28T11:22:25.735233+0000","last_fullsized":"2023-11-28T11:22:25.735233+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:00:53.225852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.734437+0000","last_change":"2023-11-28T11:22:24.741466+0000","last_active":"2023-11-28T11:22:25.734437+0000","last_peered":"2023-11-28T11:22:25.734437+0000","last_clean":"2023-11-28T11:22:25.734437+0000","last_became_active":"2023-11-28T11:22:24.741194+0000","last_became_peered":"2023-11-28T11:22:24.741194+0000","last_unstale":"2023-11-28T11:22:25.734437+0000","last_undegraded":"2023-11-28T11:22:25.734437+0000","last_fullsized":"2023-11-28T11:22:25.734437+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:50:58.725549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.918843+0000","last_change":"2023-11-28T11:22:24.750046+0000","last_active":"2023-11-28T11:22:25.918843+0000","last_peered":"2023-11-28T11:22:25.918843+0000","last_clean":"2023-11-28T11:22:25.918843+0000","last_became_active":"2023-11-28T11:22:24.749769+0000","last_became_peered":"2023-11-28T11:22:24.749769+0000","last_unstale":"2023-11-28T11:22:25.918843+0000","last_undegraded":"2023-11-28T11:22:25.918843+0000","last_fullsized":"2023-11-28T11:22:25.918843+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:48:58.095068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.734980+0000","last_change":"2023-11-28T11:22:24.763161+0000","last_active":"2023-11-28T11:22:25.734980+0000","last_peered":"2023-11-28T11:22:25.734980+0000","last_clean":"2023-11-28T11:22:25.734980+0000","last_became_active":"2023-11-28T11:22:24.762738+0000","last_became_peered":"2023-11-28T11:22:24.762738+0000","last_unstale":"2023-11-28T11:22:25.734980+0000","last_undegraded":"2023-11-28T11:22:25.734980+0000","last_fullsized":"2023-11-28T11:22:25.734980+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:39:46.327957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.106038+0000","last_change":"2023-11-28T11:22:24.744553+0000","last_active":"2023-11-28T11:22:26.106038+0000","last_peered":"2023-11-28T11:22:26.106038+0000","last_clean":"2023-11-28T11:22:26.106038+0000","last_became_active":"2023-11-28T11:22:24.742417+0000","last_became_peered":"2023-11-28T11:22:24.742417+0000","last_unstale":"2023-11-28T11:22:26.106038+0000","last_undegraded":"2023-11-28T11:22:26.106038+0000","last_fullsized":"2023-11-28T11:22:26.106038+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:20:38.630366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.106443+0000","last_change":"2023-11-28T11:22:24.747287+0000","last_active":"2023-11-28T11:22:26.106443+0000","last_peered":"2023-11-28T11:22:26.106443+0000","last_clean":"2023-11-28T11:22:26.106443+0000","last_became_active":"2023-11-28T11:22:24.744650+0000","last_became_peered":"2023-11-28T11:22:24.744650+0000","last_unstale":"2023-11-28T11:22:26.106443+0000","last_undegraded":"2023-11-28T11:22:26.106443+0000","last_fullsized":"2023-11-28T11:22:26.106443+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:25:57.624641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.918634+0000","last_change":"2023-11-28T11:22:24.743031+0000","last_active":"2023-11-28T11:22:25.918634+0000","last_peered":"2023-11-28T11:22:25.918634+0000","last_clean":"2023-11-28T11:22:25.918634+0000","last_became_active":"2023-11-28T11:22:24.742767+0000","last_became_peered":"2023-11-28T11:22:24.742767+0000","last_unstale":"2023-11-28T11:22:25.918634+0000","last_undegraded":"2023-11-28T11:22:25.918634+0000","last_fullsized":"2023-11-28T11:22:25.918634+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:45:09.980004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.919231+0000","last_change":"2023-11-28T11:22:24.756248+0000","last_active":"2023-11-28T11:22:25.919231+0000","last_peered":"2023-11-28T11:22:25.919231+0000","last_clean":"2023-11-28T11:22:25.919231+0000","last_became_active":"2023-11-28T11:22:24.754413+0000","last_became_peered":"2023-11-28T11:22:24.754413+0000","last_unstale":"2023-11-28T11:22:25.919231+0000","last_undegraded":"2023-11-28T11:22:25.919231+0000","last_fullsized":"2023-11-28T11:22:25.919231+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:25:57.624641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.919187+0000","last_change":"2023-11-28T11:22:24.747851+0000","last_active":"2023-11-28T11:22:25.919187+0000","last_peered":"2023-11-28T11:22:25.919187+0000","last_clean":"2023-11-28T11:22:25.919187+0000","last_became_active":"2023-11-28T11:22:24.747602+0000","last_became_peered":"2023-11-28T11:22:24.747602+0000","last_unstale":"2023-11-28T11:22:25.919187+0000","last_undegraded":"2023-11-28T11:22:25.919187+0000","last_fullsized":"2023-11-28T11:22:25.919187+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:02:19.560108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.918993+0000","last_change":"2023-11-28T11:22:24.751122+0000","last_active":"2023-11-28T11:22:25.918993+0000","last_peered":"2023-11-28T11:22:25.918993+0000","last_clean":"2023-11-28T11:22:25.918993+0000","last_became_active":"2023-11-28T11:22:24.750852+0000","last_became_peered":"2023-11-28T11:22:24.750852+0000","last_unstale":"2023-11-28T11:22:25.918993+0000","last_undegraded":"2023-11-28T11:22:25.918993+0000","last_fullsized":"2023-11-28T11:22:25.918993+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:07:08.708946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.735330+0000","last_change":"2023-11-28T11:22:24.753686+0000","last_active":"2023-11-28T11:22:25.735330+0000","last_peered":"2023-11-28T11:22:25.735330+0000","last_clean":"2023-11-28T11:22:25.735330+0000","last_became_active":"2023-11-28T11:22:24.752399+0000","last_became_peered":"2023-11-28T11:22:24.752399+0000","last_unstale":"2023-11-28T11:22:25.735330+0000","last_undegraded":"2023-11-28T11:22:25.735330+0000","last_fullsized":"2023-11-28T11:22:25.735330+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:30:58.618362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.105748+0000","last_change":"2023-11-28T11:22:24.749386+0000","last_active":"2023-11-28T11:22:26.105748+0000","last_peered":"2023-11-28T11:22:26.105748+0000","last_clean":"2023-11-28T11:22:26.105748+0000","last_became_active":"2023-11-28T11:22:24.749136+0000","last_became_peered":"2023-11-28T11:22:24.749136+0000","last_unstale":"2023-11-28T11:22:26.105748+0000","last_undegraded":"2023-11-28T11:22:26.105748+0000","last_fullsized":"2023-11-28T11:22:26.105748+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:33:45.044082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.918900+0000","last_change":"2023-11-28T11:22:24.757076+0000","last_active":"2023-11-28T11:22:25.918900+0000","last_peered":"2023-11-28T11:22:25.918900+0000","last_clean":"2023-11-28T11:22:25.918900+0000","last_became_active":"2023-11-28T11:22:24.756832+0000","last_became_peered":"2023-11-28T11:22:24.756832+0000","last_unstale":"2023-11-28T11:22:25.918900+0000","last_undegraded":"2023-11-28T11:22:25.918900+0000","last_fullsized":"2023-11-28T11:22:25.918900+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:39:46.327957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.106031+0000","last_change":"2023-11-28T11:22:24.743790+0000","last_active":"2023-11-28T11:22:26.106031+0000","last_peered":"2023-11-28T11:22:26.106031+0000","last_clean":"2023-11-28T11:22:26.106031+0000","last_became_active":"2023-11-28T11:22:24.743589+0000","last_became_peered":"2023-11-28T11:22:24.743589+0000","last_unstale":"2023-11-28T11:22:26.106031+0000","last_undegraded":"2023-11-28T11:22:26.106031+0000","last_fullsized":"2023-11-28T11:22:26.106031+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:21:04.288976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.108698+0000","last_change":"2023-11-28T11:22:24.757376+0000","last_active":"2023-11-28T11:22:26.108698+0000","last_peered":"2023-11-28T11:22:26.108698+0000","last_clean":"2023-11-28T11:22:26.108698+0000","last_became_active":"2023-11-28T11:22:24.757115+0000","last_became_peered":"2023-11-28T11:22:24.757115+0000","last_unstale":"2023-11-28T11:22:26.108698+0000","last_undegraded":"2023-11-28T11:22:26.108698+0000","last_fullsized":"2023-11-28T11:22:26.108698+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:50:31.355306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.734681+0000","last_change":"2023-11-28T11:22:24.748161+0000","last_active":"2023-11-28T11:22:25.734681+0000","last_peered":"2023-11-28T11:22:25.734681+0000","last_clean":"2023-11-28T11:22:25.734681+0000","last_became_active":"2023-11-28T11:22:24.747874+0000","last_became_peered":"2023-11-28T11:22:24.747874+0000","last_unstale":"2023-11-28T11:22:25.734681+0000","last_undegraded":"2023-11-28T11:22:25.734681+0000","last_fullsized":"2023-11-28T11:22:25.734681+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:48:58.095068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.734503+0000","last_change":"2023-11-28T11:22:24.758001+0000","last_active":"2023-11-28T11:22:25.734503+0000","last_peered":"2023-11-28T11:22:25.734503+0000","last_clean":"2023-11-28T11:22:25.734503+0000","last_became_active":"2023-11-28T11:22:24.756789+0000","last_became_peered":"2023-11-28T11:22:24.756789+0000","last_unstale":"2023-11-28T11:22:25.734503+0000","last_undegraded":"2023-11-28T11:22:25.734503+0000","last_fullsized":"2023-11-28T11:22:25.734503+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:27:42.158553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.108954+0000","last_change":"2023-11-28T11:22:24.757403+0000","last_active":"2023-11-28T11:22:26.108954+0000","last_peered":"2023-11-28T11:22:26.108954+0000","last_clean":"2023-11-28T11:22:26.108954+0000","last_became_active":"2023-11-28T11:22:24.757156+0000","last_became_peered":"2023-11-28T11:22:24.757156+0000","last_unstale":"2023-11-28T11:22:26.108954+0000","last_undegraded":"2023-11-28T11:22:26.108954+0000","last_fullsized":"2023-11-28T11:22:26.108954+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:31:16.730971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.105729+0000","last_change":"2023-11-28T11:22:24.760399+0000","last_active":"2023-11-28T11:22:26.105729+0000","last_peered":"2023-11-28T11:22:26.105729+0000","last_clean":"2023-11-28T11:22:26.105729+0000","last_became_active":"2023-11-28T11:22:24.760144+0000","last_became_peered":"2023-11-28T11:22:24.760144+0000","last_unstale":"2023-11-28T11:22:26.105729+0000","last_undegraded":"2023-11-28T11:22:26.105729+0000","last_fullsized":"2023-11-28T11:22:26.105729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:31:16.730971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.106179+0000","last_change":"2023-11-28T11:22:24.752618+0000","last_active":"2023-11-28T11:22:26.106179+0000","last_peered":"2023-11-28T11:22:26.106179+0000","last_clean":"2023-11-28T11:22:26.106179+0000","last_became_active":"2023-11-28T11:22:24.749820+0000","last_became_peered":"2023-11-28T11:22:24.749820+0000","last_unstale":"2023-11-28T11:22:26.106179+0000","last_undegraded":"2023-11-28T11:22:26.106179+0000","last_fullsized":"2023-11-28T11:22:26.106179+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:04:03.997360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.919019+0000","last_change":"2023-11-28T11:22:24.749121+0000","last_active":"2023-11-28T11:22:25.919019+0000","last_peered":"2023-11-28T11:22:25.919019+0000","last_clean":"2023-11-28T11:22:25.919019+0000","last_became_active":"2023-11-28T11:22:24.748888+0000","last_became_peered":"2023-11-28T11:22:24.748888+0000","last_unstale":"2023-11-28T11:22:25.919019+0000","last_undegraded":"2023-11-28T11:22:25.919019+0000","last_fullsized":"2023-11-28T11:22:25.919019+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:01:46.793779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.108756+0000","last_change":"2023-11-28T11:22:24.751973+0000","last_active":"2023-11-28T11:22:26.108756+0000","last_peered":"2023-11-28T11:22:26.108756+0000","last_clean":"2023-11-28T11:22:26.108756+0000","last_became_active":"2023-11-28T11:22:24.751205+0000","last_became_peered":"2023-11-28T11:22:24.751205+0000","last_unstale":"2023-11-28T11:22:26.108756+0000","last_undegraded":"2023-11-28T11:22:26.108756+0000","last_fullsized":"2023-11-28T11:22:26.108756+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:39:01.023278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.108769+0000","last_change":"2023-11-28T11:22:24.751505+0000","last_active":"2023-11-28T11:22:26.108769+0000","last_peered":"2023-11-28T11:22:26.108769+0000","last_clean":"2023-11-28T11:22:26.108769+0000","last_became_active":"2023-11-28T11:22:24.751187+0000","last_became_peered":"2023-11-28T11:22:24.751187+0000","last_unstale":"2023-11-28T11:22:26.108769+0000","last_undegraded":"2023-11-28T11:22:26.108769+0000","last_fullsized":"2023-11-28T11:22:26.108769+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:50:08.305771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.109336+0000","last_change":"2023-11-28T11:22:24.752027+0000","last_active":"2023-11-28T11:22:26.109336+0000","last_peered":"2023-11-28T11:22:26.109336+0000","last_clean":"2023-11-28T11:22:26.109336+0000","last_became_active":"2023-11-28T11:22:24.751727+0000","last_became_peered":"2023-11-28T11:22:24.751727+0000","last_unstale":"2023-11-28T11:22:26.109336+0000","last_undegraded":"2023-11-28T11:22:26.109336+0000","last_fullsized":"2023-11-28T11:22:26.109336+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:38:23.565208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.105589+0000","last_change":"2023-11-28T11:22:24.750048+0000","last_active":"2023-11-28T11:22:26.105589+0000","last_peered":"2023-11-28T11:22:26.105589+0000","last_clean":"2023-11-28T11:22:26.105589+0000","last_became_active":"2023-11-28T11:22:24.749800+0000","last_became_peered":"2023-11-28T11:22:24.749800+0000","last_unstale":"2023-11-28T11:22:26.105589+0000","last_undegraded":"2023-11-28T11:22:26.105589+0000","last_fullsized":"2023-11-28T11:22:26.105589+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:38:23.565208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.734590+0000","last_change":"2023-11-28T11:22:24.746691+0000","last_active":"2023-11-28T11:22:25.734590+0000","last_peered":"2023-11-28T11:22:25.734590+0000","last_clean":"2023-11-28T11:22:25.734590+0000","last_became_active":"2023-11-28T11:22:24.746414+0000","last_became_peered":"2023-11-28T11:22:24.746414+0000","last_unstale":"2023-11-28T11:22:25.734590+0000","last_undegraded":"2023-11-28T11:22:25.734590+0000","last_fullsized":"2023-11-28T11:22:25.734590+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:23:14.629301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.735126+0000","last_change":"2023-11-28T11:22:24.747468+0000","last_active":"2023-11-28T11:22:25.735126+0000","last_peered":"2023-11-28T11:22:25.735126+0000","last_clean":"2023-11-28T11:22:25.735126+0000","last_became_active":"2023-11-28T11:22:24.747206+0000","last_became_peered":"2023-11-28T11:22:24.747206+0000","last_unstale":"2023-11-28T11:22:25.735126+0000","last_undegraded":"2023-11-28T11:22:25.735126+0000","last_fullsized":"2023-11-28T11:22:25.735126+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:08:22.302877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.109034+0000","last_change":"2023-11-28T11:22:24.750628+0000","last_active":"2023-11-28T11:22:26.109034+0000","last_peered":"2023-11-28T11:22:26.109034+0000","last_clean":"2023-11-28T11:22:26.109034+0000","last_became_active":"2023-11-28T11:22:24.750343+0000","last_became_peered":"2023-11-28T11:22:24.750343+0000","last_unstale":"2023-11-28T11:22:26.109034+0000","last_undegraded":"2023-11-28T11:22:26.109034+0000","last_fullsized":"2023-11-28T11:22:26.109034+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:52:15.751540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.734736+0000","last_change":"2023-11-28T11:22:24.743191+0000","last_active":"2023-11-28T11:22:25.734736+0000","last_peered":"2023-11-28T11:22:25.734736+0000","last_clean":"2023-11-28T11:22:25.734736+0000","last_became_active":"2023-11-28T11:22:24.742926+0000","last_became_peered":"2023-11-28T11:22:24.742926+0000","last_unstale":"2023-11-28T11:22:25.734736+0000","last_undegraded":"2023-11-28T11:22:25.734736+0000","last_fullsized":"2023-11-28T11:22:25.734736+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:18:49.637085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.105773+0000","last_change":"2023-11-28T11:22:24.748844+0000","last_active":"2023-11-28T11:22:26.105773+0000","last_peered":"2023-11-28T11:22:26.105773+0000","last_clean":"2023-11-28T11:22:26.105773+0000","last_became_active":"2023-11-28T11:22:24.748563+0000","last_became_peered":"2023-11-28T11:22:24.748563+0000","last_unstale":"2023-11-28T11:22:26.105773+0000","last_undegraded":"2023-11-28T11:22:26.105773+0000","last_fullsized":"2023-11-28T11:22:26.105773+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:49:07.842588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.105513+0000","last_change":"2023-11-28T11:22:24.750190+0000","last_active":"2023-11-28T11:22:26.105513+0000","last_peered":"2023-11-28T11:22:26.105513+0000","last_clean":"2023-11-28T11:22:26.105513+0000","last_became_active":"2023-11-28T11:22:24.747715+0000","last_became_peered":"2023-11-28T11:22:24.747715+0000","last_unstale":"2023-11-28T11:22:26.105513+0000","last_undegraded":"2023-11-28T11:22:26.105513+0000","last_fullsized":"2023-11-28T11:22:26.105513+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:33:57.540987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.732631+0000","last_change":"2023-11-28T11:22:24.750462+0000","last_active":"2023-11-28T11:22:25.732631+0000","last_peered":"2023-11-28T11:22:25.732631+0000","last_clean":"2023-11-28T11:22:25.732631+0000","last_became_active":"2023-11-28T11:22:24.750208+0000","last_became_peered":"2023-11-28T11:22:24.750208+0000","last_unstale":"2023-11-28T11:22:25.732631+0000","last_undegraded":"2023-11-28T11:22:25.732631+0000","last_fullsized":"2023-11-28T11:22:25.732631+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:53:29.715029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"41'4","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.106467+0000","last_change":"2023-11-28T11:22:24.243596+0000","last_active":"2023-11-28T11:22:26.106467+0000","last_peered":"2023-11-28T11:22:26.106467+0000","last_clean":"2023-11-28T11:22:26.106467+0000","last_became_active":"2023-11-28T11:22:24.243317+0000","last_became_peered":"2023-11-28T11:22:24.243317+0000","last_unstale":"2023-11-28T11:22:26.106467+0000","last_undegraded":"2023-11-28T11:22:26.106467+0000","last_fullsized":"2023-11-28T11:22:26.106467+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:39:45.321174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.109240+0000","last_change":"2023-11-28T11:22:24.749451+0000","last_active":"2023-11-28T11:22:26.109240+0000","last_peered":"2023-11-28T11:22:26.109240+0000","last_clean":"2023-11-28T11:22:26.109240+0000","last_became_active":"2023-11-28T11:22:24.748994+0000","last_became_peered":"2023-11-28T11:22:24.748994+0000","last_unstale":"2023-11-28T11:22:26.109240+0000","last_undegraded":"2023-11-28T11:22:26.109240+0000","last_fullsized":"2023-11-28T11:22:26.109240+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:11:41.372618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"41'1","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.734790+0000","last_change":"2023-11-28T11:22:24.238405+0000","last_active":"2023-11-28T11:22:25.734790+0000","last_peered":"2023-11-28T11:22:25.734790+0000","last_clean":"2023-11-28T11:22:25.734790+0000","last_became_active":"2023-11-28T11:22:24.237714+0000","last_became_peered":"2023-11-28T11:22:24.237714+0000","last_unstale":"2023-11-28T11:22:25.734790+0000","last_undegraded":"2023-11-28T11:22:25.734790+0000","last_fullsized":"2023-11-28T11:22:25.734790+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:39:54.488254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.735262+0000","last_change":"2023-11-28T11:22:24.749958+0000","last_active":"2023-11-28T11:22:25.735262+0000","last_peered":"2023-11-28T11:22:25.735262+0000","last_clean":"2023-11-28T11:22:25.735262+0000","last_became_active":"2023-11-28T11:22:24.749659+0000","last_became_peered":"2023-11-28T11:22:24.749659+0000","last_unstale":"2023-11-28T11:22:25.735262+0000","last_undegraded":"2023-11-28T11:22:25.735262+0000","last_fullsized":"2023-11-28T11:22:25.735262+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:54:40.929864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"41'2","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.732662+0000","last_change":"2023-11-28T11:22:24.246752+0000","last_active":"2023-11-28T11:22:25.732662+0000","last_peered":"2023-11-28T11:22:25.732662+0000","last_clean":"2023-11-28T11:22:25.732662+0000","last_became_active":"2023-11-28T11:22:24.243536+0000","last_became_peered":"2023-11-28T11:22:24.243536+0000","last_unstale":"2023-11-28T11:22:25.732662+0000","last_undegraded":"2023-11-28T11:22:25.732662+0000","last_fullsized":"2023-11-28T11:22:25.732662+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:33:56.534204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.732537+0000","last_change":"2023-11-28T11:22:24.744468+0000","last_active":"2023-11-28T11:22:25.732537+0000","last_peered":"2023-11-28T11:22:25.732537+0000","last_clean":"2023-11-28T11:22:25.732537+0000","last_became_active":"2023-11-28T11:22:24.744142+0000","last_became_peered":"2023-11-28T11:22:24.744142+0000","last_unstale":"2023-11-28T11:22:25.732537+0000","last_undegraded":"2023-11-28T11:22:25.732537+0000","last_fullsized":"2023-11-28T11:22:25.732537+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:58.550893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"41'2","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.106704+0000","last_change":"2023-11-28T11:22:24.239579+0000","last_active":"2023-11-28T11:22:26.106704+0000","last_peered":"2023-11-28T11:22:26.106704+0000","last_clean":"2023-11-28T11:22:26.106704+0000","last_became_active":"2023-11-28T11:22:24.239276+0000","last_became_peered":"2023-11-28T11:22:24.239276+0000","last_unstale":"2023-11-28T11:22:26.106704+0000","last_undegraded":"2023-11-28T11:22:26.106704+0000","last_fullsized":"2023-11-28T11:22:26.106704+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:50:30.348523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.734289+0000","last_change":"2023-11-28T11:22:24.754252+0000","last_active":"2023-11-28T11:22:25.734289+0000","last_peered":"2023-11-28T11:22:25.734289+0000","last_clean":"2023-11-28T11:22:25.734289+0000","last_became_active":"2023-11-28T11:22:24.753350+0000","last_became_peered":"2023-11-28T11:22:24.753350+0000","last_unstale":"2023-11-28T11:22:25.734289+0000","last_undegraded":"2023-11-28T11:22:25.734289+0000","last_fullsized":"2023-11-28T11:22:25.734289+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:02:19.560108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"41'1","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.106163+0000","last_change":"2023-11-28T11:22:24.229901+0000","last_active":"2023-11-28T11:22:26.106163+0000","last_peered":"2023-11-28T11:22:26.106163+0000","last_clean":"2023-11-28T11:22:26.106163+0000","last_became_active":"2023-11-28T11:22:24.229638+0000","last_became_peered":"2023-11-28T11:22:24.229638+0000","last_unstale":"2023-11-28T11:22:26.106163+0000","last_undegraded":"2023-11-28T11:22:26.106163+0000","last_fullsized":"2023-11-28T11:22:26.106163+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:23:13.622518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.918857+0000","last_change":"2023-11-28T11:22:24.751654+0000","last_active":"2023-11-28T11:22:25.918857+0000","last_peered":"2023-11-28T11:22:25.918857+0000","last_clean":"2023-11-28T11:22:25.918857+0000","last_became_active":"2023-11-28T11:22:24.751370+0000","last_became_peered":"2023-11-28T11:22:24.751370+0000","last_unstale":"2023-11-28T11:22:25.918857+0000","last_undegraded":"2023-11-28T11:22:25.918857+0000","last_fullsized":"2023-11-28T11:22:25.918857+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:00:53.225852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"41'1","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.735147+0000","last_change":"2023-11-28T11:22:24.233155+0000","last_active":"2023-11-28T11:22:25.735147+0000","last_peered":"2023-11-28T11:22:25.735147+0000","last_clean":"2023-11-28T11:22:25.735147+0000","last_became_active":"2023-11-28T11:22:24.232854+0000","last_became_peered":"2023-11-28T11:22:24.232854+0000","last_unstale":"2023-11-28T11:22:25.735147+0000","last_undegraded":"2023-11-28T11:22:25.735147+0000","last_fullsized":"2023-11-28T11:22:25.735147+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:05:48.569713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.918720+0000","last_change":"2023-11-28T11:22:24.742651+0000","last_active":"2023-11-28T11:22:25.918720+0000","last_peered":"2023-11-28T11:22:25.918720+0000","last_clean":"2023-11-28T11:22:25.918720+0000","last_became_active":"2023-11-28T11:22:24.741692+0000","last_became_peered":"2023-11-28T11:22:24.741692+0000","last_unstale":"2023-11-28T11:22:25.918720+0000","last_undegraded":"2023-11-28T11:22:25.918720+0000","last_fullsized":"2023-11-28T11:22:25.918720+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:23:14.629301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.734589+0000","last_change":"2023-11-28T11:22:24.234395+0000","last_active":"2023-11-28T11:22:25.734589+0000","last_peered":"2023-11-28T11:22:25.734589+0000","last_clean":"2023-11-28T11:22:25.734589+0000","last_became_active":"2023-11-28T11:22:24.234135+0000","last_became_peered":"2023-11-28T11:22:24.234135+0000","last_unstale":"2023-11-28T11:22:25.734589+0000","last_undegraded":"2023-11-28T11:22:25.734589+0000","last_fullsized":"2023-11-28T11:22:25.734589+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:53:27.359189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.108922+0000","last_change":"2023-11-28T11:22:24.753515+0000","last_active":"2023-11-28T11:22:26.108922+0000","last_peered":"2023-11-28T11:22:26.108922+0000","last_clean":"2023-11-28T11:22:26.108922+0000","last_became_active":"2023-11-28T11:22:24.753227+0000","last_became_peered":"2023-11-28T11:22:24.753227+0000","last_unstale":"2023-11-28T11:22:26.108922+0000","last_undegraded":"2023-11-28T11:22:26.108922+0000","last_fullsized":"2023-11-28T11:22:26.108922+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:44:45.612931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.918731+0000","last_change":"2023-11-28T11:22:24.242991+0000","last_active":"2023-11-28T11:22:25.918731+0000","last_peered":"2023-11-28T11:22:25.918731+0000","last_clean":"2023-11-28T11:22:25.918731+0000","last_became_active":"2023-11-28T11:22:24.242524+0000","last_became_peered":"2023-11-28T11:22:24.242524+0000","last_unstale":"2023-11-28T11:22:25.918731+0000","last_undegraded":"2023-11-28T11:22:25.918731+0000","last_fullsized":"2023-11-28T11:22:25.918731+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:31:15.724188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.734738+0000","last_change":"2023-11-28T11:22:24.757212+0000","last_active":"2023-11-28T11:22:25.734738+0000","last_peered":"2023-11-28T11:22:25.734738+0000","last_clean":"2023-11-28T11:22:25.734738+0000","last_became_active":"2023-11-28T11:22:24.756934+0000","last_became_peered":"2023-11-28T11:22:24.756934+0000","last_unstale":"2023-11-28T11:22:25.734738+0000","last_undegraded":"2023-11-28T11:22:25.734738+0000","last_fullsized":"2023-11-28T11:22:25.734738+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:29:53.939582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.734827+0000","last_change":"2023-11-28T11:22:24.234787+0000","last_active":"2023-11-28T11:22:25.734827+0000","last_peered":"2023-11-28T11:22:25.734827+0000","last_clean":"2023-11-28T11:22:25.734827+0000","last_became_active":"2023-11-28T11:22:24.233501+0000","last_became_peered":"2023-11-28T11:22:24.233501+0000","last_unstale":"2023-11-28T11:22:25.734827+0000","last_undegraded":"2023-11-28T11:22:25.734827+0000","last_fullsized":"2023-11-28T11:22:25.734827+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:25:56.617859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.106938+0000","last_change":"2023-11-28T11:22:24.750717+0000","last_active":"2023-11-28T11:22:26.106938+0000","last_peered":"2023-11-28T11:22:26.106938+0000","last_clean":"2023-11-28T11:22:26.106938+0000","last_became_active":"2023-11-28T11:22:24.747789+0000","last_became_peered":"2023-11-28T11:22:24.747789+0000","last_unstale":"2023-11-28T11:22:26.106938+0000","last_undegraded":"2023-11-28T11:22:26.106938+0000","last_fullsized":"2023-11-28T11:22:26.106938+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:05:49.576496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.919233+0000","last_change":"2023-11-28T11:22:24.244527+0000","last_active":"2023-11-28T11:22:25.919233+0000","last_peered":"2023-11-28T11:22:25.919233+0000","last_clean":"2023-11-28T11:22:25.919233+0000","last_became_active":"2023-11-28T11:22:24.244239+0000","last_became_peered":"2023-11-28T11:22:24.244239+0000","last_unstale":"2023-11-28T11:22:25.919233+0000","last_undegraded":"2023-11-28T11:22:25.919233+0000","last_fullsized":"2023-11-28T11:22:25.919233+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:50:30.348523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.106322+0000","last_change":"2023-11-28T11:22:24.745415+0000","last_active":"2023-11-28T11:22:26.106322+0000","last_peered":"2023-11-28T11:22:26.106322+0000","last_clean":"2023-11-28T11:22:26.106322+0000","last_became_active":"2023-11-28T11:22:24.745170+0000","last_became_peered":"2023-11-28T11:22:24.745170+0000","last_unstale":"2023-11-28T11:22:26.106322+0000","last_undegraded":"2023-11-28T11:22:26.106322+0000","last_fullsized":"2023-11-28T11:22:26.106322+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:29:58.643132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"41'3","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.106887+0000","last_change":"2023-11-28T11:22:24.239784+0000","last_active":"2023-11-28T11:22:26.106887+0000","last_peered":"2023-11-28T11:22:26.106887+0000","last_clean":"2023-11-28T11:22:26.106887+0000","last_became_active":"2023-11-28T11:22:24.239484+0000","last_became_peered":"2023-11-28T11:22:24.239484+0000","last_unstale":"2023-11-28T11:22:26.106887+0000","last_undegraded":"2023-11-28T11:22:26.106887+0000","last_fullsized":"2023-11-28T11:22:26.106887+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:33:56.534204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.106758+0000","last_change":"2023-11-28T11:22:24.750309+0000","last_active":"2023-11-28T11:22:26.106758+0000","last_peered":"2023-11-28T11:22:26.106758+0000","last_clean":"2023-11-28T11:22:26.106758+0000","last_became_active":"2023-11-28T11:22:24.749432+0000","last_became_peered":"2023-11-28T11:22:24.749432+0000","last_unstale":"2023-11-28T11:22:26.106758+0000","last_undegraded":"2023-11-28T11:22:26.106758+0000","last_fullsized":"2023-11-28T11:22:26.106758+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:48:07.849967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"41'6","reported_seq":17,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.105831+0000","last_change":"2023-11-28T11:22:24.245339+0000","last_active":"2023-11-28T11:22:26.105831+0000","last_peered":"2023-11-28T11:22:26.105831+0000","last_clean":"2023-11-28T11:22:26.105831+0000","last_became_active":"2023-11-28T11:22:24.245096+0000","last_became_peered":"2023-11-28T11:22:24.245096+0000","last_unstale":"2023-11-28T11:22:26.105831+0000","last_undegraded":"2023-11-28T11:22:26.105831+0000","last_fullsized":"2023-11-28T11:22:26.105831+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:58:33.293304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.108826+0000","last_change":"2023-11-28T11:22:24.758212+0000","last_active":"2023-11-28T11:22:26.108826+0000","last_peered":"2023-11-28T11:22:26.108826+0000","last_clean":"2023-11-28T11:22:26.108826+0000","last_became_active":"2023-11-28T11:22:24.757952+0000","last_became_peered":"2023-11-28T11:22:24.757952+0000","last_unstale":"2023-11-28T11:22:26.108826+0000","last_undegraded":"2023-11-28T11:22:26.108826+0000","last_fullsized":"2023-11-28T11:22:26.108826+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:49:54.519059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"41'2","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.106290+0000","last_change":"2023-11-28T11:22:24.245144+0000","last_active":"2023-11-28T11:22:26.106290+0000","last_peered":"2023-11-28T11:22:26.106290+0000","last_clean":"2023-11-28T11:22:26.106290+0000","last_became_active":"2023-11-28T11:22:24.244868+0000","last_became_peered":"2023-11-28T11:22:24.244868+0000","last_unstale":"2023-11-28T11:22:26.106290+0000","last_undegraded":"2023-11-28T11:22:26.106290+0000","last_fullsized":"2023-11-28T11:22:26.106290+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:08:21.296095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.919210+0000","last_change":"2023-11-28T11:22:24.751160+0000","last_active":"2023-11-28T11:22:25.919210+0000","last_peered":"2023-11-28T11:22:25.919210+0000","last_clean":"2023-11-28T11:22:25.919210+0000","last_became_active":"2023-11-28T11:22:24.750895+0000","last_became_peered":"2023-11-28T11:22:24.750895+0000","last_unstale":"2023-11-28T11:22:25.919210+0000","last_undegraded":"2023-11-28T11:22:25.919210+0000","last_fullsized":"2023-11-28T11:22:25.919210+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:50:58.725549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.106543+0000","last_change":"2023-11-28T11:22:24.240189+0000","last_active":"2023-11-28T11:22:26.106543+0000","last_peered":"2023-11-28T11:22:26.106543+0000","last_clean":"2023-11-28T11:22:26.106543+0000","last_became_active":"2023-11-28T11:22:24.239869+0000","last_became_peered":"2023-11-28T11:22:24.239869+0000","last_unstale":"2023-11-28T11:22:26.106543+0000","last_undegraded":"2023-11-28T11:22:26.106543+0000","last_fullsized":"2023-11-28T11:22:26.106543+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:43:48.886755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.106063+0000","last_change":"2023-11-28T11:22:24.741396+0000","last_active":"2023-11-28T11:22:26.106063+0000","last_peered":"2023-11-28T11:22:26.106063+0000","last_clean":"2023-11-28T11:22:26.106063+0000","last_became_active":"2023-11-28T11:22:24.741095+0000","last_became_peered":"2023-11-28T11:22:24.741095+0000","last_unstale":"2023-11-28T11:22:26.106063+0000","last_undegraded":"2023-11-28T11:22:26.106063+0000","last_fullsized":"2023-11-28T11:22:26.106063+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:06:15.963909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.105953+0000","last_change":"2023-11-28T11:22:24.243978+0000","last_active":"2023-11-28T11:22:26.105953+0000","last_peered":"2023-11-28T11:22:26.105953+0000","last_clean":"2023-11-28T11:22:26.105953+0000","last_became_active":"2023-11-28T11:22:24.243620+0000","last_became_peered":"2023-11-28T11:22:24.243620+0000","last_unstale":"2023-11-28T11:22:26.105953+0000","last_undegraded":"2023-11-28T11:22:26.105953+0000","last_fullsized":"2023-11-28T11:22:26.105953+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:54:39.923082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.106517+0000","last_change":"2023-11-28T11:22:24.745735+0000","last_active":"2023-11-28T11:22:26.106517+0000","last_peered":"2023-11-28T11:22:26.106517+0000","last_clean":"2023-11-28T11:22:26.106517+0000","last_became_active":"2023-11-28T11:22:24.744262+0000","last_became_peered":"2023-11-28T11:22:24.744262+0000","last_unstale":"2023-11-28T11:22:26.106517+0000","last_undegraded":"2023-11-28T11:22:26.106517+0000","last_fullsized":"2023-11-28T11:22:26.106517+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:30:58.618362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"41'8","reported_seq":19,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.918887+0000","last_change":"2023-11-28T11:22:24.235077+0000","last_active":"2023-11-28T11:22:25.918887+0000","last_peered":"2023-11-28T11:22:25.918887+0000","last_clean":"2023-11-28T11:22:25.918887+0000","last_became_active":"2023-11-28T11:22:24.234799+0000","last_became_peered":"2023-11-28T11:22:24.234799+0000","last_unstale":"2023-11-28T11:22:25.918887+0000","last_undegraded":"2023-11-28T11:22:25.918887+0000","last_fullsized":"2023-11-28T11:22:25.918887+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:52:14.744757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.105697+0000","last_change":"2023-11-28T11:22:24.752479+0000","last_active":"2023-11-28T11:22:26.105697+0000","last_peered":"2023-11-28T11:22:26.105697+0000","last_clean":"2023-11-28T11:22:26.105697+0000","last_became_active":"2023-11-28T11:22:24.752224+0000","last_became_peered":"2023-11-28T11:22:24.752224+0000","last_unstale":"2023-11-28T11:22:26.105697+0000","last_undegraded":"2023-11-28T11:22:26.105697+0000","last_fullsized":"2023-11-28T11:22:26.105697+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:50:31.355306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"41'1","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.105840+0000","last_change":"2023-11-28T11:22:24.232009+0000","last_active":"2023-11-28T11:22:26.105840+0000","last_peered":"2023-11-28T11:22:26.105840+0000","last_clean":"2023-11-28T11:22:26.105840+0000","last_became_active":"2023-11-28T11:22:24.231755+0000","last_became_peered":"2023-11-28T11:22:24.231755+0000","last_unstale":"2023-11-28T11:22:26.105840+0000","last_undegraded":"2023-11-28T11:22:26.105840+0000","last_fullsized":"2023-11-28T11:22:26.105840+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:28:56.282597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.919227+0000","last_change":"2023-11-28T11:22:24.759542+0000","last_active":"2023-11-28T11:22:25.919227+0000","last_peered":"2023-11-28T11:22:25.919227+0000","last_clean":"2023-11-28T11:22:25.919227+0000","last_became_active":"2023-11-28T11:22:24.759092+0000","last_became_peered":"2023-11-28T11:22:24.759092+0000","last_unstale":"2023-11-28T11:22:25.919227+0000","last_undegraded":"2023-11-28T11:22:25.919227+0000","last_fullsized":"2023-11-28T11:22:25.919227+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:53:29.715029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"41'2","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.734884+0000","last_change":"2023-11-28T11:22:24.233029+0000","last_active":"2023-11-28T11:22:25.734884+0000","last_peered":"2023-11-28T11:22:25.734884+0000","last_clean":"2023-11-28T11:22:25.734884+0000","last_became_active":"2023-11-28T11:22:24.232741+0000","last_became_peered":"2023-11-28T11:22:24.232741+0000","last_unstale":"2023-11-28T11:22:25.734884+0000","last_undegraded":"2023-11-28T11:22:25.734884+0000","last_fullsized":"2023-11-28T11:22:25.734884+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:52:14.086350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.109123+0000","last_change":"2023-11-28T11:22:24.758498+0000","last_active":"2023-11-28T11:22:26.109123+0000","last_peered":"2023-11-28T11:22:26.109123+0000","last_clean":"2023-11-28T11:22:26.109123+0000","last_became_active":"2023-11-28T11:22:24.758206+0000","last_became_peered":"2023-11-28T11:22:24.758206+0000","last_unstale":"2023-11-28T11:22:26.109123+0000","last_undegraded":"2023-11-28T11:22:26.109123+0000","last_fullsized":"2023-11-28T11:22:26.109123+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:43:49.893538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"41'1","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.735350+0000","last_change":"2023-11-28T11:22:24.235752+0000","last_active":"2023-11-28T11:22:25.735350+0000","last_peered":"2023-11-28T11:22:25.735350+0000","last_clean":"2023-11-28T11:22:25.735350+0000","last_became_active":"2023-11-28T11:22:24.235409+0000","last_became_peered":"2023-11-28T11:22:24.235409+0000","last_unstale":"2023-11-28T11:22:25.735350+0000","last_undegraded":"2023-11-28T11:22:25.735350+0000","last_fullsized":"2023-11-28T11:22:25.735350+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:53:28.708246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.109188+0000","last_change":"2023-11-28T11:22:24.746370+0000","last_active":"2023-11-28T11:22:26.109188+0000","last_peered":"2023-11-28T11:22:26.109188+0000","last_clean":"2023-11-28T11:22:26.109188+0000","last_became_active":"2023-11-28T11:22:24.746021+0000","last_became_peered":"2023-11-28T11:22:24.746021+0000","last_unstale":"2023-11-28T11:22:26.109188+0000","last_undegraded":"2023-11-28T11:22:26.109188+0000","last_fullsized":"2023-11-28T11:22:26.109188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:33:45.044082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.106325+0000","last_change":"2023-11-28T11:22:24.241745+0000","last_active":"2023-11-28T11:22:26.106325+0000","last_peered":"2023-11-28T11:22:26.106325+0000","last_clean":"2023-11-28T11:22:26.106325+0000","last_became_active":"2023-11-28T11:22:24.240332+0000","last_became_peered":"2023-11-28T11:22:24.240332+0000","last_unstale":"2023-11-28T11:22:26.106325+0000","last_undegraded":"2023-11-28T11:22:26.106325+0000","last_fullsized":"2023-11-28T11:22:26.106325+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:07:07.702163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.106013+0000","last_change":"2023-11-28T11:22:24.742679+0000","last_active":"2023-11-28T11:22:26.106013+0000","last_peered":"2023-11-28T11:22:26.106013+0000","last_clean":"2023-11-28T11:22:26.106013+0000","last_became_active":"2023-11-28T11:22:24.742154+0000","last_became_peered":"2023-11-28T11:22:24.742154+0000","last_unstale":"2023-11-28T11:22:26.106013+0000","last_undegraded":"2023-11-28T11:22:26.106013+0000","last_fullsized":"2023-11-28T11:22:26.106013+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:24:32.831418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"41'2","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.106350+0000","last_change":"2023-11-28T11:22:24.230819+0000","last_active":"2023-11-28T11:22:26.106350+0000","last_peered":"2023-11-28T11:22:26.106350+0000","last_clean":"2023-11-28T11:22:26.106350+0000","last_became_active":"2023-11-28T11:22:24.230567+0000","last_became_peered":"2023-11-28T11:22:24.230567+0000","last_unstale":"2023-11-28T11:22:26.106350+0000","last_undegraded":"2023-11-28T11:22:26.106350+0000","last_fullsized":"2023-11-28T11:22:26.106350+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:47:54.789341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.731985+0000","last_change":"2023-11-28T11:22:24.745477+0000","last_active":"2023-11-28T11:22:25.731985+0000","last_peered":"2023-11-28T11:22:25.731985+0000","last_clean":"2023-11-28T11:22:25.731985+0000","last_became_active":"2023-11-28T11:22:24.745256+0000","last_became_peered":"2023-11-28T11:22:24.745256+0000","last_unstale":"2023-11-28T11:22:25.731985+0000","last_undegraded":"2023-11-28T11:22:25.731985+0000","last_fullsized":"2023-11-28T11:22:25.731985+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:53:28.365972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.735004+0000","last_change":"2023-11-28T11:22:24.238270+0000","last_active":"2023-11-28T11:22:25.735004+0000","last_peered":"2023-11-28T11:22:25.735004+0000","last_clean":"2023-11-28T11:22:25.735004+0000","last_became_active":"2023-11-28T11:22:24.237922+0000","last_became_peered":"2023-11-28T11:22:24.237922+0000","last_unstale":"2023-11-28T11:22:25.735004+0000","last_undegraded":"2023-11-28T11:22:25.735004+0000","last_fullsized":"2023-11-28T11:22:25.735004+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:54:19.649109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.105623+0000","last_change":"2023-11-28T11:22:24.746580+0000","last_active":"2023-11-28T11:22:26.105623+0000","last_peered":"2023-11-28T11:22:26.105623+0000","last_clean":"2023-11-28T11:22:26.105623+0000","last_became_active":"2023-11-28T11:22:24.745731+0000","last_became_peered":"2023-11-28T11:22:24.745731+0000","last_unstale":"2023-11-28T11:22:26.105623+0000","last_undegraded":"2023-11-28T11:22:26.105623+0000","last_fullsized":"2023-11-28T11:22:26.105623+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:52:15.751540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.918796+0000","last_change":"2023-11-28T11:22:24.238501+0000","last_active":"2023-11-28T11:22:25.918796+0000","last_peered":"2023-11-28T11:22:25.918796+0000","last_clean":"2023-11-28T11:22:25.918796+0000","last_became_active":"2023-11-28T11:22:24.237406+0000","last_became_peered":"2023-11-28T11:22:24.237406+0000","last_unstale":"2023-11-28T11:22:25.918796+0000","last_undegraded":"2023-11-28T11:22:25.918796+0000","last_fullsized":"2023-11-28T11:22:25.918796+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:33:44.037299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.918696+0000","last_change":"2023-11-28T11:22:24.743899+0000","last_active":"2023-11-28T11:22:25.918696+0000","last_peered":"2023-11-28T11:22:25.918696+0000","last_clean":"2023-11-28T11:22:25.918696+0000","last_became_active":"2023-11-28T11:22:24.743645+0000","last_became_peered":"2023-11-28T11:22:24.743645+0000","last_unstale":"2023-11-28T11:22:25.918696+0000","last_undegraded":"2023-11-28T11:22:25.918696+0000","last_fullsized":"2023-11-28T11:22:25.918696+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:54:40.929864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.106233+0000","last_change":"2023-11-28T11:22:24.237071+0000","last_active":"2023-11-28T11:22:26.106233+0000","last_peered":"2023-11-28T11:22:26.106233+0000","last_clean":"2023-11-28T11:22:26.106233+0000","last_became_active":"2023-11-28T11:22:24.236815+0000","last_became_peered":"2023-11-28T11:22:24.236815+0000","last_unstale":"2023-11-28T11:22:26.106233+0000","last_undegraded":"2023-11-28T11:22:26.106233+0000","last_fullsized":"2023-11-28T11:22:26.106233+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:38:22.558425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.734588+0000","last_change":"2023-11-28T11:22:24.755377+0000","last_active":"2023-11-28T11:22:25.734588+0000","last_peered":"2023-11-28T11:22:25.734588+0000","last_clean":"2023-11-28T11:22:25.734588+0000","last_became_active":"2023-11-28T11:22:24.755103+0000","last_became_peered":"2023-11-28T11:22:24.755103+0000","last_unstale":"2023-11-28T11:22:25.734588+0000","last_undegraded":"2023-11-28T11:22:25.734588+0000","last_fullsized":"2023-11-28T11:22:25.734588+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:01:46.793779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.918864+0000","last_change":"2023-11-28T11:22:24.237745+0000","last_active":"2023-11-28T11:22:25.918864+0000","last_peered":"2023-11-28T11:22:25.918864+0000","last_clean":"2023-11-28T11:22:25.918864+0000","last_became_active":"2023-11-28T11:22:24.236614+0000","last_became_peered":"2023-11-28T11:22:24.236614+0000","last_unstale":"2023-11-28T11:22:25.918864+0000","last_undegraded":"2023-11-28T11:22:25.918864+0000","last_fullsized":"2023-11-28T11:22:25.918864+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:39:00.016495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-28T11:22:24.736780+0000","last_change":"2023-11-28T11:20:51.020016+0000","last_active":"2023-11-28T11:22:24.736780+0000","last_peered":"2023-11-28T11:22:24.736780+0000","last_clean":"2023-11-28T11:22:24.736780+0000","last_became_active":"2023-11-28T11:20:51.019492+0000","last_became_peered":"2023-11-28T11:20:51.019492+0000","last_unstale":"2023-11-28T11:22:24.736780+0000","last_undegraded":"2023-11-28T11:22:24.736780+0000","last_fullsized":"2023-11-28T11:22:24.736780+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:27.884538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.109269+0000","last_change":"2023-11-28T11:22:24.757513+0000","last_active":"2023-11-28T11:22:26.109269+0000","last_peered":"2023-11-28T11:22:26.109269+0000","last_clean":"2023-11-28T11:22:26.109269+0000","last_became_active":"2023-11-28T11:22:24.757188+0000","last_became_peered":"2023-11-28T11:22:24.757188+0000","last_unstale":"2023-11-28T11:22:26.109269+0000","last_undegraded":"2023-11-28T11:22:26.109269+0000","last_fullsized":"2023-11-28T11:22:26.109269+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:33:57.540987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"42'1","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.152675+0000","last_change":"2023-11-28T11:22:24.237383+0000","last_active":"2023-11-28T11:22:26.152675+0000","last_peered":"2023-11-28T11:22:26.152675+0000","last_clean":"2023-11-28T11:22:26.152675+0000","last_became_active":"2023-11-28T11:22:24.236832+0000","last_became_peered":"2023-11-28T11:22:24.236832+0000","last_unstale":"2023-11-28T11:22:26.152675+0000","last_undegraded":"2023-11-28T11:22:26.152675+0000","last_fullsized":"2023-11-28T11:22:26.152675+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:00:52.219069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":31,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-28T11:22:24.735402+0000","last_change":"2023-11-28T11:20:50.521908+0000","last_active":"2023-11-28T11:22:24.735402+0000","last_peered":"2023-11-28T11:22:24.735402+0000","last_clean":"2023-11-28T11:22:24.735402+0000","last_became_active":"2023-11-28T11:20:50.521059+0000","last_became_peered":"2023-11-28T11:20:50.521059+0000","last_unstale":"2023-11-28T11:22:24.735402+0000","last_undegraded":"2023-11-28T11:22:24.735402+0000","last_fullsized":"2023-11-28T11:22:24.735402+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:05.302498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.918998+0000","last_change":"2023-11-28T11:22:24.748517+0000","last_active":"2023-11-28T11:22:25.918998+0000","last_peered":"2023-11-28T11:22:25.918998+0000","last_clean":"2023-11-28T11:22:25.918998+0000","last_became_active":"2023-11-28T11:22:24.748232+0000","last_became_peered":"2023-11-28T11:22:24.748232+0000","last_unstale":"2023-11-28T11:22:25.918998+0000","last_undegraded":"2023-11-28T11:22:25.918998+0000","last_fullsized":"2023-11-28T11:22:25.918998+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:18:49.637085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.106892+0000","last_change":"2023-11-28T11:22:24.244306+0000","last_active":"2023-11-28T11:22:26.106892+0000","last_peered":"2023-11-28T11:22:26.106892+0000","last_clean":"2023-11-28T11:22:26.106892+0000","last_became_active":"2023-11-28T11:22:24.243991+0000","last_became_peered":"2023-11-28T11:22:24.243991+0000","last_unstale":"2023-11-28T11:22:26.106892+0000","last_undegraded":"2023-11-28T11:22:26.106892+0000","last_fullsized":"2023-11-28T11:22:26.106892+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:31:15.724188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":29,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-28T11:22:24.734488+0000","last_change":"2023-11-28T11:20:50.619004+0000","last_active":"2023-11-28T11:22:24.734488+0000","last_peered":"2023-11-28T11:22:24.734488+0000","last_clean":"2023-11-28T11:22:24.734488+0000","last_became_active":"2023-11-28T11:20:50.618315+0000","last_became_peered":"2023-11-28T11:20:50.618315+0000","last_unstale":"2023-11-28T11:22:24.734488+0000","last_undegraded":"2023-11-28T11:22:24.734488+0000","last_fullsized":"2023-11-28T11:22:24.734488+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:27.884538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.919233+0000","last_change":"2023-11-28T11:22:24.747406+0000","last_active":"2023-11-28T11:22:25.919233+0000","last_peered":"2023-11-28T11:22:25.919233+0000","last_clean":"2023-11-28T11:22:25.919233+0000","last_became_active":"2023-11-28T11:22:24.747147+0000","last_became_peered":"2023-11-28T11:22:24.747147+0000","last_unstale":"2023-11-28T11:22:25.919233+0000","last_undegraded":"2023-11-28T11:22:25.919233+0000","last_fullsized":"2023-11-28T11:22:25.919233+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:47:55.796123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"41'2","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.735436+0000","last_change":"2023-11-28T11:22:24.229941+0000","last_active":"2023-11-28T11:22:25.735436+0000","last_peered":"2023-11-28T11:22:25.735436+0000","last_clean":"2023-11-28T11:22:25.735436+0000","last_became_active":"2023-11-28T11:22:24.229498+0000","last_became_peered":"2023-11-28T11:22:24.229498+0000","last_unstale":"2023-11-28T11:22:25.735436+0000","last_undegraded":"2023-11-28T11:22:25.735436+0000","last_fullsized":"2023-11-28T11:22:25.735436+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:11:40.365836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":31,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-28T11:22:24.735393+0000","last_change":"2023-11-28T11:20:50.522582+0000","last_active":"2023-11-28T11:22:24.735393+0000","last_peered":"2023-11-28T11:22:24.735393+0000","last_clean":"2023-11-28T11:22:24.735393+0000","last_became_active":"2023-11-28T11:20:50.522157+0000","last_became_peered":"2023-11-28T11:20:50.522157+0000","last_unstale":"2023-11-28T11:22:24.735393+0000","last_undegraded":"2023-11-28T11:22:24.735393+0000","last_fullsized":"2023-11-28T11:22:24.735393+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:27.884538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":183,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-28T11:22:24.722564+0000","last_change":"2023-11-28T11:20:43.294845+0000","last_active":"2023-11-28T11:22:24.722564+0000","last_peered":"2023-11-28T11:22:24.722564+0000","last_clean":"2023-11-28T11:22:24.722564+0000","last_became_active":"2023-11-28T11:20:43.294483+0000","last_became_peered":"2023-11-28T11:20:43.294483+0000","last_unstale":"2023-11-28T11:22:24.722564+0000","last_undegraded":"2023-11-28T11:22:24.722564+0000","last_fullsized":"2023-11-28T11:22:24.722564+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:42.270512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:42.270512+0000","last_clean_scrub_stamp":"2023-11-28T11:20:42.270512+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:17:03.376680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.106779+0000","last_change":"2023-11-28T11:22:24.741930+0000","last_active":"2023-11-28T11:22:26.106779+0000","last_peered":"2023-11-28T11:22:26.106779+0000","last_clean":"2023-11-28T11:22:26.106779+0000","last_became_active":"2023-11-28T11:22:24.741663+0000","last_became_peered":"2023-11-28T11:22:24.741663+0000","last_unstale":"2023-11-28T11:22:26.106779+0000","last_undegraded":"2023-11-28T11:22:26.106779+0000","last_fullsized":"2023-11-28T11:22:26.106779+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:40:41.077032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.731802+0000","last_change":"2023-11-28T11:22:24.241309+0000","last_active":"2023-11-28T11:22:25.731802+0000","last_peered":"2023-11-28T11:22:25.731802+0000","last_clean":"2023-11-28T11:22:25.731802+0000","last_became_active":"2023-11-28T11:22:24.241016+0000","last_became_peered":"2023-11-28T11:22:24.241016+0000","last_unstale":"2023-11-28T11:22:25.731802+0000","last_undegraded":"2023-11-28T11:22:25.731802+0000","last_fullsized":"2023-11-28T11:22:25.731802+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:50:30.348523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-28T11:22:24.722717+0000","last_change":"2023-11-28T11:20:50.526865+0000","last_active":"2023-11-28T11:22:24.722717+0000","last_peered":"2023-11-28T11:22:24.722717+0000","last_clean":"2023-11-28T11:22:24.722717+0000","last_became_active":"2023-11-28T11:20:50.526576+0000","last_became_peered":"2023-11-28T11:20:50.526576+0000","last_unstale":"2023-11-28T11:22:24.722717+0000","last_undegraded":"2023-11-28T11:22:24.722717+0000","last_fullsized":"2023-11-28T11:22:24.722717+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:09:18.460112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.734797+0000","last_change":"2023-11-28T11:22:24.748680+0000","last_active":"2023-11-28T11:22:25.734797+0000","last_peered":"2023-11-28T11:22:25.734797+0000","last_clean":"2023-11-28T11:22:25.734797+0000","last_became_active":"2023-11-28T11:22:24.746760+0000","last_became_peered":"2023-11-28T11:22:24.746760+0000","last_unstale":"2023-11-28T11:22:25.734797+0000","last_undegraded":"2023-11-28T11:22:25.734797+0000","last_fullsized":"2023-11-28T11:22:25.734797+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:07:08.708946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.732022+0000","last_change":"2023-11-28T11:22:24.239331+0000","last_active":"2023-11-28T11:22:25.732022+0000","last_peered":"2023-11-28T11:22:25.732022+0000","last_clean":"2023-11-28T11:22:25.732022+0000","last_became_active":"2023-11-28T11:22:24.239084+0000","last_became_peered":"2023-11-28T11:22:24.239084+0000","last_unstale":"2023-11-28T11:22:25.732022+0000","last_undegraded":"2023-11-28T11:22:25.732022+0000","last_fullsized":"2023-11-28T11:22:25.732022+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:31:15.724188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-28T11:22:24.723578+0000","last_change":"2023-11-28T11:20:51.020043+0000","last_active":"2023-11-28T11:22:24.723578+0000","last_peered":"2023-11-28T11:22:24.723578+0000","last_clean":"2023-11-28T11:22:24.723578+0000","last_became_active":"2023-11-28T11:20:51.019702+0000","last_became_peered":"2023-11-28T11:20:51.019702+0000","last_unstale":"2023-11-28T11:22:24.723578+0000","last_undegraded":"2023-11-28T11:22:24.723578+0000","last_fullsized":"2023-11-28T11:22:24.723578+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:26:20.246873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.106258+0000","last_change":"2023-11-28T11:22:24.759524+0000","last_active":"2023-11-28T11:22:26.106258+0000","last_peered":"2023-11-28T11:22:26.106258+0000","last_clean":"2023-11-28T11:22:26.106258+0000","last_became_active":"2023-11-28T11:22:24.757893+0000","last_became_peered":"2023-11-28T11:22:24.757893+0000","last_unstale":"2023-11-28T11:22:26.106258+0000","last_undegraded":"2023-11-28T11:22:26.106258+0000","last_fullsized":"2023-11-28T11:22:26.106258+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:12:27.357618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.734854+0000","last_change":"2023-11-28T11:22:24.237362+0000","last_active":"2023-11-28T11:22:25.734854+0000","last_peered":"2023-11-28T11:22:25.734854+0000","last_clean":"2023-11-28T11:22:25.734854+0000","last_became_active":"2023-11-28T11:22:24.236840+0000","last_became_peered":"2023-11-28T11:22:24.236840+0000","last_unstale":"2023-11-28T11:22:25.734854+0000","last_undegraded":"2023-11-28T11:22:25.734854+0000","last_fullsized":"2023-11-28T11:22:25.734854+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:40:40.070249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-28T11:22:24.736887+0000","last_change":"2023-11-28T11:20:50.524251+0000","last_active":"2023-11-28T11:22:24.736887+0000","last_peered":"2023-11-28T11:22:24.736887+0000","last_clean":"2023-11-28T11:22:24.736887+0000","last_became_active":"2023-11-28T11:20:50.524012+0000","last_became_peered":"2023-11-28T11:20:50.524012+0000","last_unstale":"2023-11-28T11:22:24.736887+0000","last_undegraded":"2023-11-28T11:22:24.736887+0000","last_fullsized":"2023-11-28T11:22:24.736887+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:05.302498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.106071+0000","last_change":"2023-11-28T11:22:24.750518+0000","last_active":"2023-11-28T11:22:26.106071+0000","last_peered":"2023-11-28T11:22:26.106071+0000","last_clean":"2023-11-28T11:22:26.106071+0000","last_became_active":"2023-11-28T11:22:24.750252+0000","last_became_peered":"2023-11-28T11:22:24.750252+0000","last_unstale":"2023-11-28T11:22:26.106071+0000","last_undegraded":"2023-11-28T11:22:26.106071+0000","last_fullsized":"2023-11-28T11:22:26.106071+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:35:10.458383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.106738+0000","last_change":"2023-11-28T11:22:24.238373+0000","last_active":"2023-11-28T11:22:26.106738+0000","last_peered":"2023-11-28T11:22:26.106738+0000","last_clean":"2023-11-28T11:22:26.106738+0000","last_became_active":"2023-11-28T11:22:24.236466+0000","last_became_peered":"2023-11-28T11:22:24.236466+0000","last_unstale":"2023-11-28T11:22:26.106738+0000","last_undegraded":"2023-11-28T11:22:26.106738+0000","last_fullsized":"2023-11-28T11:22:26.106738+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:52:14.744757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-28T11:22:24.742195+0000","last_change":"2023-11-28T11:20:50.524838+0000","last_active":"2023-11-28T11:22:24.742195+0000","last_peered":"2023-11-28T11:22:24.742195+0000","last_clean":"2023-11-28T11:22:24.742195+0000","last_became_active":"2023-11-28T11:20:50.524402+0000","last_became_peered":"2023-11-28T11:20:50.524402+0000","last_unstale":"2023-11-28T11:22:24.742195+0000","last_undegraded":"2023-11-28T11:22:24.742195+0000","last_fullsized":"2023-11-28T11:22:24.742195+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:22.699447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9832,"kb_used_data":1056,"kb_used_omap":0,"kb_used_meta":8768,"kb_avail":94362008,"statfs":{"total":96636764160,"available":96626696192,"internally_reserved":0,"allocated":1081344,"data_stored":808612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8978432},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.66500000000000004}]},{"osd":1,"last update":"Tue Nov 28 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.61599999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.68100000000000005}]},{"osd":3,"last update":"Tue Nov 28 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.71799999999999997}]},{"osd":4,"last update":"Tue Nov 28 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.69799999999999995}]},{"osd":5,"last update":"Tue Nov 28 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.56599999999999995}]},{"osd":6,"last update":"Tue Nov 28 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.59899999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8620,"kb_used_data":484,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363220,"statfs":{"total":96636764160,"available":96627937280,"internally_reserved":0,"allocated":495616,"data_stored":219432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.49299999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.434}]},{"osd":2,"last update":"Tue Nov 28 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.996,"5min":1.996,"15min":1.996},"last":0.68100000000000005}]},{"osd":3,"last update":"Tue Nov 28 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.622},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.45400000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":2.048,"5min":2.048,"15min":2.048},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.57399999999999995}]},{"osd":5,"last update":"Tue Nov 28 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":2.1000000000000001,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":0.70799999999999996}]},{"osd":7,"last update":"Tue Nov 28 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.56799999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8744,"kb_used_data":480,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363096,"statfs":{"total":96636764160,"available":96627810304,"internally_reserved":0,"allocated":491520,"data_stored":218846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.57899999999999996}]},{"osd":2,"last update":"Tue Nov 28 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.378},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.33700000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.497}]},{"osd":4,"last update":"Tue Nov 28 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.81599999999999995}]},{"osd":5,"last update":"Tue Nov 28 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.82799999999999996}]},{"osd":6,"last update":"Tue Nov 28 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.70399999999999996}]},{"osd":7,"last update":"Tue Nov 28 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.754}]}]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9836,"kb_used_data":1060,"kb_used_omap":0,"kb_used_meta":8768,"kb_avail":94362004,"statfs":{"total":96636764160,"available":96626692096,"internally_reserved":0,"allocated":1085440,"data_stored":809178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8978432},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:21:41 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":3.8420000000000001,"5min":3.8420000000000001,"15min":3.8420000000000001},"last":0.33600000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:21:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":3.8940000000000001,"5min":3.8940000000000001,"15min":3.8940000000000001},"last":0.38100000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:21:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":3.681,"5min":3.681,"15min":3.681},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.434}]},{"osd":4,"last update":"Tue Nov 28 11:21:41 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":3.9790000000000001,"5min":3.9790000000000001,"15min":3.9790000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":3.5779999999999998,"5min":3.5779999999999998,"15min":3.5779999999999998},"last":0.75700000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:21:41 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.998,"5min":0.998,"15min":0.998},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":3.9409999999999998,"5min":3.9409999999999998,"15min":3.9409999999999998},"last":0.74399999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:21:41 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":3.6160000000000001,"5min":3.6160000000000001,"15min":3.6160000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.79100000000000004}]},{"osd":7,"last update":"Tue Nov 28 11:21:41 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.077,"15min":1.077},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":4.0099999999999998,"5min":4.0099999999999998,"15min":4.0099999999999998},"last":0.80600000000000005},{"interface":"front","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":3.653,"5min":3.653,"15min":3.653},"last":0.73499999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8564,"kb_used_data":492,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363276,"statfs":{"total":96636764160,"available":96627994624,"internally_reserved":0,"allocated":503808,"data_stored":220346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":4.1109999999999998,"5min":4.1109999999999998,"15min":4.1109999999999998},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":3.6230000000000002,"5min":3.6230000000000002,"15min":3.6230000000000002},"last":0.315}]},{"osd":1,"last update":"Tue Nov 28 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":3.6640000000000001,"5min":3.6640000000000001,"15min":3.6640000000000001},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":4.0830000000000002,"5min":4.0830000000000002,"15min":4.0830000000000002},"last":0.40899999999999997}]},{"osd":3,"last update":"Tue Nov 28 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":4.1529999999999996,"5min":4.1529999999999996,"15min":4.1529999999999996},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.42299999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.649,"5min":1.649,"15min":1.649},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":4.0060000000000002,"5min":4.0060000000000002,"15min":4.0060000000000002},"last":0.74199999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":4.0529999999999999,"5min":4.0529999999999999,"15min":4.0529999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.71699999999999997}]},{"osd":6,"last update":"Tue Nov 28 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.75700000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.70699999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8740,"kb_used_data":476,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363100,"statfs":{"total":96636764160,"available":96627814400,"internally_reserved":0,"allocated":487424,"data_stored":212425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.57999999999999996}]},{"osd":1,"last update":"Tue Nov 28 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":4.8019999999999996,"5min":4.8019999999999996,"15min":4.8019999999999996},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":4.7539999999999996,"5min":4.7539999999999996,"15min":4.7539999999999996},"last":0.63800000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":4.8419999999999996,"5min":4.8419999999999996,"15min":4.8419999999999996},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":4.7919999999999998,"5min":4.7919999999999998,"15min":4.7919999999999998},"last":0.56200000000000006}]},{"osd":4,"last update":"Tue Nov 28 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":4.8869999999999996,"5min":4.8869999999999996,"15min":4.8869999999999996},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":5.2460000000000004,"5min":5.2460000000000004,"15min":5.2460000000000004},"last":0.752}]},{"osd":5,"last update":"Tue Nov 28 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":5.0759999999999996,"5min":5.0759999999999996,"15min":5.0759999999999996},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":5.3010000000000002,"5min":5.3010000000000002,"15min":5.3010000000000002},"last":0.68000000000000005}]},{"osd":6,"last update":"Tue Nov 28 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":5.3399999999999999,"5min":5.3399999999999999,"15min":5.3399999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":4.8330000000000002,"5min":4.8330000000000002,"15min":4.8330000000000002},"last":0.622}]},{"osd":7,"last update":"Tue Nov 28 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":4.8600000000000003,"5min":4.8600000000000003,"15min":4.8600000000000003},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":5.1909999999999998,"5min":5.1909999999999998,"15min":5.1909999999999998},"last":0.66700000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8816,"kb_used_data":488,"kb_used_omap":0,"kb_used_meta":8320,"kb_avail":94363024,"statfs":{"total":96636764160,"available":96627736576,"internally_reserved":0,"allocated":499712,"data_stored":220324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8519680},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.66900000000000004}]},{"osd":1,"last update":"Tue Nov 28 11:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.78900000000000003}]},{"osd":2,"last update":"Tue Nov 28 11:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.84199999999999997}]},{"osd":3,"last update":"Tue Nov 28 11:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.97399999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.81499999999999995}]},{"osd":6,"last update":"Tue Nov 28 11:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.95899999999999996}]},{"osd":7,"last update":"Tue Nov 28 11:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":2.1960000000000002,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":1.0149999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8680,"kb_used_data":480,"kb_used_omap":0,"kb_used_meta":8192,"kb_avail":94363160,"statfs":{"total":96636764160,"available":96627875840,"internally_reserved":0,"allocated":491520,"data_stored":218846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8388608},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:21:44 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.495}]},{"osd":1,"last update":"Tue Nov 28 11:21:44 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.59399999999999997}]},{"osd":2,"last update":"Tue Nov 28 11:21:44 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.71299999999999997}]},{"osd":3,"last update":"Tue Nov 28 11:21:44 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.81299999999999994}]},{"osd":4,"last update":"Tue Nov 28 11:21:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.73599999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:21:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.68000000000000005}]},{"osd":7,"last update":"Tue Nov 28 11:21:44 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.85399999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:22:29.096 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:22:29.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.323+0000 7ff67b9ff700 1 Processor -- start 2023-11-28T11:22:29.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.323+0000 7ff67b9ff700 1 -- start start 2023-11-28T11:22:29.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.324+0000 7ff67b9ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff67415d820 0x7ff6740cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:29.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.324+0000 7ff67b9ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff6740cfcf0 0x7ff6740b19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:29.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.324+0000 7ff67b9ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6740d0170 con 0x7ff67415a110 2023-11-28T11:22:29.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.325+0000 7ff67b9ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6740cdae0 con 0x7ff6740cfcf0 2023-11-28T11:22:29.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.325+0000 7ff67b9ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6740ce4f0 con 0x7ff67415d820 2023-11-28T11:22:29.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.325+0000 7ff679f9c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff6740cfcf0 0x7ff6740b19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:29.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.325+0000 7ff679f9c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff6740cfcf0 0x7ff6740b19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55954/0 (socket says 172.21.15.92:55954) 2023-11-28T11:22:29.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.325+0000 7ff679f9c700 1 -- 172.21.15.92:0/3186846235 learned_addr learned my addr 172.21.15.92:0/3186846235 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:29.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.325+0000 7ff678f9a700 1 --2- 172.21.15.92:0/3186846235 >> v2:172.21.15.169:3300/0 conn(0x7ff67415d820 0x7ff6740cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:29.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.325+0000 7ff679f9c700 1 -- 172.21.15.92:0/3186846235 >> v2:172.21.15.169:3300/0 conn(0x7ff67415d820 msgr2=0x7ff6740cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:29.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.325+0000 7ff679f9c700 1 --2- 172.21.15.92:0/3186846235 >> v2:172.21.15.169:3300/0 conn(0x7ff67415d820 0x7ff6740cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:29.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.325+0000 7ff679f9c700 1 -- 172.21.15.92:0/3186846235 >> v1:172.21.15.169:6789/0 conn(0x7ff67415a110 legacy=0x7ff674160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:29.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.325+0000 7ff679f9c700 1 -- 172.21.15.92:0/3186846235 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6740cf2d0 con 0x7ff6740cfcf0 2023-11-28T11:22:29.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.326+0000 7ff679f9c700 1 --2- 172.21.15.92:0/3186846235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6740cfcf0 0x7ff6740b19b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff6740d09b0 tx=0x7ff66c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3ba44ba4507198d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:29.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.326+0000 7ff66bfff700 1 -- 172.21.15.92:0/3186846235 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66c00e070 con 0x7ff6740cfcf0 2023-11-28T11:22:29.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.326+0000 7ff66bfff700 1 -- 172.21.15.92:0/3186846235 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff66c00b910 con 0x7ff6740cfcf0 2023-11-28T11:22:29.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.326+0000 7ff66bfff700 1 -- 172.21.15.92:0/3186846235 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66c00bcc0 con 0x7ff6740cfcf0 2023-11-28T11:22:29.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.326+0000 7ff67b9ff700 1 -- 172.21.15.92:0/3186846235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6740cfcf0 msgr2=0x7ff6740b19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:29.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.326+0000 7ff67b9ff700 1 --2- 172.21.15.92:0/3186846235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6740cfcf0 0x7ff6740b19b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff6740d09b0 tx=0x7ff66c00b330 comp rx=0 tx=0).stop 2023-11-28T11:22:29.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.327+0000 7ff67b9ff700 1 -- 172.21.15.92:0/3186846235 shutdown_connections 2023-11-28T11:22:29.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.327+0000 7ff67b9ff700 1 --2- 172.21.15.92:0/3186846235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6740cfcf0 0x7ff6740b19b0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:29.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.327+0000 7ff67b9ff700 1 --2- 172.21.15.92:0/3186846235 >> v2:172.21.15.169:3300/0 conn(0x7ff67415d820 0x7ff6740cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:29.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.327+0000 7ff67b9ff700 1 -- 172.21.15.92:0/3186846235 >> 172.21.15.92:0/3186846235 conn(0x7ff6740719a0 msgr2=0x7ff674071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:29.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.327+0000 7ff67b9ff700 1 -- 172.21.15.92:0/3186846235 shutdown_connections 2023-11-28T11:22:29.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.327+0000 7ff67b9ff700 1 -- 172.21.15.92:0/3186846235 wait complete. 2023-11-28T11:22:29.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.328+0000 7ff67b9ff700 1 Processor -- start 2023-11-28T11:22:29.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.328+0000 7ff67b9ff700 1 -- start start 2023-11-28T11:22:29.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.329+0000 7ff67b9ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff6740cfcf0 0x7ff6740cb960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:29.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.329+0000 7ff67b9ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff67415a110 0x7ff6740c7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:29.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.329+0000 7ff67b9ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff67415d820 0x7ff6740c7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:29.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.329+0000 7ff67b9ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6740cc970 con 0x7ff67415d820 2023-11-28T11:22:29.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.329+0000 7ff67b9ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff674158b90 con 0x7ff67415a110 2023-11-28T11:22:29.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.329+0000 7ff67b9ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6740c8eb0 con 0x7ff6740cfcf0 2023-11-28T11:22:29.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.330+0000 7ff679f9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff67415d820 0x7ff6740c7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:29.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.330+0000 7ff679f9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff67415d820 0x7ff6740c7fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55962/0 (socket says 172.21.15.92:55962) 2023-11-28T11:22:29.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.330+0000 7ff679f9c700 1 -- 172.21.15.92:0/2396462292 learned_addr learned my addr 172.21.15.92:0/2396462292 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:29.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.330+0000 7ff678f9a700 1 --2- 172.21.15.92:0/2396462292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff67415a110 0x7ff6740c7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:29.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.330+0000 7ff67979b700 1 --2- 172.21.15.92:0/2396462292 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff6740cfcf0 0x7ff6740cb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:29.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.330+0000 7ff679f9c700 1 -- 172.21.15.92:0/2396462292 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff6740cfcf0 msgr2=0x7ff6740cb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:29.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.330+0000 7ff679f9c700 1 --2- 172.21.15.92:0/2396462292 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff6740cfcf0 0x7ff6740cb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:29.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.330+0000 7ff679f9c700 1 -- 172.21.15.92:0/2396462292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff67415a110 msgr2=0x7ff6740c7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:29.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.330+0000 7ff679f9c700 1 --2- 172.21.15.92:0/2396462292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff67415a110 0x7ff6740c7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:29.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.330+0000 7ff679f9c700 1 -- 172.21.15.92:0/2396462292 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff66c009b30 con 0x7ff67415d820 2023-11-28T11:22:29.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.330+0000 7ff679f9c700 1 --2- 172.21.15.92:0/2396462292 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff67415d820 0x7ff6740c7fc0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ff66c00b370 tx=0x7ff66c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:29.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.331+0000 7ff669ffb700 1 -- 172.21.15.92:0/2396462292 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66c004880 con 0x7ff67415d820 2023-11-28T11:22:29.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.331+0000 7ff669ffb700 1 -- 172.21.15.92:0/2396462292 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff66c0099b0 con 0x7ff67415d820 2023-11-28T11:22:29.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.331+0000 7ff669ffb700 1 -- 172.21.15.92:0/2396462292 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66c01c680 con 0x7ff67415d820 2023-11-28T11:22:29.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.331+0000 7ff67b9ff700 1 -- 172.21.15.92:0/2396462292 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff67415cd40 con 0x7ff67415d820 2023-11-28T11:22:29.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.331+0000 7ff67b9ff700 1 -- 172.21.15.92:0/2396462292 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6740cd800 con 0x7ff67415d820 2023-11-28T11:22:29.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.332+0000 7ff67b9ff700 1 -- 172.21.15.92:0/2396462292 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6740cbea0 con 0x7ff67415d820 2023-11-28T11:22:29.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.333+0000 7ff669ffb700 1 -- 172.21.15.92:0/2396462292 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff66c022030 con 0x7ff67415d820 2023-11-28T11:22:29.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.334+0000 7ff669ffb700 1 --2- 172.21.15.92:0/2396462292 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff6580682f0 0x7ff65806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:29.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.334+0000 7ff669ffb700 1 -- 172.21.15.92:0/2396462292 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7ff66c087570 con 0x7ff67415d820 2023-11-28T11:22:29.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.334+0000 7ff67979b700 1 --2- 172.21.15.92:0/2396462292 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff6580682f0 0x7ff65806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:29.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.335+0000 7ff67979b700 1 --2- 172.21.15.92:0/2396462292 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff6580682f0 0x7ff65806a7a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff670007fd0 tx=0x7ff670009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:29.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.338+0000 7ff669ffb700 1 -- 172.21.15.92:0/2396462292 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff66c053c90 con 0x7ff67415d820 2023-11-28T11:22:29.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.523+0000 7ff67b9ff700 1 -- 172.21.15.92:0/2396462292 --> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff67415d020 con 0x7ff6580682f0 2023-11-28T11:22:29.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.532+0000 7ff669ffb700 1 -- 172.21.15.92:0/2396462292 <== mgr.4103 v2:172.21.15.92:6832/115482 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311324 (secure 0 0 0) 0x7ff67415d020 con 0x7ff6580682f0 2023-11-28T11:22:29.538 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:22:29.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.537+0000 7ff67b9ff700 1 -- 172.21.15.92:0/2396462292 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff6580682f0 msgr2=0x7ff65806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:29.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.537+0000 7ff67b9ff700 1 --2- 172.21.15.92:0/2396462292 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff6580682f0 0x7ff65806a7a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff670007fd0 tx=0x7ff670009040 comp rx=0 tx=0).stop 2023-11-28T11:22:29.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.537+0000 7ff67b9ff700 1 -- 172.21.15.92:0/2396462292 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff67415d820 msgr2=0x7ff6740c7fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:29.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.537+0000 7ff67b9ff700 1 --2- 172.21.15.92:0/2396462292 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff67415d820 0x7ff6740c7fc0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ff66c00b370 tx=0x7ff66c004ad0 comp rx=0 tx=0).stop 2023-11-28T11:22:29.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.537+0000 7ff67b9ff700 1 -- 172.21.15.92:0/2396462292 shutdown_connections 2023-11-28T11:22:29.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.537+0000 7ff67b9ff700 1 --2- 172.21.15.92:0/2396462292 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff6580682f0 0x7ff65806a7a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:29.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.537+0000 7ff67b9ff700 1 --2- 172.21.15.92:0/2396462292 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff67415d820 0x7ff6740c7fc0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:29.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.537+0000 7ff67b9ff700 1 --2- 172.21.15.92:0/2396462292 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff6740cfcf0 0x7ff6740cb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:29.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.537+0000 7ff67b9ff700 1 --2- 172.21.15.92:0/2396462292 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff67415a110 0x7ff6740c7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:29.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.537+0000 7ff67b9ff700 1 -- 172.21.15.92:0/2396462292 >> 172.21.15.92:0/2396462292 conn(0x7ff6740719a0 msgr2=0x7ff6740b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:29.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.537+0000 7ff67b9ff700 1 -- 172.21.15.92:0/2396462292 shutdown_connections 2023-11-28T11:22:29.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.537+0000 7ff67b9ff700 1 -- 172.21.15.92:0/2396462292 wait complete. 2023-11-28T11:22:29.542 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-11-28T11:22:29.557 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":88,"stamp":"2023-11-28T11:22:27.880247+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":157,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":71832,"kb_used_data":5016,"kb_used_omap":0,"kb_used_meta":66752,"kb_avail":754902888,"statfs":{"total":773094113280,"available":773020557312,"internally_reserved":0,"allocated":5136384,"data_stored":2928009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68354048},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.669966"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.106893+0000","last_change":"2023-11-28T11:22:24.743559+0000","last_active":"2023-11-28T11:22:26.106893+0000","last_peered":"2023-11-28T11:22:26.106893+0000","last_clean":"2023-11-28T11:22:26.106893+0000","last_became_active":"2023-11-28T11:22:24.743278+0000","last_became_peered":"2023-11-28T11:22:24.743278+0000","last_unstale":"2023-11-28T11:22:26.106893+0000","last_undegraded":"2023-11-28T11:22:26.106893+0000","last_fullsized":"2023-11-28T11:22:26.106893+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:52:15.093133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.735233+0000","last_change":"2023-11-28T11:22:24.763133+0000","last_active":"2023-11-28T11:22:25.735233+0000","last_peered":"2023-11-28T11:22:25.735233+0000","last_clean":"2023-11-28T11:22:25.735233+0000","last_became_active":"2023-11-28T11:22:24.762707+0000","last_became_peered":"2023-11-28T11:22:24.762707+0000","last_unstale":"2023-11-28T11:22:25.735233+0000","last_undegraded":"2023-11-28T11:22:25.735233+0000","last_fullsized":"2023-11-28T11:22:25.735233+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:00:53.225852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.734437+0000","last_change":"2023-11-28T11:22:24.741466+0000","last_active":"2023-11-28T11:22:25.734437+0000","last_peered":"2023-11-28T11:22:25.734437+0000","last_clean":"2023-11-28T11:22:25.734437+0000","last_became_active":"2023-11-28T11:22:24.741194+0000","last_became_peered":"2023-11-28T11:22:24.741194+0000","last_unstale":"2023-11-28T11:22:25.734437+0000","last_undegraded":"2023-11-28T11:22:25.734437+0000","last_fullsized":"2023-11-28T11:22:25.734437+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:50:58.725549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.918843+0000","last_change":"2023-11-28T11:22:24.750046+0000","last_active":"2023-11-28T11:22:25.918843+0000","last_peered":"2023-11-28T11:22:25.918843+0000","last_clean":"2023-11-28T11:22:25.918843+0000","last_became_active":"2023-11-28T11:22:24.749769+0000","last_became_peered":"2023-11-28T11:22:24.749769+0000","last_unstale":"2023-11-28T11:22:25.918843+0000","last_undegraded":"2023-11-28T11:22:25.918843+0000","last_fullsized":"2023-11-28T11:22:25.918843+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:48:58.095068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.734980+0000","last_change":"2023-11-28T11:22:24.763161+0000","last_active":"2023-11-28T11:22:25.734980+0000","last_peered":"2023-11-28T11:22:25.734980+0000","last_clean":"2023-11-28T11:22:25.734980+0000","last_became_active":"2023-11-28T11:22:24.762738+0000","last_became_peered":"2023-11-28T11:22:24.762738+0000","last_unstale":"2023-11-28T11:22:25.734980+0000","last_undegraded":"2023-11-28T11:22:25.734980+0000","last_fullsized":"2023-11-28T11:22:25.734980+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:39:46.327957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.106038+0000","last_change":"2023-11-28T11:22:24.744553+0000","last_active":"2023-11-28T11:22:26.106038+0000","last_peered":"2023-11-28T11:22:26.106038+0000","last_clean":"2023-11-28T11:22:26.106038+0000","last_became_active":"2023-11-28T11:22:24.742417+0000","last_became_peered":"2023-11-28T11:22:24.742417+0000","last_unstale":"2023-11-28T11:22:26.106038+0000","last_undegraded":"2023-11-28T11:22:26.106038+0000","last_fullsized":"2023-11-28T11:22:26.106038+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:20:38.630366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.106443+0000","last_change":"2023-11-28T11:22:24.747287+0000","last_active":"2023-11-28T11:22:26.106443+0000","last_peered":"2023-11-28T11:22:26.106443+0000","last_clean":"2023-11-28T11:22:26.106443+0000","last_became_active":"2023-11-28T11:22:24.744650+0000","last_became_peered":"2023-11-28T11:22:24.744650+0000","last_unstale":"2023-11-28T11:22:26.106443+0000","last_undegraded":"2023-11-28T11:22:26.106443+0000","last_fullsized":"2023-11-28T11:22:26.106443+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:25:57.624641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.918634+0000","last_change":"2023-11-28T11:22:24.743031+0000","last_active":"2023-11-28T11:22:25.918634+0000","last_peered":"2023-11-28T11:22:25.918634+0000","last_clean":"2023-11-28T11:22:25.918634+0000","last_became_active":"2023-11-28T11:22:24.742767+0000","last_became_peered":"2023-11-28T11:22:24.742767+0000","last_unstale":"2023-11-28T11:22:25.918634+0000","last_undegraded":"2023-11-28T11:22:25.918634+0000","last_fullsized":"2023-11-28T11:22:25.918634+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:45:09.980004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.919231+0000","last_change":"2023-11-28T11:22:24.756248+0000","last_active":"2023-11-28T11:22:25.919231+0000","last_peered":"2023-11-28T11:22:25.919231+0000","last_clean":"2023-11-28T11:22:25.919231+0000","last_became_active":"2023-11-28T11:22:24.754413+0000","last_became_peered":"2023-11-28T11:22:24.754413+0000","last_unstale":"2023-11-28T11:22:25.919231+0000","last_undegraded":"2023-11-28T11:22:25.919231+0000","last_fullsized":"2023-11-28T11:22:25.919231+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:25:57.624641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.919187+0000","last_change":"2023-11-28T11:22:24.747851+0000","last_active":"2023-11-28T11:22:25.919187+0000","last_peered":"2023-11-28T11:22:25.919187+0000","last_clean":"2023-11-28T11:22:25.919187+0000","last_became_active":"2023-11-28T11:22:24.747602+0000","last_became_peered":"2023-11-28T11:22:24.747602+0000","last_unstale":"2023-11-28T11:22:25.919187+0000","last_undegraded":"2023-11-28T11:22:25.919187+0000","last_fullsized":"2023-11-28T11:22:25.919187+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:02:19.560108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.918993+0000","last_change":"2023-11-28T11:22:24.751122+0000","last_active":"2023-11-28T11:22:25.918993+0000","last_peered":"2023-11-28T11:22:25.918993+0000","last_clean":"2023-11-28T11:22:25.918993+0000","last_became_active":"2023-11-28T11:22:24.750852+0000","last_became_peered":"2023-11-28T11:22:24.750852+0000","last_unstale":"2023-11-28T11:22:25.918993+0000","last_undegraded":"2023-11-28T11:22:25.918993+0000","last_fullsized":"2023-11-28T11:22:25.918993+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:07:08.708946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.735330+0000","last_change":"2023-11-28T11:22:24.753686+0000","last_active":"2023-11-28T11:22:25.735330+0000","last_peered":"2023-11-28T11:22:25.735330+0000","last_clean":"2023-11-28T11:22:25.735330+0000","last_became_active":"2023-11-28T11:22:24.752399+0000","last_became_peered":"2023-11-28T11:22:24.752399+0000","last_unstale":"2023-11-28T11:22:25.735330+0000","last_undegraded":"2023-11-28T11:22:25.735330+0000","last_fullsized":"2023-11-28T11:22:25.735330+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:30:58.618362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.105748+0000","last_change":"2023-11-28T11:22:24.749386+0000","last_active":"2023-11-28T11:22:26.105748+0000","last_peered":"2023-11-28T11:22:26.105748+0000","last_clean":"2023-11-28T11:22:26.105748+0000","last_became_active":"2023-11-28T11:22:24.749136+0000","last_became_peered":"2023-11-28T11:22:24.749136+0000","last_unstale":"2023-11-28T11:22:26.105748+0000","last_undegraded":"2023-11-28T11:22:26.105748+0000","last_fullsized":"2023-11-28T11:22:26.105748+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:33:45.044082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.918900+0000","last_change":"2023-11-28T11:22:24.757076+0000","last_active":"2023-11-28T11:22:25.918900+0000","last_peered":"2023-11-28T11:22:25.918900+0000","last_clean":"2023-11-28T11:22:25.918900+0000","last_became_active":"2023-11-28T11:22:24.756832+0000","last_became_peered":"2023-11-28T11:22:24.756832+0000","last_unstale":"2023-11-28T11:22:25.918900+0000","last_undegraded":"2023-11-28T11:22:25.918900+0000","last_fullsized":"2023-11-28T11:22:25.918900+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:39:46.327957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.106031+0000","last_change":"2023-11-28T11:22:24.743790+0000","last_active":"2023-11-28T11:22:26.106031+0000","last_peered":"2023-11-28T11:22:26.106031+0000","last_clean":"2023-11-28T11:22:26.106031+0000","last_became_active":"2023-11-28T11:22:24.743589+0000","last_became_peered":"2023-11-28T11:22:24.743589+0000","last_unstale":"2023-11-28T11:22:26.106031+0000","last_undegraded":"2023-11-28T11:22:26.106031+0000","last_fullsized":"2023-11-28T11:22:26.106031+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:21:04.288976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.108698+0000","last_change":"2023-11-28T11:22:24.757376+0000","last_active":"2023-11-28T11:22:26.108698+0000","last_peered":"2023-11-28T11:22:26.108698+0000","last_clean":"2023-11-28T11:22:26.108698+0000","last_became_active":"2023-11-28T11:22:24.757115+0000","last_became_peered":"2023-11-28T11:22:24.757115+0000","last_unstale":"2023-11-28T11:22:26.108698+0000","last_undegraded":"2023-11-28T11:22:26.108698+0000","last_fullsized":"2023-11-28T11:22:26.108698+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:50:31.355306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.734681+0000","last_change":"2023-11-28T11:22:24.748161+0000","last_active":"2023-11-28T11:22:25.734681+0000","last_peered":"2023-11-28T11:22:25.734681+0000","last_clean":"2023-11-28T11:22:25.734681+0000","last_became_active":"2023-11-28T11:22:24.747874+0000","last_became_peered":"2023-11-28T11:22:24.747874+0000","last_unstale":"2023-11-28T11:22:25.734681+0000","last_undegraded":"2023-11-28T11:22:25.734681+0000","last_fullsized":"2023-11-28T11:22:25.734681+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:48:58.095068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.734503+0000","last_change":"2023-11-28T11:22:24.758001+0000","last_active":"2023-11-28T11:22:25.734503+0000","last_peered":"2023-11-28T11:22:25.734503+0000","last_clean":"2023-11-28T11:22:25.734503+0000","last_became_active":"2023-11-28T11:22:24.756789+0000","last_became_peered":"2023-11-28T11:22:24.756789+0000","last_unstale":"2023-11-28T11:22:25.734503+0000","last_undegraded":"2023-11-28T11:22:25.734503+0000","last_fullsized":"2023-11-28T11:22:25.734503+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:27:42.158553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.108954+0000","last_change":"2023-11-28T11:22:24.757403+0000","last_active":"2023-11-28T11:22:26.108954+0000","last_peered":"2023-11-28T11:22:26.108954+0000","last_clean":"2023-11-28T11:22:26.108954+0000","last_became_active":"2023-11-28T11:22:24.757156+0000","last_became_peered":"2023-11-28T11:22:24.757156+0000","last_unstale":"2023-11-28T11:22:26.108954+0000","last_undegraded":"2023-11-28T11:22:26.108954+0000","last_fullsized":"2023-11-28T11:22:26.108954+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:31:16.730971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.105729+0000","last_change":"2023-11-28T11:22:24.760399+0000","last_active":"2023-11-28T11:22:26.105729+0000","last_peered":"2023-11-28T11:22:26.105729+0000","last_clean":"2023-11-28T11:22:26.105729+0000","last_became_active":"2023-11-28T11:22:24.760144+0000","last_became_peered":"2023-11-28T11:22:24.760144+0000","last_unstale":"2023-11-28T11:22:26.105729+0000","last_undegraded":"2023-11-28T11:22:26.105729+0000","last_fullsized":"2023-11-28T11:22:26.105729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:31:16.730971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.106179+0000","last_change":"2023-11-28T11:22:24.752618+0000","last_active":"2023-11-28T11:22:26.106179+0000","last_peered":"2023-11-28T11:22:26.106179+0000","last_clean":"2023-11-28T11:22:26.106179+0000","last_became_active":"2023-11-28T11:22:24.749820+0000","last_became_peered":"2023-11-28T11:22:24.749820+0000","last_unstale":"2023-11-28T11:22:26.106179+0000","last_undegraded":"2023-11-28T11:22:26.106179+0000","last_fullsized":"2023-11-28T11:22:26.106179+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:04:03.997360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.919019+0000","last_change":"2023-11-28T11:22:24.749121+0000","last_active":"2023-11-28T11:22:25.919019+0000","last_peered":"2023-11-28T11:22:25.919019+0000","last_clean":"2023-11-28T11:22:25.919019+0000","last_became_active":"2023-11-28T11:22:24.748888+0000","last_became_peered":"2023-11-28T11:22:24.748888+0000","last_unstale":"2023-11-28T11:22:25.919019+0000","last_undegraded":"2023-11-28T11:22:25.919019+0000","last_fullsized":"2023-11-28T11:22:25.919019+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:01:46.793779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.108756+0000","last_change":"2023-11-28T11:22:24.751973+0000","last_active":"2023-11-28T11:22:26.108756+0000","last_peered":"2023-11-28T11:22:26.108756+0000","last_clean":"2023-11-28T11:22:26.108756+0000","last_became_active":"2023-11-28T11:22:24.751205+0000","last_became_peered":"2023-11-28T11:22:24.751205+0000","last_unstale":"2023-11-28T11:22:26.108756+0000","last_undegraded":"2023-11-28T11:22:26.108756+0000","last_fullsized":"2023-11-28T11:22:26.108756+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:39:01.023278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.108769+0000","last_change":"2023-11-28T11:22:24.751505+0000","last_active":"2023-11-28T11:22:26.108769+0000","last_peered":"2023-11-28T11:22:26.108769+0000","last_clean":"2023-11-28T11:22:26.108769+0000","last_became_active":"2023-11-28T11:22:24.751187+0000","last_became_peered":"2023-11-28T11:22:24.751187+0000","last_unstale":"2023-11-28T11:22:26.108769+0000","last_undegraded":"2023-11-28T11:22:26.108769+0000","last_fullsized":"2023-11-28T11:22:26.108769+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:50:08.305771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.109336+0000","last_change":"2023-11-28T11:22:24.752027+0000","last_active":"2023-11-28T11:22:26.109336+0000","last_peered":"2023-11-28T11:22:26.109336+0000","last_clean":"2023-11-28T11:22:26.109336+0000","last_became_active":"2023-11-28T11:22:24.751727+0000","last_became_peered":"2023-11-28T11:22:24.751727+0000","last_unstale":"2023-11-28T11:22:26.109336+0000","last_undegraded":"2023-11-28T11:22:26.109336+0000","last_fullsized":"2023-11-28T11:22:26.109336+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:38:23.565208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.105589+0000","last_change":"2023-11-28T11:22:24.750048+0000","last_active":"2023-11-28T11:22:26.105589+0000","last_peered":"2023-11-28T11:22:26.105589+0000","last_clean":"2023-11-28T11:22:26.105589+0000","last_became_active":"2023-11-28T11:22:24.749800+0000","last_became_peered":"2023-11-28T11:22:24.749800+0000","last_unstale":"2023-11-28T11:22:26.105589+0000","last_undegraded":"2023-11-28T11:22:26.105589+0000","last_fullsized":"2023-11-28T11:22:26.105589+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:38:23.565208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.734590+0000","last_change":"2023-11-28T11:22:24.746691+0000","last_active":"2023-11-28T11:22:25.734590+0000","last_peered":"2023-11-28T11:22:25.734590+0000","last_clean":"2023-11-28T11:22:25.734590+0000","last_became_active":"2023-11-28T11:22:24.746414+0000","last_became_peered":"2023-11-28T11:22:24.746414+0000","last_unstale":"2023-11-28T11:22:25.734590+0000","last_undegraded":"2023-11-28T11:22:25.734590+0000","last_fullsized":"2023-11-28T11:22:25.734590+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:23:14.629301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.735126+0000","last_change":"2023-11-28T11:22:24.747468+0000","last_active":"2023-11-28T11:22:25.735126+0000","last_peered":"2023-11-28T11:22:25.735126+0000","last_clean":"2023-11-28T11:22:25.735126+0000","last_became_active":"2023-11-28T11:22:24.747206+0000","last_became_peered":"2023-11-28T11:22:24.747206+0000","last_unstale":"2023-11-28T11:22:25.735126+0000","last_undegraded":"2023-11-28T11:22:25.735126+0000","last_fullsized":"2023-11-28T11:22:25.735126+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:08:22.302877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.109034+0000","last_change":"2023-11-28T11:22:24.750628+0000","last_active":"2023-11-28T11:22:26.109034+0000","last_peered":"2023-11-28T11:22:26.109034+0000","last_clean":"2023-11-28T11:22:26.109034+0000","last_became_active":"2023-11-28T11:22:24.750343+0000","last_became_peered":"2023-11-28T11:22:24.750343+0000","last_unstale":"2023-11-28T11:22:26.109034+0000","last_undegraded":"2023-11-28T11:22:26.109034+0000","last_fullsized":"2023-11-28T11:22:26.109034+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:52:15.751540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.734736+0000","last_change":"2023-11-28T11:22:24.743191+0000","last_active":"2023-11-28T11:22:25.734736+0000","last_peered":"2023-11-28T11:22:25.734736+0000","last_clean":"2023-11-28T11:22:25.734736+0000","last_became_active":"2023-11-28T11:22:24.742926+0000","last_became_peered":"2023-11-28T11:22:24.742926+0000","last_unstale":"2023-11-28T11:22:25.734736+0000","last_undegraded":"2023-11-28T11:22:25.734736+0000","last_fullsized":"2023-11-28T11:22:25.734736+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:18:49.637085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.105773+0000","last_change":"2023-11-28T11:22:24.748844+0000","last_active":"2023-11-28T11:22:26.105773+0000","last_peered":"2023-11-28T11:22:26.105773+0000","last_clean":"2023-11-28T11:22:26.105773+0000","last_became_active":"2023-11-28T11:22:24.748563+0000","last_became_peered":"2023-11-28T11:22:24.748563+0000","last_unstale":"2023-11-28T11:22:26.105773+0000","last_undegraded":"2023-11-28T11:22:26.105773+0000","last_fullsized":"2023-11-28T11:22:26.105773+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:49:07.842588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.105513+0000","last_change":"2023-11-28T11:22:24.750190+0000","last_active":"2023-11-28T11:22:26.105513+0000","last_peered":"2023-11-28T11:22:26.105513+0000","last_clean":"2023-11-28T11:22:26.105513+0000","last_became_active":"2023-11-28T11:22:24.747715+0000","last_became_peered":"2023-11-28T11:22:24.747715+0000","last_unstale":"2023-11-28T11:22:26.105513+0000","last_undegraded":"2023-11-28T11:22:26.105513+0000","last_fullsized":"2023-11-28T11:22:26.105513+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:33:57.540987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.732631+0000","last_change":"2023-11-28T11:22:24.750462+0000","last_active":"2023-11-28T11:22:25.732631+0000","last_peered":"2023-11-28T11:22:25.732631+0000","last_clean":"2023-11-28T11:22:25.732631+0000","last_became_active":"2023-11-28T11:22:24.750208+0000","last_became_peered":"2023-11-28T11:22:24.750208+0000","last_unstale":"2023-11-28T11:22:25.732631+0000","last_undegraded":"2023-11-28T11:22:25.732631+0000","last_fullsized":"2023-11-28T11:22:25.732631+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:53:29.715029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"41'4","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.106467+0000","last_change":"2023-11-28T11:22:24.243596+0000","last_active":"2023-11-28T11:22:26.106467+0000","last_peered":"2023-11-28T11:22:26.106467+0000","last_clean":"2023-11-28T11:22:26.106467+0000","last_became_active":"2023-11-28T11:22:24.243317+0000","last_became_peered":"2023-11-28T11:22:24.243317+0000","last_unstale":"2023-11-28T11:22:26.106467+0000","last_undegraded":"2023-11-28T11:22:26.106467+0000","last_fullsized":"2023-11-28T11:22:26.106467+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:39:45.321174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.109240+0000","last_change":"2023-11-28T11:22:24.749451+0000","last_active":"2023-11-28T11:22:26.109240+0000","last_peered":"2023-11-28T11:22:26.109240+0000","last_clean":"2023-11-28T11:22:26.109240+0000","last_became_active":"2023-11-28T11:22:24.748994+0000","last_became_peered":"2023-11-28T11:22:24.748994+0000","last_unstale":"2023-11-28T11:22:26.109240+0000","last_undegraded":"2023-11-28T11:22:26.109240+0000","last_fullsized":"2023-11-28T11:22:26.109240+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:11:41.372618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"41'1","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.734790+0000","last_change":"2023-11-28T11:22:24.238405+0000","last_active":"2023-11-28T11:22:25.734790+0000","last_peered":"2023-11-28T11:22:25.734790+0000","last_clean":"2023-11-28T11:22:25.734790+0000","last_became_active":"2023-11-28T11:22:24.237714+0000","last_became_peered":"2023-11-28T11:22:24.237714+0000","last_unstale":"2023-11-28T11:22:25.734790+0000","last_undegraded":"2023-11-28T11:22:25.734790+0000","last_fullsized":"2023-11-28T11:22:25.734790+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:39:54.488254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.735262+0000","last_change":"2023-11-28T11:22:24.749958+0000","last_active":"2023-11-28T11:22:25.735262+0000","last_peered":"2023-11-28T11:22:25.735262+0000","last_clean":"2023-11-28T11:22:25.735262+0000","last_became_active":"2023-11-28T11:22:24.749659+0000","last_became_peered":"2023-11-28T11:22:24.749659+0000","last_unstale":"2023-11-28T11:22:25.735262+0000","last_undegraded":"2023-11-28T11:22:25.735262+0000","last_fullsized":"2023-11-28T11:22:25.735262+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:54:40.929864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"41'2","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.732662+0000","last_change":"2023-11-28T11:22:24.246752+0000","last_active":"2023-11-28T11:22:25.732662+0000","last_peered":"2023-11-28T11:22:25.732662+0000","last_clean":"2023-11-28T11:22:25.732662+0000","last_became_active":"2023-11-28T11:22:24.243536+0000","last_became_peered":"2023-11-28T11:22:24.243536+0000","last_unstale":"2023-11-28T11:22:25.732662+0000","last_undegraded":"2023-11-28T11:22:25.732662+0000","last_fullsized":"2023-11-28T11:22:25.732662+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:33:56.534204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.732537+0000","last_change":"2023-11-28T11:22:24.744468+0000","last_active":"2023-11-28T11:22:25.732537+0000","last_peered":"2023-11-28T11:22:25.732537+0000","last_clean":"2023-11-28T11:22:25.732537+0000","last_became_active":"2023-11-28T11:22:24.744142+0000","last_became_peered":"2023-11-28T11:22:24.744142+0000","last_unstale":"2023-11-28T11:22:25.732537+0000","last_undegraded":"2023-11-28T11:22:25.732537+0000","last_fullsized":"2023-11-28T11:22:25.732537+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:58.550893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"41'2","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.106704+0000","last_change":"2023-11-28T11:22:24.239579+0000","last_active":"2023-11-28T11:22:26.106704+0000","last_peered":"2023-11-28T11:22:26.106704+0000","last_clean":"2023-11-28T11:22:26.106704+0000","last_became_active":"2023-11-28T11:22:24.239276+0000","last_became_peered":"2023-11-28T11:22:24.239276+0000","last_unstale":"2023-11-28T11:22:26.106704+0000","last_undegraded":"2023-11-28T11:22:26.106704+0000","last_fullsized":"2023-11-28T11:22:26.106704+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:50:30.348523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.734289+0000","last_change":"2023-11-28T11:22:24.754252+0000","last_active":"2023-11-28T11:22:25.734289+0000","last_peered":"2023-11-28T11:22:25.734289+0000","last_clean":"2023-11-28T11:22:25.734289+0000","last_became_active":"2023-11-28T11:22:24.753350+0000","last_became_peered":"2023-11-28T11:22:24.753350+0000","last_unstale":"2023-11-28T11:22:25.734289+0000","last_undegraded":"2023-11-28T11:22:25.734289+0000","last_fullsized":"2023-11-28T11:22:25.734289+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:02:19.560108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"41'1","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.106163+0000","last_change":"2023-11-28T11:22:24.229901+0000","last_active":"2023-11-28T11:22:26.106163+0000","last_peered":"2023-11-28T11:22:26.106163+0000","last_clean":"2023-11-28T11:22:26.106163+0000","last_became_active":"2023-11-28T11:22:24.229638+0000","last_became_peered":"2023-11-28T11:22:24.229638+0000","last_unstale":"2023-11-28T11:22:26.106163+0000","last_undegraded":"2023-11-28T11:22:26.106163+0000","last_fullsized":"2023-11-28T11:22:26.106163+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:23:13.622518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.918857+0000","last_change":"2023-11-28T11:22:24.751654+0000","last_active":"2023-11-28T11:22:25.918857+0000","last_peered":"2023-11-28T11:22:25.918857+0000","last_clean":"2023-11-28T11:22:25.918857+0000","last_became_active":"2023-11-28T11:22:24.751370+0000","last_became_peered":"2023-11-28T11:22:24.751370+0000","last_unstale":"2023-11-28T11:22:25.918857+0000","last_undegraded":"2023-11-28T11:22:25.918857+0000","last_fullsized":"2023-11-28T11:22:25.918857+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:00:53.225852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"41'1","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.735147+0000","last_change":"2023-11-28T11:22:24.233155+0000","last_active":"2023-11-28T11:22:25.735147+0000","last_peered":"2023-11-28T11:22:25.735147+0000","last_clean":"2023-11-28T11:22:25.735147+0000","last_became_active":"2023-11-28T11:22:24.232854+0000","last_became_peered":"2023-11-28T11:22:24.232854+0000","last_unstale":"2023-11-28T11:22:25.735147+0000","last_undegraded":"2023-11-28T11:22:25.735147+0000","last_fullsized":"2023-11-28T11:22:25.735147+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:05:48.569713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.918720+0000","last_change":"2023-11-28T11:22:24.742651+0000","last_active":"2023-11-28T11:22:25.918720+0000","last_peered":"2023-11-28T11:22:25.918720+0000","last_clean":"2023-11-28T11:22:25.918720+0000","last_became_active":"2023-11-28T11:22:24.741692+0000","last_became_peered":"2023-11-28T11:22:24.741692+0000","last_unstale":"2023-11-28T11:22:25.918720+0000","last_undegraded":"2023-11-28T11:22:25.918720+0000","last_fullsized":"2023-11-28T11:22:25.918720+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:23:14.629301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.734589+0000","last_change":"2023-11-28T11:22:24.234395+0000","last_active":"2023-11-28T11:22:25.734589+0000","last_peered":"2023-11-28T11:22:25.734589+0000","last_clean":"2023-11-28T11:22:25.734589+0000","last_became_active":"2023-11-28T11:22:24.234135+0000","last_became_peered":"2023-11-28T11:22:24.234135+0000","last_unstale":"2023-11-28T11:22:25.734589+0000","last_undegraded":"2023-11-28T11:22:25.734589+0000","last_fullsized":"2023-11-28T11:22:25.734589+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:53:27.359189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.108922+0000","last_change":"2023-11-28T11:22:24.753515+0000","last_active":"2023-11-28T11:22:26.108922+0000","last_peered":"2023-11-28T11:22:26.108922+0000","last_clean":"2023-11-28T11:22:26.108922+0000","last_became_active":"2023-11-28T11:22:24.753227+0000","last_became_peered":"2023-11-28T11:22:24.753227+0000","last_unstale":"2023-11-28T11:22:26.108922+0000","last_undegraded":"2023-11-28T11:22:26.108922+0000","last_fullsized":"2023-11-28T11:22:26.108922+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:44:45.612931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.918731+0000","last_change":"2023-11-28T11:22:24.242991+0000","last_active":"2023-11-28T11:22:25.918731+0000","last_peered":"2023-11-28T11:22:25.918731+0000","last_clean":"2023-11-28T11:22:25.918731+0000","last_became_active":"2023-11-28T11:22:24.242524+0000","last_became_peered":"2023-11-28T11:22:24.242524+0000","last_unstale":"2023-11-28T11:22:25.918731+0000","last_undegraded":"2023-11-28T11:22:25.918731+0000","last_fullsized":"2023-11-28T11:22:25.918731+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:31:15.724188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.734738+0000","last_change":"2023-11-28T11:22:24.757212+0000","last_active":"2023-11-28T11:22:25.734738+0000","last_peered":"2023-11-28T11:22:25.734738+0000","last_clean":"2023-11-28T11:22:25.734738+0000","last_became_active":"2023-11-28T11:22:24.756934+0000","last_became_peered":"2023-11-28T11:22:24.756934+0000","last_unstale":"2023-11-28T11:22:25.734738+0000","last_undegraded":"2023-11-28T11:22:25.734738+0000","last_fullsized":"2023-11-28T11:22:25.734738+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:29:53.939582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.734827+0000","last_change":"2023-11-28T11:22:24.234787+0000","last_active":"2023-11-28T11:22:25.734827+0000","last_peered":"2023-11-28T11:22:25.734827+0000","last_clean":"2023-11-28T11:22:25.734827+0000","last_became_active":"2023-11-28T11:22:24.233501+0000","last_became_peered":"2023-11-28T11:22:24.233501+0000","last_unstale":"2023-11-28T11:22:25.734827+0000","last_undegraded":"2023-11-28T11:22:25.734827+0000","last_fullsized":"2023-11-28T11:22:25.734827+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:25:56.617859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.106938+0000","last_change":"2023-11-28T11:22:24.750717+0000","last_active":"2023-11-28T11:22:26.106938+0000","last_peered":"2023-11-28T11:22:26.106938+0000","last_clean":"2023-11-28T11:22:26.106938+0000","last_became_active":"2023-11-28T11:22:24.747789+0000","last_became_peered":"2023-11-28T11:22:24.747789+0000","last_unstale":"2023-11-28T11:22:26.106938+0000","last_undegraded":"2023-11-28T11:22:26.106938+0000","last_fullsized":"2023-11-28T11:22:26.106938+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:05:49.576496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.919233+0000","last_change":"2023-11-28T11:22:24.244527+0000","last_active":"2023-11-28T11:22:25.919233+0000","last_peered":"2023-11-28T11:22:25.919233+0000","last_clean":"2023-11-28T11:22:25.919233+0000","last_became_active":"2023-11-28T11:22:24.244239+0000","last_became_peered":"2023-11-28T11:22:24.244239+0000","last_unstale":"2023-11-28T11:22:25.919233+0000","last_undegraded":"2023-11-28T11:22:25.919233+0000","last_fullsized":"2023-11-28T11:22:25.919233+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:50:30.348523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.106322+0000","last_change":"2023-11-28T11:22:24.745415+0000","last_active":"2023-11-28T11:22:26.106322+0000","last_peered":"2023-11-28T11:22:26.106322+0000","last_clean":"2023-11-28T11:22:26.106322+0000","last_became_active":"2023-11-28T11:22:24.745170+0000","last_became_peered":"2023-11-28T11:22:24.745170+0000","last_unstale":"2023-11-28T11:22:26.106322+0000","last_undegraded":"2023-11-28T11:22:26.106322+0000","last_fullsized":"2023-11-28T11:22:26.106322+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:29:58.643132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"41'3","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.106887+0000","last_change":"2023-11-28T11:22:24.239784+0000","last_active":"2023-11-28T11:22:26.106887+0000","last_peered":"2023-11-28T11:22:26.106887+0000","last_clean":"2023-11-28T11:22:26.106887+0000","last_became_active":"2023-11-28T11:22:24.239484+0000","last_became_peered":"2023-11-28T11:22:24.239484+0000","last_unstale":"2023-11-28T11:22:26.106887+0000","last_undegraded":"2023-11-28T11:22:26.106887+0000","last_fullsized":"2023-11-28T11:22:26.106887+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:33:56.534204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.106758+0000","last_change":"2023-11-28T11:22:24.750309+0000","last_active":"2023-11-28T11:22:26.106758+0000","last_peered":"2023-11-28T11:22:26.106758+0000","last_clean":"2023-11-28T11:22:26.106758+0000","last_became_active":"2023-11-28T11:22:24.749432+0000","last_became_peered":"2023-11-28T11:22:24.749432+0000","last_unstale":"2023-11-28T11:22:26.106758+0000","last_undegraded":"2023-11-28T11:22:26.106758+0000","last_fullsized":"2023-11-28T11:22:26.106758+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:48:07.849967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"41'6","reported_seq":17,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.105831+0000","last_change":"2023-11-28T11:22:24.245339+0000","last_active":"2023-11-28T11:22:26.105831+0000","last_peered":"2023-11-28T11:22:26.105831+0000","last_clean":"2023-11-28T11:22:26.105831+0000","last_became_active":"2023-11-28T11:22:24.245096+0000","last_became_peered":"2023-11-28T11:22:24.245096+0000","last_unstale":"2023-11-28T11:22:26.105831+0000","last_undegraded":"2023-11-28T11:22:26.105831+0000","last_fullsized":"2023-11-28T11:22:26.105831+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:58:33.293304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.108826+0000","last_change":"2023-11-28T11:22:24.758212+0000","last_active":"2023-11-28T11:22:26.108826+0000","last_peered":"2023-11-28T11:22:26.108826+0000","last_clean":"2023-11-28T11:22:26.108826+0000","last_became_active":"2023-11-28T11:22:24.757952+0000","last_became_peered":"2023-11-28T11:22:24.757952+0000","last_unstale":"2023-11-28T11:22:26.108826+0000","last_undegraded":"2023-11-28T11:22:26.108826+0000","last_fullsized":"2023-11-28T11:22:26.108826+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:49:54.519059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"41'2","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.106290+0000","last_change":"2023-11-28T11:22:24.245144+0000","last_active":"2023-11-28T11:22:26.106290+0000","last_peered":"2023-11-28T11:22:26.106290+0000","last_clean":"2023-11-28T11:22:26.106290+0000","last_became_active":"2023-11-28T11:22:24.244868+0000","last_became_peered":"2023-11-28T11:22:24.244868+0000","last_unstale":"2023-11-28T11:22:26.106290+0000","last_undegraded":"2023-11-28T11:22:26.106290+0000","last_fullsized":"2023-11-28T11:22:26.106290+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:08:21.296095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.919210+0000","last_change":"2023-11-28T11:22:24.751160+0000","last_active":"2023-11-28T11:22:25.919210+0000","last_peered":"2023-11-28T11:22:25.919210+0000","last_clean":"2023-11-28T11:22:25.919210+0000","last_became_active":"2023-11-28T11:22:24.750895+0000","last_became_peered":"2023-11-28T11:22:24.750895+0000","last_unstale":"2023-11-28T11:22:25.919210+0000","last_undegraded":"2023-11-28T11:22:25.919210+0000","last_fullsized":"2023-11-28T11:22:25.919210+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:50:58.725549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.106543+0000","last_change":"2023-11-28T11:22:24.240189+0000","last_active":"2023-11-28T11:22:26.106543+0000","last_peered":"2023-11-28T11:22:26.106543+0000","last_clean":"2023-11-28T11:22:26.106543+0000","last_became_active":"2023-11-28T11:22:24.239869+0000","last_became_peered":"2023-11-28T11:22:24.239869+0000","last_unstale":"2023-11-28T11:22:26.106543+0000","last_undegraded":"2023-11-28T11:22:26.106543+0000","last_fullsized":"2023-11-28T11:22:26.106543+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:43:48.886755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.106063+0000","last_change":"2023-11-28T11:22:24.741396+0000","last_active":"2023-11-28T11:22:26.106063+0000","last_peered":"2023-11-28T11:22:26.106063+0000","last_clean":"2023-11-28T11:22:26.106063+0000","last_became_active":"2023-11-28T11:22:24.741095+0000","last_became_peered":"2023-11-28T11:22:24.741095+0000","last_unstale":"2023-11-28T11:22:26.106063+0000","last_undegraded":"2023-11-28T11:22:26.106063+0000","last_fullsized":"2023-11-28T11:22:26.106063+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:06:15.963909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.105953+0000","last_change":"2023-11-28T11:22:24.243978+0000","last_active":"2023-11-28T11:22:26.105953+0000","last_peered":"2023-11-28T11:22:26.105953+0000","last_clean":"2023-11-28T11:22:26.105953+0000","last_became_active":"2023-11-28T11:22:24.243620+0000","last_became_peered":"2023-11-28T11:22:24.243620+0000","last_unstale":"2023-11-28T11:22:26.105953+0000","last_undegraded":"2023-11-28T11:22:26.105953+0000","last_fullsized":"2023-11-28T11:22:26.105953+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:54:39.923082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.106517+0000","last_change":"2023-11-28T11:22:24.745735+0000","last_active":"2023-11-28T11:22:26.106517+0000","last_peered":"2023-11-28T11:22:26.106517+0000","last_clean":"2023-11-28T11:22:26.106517+0000","last_became_active":"2023-11-28T11:22:24.744262+0000","last_became_peered":"2023-11-28T11:22:24.744262+0000","last_unstale":"2023-11-28T11:22:26.106517+0000","last_undegraded":"2023-11-28T11:22:26.106517+0000","last_fullsized":"2023-11-28T11:22:26.106517+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:30:58.618362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"41'8","reported_seq":19,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.918887+0000","last_change":"2023-11-28T11:22:24.235077+0000","last_active":"2023-11-28T11:22:25.918887+0000","last_peered":"2023-11-28T11:22:25.918887+0000","last_clean":"2023-11-28T11:22:25.918887+0000","last_became_active":"2023-11-28T11:22:24.234799+0000","last_became_peered":"2023-11-28T11:22:24.234799+0000","last_unstale":"2023-11-28T11:22:25.918887+0000","last_undegraded":"2023-11-28T11:22:25.918887+0000","last_fullsized":"2023-11-28T11:22:25.918887+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:52:14.744757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.105697+0000","last_change":"2023-11-28T11:22:24.752479+0000","last_active":"2023-11-28T11:22:26.105697+0000","last_peered":"2023-11-28T11:22:26.105697+0000","last_clean":"2023-11-28T11:22:26.105697+0000","last_became_active":"2023-11-28T11:22:24.752224+0000","last_became_peered":"2023-11-28T11:22:24.752224+0000","last_unstale":"2023-11-28T11:22:26.105697+0000","last_undegraded":"2023-11-28T11:22:26.105697+0000","last_fullsized":"2023-11-28T11:22:26.105697+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:50:31.355306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"41'1","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.105840+0000","last_change":"2023-11-28T11:22:24.232009+0000","last_active":"2023-11-28T11:22:26.105840+0000","last_peered":"2023-11-28T11:22:26.105840+0000","last_clean":"2023-11-28T11:22:26.105840+0000","last_became_active":"2023-11-28T11:22:24.231755+0000","last_became_peered":"2023-11-28T11:22:24.231755+0000","last_unstale":"2023-11-28T11:22:26.105840+0000","last_undegraded":"2023-11-28T11:22:26.105840+0000","last_fullsized":"2023-11-28T11:22:26.105840+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:28:56.282597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.919227+0000","last_change":"2023-11-28T11:22:24.759542+0000","last_active":"2023-11-28T11:22:25.919227+0000","last_peered":"2023-11-28T11:22:25.919227+0000","last_clean":"2023-11-28T11:22:25.919227+0000","last_became_active":"2023-11-28T11:22:24.759092+0000","last_became_peered":"2023-11-28T11:22:24.759092+0000","last_unstale":"2023-11-28T11:22:25.919227+0000","last_undegraded":"2023-11-28T11:22:25.919227+0000","last_fullsized":"2023-11-28T11:22:25.919227+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:53:29.715029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"41'2","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.734884+0000","last_change":"2023-11-28T11:22:24.233029+0000","last_active":"2023-11-28T11:22:25.734884+0000","last_peered":"2023-11-28T11:22:25.734884+0000","last_clean":"2023-11-28T11:22:25.734884+0000","last_became_active":"2023-11-28T11:22:24.232741+0000","last_became_peered":"2023-11-28T11:22:24.232741+0000","last_unstale":"2023-11-28T11:22:25.734884+0000","last_undegraded":"2023-11-28T11:22:25.734884+0000","last_fullsized":"2023-11-28T11:22:25.734884+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:52:14.086350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.109123+0000","last_change":"2023-11-28T11:22:24.758498+0000","last_active":"2023-11-28T11:22:26.109123+0000","last_peered":"2023-11-28T11:22:26.109123+0000","last_clean":"2023-11-28T11:22:26.109123+0000","last_became_active":"2023-11-28T11:22:24.758206+0000","last_became_peered":"2023-11-28T11:22:24.758206+0000","last_unstale":"2023-11-28T11:22:26.109123+0000","last_undegraded":"2023-11-28T11:22:26.109123+0000","last_fullsized":"2023-11-28T11:22:26.109123+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:43:49.893538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"41'1","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.735350+0000","last_change":"2023-11-28T11:22:24.235752+0000","last_active":"2023-11-28T11:22:25.735350+0000","last_peered":"2023-11-28T11:22:25.735350+0000","last_clean":"2023-11-28T11:22:25.735350+0000","last_became_active":"2023-11-28T11:22:24.235409+0000","last_became_peered":"2023-11-28T11:22:24.235409+0000","last_unstale":"2023-11-28T11:22:25.735350+0000","last_undegraded":"2023-11-28T11:22:25.735350+0000","last_fullsized":"2023-11-28T11:22:25.735350+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:53:28.708246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.109188+0000","last_change":"2023-11-28T11:22:24.746370+0000","last_active":"2023-11-28T11:22:26.109188+0000","last_peered":"2023-11-28T11:22:26.109188+0000","last_clean":"2023-11-28T11:22:26.109188+0000","last_became_active":"2023-11-28T11:22:24.746021+0000","last_became_peered":"2023-11-28T11:22:24.746021+0000","last_unstale":"2023-11-28T11:22:26.109188+0000","last_undegraded":"2023-11-28T11:22:26.109188+0000","last_fullsized":"2023-11-28T11:22:26.109188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:33:45.044082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.106325+0000","last_change":"2023-11-28T11:22:24.241745+0000","last_active":"2023-11-28T11:22:26.106325+0000","last_peered":"2023-11-28T11:22:26.106325+0000","last_clean":"2023-11-28T11:22:26.106325+0000","last_became_active":"2023-11-28T11:22:24.240332+0000","last_became_peered":"2023-11-28T11:22:24.240332+0000","last_unstale":"2023-11-28T11:22:26.106325+0000","last_undegraded":"2023-11-28T11:22:26.106325+0000","last_fullsized":"2023-11-28T11:22:26.106325+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:07:07.702163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.106013+0000","last_change":"2023-11-28T11:22:24.742679+0000","last_active":"2023-11-28T11:22:26.106013+0000","last_peered":"2023-11-28T11:22:26.106013+0000","last_clean":"2023-11-28T11:22:26.106013+0000","last_became_active":"2023-11-28T11:22:24.742154+0000","last_became_peered":"2023-11-28T11:22:24.742154+0000","last_unstale":"2023-11-28T11:22:26.106013+0000","last_undegraded":"2023-11-28T11:22:26.106013+0000","last_fullsized":"2023-11-28T11:22:26.106013+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:24:32.831418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"41'2","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.106350+0000","last_change":"2023-11-28T11:22:24.230819+0000","last_active":"2023-11-28T11:22:26.106350+0000","last_peered":"2023-11-28T11:22:26.106350+0000","last_clean":"2023-11-28T11:22:26.106350+0000","last_became_active":"2023-11-28T11:22:24.230567+0000","last_became_peered":"2023-11-28T11:22:24.230567+0000","last_unstale":"2023-11-28T11:22:26.106350+0000","last_undegraded":"2023-11-28T11:22:26.106350+0000","last_fullsized":"2023-11-28T11:22:26.106350+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:47:54.789341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.731985+0000","last_change":"2023-11-28T11:22:24.745477+0000","last_active":"2023-11-28T11:22:25.731985+0000","last_peered":"2023-11-28T11:22:25.731985+0000","last_clean":"2023-11-28T11:22:25.731985+0000","last_became_active":"2023-11-28T11:22:24.745256+0000","last_became_peered":"2023-11-28T11:22:24.745256+0000","last_unstale":"2023-11-28T11:22:25.731985+0000","last_undegraded":"2023-11-28T11:22:25.731985+0000","last_fullsized":"2023-11-28T11:22:25.731985+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:53:28.365972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.735004+0000","last_change":"2023-11-28T11:22:24.238270+0000","last_active":"2023-11-28T11:22:25.735004+0000","last_peered":"2023-11-28T11:22:25.735004+0000","last_clean":"2023-11-28T11:22:25.735004+0000","last_became_active":"2023-11-28T11:22:24.237922+0000","last_became_peered":"2023-11-28T11:22:24.237922+0000","last_unstale":"2023-11-28T11:22:25.735004+0000","last_undegraded":"2023-11-28T11:22:25.735004+0000","last_fullsized":"2023-11-28T11:22:25.735004+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:54:19.649109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.105623+0000","last_change":"2023-11-28T11:22:24.746580+0000","last_active":"2023-11-28T11:22:26.105623+0000","last_peered":"2023-11-28T11:22:26.105623+0000","last_clean":"2023-11-28T11:22:26.105623+0000","last_became_active":"2023-11-28T11:22:24.745731+0000","last_became_peered":"2023-11-28T11:22:24.745731+0000","last_unstale":"2023-11-28T11:22:26.105623+0000","last_undegraded":"2023-11-28T11:22:26.105623+0000","last_fullsized":"2023-11-28T11:22:26.105623+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:52:15.751540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.918796+0000","last_change":"2023-11-28T11:22:24.238501+0000","last_active":"2023-11-28T11:22:25.918796+0000","last_peered":"2023-11-28T11:22:25.918796+0000","last_clean":"2023-11-28T11:22:25.918796+0000","last_became_active":"2023-11-28T11:22:24.237406+0000","last_became_peered":"2023-11-28T11:22:24.237406+0000","last_unstale":"2023-11-28T11:22:25.918796+0000","last_undegraded":"2023-11-28T11:22:25.918796+0000","last_fullsized":"2023-11-28T11:22:25.918796+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:33:44.037299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.918696+0000","last_change":"2023-11-28T11:22:24.743899+0000","last_active":"2023-11-28T11:22:25.918696+0000","last_peered":"2023-11-28T11:22:25.918696+0000","last_clean":"2023-11-28T11:22:25.918696+0000","last_became_active":"2023-11-28T11:22:24.743645+0000","last_became_peered":"2023-11-28T11:22:24.743645+0000","last_unstale":"2023-11-28T11:22:25.918696+0000","last_undegraded":"2023-11-28T11:22:25.918696+0000","last_fullsized":"2023-11-28T11:22:25.918696+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:54:40.929864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.106233+0000","last_change":"2023-11-28T11:22:24.237071+0000","last_active":"2023-11-28T11:22:26.106233+0000","last_peered":"2023-11-28T11:22:26.106233+0000","last_clean":"2023-11-28T11:22:26.106233+0000","last_became_active":"2023-11-28T11:22:24.236815+0000","last_became_peered":"2023-11-28T11:22:24.236815+0000","last_unstale":"2023-11-28T11:22:26.106233+0000","last_undegraded":"2023-11-28T11:22:26.106233+0000","last_fullsized":"2023-11-28T11:22:26.106233+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:38:22.558425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.734588+0000","last_change":"2023-11-28T11:22:24.755377+0000","last_active":"2023-11-28T11:22:25.734588+0000","last_peered":"2023-11-28T11:22:25.734588+0000","last_clean":"2023-11-28T11:22:25.734588+0000","last_became_active":"2023-11-28T11:22:24.755103+0000","last_became_peered":"2023-11-28T11:22:24.755103+0000","last_unstale":"2023-11-28T11:22:25.734588+0000","last_undegraded":"2023-11-28T11:22:25.734588+0000","last_fullsized":"2023-11-28T11:22:25.734588+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:01:46.793779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.918864+0000","last_change":"2023-11-28T11:22:24.237745+0000","last_active":"2023-11-28T11:22:25.918864+0000","last_peered":"2023-11-28T11:22:25.918864+0000","last_clean":"2023-11-28T11:22:25.918864+0000","last_became_active":"2023-11-28T11:22:24.236614+0000","last_became_peered":"2023-11-28T11:22:24.236614+0000","last_unstale":"2023-11-28T11:22:25.918864+0000","last_undegraded":"2023-11-28T11:22:25.918864+0000","last_fullsized":"2023-11-28T11:22:25.918864+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:39:00.016495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-28T11:22:24.736780+0000","last_change":"2023-11-28T11:20:51.020016+0000","last_active":"2023-11-28T11:22:24.736780+0000","last_peered":"2023-11-28T11:22:24.736780+0000","last_clean":"2023-11-28T11:22:24.736780+0000","last_became_active":"2023-11-28T11:20:51.019492+0000","last_became_peered":"2023-11-28T11:20:51.019492+0000","last_unstale":"2023-11-28T11:22:24.736780+0000","last_undegraded":"2023-11-28T11:22:24.736780+0000","last_fullsized":"2023-11-28T11:22:24.736780+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:27.884538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.109269+0000","last_change":"2023-11-28T11:22:24.757513+0000","last_active":"2023-11-28T11:22:26.109269+0000","last_peered":"2023-11-28T11:22:26.109269+0000","last_clean":"2023-11-28T11:22:26.109269+0000","last_became_active":"2023-11-28T11:22:24.757188+0000","last_became_peered":"2023-11-28T11:22:24.757188+0000","last_unstale":"2023-11-28T11:22:26.109269+0000","last_undegraded":"2023-11-28T11:22:26.109269+0000","last_fullsized":"2023-11-28T11:22:26.109269+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:33:57.540987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"42'1","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.152675+0000","last_change":"2023-11-28T11:22:24.237383+0000","last_active":"2023-11-28T11:22:26.152675+0000","last_peered":"2023-11-28T11:22:26.152675+0000","last_clean":"2023-11-28T11:22:26.152675+0000","last_became_active":"2023-11-28T11:22:24.236832+0000","last_became_peered":"2023-11-28T11:22:24.236832+0000","last_unstale":"2023-11-28T11:22:26.152675+0000","last_undegraded":"2023-11-28T11:22:26.152675+0000","last_fullsized":"2023-11-28T11:22:26.152675+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:00:52.219069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":31,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-28T11:22:24.735402+0000","last_change":"2023-11-28T11:20:50.521908+0000","last_active":"2023-11-28T11:22:24.735402+0000","last_peered":"2023-11-28T11:22:24.735402+0000","last_clean":"2023-11-28T11:22:24.735402+0000","last_became_active":"2023-11-28T11:20:50.521059+0000","last_became_peered":"2023-11-28T11:20:50.521059+0000","last_unstale":"2023-11-28T11:22:24.735402+0000","last_undegraded":"2023-11-28T11:22:24.735402+0000","last_fullsized":"2023-11-28T11:22:24.735402+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:05.302498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.918998+0000","last_change":"2023-11-28T11:22:24.748517+0000","last_active":"2023-11-28T11:22:25.918998+0000","last_peered":"2023-11-28T11:22:25.918998+0000","last_clean":"2023-11-28T11:22:25.918998+0000","last_became_active":"2023-11-28T11:22:24.748232+0000","last_became_peered":"2023-11-28T11:22:24.748232+0000","last_unstale":"2023-11-28T11:22:25.918998+0000","last_undegraded":"2023-11-28T11:22:25.918998+0000","last_fullsized":"2023-11-28T11:22:25.918998+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:18:49.637085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.106892+0000","last_change":"2023-11-28T11:22:24.244306+0000","last_active":"2023-11-28T11:22:26.106892+0000","last_peered":"2023-11-28T11:22:26.106892+0000","last_clean":"2023-11-28T11:22:26.106892+0000","last_became_active":"2023-11-28T11:22:24.243991+0000","last_became_peered":"2023-11-28T11:22:24.243991+0000","last_unstale":"2023-11-28T11:22:26.106892+0000","last_undegraded":"2023-11-28T11:22:26.106892+0000","last_fullsized":"2023-11-28T11:22:26.106892+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:31:15.724188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":29,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-28T11:22:24.734488+0000","last_change":"2023-11-28T11:20:50.619004+0000","last_active":"2023-11-28T11:22:24.734488+0000","last_peered":"2023-11-28T11:22:24.734488+0000","last_clean":"2023-11-28T11:22:24.734488+0000","last_became_active":"2023-11-28T11:20:50.618315+0000","last_became_peered":"2023-11-28T11:20:50.618315+0000","last_unstale":"2023-11-28T11:22:24.734488+0000","last_undegraded":"2023-11-28T11:22:24.734488+0000","last_fullsized":"2023-11-28T11:22:24.734488+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:27.884538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.919233+0000","last_change":"2023-11-28T11:22:24.747406+0000","last_active":"2023-11-28T11:22:25.919233+0000","last_peered":"2023-11-28T11:22:25.919233+0000","last_clean":"2023-11-28T11:22:25.919233+0000","last_became_active":"2023-11-28T11:22:24.747147+0000","last_became_peered":"2023-11-28T11:22:24.747147+0000","last_unstale":"2023-11-28T11:22:25.919233+0000","last_undegraded":"2023-11-28T11:22:25.919233+0000","last_fullsized":"2023-11-28T11:22:25.919233+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:47:55.796123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"41'2","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.735436+0000","last_change":"2023-11-28T11:22:24.229941+0000","last_active":"2023-11-28T11:22:25.735436+0000","last_peered":"2023-11-28T11:22:25.735436+0000","last_clean":"2023-11-28T11:22:25.735436+0000","last_became_active":"2023-11-28T11:22:24.229498+0000","last_became_peered":"2023-11-28T11:22:24.229498+0000","last_unstale":"2023-11-28T11:22:25.735436+0000","last_undegraded":"2023-11-28T11:22:25.735436+0000","last_fullsized":"2023-11-28T11:22:25.735436+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:11:40.365836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":31,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-28T11:22:24.735393+0000","last_change":"2023-11-28T11:20:50.522582+0000","last_active":"2023-11-28T11:22:24.735393+0000","last_peered":"2023-11-28T11:22:24.735393+0000","last_clean":"2023-11-28T11:22:24.735393+0000","last_became_active":"2023-11-28T11:20:50.522157+0000","last_became_peered":"2023-11-28T11:20:50.522157+0000","last_unstale":"2023-11-28T11:22:24.735393+0000","last_undegraded":"2023-11-28T11:22:24.735393+0000","last_fullsized":"2023-11-28T11:22:24.735393+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:27.884538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":183,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-28T11:22:24.722564+0000","last_change":"2023-11-28T11:20:43.294845+0000","last_active":"2023-11-28T11:22:24.722564+0000","last_peered":"2023-11-28T11:22:24.722564+0000","last_clean":"2023-11-28T11:22:24.722564+0000","last_became_active":"2023-11-28T11:20:43.294483+0000","last_became_peered":"2023-11-28T11:20:43.294483+0000","last_unstale":"2023-11-28T11:22:24.722564+0000","last_undegraded":"2023-11-28T11:22:24.722564+0000","last_fullsized":"2023-11-28T11:22:24.722564+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:42.270512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:42.270512+0000","last_clean_scrub_stamp":"2023-11-28T11:20:42.270512+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:17:03.376680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.106779+0000","last_change":"2023-11-28T11:22:24.741930+0000","last_active":"2023-11-28T11:22:26.106779+0000","last_peered":"2023-11-28T11:22:26.106779+0000","last_clean":"2023-11-28T11:22:26.106779+0000","last_became_active":"2023-11-28T11:22:24.741663+0000","last_became_peered":"2023-11-28T11:22:24.741663+0000","last_unstale":"2023-11-28T11:22:26.106779+0000","last_undegraded":"2023-11-28T11:22:26.106779+0000","last_fullsized":"2023-11-28T11:22:26.106779+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:40:41.077032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.731802+0000","last_change":"2023-11-28T11:22:24.241309+0000","last_active":"2023-11-28T11:22:25.731802+0000","last_peered":"2023-11-28T11:22:25.731802+0000","last_clean":"2023-11-28T11:22:25.731802+0000","last_became_active":"2023-11-28T11:22:24.241016+0000","last_became_peered":"2023-11-28T11:22:24.241016+0000","last_unstale":"2023-11-28T11:22:25.731802+0000","last_undegraded":"2023-11-28T11:22:25.731802+0000","last_fullsized":"2023-11-28T11:22:25.731802+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:50:30.348523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-28T11:22:24.722717+0000","last_change":"2023-11-28T11:20:50.526865+0000","last_active":"2023-11-28T11:22:24.722717+0000","last_peered":"2023-11-28T11:22:24.722717+0000","last_clean":"2023-11-28T11:22:24.722717+0000","last_became_active":"2023-11-28T11:20:50.526576+0000","last_became_peered":"2023-11-28T11:20:50.526576+0000","last_unstale":"2023-11-28T11:22:24.722717+0000","last_undegraded":"2023-11-28T11:22:24.722717+0000","last_fullsized":"2023-11-28T11:22:24.722717+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:09:18.460112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.734797+0000","last_change":"2023-11-28T11:22:24.748680+0000","last_active":"2023-11-28T11:22:25.734797+0000","last_peered":"2023-11-28T11:22:25.734797+0000","last_clean":"2023-11-28T11:22:25.734797+0000","last_became_active":"2023-11-28T11:22:24.746760+0000","last_became_peered":"2023-11-28T11:22:24.746760+0000","last_unstale":"2023-11-28T11:22:25.734797+0000","last_undegraded":"2023-11-28T11:22:25.734797+0000","last_fullsized":"2023-11-28T11:22:25.734797+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:07:08.708946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.732022+0000","last_change":"2023-11-28T11:22:24.239331+0000","last_active":"2023-11-28T11:22:25.732022+0000","last_peered":"2023-11-28T11:22:25.732022+0000","last_clean":"2023-11-28T11:22:25.732022+0000","last_became_active":"2023-11-28T11:22:24.239084+0000","last_became_peered":"2023-11-28T11:22:24.239084+0000","last_unstale":"2023-11-28T11:22:25.732022+0000","last_undegraded":"2023-11-28T11:22:25.732022+0000","last_fullsized":"2023-11-28T11:22:25.732022+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:31:15.724188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-28T11:22:24.723578+0000","last_change":"2023-11-28T11:20:51.020043+0000","last_active":"2023-11-28T11:22:24.723578+0000","last_peered":"2023-11-28T11:22:24.723578+0000","last_clean":"2023-11-28T11:22:24.723578+0000","last_became_active":"2023-11-28T11:20:51.019702+0000","last_became_peered":"2023-11-28T11:20:51.019702+0000","last_unstale":"2023-11-28T11:22:24.723578+0000","last_undegraded":"2023-11-28T11:22:24.723578+0000","last_fullsized":"2023-11-28T11:22:24.723578+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:26:20.246873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.106258+0000","last_change":"2023-11-28T11:22:24.759524+0000","last_active":"2023-11-28T11:22:26.106258+0000","last_peered":"2023-11-28T11:22:26.106258+0000","last_clean":"2023-11-28T11:22:26.106258+0000","last_became_active":"2023-11-28T11:22:24.757893+0000","last_became_peered":"2023-11-28T11:22:24.757893+0000","last_unstale":"2023-11-28T11:22:26.106258+0000","last_undegraded":"2023-11-28T11:22:26.106258+0000","last_fullsized":"2023-11-28T11:22:26.106258+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:12:27.357618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:25.734854+0000","last_change":"2023-11-28T11:22:24.237362+0000","last_active":"2023-11-28T11:22:25.734854+0000","last_peered":"2023-11-28T11:22:25.734854+0000","last_clean":"2023-11-28T11:22:25.734854+0000","last_became_active":"2023-11-28T11:22:24.236840+0000","last_became_peered":"2023-11-28T11:22:24.236840+0000","last_unstale":"2023-11-28T11:22:25.734854+0000","last_undegraded":"2023-11-28T11:22:25.734854+0000","last_fullsized":"2023-11-28T11:22:25.734854+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:40:40.070249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-28T11:22:24.736887+0000","last_change":"2023-11-28T11:20:50.524251+0000","last_active":"2023-11-28T11:22:24.736887+0000","last_peered":"2023-11-28T11:22:24.736887+0000","last_clean":"2023-11-28T11:22:24.736887+0000","last_became_active":"2023-11-28T11:20:50.524012+0000","last_became_peered":"2023-11-28T11:20:50.524012+0000","last_unstale":"2023-11-28T11:22:24.736887+0000","last_undegraded":"2023-11-28T11:22:24.736887+0000","last_fullsized":"2023-11-28T11:22:24.736887+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:05.302498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.106071+0000","last_change":"2023-11-28T11:22:24.750518+0000","last_active":"2023-11-28T11:22:26.106071+0000","last_peered":"2023-11-28T11:22:26.106071+0000","last_clean":"2023-11-28T11:22:26.106071+0000","last_became_active":"2023-11-28T11:22:24.750252+0000","last_became_peered":"2023-11-28T11:22:24.750252+0000","last_unstale":"2023-11-28T11:22:26.106071+0000","last_undegraded":"2023-11-28T11:22:26.106071+0000","last_fullsized":"2023-11-28T11:22:26.106071+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:24.209503+0000","last_clean_scrub_stamp":"2023-11-28T11:22:24.209503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:35:10.458383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-28T11:22:26.106738+0000","last_change":"2023-11-28T11:22:24.238373+0000","last_active":"2023-11-28T11:22:26.106738+0000","last_peered":"2023-11-28T11:22:26.106738+0000","last_clean":"2023-11-28T11:22:26.106738+0000","last_became_active":"2023-11-28T11:22:24.236466+0000","last_became_peered":"2023-11-28T11:22:24.236466+0000","last_unstale":"2023-11-28T11:22:26.106738+0000","last_undegraded":"2023-11-28T11:22:26.106738+0000","last_fullsized":"2023-11-28T11:22:26.106738+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:22:23.202720+0000","last_clean_scrub_stamp":"2023-11-28T11:22:23.202720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:52:14.744757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-11-28T11:22:24.742195+0000","last_change":"2023-11-28T11:20:50.524838+0000","last_active":"2023-11-28T11:22:24.742195+0000","last_peered":"2023-11-28T11:22:24.742195+0000","last_clean":"2023-11-28T11:22:24.742195+0000","last_became_active":"2023-11-28T11:20:50.524402+0000","last_became_peered":"2023-11-28T11:20:50.524402+0000","last_unstale":"2023-11-28T11:22:24.742195+0000","last_undegraded":"2023-11-28T11:22:24.742195+0000","last_fullsized":"2023-11-28T11:22:24.742195+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:22.699447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9832,"kb_used_data":1056,"kb_used_omap":0,"kb_used_meta":8768,"kb_avail":94362008,"statfs":{"total":96636764160,"available":96626696192,"internally_reserved":0,"allocated":1081344,"data_stored":808612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8978432},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.66500000000000004}]},{"osd":1,"last update":"Tue Nov 28 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.61599999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.68100000000000005}]},{"osd":3,"last update":"Tue Nov 28 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.71799999999999997}]},{"osd":4,"last update":"Tue Nov 28 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.69799999999999995}]},{"osd":5,"last update":"Tue Nov 28 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.56599999999999995}]},{"osd":6,"last update":"Tue Nov 28 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.59899999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8620,"kb_used_data":484,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363220,"statfs":{"total":96636764160,"available":96627937280,"internally_reserved":0,"allocated":495616,"data_stored":219432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.49299999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.434}]},{"osd":2,"last update":"Tue Nov 28 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.996,"5min":1.996,"15min":1.996},"last":0.68100000000000005}]},{"osd":3,"last update":"Tue Nov 28 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.622},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.45400000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":2.048,"5min":2.048,"15min":2.048},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.57399999999999995}]},{"osd":5,"last update":"Tue Nov 28 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":2.1000000000000001,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":0.70799999999999996}]},{"osd":7,"last update":"Tue Nov 28 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.56799999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8744,"kb_used_data":480,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363096,"statfs":{"total":96636764160,"available":96627810304,"internally_reserved":0,"allocated":491520,"data_stored":218846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.57899999999999996}]},{"osd":2,"last update":"Tue Nov 28 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.378},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.33700000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.497}]},{"osd":4,"last update":"Tue Nov 28 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.81599999999999995}]},{"osd":5,"last update":"Tue Nov 28 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.82799999999999996}]},{"osd":6,"last update":"Tue Nov 28 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.70399999999999996}]},{"osd":7,"last update":"Tue Nov 28 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.754}]}]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9836,"kb_used_data":1060,"kb_used_omap":0,"kb_used_meta":8768,"kb_avail":94362004,"statfs":{"total":96636764160,"available":96626692096,"internally_reserved":0,"allocated":1085440,"data_stored":809178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8978432},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:21:41 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":3.8420000000000001,"5min":3.8420000000000001,"15min":3.8420000000000001},"last":0.33600000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:21:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":3.8940000000000001,"5min":3.8940000000000001,"15min":3.8940000000000001},"last":0.38100000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:21:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":3.681,"5min":3.681,"15min":3.681},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.434}]},{"osd":4,"last update":"Tue Nov 28 11:21:41 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":3.9790000000000001,"5min":3.9790000000000001,"15min":3.9790000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":3.5779999999999998,"5min":3.5779999999999998,"15min":3.5779999999999998},"last":0.75700000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:21:41 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.998,"5min":0.998,"15min":0.998},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":3.9409999999999998,"5min":3.9409999999999998,"15min":3.9409999999999998},"last":0.74399999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:21:41 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":3.6160000000000001,"5min":3.6160000000000001,"15min":3.6160000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.79100000000000004}]},{"osd":7,"last update":"Tue Nov 28 11:21:41 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.077,"15min":1.077},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":4.0099999999999998,"5min":4.0099999999999998,"15min":4.0099999999999998},"last":0.80600000000000005},{"interface":"front","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":3.653,"5min":3.653,"15min":3.653},"last":0.73499999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8564,"kb_used_data":492,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363276,"statfs":{"total":96636764160,"available":96627994624,"internally_reserved":0,"allocated":503808,"data_stored":220346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":4.1109999999999998,"5min":4.1109999999999998,"15min":4.1109999999999998},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":3.6230000000000002,"5min":3.6230000000000002,"15min":3.6230000000000002},"last":0.315}]},{"osd":1,"last update":"Tue Nov 28 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":3.6640000000000001,"5min":3.6640000000000001,"15min":3.6640000000000001},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":4.0830000000000002,"5min":4.0830000000000002,"15min":4.0830000000000002},"last":0.40899999999999997}]},{"osd":3,"last update":"Tue Nov 28 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":4.1529999999999996,"5min":4.1529999999999996,"15min":4.1529999999999996},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.42299999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.649,"5min":1.649,"15min":1.649},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":4.0060000000000002,"5min":4.0060000000000002,"15min":4.0060000000000002},"last":0.74199999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":4.0529999999999999,"5min":4.0529999999999999,"15min":4.0529999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.71699999999999997}]},{"osd":6,"last update":"Tue Nov 28 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.75700000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.70699999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8740,"kb_used_data":476,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363100,"statfs":{"total":96636764160,"available":96627814400,"internally_reserved":0,"allocated":487424,"data_stored":212425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.57999999999999996}]},{"osd":1,"last update":"Tue Nov 28 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":4.8019999999999996,"5min":4.8019999999999996,"15min":4.8019999999999996},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":4.7539999999999996,"5min":4.7539999999999996,"15min":4.7539999999999996},"last":0.63800000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":4.8419999999999996,"5min":4.8419999999999996,"15min":4.8419999999999996},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":4.7919999999999998,"5min":4.7919999999999998,"15min":4.7919999999999998},"last":0.56200000000000006}]},{"osd":4,"last update":"Tue Nov 28 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":4.8869999999999996,"5min":4.8869999999999996,"15min":4.8869999999999996},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":5.2460000000000004,"5min":5.2460000000000004,"15min":5.2460000000000004},"last":0.752}]},{"osd":5,"last update":"Tue Nov 28 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":5.0759999999999996,"5min":5.0759999999999996,"15min":5.0759999999999996},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":5.3010000000000002,"5min":5.3010000000000002,"15min":5.3010000000000002},"last":0.68000000000000005}]},{"osd":6,"last update":"Tue Nov 28 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":5.3399999999999999,"5min":5.3399999999999999,"15min":5.3399999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":4.8330000000000002,"5min":4.8330000000000002,"15min":4.8330000000000002},"last":0.622}]},{"osd":7,"last update":"Tue Nov 28 11:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":4.8600000000000003,"5min":4.8600000000000003,"15min":4.8600000000000003},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":5.1909999999999998,"5min":5.1909999999999998,"15min":5.1909999999999998},"last":0.66700000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8816,"kb_used_data":488,"kb_used_omap":0,"kb_used_meta":8320,"kb_avail":94363024,"statfs":{"total":96636764160,"available":96627736576,"internally_reserved":0,"allocated":499712,"data_stored":220324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8519680},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.66900000000000004}]},{"osd":1,"last update":"Tue Nov 28 11:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.78900000000000003}]},{"osd":2,"last update":"Tue Nov 28 11:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.84199999999999997}]},{"osd":3,"last update":"Tue Nov 28 11:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.97399999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.81499999999999995}]},{"osd":6,"last update":"Tue Nov 28 11:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.95899999999999996}]},{"osd":7,"last update":"Tue Nov 28 11:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":2.1960000000000002,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":1.0149999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8680,"kb_used_data":480,"kb_used_omap":0,"kb_used_meta":8192,"kb_avail":94363160,"statfs":{"total":96636764160,"available":96627875840,"internally_reserved":0,"allocated":491520,"data_stored":218846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8388608},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:21:44 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.495}]},{"osd":1,"last update":"Tue Nov 28 11:21:44 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.59399999999999997}]},{"osd":2,"last update":"Tue Nov 28 11:21:44 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.71299999999999997}]},{"osd":3,"last update":"Tue Nov 28 11:21:44 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.81299999999999994}]},{"osd":4,"last update":"Tue Nov 28 11:21:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.73599999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:21:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.68000000000000005}]},{"osd":7,"last update":"Tue Nov 28 11:21:44 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.85399999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:22:29.562 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T11:22:29.562 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2023-11-28T11:22:29.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.800+0000 7f66d01ff700 1 Processor -- start 2023-11-28T11:22:29.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.800+0000 7f66d01ff700 1 -- start start 2023-11-28T11:22:29.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.801+0000 7f66d01ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66c815d590 0x7f66c80b2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:29.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.802+0000 7f66d01ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66c815d9e0 con 0x7f66c815dee0 2023-11-28T11:22:29.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.802+0000 7f66d01ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66c80cd4d0 con 0x7f66c80d2e10 2023-11-28T11:22:29.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.802+0000 7f66d01ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66c80cdce0 con 0x7f66c815d590 2023-11-28T11:22:29.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.802+0000 7f66ce79c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f66c815dee0 0x7f66c80cf900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:41118/0 (socket says 172.21.15.92:41118) 2023-11-28T11:22:29.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.802+0000 7f66ce79c700 1 -- 172.21.15.92:0/1687450891 learned_addr learned my addr 172.21.15.92:0/1687450891 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:22:29.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.802+0000 7f66cd79a700 1 --2- 172.21.15.92:0/1687450891 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66c815d590 0x7f66c80b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:29.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.802+0000 7f66ccf99700 1 -- 172.21.15.92:0/1687450891 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1064994773 0 0) 0x7f66c0001490 con 0x7f66c815dee0 2023-11-28T11:22:29.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.802+0000 7f66ccf99700 1 -- 172.21.15.92:0/1687450891 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66b40035b0 con 0x7f66c815dee0 2023-11-28T11:22:29.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.802+0000 7f66ccf99700 1 -- 172.21.15.92:0/1687450891 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 179734083 0 0) 0x7f66c80cd4d0 con 0x7f66c80d2e10 2023-11-28T11:22:29.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.802+0000 7f66ccf99700 1 -- 172.21.15.92:0/1687450891 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66c0001490 con 0x7f66c80d2e10 2023-11-28T11:22:29.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.803+0000 7f66cd79a700 1 -- 172.21.15.92:0/1687450891 >> v1:172.21.15.169:6789/0 conn(0x7f66c80d2e10 legacy=0x7f66c80d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:29.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.803+0000 7f66cd79a700 1 -- 172.21.15.92:0/1687450891 >> v1:172.21.15.92:6789/0 conn(0x7f66c815dee0 legacy=0x7f66c80cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:29.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.803+0000 7f66cd79a700 1 -- 172.21.15.92:0/1687450891 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c80cf470 con 0x7f66c815d590 2023-11-28T11:22:29.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.803+0000 7f66cd79a700 1 --2- 172.21.15.92:0/1687450891 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66c815d590 0x7f66c80b2280 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f66b8009870 tx=0x7f66b8009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3355d8d244c51c4e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:29.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.803+0000 7f66ccf99700 1 -- 172.21.15.92:0/1687450891 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b8011830 con 0x7f66c815d590 2023-11-28T11:22:29.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.803+0000 7f66ccf99700 1 -- 172.21.15.92:0/1687450891 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66b8011990 con 0x7f66c815d590 2023-11-28T11:22:29.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.803+0000 7f66ccf99700 1 -- 172.21.15.92:0/1687450891 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b8011d20 con 0x7f66c815d590 2023-11-28T11:22:29.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.804+0000 7f66d01ff700 1 -- 172.21.15.92:0/1687450891 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66c815d590 msgr2=0x7f66c80b2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:29.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.804+0000 7f66d01ff700 1 --2- 172.21.15.92:0/1687450891 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66c815d590 0x7f66c80b2280 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f66b8009870 tx=0x7f66b8009b80 comp rx=0 tx=0).stop 2023-11-28T11:22:29.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.804+0000 7f66d01ff700 1 -- 172.21.15.92:0/1687450891 shutdown_connections 2023-11-28T11:22:29.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.804+0000 7f66d01ff700 1 --2- 172.21.15.92:0/1687450891 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66c815d590 0x7f66c80b2280 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:29.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.804+0000 7f66d01ff700 1 -- 172.21.15.92:0/1687450891 >> 172.21.15.92:0/1687450891 conn(0x7f66c80719a0 msgr2=0x7f66c8071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:29.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.804+0000 7f66d01ff700 1 -- 172.21.15.92:0/1687450891 shutdown_connections 2023-11-28T11:22:29.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.804+0000 7f66d01ff700 1 -- 172.21.15.92:0/1687450891 wait complete. 2023-11-28T11:22:29.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.806+0000 7f66d01ff700 1 Processor -- start 2023-11-28T11:22:29.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.806+0000 7f66d01ff700 1 -- start start 2023-11-28T11:22:29.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.807+0000 7f66d01ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66c80d2e10 0x7f66c80c7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:29.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.807+0000 7f66d01ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66c815d590 0x7f66c814b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:29.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.807+0000 7f66d01ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66c815dee0 0x7f66c815ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:29.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.807+0000 7f66d01ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c80c76b0 con 0x7f66c80d2e10 2023-11-28T11:22:29.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.807+0000 7f66d01ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c80c6aa0 con 0x7f66c815d590 2023-11-28T11:22:29.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.807+0000 7f66d01ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66c80c6d80 con 0x7f66c815dee0 2023-11-28T11:22:29.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.807+0000 7f66cdf9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66c80d2e10 0x7f66c80c7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:29.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.807+0000 7f66cdf9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66c80d2e10 0x7f66c80c7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55964/0 (socket says 172.21.15.92:55964) 2023-11-28T11:22:29.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.807+0000 7f66cdf9b700 1 -- 172.21.15.92:0/916429145 learned_addr learned my addr 172.21.15.92:0/916429145 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:29.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.807+0000 7f66ce79c700 1 --2- 172.21.15.92:0/916429145 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66c815dee0 0x7f66c815ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:29.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.807+0000 7f66cd79a700 1 --2- 172.21.15.92:0/916429145 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66c815d590 0x7f66c814b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:29.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.807+0000 7f66cdf9b700 1 -- 172.21.15.92:0/916429145 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66c815dee0 msgr2=0x7f66c815ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:29.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.807+0000 7f66cdf9b700 1 --2- 172.21.15.92:0/916429145 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66c815dee0 0x7f66c815ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:29.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.807+0000 7f66cdf9b700 1 -- 172.21.15.92:0/916429145 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66c815d590 msgr2=0x7f66c814b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:29.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.807+0000 7f66cdf9b700 1 --2- 172.21.15.92:0/916429145 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66c815d590 0x7f66c814b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:29.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.807+0000 7f66cdf9b700 1 -- 172.21.15.92:0/916429145 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b80094a0 con 0x7f66c80d2e10 2023-11-28T11:22:29.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.808+0000 7f66cdf9b700 1 --2- 172.21.15.92:0/916429145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66c80d2e10 0x7f66c80c7b40 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f66c400c520 tx=0x7f66c400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:29.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.808+0000 7f66be7fc700 1 -- 172.21.15.92:0/916429145 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c40052d0 con 0x7f66c80d2e10 2023-11-28T11:22:29.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.808+0000 7f66be7fc700 1 -- 172.21.15.92:0/916429145 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66c4005430 con 0x7f66c80d2e10 2023-11-28T11:22:29.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.808+0000 7f66d01ff700 1 -- 172.21.15.92:0/916429145 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66c80cd720 con 0x7f66c80d2e10 2023-11-28T11:22:29.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.808+0000 7f66be7fc700 1 -- 172.21.15.92:0/916429145 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c40038c0 con 0x7f66c80d2e10 2023-11-28T11:22:29.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.808+0000 7f66d01ff700 1 -- 172.21.15.92:0/916429145 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66c8156fa0 con 0x7f66c80d2e10 2023-11-28T11:22:29.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.809+0000 7f66d01ff700 1 -- 172.21.15.92:0/916429145 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66c80cda00 con 0x7f66c80d2e10 2023-11-28T11:22:29.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.810+0000 7f66be7fc700 1 -- 172.21.15.92:0/916429145 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f66c4019030 con 0x7f66c80d2e10 2023-11-28T11:22:29.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.811+0000 7f66be7fc700 1 --2- 172.21.15.92:0/916429145 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f66ac0682f0 0x7f66ac06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:29.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.811+0000 7f66be7fc700 1 -- 172.21.15.92:0/916429145 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f66c40851d0 con 0x7f66c80d2e10 2023-11-28T11:22:29.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.811+0000 7f66cd79a700 1 --2- 172.21.15.92:0/916429145 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f66ac0682f0 0x7f66ac06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:29.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.811+0000 7f66cd79a700 1 --2- 172.21.15.92:0/916429145 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f66ac0682f0 0x7f66ac06a7a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f66b80097c0 tx=0x7f66b801c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:29.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:29.815+0000 7f66be7fc700 1 -- 172.21.15.92:0/916429145 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f66c4051800 con 0x7f66c80d2e10 2023-11-28T11:22:30.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.081+0000 7f66d01ff700 1 -- 172.21.15.92:0/916429145 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7f66c80c7990 con 0x7f66c80d2e10 2023-11-28T11:22:30.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.085+0000 7f66be7fc700 1 -- 172.21.15.92:0/916429145 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 184+0+0 (secure 0 0 0) 0x7f66c4054e20 con 0x7f66c80d2e10 2023-11-28T11:22:30.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.087+0000 7f66d01ff700 1 -- 172.21.15.92:0/916429145 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f66ac0682f0 msgr2=0x7f66ac06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:30.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.087+0000 7f66d01ff700 1 --2- 172.21.15.92:0/916429145 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f66ac0682f0 0x7f66ac06a7a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f66b80097c0 tx=0x7f66b801c040 comp rx=0 tx=0).stop 2023-11-28T11:22:30.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.087+0000 7f66d01ff700 1 -- 172.21.15.92:0/916429145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66c80d2e10 msgr2=0x7f66c80c7b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:30.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.087+0000 7f66d01ff700 1 --2- 172.21.15.92:0/916429145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66c80d2e10 0x7f66c80c7b40 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f66c400c520 tx=0x7f66c400c830 comp rx=0 tx=0).stop 2023-11-28T11:22:30.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.087+0000 7f66d01ff700 1 -- 172.21.15.92:0/916429145 shutdown_connections 2023-11-28T11:22:30.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.088+0000 7f66d01ff700 1 --2- 172.21.15.92:0/916429145 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66c815dee0 0x7f66c815ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:30.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.088+0000 7f66d01ff700 1 --2- 172.21.15.92:0/916429145 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f66ac0682f0 0x7f66ac06a7a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:30.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.088+0000 7f66d01ff700 1 --2- 172.21.15.92:0/916429145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66c80d2e10 0x7f66c80c7b40 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:30.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.088+0000 7f66d01ff700 1 --2- 172.21.15.92:0/916429145 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66c815d590 0x7f66c814b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:30.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.088+0000 7f66d01ff700 1 -- 172.21.15.92:0/916429145 >> 172.21.15.92:0/916429145 conn(0x7f66c80719a0 msgr2=0x7f66c815c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:30.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.088+0000 7f66d01ff700 1 -- 172.21.15.92:0/916429145 shutdown_connections 2023-11-28T11:22:30.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.088+0000 7f66d01ff700 1 -- 172.21.15.92:0/916429145 wait complete. 2023-11-28T11:22:30.091 INFO:teuthology.orchestra.run.smithi092.stderr:updated caps for client.0 2023-11-28T11:22:30.108 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:22:30.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.301+0000 7f39df9ff700 1 Processor -- start 2023-11-28T11:22:30.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.301+0000 7f39df9ff700 1 -- start start 2023-11-28T11:22:30.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.302+0000 7f39df9ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f39d80b8510 0x7f39d806b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:30.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.302+0000 7f39df9ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f39d80cd460 0x7f39d806beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:30.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.303+0000 7f39df9ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39d80b88e0 con 0x7f39d806c3f0 2023-11-28T11:22:30.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.303+0000 7f39df9ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39d80cdbd0 con 0x7f39d80b8510 2023-11-28T11:22:30.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.303+0000 7f39df9ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39d80d02d0 con 0x7f39d80cd460 2023-11-28T11:22:30.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.303+0000 7f39dd79b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f39d80b8510 0x7f39d806b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:30.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.303+0000 7f39dd79b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f39d80b8510 0x7f39d806b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55970/0 (socket says 172.21.15.92:55970) 2023-11-28T11:22:30.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.303+0000 7f39dd79b700 1 -- 172.21.15.92:0/2863976675 learned_addr learned my addr 172.21.15.92:0/2863976675 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:30.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.303+0000 7f39dcf9a700 1 --2- 172.21.15.92:0/2863976675 >> v2:172.21.15.169:3300/0 conn(0x7f39d80cd460 0x7f39d806beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:30.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.303+0000 7f39dd79b700 1 -- 172.21.15.92:0/2863976675 >> v2:172.21.15.169:3300/0 conn(0x7f39d80cd460 msgr2=0x7f39d806beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:30.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.303+0000 7f39dd79b700 1 --2- 172.21.15.92:0/2863976675 >> v2:172.21.15.169:3300/0 conn(0x7f39d80cd460 0x7f39d806beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:30.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.303+0000 7f39dd79b700 1 -- 172.21.15.92:0/2863976675 >> v1:172.21.15.169:6789/0 conn(0x7f39d806c3f0 legacy=0x7f39d815cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:30.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.303+0000 7f39dd79b700 1 -- 172.21.15.92:0/2863976675 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39d80cd8f0 con 0x7f39d80b8510 2023-11-28T11:22:30.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.303+0000 7f39dcf9a700 1 --2- 172.21.15.92:0/2863976675 >> v2:172.21.15.169:3300/0 conn(0x7f39d80cd460 0x7f39d806beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:22:30.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.304+0000 7f39dd79b700 1 --2- 172.21.15.92:0/2863976675 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39d80b8510 0x7f39d806b970 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f39d4009920 tx=0x7f39d4009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94f78f92be0a1dd7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:30.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.304+0000 7f39cffff700 1 -- 172.21.15.92:0/2863976675 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d400b9e0 con 0x7f39d80b8510 2023-11-28T11:22:30.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.304+0000 7f39cffff700 1 -- 172.21.15.92:0/2863976675 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39d400bb40 con 0x7f39d80b8510 2023-11-28T11:22:30.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.304+0000 7f39cffff700 1 -- 172.21.15.92:0/2863976675 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d4009e70 con 0x7f39d80b8510 2023-11-28T11:22:30.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.304+0000 7f39df9ff700 1 -- 172.21.15.92:0/2863976675 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39d80b8510 msgr2=0x7f39d806b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:30.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.304+0000 7f39df9ff700 1 --2- 172.21.15.92:0/2863976675 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39d80b8510 0x7f39d806b970 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f39d4009920 tx=0x7f39d4009c30 comp rx=0 tx=0).stop 2023-11-28T11:22:30.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.304+0000 7f39df9ff700 1 -- 172.21.15.92:0/2863976675 shutdown_connections 2023-11-28T11:22:30.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.304+0000 7f39df9ff700 1 --2- 172.21.15.92:0/2863976675 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39d80b8510 0x7f39d806b970 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:30.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.304+0000 7f39df9ff700 1 --2- 172.21.15.92:0/2863976675 >> v2:172.21.15.169:3300/0 conn(0x7f39d80cd460 0x7f39d806beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:30.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.304+0000 7f39df9ff700 1 -- 172.21.15.92:0/2863976675 >> 172.21.15.92:0/2863976675 conn(0x7f39d80719a0 msgr2=0x7f39d8071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:30.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.304+0000 7f39df9ff700 1 -- 172.21.15.92:0/2863976675 shutdown_connections 2023-11-28T11:22:30.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.304+0000 7f39df9ff700 1 -- 172.21.15.92:0/2863976675 wait complete. 2023-11-28T11:22:30.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.306+0000 7f39df9ff700 1 Processor -- start 2023-11-28T11:22:30.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.306+0000 7f39df9ff700 1 -- start start 2023-11-28T11:22:30.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.307+0000 7f39df9ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39d806c3f0 0x7f39d81553f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:30.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.307+0000 7f39df9ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f39d80b8510 0x7f39d81f2f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:30.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.307+0000 7f39df9ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39d80cd460 0x7f39d80aab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:30.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.307+0000 7f39df9ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d8155930 con 0x7f39d806c3f0 2023-11-28T11:22:30.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.307+0000 7f39df9ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d80a9840 con 0x7f39d80cd460 2023-11-28T11:22:30.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.307+0000 7f39df9ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39d81f34d0 con 0x7f39d80b8510 2023-11-28T11:22:30.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.307+0000 7f39dd79b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39d806c3f0 0x7f39d81553f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:30.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.307+0000 7f39dd79b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39d806c3f0 0x7f39d81553f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55972/0 (socket says 172.21.15.92:55972) 2023-11-28T11:22:30.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.307+0000 7f39dd79b700 1 -- 172.21.15.92:0/346027150 learned_addr learned my addr 172.21.15.92:0/346027150 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:30.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.308+0000 7f39dd79b700 1 -- 172.21.15.92:0/346027150 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f39d80b8510 msgr2=0x7f39d81f2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:30.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.308+0000 7f39dd79b700 1 --2- 172.21.15.92:0/346027150 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f39d80b8510 0x7f39d81f2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:30.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.308+0000 7f39dd79b700 1 -- 172.21.15.92:0/346027150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39d80cd460 msgr2=0x7f39d80aab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:30.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.308+0000 7f39dd79b700 1 --2- 172.21.15.92:0/346027150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39d80cd460 0x7f39d80aab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:30.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.308+0000 7f39dd79b700 1 -- 172.21.15.92:0/346027150 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39d4009550 con 0x7f39d806c3f0 2023-11-28T11:22:30.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.308+0000 7f39dd79b700 1 --2- 172.21.15.92:0/346027150 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39d806c3f0 0x7f39d81553f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f39d4009450 tx=0x7f39d4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:30.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.308+0000 7f39cdffb700 1 -- 172.21.15.92:0/346027150 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d40036a0 con 0x7f39d806c3f0 2023-11-28T11:22:30.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.308+0000 7f39cdffb700 1 -- 172.21.15.92:0/346027150 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39d4003820 con 0x7f39d806c3f0 2023-11-28T11:22:30.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.308+0000 7f39df9ff700 1 -- 172.21.15.92:0/346027150 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39d80cfff0 con 0x7f39d806c3f0 2023-11-28T11:22:30.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.308+0000 7f39cdffb700 1 -- 172.21.15.92:0/346027150 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d4003bd0 con 0x7f39d806c3f0 2023-11-28T11:22:30.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.308+0000 7f39df9ff700 1 -- 172.21.15.92:0/346027150 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39d80cfd10 con 0x7f39d806c3f0 2023-11-28T11:22:30.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.309+0000 7f39df9ff700 1 -- 172.21.15.92:0/346027150 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39d8067880 con 0x7f39d806c3f0 2023-11-28T11:22:30.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.310+0000 7f39cdffb700 1 -- 172.21.15.92:0/346027150 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f39d402d070 con 0x7f39d806c3f0 2023-11-28T11:22:30.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.311+0000 7f39cdffb700 1 --2- 172.21.15.92:0/346027150 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f39bc0682f0 0x7f39bc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:30.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.311+0000 7f39cdffb700 1 -- 172.21.15.92:0/346027150 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f39d4021030 con 0x7f39d806c3f0 2023-11-28T11:22:30.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.312+0000 7f39dcf9a700 1 --2- 172.21.15.92:0/346027150 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f39bc0682f0 0x7f39bc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:30.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.312+0000 7f39dcf9a700 1 --2- 172.21.15.92:0/346027150 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f39bc0682f0 0x7f39bc06a7a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f39c8006fd0 tx=0x7f39c8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:30.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.316+0000 7f39cdffb700 1 -- 172.21.15.92:0/346027150 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f39d4053c50 con 0x7f39d806c3f0 2023-11-28T11:22:30.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.548+0000 7f39df9ff700 1 -- 172.21.15.92:0/346027150 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f39d80d2940 con 0x7f39d806c3f0 2023-11-28T11:22:30.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.548+0000 7f39cdffb700 1 -- 172.21.15.92:0/346027150 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4645 (secure 0 0 0) 0x7f39d402b090 con 0x7f39d806c3f0 2023-11-28T11:22:30.550 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:22:30.551 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4976,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/3102603064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3102603064},{"type":"v1","addr":"172.21.15.169:6834","nonce":3102603064}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4979,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/79247121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":79247121},{"type":"v1","addr":"172.21.15.92:6837","nonce":79247121}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5221,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/219797192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":219797192},{"type":"v1","addr":"172.21.15.169:6835","nonce":219797192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:22:24.715447+0000","modified":"2023-11-28T11:22:27.733117+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5298},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5298":{"gid":5298,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.92:6835/1577294649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1577294649},{"type":"v1","addr":"172.21.15.92:6835","nonce":1577294649}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T11:22:30.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.551+0000 7f39df9ff700 1 -- 172.21.15.92:0/346027150 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f39bc0682f0 msgr2=0x7f39bc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:30.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.551+0000 7f39df9ff700 1 --2- 172.21.15.92:0/346027150 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f39bc0682f0 0x7f39bc06a7a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f39c8006fd0 tx=0x7f39c8008040 comp rx=0 tx=0).stop 2023-11-28T11:22:30.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.551+0000 7f39df9ff700 1 -- 172.21.15.92:0/346027150 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39d806c3f0 msgr2=0x7f39d81553f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:30.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.551+0000 7f39df9ff700 1 --2- 172.21.15.92:0/346027150 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39d806c3f0 0x7f39d81553f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f39d4009450 tx=0x7f39d4004750 comp rx=0 tx=0).stop 2023-11-28T11:22:30.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.551+0000 7f39df9ff700 1 -- 172.21.15.92:0/346027150 shutdown_connections 2023-11-28T11:22:30.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.551+0000 7f39df9ff700 1 --2- 172.21.15.92:0/346027150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f39d80cd460 0x7f39d80aab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:30.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.551+0000 7f39df9ff700 1 --2- 172.21.15.92:0/346027150 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f39bc0682f0 0x7f39bc06a7a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:30.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.551+0000 7f39df9ff700 1 --2- 172.21.15.92:0/346027150 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39d806c3f0 0x7f39d81553f0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:30.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.551+0000 7f39df9ff700 1 --2- 172.21.15.92:0/346027150 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f39d80b8510 0x7f39d81f2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:30.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.551+0000 7f39df9ff700 1 -- 172.21.15.92:0/346027150 >> 172.21.15.92:0/346027150 conn(0x7f39d80719a0 msgr2=0x7f39d80b5d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:30.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.551+0000 7f39df9ff700 1 -- 172.21.15.92:0/346027150 shutdown_connections 2023-11-28T11:22:30.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:30.551+0000 7f39df9ff700 1 -- 172.21.15.92:0/346027150 wait complete. 2023-11-28T11:22:30.554 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 17 2023-11-28T11:22:30.973 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T11:22:24.715447+0000', 'modified': '2023-11-28T11:22:27.733117+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5298}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5298': {'gid': 5298, 'name': 'a', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.92:6835/1577294649', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6834', 'nonce': 1577294649}, {'type': 'v1', 'addr': '172.21.15.92:6835', 'nonce': 1577294649}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T11:22:30.973 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T11:22:30.973 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-28T11:22:31.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.171+0000 7f9eb7bff700 1 Processor -- start 2023-11-28T11:22:31.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.171+0000 7f9eb7bff700 1 -- start start 2023-11-28T11:22:31.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.172+0000 7f9eb7bff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9eb00cd460 0x7f9eb006c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:31.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.172+0000 7f9eb7bff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9eb00d0d60 0x7f9eb00d11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:31.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.172+0000 7f9eb7bff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9eb00b88e0 con 0x7f9eb00b8510 2023-11-28T11:22:31.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.172+0000 7f9eb7bff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9eb00cdbd0 con 0x7f9eb00d0d60 2023-11-28T11:22:31.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.172+0000 7f9eb7bff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb00d02d0 con 0x7f9eb00cd460 2023-11-28T11:22:31.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.173+0000 7f9eb519a700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9eb00cd460 0x7f9eb006c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:31.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.173+0000 7f9eb619c700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9eb00d0d60 0x7f9eb00d11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:31.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.173+0000 7f9eb599b700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f9eb00b8510 0x7f9eb006b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.92:37966/0 (socket says 172.21.15.92:37966) 2023-11-28T11:22:31.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.173+0000 7f9eb599b700 1 -- 172.21.15.92:0/940807005 learned_addr learned my addr 172.21.15.92:0/940807005 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:22:31.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.173+0000 7f9eb4999700 1 -- 172.21.15.92:0/940807005 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1320039664 0 0) 0x7f9eb00b88e0 con 0x7f9eb00b8510 2023-11-28T11:22:31.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.173+0000 7f9eb4999700 1 -- 172.21.15.92:0/940807005 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e9c0035b0 con 0x7f9eb00b8510 2023-11-28T11:22:31.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.173+0000 7f9eb519a700 1 -- 172.21.15.92:0/940807005 >> v2:172.21.15.169:3300/0 conn(0x7f9eb00d0d60 msgr2=0x7f9eb00d11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:31.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.173+0000 7f9eb519a700 1 --2- 172.21.15.92:0/940807005 >> v2:172.21.15.169:3300/0 conn(0x7f9eb00d0d60 0x7f9eb00d11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:31.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.173+0000 7f9eb519a700 1 -- 172.21.15.92:0/940807005 >> v1:172.21.15.169:6789/0 conn(0x7f9eb00b8510 legacy=0x7f9eb006b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:31.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.173+0000 7f9eb519a700 1 -- 172.21.15.92:0/940807005 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb00cd8f0 con 0x7f9eb00cd460 2023-11-28T11:22:31.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.173+0000 7f9eb619c700 1 --2- 172.21.15.92:0/940807005 >> v2:172.21.15.169:3300/0 conn(0x7f9eb00d0d60 0x7f9eb00d11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:22:31.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.174+0000 7f9eb519a700 1 --2- 172.21.15.92:0/940807005 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9eb00cd460 0x7f9eb006c080 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0009840 tx=0x7f9ea0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddbe4b61799822a0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:31.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.174+0000 7f9eb4999700 1 -- 172.21.15.92:0/940807005 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0011830 con 0x7f9eb00cd460 2023-11-28T11:22:31.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.174+0000 7f9eb4999700 1 -- 172.21.15.92:0/940807005 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ea0011990 con 0x7f9eb00cd460 2023-11-28T11:22:31.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.174+0000 7f9eb4999700 1 -- 172.21.15.92:0/940807005 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0011d20 con 0x7f9eb00cd460 2023-11-28T11:22:31.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.175+0000 7f9eb7bff700 1 -- 172.21.15.92:0/940807005 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9eb00cd460 msgr2=0x7f9eb006c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:31.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.175+0000 7f9eb7bff700 1 --2- 172.21.15.92:0/940807005 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9eb00cd460 0x7f9eb006c080 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0009840 tx=0x7f9ea0009b50 comp rx=0 tx=0).stop 2023-11-28T11:22:31.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.175+0000 7f9eb7bff700 1 -- 172.21.15.92:0/940807005 shutdown_connections 2023-11-28T11:22:31.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.175+0000 7f9eb7bff700 1 --2- 172.21.15.92:0/940807005 >> v2:172.21.15.169:3300/0 conn(0x7f9eb00d0d60 0x7f9eb00d11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:31.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.175+0000 7f9eb7bff700 1 --2- 172.21.15.92:0/940807005 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9eb00cd460 0x7f9eb006c080 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:31.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.175+0000 7f9eb7bff700 1 -- 172.21.15.92:0/940807005 >> 172.21.15.92:0/940807005 conn(0x7f9eb00719a0 msgr2=0x7f9eb0071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:31.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.175+0000 7f9eb7bff700 1 -- 172.21.15.92:0/940807005 shutdown_connections 2023-11-28T11:22:31.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.175+0000 7f9eb7bff700 1 -- 172.21.15.92:0/940807005 wait complete. 2023-11-28T11:22:31.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.177+0000 7f9eb7bff700 1 Processor -- start 2023-11-28T11:22:31.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.177+0000 7f9eb7bff700 1 -- start start 2023-11-28T11:22:31.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.178+0000 7f9eb7bff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9eb00b8510 0x7f9eb01569d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:31.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.178+0000 7f9eb7bff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9eb00cd460 0x7f9eb0154f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:31.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.178+0000 7f9eb7bff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9eb00d0d60 0x7f9eb00b5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:31.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.178+0000 7f9eb7bff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb01476f0 con 0x7f9eb00d0d60 2023-11-28T11:22:31.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.178+0000 7f9eb7bff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb0155930 con 0x7f9eb00cd460 2023-11-28T11:22:31.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.178+0000 7f9eb7bff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb0155c10 con 0x7f9eb00b8510 2023-11-28T11:22:31.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.178+0000 7f9eb619c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9eb00d0d60 0x7f9eb00b5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:31.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.178+0000 7f9eb599b700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9eb00b8510 0x7f9eb01569d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:31.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.178+0000 7f9eb519a700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9eb00cd460 0x7f9eb0154f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:31.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.178+0000 7f9eb619c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9eb00d0d60 0x7f9eb00b5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55978/0 (socket says 172.21.15.92:55978) 2023-11-28T11:22:31.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.178+0000 7f9eb619c700 1 -- 172.21.15.92:0/3490001503 learned_addr learned my addr 172.21.15.92:0/3490001503 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:31.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.178+0000 7f9eb619c700 1 -- 172.21.15.92:0/3490001503 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9eb00b8510 msgr2=0x7f9eb01569d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:31.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.178+0000 7f9eb619c700 1 --2- 172.21.15.92:0/3490001503 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9eb00b8510 0x7f9eb01569d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:31.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.178+0000 7f9eb619c700 1 -- 172.21.15.92:0/3490001503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9eb00cd460 msgr2=0x7f9eb0154f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:31.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.178+0000 7f9eb619c700 1 --2- 172.21.15.92:0/3490001503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9eb00cd460 0x7f9eb0154f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:31.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.178+0000 7f9eb619c700 1 -- 172.21.15.92:0/3490001503 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea8009a70 con 0x7f9eb00d0d60 2023-11-28T11:22:31.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.179+0000 7f9eb619c700 1 --2- 172.21.15.92:0/3490001503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9eb00d0d60 0x7f9eb00b5f80 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f9ea800efc0 tx=0x7f9ea800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:31.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.179+0000 7f9ea67fc700 1 -- 172.21.15.92:0/3490001503 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea800ca90 con 0x7f9eb00d0d60 2023-11-28T11:22:31.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.179+0000 7f9ea67fc700 1 -- 172.21.15.92:0/3490001503 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ea800f040 con 0x7f9eb00d0d60 2023-11-28T11:22:31.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.179+0000 7f9eb7bff700 1 -- 172.21.15.92:0/3490001503 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ea00094a0 con 0x7f9eb00d0d60 2023-11-28T11:22:31.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.179+0000 7f9ea67fc700 1 -- 172.21.15.92:0/3490001503 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea800ce30 con 0x7f9eb00d0d60 2023-11-28T11:22:31.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.179+0000 7f9eb7bff700 1 -- 172.21.15.92:0/3490001503 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eb00cfff0 con 0x7f9eb00d0d60 2023-11-28T11:22:31.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.181+0000 7f9ea67fc700 1 -- 172.21.15.92:0/3490001503 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9ea8014480 con 0x7f9eb00d0d60 2023-11-28T11:22:31.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.182+0000 7f9ea67fc700 1 --2- 172.21.15.92:0/3490001503 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9e94068320 0x7f9e9406a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:31.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.182+0000 7f9ea67fc700 1 -- 172.21.15.92:0/3490001503 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f9ea8086700 con 0x7f9eb00d0d60 2023-11-28T11:22:31.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.182+0000 7f9eb599b700 1 --2- 172.21.15.92:0/3490001503 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9e94068320 0x7f9e9406a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:31.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.183+0000 7f9eb599b700 1 --2- 172.21.15.92:0/3490001503 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9e94068320 0x7f9e9406a7d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9eac007fd0 tx=0x7f9eac007dc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:31.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.184+0000 7f9ecb4fab80 1 Processor -- start 2023-11-28T11:22:31.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.184+0000 7f9ecb4fab80 1 -- start start 2023-11-28T11:22:31.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.185+0000 7f9ecb4fab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556a0740f50 0x5556a06fa9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:31.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.185+0000 7f9ecb4fab80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556a0737ac0 0x5556a0741320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:31.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.185+0000 7f9ecb4fab80 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556a0723660 0x5556a06fbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:31.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.185+0000 7f9ecb4fab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556a0678de0 con 0x5556a0740f50 2023-11-28T11:22:31.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.185+0000 7f9ecb4fab80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556a05e5140 con 0x5556a0737ac0 2023-11-28T11:22:31.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.185+0000 7f9ecb4fab80 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5556a0653db0 con 0x5556a0723660 2023-11-28T11:22:31.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.185+0000 7f9eb599b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556a0740f50 0x5556a06fa9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:31.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.185+0000 7f9eb599b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556a0740f50 0x5556a06fa9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55990/0 (socket says 172.21.15.92:55990) 2023-11-28T11:22:31.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.185+0000 7f9eb599b700 1 -- 172.21.15.92:0/1204670795 learned_addr learned my addr 172.21.15.92:0/1204670795 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:31.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.185+0000 7f9eb599b700 1 -- 172.21.15.92:0/1204670795 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556a0723660 msgr2=0x5556a06fbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:31.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.185+0000 7f9eb599b700 1 --2- 172.21.15.92:0/1204670795 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556a0723660 0x5556a06fbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:31.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.185+0000 7f9eb599b700 1 -- 172.21.15.92:0/1204670795 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556a0737ac0 msgr2=0x5556a0741320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:31.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.185+0000 7f9eb599b700 1 --2- 172.21.15.92:0/1204670795 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556a0737ac0 0x5556a0741320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:31.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.185+0000 7f9eb599b700 1 -- 172.21.15.92:0/1204670795 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556a0441790 con 0x5556a0740f50 2023-11-28T11:22:31.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.186+0000 7f9eb599b700 1 --2- 172.21.15.92:0/1204670795 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556a0740f50 0x5556a06fa9f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f9eac005bd0 tx=0x7f9eac012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9ca8d1dfac68426 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:31.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.186+0000 7f9e937fe700 1 -- 172.21.15.92:0/1204670795 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eac0063c0 con 0x5556a0740f50 2023-11-28T11:22:31.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.186+0000 7f9e937fe700 1 -- 172.21.15.92:0/1204670795 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eac006520 con 0x5556a0740f50 2023-11-28T11:22:31.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.186+0000 7f9e937fe700 1 -- 172.21.15.92:0/1204670795 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eac001370 con 0x5556a0740f50 2023-11-28T11:22:31.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.186+0000 7f9ecb4fab80 1 -- 172.21.15.92:0/1204670795 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556a0740f50 msgr2=0x5556a06fa9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:31.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.186+0000 7f9ecb4fab80 1 --2- 172.21.15.92:0/1204670795 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556a0740f50 0x5556a06fa9f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f9eac005bd0 tx=0x7f9eac012000 comp rx=0 tx=0).stop 2023-11-28T11:22:31.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.187+0000 7f9ecb4fab80 1 -- 172.21.15.92:0/1204670795 shutdown_connections 2023-11-28T11:22:31.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.187+0000 7f9ecb4fab80 1 --2- 172.21.15.92:0/1204670795 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556a0723660 0x5556a06fbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:31.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.187+0000 7f9ecb4fab80 1 --2- 172.21.15.92:0/1204670795 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556a0740f50 0x5556a06fa9f0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:31.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.187+0000 7f9ecb4fab80 1 --2- 172.21.15.92:0/1204670795 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556a0737ac0 0x5556a0741320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:31.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.187+0000 7f9ecb4fab80 1 -- 172.21.15.92:0/1204670795 >> 172.21.15.92:0/1204670795 conn(0x5556a06f0d20 msgr2=0x5556a06f9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:31.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.187+0000 7f9ecb4fab80 1 -- 172.21.15.92:0/1204670795 shutdown_connections 2023-11-28T11:22:31.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.187+0000 7f9ecb4fab80 1 -- 172.21.15.92:0/1204670795 wait complete. 2023-11-28T11:22:31.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.187+0000 7f9ecb4fab80 1 -- start start 2023-11-28T11:22:31.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.188+0000 7f9ecb4fab80 1 Processor -- start 2023-11-28T11:22:31.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.189+0000 7f9ecb4fab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556a0723660 0x5556a071ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:31.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.189+0000 7f9ecb4fab80 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556a0737ac0 0x5556a073fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:31.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.189+0000 7f9ecb4fab80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556a0740f50 0x5556a07402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:31.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.189+0000 7f9ecb4fab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556a064d120 con 0x5556a0723660 2023-11-28T11:22:31.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.189+0000 7f9ecb4fab80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556a0442390 con 0x5556a0740f50 2023-11-28T11:22:31.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.189+0000 7f9ecb4fab80 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5556a05e5340 con 0x5556a0737ac0 2023-11-28T11:22:31.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.189+0000 7f9ecb4fab80 10 client.0 _ll_register_callbacks cb 0x5556a05eb400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:22:31.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.189+0000 7f9eb599b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556a0723660 0x5556a071ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:31.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.189+0000 7f9eb599b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556a0723660 0x5556a071ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55996/0 (socket says 172.21.15.92:55996) 2023-11-28T11:22:31.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.189+0000 7f9eb599b700 1 -- 172.21.15.92:0/1848644102 learned_addr learned my addr 172.21.15.92:0/1848644102 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:31.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.189+0000 7f9eb619c700 1 --2- 172.21.15.92:0/1848644102 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556a0740f50 0x5556a07402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:31.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.189+0000 7f9eb599b700 1 -- 172.21.15.92:0/1848644102 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556a0737ac0 msgr2=0x5556a073fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:31.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.189+0000 7f9eb519a700 1 --2- 172.21.15.92:0/1848644102 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556a0737ac0 0x5556a073fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:31.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.189+0000 7f9eb599b700 1 --2- 172.21.15.92:0/1848644102 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556a0737ac0 0x5556a073fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:31.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.189+0000 7f9eb599b700 1 -- 172.21.15.92:0/1848644102 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556a0740f50 msgr2=0x5556a07402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:31.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.189+0000 7f9eb599b700 1 --2- 172.21.15.92:0/1848644102 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556a0740f50 0x5556a07402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:31.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.189+0000 7f9eb599b700 1 -- 172.21.15.92:0/1848644102 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e9c0035b0 con 0x5556a0723660 2023-11-28T11:22:31.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.189+0000 7f9eb619c700 1 --2- 172.21.15.92:0/1848644102 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556a0740f50 0x5556a07402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:22:31.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.189+0000 7f9eb519a700 1 --2- 172.21.15.92:0/1848644102 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556a0737ac0 0x5556a073fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:22:31.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.190+0000 7f9eb599b700 1 --2- 172.21.15.92:0/1848644102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556a0723660 0x5556a071ecb0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f9eac005f90 tx=0x7f9eac000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=829e10b4a43f834 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:31.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.190+0000 7f9e90ff9700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-11-28T11:22:31.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.190+0000 7f9e90ff9700 1 -- 172.21.15.92:0/1848644102 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eac0063c0 con 0x5556a0723660 2023-11-28T11:22:31.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.190+0000 7f9e90ff9700 1 -- 172.21.15.92:0/1848644102 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f9eac01b050 con 0x5556a0723660 2023-11-28T11:22:31.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.190+0000 7f9ecb4fab80 1 -- 172.21.15.92:0/1848644102 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5556a0651030 con 0x5556a0723660 2023-11-28T11:22:31.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.190+0000 7f9e90ff9700 1 -- 172.21.15.92:0/1848644102 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9eac006690 con 0x5556a0723660 2023-11-28T11:22:31.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.190+0000 7f9ecb4fab80 10 client.5478 fetch_fsmap learned FSMap version 17 2023-11-28T11:22:31.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.190+0000 7f9ecb4fab80 1 -- 172.21.15.92:0/1848644102 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x5556a052e0a0 con 0x5556a0723660 2023-11-28T11:22:31.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.190+0000 7f9e90ff9700 1 -- 172.21.15.92:0/1848644102 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5556a0651030 con 0x5556a0723660 2023-11-28T11:22:31.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.190+0000 7f9e90ff9700 1 -- 172.21.15.92:0/1848644102 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 17) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f9eac012070 con 0x5556a0723660 2023-11-28T11:22:31.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.190+0000 7f9ecb4fab80 10 client.5478 fetch_fsmap finished waiting for FSMap version 17 2023-11-28T11:22:31.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.190+0000 7f9ecb4fab80 10 client.5478 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T11:22:31.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.191+0000 7f9ecb4fab80 20 client.5478 populate_metadata read hostname 'smithi092' 2023-11-28T11:22:31.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.191+0000 7f9ecb4fab80 1 --2- 172.21.15.92:0/1848644102 >> [v2:172.21.15.92:6834/1577294649,v1:172.21.15.92:6835/1577294649] conn(0x5556a0728c60 0x5556a0729030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:31.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.191+0000 7f9ecb4fab80 4 client.5478 mds_command: new command op to 5298 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:22:31.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.191+0000 7f9ecb4fab80 1 -- 172.21.15.92:0/1848644102 --> [v2:172.21.15.92:6834/1577294649,v1:172.21.15.92:6835/1577294649] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5556a049cef0 con 0x5556a0728c60 2023-11-28T11:22:31.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.191+0000 7f9eb519a700 1 --2- 172.21.15.92:0/1848644102 >> [v2:172.21.15.92:6834/1577294649,v1:172.21.15.92:6835/1577294649] conn(0x5556a0728c60 0x5556a0729030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:31.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.191+0000 7f9eb519a700 1 --2- 172.21.15.92:0/1848644102 >> [v2:172.21.15.92:6834/1577294649,v1:172.21.15.92:6835/1577294649] conn(0x5556a0728c60 0x5556a0729030 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33d4b17f525045d2 server_cookie=a7ab7e77b94c6e38 in_seq=0 out_seq=0 2023-11-28T11:22:31.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.191+0000 7f9e90ff9700 10 client.5478 ms_handle_connect on v2:172.21.15.92:6834/1577294649 2023-11-28T11:22:31.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.192+0000 7f9e90ff9700 1 -- 172.21.15.92:0/1848644102 <== mds.0 v2:172.21.15.92:6834/1577294649 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x5556a049cef0 con 0x5556a0728c60 2023-11-28T11:22:31.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.192+0000 7f9e90ff9700 10 client.5478 handle_command_reply: tid=0 2023-11-28T11:22:31.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.192+0000 7f9ecb4fab80 1 client.5478 shutdown 2023-11-28T11:22:31.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.192+0000 7f9ecb4fab80 1 -- 172.21.15.92:0/1848644102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556a0723660 msgr2=0x5556a071ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:31.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.192+0000 7f9ecb4fab80 1 --2- 172.21.15.92:0/1848644102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556a0723660 0x5556a071ecb0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f9eac005f90 tx=0x7f9eac000ed0 comp rx=0 tx=0).stop 2023-11-28T11:22:31.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.192+0000 7f9ecb4fab80 1 -- 172.21.15.92:0/1848644102 shutdown_connections 2023-11-28T11:22:31.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.192+0000 7f9ecb4fab80 1 --2- 172.21.15.92:0/1848644102 >> [v2:172.21.15.92:6834/1577294649,v1:172.21.15.92:6835/1577294649] conn(0x5556a0728c60 0x5556a0729030 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:31.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.192+0000 7f9ecb4fab80 1 --2- 172.21.15.92:0/1848644102 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556a0740f50 0x5556a07402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:31.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.192+0000 7f9ecb4fab80 1 --2- 172.21.15.92:0/1848644102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556a0723660 0x5556a071ecb0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:31.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.192+0000 7f9e90ff9700 0 client.5478 ms_handle_reset on v2:172.21.15.92:6834/1577294649 2023-11-28T11:22:31.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.192+0000 7f9ecb4fab80 1 --2- 172.21.15.92:0/1848644102 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556a0737ac0 0x5556a073fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:31.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.192+0000 7f9ecb4fab80 1 -- 172.21.15.92:0/1848644102 >> 172.21.15.92:0/1848644102 conn(0x5556a06f0d20 msgr2=0x5556a0722d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:31.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.193+0000 7f9ecb4fab80 1 -- 172.21.15.92:0/1848644102 shutdown_connections 2023-11-28T11:22:31.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.193+0000 7f9ecb4fab80 1 -- 172.21.15.92:0/1848644102 wait complete. 2023-11-28T11:22:31.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.193+0000 7f9ecb4fab80 20 client.5478 trim_cache size 0 max 16384 2023-11-28T11:22:31.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.210+0000 7f9ecb4fab80 1 Processor -- start 2023-11-28T11:22:31.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.210+0000 7f9ecb4fab80 1 -- start start 2023-11-28T11:22:31.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.211+0000 7f9ecb4fab80 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556a0723660 0x5556a06f21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:31.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.211+0000 7f9ecb4fab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556a0737ac0 0x5556a0722d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:31.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.211+0000 7f9ecb4fab80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556a0740f50 0x5556a06fa060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:31.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.211+0000 7f9ecb4fab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556a06f2730 con 0x5556a0737ac0 2023-11-28T11:22:31.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.211+0000 7f9ecb4fab80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556a073e580 con 0x5556a0740f50 2023-11-28T11:22:31.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.211+0000 7f9ecb4fab80 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5556a073f7e0 con 0x5556a0723660 2023-11-28T11:22:31.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.211+0000 7f9eb519a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556a0737ac0 0x5556a0722d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:31.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.211+0000 7f9eb519a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556a0737ac0 0x5556a0722d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56010/0 (socket says 172.21.15.92:56010) 2023-11-28T11:22:31.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.211+0000 7f9eb519a700 1 -- 172.21.15.92:0/3826871572 learned_addr learned my addr 172.21.15.92:0/3826871572 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:31.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.212+0000 7f9eb519a700 1 -- 172.21.15.92:0/3826871572 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556a0723660 msgr2=0x5556a06f21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:31.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.212+0000 7f9eb519a700 1 --2- 172.21.15.92:0/3826871572 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556a0723660 0x5556a06f21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:31.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.212+0000 7f9eb519a700 1 -- 172.21.15.92:0/3826871572 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556a0740f50 msgr2=0x5556a06fa060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:31.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.212+0000 7f9eb619c700 1 --2- 172.21.15.92:0/3826871572 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556a0740f50 0x5556a06fa060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:31.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.212+0000 7f9eb519a700 1 --2- 172.21.15.92:0/3826871572 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556a0740f50 0x5556a06fa060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:31.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.212+0000 7f9eb519a700 1 -- 172.21.15.92:0/3826871572 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556a06142e0 con 0x5556a0737ac0 2023-11-28T11:22:31.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.212+0000 7f9eb619c700 1 --2- 172.21.15.92:0/3826871572 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556a0740f50 0x5556a06fa060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:22:31.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.212+0000 7f9eb519a700 1 --2- 172.21.15.92:0/3826871572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556a0737ac0 0x5556a0722d30 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0013000 tx=0x7f9ea0010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fae66c4e701ecb0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:31.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.212+0000 7f9e93fff700 1 -- 172.21.15.92:0/3826871572 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0020040 con 0x5556a0737ac0 2023-11-28T11:22:31.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.212+0000 7f9e93fff700 1 -- 172.21.15.92:0/3826871572 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ea0011d20 con 0x5556a0737ac0 2023-11-28T11:22:31.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.212+0000 7f9e93fff700 1 -- 172.21.15.92:0/3826871572 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0028260 con 0x5556a0737ac0 2023-11-28T11:22:31.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.213+0000 7f9ecb4fab80 1 -- 172.21.15.92:0/3826871572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556a0737ac0 msgr2=0x5556a0722d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:31.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.213+0000 7f9ecb4fab80 1 --2- 172.21.15.92:0/3826871572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556a0737ac0 0x5556a0722d30 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0013000 tx=0x7f9ea0010000 comp rx=0 tx=0).stop 2023-11-28T11:22:31.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.213+0000 7f9ecb4fab80 1 -- 172.21.15.92:0/3826871572 shutdown_connections 2023-11-28T11:22:31.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.213+0000 7f9ecb4fab80 1 --2- 172.21.15.92:0/3826871572 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556a0740f50 0x5556a06fa060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:31.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.213+0000 7f9ecb4fab80 1 --2- 172.21.15.92:0/3826871572 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556a0723660 0x5556a06f21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:31.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.213+0000 7f9ecb4fab80 1 --2- 172.21.15.92:0/3826871572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556a0737ac0 0x5556a0722d30 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:31.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.213+0000 7f9ecb4fab80 1 -- 172.21.15.92:0/3826871572 >> 172.21.15.92:0/3826871572 conn(0x5556a06f0d20 msgr2=0x5556a073dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:31.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.213+0000 7f9ecb4fab80 1 -- 172.21.15.92:0/3826871572 shutdown_connections 2023-11-28T11:22:31.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.213+0000 7f9ecb4fab80 1 -- 172.21.15.92:0/3826871572 wait complete. 2023-11-28T11:22:31.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.214+0000 7f9ecb4fab80 1 -- start start 2023-11-28T11:22:31.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.214+0000 7f9ecb4fab80 1 Processor -- start 2023-11-28T11:22:31.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.215+0000 7f9ecb4fab80 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556a0723660 0x5556a0728600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:31.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.215+0000 7f9ecb4fab80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556a0737ac0 0x5556a07a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:31.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.215+0000 7f9ecb4fab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556a0740f50 0x5556a07a8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:31.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.215+0000 7f9ecb4fab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556a06face0 con 0x5556a0740f50 2023-11-28T11:22:31.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.215+0000 7f9ecb4fab80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556a07a9470 con 0x5556a0737ac0 2023-11-28T11:22:31.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.215+0000 7f9ecb4fab80 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5556a0729030 con 0x5556a0723660 2023-11-28T11:22:31.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.215+0000 7f9ecb4fab80 10 client.0 _ll_register_callbacks cb 0x5556a05eb400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:22:31.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.215+0000 7f9eb619c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556a0740f50 0x5556a07a8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:31.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.215+0000 7f9eb619c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556a0740f50 0x5556a07a8cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56020/0 (socket says 172.21.15.92:56020) 2023-11-28T11:22:31.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.215+0000 7f9eb619c700 1 -- 172.21.15.92:0/2298641032 learned_addr learned my addr 172.21.15.92:0/2298641032 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:31.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.215+0000 7f9eb599b700 1 --2- 172.21.15.92:0/2298641032 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556a0723660 0x5556a0728600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:31.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.215+0000 7f9eb519a700 1 --2- 172.21.15.92:0/2298641032 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556a0737ac0 0x5556a07a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:31.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.215+0000 7f9eb619c700 1 -- 172.21.15.92:0/2298641032 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556a0723660 msgr2=0x5556a0728600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:31.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.215+0000 7f9eb619c700 1 --2- 172.21.15.92:0/2298641032 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556a0723660 0x5556a0728600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:31.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.215+0000 7f9eb619c700 1 -- 172.21.15.92:0/2298641032 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556a0737ac0 msgr2=0x5556a07a51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:31.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.215+0000 7f9eb619c700 1 --2- 172.21.15.92:0/2298641032 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556a0737ac0 0x5556a07a51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:31.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.215+0000 7f9eb619c700 1 -- 172.21.15.92:0/2298641032 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eb00cd8f0 con 0x5556a0740f50 2023-11-28T11:22:31.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.216+0000 7f9eb619c700 1 --2- 172.21.15.92:0/2298641032 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556a0740f50 0x5556a07a8cb0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8052e00 tx=0x7f9ea8007ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=555bcc6ba62577ba server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:31.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.216+0000 7f9e90ff9700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-11-28T11:22:31.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.216+0000 7f9e90ff9700 1 -- 172.21.15.92:0/2298641032 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea8082070 con 0x5556a0740f50 2023-11-28T11:22:31.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.216+0000 7f9ecb4fab80 1 -- 172.21.15.92:0/2298641032 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9eac006520 con 0x5556a0740f50 2023-11-28T11:22:31.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.216+0000 7f9e90ff9700 1 -- 172.21.15.92:0/2298641032 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f9ea80526f0 con 0x5556a0740f50 2023-11-28T11:22:31.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.216+0000 7f9e90ff9700 1 -- 172.21.15.92:0/2298641032 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ea806b070 con 0x5556a0740f50 2023-11-28T11:22:31.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.216+0000 7f9ecb4fab80 10 client.5484 fetch_fsmap learned FSMap version 17 2023-11-28T11:22:31.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.216+0000 7f9ecb4fab80 1 -- 172.21.15.92:0/2298641032 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x5556a04a21d0 con 0x5556a0740f50 2023-11-28T11:22:31.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.216+0000 7f9e90ff9700 1 -- 172.21.15.92:0/2298641032 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eac006520 con 0x5556a0740f50 2023-11-28T11:22:31.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.216+0000 7f9e90ff9700 1 -- 172.21.15.92:0/2298641032 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 17) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f9ea80528b0 con 0x5556a0740f50 2023-11-28T11:22:31.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.216+0000 7f9ecb4fab80 10 client.5484 fetch_fsmap finished waiting for FSMap version 17 2023-11-28T11:22:31.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.216+0000 7f9ecb4fab80 10 client.5484 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T11:22:31.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.216+0000 7f9ecb4fab80 20 client.5484 populate_metadata read hostname 'smithi092' 2023-11-28T11:22:31.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.216+0000 7f9ecb4fab80 1 --2- 172.21.15.92:0/2298641032 >> [v2:172.21.15.92:6834/1577294649,v1:172.21.15.92:6835/1577294649] conn(0x5556a0725af0 0x5556a0725ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:31.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.216+0000 7f9ecb4fab80 4 client.5484 mds_command: new command op to 5298 tid=0[{"prefix": "status"}] 2023-11-28T11:22:31.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.216+0000 7f9ecb4fab80 1 -- 172.21.15.92:0/2298641032 --> [v2:172.21.15.92:6834/1577294649,v1:172.21.15.92:6835/1577294649] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5556a04d2040 con 0x5556a0725af0 2023-11-28T11:22:31.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.217+0000 7f9eb599b700 1 --2- 172.21.15.92:0/2298641032 >> [v2:172.21.15.92:6834/1577294649,v1:172.21.15.92:6835/1577294649] conn(0x5556a0725af0 0x5556a0725ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:31.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.217+0000 7f9eb599b700 1 --2- 172.21.15.92:0/2298641032 >> [v2:172.21.15.92:6834/1577294649,v1:172.21.15.92:6835/1577294649] conn(0x5556a0725af0 0x5556a0725ec0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4171c6d823a8600 server_cookie=19783e7064b5f9cc in_seq=0 out_seq=0 2023-11-28T11:22:31.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.217+0000 7f9e90ff9700 10 client.5484 ms_handle_connect on v2:172.21.15.92:6834/1577294649 2023-11-28T11:22:31.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.217+0000 7f9e90ff9700 1 -- 172.21.15.92:0/2298641032 <== mds.0 v2:172.21.15.92:6834/1577294649 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5556a04d2040 con 0x5556a0725af0 2023-11-28T11:22:31.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.217+0000 7f9e90ff9700 10 client.5484 handle_command_reply: tid=0 2023-11-28T11:22:31.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.217+0000 7f9ecb4fab80 1 client.5484 shutdown 2023-11-28T11:22:31.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.217+0000 7f9ecb4fab80 1 -- 172.21.15.92:0/2298641032 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556a0740f50 msgr2=0x5556a07a8cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:31.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.217+0000 7f9ecb4fab80 1 --2- 172.21.15.92:0/2298641032 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556a0740f50 0x5556a07a8cb0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8052e00 tx=0x7f9ea8007ee0 comp rx=0 tx=0).stop 2023-11-28T11:22:31.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.217+0000 7f9ecb4fab80 1 -- 172.21.15.92:0/2298641032 shutdown_connections 2023-11-28T11:22:31.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.217+0000 7f9ecb4fab80 1 --2- 172.21.15.92:0/2298641032 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556a0740f50 0x5556a07a8cb0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:31.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.217+0000 7f9ecb4fab80 1 --2- 172.21.15.92:0/2298641032 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5556a0723660 0x5556a0728600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:31.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.217+0000 7f9ecb4fab80 1 --2- 172.21.15.92:0/2298641032 >> [v2:172.21.15.92:6834/1577294649,v1:172.21.15.92:6835/1577294649] conn(0x5556a0725af0 0x5556a0725ec0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:31.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.217+0000 7f9ecb4fab80 1 --2- 172.21.15.92:0/2298641032 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5556a0737ac0 0x5556a07a51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:31.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.217+0000 7f9e90ff9700 0 client.5484 ms_handle_reset on v2:172.21.15.92:6834/1577294649 2023-11-28T11:22:31.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.217+0000 7f9ecb4fab80 1 -- 172.21.15.92:0/2298641032 >> 172.21.15.92:0/2298641032 conn(0x5556a06f0d20 msgr2=0x5556a073a940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:31.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.218+0000 7f9ecb4fab80 1 -- 172.21.15.92:0/2298641032 shutdown_connections 2023-11-28T11:22:31.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.218+0000 7f9ecb4fab80 1 -- 172.21.15.92:0/2298641032 wait complete. 2023-11-28T11:22:31.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.218+0000 7f9ecb4fab80 20 client.5484 trim_cache size 0 max 16384 2023-11-28T11:22:31.226 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-28T11:22:31.226 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "68956e0c-c16f-47cd-a220-f98364b0831d", 2023-11-28T11:22:31.226 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 0, 2023-11-28T11:22:31.226 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 5298, 2023-11-28T11:22:31.226 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-11-28T11:22:31.226 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-11-28T11:22:31.226 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-11-28T11:22:31.227 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 6.4997714709999999, 2023-11-28T11:22:31.227 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 17, 2023-11-28T11:22:31.227 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 42, 2023-11-28T11:22:31.227 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 42, 2023-11-28T11:22:31.227 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 15.071923997000001 2023-11-28T11:22:31.227 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-28T11:22:31.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.218+0000 7f9eb7bff700 1 -- 172.21.15.92:0/3490001503 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9e94068320 msgr2=0x7f9e9406a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:31.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.218+0000 7f9eb7bff700 1 --2- 172.21.15.92:0/3490001503 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9e94068320 0x7f9e9406a7d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9eac007fd0 tx=0x7f9eac007dc0 comp rx=0 tx=0).stop 2023-11-28T11:22:31.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.218+0000 7f9eb7bff700 1 -- 172.21.15.92:0/3490001503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9eb00d0d60 msgr2=0x7f9eb00b5f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:31.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.218+0000 7f9eb7bff700 1 --2- 172.21.15.92:0/3490001503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9eb00d0d60 0x7f9eb00b5f80 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f9ea800efc0 tx=0x7f9ea800c3a0 comp rx=0 tx=0).stop 2023-11-28T11:22:31.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.218+0000 7f9eb7bff700 1 -- 172.21.15.92:0/3490001503 shutdown_connections 2023-11-28T11:22:31.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.218+0000 7f9eb7bff700 1 --2- 172.21.15.92:0/3490001503 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9e94068320 0x7f9e9406a7d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:31.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.218+0000 7f9eb7bff700 1 --2- 172.21.15.92:0/3490001503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9eb00d0d60 0x7f9eb00b5f80 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:31.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.218+0000 7f9eb7bff700 1 --2- 172.21.15.92:0/3490001503 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9eb00b8510 0x7f9eb01569d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:31.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.218+0000 7f9eb7bff700 1 --2- 172.21.15.92:0/3490001503 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9eb00cd460 0x7f9eb0154f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:31.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.218+0000 7f9eb7bff700 1 -- 172.21.15.92:0/3490001503 >> 172.21.15.92:0/3490001503 conn(0x7f9eb00719a0 msgr2=0x7f9eb00d11e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:31.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.219+0000 7f9eb7bff700 1 -- 172.21.15.92:0/3490001503 shutdown_connections 2023-11-28T11:22:31.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.219+0000 7f9eb7bff700 1 -- 172.21.15.92:0/3490001503 wait complete. 2023-11-28T11:22:31.238 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T11:22:31.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.431+0000 7fc21edff700 1 Processor -- start 2023-11-28T11:22:31.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.432+0000 7fc21edff700 1 -- start start 2023-11-28T11:22:31.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.433+0000 7fc21edff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc2180b8510 0x7fc21806b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:31.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.433+0000 7fc21edff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2180cd460 0x7fc21806beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:31.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.433+0000 7fc21edff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2180b88e0 con 0x7fc21806c3f0 2023-11-28T11:22:31.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.433+0000 7fc21edff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2180cdbd0 con 0x7fc2180b8510 2023-11-28T11:22:31.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.433+0000 7fc21edff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2180d02d0 con 0x7fc2180cd460 2023-11-28T11:22:31.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.433+0000 7fc21cb9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc2180b8510 0x7fc21806b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:31.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.433+0000 7fc21cb9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc2180b8510 0x7fc21806b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56024/0 (socket says 172.21.15.92:56024) 2023-11-28T11:22:31.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.433+0000 7fc21cb9b700 1 -- 172.21.15.92:0/1783235209 learned_addr learned my addr 172.21.15.92:0/1783235209 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:31.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.433+0000 7fc217fff700 1 --2- 172.21.15.92:0/1783235209 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2180cd460 0x7fc21806beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:31.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.434+0000 7fc21cb9b700 1 -- 172.21.15.92:0/1783235209 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2180cd460 msgr2=0x7fc21806beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:31.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.434+0000 7fc21cb9b700 1 --2- 172.21.15.92:0/1783235209 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2180cd460 0x7fc21806beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:31.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.434+0000 7fc21cb9b700 1 -- 172.21.15.92:0/1783235209 >> v1:172.21.15.169:6789/0 conn(0x7fc21806c3f0 legacy=0x7fc21815cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:31.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.434+0000 7fc21cb9b700 1 -- 172.21.15.92:0/1783235209 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2180cd8f0 con 0x7fc2180b8510 2023-11-28T11:22:31.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.434+0000 7fc21cb9b700 1 --2- 172.21.15.92:0/1783235209 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2180b8510 0x7fc21806b970 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc20c009920 tx=0x7fc20c009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85834a0fc6dbab79 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:31.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.434+0000 7fc2177fe700 1 -- 172.21.15.92:0/1783235209 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20c00ba20 con 0x7fc2180b8510 2023-11-28T11:22:31.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.434+0000 7fc2177fe700 1 -- 172.21.15.92:0/1783235209 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc20c00bb80 con 0x7fc2180b8510 2023-11-28T11:22:31.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.434+0000 7fc2177fe700 1 -- 172.21.15.92:0/1783235209 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20c009e70 con 0x7fc2180b8510 2023-11-28T11:22:31.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.435+0000 7fc21edff700 1 -- 172.21.15.92:0/1783235209 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2180b8510 msgr2=0x7fc21806b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:31.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.435+0000 7fc21edff700 1 --2- 172.21.15.92:0/1783235209 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2180b8510 0x7fc21806b970 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc20c009920 tx=0x7fc20c009c30 comp rx=0 tx=0).stop 2023-11-28T11:22:31.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.435+0000 7fc21edff700 1 -- 172.21.15.92:0/1783235209 shutdown_connections 2023-11-28T11:22:31.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.435+0000 7fc21edff700 1 --2- 172.21.15.92:0/1783235209 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2180b8510 0x7fc21806b970 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:31.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.435+0000 7fc21edff700 1 --2- 172.21.15.92:0/1783235209 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2180cd460 0x7fc21806beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:31.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.435+0000 7fc21edff700 1 -- 172.21.15.92:0/1783235209 >> 172.21.15.92:0/1783235209 conn(0x7fc2180719a0 msgr2=0x7fc218071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:31.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.435+0000 7fc21edff700 1 -- 172.21.15.92:0/1783235209 shutdown_connections 2023-11-28T11:22:31.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.435+0000 7fc21edff700 1 -- 172.21.15.92:0/1783235209 wait complete. 2023-11-28T11:22:31.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.436+0000 7fc21edff700 1 Processor -- start 2023-11-28T11:22:31.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.437+0000 7fc21edff700 1 -- start start 2023-11-28T11:22:31.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.437+0000 7fc21edff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc21806c3f0 0x7fc2180cab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:31.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.437+0000 7fc21edff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2180b8510 0x7fc218158540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:31.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.437+0000 7fc21edff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2180cd460 0x7fc2180d15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:31.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.437+0000 7fc21edff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc218157be0 con 0x7fc2180b8510 2023-11-28T11:22:31.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.437+0000 7fc21edff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2180ca080 con 0x7fc21806c3f0 2023-11-28T11:22:31.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.437+0000 7fc21edff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2180ca360 con 0x7fc2180cd460 2023-11-28T11:22:31.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.438+0000 7fc217fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2180b8510 0x7fc218158540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:31.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.438+0000 7fc217fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2180b8510 0x7fc218158540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56038/0 (socket says 172.21.15.92:56038) 2023-11-28T11:22:31.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.438+0000 7fc217fff700 1 -- 172.21.15.92:0/766094364 learned_addr learned my addr 172.21.15.92:0/766094364 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:31.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.438+0000 7fc21cb9b700 1 --2- 172.21.15.92:0/766094364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc21806c3f0 0x7fc2180cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:31.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.438+0000 7fc21d39c700 1 --2- 172.21.15.92:0/766094364 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2180cd460 0x7fc2180d15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:31.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.438+0000 7fc217fff700 1 -- 172.21.15.92:0/766094364 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2180cd460 msgr2=0x7fc2180d15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:31.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.438+0000 7fc217fff700 1 --2- 172.21.15.92:0/766094364 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2180cd460 0x7fc2180d15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:31.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.438+0000 7fc217fff700 1 -- 172.21.15.92:0/766094364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc21806c3f0 msgr2=0x7fc2180cab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:31.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.438+0000 7fc217fff700 1 --2- 172.21.15.92:0/766094364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc21806c3f0 0x7fc2180cab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:31.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.438+0000 7fc217fff700 1 -- 172.21.15.92:0/766094364 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc20c009550 con 0x7fc2180b8510 2023-11-28T11:22:31.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.438+0000 7fc21d39c700 1 --2- 172.21.15.92:0/766094364 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2180cd460 0x7fc2180d15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:22:31.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.438+0000 7fc217fff700 1 --2- 172.21.15.92:0/766094364 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2180b8510 0x7fc218158540 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fc20400d800 tx=0x7fc20400db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:31.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.439+0000 7fc2157fa700 1 -- 172.21.15.92:0/766094364 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2040125f0 con 0x7fc2180b8510 2023-11-28T11:22:31.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.439+0000 7fc2157fa700 1 -- 172.21.15.92:0/766094364 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc204012750 con 0x7fc2180b8510 2023-11-28T11:22:31.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.439+0000 7fc2157fa700 1 -- 172.21.15.92:0/766094364 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc204012b00 con 0x7fc2180b8510 2023-11-28T11:22:31.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.439+0000 7fc21edff700 1 -- 172.21.15.92:0/766094364 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2180cfff0 con 0x7fc2180b8510 2023-11-28T11:22:31.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.439+0000 7fc21edff700 1 -- 172.21.15.92:0/766094364 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2180cfd10 con 0x7fc2180b8510 2023-11-28T11:22:31.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.440+0000 7fc21edff700 1 -- 172.21.15.92:0/766094364 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2180c9ee0 con 0x7fc2180b8510 2023-11-28T11:22:31.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.441+0000 7fc2157fa700 1 -- 172.21.15.92:0/766094364 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc204012c60 con 0x7fc2180b8510 2023-11-28T11:22:31.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.442+0000 7fc2157fa700 1 --2- 172.21.15.92:0/766094364 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc1fc0682f0 0x7fc1fc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:31.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.442+0000 7fc2157fa700 1 -- 172.21.15.92:0/766094364 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fc204086230 con 0x7fc2180b8510 2023-11-28T11:22:31.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.442+0000 7fc21cb9b700 1 --2- 172.21.15.92:0/766094364 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc1fc0682f0 0x7fc1fc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:31.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.443+0000 7fc21cb9b700 1 --2- 172.21.15.92:0/766094364 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc1fc0682f0 0x7fc1fc06a7a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc20c009c70 tx=0x7fc20c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:31.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.446+0000 7fc2157fa700 1 -- 172.21.15.92:0/766094364 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc2180c9ee0 con 0x7fc2180b8510 2023-11-28T11:22:31.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.677+0000 7fc21edff700 1 -- 172.21.15.92:0/766094364 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc2180c9ee0 con 0x7fc2180b8510 2023-11-28T11:22:31.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.678+0000 7fc2157fa700 1 -- 172.21.15.92:0/766094364 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc2180c9ee0 con 0x7fc2180b8510 2023-11-28T11:22:31.680 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T11:22:31.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.680+0000 7fc21edff700 1 -- 172.21.15.92:0/766094364 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc1fc0682f0 msgr2=0x7fc1fc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:31.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.680+0000 7fc21edff700 1 --2- 172.21.15.92:0/766094364 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc1fc0682f0 0x7fc1fc06a7a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc20c009c70 tx=0x7fc20c008030 comp rx=0 tx=0).stop 2023-11-28T11:22:31.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.680+0000 7fc21edff700 1 -- 172.21.15.92:0/766094364 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2180b8510 msgr2=0x7fc218158540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:31.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.680+0000 7fc21edff700 1 --2- 172.21.15.92:0/766094364 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2180b8510 0x7fc218158540 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fc20400d800 tx=0x7fc20400db10 comp rx=0 tx=0).stop 2023-11-28T11:22:31.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.680+0000 7fc21edff700 1 -- 172.21.15.92:0/766094364 shutdown_connections 2023-11-28T11:22:31.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.680+0000 7fc21edff700 1 --2- 172.21.15.92:0/766094364 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2180cd460 0x7fc2180d15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:31.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.680+0000 7fc21edff700 1 --2- 172.21.15.92:0/766094364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc21806c3f0 0x7fc2180cab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:31.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.680+0000 7fc21edff700 1 --2- 172.21.15.92:0/766094364 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc1fc0682f0 0x7fc1fc06a7a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:31.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.680+0000 7fc21edff700 1 --2- 172.21.15.92:0/766094364 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2180b8510 0x7fc218158540 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:31.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.680+0000 7fc21edff700 1 -- 172.21.15.92:0/766094364 >> 172.21.15.92:0/766094364 conn(0x7fc2180719a0 msgr2=0x7fc2180b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:31.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.680+0000 7fc21edff700 1 -- 172.21.15.92:0/766094364 shutdown_connections 2023-11-28T11:22:31.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:31.681+0000 7fc21edff700 1 -- 172.21.15.92:0/766094364 wait complete. 2023-11-28T11:22:31.700 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-28T11:22:31.701 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T11:22:31.701 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi092.front.sepia.ceph.com 2023-11-28T11:22:31.701 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T11:22:31.701 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T11:22:31.701 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-28T11:22:31.701 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-28T11:22:31.701 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T11:22:31.701 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:22:31.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:22:31.701 DEBUG:teuthology.orchestra.run.smithi092:> ip addr 2023-11-28T11:22:31.748 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:22:31.749 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:22:31.749 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:22:31.749 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:22:31.749 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-11-28T11:22:31.749 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:22:31.749 INFO:teuthology.orchestra.run.smithi092.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:22:31.749 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b2 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:22:31.749 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f0 2023-11-28T11:22:31.749 INFO:teuthology.orchestra.run.smithi092.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:22:31.749 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b3 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:22:31.749 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f1 2023-11-28T11:22:31.750 INFO:teuthology.orchestra.run.smithi092.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:22:31.750 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:44 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:22:31.750 INFO:teuthology.orchestra.run.smithi092.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T11:22:31.750 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:45 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:22:31.750 INFO:teuthology.orchestra.run.smithi092.stdout: inet 172.21.15.92/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T11:22:31.750 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft 41903sec preferred_lft 41903sec 2023-11-28T11:22:31.750 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::ec4:7aff:fe8f:cd45/64 scope link 2023-11-28T11:22:31.750 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:22:31.750 INFO:teuthology.orchestra.run.smithi092.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T11:22:31.750 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 02:f9:a8:fe:83:2e brd ff:ff:ff:ff:ff:ff 2023-11-28T11:22:31.750 INFO:teuthology.orchestra.run.smithi092.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T11:22:31.750 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:22:31.750 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::3ca6:3fff:feb6:2c13/64 scope link 2023-11-28T11:22:31.751 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:22:31.751 INFO:teuthology.orchestra.run.smithi092.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T11:22:31.751 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 4e:8f:e4:5b:18:ce brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T11:22:31.751 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::4c8f:e4ff:fe5b:18ce/64 scope link 2023-11-28T11:22:31.751 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:22:31.751 INFO:teuthology.orchestra.run.smithi092.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T11:22:31.751 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 02:f9:a8:fe:83:2e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T11:22:31.751 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::f9:a8ff:fefe:832e/64 scope link 2023-11-28T11:22:31.751 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:22:31.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:22:31.753 DEBUG:teuthology.orchestra.run.smithi092:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T11:22:31.860 INFO:teuthology.orchestra.run.smithi092.stdout:1 2023-11-28T11:22:31.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:22:31.863 DEBUG:teuthology.orchestra.run.smithi092:> route 2023-11-28T11:22:31.912 INFO:teuthology.orchestra.run.smithi092.stdout:Kernel IP routing table 2023-11-28T11:22:31.913 INFO:teuthology.orchestra.run.smithi092.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T11:22:31.913 INFO:teuthology.orchestra.run.smithi092.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T11:22:31.913 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T11:22:31.913 INFO:teuthology.orchestra.run.smithi092.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T11:22:31.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:22:31.914 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-11-28T11:22:31.914 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-11-28T11:22:31.914 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T11:22:31.914 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T11:22:31.914 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T11:22:31.915 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-11-28T11:22:32.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:22:32.084 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-11-28T11:22:32.131 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T11:22:32.131 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T11:22:32.132 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:22:32.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:22:32.132 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set brx.0 up 2023-11-28T11:22:32.245 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:22:32.245 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-28T11:22:32.291 INFO:teuthology.orchestra.run.smithi092.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:22:32.291 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:22:32.291 DEBUG:teuthology.orchestra.run.smithi092:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-28T11:22:32.377 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-11-28T11:22:32.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:22:32.377 DEBUG:teuthology.orchestra.run.smithi092:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:22:32.522 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:22:32.522 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T11:22:32.523 INFO:teuthology.orchestra.run.smithi092.stdout:invalid new device string format 2023-11-28T11:22:32.538 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:22:32.538 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T11:22:32.538 INFO:teuthology.orchestra.run.smithi092.stdout:invalid new device string format 2023-11-28T11:22:32.539 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.169:3300,172.21.15.169:3301" 2023-11-28T11:22:32.539 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:22:32.563 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:22:32.563 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T11:22:32.563 INFO:teuthology.orchestra.run.smithi092.stdout:invalid new device string format 2023-11-28T11:22:32.563 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.169:3300,172.21.15.169:3301" 2023-11-28T11:22:32.563 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:22:32.563 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:6789,172.21.15.169:6789,172.21.15.169:6790" 2023-11-28T11:22:32.563 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.92:6789,172.21.15.169:6789,172.21.15.169:6790:/ 2023-11-28T11:22:32.563 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=68956e0c-c16f-47cd-a220-f98364b0831d" will pass to kernel 2023-11-28T11:22:32.569 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T11:22:32.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:22:32.569 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T11:22:32.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:22:32.640 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:22:32.740 INFO:teuthology.orchestra.run.smithi092.stdout:client5380 2023-11-28T11:22:32.740 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-28T11:22:32.740 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:22:32.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:22:32.790 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir) 2023-11-28T11:22:32.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:22:32.884 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir/childdir) 2023-11-28T11:22:32.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:22:32.933 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch rfile) 2023-11-28T11:22:32.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:22:32.981 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/pfile) 2023-11-28T11:22:33.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:22:33.033 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/childdir/cfile) 2023-11-28T11:22:33.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:22:33.081 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2023-11-28T11:22:33.134 INFO:teuthology.orchestra.run.smithi092.stdout:./ 2023-11-28T11:22:33.134 INFO:teuthology.orchestra.run.smithi092.stdout:./rfile 2023-11-28T11:22:33.134 INFO:teuthology.orchestra.run.smithi092.stdout:./parentdir 2023-11-28T11:22:33.134 INFO:teuthology.orchestra.run.smithi092.stdout:./parentdir/pfile 2023-11-28T11:22:33.134 INFO:teuthology.orchestra.run.smithi092.stdout:./parentdir/childdir 2023-11-28T11:22:33.134 INFO:teuthology.orchestra.run.smithi092.stdout:./parentdir/childdir/cfile 2023-11-28T11:22:33.135 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T11:22:33.136 DEBUG:teuthology.orchestra.run.smithi092:> import os 2023-11-28T11:22:33.136 DEBUG:teuthology.orchestra.run.smithi092:> import stat 2023-11-28T11:22:33.136 DEBUG:teuthology.orchestra.run.smithi092:> 2023-11-28T11:22:33.136 DEBUG:teuthology.orchestra.run.smithi092:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2023-11-28T11:22:33.136 DEBUG:teuthology.orchestra.run.smithi092:> ' 2023-11-28T11:22:33.249 INFO:teuthology.orchestra.run.smithi092.stdout:1 2023-11-28T11:22:33.434 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T11:22:33.434 DEBUG:teuthology.orchestra.run.smithi092:> import os 2023-11-28T11:22:33.435 DEBUG:teuthology.orchestra.run.smithi092:> import stat 2023-11-28T11:22:33.435 DEBUG:teuthology.orchestra.run.smithi092:> 2023-11-28T11:22:33.435 DEBUG:teuthology.orchestra.run.smithi092:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./rfile").st_ino) 2023-11-28T11:22:33.435 DEBUG:teuthology.orchestra.run.smithi092:> ' 2023-11-28T11:22:33.589 INFO:teuthology.orchestra.run.smithi092.stdout:1099511627778 2023-11-28T11:22:33.774 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T11:22:33.774 DEBUG:teuthology.orchestra.run.smithi092:> import os 2023-11-28T11:22:33.774 DEBUG:teuthology.orchestra.run.smithi092:> import stat 2023-11-28T11:22:33.774 DEBUG:teuthology.orchestra.run.smithi092:> 2023-11-28T11:22:33.774 DEBUG:teuthology.orchestra.run.smithi092:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir").st_ino) 2023-11-28T11:22:33.774 DEBUG:teuthology.orchestra.run.smithi092:> ' 2023-11-28T11:22:33.929 INFO:teuthology.orchestra.run.smithi092.stdout:1099511627776 2023-11-28T11:22:34.114 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T11:22:37.888 DEBUG:teuthology.orchestra.run.smithi092:> import os 2023-11-28T11:22:37.888 DEBUG:teuthology.orchestra.run.smithi092:> import stat 2023-11-28T11:22:37.888 DEBUG:teuthology.orchestra.run.smithi092:> 2023-11-28T11:22:37.888 DEBUG:teuthology.orchestra.run.smithi092:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/pfile").st_ino) 2023-11-28T11:22:37.888 DEBUG:teuthology.orchestra.run.smithi092:> ' 2023-11-28T11:22:38.011 INFO:teuthology.orchestra.run.smithi092.stdout:1099511628279 2023-11-28T11:22:38.196 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T11:22:38.196 DEBUG:teuthology.orchestra.run.smithi092:> import os 2023-11-28T11:22:38.196 DEBUG:teuthology.orchestra.run.smithi092:> import stat 2023-11-28T11:22:38.196 DEBUG:teuthology.orchestra.run.smithi092:> 2023-11-28T11:22:38.196 DEBUG:teuthology.orchestra.run.smithi092:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir").st_ino) 2023-11-28T11:22:38.196 DEBUG:teuthology.orchestra.run.smithi092:> ' 2023-11-28T11:22:38.351 INFO:teuthology.orchestra.run.smithi092.stdout:1099511627777 2023-11-28T11:22:38.537 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T11:22:38.537 DEBUG:teuthology.orchestra.run.smithi092:> import os 2023-11-28T11:22:38.537 DEBUG:teuthology.orchestra.run.smithi092:> import stat 2023-11-28T11:22:38.537 DEBUG:teuthology.orchestra.run.smithi092:> 2023-11-28T11:22:38.537 DEBUG:teuthology.orchestra.run.smithi092:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir/cfile").st_ino) 2023-11-28T11:22:38.537 DEBUG:teuthology.orchestra.run.smithi092:> ' 2023-11-28T11:22:38.690 INFO:teuthology.orchestra.run.smithi092.stdout:1099511628280 2023-11-28T11:22:38.876 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-28T11:22:38.876 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:22:38.963 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-28T11:22:38.963 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:22:39.050 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:22:39.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.287+0000 7fbb865ff700 1 Processor -- start 2023-11-28T11:22:39.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.287+0000 7fbb865ff700 1 -- start start 2023-11-28T11:22:39.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.288+0000 7fbb865ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb800d1d70 0x7fbb80167270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:39.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.288+0000 7fbb865ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb8006c790 con 0x7fbb800cd460 2023-11-28T11:22:39.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.288+0000 7fbb865ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb800b88e0 con 0x7fbb800b8510 2023-11-28T11:22:39.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.288+0000 7fbb865ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb800cdbd0 con 0x7fbb800d1d70 2023-11-28T11:22:39.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.288+0000 7fbb7f7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fbb800cd460 0x7fbb8006c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:38990/0 (socket says 172.21.15.92:38990) 2023-11-28T11:22:39.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.288+0000 7fbb7f7fe700 1 -- 172.21.15.92:0/928507781 learned_addr learned my addr 172.21.15.92:0/928507781 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:22:39.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.289+0000 7fbb84b9c700 1 --2- 172.21.15.92:0/928507781 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb800d1d70 0x7fbb80167270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:39.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.289+0000 7fbb7effd700 1 -- 172.21.15.92:0/928507781 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4208398117 0 0) 0x7fbb8006c790 con 0x7fbb800cd460 2023-11-28T11:22:39.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.289+0000 7fbb7effd700 1 -- 172.21.15.92:0/928507781 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb680035b0 con 0x7fbb800cd460 2023-11-28T11:22:39.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.289+0000 7fbb7effd700 1 -- 172.21.15.92:0/928507781 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3343525104 0 0) 0x7fbb800b88e0 con 0x7fbb800b8510 2023-11-28T11:22:39.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.289+0000 7fbb7effd700 1 -- 172.21.15.92:0/928507781 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb8006c790 con 0x7fbb800b8510 2023-11-28T11:22:39.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.289+0000 7fbb7effd700 1 -- 172.21.15.92:0/928507781 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4003961419 0 0) 0x7fbb680035b0 con 0x7fbb800cd460 2023-11-28T11:22:39.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.289+0000 7fbb7effd700 1 -- 172.21.15.92:0/928507781 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb800d1d70 msgr2=0x7fbb80167270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:39.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.289+0000 7fbb7effd700 1 --2- 172.21.15.92:0/928507781 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb800d1d70 0x7fbb80167270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:39.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.289+0000 7fbb7effd700 1 -- 172.21.15.92:0/928507781 >> v1:172.21.15.169:6789/0 conn(0x7fbb800b8510 legacy=0x7fbb8006b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:39.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.289+0000 7fbb7effd700 1 -- 172.21.15.92:0/928507781 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb800cd8f0 con 0x7fbb800cd460 2023-11-28T11:22:39.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.289+0000 7fbb7effd700 1 -- 172.21.15.92:0/928507781 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532161587 0 0) 0x7fbb74001670 con 0x7fbb800cd460 2023-11-28T11:22:39.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.289+0000 7fbb7effd700 1 -- 172.21.15.92:0/928507781 >> v1:172.21.15.92:6789/0 conn(0x7fbb800cd460 legacy=0x7fbb8006c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:39.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.289+0000 7fbb7effd700 1 --2- 172.21.15.92:0/928507781 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb68003bc0 0x7fbb68003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:39.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.289+0000 7fbb7effd700 1 --2- 172.21.15.92:0/928507781 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb68004850 0x7fbb68004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:39.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.289+0000 7fbb7effd700 1 --2- 172.21.15.92:0/928507781 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb68005200 0x7fbb6800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:39.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.289+0000 7fbb7effd700 1 -- 172.21.15.92:0/928507781 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb800cdbd0 con 0x7fbb68005200 2023-11-28T11:22:39.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.289+0000 7fbb7effd700 1 -- 172.21.15.92:0/928507781 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6800c030 con 0x7fbb68004850 2023-11-28T11:22:39.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.289+0000 7fbb7effd700 1 -- 172.21.15.92:0/928507781 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6800c1f0 con 0x7fbb68003bc0 2023-11-28T11:22:39.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.289+0000 7fbb84b9c700 1 --2- 172.21.15.92:0/928507781 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb800d1d70 0x7fbb80167270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:22:39.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.290+0000 7fbb7f7fe700 1 --2- 172.21.15.92:0/928507781 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb68004850 0x7fbb68004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:39.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.290+0000 7fbb84b9c700 1 --2- 172.21.15.92:0/928507781 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb68005200 0x7fbb6800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:39.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.290+0000 7fbb7ffff700 1 --2- 172.21.15.92:0/928507781 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb68003bc0 0x7fbb68003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:39.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.290+0000 7fbb84b9c700 1 -- 172.21.15.92:0/928507781 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb68003bc0 msgr2=0x7fbb68003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:39.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.290+0000 7fbb84b9c700 1 --2- 172.21.15.92:0/928507781 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb68003bc0 0x7fbb68003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:39.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.290+0000 7fbb84b9c700 1 -- 172.21.15.92:0/928507781 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb68004850 msgr2=0x7fbb68004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:39.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.290+0000 7fbb84b9c700 1 --2- 172.21.15.92:0/928507781 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb68004850 0x7fbb68004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:39.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.290+0000 7fbb84b9c700 1 -- 172.21.15.92:0/928507781 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb680035b0 con 0x7fbb68005200 2023-11-28T11:22:39.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.290+0000 7fbb84b9c700 1 --2- 172.21.15.92:0/928507781 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb68005200 0x7fbb6800ba10 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fbb7800e1f0 tx=0x7fbb68005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd69963f486bc2ee server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:39.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.291+0000 7fbb7effd700 1 -- 172.21.15.92:0/928507781 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7800ec90 con 0x7fbb68005200 2023-11-28T11:22:39.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.291+0000 7fbb7effd700 1 -- 172.21.15.92:0/928507781 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb7800f040 con 0x7fbb68005200 2023-11-28T11:22:39.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.291+0000 7fbb7effd700 1 -- 172.21.15.92:0/928507781 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb78003a20 con 0x7fbb68005200 2023-11-28T11:22:39.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.291+0000 7fbb865ff700 1 -- 172.21.15.92:0/928507781 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb68005200 msgr2=0x7fbb6800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:39.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.291+0000 7fbb865ff700 1 --2- 172.21.15.92:0/928507781 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb68005200 0x7fbb6800ba10 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fbb7800e1f0 tx=0x7fbb68005880 comp rx=0 tx=0).stop 2023-11-28T11:22:39.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.291+0000 7fbb84b9c700 1 -- 172.21.15.92:0/928507781 reap_dead start 2023-11-28T11:22:39.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.291+0000 7fbb865ff700 1 -- 172.21.15.92:0/928507781 shutdown_connections 2023-11-28T11:22:39.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.291+0000 7fbb865ff700 1 -- 172.21.15.92:0/928507781 >> 172.21.15.92:0/928507781 conn(0x7fbb800719a0 msgr2=0x7fbb80071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:39.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.291+0000 7fbb865ff700 1 -- 172.21.15.92:0/928507781 shutdown_connections 2023-11-28T11:22:39.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.291+0000 7fbb865ff700 1 -- 172.21.15.92:0/928507781 wait complete. 2023-11-28T11:22:39.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.293+0000 7fbb865ff700 1 Processor -- start 2023-11-28T11:22:39.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.293+0000 7fbb865ff700 1 -- start start 2023-11-28T11:22:39.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.294+0000 7fbb865ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb68005200 0x7fbb8015b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:39.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.294+0000 7fbb865ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb80154230 0x7fbb800c9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:39.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.294+0000 7fbb865ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb801f2270 0x7fbb801f26c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:39.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.294+0000 7fbb865ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb80158a10 con 0x7fbb80154230 2023-11-28T11:22:39.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.294+0000 7fbb865ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb80156ba0 con 0x7fbb801f2270 2023-11-28T11:22:39.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.294+0000 7fbb865ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb80156e40 con 0x7fbb68005200 2023-11-28T11:22:39.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.294+0000 7fbb7f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb80154230 0x7fbb800c9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:39.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.295+0000 7fbb7f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb80154230 0x7fbb800c9110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52844/0 (socket says 172.21.15.92:52844) 2023-11-28T11:22:39.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.295+0000 7fbb7f7fe700 1 -- 172.21.15.92:0/1099274779 learned_addr learned my addr 172.21.15.92:0/1099274779 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:39.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.295+0000 7fbb84b9c700 1 --2- 172.21.15.92:0/1099274779 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb801f2270 0x7fbb801f26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:39.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.295+0000 7fbb7ffff700 1 --2- 172.21.15.92:0/1099274779 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb68005200 0x7fbb8015b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:39.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.295+0000 7fbb7f7fe700 1 -- 172.21.15.92:0/1099274779 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb68005200 msgr2=0x7fbb8015b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:39.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.295+0000 7fbb7f7fe700 1 --2- 172.21.15.92:0/1099274779 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb68005200 0x7fbb8015b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:39.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.295+0000 7fbb7f7fe700 1 -- 172.21.15.92:0/1099274779 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb801f2270 msgr2=0x7fbb801f26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:39.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.295+0000 7fbb7f7fe700 1 --2- 172.21.15.92:0/1099274779 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb801f2270 0x7fbb801f26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:39.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.295+0000 7fbb7f7fe700 1 -- 172.21.15.92:0/1099274779 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb800cfff0 con 0x7fbb80154230 2023-11-28T11:22:39.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.295+0000 7fbb84b9c700 1 --2- 172.21.15.92:0/1099274779 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb801f2270 0x7fbb801f26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:22:39.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.295+0000 7fbb7ffff700 1 --2- 172.21.15.92:0/1099274779 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb68005200 0x7fbb8015b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:22:39.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.295+0000 7fbb7f7fe700 1 --2- 172.21.15.92:0/1099274779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb80154230 0x7fbb800c9110 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fbb740072c0 tx=0x7fbb740075d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:39.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.295+0000 7fbb7cff9700 1 -- 172.21.15.92:0/1099274779 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb740077b0 con 0x7fbb80154230 2023-11-28T11:22:39.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.295+0000 7fbb7cff9700 1 -- 172.21.15.92:0/1099274779 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb7400f850 con 0x7fbb80154230 2023-11-28T11:22:39.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.295+0000 7fbb865ff700 1 -- 172.21.15.92:0/1099274779 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb800cfd10 con 0x7fbb80154230 2023-11-28T11:22:39.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.295+0000 7fbb7cff9700 1 -- 172.21.15.92:0/1099274779 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7400fc00 con 0x7fbb80154230 2023-11-28T11:22:39.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.295+0000 7fbb865ff700 1 -- 172.21.15.92:0/1099274779 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb800d29c0 con 0x7fbb80154230 2023-11-28T11:22:39.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.296+0000 7fbb865ff700 1 -- 172.21.15.92:0/1099274779 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb80067880 con 0x7fbb80154230 2023-11-28T11:22:39.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.297+0000 7fbb7cff9700 1 -- 172.21.15.92:0/1099274779 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbb7400fd60 con 0x7fbb80154230 2023-11-28T11:22:39.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.298+0000 7fbb7cff9700 1 --2- 172.21.15.92:0/1099274779 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fbb6c0682f0 0x7fbb6c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:39.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.298+0000 7fbb7cff9700 1 -- 172.21.15.92:0/1099274779 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fbb74086100 con 0x7fbb80154230 2023-11-28T11:22:39.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.298+0000 7fbb7ffff700 1 --2- 172.21.15.92:0/1099274779 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fbb6c0682f0 0x7fbb6c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:39.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.299+0000 7fbb7ffff700 1 --2- 172.21.15.92:0/1099274779 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fbb6c0682f0 0x7fbb6c06a7a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbb70000c00 tx=0x7fbb70009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:39.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.302+0000 7fbb7cff9700 1 -- 172.21.15.92:0/1099274779 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbb74052710 con 0x7fbb80154230 2023-11-28T11:22:39.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.495+0000 7fbb865ff700 1 -- 172.21.15.92:0/1099274779 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbb8015d150 con 0x7fbb80154230 2023-11-28T11:22:39.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.496+0000 7fbb7cff9700 1 -- 172.21.15.92:0/1099274779 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+18279 (secure 0 0 0) 0x7fbb74055d30 con 0x7fbb80154230 2023-11-28T11:22:39.497 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:22:39.498 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":44,"fsid":"68956e0c-c16f-47cd-a220-f98364b0831d","created":"2023-11-28T11:20:28.697573+0000","modified":"2023-11-28T11:22:38.531199+0000","last_up_change":"2023-11-28T11:20:41.257011+0000","last_in_change":"2023-11-28T11:20:32.249767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:20:41.925237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:20:48.682497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:22:22.942125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-28T11:22:23.684528+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6abe483f-0e03-4a26-b9cf-62041291747e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":115823},{"type":"v1","addr":"172.21.15.92:6809","nonce":115823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":115823},{"type":"v1","addr":"172.21.15.92:6813","nonce":115823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":115823},{"type":"v1","addr":"172.21.15.92:6821","nonce":115823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":115823},{"type":"v1","addr":"172.21.15.92:6817","nonce":115823}]},"public_addr":"172.21.15.92:6809/115823","cluster_addr":"172.21.15.92:6813/115823","heartbeat_back_addr":"172.21.15.92:6821/115823","heartbeat_front_addr":"172.21.15.92:6817/115823","state":["exists","up"]},{"osd":1,"uuid":"13c1de73-1a8c-4d7e-8a28-9dae52dfe089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":115824},{"type":"v1","addr":"172.21.15.92:6812","nonce":115824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":115824},{"type":"v1","addr":"172.21.15.92:6816","nonce":115824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":115824},{"type":"v1","addr":"172.21.15.92:6823","nonce":115824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":115824},{"type":"v1","addr":"172.21.15.92:6820","nonce":115824}]},"public_addr":"172.21.15.92:6812/115824","cluster_addr":"172.21.15.92:6816/115824","heartbeat_back_addr":"172.21.15.92:6823/115824","heartbeat_front_addr":"172.21.15.92:6820/115824","state":["exists","up"]},{"osd":2,"uuid":"25788a55-112f-4f00-810d-2bc37cf76778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":115821},{"type":"v1","addr":"172.21.15.92:6801","nonce":115821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":115821},{"type":"v1","addr":"172.21.15.92:6803","nonce":115821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":115821},{"type":"v1","addr":"172.21.15.92:6807","nonce":115821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":115821},{"type":"v1","addr":"172.21.15.92:6805","nonce":115821}]},"public_addr":"172.21.15.92:6801/115821","cluster_addr":"172.21.15.92:6803/115821","heartbeat_back_addr":"172.21.15.92:6807/115821","heartbeat_front_addr":"172.21.15.92:6805/115821","state":["exists","up"]},{"osd":3,"uuid":"adf54e24-33a9-41fe-9ce3-98a5ab6c158e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":115825},{"type":"v1","addr":"172.21.15.92:6825","nonce":115825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":115825},{"type":"v1","addr":"172.21.15.92:6827","nonce":115825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":115825},{"type":"v1","addr":"172.21.15.92:6831","nonce":115825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":115825},{"type":"v1","addr":"172.21.15.92:6829","nonce":115825}]},"public_addr":"172.21.15.92:6825/115825","cluster_addr":"172.21.15.92:6827/115825","heartbeat_back_addr":"172.21.15.92:6831/115825","heartbeat_front_addr":"172.21.15.92:6829/115825","state":["exists","up"]},{"osd":4,"uuid":"ccb9b747-f399-44ea-9a18-9db3152227e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":116035},{"type":"v1","addr":"172.21.15.169:6817","nonce":116035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":116035},{"type":"v1","addr":"172.21.15.169:6819","nonce":116035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":116035},{"type":"v1","addr":"172.21.15.169:6826","nonce":116035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":116035},{"type":"v1","addr":"172.21.15.169:6822","nonce":116035}]},"public_addr":"172.21.15.169:6817/116035","cluster_addr":"172.21.15.169:6819/116035","heartbeat_back_addr":"172.21.15.169:6826/116035","heartbeat_front_addr":"172.21.15.169:6822/116035","state":["exists","up"]},{"osd":5,"uuid":"308a1e63-75aa-4d7d-8a7c-cecc24714fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":116033},{"type":"v1","addr":"172.21.15.169:6801","nonce":116033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":116033},{"type":"v1","addr":"172.21.15.169:6803","nonce":116033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":116033},{"type":"v1","addr":"172.21.15.169:6807","nonce":116033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":116033},{"type":"v1","addr":"172.21.15.169:6805","nonce":116033}]},"public_addr":"172.21.15.169:6801/116033","cluster_addr":"172.21.15.169:6803/116033","heartbeat_back_addr":"172.21.15.169:6807/116033","heartbeat_front_addr":"172.21.15.169:6805/116033","state":["exists","up"]},{"osd":6,"uuid":"9c459b7b-b821-4b4b-b91e-6aaae271457c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":116034},{"type":"v1","addr":"172.21.15.169:6809","nonce":116034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":116034},{"type":"v1","addr":"172.21.15.169:6811","nonce":116034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":116034},{"type":"v1","addr":"172.21.15.169:6815","nonce":116034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":116034},{"type":"v1","addr":"172.21.15.169:6813","nonce":116034}]},"public_addr":"172.21.15.169:6809/116034","cluster_addr":"172.21.15.169:6811/116034","heartbeat_back_addr":"172.21.15.169:6815/116034","heartbeat_front_addr":"172.21.15.169:6813/116034","state":["exists","up"]},{"osd":7,"uuid":"1ca9aa4a-755d-4e28-a937-4d8dccab4df3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":116044},{"type":"v1","addr":"172.21.15.169:6823","nonce":116044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6825","nonce":116044},{"type":"v1","addr":"172.21.15.169:6827","nonce":116044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":116044},{"type":"v1","addr":"172.21.15.169:6831","nonce":116044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":116044},{"type":"v1","addr":"172.21.15.169:6829","nonce":116044}]},"public_addr":"172.21.15.169:6823/116044","cluster_addr":"172.21.15.169:6827/116044","heartbeat_back_addr":"172.21.15.169:6831/116044","heartbeat_front_addr":"172.21.15.169:6829/116044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.586781+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.869778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.508594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.860493+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.872267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.746933+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.611520+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.871446+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.f","mappings":[{"from":0,"to":4}]},{"pgid":"7.3e","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:22:39.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.499+0000 7fbb865ff700 1 -- 172.21.15.92:0/1099274779 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fbb6c0682f0 msgr2=0x7fbb6c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:39.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.499+0000 7fbb865ff700 1 --2- 172.21.15.92:0/1099274779 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fbb6c0682f0 0x7fbb6c06a7a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbb70000c00 tx=0x7fbb70009040 comp rx=0 tx=0).stop 2023-11-28T11:22:39.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.499+0000 7fbb865ff700 1 -- 172.21.15.92:0/1099274779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb80154230 msgr2=0x7fbb800c9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:39.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.499+0000 7fbb865ff700 1 --2- 172.21.15.92:0/1099274779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb80154230 0x7fbb800c9110 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fbb740072c0 tx=0x7fbb740075d0 comp rx=0 tx=0).stop 2023-11-28T11:22:39.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.499+0000 7fbb865ff700 1 -- 172.21.15.92:0/1099274779 shutdown_connections 2023-11-28T11:22:39.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.499+0000 7fbb865ff700 1 --2- 172.21.15.92:0/1099274779 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb801f2270 0x7fbb801f26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:39.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.499+0000 7fbb865ff700 1 --2- 172.21.15.92:0/1099274779 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb68005200 0x7fbb8015b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:39.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.499+0000 7fbb865ff700 1 --2- 172.21.15.92:0/1099274779 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fbb6c0682f0 0x7fbb6c06a7a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:39.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.499+0000 7fbb865ff700 1 --2- 172.21.15.92:0/1099274779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb80154230 0x7fbb800c9110 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:39.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.499+0000 7fbb865ff700 1 -- 172.21.15.92:0/1099274779 >> 172.21.15.92:0/1099274779 conn(0x7fbb800719a0 msgr2=0x7fbb800cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:39.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.499+0000 7fbb865ff700 1 -- 172.21.15.92:0/1099274779 shutdown_connections 2023-11-28T11:22:39.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.499+0000 7fbb865ff700 1 -- 172.21.15.92:0/1099274779 wait complete. 2023-11-28T11:22:39.518 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:20:41.925237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:20:48.682497+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:22:22.942125+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:22:23.684528+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:22:39.518 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:22:39.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.717+0000 7f26631ff700 1 Processor -- start 2023-11-28T11:22:39.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.717+0000 7f26631ff700 1 -- start start 2023-11-28T11:22:39.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.718+0000 7f26631ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f265c0b8510 0x7f265c06b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:39.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.718+0000 7f26631ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f265c0cd460 0x7f265c06beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:39.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.718+0000 7f26631ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f265c0b88e0 con 0x7f265c06c3f0 2023-11-28T11:22:39.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.718+0000 7f26631ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f265c0cdbd0 con 0x7f265c0cd460 2023-11-28T11:22:39.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.718+0000 7f26631ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f265c06c830 con 0x7f265c0b8510 2023-11-28T11:22:39.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.718+0000 7f265bfff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f265c0cd460 0x7f265c06beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:39.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.718+0000 7f2660f9b700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f265c0b8510 0x7f265c06b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:39.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.718+0000 7f265bfff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f265c0cd460 0x7f265c06beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.92:51858/0 (socket says 172.21.15.92:51858) 2023-11-28T11:22:39.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.718+0000 7f265bfff700 1 -- 172.21.15.92:0/2186627520 learned_addr learned my addr 172.21.15.92:0/2186627520 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:39.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.719+0000 7f2660f9b700 1 -- 172.21.15.92:0/2186627520 >> v2:172.21.15.169:3300/0 conn(0x7f265c0cd460 msgr2=0x7f265c06beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:39.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.719+0000 7f2660f9b700 1 --2- 172.21.15.92:0/2186627520 >> v2:172.21.15.169:3300/0 conn(0x7f265c0cd460 0x7f265c06beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:39.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.719+0000 7f2660f9b700 1 -- 172.21.15.92:0/2186627520 >> v1:172.21.15.169:6789/0 conn(0x7f265c06c3f0 legacy=0x7f265c15cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:39.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.719+0000 7f2660f9b700 1 -- 172.21.15.92:0/2186627520 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f265c0cd8f0 con 0x7f265c0b8510 2023-11-28T11:22:39.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.719+0000 7f2660f9b700 1 --2- 172.21.15.92:0/2186627520 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f265c0b8510 0x7f265c06b970 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2650009840 tx=0x7f2650009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41befa43bb462c0c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:39.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.719+0000 7f265b7fe700 1 -- 172.21.15.92:0/2186627520 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2650011830 con 0x7f265c0b8510 2023-11-28T11:22:39.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.719+0000 7f265b7fe700 1 -- 172.21.15.92:0/2186627520 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2650011990 con 0x7f265c0b8510 2023-11-28T11:22:39.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.719+0000 7f265b7fe700 1 -- 172.21.15.92:0/2186627520 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2650011d20 con 0x7f265c0b8510 2023-11-28T11:22:39.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.720+0000 7f26631ff700 1 -- 172.21.15.92:0/2186627520 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f265c0b8510 msgr2=0x7f265c06b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:39.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.720+0000 7f26631ff700 1 --2- 172.21.15.92:0/2186627520 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f265c0b8510 0x7f265c06b970 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2650009840 tx=0x7f2650009b50 comp rx=0 tx=0).stop 2023-11-28T11:22:39.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.720+0000 7f26631ff700 1 -- 172.21.15.92:0/2186627520 shutdown_connections 2023-11-28T11:22:39.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.720+0000 7f26631ff700 1 --2- 172.21.15.92:0/2186627520 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f265c0b8510 0x7f265c06b970 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:39.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.720+0000 7f26631ff700 1 --2- 172.21.15.92:0/2186627520 >> v2:172.21.15.169:3300/0 conn(0x7f265c0cd460 0x7f265c06beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:39.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.720+0000 7f26631ff700 1 -- 172.21.15.92:0/2186627520 >> 172.21.15.92:0/2186627520 conn(0x7f265c0719a0 msgr2=0x7f265c071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:39.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.720+0000 7f26631ff700 1 -- 172.21.15.92:0/2186627520 shutdown_connections 2023-11-28T11:22:39.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.720+0000 7f26631ff700 1 -- 172.21.15.92:0/2186627520 wait complete. 2023-11-28T11:22:39.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.722+0000 7f26631ff700 1 Processor -- start 2023-11-28T11:22:39.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.722+0000 7f26631ff700 1 -- start start 2023-11-28T11:22:39.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.723+0000 7f26631ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f265c06c3f0 0x7f265c15b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:39.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.723+0000 7f26631ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f265c0b8510 0x7f265c15a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:39.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.723+0000 7f26631ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f265c0cd460 0x7f265c15aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:39.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.723+0000 7f26631ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f265c1e7e10 con 0x7f265c06c3f0 2023-11-28T11:22:39.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.723+0000 7f26631ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f265c1e80f0 con 0x7f265c0cd460 2023-11-28T11:22:39.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.723+0000 7f26631ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f265c1f22d0 con 0x7f265c0b8510 2023-11-28T11:22:39.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.723+0000 7f2660f9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f265c06c3f0 0x7f265c15b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:39.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.723+0000 7f2660f9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f265c06c3f0 0x7f265c15b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52854/0 (socket says 172.21.15.92:52854) 2023-11-28T11:22:39.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.723+0000 7f2660f9b700 1 -- 172.21.15.92:0/1032701720 learned_addr learned my addr 172.21.15.92:0/1032701720 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:39.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.723+0000 7f265bfff700 1 --2- 172.21.15.92:0/1032701720 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f265c0b8510 0x7f265c15a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:39.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.723+0000 7f266179c700 1 --2- 172.21.15.92:0/1032701720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f265c0cd460 0x7f265c15aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:39.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.723+0000 7f2660f9b700 1 -- 172.21.15.92:0/1032701720 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f265c0b8510 msgr2=0x7f265c15a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:39.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.723+0000 7f2660f9b700 1 --2- 172.21.15.92:0/1032701720 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f265c0b8510 0x7f265c15a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:39.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.723+0000 7f2660f9b700 1 -- 172.21.15.92:0/1032701720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f265c0cd460 msgr2=0x7f265c15aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:39.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.723+0000 7f2660f9b700 1 --2- 172.21.15.92:0/1032701720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f265c0cd460 0x7f265c15aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:39.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.723+0000 7f2660f9b700 1 -- 172.21.15.92:0/1032701720 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26500094a0 con 0x7f265c06c3f0 2023-11-28T11:22:39.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.723+0000 7f265bfff700 1 --2- 172.21.15.92:0/1032701720 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f265c0b8510 0x7f265c15a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:22:39.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.724+0000 7f2660f9b700 1 --2- 172.21.15.92:0/1032701720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f265c06c3f0 0x7f265c15b0b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2650009fd0 tx=0x7f2650004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:39.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.724+0000 7f26597fa700 1 -- 172.21.15.92:0/1032701720 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26500038a0 con 0x7f265c06c3f0 2023-11-28T11:22:39.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.724+0000 7f26597fa700 1 -- 172.21.15.92:0/1032701720 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f265001f070 con 0x7f265c06c3f0 2023-11-28T11:22:39.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.724+0000 7f26631ff700 1 -- 172.21.15.92:0/1032701720 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f265c0cfff0 con 0x7f265c06c3f0 2023-11-28T11:22:39.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.724+0000 7f26597fa700 1 -- 172.21.15.92:0/1032701720 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2650003c40 con 0x7f265c06c3f0 2023-11-28T11:22:39.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.724+0000 7f26631ff700 1 -- 172.21.15.92:0/1032701720 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f265c0cfd10 con 0x7f265c06c3f0 2023-11-28T11:22:39.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.725+0000 7f26631ff700 1 -- 172.21.15.92:0/1032701720 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f265c067880 con 0x7f265c06c3f0 2023-11-28T11:22:39.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.726+0000 7f26597fa700 1 -- 172.21.15.92:0/1032701720 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f265000f050 con 0x7f265c06c3f0 2023-11-28T11:22:39.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.727+0000 7f26597fa700 1 --2- 172.21.15.92:0/1032701720 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f26400682f0 0x7f264006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:39.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.727+0000 7f26597fa700 1 -- 172.21.15.92:0/1032701720 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f2650021070 con 0x7f265c06c3f0 2023-11-28T11:22:39.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.727+0000 7f265bfff700 1 --2- 172.21.15.92:0/1032701720 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f26400682f0 0x7f264006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:39.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.728+0000 7f265bfff700 1 --2- 172.21.15.92:0/1032701720 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f26400682f0 0x7f264006a7a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f265c0b3e60 tx=0x7f2648008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:39.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.731+0000 7f26597fa700 1 -- 172.21.15.92:0/1032701720 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2650053ef0 con 0x7f265c06c3f0 2023-11-28T11:22:39.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.924+0000 7f26631ff700 1 -- 172.21.15.92:0/1032701720 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f265c0d2940 con 0x7f265c06c3f0 2023-11-28T11:22:39.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.925+0000 7f26597fa700 1 -- 172.21.15.92:0/1032701720 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v44) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2650015020 con 0x7f265c06c3f0 2023-11-28T11:22:39.926 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-11-28T11:22:39.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.927+0000 7f26631ff700 1 -- 172.21.15.92:0/1032701720 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f26400682f0 msgr2=0x7f264006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:39.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.927+0000 7f26631ff700 1 --2- 172.21.15.92:0/1032701720 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f26400682f0 0x7f264006a7a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f265c0b3e60 tx=0x7f2648008040 comp rx=0 tx=0).stop 2023-11-28T11:22:39.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.927+0000 7f26631ff700 1 -- 172.21.15.92:0/1032701720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f265c06c3f0 msgr2=0x7f265c15b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:39.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.927+0000 7f26631ff700 1 --2- 172.21.15.92:0/1032701720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f265c06c3f0 0x7f265c15b0b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2650009fd0 tx=0x7f2650004750 comp rx=0 tx=0).stop 2023-11-28T11:22:39.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.927+0000 7f26631ff700 1 -- 172.21.15.92:0/1032701720 shutdown_connections 2023-11-28T11:22:39.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.927+0000 7f26631ff700 1 --2- 172.21.15.92:0/1032701720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f265c0cd460 0x7f265c15aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:39.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.927+0000 7f26631ff700 1 --2- 172.21.15.92:0/1032701720 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f26400682f0 0x7f264006a7a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:39.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.927+0000 7f26631ff700 1 --2- 172.21.15.92:0/1032701720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f265c06c3f0 0x7f265c15b0b0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:39.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.927+0000 7f26631ff700 1 --2- 172.21.15.92:0/1032701720 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f265c0b8510 0x7f265c15a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:39.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.927+0000 7f26631ff700 1 -- 172.21.15.92:0/1032701720 >> 172.21.15.92:0/1032701720 conn(0x7f265c0719a0 msgr2=0x7f265c0b5e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:39.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.927+0000 7f26631ff700 1 -- 172.21.15.92:0/1032701720 shutdown_connections 2023-11-28T11:22:39.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:39.927+0000 7f26631ff700 1 -- 172.21.15.92:0/1032701720 wait complete. 2023-11-28T11:22:39.946 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:22:40.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.144+0000 7f5a7a1ff700 1 Processor -- start 2023-11-28T11:22:40.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.145+0000 7f5a7a1ff700 1 -- start start 2023-11-28T11:22:40.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.146+0000 7f5a7a1ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5a740b8510 0x7f5a7406b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:40.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.146+0000 7f5a7a1ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a740d0d60 0x7f5a740d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:40.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.146+0000 7f5a7a1ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a740b88e0 con 0x7f5a740cd460 2023-11-28T11:22:40.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.146+0000 7f5a7a1ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a740cdbd0 con 0x7f5a740b8510 2023-11-28T11:22:40.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.146+0000 7f5a7a1ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a740d02d0 con 0x7f5a740d0d60 2023-11-28T11:22:40.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.146+0000 7f5a72ffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f5a740cd460 0x7f5a7406beb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:38998/0 (socket says 172.21.15.92:38998) 2023-11-28T11:22:40.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.146+0000 7f5a72ffd700 1 -- 172.21.15.92:0/3343290368 learned_addr learned my addr 172.21.15.92:0/3343290368 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:22:40.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.146+0000 7f5a737fe700 1 --2- 172.21.15.92:0/3343290368 >> v2:172.21.15.169:3300/0 conn(0x7f5a740b8510 0x7f5a7406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:40.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.146+0000 7f5a73fff700 1 --2- 172.21.15.92:0/3343290368 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a740d0d60 0x7f5a740d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:40.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.146+0000 7f5a727fc700 1 -- 172.21.15.92:0/3343290368 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2607132170 0 0) 0x7f5a740b88e0 con 0x7f5a740cd460 2023-11-28T11:22:40.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.146+0000 7f5a727fc700 1 -- 172.21.15.92:0/3343290368 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a5c0035b0 con 0x7f5a740cd460 2023-11-28T11:22:40.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.147+0000 7f5a727fc700 1 -- 172.21.15.92:0/3343290368 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2350357898 0 0) 0x7f5a5c0035b0 con 0x7f5a740cd460 2023-11-28T11:22:40.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.147+0000 7f5a727fc700 1 -- 172.21.15.92:0/3343290368 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a740d0d60 msgr2=0x7f5a740d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:40.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.147+0000 7f5a727fc700 1 --2- 172.21.15.92:0/3343290368 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a740d0d60 0x7f5a740d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:40.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.147+0000 7f5a727fc700 1 -- 172.21.15.92:0/3343290368 >> v2:172.21.15.169:3300/0 conn(0x7f5a740b8510 msgr2=0x7f5a7406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:40.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.147+0000 7f5a727fc700 1 --2- 172.21.15.92:0/3343290368 >> v2:172.21.15.169:3300/0 conn(0x7f5a740b8510 0x7f5a7406b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:40.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.147+0000 7f5a727fc700 1 -- 172.21.15.92:0/3343290368 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a740cd8f0 con 0x7f5a740cd460 2023-11-28T11:22:40.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.147+0000 7f5a727fc700 1 -- 172.21.15.92:0/3343290368 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532161587 0 0) 0x7f5a68001670 con 0x7f5a740cd460 2023-11-28T11:22:40.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.147+0000 7f5a727fc700 1 -- 172.21.15.92:0/3343290368 >> v1:172.21.15.92:6789/0 conn(0x7f5a740cd460 legacy=0x7f5a7406beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:40.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.147+0000 7f5a727fc700 1 --2- 172.21.15.92:0/3343290368 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a740d0d60 0x7f5a5c003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:40.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.147+0000 7f5a727fc700 1 --2- 172.21.15.92:0/3343290368 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a5c003fb0 0x7f5a5c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:40.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.147+0000 7f5a727fc700 1 --2- 172.21.15.92:0/3343290368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a5c004d90 0x7f5a5c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:40.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.147+0000 7f5a727fc700 1 -- 172.21.15.92:0/3343290368 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a740cdbd0 con 0x7f5a5c004d90 2023-11-28T11:22:40.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.147+0000 7f5a727fc700 1 -- 172.21.15.92:0/3343290368 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a740d02d0 con 0x7f5a5c003fb0 2023-11-28T11:22:40.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.147+0000 7f5a727fc700 1 -- 172.21.15.92:0/3343290368 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5c00ba10 con 0x7f5a740d0d60 2023-11-28T11:22:40.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.147+0000 7f5a73fff700 1 --2- 172.21.15.92:0/3343290368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a5c004d90 0x7f5a5c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:40.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.147+0000 7f5a72ffd700 1 --2- 172.21.15.92:0/3343290368 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a5c003fb0 0x7f5a5c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:40.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.147+0000 7f5a737fe700 1 --2- 172.21.15.92:0/3343290368 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a740d0d60 0x7f5a5c003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:40.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.147+0000 7f5a73fff700 1 -- 172.21.15.92:0/3343290368 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a740d0d60 msgr2=0x7f5a5c003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:40.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.147+0000 7f5a73fff700 1 --2- 172.21.15.92:0/3343290368 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a740d0d60 0x7f5a5c003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:40.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.148+0000 7f5a73fff700 1 -- 172.21.15.92:0/3343290368 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a5c003fb0 msgr2=0x7f5a5c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:40.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.148+0000 7f5a73fff700 1 --2- 172.21.15.92:0/3343290368 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a5c003fb0 0x7f5a5c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:40.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.148+0000 7f5a73fff700 1 -- 172.21.15.92:0/3343290368 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a5c0035b0 con 0x7f5a5c004d90 2023-11-28T11:22:40.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.148+0000 7f5a73fff700 1 --2- 172.21.15.92:0/3343290368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a5c004d90 0x7f5a5c005200 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c009fc0 tx=0x7f5a5c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4965aa3f3d82c518 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:40.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.148+0000 7f5a727fc700 1 -- 172.21.15.92:0/3343290368 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c00ec30 con 0x7f5a5c004d90 2023-11-28T11:22:40.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.148+0000 7f5a727fc700 1 -- 172.21.15.92:0/3343290368 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a6c00f040 con 0x7f5a5c004d90 2023-11-28T11:22:40.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.148+0000 7f5a727fc700 1 -- 172.21.15.92:0/3343290368 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c014480 con 0x7f5a5c004d90 2023-11-28T11:22:40.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.148+0000 7f5a7a1ff700 1 -- 172.21.15.92:0/3343290368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a5c004d90 msgr2=0x7f5a5c005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:40.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.148+0000 7f5a7a1ff700 1 --2- 172.21.15.92:0/3343290368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a5c004d90 0x7f5a5c005200 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c009fc0 tx=0x7f5a5c005880 comp rx=0 tx=0).stop 2023-11-28T11:22:40.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.148+0000 7f5a73fff700 1 -- 172.21.15.92:0/3343290368 reap_dead start 2023-11-28T11:22:40.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.149+0000 7f5a7a1ff700 1 -- 172.21.15.92:0/3343290368 shutdown_connections 2023-11-28T11:22:40.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.149+0000 7f5a7a1ff700 1 -- 172.21.15.92:0/3343290368 >> 172.21.15.92:0/3343290368 conn(0x7f5a740719a0 msgr2=0x7f5a74071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:40.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.149+0000 7f5a7a1ff700 1 -- 172.21.15.92:0/3343290368 shutdown_connections 2023-11-28T11:22:40.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.149+0000 7f5a7a1ff700 1 -- 172.21.15.92:0/3343290368 wait complete. 2023-11-28T11:22:40.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.150+0000 7f5a7a1ff700 1 Processor -- start 2023-11-28T11:22:40.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.151+0000 7f5a7a1ff700 1 -- start start 2023-11-28T11:22:40.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.151+0000 7f5a7a1ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a5c004d90 0x7f5a74154230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:40.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.151+0000 7f5a7a1ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a74154770 0x7f5a740b6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:40.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.151+0000 7f5a7a1ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a740b7260 0x7f5a741596d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:40.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.151+0000 7f5a7a1ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a740b76d0 con 0x7f5a740b7260 2023-11-28T11:22:40.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.151+0000 7f5a7a1ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a74159240 con 0x7f5a5c004d90 2023-11-28T11:22:40.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.151+0000 7f5a7a1ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a74158670 con 0x7f5a74154770 2023-11-28T11:22:40.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.152+0000 7f5a73fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a740b7260 0x7f5a741596d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:40.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.152+0000 7f5a73fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a740b7260 0x7f5a741596d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52884/0 (socket says 172.21.15.92:52884) 2023-11-28T11:22:40.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.152+0000 7f5a73fff700 1 -- 172.21.15.92:0/2417458205 learned_addr learned my addr 172.21.15.92:0/2417458205 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:40.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.152+0000 7f5a737fe700 1 --2- 172.21.15.92:0/2417458205 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a5c004d90 0x7f5a74154230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:40.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.152+0000 7f5a72ffd700 1 --2- 172.21.15.92:0/2417458205 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a74154770 0x7f5a740b6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:40.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.152+0000 7f5a73fff700 1 -- 172.21.15.92:0/2417458205 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a74154770 msgr2=0x7f5a740b6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:40.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.152+0000 7f5a73fff700 1 --2- 172.21.15.92:0/2417458205 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a74154770 0x7f5a740b6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:40.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.152+0000 7f5a73fff700 1 -- 172.21.15.92:0/2417458205 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a5c004d90 msgr2=0x7f5a74154230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:40.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.152+0000 7f5a73fff700 1 --2- 172.21.15.92:0/2417458205 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a5c004d90 0x7f5a74154230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:40.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.152+0000 7f5a73fff700 1 -- 172.21.15.92:0/2417458205 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a740cfff0 con 0x7f5a740b7260 2023-11-28T11:22:40.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.152+0000 7f5a737fe700 1 --2- 172.21.15.92:0/2417458205 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a5c004d90 0x7f5a74154230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:22:40.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.152+0000 7f5a73fff700 1 --2- 172.21.15.92:0/2417458205 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a740b7260 0x7f5a741596d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c017040 tx=0x7f5a6c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:40.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.152+0000 7f5a5bfff700 1 -- 172.21.15.92:0/2417458205 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c007000 con 0x7f5a740b7260 2023-11-28T11:22:40.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.152+0000 7f5a5bfff700 1 -- 172.21.15.92:0/2417458205 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a6c003e50 con 0x7f5a740b7260 2023-11-28T11:22:40.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.152+0000 7f5a7a1ff700 1 -- 172.21.15.92:0/2417458205 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a740cfd10 con 0x7f5a740b7260 2023-11-28T11:22:40.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.152+0000 7f5a5bfff700 1 -- 172.21.15.92:0/2417458205 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c014b50 con 0x7f5a740b7260 2023-11-28T11:22:40.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.153+0000 7f5a7a1ff700 1 -- 172.21.15.92:0/2417458205 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a740d29c0 con 0x7f5a740b7260 2023-11-28T11:22:40.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.154+0000 7f5a7a1ff700 1 -- 172.21.15.92:0/2417458205 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a74067880 con 0x7f5a740b7260 2023-11-28T11:22:40.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.154+0000 7f5a5bfff700 1 -- 172.21.15.92:0/2417458205 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5a6c02d070 con 0x7f5a740b7260 2023-11-28T11:22:40.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.155+0000 7f5a5bfff700 1 --2- 172.21.15.92:0/2417458205 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5a600682f0 0x7f5a6006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:40.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.155+0000 7f5a5bfff700 1 -- 172.21.15.92:0/2417458205 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f5a6c0875e0 con 0x7f5a740b7260 2023-11-28T11:22:40.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.156+0000 7f5a737fe700 1 --2- 172.21.15.92:0/2417458205 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5a600682f0 0x7f5a6006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:40.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.156+0000 7f5a737fe700 1 --2- 172.21.15.92:0/2417458205 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5a600682f0 0x7f5a6006a7a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5a740d3b10 tx=0x7f5a64008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:40.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.160+0000 7f5a5bfff700 1 -- 172.21.15.92:0/2417458205 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5a6c053bd0 con 0x7f5a740b7260 2023-11-28T11:22:40.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.355+0000 7f5a7a1ff700 1 -- 172.21.15.92:0/2417458205 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5a740d1690 con 0x7f5a740b7260 2023-11-28T11:22:40.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.355+0000 7f5a5bfff700 1 -- 172.21.15.92:0/2417458205 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v44) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5a6c0571f0 con 0x7f5a740b7260 2023-11-28T11:22:40.356 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-11-28T11:22:40.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.358+0000 7f5a7a1ff700 1 -- 172.21.15.92:0/2417458205 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5a600682f0 msgr2=0x7f5a6006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:40.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.358+0000 7f5a7a1ff700 1 --2- 172.21.15.92:0/2417458205 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5a600682f0 0x7f5a6006a7a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5a740d3b10 tx=0x7f5a64008040 comp rx=0 tx=0).stop 2023-11-28T11:22:40.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.358+0000 7f5a7a1ff700 1 -- 172.21.15.92:0/2417458205 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a740b7260 msgr2=0x7f5a741596d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:40.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.358+0000 7f5a7a1ff700 1 --2- 172.21.15.92:0/2417458205 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a740b7260 0x7f5a741596d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c017040 tx=0x7f5a6c009750 comp rx=0 tx=0).stop 2023-11-28T11:22:40.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.358+0000 7f5a7a1ff700 1 -- 172.21.15.92:0/2417458205 shutdown_connections 2023-11-28T11:22:40.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.358+0000 7f5a7a1ff700 1 --2- 172.21.15.92:0/2417458205 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5a600682f0 0x7f5a6006a7a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:40.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.358+0000 7f5a7a1ff700 1 --2- 172.21.15.92:0/2417458205 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a740b7260 0x7f5a741596d0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:40.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.358+0000 7f5a7a1ff700 1 --2- 172.21.15.92:0/2417458205 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a5c004d90 0x7f5a74154230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:40.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.358+0000 7f5a7a1ff700 1 --2- 172.21.15.92:0/2417458205 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a74154770 0x7f5a740b6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:40.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.358+0000 7f5a7a1ff700 1 -- 172.21.15.92:0/2417458205 >> 172.21.15.92:0/2417458205 conn(0x7f5a740719a0 msgr2=0x7f5a740cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:40.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.358+0000 7f5a7a1ff700 1 -- 172.21.15.92:0/2417458205 shutdown_connections 2023-11-28T11:22:40.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.358+0000 7f5a7a1ff700 1 -- 172.21.15.92:0/2417458205 wait complete. 2023-11-28T11:22:40.377 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:22:40.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.576+0000 7f2477fff700 1 Processor -- start 2023-11-28T11:22:40.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.576+0000 7f2477fff700 1 -- start start 2023-11-28T11:22:40.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.577+0000 7f2477fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24700cd460 0x7f247006c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:40.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.577+0000 7f2477fff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24700b88e0 con 0x7f24700d0d60 2023-11-28T11:22:40.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.577+0000 7f2477fff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24700cd8f0 con 0x7f24700b8510 2023-11-28T11:22:40.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.577+0000 7f2477fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24700cdbd0 con 0x7f24700cd460 2023-11-28T11:22:40.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.577+0000 7f247659c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f24700d0d60 0x7f24700d11e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:39002/0 (socket says 172.21.15.92:39002) 2023-11-28T11:22:40.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.577+0000 7f247659c700 1 -- 172.21.15.92:0/2696404840 learned_addr learned my addr 172.21.15.92:0/2696404840 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:22:40.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.577+0000 7f247559a700 1 --2- 172.21.15.92:0/2696404840 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24700cd460 0x7f247006c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:40.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.578+0000 7f2474d99700 1 -- 172.21.15.92:0/2696404840 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3216485211 0 0) 0x7f24700b88e0 con 0x7f24700d0d60 2023-11-28T11:22:40.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.578+0000 7f2474d99700 1 -- 172.21.15.92:0/2696404840 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f245c0035b0 con 0x7f24700d0d60 2023-11-28T11:22:40.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.578+0000 7f2474d99700 1 -- 172.21.15.92:0/2696404840 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3517435406 0 0) 0x7f24700cd8f0 con 0x7f24700b8510 2023-11-28T11:22:40.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.578+0000 7f2474d99700 1 -- 172.21.15.92:0/2696404840 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24700b88e0 con 0x7f24700b8510 2023-11-28T11:22:40.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.578+0000 7f247559a700 1 -- 172.21.15.92:0/2696404840 >> v1:172.21.15.169:6789/0 conn(0x7f24700b8510 legacy=0x7f247006b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:40.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.578+0000 7f247559a700 1 -- 172.21.15.92:0/2696404840 >> v1:172.21.15.92:6789/0 conn(0x7f24700d0d60 legacy=0x7f24700d11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:40.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.578+0000 7f2474d99700 1 -- 172.21.15.92:0/2696404840 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1604777559 0 0) 0x7f245c0035b0 con 0x7f24700d0d60 2023-11-28T11:22:40.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.578+0000 7f247559a700 1 -- 172.21.15.92:0/2696404840 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24700cfd10 con 0x7f24700cd460 2023-11-28T11:22:40.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.578+0000 7f247559a700 1 --2- 172.21.15.92:0/2696404840 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24700cd460 0x7f247006c080 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2460009870 tx=0x7f2460009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d78e39bf16acdbce server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:40.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.579+0000 7f2474d99700 1 -- 172.21.15.92:0/2696404840 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24680019f0 con 0x7f24700cd460 2023-11-28T11:22:40.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.579+0000 7f2474d99700 1 -- 172.21.15.92:0/2696404840 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2460004030 con 0x7f24700cd460 2023-11-28T11:22:40.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.579+0000 7f2474d99700 1 -- 172.21.15.92:0/2696404840 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24600043c0 con 0x7f24700cd460 2023-11-28T11:22:40.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.579+0000 7f2477fff700 1 -- 172.21.15.92:0/2696404840 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24700cd460 msgr2=0x7f247006c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:40.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.579+0000 7f2477fff700 1 --2- 172.21.15.92:0/2696404840 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24700cd460 0x7f247006c080 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2460009870 tx=0x7f2460009b80 comp rx=0 tx=0).stop 2023-11-28T11:22:40.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.579+0000 7f2477fff700 1 -- 172.21.15.92:0/2696404840 shutdown_connections 2023-11-28T11:22:40.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.579+0000 7f2477fff700 1 --2- 172.21.15.92:0/2696404840 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24700cd460 0x7f247006c080 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:40.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.579+0000 7f2477fff700 1 -- 172.21.15.92:0/2696404840 >> 172.21.15.92:0/2696404840 conn(0x7f24700719a0 msgr2=0x7f2470071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:40.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.579+0000 7f2477fff700 1 -- 172.21.15.92:0/2696404840 shutdown_connections 2023-11-28T11:22:40.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.579+0000 7f2477fff700 1 -- 172.21.15.92:0/2696404840 wait complete. 2023-11-28T11:22:40.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.581+0000 7f2477fff700 1 Processor -- start 2023-11-28T11:22:40.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.581+0000 7f2477fff700 1 -- start start 2023-11-28T11:22:40.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.582+0000 7f2477fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24700b8510 0x7f24701f3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:40.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.582+0000 7f2477fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24700cd460 0x7f24700c5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:40.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.582+0000 7f2477fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24700d0d60 0x7f24701f2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:40.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.582+0000 7f2477fff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24701f2fb0 con 0x7f24700cd460 2023-11-28T11:22:40.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.582+0000 7f2477fff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24701f3290 con 0x7f24700d0d60 2023-11-28T11:22:40.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.582+0000 7f2477fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24700cbf30 con 0x7f24700b8510 2023-11-28T11:22:40.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.582+0000 7f247559a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24700cd460 0x7f24700c5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:40.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.582+0000 7f247559a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24700cd460 0x7f24700c5e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52896/0 (socket says 172.21.15.92:52896) 2023-11-28T11:22:40.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.582+0000 7f247559a700 1 -- 172.21.15.92:0/3191306069 learned_addr learned my addr 172.21.15.92:0/3191306069 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:40.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.583+0000 7f247659c700 1 --2- 172.21.15.92:0/3191306069 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24700d0d60 0x7f24701f2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:40.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.583+0000 7f2475d9b700 1 --2- 172.21.15.92:0/3191306069 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24700b8510 0x7f24701f3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:40.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.583+0000 7f247559a700 1 -- 172.21.15.92:0/3191306069 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24700b8510 msgr2=0x7f24701f3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:40.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.583+0000 7f247559a700 1 --2- 172.21.15.92:0/3191306069 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24700b8510 0x7f24701f3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:40.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.583+0000 7f247559a700 1 -- 172.21.15.92:0/3191306069 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24700d0d60 msgr2=0x7f24701f2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:40.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.583+0000 7f247559a700 1 --2- 172.21.15.92:0/3191306069 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24700d0d60 0x7f24701f2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:40.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.583+0000 7f247559a700 1 -- 172.21.15.92:0/3191306069 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24600094a0 con 0x7f24700cd460 2023-11-28T11:22:40.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.583+0000 7f247659c700 1 --2- 172.21.15.92:0/3191306069 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24700d0d60 0x7f24701f2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:22:40.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.583+0000 7f2475d9b700 1 --2- 172.21.15.92:0/3191306069 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24700b8510 0x7f24701f3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:22:40.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.583+0000 7f247559a700 1 --2- 172.21.15.92:0/3191306069 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24700cd460 0x7f24700c5e70 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f2460000c00 tx=0x7f2460004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:40.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.583+0000 7f24667fc700 1 -- 172.21.15.92:0/3191306069 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2460011630 con 0x7f24700cd460 2023-11-28T11:22:40.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.583+0000 7f24667fc700 1 -- 172.21.15.92:0/3191306069 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f246001f070 con 0x7f24700cd460 2023-11-28T11:22:40.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.583+0000 7f2477fff700 1 -- 172.21.15.92:0/3191306069 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24700d29c0 con 0x7f24700cd460 2023-11-28T11:22:40.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.583+0000 7f24667fc700 1 -- 172.21.15.92:0/3191306069 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24600119f0 con 0x7f24700cd460 2023-11-28T11:22:40.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.584+0000 7f2477fff700 1 -- 172.21.15.92:0/3191306069 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24700cfff0 con 0x7f24700cd460 2023-11-28T11:22:40.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.584+0000 7f2477fff700 1 -- 172.21.15.92:0/3191306069 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24700d2b50 con 0x7f24700cd460 2023-11-28T11:22:40.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.585+0000 7f24667fc700 1 -- 172.21.15.92:0/3191306069 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2460017020 con 0x7f24700cd460 2023-11-28T11:22:40.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.586+0000 7f24667fc700 1 --2- 172.21.15.92:0/3191306069 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f24540682f0 0x7f245406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:40.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.586+0000 7f24667fc700 1 -- 172.21.15.92:0/3191306069 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f2460021070 con 0x7f24700cd460 2023-11-28T11:22:40.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.587+0000 7f2475d9b700 1 --2- 172.21.15.92:0/3191306069 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f24540682f0 0x7f245406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:40.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.587+0000 7f2475d9b700 1 --2- 172.21.15.92:0/3191306069 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f24540682f0 0x7f245406a7a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f246c0011e0 tx=0x7f246c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:40.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.590+0000 7f24667fc700 1 -- 172.21.15.92:0/3191306069 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2460053e40 con 0x7f24700cd460 2023-11-28T11:22:40.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.786+0000 7f2477fff700 1 -- 172.21.15.92:0/3191306069 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f24700cc450 con 0x7f24700cd460 2023-11-28T11:22:40.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.787+0000 7f24667fc700 1 -- 172.21.15.92:0/3191306069 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v44) v1 ==== 104+0+11 (secure 0 0 0) 0x7f24700cc450 con 0x7f24700cd460 2023-11-28T11:22:40.788 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-11-28T11:22:40.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.789+0000 7f2477fff700 1 -- 172.21.15.92:0/3191306069 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f24540682f0 msgr2=0x7f245406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:40.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.789+0000 7f2477fff700 1 --2- 172.21.15.92:0/3191306069 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f24540682f0 0x7f245406a7a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f246c0011e0 tx=0x7f246c009040 comp rx=0 tx=0).stop 2023-11-28T11:22:40.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.789+0000 7f2477fff700 1 -- 172.21.15.92:0/3191306069 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24700cd460 msgr2=0x7f24700c5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:40.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.789+0000 7f2477fff700 1 --2- 172.21.15.92:0/3191306069 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24700cd460 0x7f24700c5e70 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f2460000c00 tx=0x7f2460004750 comp rx=0 tx=0).stop 2023-11-28T11:22:40.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.789+0000 7f2477fff700 1 -- 172.21.15.92:0/3191306069 shutdown_connections 2023-11-28T11:22:40.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.789+0000 7f2477fff700 1 --2- 172.21.15.92:0/3191306069 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f24700d0d60 0x7f24701f2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:40.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.789+0000 7f2477fff700 1 --2- 172.21.15.92:0/3191306069 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f24700b8510 0x7f24701f3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:40.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.789+0000 7f2477fff700 1 --2- 172.21.15.92:0/3191306069 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f24540682f0 0x7f245406a7a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:40.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.789+0000 7f2477fff700 1 --2- 172.21.15.92:0/3191306069 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24700cd460 0x7f24700c5e70 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:40.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.789+0000 7f2477fff700 1 -- 172.21.15.92:0/3191306069 >> 172.21.15.92:0/3191306069 conn(0x7f24700719a0 msgr2=0x7f247006b970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:40.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.789+0000 7f2477fff700 1 -- 172.21.15.92:0/3191306069 shutdown_connections 2023-11-28T11:22:40.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:40.790+0000 7f2477fff700 1 -- 172.21.15.92:0/3191306069 wait complete. 2023-11-28T11:22:40.809 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:22:41.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.007+0000 7f557a1ff700 1 Processor -- start 2023-11-28T11:22:41.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.008+0000 7f557a1ff700 1 -- start start 2023-11-28T11:22:41.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.009+0000 7f557a1ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55740d2df0 0x7f55740cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:41.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.009+0000 7f557a1ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f55740b1c60 0x7f55740cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:41.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.009+0000 7f557a1ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55740d1950 con 0x7f55740b4fc0 2023-11-28T11:22:41.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.009+0000 7f557a1ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55740d3d30 con 0x7f55740b1c60 2023-11-28T11:22:41.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.009+0000 7f557a1ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55740b2500 con 0x7f55740d2df0 2023-11-28T11:22:41.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.009+0000 7f5573fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f55740b1c60 0x7f55740cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:41.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.009+0000 7f5572ffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f55740b4fc0 0x7f55740d31c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:39012/0 (socket says 172.21.15.92:39012) 2023-11-28T11:22:41.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.009+0000 7f5572ffd700 1 -- 172.21.15.92:0/2307612258 learned_addr learned my addr 172.21.15.92:0/2307612258 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:22:41.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.009+0000 7f55737fe700 1 --2- 172.21.15.92:0/2307612258 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55740d2df0 0x7f55740cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:41.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.009+0000 7f5573fff700 1 -- 172.21.15.92:0/2307612258 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55740d2df0 msgr2=0x7f55740cd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:41.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.009+0000 7f5573fff700 1 --2- 172.21.15.92:0/2307612258 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55740d2df0 0x7f55740cd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:41.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.009+0000 7f5573fff700 1 -- 172.21.15.92:0/2307612258 >> v1:172.21.15.92:6789/0 conn(0x7f55740b4fc0 legacy=0x7f55740d31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:41.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.009+0000 7f5573fff700 1 -- 172.21.15.92:0/2307612258 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55740d3a50 con 0x7f55740b1c60 2023-11-28T11:22:41.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.009+0000 7f55727fc700 1 -- 172.21.15.92:0/2307612258 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1626818089 0 0) 0x7f55740d1950 con 0x7f55740b4fc0 2023-11-28T11:22:41.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.010+0000 7f5573fff700 1 --2- 172.21.15.92:0/2307612258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55740b1c60 0x7f55740cf010 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f556c009d90 tx=0x7f556c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48f9e968faaf84ce server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:41.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.010+0000 7f55727fc700 1 -- 172.21.15.92:0/2307612258 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556c00e070 con 0x7f55740b1c60 2023-11-28T11:22:41.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.010+0000 7f55727fc700 1 -- 172.21.15.92:0/2307612258 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f556c00bbd0 con 0x7f55740b1c60 2023-11-28T11:22:41.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.010+0000 7f55727fc700 1 -- 172.21.15.92:0/2307612258 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556c012430 con 0x7f55740b1c60 2023-11-28T11:22:41.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.010+0000 7f557a1ff700 1 -- 172.21.15.92:0/2307612258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55740b1c60 msgr2=0x7f55740cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:41.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.010+0000 7f557a1ff700 1 --2- 172.21.15.92:0/2307612258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55740b1c60 0x7f55740cf010 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f556c009d90 tx=0x7f556c00b330 comp rx=0 tx=0).stop 2023-11-28T11:22:41.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.011+0000 7f557a1ff700 1 -- 172.21.15.92:0/2307612258 shutdown_connections 2023-11-28T11:22:41.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.011+0000 7f557a1ff700 1 --2- 172.21.15.92:0/2307612258 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55740d2df0 0x7f55740cd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:41.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.011+0000 7f557a1ff700 1 --2- 172.21.15.92:0/2307612258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55740b1c60 0x7f55740cf010 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:41.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.011+0000 7f557a1ff700 1 -- 172.21.15.92:0/2307612258 >> 172.21.15.92:0/2307612258 conn(0x7f55740719a0 msgr2=0x7f5574071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:41.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.011+0000 7f557a1ff700 1 -- 172.21.15.92:0/2307612258 shutdown_connections 2023-11-28T11:22:41.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.011+0000 7f557a1ff700 1 -- 172.21.15.92:0/2307612258 wait complete. 2023-11-28T11:22:41.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.012+0000 7f557a1ff700 1 Processor -- start 2023-11-28T11:22:41.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.012+0000 7f557a1ff700 1 -- start start 2023-11-28T11:22:41.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.013+0000 7f557a1ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55740b1c60 0x7f55740c9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:41.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.013+0000 7f557a1ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55740b4fc0 0x7f55741f2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:41.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.013+0000 7f557a1ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55740d2df0 0x7f55741f2a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:41.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.013+0000 7f557a1ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55741e6150 con 0x7f55740d2df0 2023-11-28T11:22:41.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.013+0000 7f557a1ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55741e6430 con 0x7f55740b1c60 2023-11-28T11:22:41.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.013+0000 7f557a1ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55741e6710 con 0x7f55740b4fc0 2023-11-28T11:22:41.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.014+0000 7f5573fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55740d2df0 0x7f55741f2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:41.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.014+0000 7f5573fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55740d2df0 0x7f55741f2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52904/0 (socket says 172.21.15.92:52904) 2023-11-28T11:22:41.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.014+0000 7f5573fff700 1 -- 172.21.15.92:0/801480225 learned_addr learned my addr 172.21.15.92:0/801480225 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:41.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.014+0000 7f5572ffd700 1 --2- 172.21.15.92:0/801480225 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55740b4fc0 0x7f55741f2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:41.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.014+0000 7f55737fe700 1 --2- 172.21.15.92:0/801480225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55740b1c60 0x7f55740c9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:41.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.014+0000 7f5573fff700 1 -- 172.21.15.92:0/801480225 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55740b4fc0 msgr2=0x7f55741f2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:41.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.014+0000 7f5573fff700 1 --2- 172.21.15.92:0/801480225 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55740b4fc0 0x7f55741f2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:41.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.014+0000 7f5573fff700 1 -- 172.21.15.92:0/801480225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55740b1c60 msgr2=0x7f55740c9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:41.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.014+0000 7f5573fff700 1 --2- 172.21.15.92:0/801480225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55740b1c60 0x7f55740c9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:41.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.014+0000 7f5573fff700 1 -- 172.21.15.92:0/801480225 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f556c009a70 con 0x7f55740d2df0 2023-11-28T11:22:41.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.014+0000 7f55737fe700 1 --2- 172.21.15.92:0/801480225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55740b1c60 0x7f55740c9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:22:41.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.014+0000 7f5572ffd700 1 --2- 172.21.15.92:0/801480225 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55740b4fc0 0x7f55741f2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:22:41.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.014+0000 7f5573fff700 1 --2- 172.21.15.92:0/801480225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55740d2df0 0x7f55741f2a80 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f557408c040 tx=0x7f556c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:41.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.014+0000 7f555bfff700 1 -- 172.21.15.92:0/801480225 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556c012430 con 0x7f55740d2df0 2023-11-28T11:22:41.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.015+0000 7f555bfff700 1 -- 172.21.15.92:0/801480225 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f556c020070 con 0x7f55740d2df0 2023-11-28T11:22:41.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.015+0000 7f557a1ff700 1 -- 172.21.15.92:0/801480225 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55740b88f0 con 0x7f55740d2df0 2023-11-28T11:22:41.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.015+0000 7f555bfff700 1 -- 172.21.15.92:0/801480225 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556c004280 con 0x7f55740d2df0 2023-11-28T11:22:41.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.015+0000 7f557a1ff700 1 -- 172.21.15.92:0/801480225 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55740d15e0 con 0x7f55740d2df0 2023-11-28T11:22:41.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.016+0000 7f557a1ff700 1 -- 172.21.15.92:0/801480225 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55740b21b0 con 0x7f55740d2df0 2023-11-28T11:22:41.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.017+0000 7f555bfff700 1 -- 172.21.15.92:0/801480225 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f556c0043e0 con 0x7f55740d2df0 2023-11-28T11:22:41.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.017+0000 7f555bfff700 1 --2- 172.21.15.92:0/801480225 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f55600682f0 0x7f556006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:41.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.017+0000 7f555bfff700 1 -- 172.21.15.92:0/801480225 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f556c022070 con 0x7f55740d2df0 2023-11-28T11:22:41.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.018+0000 7f55737fe700 1 --2- 172.21.15.92:0/801480225 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f55600682f0 0x7f556006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:41.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.018+0000 7f55737fe700 1 --2- 172.21.15.92:0/801480225 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f55600682f0 0x7f556006a7a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5564009820 tx=0x7f5564008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:41.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.022+0000 7f555bfff700 1 -- 172.21.15.92:0/801480225 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f556c0549e0 con 0x7f55740d2df0 2023-11-28T11:22:41.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.213+0000 7f557a1ff700 1 -- 172.21.15.92:0/801480225 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5574067170 con 0x7f55740d2df0 2023-11-28T11:22:41.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.213+0000 7f555bfff700 1 -- 172.21.15.92:0/801480225 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v44) v1 ==== 100+0+11 (secure 0 0 0) 0x7f556c058000 con 0x7f55740d2df0 2023-11-28T11:22:41.214 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-11-28T11:22:41.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.216+0000 7f557a1ff700 1 -- 172.21.15.92:0/801480225 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f55600682f0 msgr2=0x7f556006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:41.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.216+0000 7f557a1ff700 1 --2- 172.21.15.92:0/801480225 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f55600682f0 0x7f556006a7a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5564009820 tx=0x7f5564008040 comp rx=0 tx=0).stop 2023-11-28T11:22:41.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.216+0000 7f557a1ff700 1 -- 172.21.15.92:0/801480225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55740d2df0 msgr2=0x7f55741f2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:41.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.216+0000 7f557a1ff700 1 --2- 172.21.15.92:0/801480225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55740d2df0 0x7f55741f2a80 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f557408c040 tx=0x7f556c004ad0 comp rx=0 tx=0).stop 2023-11-28T11:22:41.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.216+0000 7f557a1ff700 1 -- 172.21.15.92:0/801480225 shutdown_connections 2023-11-28T11:22:41.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.216+0000 7f557a1ff700 1 --2- 172.21.15.92:0/801480225 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f55600682f0 0x7f556006a7a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:41.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.216+0000 7f557a1ff700 1 --2- 172.21.15.92:0/801480225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55740d2df0 0x7f55741f2a80 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:41.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.216+0000 7f557a1ff700 1 --2- 172.21.15.92:0/801480225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f55740b1c60 0x7f55740c9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:41.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.216+0000 7f557a1ff700 1 --2- 172.21.15.92:0/801480225 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f55740b4fc0 0x7f55741f2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:41.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.216+0000 7f557a1ff700 1 -- 172.21.15.92:0/801480225 >> 172.21.15.92:0/801480225 conn(0x7f55740719a0 msgr2=0x7f55740b2670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:41.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.216+0000 7f557a1ff700 1 -- 172.21.15.92:0/801480225 shutdown_connections 2023-11-28T11:22:41.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.216+0000 7f557a1ff700 1 -- 172.21.15.92:0/801480225 wait complete. 2023-11-28T11:22:41.235 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T11:22:41.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.430+0000 7ff4533ff700 1 Processor -- start 2023-11-28T11:22:41.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.430+0000 7ff4533ff700 1 -- start start 2023-11-28T11:22:41.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.431+0000 7ff4533ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff44c0b8510 0x7ff44c06b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:41.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.431+0000 7ff4533ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff44c0cd460 0x7ff44c06beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:41.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.431+0000 7ff4533ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff44c0b88e0 con 0x7ff44c06c3f0 2023-11-28T11:22:41.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.431+0000 7ff4533ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff44c0cdbd0 con 0x7ff44c0b8510 2023-11-28T11:22:41.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.431+0000 7ff4533ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff44c0d02d0 con 0x7ff44c0cd460 2023-11-28T11:22:41.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.432+0000 7ff45119b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff44c0b8510 0x7ff44c06b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:41.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.432+0000 7ff45199c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff44c06c3f0 0x7ff44c15cc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:39026/0 (socket says 172.21.15.92:39026) 2023-11-28T11:22:41.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.432+0000 7ff45199c700 1 -- 172.21.15.92:0/358313685 learned_addr learned my addr 172.21.15.92:0/358313685 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:22:41.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.432+0000 7ff45099a700 1 --2- 172.21.15.92:0/358313685 >> v2:172.21.15.169:3300/0 conn(0x7ff44c0cd460 0x7ff44c06beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:41.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.432+0000 7ff45119b700 1 -- 172.21.15.92:0/358313685 >> v2:172.21.15.169:3300/0 conn(0x7ff44c0cd460 msgr2=0x7ff44c06beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:41.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.432+0000 7ff45119b700 1 --2- 172.21.15.92:0/358313685 >> v2:172.21.15.169:3300/0 conn(0x7ff44c0cd460 0x7ff44c06beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:41.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.432+0000 7ff45119b700 1 -- 172.21.15.92:0/358313685 >> v1:172.21.15.92:6789/0 conn(0x7ff44c06c3f0 legacy=0x7ff44c15cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:41.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.432+0000 7ff45119b700 1 -- 172.21.15.92:0/358313685 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff44c0cd8f0 con 0x7ff44c0b8510 2023-11-28T11:22:41.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.432+0000 7ff443fff700 1 -- 172.21.15.92:0/358313685 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 619171563 0 0) 0x7ff44c0b88e0 con 0x7ff44c06c3f0 2023-11-28T11:22:41.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.432+0000 7ff45119b700 1 --2- 172.21.15.92:0/358313685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff44c0b8510 0x7ff44c06b970 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff448009870 tx=0x7ff448009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a66dfb926389261b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:41.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.433+0000 7ff443fff700 1 -- 172.21.15.92:0/358313685 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff448011830 con 0x7ff44c0b8510 2023-11-28T11:22:41.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.433+0000 7ff443fff700 1 -- 172.21.15.92:0/358313685 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff448011990 con 0x7ff44c0b8510 2023-11-28T11:22:41.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.433+0000 7ff443fff700 1 -- 172.21.15.92:0/358313685 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff448011d20 con 0x7ff44c0b8510 2023-11-28T11:22:41.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.433+0000 7ff4533ff700 1 -- 172.21.15.92:0/358313685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff44c0b8510 msgr2=0x7ff44c06b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:41.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.433+0000 7ff4533ff700 1 --2- 172.21.15.92:0/358313685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff44c0b8510 0x7ff44c06b970 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff448009870 tx=0x7ff448009b80 comp rx=0 tx=0).stop 2023-11-28T11:22:41.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.433+0000 7ff4533ff700 1 -- 172.21.15.92:0/358313685 shutdown_connections 2023-11-28T11:22:41.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.433+0000 7ff4533ff700 1 --2- 172.21.15.92:0/358313685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff44c0b8510 0x7ff44c06b970 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:41.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.433+0000 7ff4533ff700 1 --2- 172.21.15.92:0/358313685 >> v2:172.21.15.169:3300/0 conn(0x7ff44c0cd460 0x7ff44c06beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:41.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.433+0000 7ff4533ff700 1 -- 172.21.15.92:0/358313685 >> 172.21.15.92:0/358313685 conn(0x7ff44c0719a0 msgr2=0x7ff44c071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:41.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.433+0000 7ff4533ff700 1 -- 172.21.15.92:0/358313685 shutdown_connections 2023-11-28T11:22:41.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.433+0000 7ff4533ff700 1 -- 172.21.15.92:0/358313685 wait complete. 2023-11-28T11:22:41.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.436+0000 7ff4533ff700 1 Processor -- start 2023-11-28T11:22:41.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.436+0000 7ff4533ff700 1 -- start start 2023-11-28T11:22:41.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.437+0000 7ff4533ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff44c06c3f0 0x7ff44c1578e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:41.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.437+0000 7ff4533ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff44c0b8510 0x7ff44c1f1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:41.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.437+0000 7ff4533ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff44c0cd460 0x7ff44c0c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:41.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.437+0000 7ff4533ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44c158e70 con 0x7ff44c0b8510 2023-11-28T11:22:41.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.437+0000 7ff4533ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44c158250 con 0x7ff44c0cd460 2023-11-28T11:22:41.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.437+0000 7ff4533ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff44c158530 con 0x7ff44c06c3f0 2023-11-28T11:22:41.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.437+0000 7ff45099a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff44c0b8510 0x7ff44c1f1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:41.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.437+0000 7ff45099a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff44c0b8510 0x7ff44c1f1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52912/0 (socket says 172.21.15.92:52912) 2023-11-28T11:22:41.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.437+0000 7ff45099a700 1 -- 172.21.15.92:0/4058967442 learned_addr learned my addr 172.21.15.92:0/4058967442 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:41.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.438+0000 7ff45119b700 1 --2- 172.21.15.92:0/4058967442 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff44c06c3f0 0x7ff44c1578e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:41.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.438+0000 7ff45099a700 1 -- 172.21.15.92:0/4058967442 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff44c06c3f0 msgr2=0x7ff44c1578e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:41.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.438+0000 7ff45099a700 1 --2- 172.21.15.92:0/4058967442 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff44c06c3f0 0x7ff44c1578e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:41.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.438+0000 7ff45099a700 1 -- 172.21.15.92:0/4058967442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff44c0cd460 msgr2=0x7ff44c0c9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:41.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.438+0000 7ff45199c700 1 --2- 172.21.15.92:0/4058967442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff44c0cd460 0x7ff44c0c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:41.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.438+0000 7ff45099a700 1 --2- 172.21.15.92:0/4058967442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff44c0cd460 0x7ff44c0c9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:41.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.438+0000 7ff45099a700 1 -- 172.21.15.92:0/4058967442 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4480094a0 con 0x7ff44c0b8510 2023-11-28T11:22:41.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.438+0000 7ff45099a700 1 --2- 172.21.15.92:0/4058967442 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff44c0b8510 0x7ff44c1f1930 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff43c00d7d0 tx=0x7ff43c00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:41.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.438+0000 7ff441ffb700 1 -- 172.21.15.92:0/4058967442 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43c0125f0 con 0x7ff44c0b8510 2023-11-28T11:22:41.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.438+0000 7ff441ffb700 1 -- 172.21.15.92:0/4058967442 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff43c012770 con 0x7ff44c0b8510 2023-11-28T11:22:41.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.438+0000 7ff4533ff700 1 -- 172.21.15.92:0/4058967442 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff44c0cfff0 con 0x7ff44c0b8510 2023-11-28T11:22:41.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.438+0000 7ff441ffb700 1 -- 172.21.15.92:0/4058967442 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43c012b20 con 0x7ff44c0b8510 2023-11-28T11:22:41.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.438+0000 7ff4533ff700 1 -- 172.21.15.92:0/4058967442 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff44c0cfd10 con 0x7ff44c0b8510 2023-11-28T11:22:41.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.439+0000 7ff4533ff700 1 -- 172.21.15.92:0/4058967442 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff44c158c40 con 0x7ff44c0b8510 2023-11-28T11:22:41.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.440+0000 7ff441ffb700 1 -- 172.21.15.92:0/4058967442 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff43c012c80 con 0x7ff44c0b8510 2023-11-28T11:22:41.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.441+0000 7ff441ffb700 1 --2- 172.21.15.92:0/4058967442 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff430068320 0x7ff43006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:41.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.441+0000 7ff441ffb700 1 -- 172.21.15.92:0/4058967442 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6440+0+0 (secure 0 0 0) 0x7ff43c086370 con 0x7ff44c0b8510 2023-11-28T11:22:41.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.441+0000 7ff45119b700 1 --2- 172.21.15.92:0/4058967442 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff430068320 0x7ff43006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:41.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.442+0000 7ff45119b700 1 --2- 172.21.15.92:0/4058967442 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff430068320 0x7ff43006a7d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff4480097c0 tx=0x7ff448008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:41.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.445+0000 7ff441ffb700 1 -- 172.21.15.92:0/4058967442 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff44c158c40 con 0x7ff44c0b8510 2023-11-28T11:22:41.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.649+0000 7ff4533ff700 1 -- 172.21.15.92:0/4058967442 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff44c158c40 con 0x7ff44c0b8510 2023-11-28T11:22:41.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.650+0000 7ff441ffb700 1 -- 172.21.15.92:0/4058967442 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v44) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff44c158c40 con 0x7ff44c0b8510 2023-11-28T11:22:41.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.652+0000 7ff4533ff700 1 -- 172.21.15.92:0/4058967442 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff430068320 msgr2=0x7ff43006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:41.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.652+0000 7ff4533ff700 1 --2- 172.21.15.92:0/4058967442 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff430068320 0x7ff43006a7d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff4480097c0 tx=0x7ff448008030 comp rx=0 tx=0).stop 2023-11-28T11:22:41.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.652+0000 7ff4533ff700 1 -- 172.21.15.92:0/4058967442 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff44c0b8510 msgr2=0x7ff44c1f1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:41.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.652+0000 7ff4533ff700 1 --2- 172.21.15.92:0/4058967442 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff44c0b8510 0x7ff44c1f1930 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff43c00d7d0 tx=0x7ff43c00db90 comp rx=0 tx=0).stop 2023-11-28T11:22:41.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.652+0000 7ff4533ff700 1 -- 172.21.15.92:0/4058967442 shutdown_connections 2023-11-28T11:22:41.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.652+0000 7ff4533ff700 1 --2- 172.21.15.92:0/4058967442 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff44c0cd460 0x7ff44c0c9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:41.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.652+0000 7ff4533ff700 1 --2- 172.21.15.92:0/4058967442 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff44c06c3f0 0x7ff44c1578e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:41.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.652+0000 7ff4533ff700 1 --2- 172.21.15.92:0/4058967442 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff430068320 0x7ff43006a7d0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:41.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.652+0000 7ff4533ff700 1 --2- 172.21.15.92:0/4058967442 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff44c0b8510 0x7ff44c1f1930 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:41.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.652+0000 7ff4533ff700 1 -- 172.21.15.92:0/4058967442 >> 172.21.15.92:0/4058967442 conn(0x7ff44c0719a0 msgr2=0x7ff44c0b41b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:41.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.652+0000 7ff4533ff700 1 -- 172.21.15.92:0/4058967442 shutdown_connections 2023-11-28T11:22:41.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:41.652+0000 7ff4533ff700 1 -- 172.21.15.92:0/4058967442 wait complete. 2023-11-28T11:22:41.656 INFO:teuthology.orchestra.run.smithi092.stderr:listed 0 entries 2023-11-28T11:22:41.671 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-28T11:22:41.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:22:41.671 DEBUG:teuthology.orchestra.run.smithi092:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-28T11:22:41.776 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2023-11-28T11:22:41.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:22:41.777 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T11:22:41.838 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:22:42.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.035+0000 7f749f5ff700 1 Processor -- start 2023-11-28T11:22:42.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.035+0000 7f749f5ff700 1 -- start start 2023-11-28T11:22:42.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.036+0000 7f749f5ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f74980b8510 0x7f749806b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:42.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.036+0000 7f749f5ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f74980d0d60 0x7f74980d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:42.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.036+0000 7f749f5ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74980b88e0 con 0x7f74980cd460 2023-11-28T11:22:42.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.036+0000 7f749f5ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74980cdbd0 con 0x7f74980b8510 2023-11-28T11:22:42.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.036+0000 7f749f5ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74980d02d0 con 0x7f74980d0d60 2023-11-28T11:22:42.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.036+0000 7f749d39b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f74980b8510 0x7f749806b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:42.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.036+0000 7f749cb9a700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f74980cd460 0x7f749806beb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:39040/0 (socket says 172.21.15.92:39040) 2023-11-28T11:22:42.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.037+0000 7f749cb9a700 1 -- 172.21.15.92:0/2682758600 learned_addr learned my addr 172.21.15.92:0/2682758600 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:22:42.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.037+0000 7f749db9c700 1 --2- 172.21.15.92:0/2682758600 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f74980d0d60 0x7f74980d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:42.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.037+0000 7f748ffff700 1 -- 172.21.15.92:0/2682758600 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4175958171 0 0) 0x7f74980b88e0 con 0x7f74980cd460 2023-11-28T11:22:42.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.037+0000 7f748ffff700 1 -- 172.21.15.92:0/2682758600 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74840035b0 con 0x7f74980cd460 2023-11-28T11:22:42.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.037+0000 7f749d39b700 1 -- 172.21.15.92:0/2682758600 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f74980d0d60 msgr2=0x7f74980d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:42.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.037+0000 7f749d39b700 1 --2- 172.21.15.92:0/2682758600 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f74980d0d60 0x7f74980d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:42.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.037+0000 7f749d39b700 1 -- 172.21.15.92:0/2682758600 >> v1:172.21.15.92:6789/0 conn(0x7f74980cd460 legacy=0x7f749806beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:42.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.037+0000 7f749d39b700 1 -- 172.21.15.92:0/2682758600 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74980cd8f0 con 0x7f74980b8510 2023-11-28T11:22:42.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.037+0000 7f749d39b700 1 --2- 172.21.15.92:0/2682758600 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74980b8510 0x7f749806b970 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f7494009870 tx=0x7f7494009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee72b1039dbfbc3c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:42.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.038+0000 7f748ffff700 1 -- 172.21.15.92:0/2682758600 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7494011830 con 0x7f74980b8510 2023-11-28T11:22:42.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.038+0000 7f748ffff700 1 -- 172.21.15.92:0/2682758600 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7494011990 con 0x7f74980b8510 2023-11-28T11:22:42.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.038+0000 7f748ffff700 1 -- 172.21.15.92:0/2682758600 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7494011d20 con 0x7f74980b8510 2023-11-28T11:22:42.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.038+0000 7f749f5ff700 1 -- 172.21.15.92:0/2682758600 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74980b8510 msgr2=0x7f749806b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:42.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.038+0000 7f749f5ff700 1 --2- 172.21.15.92:0/2682758600 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74980b8510 0x7f749806b970 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f7494009870 tx=0x7f7494009b80 comp rx=0 tx=0).stop 2023-11-28T11:22:42.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.038+0000 7f749f5ff700 1 -- 172.21.15.92:0/2682758600 shutdown_connections 2023-11-28T11:22:42.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.038+0000 7f749f5ff700 1 --2- 172.21.15.92:0/2682758600 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f74980d0d60 0x7f74980d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:42.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.038+0000 7f749f5ff700 1 --2- 172.21.15.92:0/2682758600 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74980b8510 0x7f749806b970 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:42.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.038+0000 7f749f5ff700 1 -- 172.21.15.92:0/2682758600 >> 172.21.15.92:0/2682758600 conn(0x7f74980719a0 msgr2=0x7f7498071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:42.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.038+0000 7f749f5ff700 1 -- 172.21.15.92:0/2682758600 shutdown_connections 2023-11-28T11:22:42.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.038+0000 7f749f5ff700 1 -- 172.21.15.92:0/2682758600 wait complete. 2023-11-28T11:22:42.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.040+0000 7f749f5ff700 1 Processor -- start 2023-11-28T11:22:42.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.040+0000 7f749f5ff700 1 -- start start 2023-11-28T11:22:42.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.041+0000 7f749f5ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f74980b8510 0x7f74980c95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:42.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.041+0000 7f749f5ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74980cd460 0x7f74980a9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:42.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.041+0000 7f749f5ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74980d0d60 0x7f74981f5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:42.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.041+0000 7f749f5ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74981e5bc0 con 0x7f74980cd460 2023-11-28T11:22:42.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.041+0000 7f749f5ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74981e5ea0 con 0x7f74980d0d60 2023-11-28T11:22:42.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.041+0000 7f749f5ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74981e6180 con 0x7f74980b8510 2023-11-28T11:22:42.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.041+0000 7f749cb9a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74980cd460 0x7f74980a9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:42.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.041+0000 7f749cb9a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74980cd460 0x7f74980a9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52936/0 (socket says 172.21.15.92:52936) 2023-11-28T11:22:42.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.041+0000 7f749cb9a700 1 -- 172.21.15.92:0/1602069538 learned_addr learned my addr 172.21.15.92:0/1602069538 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:42.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.041+0000 7f749d39b700 1 --2- 172.21.15.92:0/1602069538 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f74980b8510 0x7f74980c95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:42.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.041+0000 7f749db9c700 1 --2- 172.21.15.92:0/1602069538 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74980d0d60 0x7f74981f5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:42.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.042+0000 7f749cb9a700 1 -- 172.21.15.92:0/1602069538 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f74980b8510 msgr2=0x7f74980c95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:42.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.042+0000 7f749cb9a700 1 --2- 172.21.15.92:0/1602069538 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f74980b8510 0x7f74980c95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:42.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.042+0000 7f749cb9a700 1 -- 172.21.15.92:0/1602069538 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74980d0d60 msgr2=0x7f74981f5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:42.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.042+0000 7f749cb9a700 1 --2- 172.21.15.92:0/1602069538 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74980d0d60 0x7f74981f5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:42.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.042+0000 7f749cb9a700 1 -- 172.21.15.92:0/1602069538 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74940094a0 con 0x7f74980cd460 2023-11-28T11:22:42.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.042+0000 7f749db9c700 1 --2- 172.21.15.92:0/1602069538 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74980d0d60 0x7f74981f5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:22:42.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.042+0000 7f749d39b700 1 --2- 172.21.15.92:0/1602069538 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f74980b8510 0x7f74980c95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:22:42.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.042+0000 7f749cb9a700 1 --2- 172.21.15.92:0/1602069538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74980cd460 0x7f74980a9840 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f748800c4f0 tx=0x7f748800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:42.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.042+0000 7f748dffb700 1 -- 172.21.15.92:0/1602069538 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7488005d70 con 0x7f74980cd460 2023-11-28T11:22:42.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.042+0000 7f748dffb700 1 -- 172.21.15.92:0/1602069538 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74880070d0 con 0x7f74980cd460 2023-11-28T11:22:42.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.042+0000 7f749f5ff700 1 -- 172.21.15.92:0/1602069538 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74980cfff0 con 0x7f74980cd460 2023-11-28T11:22:42.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.042+0000 7f748dffb700 1 -- 172.21.15.92:0/1602069538 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7488007440 con 0x7f74980cd460 2023-11-28T11:22:42.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.042+0000 7f749f5ff700 1 -- 172.21.15.92:0/1602069538 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74980cfd10 con 0x7f74980cd460 2023-11-28T11:22:42.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.043+0000 7f749f5ff700 1 -- 172.21.15.92:0/1602069538 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7498148e30 con 0x7f74980cd460 2023-11-28T11:22:42.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.044+0000 7f748dffb700 1 -- 172.21.15.92:0/1602069538 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7488019030 con 0x7f74980cd460 2023-11-28T11:22:42.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.045+0000 7f748dffb700 1 --2- 172.21.15.92:0/1602069538 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f747c0682f0 0x7f747c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:42.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.046+0000 7f748dffb700 1 -- 172.21.15.92:0/1602069538 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f74880851e0 con 0x7f74980cd460 2023-11-28T11:22:42.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.046+0000 7f749d39b700 1 --2- 172.21.15.92:0/1602069538 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f747c0682f0 0x7f747c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:42.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.046+0000 7f749d39b700 1 --2- 172.21.15.92:0/1602069538 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f747c0682f0 0x7f747c06a7a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f74940090b0 tx=0x7f7494008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:42.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.050+0000 7f748dffb700 1 -- 172.21.15.92:0/1602069538 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f748804b5e0 con 0x7f74980cd460 2023-11-28T11:22:42.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.281+0000 7f749f5ff700 1 -- 172.21.15.92:0/1602069538 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7498067880 con 0x7f74980cd460 2023-11-28T11:22:42.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.282+0000 7f748dffb700 1 -- 172.21.15.92:0/1602069538 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4645 (secure 0 0 0) 0x7f7488051880 con 0x7f74980cd460 2023-11-28T11:22:42.283 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:22:42.283 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4976,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/3102603064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3102603064},{"type":"v1","addr":"172.21.15.169:6834","nonce":3102603064}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4979,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/79247121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":79247121},{"type":"v1","addr":"172.21.15.92:6837","nonce":79247121}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5221,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/219797192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":219797192},{"type":"v1","addr":"172.21.15.169:6835","nonce":219797192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:22:24.715447+0000","modified":"2023-11-28T11:22:27.733117+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5298},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5298":{"gid":5298,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.92:6835/1577294649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1577294649},{"type":"v1","addr":"172.21.15.92:6835","nonce":1577294649}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T11:22:42.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.284+0000 7f749f5ff700 1 -- 172.21.15.92:0/1602069538 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f747c0682f0 msgr2=0x7f747c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:42.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.284+0000 7f749f5ff700 1 --2- 172.21.15.92:0/1602069538 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f747c0682f0 0x7f747c06a7a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f74940090b0 tx=0x7f7494008030 comp rx=0 tx=0).stop 2023-11-28T11:22:42.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.284+0000 7f749f5ff700 1 -- 172.21.15.92:0/1602069538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74980cd460 msgr2=0x7f74980a9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:42.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.284+0000 7f749f5ff700 1 --2- 172.21.15.92:0/1602069538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74980cd460 0x7f74980a9840 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f748800c4f0 tx=0x7f748800c8b0 comp rx=0 tx=0).stop 2023-11-28T11:22:42.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.285+0000 7f749f5ff700 1 -- 172.21.15.92:0/1602069538 shutdown_connections 2023-11-28T11:22:42.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.285+0000 7f749f5ff700 1 --2- 172.21.15.92:0/1602069538 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f74980d0d60 0x7f74981f5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:42.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.285+0000 7f749f5ff700 1 --2- 172.21.15.92:0/1602069538 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f74980b8510 0x7f74980c95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:42.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.285+0000 7f749f5ff700 1 --2- 172.21.15.92:0/1602069538 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f747c0682f0 0x7f747c06a7a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:42.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.285+0000 7f749f5ff700 1 --2- 172.21.15.92:0/1602069538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74980cd460 0x7f74980a9840 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:42.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.285+0000 7f749f5ff700 1 -- 172.21.15.92:0/1602069538 >> 172.21.15.92:0/1602069538 conn(0x7f74980719a0 msgr2=0x7f74980b3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:42.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.285+0000 7f749f5ff700 1 -- 172.21.15.92:0/1602069538 shutdown_connections 2023-11-28T11:22:42.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.285+0000 7f749f5ff700 1 -- 172.21.15.92:0/1602069538 wait complete. 2023-11-28T11:22:42.287 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 17 2023-11-28T11:22:42.303 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2023-11-28T11:22:42.534 INFO:teuthology.orchestra.run.smithi092.stdout:{"message":"","return_code":0} 2023-11-28T11:22:42.534 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-28T11:22:42.534 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:22:42.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.731+0000 7f5d4f1ff700 1 Processor -- start 2023-11-28T11:22:42.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.732+0000 7f5d4f1ff700 1 -- start start 2023-11-28T11:22:42.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.733+0000 7f5d4f1ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5d480cd460 0x7f5d4806c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:42.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.733+0000 7f5d4f1ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d480d0d60 0x7f5d480d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:42.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.733+0000 7f5d4f1ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d480b88e0 con 0x7f5d480b8510 2023-11-28T11:22:42.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.733+0000 7f5d4f1ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d480cdbd0 con 0x7f5d480cd460 2023-11-28T11:22:42.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.733+0000 7f5d4f1ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d480d02d0 con 0x7f5d480d0d60 2023-11-28T11:22:42.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.733+0000 7f5d47fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5d480cd460 0x7f5d4806c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:42.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.733+0000 7f5d47fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5d480cd460 0x7f5d4806c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52950/0 (socket says 172.21.15.92:52950) 2023-11-28T11:22:42.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.733+0000 7f5d47fff700 1 -- 172.21.15.92:0/1067016637 learned_addr learned my addr 172.21.15.92:0/1067016637 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:42.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.733+0000 7f5d4d79c700 1 --2- 172.21.15.92:0/1067016637 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d480d0d60 0x7f5d480d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:42.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.733+0000 7f5d47fff700 1 -- 172.21.15.92:0/1067016637 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d480d0d60 msgr2=0x7f5d480d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:42.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.733+0000 7f5d47fff700 1 --2- 172.21.15.92:0/1067016637 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d480d0d60 0x7f5d480d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:42.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.733+0000 7f5d47fff700 1 -- 172.21.15.92:0/1067016637 >> v1:172.21.15.169:6789/0 conn(0x7f5d480b8510 legacy=0x7f5d4806b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:42.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.733+0000 7f5d47fff700 1 -- 172.21.15.92:0/1067016637 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d480cd8f0 con 0x7f5d480cd460 2023-11-28T11:22:42.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.734+0000 7f5d47fff700 1 --2- 172.21.15.92:0/1067016637 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d480cd460 0x7f5d4806c080 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f5d34009920 tx=0x7f5d34009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbb0d18a5c59d2f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:42.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.734+0000 7f5d477fe700 1 -- 172.21.15.92:0/1067016637 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3400ba20 con 0x7f5d480cd460 2023-11-28T11:22:42.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.734+0000 7f5d477fe700 1 -- 172.21.15.92:0/1067016637 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d3400bb80 con 0x7f5d480cd460 2023-11-28T11:22:42.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.734+0000 7f5d477fe700 1 -- 172.21.15.92:0/1067016637 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d34009e70 con 0x7f5d480cd460 2023-11-28T11:22:42.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.734+0000 7f5d4f1ff700 1 -- 172.21.15.92:0/1067016637 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d480cd460 msgr2=0x7f5d4806c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:42.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.734+0000 7f5d4f1ff700 1 --2- 172.21.15.92:0/1067016637 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d480cd460 0x7f5d4806c080 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f5d34009920 tx=0x7f5d34009c30 comp rx=0 tx=0).stop 2023-11-28T11:22:42.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.735+0000 7f5d4f1ff700 1 -- 172.21.15.92:0/1067016637 shutdown_connections 2023-11-28T11:22:42.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.735+0000 7f5d4f1ff700 1 --2- 172.21.15.92:0/1067016637 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d480d0d60 0x7f5d480d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:42.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.735+0000 7f5d4f1ff700 1 --2- 172.21.15.92:0/1067016637 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d480cd460 0x7f5d4806c080 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:42.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.735+0000 7f5d4f1ff700 1 -- 172.21.15.92:0/1067016637 >> 172.21.15.92:0/1067016637 conn(0x7f5d480719a0 msgr2=0x7f5d48071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:42.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.735+0000 7f5d4f1ff700 1 -- 172.21.15.92:0/1067016637 shutdown_connections 2023-11-28T11:22:42.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.735+0000 7f5d4f1ff700 1 -- 172.21.15.92:0/1067016637 wait complete. 2023-11-28T11:22:42.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.736+0000 7f5d4f1ff700 1 Processor -- start 2023-11-28T11:22:42.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.737+0000 7f5d4f1ff700 1 -- start start 2023-11-28T11:22:42.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.737+0000 7f5d4f1ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d480b8510 0x7f5d481e4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:42.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.737+0000 7f5d4f1ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d480cd460 0x7f5d480cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:42.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.737+0000 7f5d4f1ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d480d0d60 0x7f5d481f2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:42.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.737+0000 7f5d4f1ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d480cbdc0 con 0x7f5d480b8510 2023-11-28T11:22:42.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.737+0000 7f5d4f1ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d480caf30 con 0x7f5d480cd460 2023-11-28T11:22:42.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.737+0000 7f5d4f1ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d480cb140 con 0x7f5d480d0d60 2023-11-28T11:22:42.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.738+0000 7f5d4cf9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d480b8510 0x7f5d481e4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:42.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.738+0000 7f5d4cf9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d480b8510 0x7f5d481e4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52960/0 (socket says 172.21.15.92:52960) 2023-11-28T11:22:42.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.738+0000 7f5d4cf9b700 1 -- 172.21.15.92:0/2741553153 learned_addr learned my addr 172.21.15.92:0/2741553153 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:42.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.738+0000 7f5d47fff700 1 --2- 172.21.15.92:0/2741553153 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d480cd460 0x7f5d480cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:42.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.738+0000 7f5d4d79c700 1 --2- 172.21.15.92:0/2741553153 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d480d0d60 0x7f5d481f2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:42.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.738+0000 7f5d4cf9b700 1 -- 172.21.15.92:0/2741553153 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d480d0d60 msgr2=0x7f5d481f2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:42.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.738+0000 7f5d4cf9b700 1 --2- 172.21.15.92:0/2741553153 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d480d0d60 0x7f5d481f2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:42.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.738+0000 7f5d4cf9b700 1 -- 172.21.15.92:0/2741553153 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d480cd460 msgr2=0x7f5d480cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:42.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.738+0000 7f5d4cf9b700 1 --2- 172.21.15.92:0/2741553153 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d480cd460 0x7f5d480cc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:42.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.738+0000 7f5d4cf9b700 1 -- 172.21.15.92:0/2741553153 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d34009550 con 0x7f5d480b8510 2023-11-28T11:22:42.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.738+0000 7f5d4cf9b700 1 --2- 172.21.15.92:0/2741553153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d480b8510 0x7f5d481e4010 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c00c4e0 tx=0x7f5d3c00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:42.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.739+0000 7f5d457fa700 1 -- 172.21.15.92:0/2741553153 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c0052d0 con 0x7f5d480b8510 2023-11-28T11:22:42.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.739+0000 7f5d457fa700 1 -- 172.21.15.92:0/2741553153 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d3c005430 con 0x7f5d480b8510 2023-11-28T11:22:42.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.739+0000 7f5d4f1ff700 1 -- 172.21.15.92:0/2741553153 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d480cfff0 con 0x7f5d480b8510 2023-11-28T11:22:42.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.739+0000 7f5d457fa700 1 -- 172.21.15.92:0/2741553153 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c0038c0 con 0x7f5d480b8510 2023-11-28T11:22:42.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.739+0000 7f5d4f1ff700 1 -- 172.21.15.92:0/2741553153 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d480cfd10 con 0x7f5d480b8510 2023-11-28T11:22:42.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.740+0000 7f5d4f1ff700 1 -- 172.21.15.92:0/2741553153 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d480cbf00 con 0x7f5d480b8510 2023-11-28T11:22:42.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.741+0000 7f5d457fa700 1 -- 172.21.15.92:0/2741553153 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5d3c019030 con 0x7f5d480b8510 2023-11-28T11:22:42.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.742+0000 7f5d457fa700 1 --2- 172.21.15.92:0/2741553153 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5d2c0682f0 0x7f5d2c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:42.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.742+0000 7f5d457fa700 1 -- 172.21.15.92:0/2741553153 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f5d3c0850d0 con 0x7f5d480b8510 2023-11-28T11:22:42.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.742+0000 7f5d47fff700 1 --2- 172.21.15.92:0/2741553153 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5d2c0682f0 0x7f5d2c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:42.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.743+0000 7f5d47fff700 1 --2- 172.21.15.92:0/2741553153 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5d2c0682f0 0x7f5d2c06a7a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5d480b1de0 tx=0x7f5d34008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:42.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.746+0000 7f5d457fa700 1 -- 172.21.15.92:0/2741553153 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5d480cbf00 con 0x7f5d480b8510 2023-11-28T11:22:42.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.978+0000 7f5d4f1ff700 1 -- 172.21.15.92:0/2741553153 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5d480cbf00 con 0x7f5d480b8510 2023-11-28T11:22:42.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.979+0000 7f5d457fa700 1 -- 172.21.15.92:0/2741553153 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4645 (secure 0 0 0) 0x7f5d480cbf00 con 0x7f5d480b8510 2023-11-28T11:22:42.980 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:22:42.980 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4976,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/3102603064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3102603064},{"type":"v1","addr":"172.21.15.169:6834","nonce":3102603064}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4979,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/79247121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":79247121},{"type":"v1","addr":"172.21.15.92:6837","nonce":79247121}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5221,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/219797192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":219797192},{"type":"v1","addr":"172.21.15.169:6835","nonce":219797192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:22:24.715447+0000","modified":"2023-11-28T11:22:27.733117+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5298},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5298":{"gid":5298,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.92:6835/1577294649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1577294649},{"type":"v1","addr":"172.21.15.92:6835","nonce":1577294649}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T11:22:42.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.981+0000 7f5d4f1ff700 1 -- 172.21.15.92:0/2741553153 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5d2c0682f0 msgr2=0x7f5d2c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:42.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.981+0000 7f5d4f1ff700 1 --2- 172.21.15.92:0/2741553153 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5d2c0682f0 0x7f5d2c06a7a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5d480b1de0 tx=0x7f5d34008030 comp rx=0 tx=0).stop 2023-11-28T11:22:42.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.981+0000 7f5d4f1ff700 1 -- 172.21.15.92:0/2741553153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d480b8510 msgr2=0x7f5d481e4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:42.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.981+0000 7f5d4f1ff700 1 --2- 172.21.15.92:0/2741553153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d480b8510 0x7f5d481e4010 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c00c4e0 tx=0x7f5d3c00c7f0 comp rx=0 tx=0).stop 2023-11-28T11:22:42.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.981+0000 7f5d4f1ff700 1 -- 172.21.15.92:0/2741553153 shutdown_connections 2023-11-28T11:22:42.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.981+0000 7f5d4f1ff700 1 --2- 172.21.15.92:0/2741553153 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d480d0d60 0x7f5d481f2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:42.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.981+0000 7f5d4f1ff700 1 --2- 172.21.15.92:0/2741553153 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5d2c0682f0 0x7f5d2c06a7a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:42.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.981+0000 7f5d4f1ff700 1 --2- 172.21.15.92:0/2741553153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d480b8510 0x7f5d481e4010 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:42.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.981+0000 7f5d4f1ff700 1 --2- 172.21.15.92:0/2741553153 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d480cd460 0x7f5d480cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:42.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.981+0000 7f5d4f1ff700 1 -- 172.21.15.92:0/2741553153 >> 172.21.15.92:0/2741553153 conn(0x7f5d480719a0 msgr2=0x7f5d480b3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:42.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.982+0000 7f5d4f1ff700 1 -- 172.21.15.92:0/2741553153 shutdown_connections 2023-11-28T11:22:42.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:42.982+0000 7f5d4f1ff700 1 -- 172.21.15.92:0/2741553153 wait complete. 2023-11-28T11:22:42.984 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 17 2023-11-28T11:22:43.001 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json tag path /parentdir mytag 2023-11-28T11:22:43.222 INFO:teuthology.orchestra.run.smithi092.stdout:{"return_code":0,"scrub_tag":"mytag","mode":"asynchronous"} 2023-11-28T11:22:43.222 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag" } 2023-11-28T11:22:48.185 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:22:48.190 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:22:53.223 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000000.00000000 scrub_tag 2023-11-28T11:22:53.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.326+0000 7fbce10d50c0 1 Processor -- start 2023-11-28T11:22:53.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.326+0000 7fbce10d50c0 1 -- start start 2023-11-28T11:22:53.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.327+0000 7fbce10d50c0 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558fd5168a40 0x558fd5150ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:53.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.327+0000 7fbce10d50c0 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558fd516d210 0x558fd5162db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:53.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.327+0000 7fbce10d50c0 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558fd51f84c0 0x558fd51f69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:53.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.327+0000 7fbce10d50c0 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fd505afe0 con 0x558fd51f84c0 2023-11-28T11:22:53.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.327+0000 7fbce10d50c0 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fd51ec020 con 0x558fd5168a40 2023-11-28T11:22:53.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.327+0000 7fbce10d50c0 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x558fd51eff80 con 0x558fd516d210 2023-11-28T11:22:53.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.328+0000 7fbcd2f9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558fd51f84c0 0x558fd51f69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:53.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.328+0000 7fbcd2f9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558fd51f84c0 0x558fd51f69b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50562/0 (socket says 172.21.15.92:50562) 2023-11-28T11:22:53.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.328+0000 7fbcd2f9c700 1 -- 172.21.15.92:0/2430938521 learned_addr learned my addr 172.21.15.92:0/2430938521 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:53.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.328+0000 7fbcd279b700 1 --2- 172.21.15.92:0/2430938521 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558fd5168a40 0x558fd5150ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:53.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.328+0000 7fbcd2f9c700 1 -- 172.21.15.92:0/2430938521 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558fd516d210 msgr2=0x558fd5162db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:53.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.328+0000 7fbcd2f9c700 1 --2- 172.21.15.92:0/2430938521 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558fd516d210 0x558fd5162db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:53.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.328+0000 7fbcd2f9c700 1 -- 172.21.15.92:0/2430938521 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558fd5168a40 msgr2=0x558fd5150ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:53.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.328+0000 7fbcd2f9c700 1 --2- 172.21.15.92:0/2430938521 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558fd5168a40 0x558fd5150ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:53.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.328+0000 7fbcd2f9c700 1 -- 172.21.15.92:0/2430938521 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcc4005de0 con 0x558fd51f84c0 2023-11-28T11:22:53.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.328+0000 7fbcd279b700 1 --2- 172.21.15.92:0/2430938521 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558fd5168a40 0x558fd5150ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:22:53.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.328+0000 7fbcd2f9c700 1 --2- 172.21.15.92:0/2430938521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558fd51f84c0 0x558fd51f69b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fbccc006f80 tx=0x7fbccc001820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b579bce3c0dc0d3f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:53.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.328+0000 7fbcd0f98700 1 -- 172.21.15.92:0/2430938521 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbccc012600 con 0x558fd51f84c0 2023-11-28T11:22:53.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.329+0000 7fbcd0f98700 1 -- 172.21.15.92:0/2430938521 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbccc012760 con 0x558fd51f84c0 2023-11-28T11:22:53.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.329+0000 7fbcd0f98700 1 -- 172.21.15.92:0/2430938521 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbccc012af0 con 0x558fd51f84c0 2023-11-28T11:22:53.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.329+0000 7fbce10d50c0 1 -- 172.21.15.92:0/2430938521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558fd51f84c0 msgr2=0x558fd51f69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:53.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.329+0000 7fbce10d50c0 1 --2- 172.21.15.92:0/2430938521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558fd51f84c0 0x558fd51f69b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fbccc006f80 tx=0x7fbccc001820 comp rx=0 tx=0).stop 2023-11-28T11:22:53.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.329+0000 7fbce10d50c0 1 -- 172.21.15.92:0/2430938521 shutdown_connections 2023-11-28T11:22:53.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.329+0000 7fbce10d50c0 1 --2- 172.21.15.92:0/2430938521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558fd51f84c0 0x558fd51f69b0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:53.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.329+0000 7fbce10d50c0 1 --2- 172.21.15.92:0/2430938521 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558fd5168a40 0x558fd5150ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:53.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.329+0000 7fbce10d50c0 1 --2- 172.21.15.92:0/2430938521 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558fd516d210 0x558fd5162db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:53.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.329+0000 7fbce10d50c0 1 -- 172.21.15.92:0/2430938521 >> 172.21.15.92:0/2430938521 conn(0x558fd51235b0 msgr2=0x558fd5161780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:53.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.329+0000 7fbce10d50c0 1 -- 172.21.15.92:0/2430938521 shutdown_connections 2023-11-28T11:22:53.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.329+0000 7fbce10d50c0 1 -- 172.21.15.92:0/2430938521 wait complete. 2023-11-28T11:22:53.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.330+0000 7fbce10d50c0 1 Processor -- start 2023-11-28T11:22:53.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.330+0000 7fbce10d50c0 1 -- start start 2023-11-28T11:22:53.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.331+0000 7fbce10d50c0 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558fd5168a40 0x558fd528e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:53.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.331+0000 7fbce10d50c0 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558fd516d210 0x558fd51eae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:53.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.331+0000 7fbce10d50c0 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558fd51f84c0 0x558fd5151130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:53.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.331+0000 7fbce10d50c0 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fd528ed50 con 0x558fd51f84c0 2023-11-28T11:22:53.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.331+0000 7fbce10d50c0 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fd51eb370 con 0x558fd516d210 2023-11-28T11:22:53.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.331+0000 7fbce10d50c0 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x558fd528e380 con 0x558fd5168a40 2023-11-28T11:22:53.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.331+0000 7fbcd2f9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558fd51f84c0 0x558fd5151130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:53.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.331+0000 7fbcd2f9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558fd51f84c0 0x558fd5151130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50574/0 (socket says 172.21.15.92:50574) 2023-11-28T11:22:53.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.331+0000 7fbcd2f9c700 1 -- 172.21.15.92:0/278542766 learned_addr learned my addr 172.21.15.92:0/278542766 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:53.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.332+0000 7fbcd1f9a700 1 --2- 172.21.15.92:0/278542766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558fd516d210 0x558fd51eae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:53.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.332+0000 7fbcd279b700 1 --2- 172.21.15.92:0/278542766 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558fd5168a40 0x558fd528e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:53.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.332+0000 7fbcd2f9c700 1 -- 172.21.15.92:0/278542766 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558fd5168a40 msgr2=0x558fd528e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:53.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.332+0000 7fbcd2f9c700 1 --2- 172.21.15.92:0/278542766 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x558fd5168a40 0x558fd528e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:53.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.332+0000 7fbcd2f9c700 1 -- 172.21.15.92:0/278542766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558fd516d210 msgr2=0x558fd51eae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:53.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.332+0000 7fbcd2f9c700 1 --2- 172.21.15.92:0/278542766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x558fd516d210 0x558fd51eae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:53.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.332+0000 7fbcd2f9c700 1 -- 172.21.15.92:0/278542766 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558fd50664a0 con 0x558fd51f84c0 2023-11-28T11:22:53.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.332+0000 7fbcd2f9c700 1 --2- 172.21.15.92:0/278542766 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558fd51f84c0 0x558fd5151130 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x558fd51f1840 tx=0x7fbccc001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:53.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.332+0000 7fbcc37fe700 1 -- 172.21.15.92:0/278542766 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbccc012580 con 0x558fd51f84c0 2023-11-28T11:22:53.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.332+0000 7fbcc37fe700 1 -- 172.21.15.92:0/278542766 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbccc01e690 con 0x558fd51f84c0 2023-11-28T11:22:53.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.332+0000 7fbce10d50c0 1 -- 172.21.15.92:0/278542766 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558fd5073120 con 0x558fd51f84c0 2023-11-28T11:22:53.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.332+0000 7fbcc37fe700 1 -- 172.21.15.92:0/278542766 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbccc01ea40 con 0x558fd51f84c0 2023-11-28T11:22:53.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.333+0000 7fbce10d50c0 1 -- 172.21.15.92:0/278542766 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558fd5066830 con 0x558fd51f84c0 2023-11-28T11:22:53.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.334+0000 7fbcc37fe700 1 -- 172.21.15.92:0/278542766 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbccc01b040 con 0x558fd51f84c0 2023-11-28T11:22:53.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.335+0000 7fbcc37fe700 1 --2- 172.21.15.92:0/278542766 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fbcb4066c10 0x7fbcb40690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:53.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.335+0000 7fbcc37fe700 1 -- 172.21.15.92:0/278542766 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fbccc086690 con 0x558fd51f84c0 2023-11-28T11:22:53.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.336+0000 7fbcd279b700 1 --2- 172.21.15.92:0/278542766 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fbcb4066c10 0x7fbcb40690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:53.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.336+0000 7fbce10d50c0 1 --2- 172.21.15.92:0/278542766 >> [v2:172.21.15.92:6808/115823,v1:172.21.15.92:6809/115823] conn(0x558fd51648d0 0x558fd5164cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:53.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.336+0000 7fbce10d50c0 1 -- 172.21.15.92:0/278542766 --> [v2:172.21.15.92:6808/115823,v1:172.21.15.92:6809/115823] -- osd_op(unknown.0.0:1 6.10 6:0cf6ad0f:::10000000000.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x558fd5165210 con 0x558fd51648d0 2023-11-28T11:22:53.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.336+0000 7fbcd1f9a700 1 --2- 172.21.15.92:0/278542766 >> [v2:172.21.15.92:6808/115823,v1:172.21.15.92:6809/115823] conn(0x558fd51648d0 0x558fd5164cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:53.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.336+0000 7fbcd279b700 1 --2- 172.21.15.92:0/278542766 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fbcb4066c10 0x7fbcb40690c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4015040 tx=0x7fbcc400f000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:53.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.337+0000 7fbcd1f9a700 1 --2- 172.21.15.92:0/278542766 >> [v2:172.21.15.92:6808/115823,v1:172.21.15.92:6809/115823] conn(0x558fd51648d0 0x558fd5164cd0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:53.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.337+0000 7fbcd1f9a700 1 -- 172.21.15.92:0/278542766 <== osd.0 v2:172.21.15.92:6808/115823 1 ==== osd_op_reply(1 10000000000.00000000 [getxattr (9) out=9b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7fbcc80096b0 con 0x558fd51648d0 2023-11-28T11:22:53.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.337+0000 7fbce10d50c0 1 -- 172.21.15.92:0/278542766 >> [v2:172.21.15.92:6808/115823,v1:172.21.15.92:6809/115823] conn(0x558fd51648d0 msgr2=0x558fd5164cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:53.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.337+0000 7fbce10d50c0 1 --2- 172.21.15.92:0/278542766 >> [v2:172.21.15.92:6808/115823,v1:172.21.15.92:6809/115823] conn(0x558fd51648d0 0x558fd5164cd0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:53.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.337+0000 7fbce10d50c0 1 -- 172.21.15.92:0/278542766 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fbcb4066c10 msgr2=0x7fbcb40690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:53.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.337+0000 7fbce10d50c0 1 --2- 172.21.15.92:0/278542766 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fbcb4066c10 0x7fbcb40690c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4015040 tx=0x7fbcc400f000 comp rx=0 tx=0).stop 2023-11-28T11:22:53.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.337+0000 7fbce10d50c0 1 -- 172.21.15.92:0/278542766 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558fd51f84c0 msgr2=0x558fd5151130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:53.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.337+0000 7fbce10d50c0 1 --2- 172.21.15.92:0/278542766 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558fd51f84c0 0x558fd5151130 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x558fd51f1840 tx=0x7fbccc001250 comp rx=0 tx=0).stop 2023-11-28T11:22:53.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.337+0000 7fbcd2f9c700 1 -- 172.21.15.92:0/278542766 reap_dead start 2023-11-28T11:22:53.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.337+0000 7fbce10d50c0 1 -- 172.21.15.92:0/278542766 shutdown_connections 2023-11-28T11:22:53.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.337+0000 7fbce10d50c0 1 -- 172.21.15.92:0/278542766 >> 172.21.15.92:0/278542766 conn(0x558fd51235b0 msgr2=0x558fd5161780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:53.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.337+0000 7fbce10d50c0 1 -- 172.21.15.92:0/278542766 shutdown_connections 2023-11-28T11:22:53.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.337+0000 7fbce10d50c0 1 -- 172.21.15.92:0/278542766 wait complete. 2023-11-28T11:22:53.342 INFO:teuthology.orchestra.run.smithi092.stdout:mytag 2023-11-28T11:22:53.342 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000001.00000000 scrub_tag 2023-11-28T11:22:53.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.430+0000 7f83474450c0 1 Processor -- start 2023-11-28T11:22:53.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.430+0000 7f83474450c0 1 -- start start 2023-11-28T11:22:53.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.431+0000 7f83474450c0 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5631d05332f0 0x5631d0517e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:53.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.431+0000 7f83474450c0 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5631d0533cf0 0x5631d0530800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:53.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.431+0000 7f83474450c0 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631d05bb530 0x5631d05bb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:53.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.431+0000 7f83474450c0 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631d048f000 con 0x5631d05bb530 2023-11-28T11:22:53.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.431+0000 7f83474450c0 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631d0421fe0 con 0x5631d05332f0 2023-11-28T11:22:53.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.431+0000 7f83474450c0 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5631d0516e30 con 0x5631d0533cf0 2023-11-28T11:22:53.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.431+0000 7f833939c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631d05bb530 0x5631d05bb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:53.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.431+0000 7f833939c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631d05bb530 0x5631d05bb9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50582/0 (socket says 172.21.15.92:50582) 2023-11-28T11:22:53.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.431+0000 7f833939c700 1 -- 172.21.15.92:0/2312922388 learned_addr learned my addr 172.21.15.92:0/2312922388 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:53.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.431+0000 7f8338b9b700 1 --2- 172.21.15.92:0/2312922388 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5631d05332f0 0x5631d0517e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:53.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.432+0000 7f8333fff700 1 --2- 172.21.15.92:0/2312922388 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5631d0533cf0 0x5631d0530800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:53.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.432+0000 7f833939c700 1 -- 172.21.15.92:0/2312922388 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5631d0533cf0 msgr2=0x5631d0530800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:53.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.432+0000 7f833939c700 1 --2- 172.21.15.92:0/2312922388 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5631d0533cf0 0x5631d0530800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:53.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.432+0000 7f833939c700 1 -- 172.21.15.92:0/2312922388 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5631d05332f0 msgr2=0x5631d0517e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:53.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.432+0000 7f833939c700 1 --2- 172.21.15.92:0/2312922388 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5631d05332f0 0x5631d0517e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:53.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.432+0000 7f833939c700 1 -- 172.21.15.92:0/2312922388 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832c005d20 con 0x5631d05bb530 2023-11-28T11:22:53.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.432+0000 7f8338b9b700 1 --2- 172.21.15.92:0/2312922388 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5631d05332f0 0x5631d0517e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:22:53.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.432+0000 7f8333fff700 1 --2- 172.21.15.92:0/2312922388 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5631d0533cf0 0x5631d0530800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:22:53.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.432+0000 7f833939c700 1 --2- 172.21.15.92:0/2312922388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631d05bb530 0x5631d05bb9d0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f8334006f50 tx=0x7f8334001820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fabcada50773745 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:53.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.432+0000 7f8332ffd700 1 -- 172.21.15.92:0/2312922388 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8334012600 con 0x5631d05bb530 2023-11-28T11:22:53.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.432+0000 7f8332ffd700 1 -- 172.21.15.92:0/2312922388 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8334012760 con 0x5631d05bb530 2023-11-28T11:22:53.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.432+0000 7f8332ffd700 1 -- 172.21.15.92:0/2312922388 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8334012b10 con 0x5631d05bb530 2023-11-28T11:22:53.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.433+0000 7f83474450c0 1 -- 172.21.15.92:0/2312922388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631d05bb530 msgr2=0x5631d05bb9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:53.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.433+0000 7f83474450c0 1 --2- 172.21.15.92:0/2312922388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631d05bb530 0x5631d05bb9d0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f8334006f50 tx=0x7f8334001820 comp rx=0 tx=0).stop 2023-11-28T11:22:53.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.433+0000 7f83474450c0 1 -- 172.21.15.92:0/2312922388 shutdown_connections 2023-11-28T11:22:53.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.433+0000 7f83474450c0 1 --2- 172.21.15.92:0/2312922388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631d05bb530 0x5631d05bb9d0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:53.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.433+0000 7f83474450c0 1 --2- 172.21.15.92:0/2312922388 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5631d05332f0 0x5631d0517e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:53.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.433+0000 7f83474450c0 1 --2- 172.21.15.92:0/2312922388 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5631d0533cf0 0x5631d0530800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:53.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.433+0000 7f83474450c0 1 -- 172.21.15.92:0/2312922388 >> 172.21.15.92:0/2312922388 conn(0x5631d04ea5b0 msgr2=0x5631d0529610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:53.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.433+0000 7f83474450c0 1 -- 172.21.15.92:0/2312922388 shutdown_connections 2023-11-28T11:22:53.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.433+0000 7f83474450c0 1 -- 172.21.15.92:0/2312922388 wait complete. 2023-11-28T11:22:53.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.434+0000 7f83474450c0 1 Processor -- start 2023-11-28T11:22:53.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.434+0000 7f83474450c0 1 -- start start 2023-11-28T11:22:53.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.435+0000 7f83474450c0 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631d05332f0 0x5631d05bbb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:53.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.435+0000 7f83474450c0 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5631d0533cf0 0x5631d05b2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:53.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.435+0000 7f83474450c0 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5631d05bb530 0x5631d05b2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:53.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.435+0000 7f83474450c0 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631d05c3a50 con 0x5631d05332f0 2023-11-28T11:22:53.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.435+0000 7f83474450c0 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631d05c3520 con 0x5631d05bb530 2023-11-28T11:22:53.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.435+0000 7f83474450c0 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5631d0515410 con 0x5631d0533cf0 2023-11-28T11:22:53.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.435+0000 7f8338b9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631d05332f0 0x5631d05bbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:53.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.435+0000 7f8338b9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631d05332f0 0x5631d05bbb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50590/0 (socket says 172.21.15.92:50590) 2023-11-28T11:22:53.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.435+0000 7f8338b9b700 1 -- 172.21.15.92:0/1310631883 learned_addr learned my addr 172.21.15.92:0/1310631883 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:53.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.436+0000 7f833939c700 1 --2- 172.21.15.92:0/1310631883 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5631d05bb530 0x5631d05b2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:53.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.436+0000 7f8333fff700 1 --2- 172.21.15.92:0/1310631883 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5631d0533cf0 0x5631d05b2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:53.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.436+0000 7f8338b9b700 1 -- 172.21.15.92:0/1310631883 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5631d0533cf0 msgr2=0x5631d05b2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:53.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.436+0000 7f8338b9b700 1 --2- 172.21.15.92:0/1310631883 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5631d0533cf0 0x5631d05b2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:53.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.436+0000 7f8338b9b700 1 -- 172.21.15.92:0/1310631883 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5631d05bb530 msgr2=0x5631d05b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:53.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.436+0000 7f8338b9b700 1 --2- 172.21.15.92:0/1310631883 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5631d05bb530 0x5631d05b2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:53.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.436+0000 7f8338b9b700 1 -- 172.21.15.92:0/1310631883 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5631d04544f0 con 0x5631d05332f0 2023-11-28T11:22:53.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.436+0000 7f8338b9b700 1 --2- 172.21.15.92:0/1310631883 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631d05332f0 0x5631d05bbb00 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f832c015040 tx=0x7f832c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:53.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.436+0000 7f8331ffb700 1 -- 172.21.15.92:0/1310631883 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832c01c820 con 0x5631d05332f0 2023-11-28T11:22:53.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.436+0000 7f8331ffb700 1 -- 172.21.15.92:0/1310631883 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f832c01c980 con 0x5631d05332f0 2023-11-28T11:22:53.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.436+0000 7f83474450c0 1 -- 172.21.15.92:0/1310631883 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5631d042d830 con 0x5631d05332f0 2023-11-28T11:22:53.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.436+0000 7f8331ffb700 1 -- 172.21.15.92:0/1310631883 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832c01cd10 con 0x5631d05332f0 2023-11-28T11:22:53.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.436+0000 7f83474450c0 1 -- 172.21.15.92:0/1310631883 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5631d043a360 con 0x5631d05332f0 2023-11-28T11:22:53.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.438+0000 7f8331ffb700 1 -- 172.21.15.92:0/1310631883 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f832c02e070 con 0x5631d05332f0 2023-11-28T11:22:53.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.439+0000 7f8331ffb700 1 --2- 172.21.15.92:0/1310631883 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f831c066c10 0x7f831c0690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:53.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.439+0000 7f8331ffb700 1 -- 172.21.15.92:0/1310631883 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f832c085ff0 con 0x5631d05332f0 2023-11-28T11:22:53.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.439+0000 7f8333fff700 1 --2- 172.21.15.92:0/1310631883 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f831c066c10 0x7f831c0690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:53.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.439+0000 7f83474450c0 1 --2- 172.21.15.92:0/1310631883 >> [v2:172.21.15.169:6800/116033,v1:172.21.15.169:6801/116033] conn(0x5631d05bc040 0x5631d0648d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:53.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.439+0000 7f83474450c0 1 -- 172.21.15.92:0/1310631883 --> [v2:172.21.15.169:6800/116033,v1:172.21.15.169:6801/116033] -- osd_op(unknown.0.0:1 6.7 6:e2e07930:::10000000001.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x5631d0649250 con 0x5631d05bc040 2023-11-28T11:22:53.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.440+0000 7f833939c700 1 --2- 172.21.15.92:0/1310631883 >> [v2:172.21.15.169:6800/116033,v1:172.21.15.169:6801/116033] conn(0x5631d05bc040 0x5631d0648d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:53.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.440+0000 7f8333fff700 1 --2- 172.21.15.92:0/1310631883 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f831c066c10 0x7f831c0690c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x5631d064ba00 tx=0x7f8324001670 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:53.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.440+0000 7f833939c700 1 --2- 172.21.15.92:0/1310631883 >> [v2:172.21.15.169:6800/116033,v1:172.21.15.169:6801/116033] conn(0x5631d05bc040 0x5631d0648d10 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:53.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.441+0000 7f833939c700 1 -- 172.21.15.92:0/1310631883 <== osd.5 v2:172.21.15.169:6800/116033 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (9) out=9b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f83340128d0 con 0x5631d05bc040 2023-11-28T11:22:53.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.441+0000 7f83474450c0 1 -- 172.21.15.92:0/1310631883 >> [v2:172.21.15.169:6800/116033,v1:172.21.15.169:6801/116033] conn(0x5631d05bc040 msgr2=0x5631d0648d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:53.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.441+0000 7f83474450c0 1 --2- 172.21.15.92:0/1310631883 >> [v2:172.21.15.169:6800/116033,v1:172.21.15.169:6801/116033] conn(0x5631d05bc040 0x5631d0648d10 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:53.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.441+0000 7f83474450c0 1 -- 172.21.15.92:0/1310631883 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f831c066c10 msgr2=0x7f831c0690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:53.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.441+0000 7f83474450c0 1 --2- 172.21.15.92:0/1310631883 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f831c066c10 0x7f831c0690c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x5631d064ba00 tx=0x7f8324001670 comp rx=0 tx=0).stop 2023-11-28T11:22:53.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.441+0000 7f83474450c0 1 -- 172.21.15.92:0/1310631883 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631d05332f0 msgr2=0x5631d05bbb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:53.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.441+0000 7f83474450c0 1 --2- 172.21.15.92:0/1310631883 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5631d05332f0 0x5631d05bbb00 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f832c015040 tx=0x7f832c002fb0 comp rx=0 tx=0).stop 2023-11-28T11:22:53.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.441+0000 7f833939c700 1 -- 172.21.15.92:0/1310631883 reap_dead start 2023-11-28T11:22:53.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.441+0000 7f83474450c0 1 -- 172.21.15.92:0/1310631883 shutdown_connections 2023-11-28T11:22:53.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.441+0000 7f83474450c0 1 -- 172.21.15.92:0/1310631883 >> 172.21.15.92:0/1310631883 conn(0x5631d04ea5b0 msgr2=0x5631d0529610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:53.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.441+0000 7f83474450c0 1 -- 172.21.15.92:0/1310631883 shutdown_connections 2023-11-28T11:22:53.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.441+0000 7f83474450c0 1 -- 172.21.15.92:0/1310631883 wait complete. 2023-11-28T11:22:53.446 INFO:teuthology.orchestra.run.smithi092.stdout:mytag 2023-11-28T11:22:53.447 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2023-11-28T11:22:53.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.533+0000 7f2596b070c0 1 Processor -- start 2023-11-28T11:22:53.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.533+0000 7f2596b070c0 1 -- start start 2023-11-28T11:22:53.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.534+0000 7f2596b070c0 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ff685d8a40 0x55ff68666200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:53.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.534+0000 7f2596b070c0 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff685dd210 0x55ff685dbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:53.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.534+0000 7f2596b070c0 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ff686684c0 0x55ff685c0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:53.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.534+0000 7f2596b070c0 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff684cafe0 con 0x55ff685dd210 2023-11-28T11:22:53.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.534+0000 7f2596b070c0 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff68666740 con 0x55ff686684c0 2023-11-28T11:22:53.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.534+0000 7f2596b070c0 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ff685c0aa0 con 0x55ff685d8a40 2023-11-28T11:22:53.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.534+0000 7f258799a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff685dd210 0x55ff685dbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:53.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.534+0000 7f258799a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff685dd210 0x55ff685dbbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50614/0 (socket says 172.21.15.92:50614) 2023-11-28T11:22:53.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.534+0000 7f258799a700 1 -- 172.21.15.92:0/3568455978 learned_addr learned my addr 172.21.15.92:0/3568455978 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:53.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.534+0000 7f258819b700 1 --2- 172.21.15.92:0/3568455978 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ff685d8a40 0x55ff68666200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:53.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.534+0000 7f258899c700 1 --2- 172.21.15.92:0/3568455978 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ff686684c0 0x55ff685c0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:53.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.535+0000 7f258799a700 1 -- 172.21.15.92:0/3568455978 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ff685d8a40 msgr2=0x55ff68666200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:53.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.535+0000 7f258799a700 1 --2- 172.21.15.92:0/3568455978 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ff685d8a40 0x55ff68666200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:53.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.535+0000 7f258799a700 1 -- 172.21.15.92:0/3568455978 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ff686684c0 msgr2=0x55ff685c0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:53.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.535+0000 7f258799a700 1 --2- 172.21.15.92:0/3568455978 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ff686684c0 0x55ff685c0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:53.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.535+0000 7f258799a700 1 -- 172.21.15.92:0/3568455978 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2578005de0 con 0x55ff685dd210 2023-11-28T11:22:53.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.535+0000 7f258799a700 1 --2- 172.21.15.92:0/3568455978 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff685dd210 0x55ff685dbbd0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f257c00a770 tx=0x7f257c00aa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9403d3303e2f40e4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:53.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.535+0000 7f2586998700 1 -- 172.21.15.92:0/3568455978 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c00f070 con 0x55ff685dd210 2023-11-28T11:22:53.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.535+0000 7f2586998700 1 -- 172.21.15.92:0/3568455978 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f257c007600 con 0x55ff685dd210 2023-11-28T11:22:53.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.535+0000 7f2586998700 1 -- 172.21.15.92:0/3568455978 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c007970 con 0x55ff685dd210 2023-11-28T11:22:53.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.536+0000 7f2596b070c0 1 -- 172.21.15.92:0/3568455978 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff685dd210 msgr2=0x55ff685dbbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:53.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.536+0000 7f2596b070c0 1 --2- 172.21.15.92:0/3568455978 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff685dd210 0x55ff685dbbd0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f257c00a770 tx=0x7f257c00aa80 comp rx=0 tx=0).stop 2023-11-28T11:22:53.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.536+0000 7f2596b070c0 1 -- 172.21.15.92:0/3568455978 shutdown_connections 2023-11-28T11:22:53.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.536+0000 7f2596b070c0 1 --2- 172.21.15.92:0/3568455978 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ff686684c0 0x55ff685c0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:53.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.536+0000 7f2596b070c0 1 --2- 172.21.15.92:0/3568455978 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ff685d8a40 0x55ff68666200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:53.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.536+0000 7f2596b070c0 1 --2- 172.21.15.92:0/3568455978 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff685dd210 0x55ff685dbbd0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:53.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.536+0000 7f2596b070c0 1 -- 172.21.15.92:0/3568455978 >> 172.21.15.92:0/3568455978 conn(0x55ff685935b0 msgr2=0x55ff685d1780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:53.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.536+0000 7f2596b070c0 1 -- 172.21.15.92:0/3568455978 shutdown_connections 2023-11-28T11:22:53.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.536+0000 7f2596b070c0 1 -- 172.21.15.92:0/3568455978 wait complete. 2023-11-28T11:22:53.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.536+0000 7f2596b070c0 1 Processor -- start 2023-11-28T11:22:53.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.537+0000 7f2596b070c0 1 -- start start 2023-11-28T11:22:53.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.537+0000 7f2596b070c0 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ff685d8a40 0x55ff68659ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:53.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.537+0000 7f2596b070c0 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff685dd210 0x55ff68660e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:53.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.537+0000 7f2596b070c0 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ff686684c0 0x55ff68661370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:53.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.537+0000 7f2596b070c0 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff685d5b60 con 0x55ff685dd210 2023-11-28T11:22:53.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.537+0000 7f2596b070c0 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff685d4b10 con 0x55ff686684c0 2023-11-28T11:22:53.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.537+0000 7f2596b070c0 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ff6865f890 con 0x55ff685d8a40 2023-11-28T11:22:53.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.538+0000 7f258799a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff685dd210 0x55ff68660e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:53.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.538+0000 7f258799a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff685dd210 0x55ff68660e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50626/0 (socket says 172.21.15.92:50626) 2023-11-28T11:22:53.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.538+0000 7f258799a700 1 -- 172.21.15.92:0/1564799267 learned_addr learned my addr 172.21.15.92:0/1564799267 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:53.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.538+0000 7f258819b700 1 --2- 172.21.15.92:0/1564799267 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ff685d8a40 0x55ff68659ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:53.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.538+0000 7f258899c700 1 --2- 172.21.15.92:0/1564799267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ff686684c0 0x55ff68661370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:53.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.538+0000 7f258799a700 1 -- 172.21.15.92:0/1564799267 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ff685d8a40 msgr2=0x55ff68659ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:53.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.538+0000 7f258799a700 1 --2- 172.21.15.92:0/1564799267 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55ff685d8a40 0x55ff68659ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:53.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.538+0000 7f258799a700 1 -- 172.21.15.92:0/1564799267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ff686684c0 msgr2=0x55ff68661370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:53.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.538+0000 7f258799a700 1 --2- 172.21.15.92:0/1564799267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ff686684c0 0x55ff68661370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:53.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.538+0000 7f258799a700 1 -- 172.21.15.92:0/1564799267 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f257c00a420 con 0x55ff685dd210 2023-11-28T11:22:53.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.538+0000 7f258899c700 1 --2- 172.21.15.92:0/1564799267 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55ff686684c0 0x55ff68661370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:22:53.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.538+0000 7f258799a700 1 --2- 172.21.15.92:0/1564799267 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff685dd210 0x55ff68660e30 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f257c017040 tx=0x7f257c003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:53.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.539+0000 7f2585996700 1 -- 172.21.15.92:0/1564799267 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c014880 con 0x55ff685dd210 2023-11-28T11:22:53.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.539+0000 7f2585996700 1 -- 172.21.15.92:0/1564799267 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f257c0149e0 con 0x55ff685dd210 2023-11-28T11:22:53.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.539+0000 7f2596b070c0 1 -- 172.21.15.92:0/1564799267 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ff684e3120 con 0x55ff685dd210 2023-11-28T11:22:53.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.539+0000 7f2585996700 1 -- 172.21.15.92:0/1564799267 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c014d90 con 0x55ff685dd210 2023-11-28T11:22:53.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.539+0000 7f2596b070c0 1 -- 172.21.15.92:0/1564799267 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ff684d6830 con 0x55ff685dd210 2023-11-28T11:22:53.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.541+0000 7f2585996700 1 -- 172.21.15.92:0/1564799267 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f257c01f440 con 0x55ff685dd210 2023-11-28T11:22:53.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.541+0000 7f2585996700 1 --2- 172.21.15.92:0/1564799267 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f2574066c10 0x7f25740690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:53.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.541+0000 7f2585996700 1 -- 172.21.15.92:0/1564799267 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f257c0860c0 con 0x55ff685dd210 2023-11-28T11:22:53.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.542+0000 7f258819b700 1 --2- 172.21.15.92:0/1564799267 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f2574066c10 0x7f25740690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:53.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.542+0000 7f2596b070c0 1 --2- 172.21.15.92:0/1564799267 >> [v2:172.21.15.92:6808/115823,v1:172.21.15.92:6809/115823] conn(0x55ff6865bc80 0x55ff68663ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:53.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.542+0000 7f2596b070c0 1 -- 172.21.15.92:0/1564799267 --> [v2:172.21.15.92:6808/115823,v1:172.21.15.92:6809/115823] -- osd_op(unknown.0.0:1 7.36 7:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x55ff68664010 con 0x55ff6865bc80 2023-11-28T11:22:53.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.542+0000 7f258899c700 1 --2- 172.21.15.92:0/1564799267 >> [v2:172.21.15.92:6808/115823,v1:172.21.15.92:6809/115823] conn(0x55ff6865bc80 0x55ff68663ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:53.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.542+0000 7f258819b700 1 --2- 172.21.15.92:0/1564799267 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f2574066c10 0x7f25740690c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f257800a4f0 tx=0x7f257800f000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:53.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.542+0000 7f258899c700 1 --2- 172.21.15.92:0/1564799267 >> [v2:172.21.15.92:6808/115823,v1:172.21.15.92:6809/115823] conn(0x55ff6865bc80 0x55ff68663ad0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:53.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.543+0000 7f258899c700 1 -- 172.21.15.92:0/1564799267 <== osd.0 v2:172.21.15.92:6808/115823 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr (9) out=9b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f2580006020 con 0x55ff6865bc80 2023-11-28T11:22:53.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.543+0000 7f2596b070c0 1 -- 172.21.15.92:0/1564799267 >> [v2:172.21.15.92:6808/115823,v1:172.21.15.92:6809/115823] conn(0x55ff6865bc80 msgr2=0x55ff68663ad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:53.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.543+0000 7f2596b070c0 1 --2- 172.21.15.92:0/1564799267 >> [v2:172.21.15.92:6808/115823,v1:172.21.15.92:6809/115823] conn(0x55ff6865bc80 0x55ff68663ad0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:53.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.543+0000 7f2596b070c0 1 -- 172.21.15.92:0/1564799267 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f2574066c10 msgr2=0x7f25740690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:53.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.543+0000 7f2596b070c0 1 --2- 172.21.15.92:0/1564799267 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f2574066c10 0x7f25740690c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f257800a4f0 tx=0x7f257800f000 comp rx=0 tx=0).stop 2023-11-28T11:22:53.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.543+0000 7f2596b070c0 1 -- 172.21.15.92:0/1564799267 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff685dd210 msgr2=0x55ff68660e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:53.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.543+0000 7f2596b070c0 1 --2- 172.21.15.92:0/1564799267 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff685dd210 0x55ff68660e30 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f257c017040 tx=0x7f257c003020 comp rx=0 tx=0).stop 2023-11-28T11:22:53.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.543+0000 7f258899c700 1 -- 172.21.15.92:0/1564799267 reap_dead start 2023-11-28T11:22:53.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.543+0000 7f2596b070c0 1 -- 172.21.15.92:0/1564799267 shutdown_connections 2023-11-28T11:22:53.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.543+0000 7f2596b070c0 1 -- 172.21.15.92:0/1564799267 >> 172.21.15.92:0/1564799267 conn(0x55ff685935b0 msgr2=0x55ff685dbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:53.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.543+0000 7f2596b070c0 1 -- 172.21.15.92:0/1564799267 shutdown_connections 2023-11-28T11:22:53.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.543+0000 7f2596b070c0 1 -- 172.21.15.92:0/1564799267 wait complete. 2023-11-28T11:22:53.548 INFO:teuthology.orchestra.run.smithi092.stdout:mytag 2023-11-28T11:22:53.548 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2023-11-28T11:22:53.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.634+0000 7f2af904f0c0 1 Processor -- start 2023-11-28T11:22:53.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.634+0000 7f2af904f0c0 1 -- start start 2023-11-28T11:22:53.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.635+0000 7f2af904f0c0 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b75dea6f20 0x55b75de9e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:53.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.635+0000 7f2af904f0c0 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b75dea72f0 0x55b75dea4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:53.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.635+0000 7f2af904f0c0 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b75dea8400 0x55b75de87510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:53.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.635+0000 7f2af904f0c0 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b75de03000 con 0x55b75dea8400 2023-11-28T11:22:53.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.635+0000 7f2af904f0c0 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b75dd95fe0 con 0x55b75dea6f20 2023-11-28T11:22:53.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.635+0000 7f2af904f0c0 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b75df277a0 con 0x55b75dea72f0 2023-11-28T11:22:53.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.635+0000 7f2aeaf9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b75dea8400 0x55b75de87510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:53.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.635+0000 7f2aeaf9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b75dea8400 0x55b75de87510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50640/0 (socket says 172.21.15.92:50640) 2023-11-28T11:22:53.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.635+0000 7f2aeaf9c700 1 -- 172.21.15.92:0/1026436536 learned_addr learned my addr 172.21.15.92:0/1026436536 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:53.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.635+0000 7f2ae9f9a700 1 --2- 172.21.15.92:0/1026436536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b75dea72f0 0x55b75dea4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:53.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.635+0000 7f2aea79b700 1 --2- 172.21.15.92:0/1026436536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b75dea6f20 0x55b75de9e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:53.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.635+0000 7f2aeaf9c700 1 -- 172.21.15.92:0/1026436536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b75dea72f0 msgr2=0x55b75dea4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:53.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.635+0000 7f2aeaf9c700 1 --2- 172.21.15.92:0/1026436536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b75dea72f0 0x55b75dea4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:53.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.635+0000 7f2aeaf9c700 1 -- 172.21.15.92:0/1026436536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b75dea6f20 msgr2=0x55b75de9e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:53.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.635+0000 7f2aeaf9c700 1 --2- 172.21.15.92:0/1026436536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b75dea6f20 0x55b75de9e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:53.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.635+0000 7f2aeaf9c700 1 -- 172.21.15.92:0/1026436536 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ae400a100 con 0x55b75dea8400 2023-11-28T11:22:53.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.636+0000 7f2aeaf9c700 1 --2- 172.21.15.92:0/1026436536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b75dea8400 0x55b75de87510 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x55b75dea37e0 tx=0x7f2ae400f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a16e90fa3e4a62b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:53.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.636+0000 7f2ae8f98700 1 -- 172.21.15.92:0/1026436536 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae400f790 con 0x55b75dea8400 2023-11-28T11:22:53.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.636+0000 7f2ae8f98700 1 -- 172.21.15.92:0/1026436536 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ae400fa60 con 0x55b75dea8400 2023-11-28T11:22:53.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.636+0000 7f2ae8f98700 1 -- 172.21.15.92:0/1026436536 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae400fe10 con 0x55b75dea8400 2023-11-28T11:22:53.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.636+0000 7f2af904f0c0 1 -- 172.21.15.92:0/1026436536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b75dea8400 msgr2=0x55b75de87510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:53.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.636+0000 7f2af904f0c0 1 --2- 172.21.15.92:0/1026436536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b75dea8400 0x55b75de87510 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x55b75dea37e0 tx=0x7f2ae400f000 comp rx=0 tx=0).stop 2023-11-28T11:22:53.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.636+0000 7f2af904f0c0 1 -- 172.21.15.92:0/1026436536 shutdown_connections 2023-11-28T11:22:53.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.636+0000 7f2af904f0c0 1 --2- 172.21.15.92:0/1026436536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b75dea8400 0x55b75de87510 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:53.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.636+0000 7f2af904f0c0 1 --2- 172.21.15.92:0/1026436536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b75dea6f20 0x55b75de9e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:53.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.636+0000 7f2af904f0c0 1 --2- 172.21.15.92:0/1026436536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b75dea72f0 0x55b75dea4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:53.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.636+0000 7f2af904f0c0 1 -- 172.21.15.92:0/1026436536 >> 172.21.15.92:0/1026436536 conn(0x55b75de5e5b0 msgr2=0x55b75dea6390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:53.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.636+0000 7f2af904f0c0 1 -- 172.21.15.92:0/1026436536 shutdown_connections 2023-11-28T11:22:53.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.636+0000 7f2af904f0c0 1 -- 172.21.15.92:0/1026436536 wait complete. 2023-11-28T11:22:53.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.637+0000 7f2af904f0c0 1 Processor -- start 2023-11-28T11:22:53.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.637+0000 7f2af904f0c0 1 -- start start 2023-11-28T11:22:53.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.638+0000 7f2af904f0c0 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b75dea6f20 0x55b75df26480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:53.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.638+0000 7f2af904f0c0 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b75dea72f0 0x55b75dea6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:53.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.638+0000 7f2af904f0c0 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b75dea8400 0x55b75df31160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:53.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.638+0000 7f2af904f0c0 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b75dea68d0 con 0x55b75dea72f0 2023-11-28T11:22:53.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.638+0000 7f2af904f0c0 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b75de9f370 con 0x55b75dea8400 2023-11-28T11:22:53.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.638+0000 7f2af904f0c0 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b75de9f650 con 0x55b75dea6f20 2023-11-28T11:22:53.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.638+0000 7f2ae9f9a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b75dea72f0 0x55b75dea6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:53.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.638+0000 7f2ae9f9a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b75dea72f0 0x55b75dea6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50650/0 (socket says 172.21.15.92:50650) 2023-11-28T11:22:53.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.638+0000 7f2ae9f9a700 1 -- 172.21.15.92:0/446239739 learned_addr learned my addr 172.21.15.92:0/446239739 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:53.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.639+0000 7f2ae9f9a700 1 -- 172.21.15.92:0/446239739 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b75dea6f20 msgr2=0x55b75df26480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:53.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.639+0000 7f2ae9f9a700 1 --2- 172.21.15.92:0/446239739 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55b75dea6f20 0x55b75df26480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:53.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.639+0000 7f2ae9f9a700 1 -- 172.21.15.92:0/446239739 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b75dea8400 msgr2=0x55b75df31160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:53.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.639+0000 7f2ae9f9a700 1 --2- 172.21.15.92:0/446239739 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55b75dea8400 0x55b75df31160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:53.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.639+0000 7f2ae9f9a700 1 -- 172.21.15.92:0/446239739 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b75ddae120 con 0x55b75dea72f0 2023-11-28T11:22:53.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.639+0000 7f2ae9f9a700 1 --2- 172.21.15.92:0/446239739 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b75dea72f0 0x55b75dea6390 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f2ae0000f60 tx=0x7f2ae000b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:53.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.639+0000 7f2adb7fe700 1 -- 172.21.15.92:0/446239739 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae000de70 con 0x55b75dea72f0 2023-11-28T11:22:53.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.639+0000 7f2adb7fe700 1 -- 172.21.15.92:0/446239739 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ae000f450 con 0x55b75dea72f0 2023-11-28T11:22:53.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.639+0000 7f2af904f0c0 1 -- 172.21.15.92:0/446239739 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b75dda1830 con 0x55b75dea72f0 2023-11-28T11:22:53.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.639+0000 7f2adb7fe700 1 -- 172.21.15.92:0/446239739 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae000f800 con 0x55b75dea72f0 2023-11-28T11:22:53.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.639+0000 7f2af904f0c0 1 -- 172.21.15.92:0/446239739 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b75ddae360 con 0x55b75dea72f0 2023-11-28T11:22:53.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.641+0000 7f2adb7fe700 1 -- 172.21.15.92:0/446239739 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2ae000f960 con 0x55b75dea72f0 2023-11-28T11:22:53.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.642+0000 7f2adb7fe700 1 --2- 172.21.15.92:0/446239739 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f2acc066c10 0x7f2acc0690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:53.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.642+0000 7f2adb7fe700 1 -- 172.21.15.92:0/446239739 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f2ae0085180 con 0x55b75dea72f0 2023-11-28T11:22:53.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.642+0000 7f2aea79b700 1 --2- 172.21.15.92:0/446239739 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f2acc066c10 0x7f2acc0690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:53.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.642+0000 7f2af904f0c0 1 --2- 172.21.15.92:0/446239739 >> [v2:172.21.15.169:6808/116034,v1:172.21.15.169:6809/116034] conn(0x55b75df26c10 0x55b75dfc8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:53.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.642+0000 7f2af904f0c0 1 -- 172.21.15.92:0/446239739 --> [v2:172.21.15.169:6808/116034,v1:172.21.15.169:6809/116034] -- osd_op(unknown.0.0:1 7.19 7:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x55b75dfc93f0 con 0x55b75df26c10 2023-11-28T11:22:53.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.643+0000 7f2aea79b700 1 --2- 172.21.15.92:0/446239739 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f2acc066c10 0x7f2acc0690c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2adc000f60 tx=0x7f2adc003410 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:53.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.643+0000 7f2aeaf9c700 1 --2- 172.21.15.92:0/446239739 >> [v2:172.21.15.169:6808/116034,v1:172.21.15.169:6809/116034] conn(0x55b75df26c10 0x55b75dfc8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:53.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.644+0000 7f2aeaf9c700 1 --2- 172.21.15.92:0/446239739 >> [v2:172.21.15.169:6808/116034,v1:172.21.15.169:6809/116034] conn(0x55b75df26c10 0x55b75dfc8eb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:53.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.644+0000 7f2aeaf9c700 1 -- 172.21.15.92:0/446239739 <== osd.6 v2:172.21.15.169:6808/116034 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (9) out=9b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f2ae400fbd0 con 0x55b75df26c10 2023-11-28T11:22:53.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.644+0000 7f2af904f0c0 1 -- 172.21.15.92:0/446239739 >> [v2:172.21.15.169:6808/116034,v1:172.21.15.169:6809/116034] conn(0x55b75df26c10 msgr2=0x55b75dfc8eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:53.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.644+0000 7f2af904f0c0 1 --2- 172.21.15.92:0/446239739 >> [v2:172.21.15.169:6808/116034,v1:172.21.15.169:6809/116034] conn(0x55b75df26c10 0x55b75dfc8eb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:53.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.644+0000 7f2af904f0c0 1 -- 172.21.15.92:0/446239739 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f2acc066c10 msgr2=0x7f2acc0690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:53.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.644+0000 7f2af904f0c0 1 --2- 172.21.15.92:0/446239739 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f2acc066c10 0x7f2acc0690c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2adc000f60 tx=0x7f2adc003410 comp rx=0 tx=0).stop 2023-11-28T11:22:53.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.644+0000 7f2af904f0c0 1 -- 172.21.15.92:0/446239739 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b75dea72f0 msgr2=0x55b75dea6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:53.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.644+0000 7f2af904f0c0 1 --2- 172.21.15.92:0/446239739 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b75dea72f0 0x55b75dea6390 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f2ae0000f60 tx=0x7f2ae000b5d0 comp rx=0 tx=0).stop 2023-11-28T11:22:53.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.644+0000 7f2aeaf9c700 1 -- 172.21.15.92:0/446239739 reap_dead start 2023-11-28T11:22:53.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.644+0000 7f2af904f0c0 1 -- 172.21.15.92:0/446239739 shutdown_connections 2023-11-28T11:22:53.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.644+0000 7f2af904f0c0 1 -- 172.21.15.92:0/446239739 >> 172.21.15.92:0/446239739 conn(0x55b75de5e5b0 msgr2=0x55b75de9df40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:53.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.645+0000 7f2af904f0c0 1 -- 172.21.15.92:0/446239739 shutdown_connections 2023-11-28T11:22:53.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.645+0000 7f2af904f0c0 1 -- 172.21.15.92:0/446239739 wait complete. 2023-11-28T11:22:53.649 INFO:teuthology.orchestra.run.smithi092.stdout:mytag 2023-11-28T11:22:53.649 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2023-11-28T11:22:53.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.735+0000 7ff60123f0c0 1 Processor -- start 2023-11-28T11:22:53.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.735+0000 7ff60123f0c0 1 -- start start 2023-11-28T11:22:53.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.736+0000 7ff60123f0c0 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5642640dfa40 0x564264168f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:53.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.736+0000 7ff60123f0c0 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5642640e43e0 0x5642641633b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:53.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.736+0000 7ff60123f0c0 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56426416f4c0 0x56426416db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:53.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.736+0000 7ff60123f0c0 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56426403f000 con 0x5642640e43e0 2023-11-28T11:22:53.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.736+0000 7ff60123f0c0 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x564263fd1fe0 con 0x5642640dfa40 2023-11-28T11:22:53.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.736+0000 7ff60123f0c0 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5642640e1ea0 con 0x56426416f4c0 2023-11-28T11:22:53.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.736+0000 7ff5f239a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5642640e43e0 0x5642641633b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:53.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.737+0000 7ff5f239a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5642640e43e0 0x5642641633b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50666/0 (socket says 172.21.15.92:50666) 2023-11-28T11:22:53.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.737+0000 7ff5f239a700 1 -- 172.21.15.92:0/3852157758 learned_addr learned my addr 172.21.15.92:0/3852157758 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:53.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.737+0000 7ff5f339c700 1 --2- 172.21.15.92:0/3852157758 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56426416f4c0 0x56426416db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:53.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.737+0000 7ff5f2b9b700 1 --2- 172.21.15.92:0/3852157758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5642640dfa40 0x564264168f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:53.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.737+0000 7ff5f239a700 1 -- 172.21.15.92:0/3852157758 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56426416f4c0 msgr2=0x56426416db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:53.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.737+0000 7ff5f239a700 1 --2- 172.21.15.92:0/3852157758 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56426416f4c0 0x56426416db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:53.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.737+0000 7ff5f239a700 1 -- 172.21.15.92:0/3852157758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5642640dfa40 msgr2=0x564264168f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:53.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.737+0000 7ff5f239a700 1 --2- 172.21.15.92:0/3852157758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5642640dfa40 0x564264168f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:53.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.737+0000 7ff5f239a700 1 -- 172.21.15.92:0/3852157758 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e4005d20 con 0x5642640e43e0 2023-11-28T11:22:53.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.737+0000 7ff5f2b9b700 1 --2- 172.21.15.92:0/3852157758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5642640dfa40 0x564264168f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:22:53.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.737+0000 7ff5f239a700 1 --2- 172.21.15.92:0/3852157758 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5642640e43e0 0x5642641633b0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ff5e800b290 tx=0x7ff5e800b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beb9737607b3d495 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:53.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.737+0000 7ff5f1398700 1 -- 172.21.15.92:0/3852157758 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e8014070 con 0x5642640e43e0 2023-11-28T11:22:53.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.737+0000 7ff5f1398700 1 -- 172.21.15.92:0/3852157758 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5e800f3f0 con 0x5642640e43e0 2023-11-28T11:22:53.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.737+0000 7ff5f1398700 1 -- 172.21.15.92:0/3852157758 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e800f7a0 con 0x5642640e43e0 2023-11-28T11:22:53.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.738+0000 7ff60123f0c0 1 -- 172.21.15.92:0/3852157758 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5642640e43e0 msgr2=0x5642641633b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:53.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.738+0000 7ff60123f0c0 1 --2- 172.21.15.92:0/3852157758 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5642640e43e0 0x5642641633b0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ff5e800b290 tx=0x7ff5e800b5a0 comp rx=0 tx=0).stop 2023-11-28T11:22:53.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.738+0000 7ff60123f0c0 1 -- 172.21.15.92:0/3852157758 shutdown_connections 2023-11-28T11:22:53.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.738+0000 7ff60123f0c0 1 --2- 172.21.15.92:0/3852157758 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56426416f4c0 0x56426416db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:53.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.738+0000 7ff60123f0c0 1 --2- 172.21.15.92:0/3852157758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5642640dfa40 0x564264168f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:53.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.738+0000 7ff60123f0c0 1 --2- 172.21.15.92:0/3852157758 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5642640e43e0 0x5642641633b0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:53.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.738+0000 7ff60123f0c0 1 -- 172.21.15.92:0/3852157758 >> 172.21.15.92:0/3852157758 conn(0x56426409a5b0 msgr2=0x5642640d9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:53.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.738+0000 7ff60123f0c0 1 -- 172.21.15.92:0/3852157758 shutdown_connections 2023-11-28T11:22:53.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.738+0000 7ff60123f0c0 1 -- 172.21.15.92:0/3852157758 wait complete. 2023-11-28T11:22:53.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.739+0000 7ff60123f0c0 1 Processor -- start 2023-11-28T11:22:53.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.739+0000 7ff60123f0c0 1 -- start start 2023-11-28T11:22:53.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.740+0000 7ff60123f0c0 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5642640dfa40 0x5642641fd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:53.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.740+0000 7ff60123f0c0 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5642640e43e0 0x56426416bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:53.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.740+0000 7ff60123f0c0 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56426416f4c0 0x56426416c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:53.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.740+0000 7ff60123f0c0 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x564264166820 con 0x56426416f4c0 2023-11-28T11:22:53.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.740+0000 7ff60123f0c0 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x564264169680 con 0x5642640dfa40 2023-11-28T11:22:53.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.740+0000 7ff60123f0c0 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5642641698d0 con 0x5642640e43e0 2023-11-28T11:22:53.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.740+0000 7ff5f339c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56426416f4c0 0x56426416c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:53.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.740+0000 7ff5f339c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56426416f4c0 0x56426416c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50678/0 (socket says 172.21.15.92:50678) 2023-11-28T11:22:53.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.740+0000 7ff5f339c700 1 -- 172.21.15.92:0/603957689 learned_addr learned my addr 172.21.15.92:0/603957689 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:53.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.740+0000 7ff5f239a700 1 --2- 172.21.15.92:0/603957689 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5642640e43e0 0x56426416bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:53.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.740+0000 7ff5f2b9b700 1 --2- 172.21.15.92:0/603957689 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5642640dfa40 0x5642641fd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:53.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.741+0000 7ff5f339c700 1 -- 172.21.15.92:0/603957689 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5642640e43e0 msgr2=0x56426416bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:53.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.741+0000 7ff5f339c700 1 --2- 172.21.15.92:0/603957689 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5642640e43e0 0x56426416bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:53.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.741+0000 7ff5f339c700 1 -- 172.21.15.92:0/603957689 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5642640dfa40 msgr2=0x5642641fd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:53.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.741+0000 7ff5f339c700 1 --2- 172.21.15.92:0/603957689 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5642640dfa40 0x5642641fd3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:53.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.741+0000 7ff5f339c700 1 -- 172.21.15.92:0/603957689 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e8001bb0 con 0x56426416f4c0 2023-11-28T11:22:53.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.741+0000 7ff5f2b9b700 1 --2- 172.21.15.92:0/603957689 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5642640dfa40 0x5642641fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:22:53.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.741+0000 7ff5f339c700 1 --2- 172.21.15.92:0/603957689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56426416f4c0 0x56426416c0f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec006f70 tx=0x7ff5ec00d440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:53.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.741+0000 7ff5dbfff700 1 -- 172.21.15.92:0/603957689 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec00dd20 con 0x56426416f4c0 2023-11-28T11:22:53.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.741+0000 7ff5dbfff700 1 -- 172.21.15.92:0/603957689 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5ec00dea0 con 0x56426416f4c0 2023-11-28T11:22:53.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.741+0000 7ff5dbfff700 1 -- 172.21.15.92:0/603957689 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec003310 con 0x56426416f4c0 2023-11-28T11:22:53.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.741+0000 7ff60123f0c0 1 -- 172.21.15.92:0/603957689 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564263fdd830 con 0x56426416f4c0 2023-11-28T11:22:53.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.741+0000 7ff60123f0c0 1 -- 172.21.15.92:0/603957689 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564263fea360 con 0x56426416f4c0 2023-11-28T11:22:53.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.743+0000 7ff5dbfff700 1 -- 172.21.15.92:0/603957689 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff5ec01a030 con 0x56426416f4c0 2023-11-28T11:22:53.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.744+0000 7ff5dbfff700 1 --2- 172.21.15.92:0/603957689 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff5e0066c10 0x7ff5e00690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:53.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.744+0000 7ff5dbfff700 1 -- 172.21.15.92:0/603957689 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6440+0+0 (secure 0 0 0) 0x7ff5ec083ff0 con 0x56426416f4c0 2023-11-28T11:22:53.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.744+0000 7ff5f2b9b700 1 --2- 172.21.15.92:0/603957689 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff5e0066c10 0x7ff5e00690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:53.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.744+0000 7ff60123f0c0 1 --2- 172.21.15.92:0/603957689 >> [v2:172.21.15.169:6808/116034,v1:172.21.15.169:6809/116034] conn(0x564264164a20 0x564264164e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:53.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.744+0000 7ff60123f0c0 1 -- 172.21.15.92:0/603957689 --> [v2:172.21.15.169:6808/116034,v1:172.21.15.169:6809/116034] -- osd_op(unknown.0.0:1 7.12 7:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) v8 -- 0x5642641625a0 con 0x564264164a20 2023-11-28T11:22:53.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.745+0000 7ff5f2b9b700 1 --2- 172.21.15.92:0/603957689 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff5e0066c10 0x7ff5e00690c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4000c00 tx=0x7ff5e400e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:53.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.745+0000 7ff5f239a700 1 --2- 172.21.15.92:0/603957689 >> [v2:172.21.15.169:6808/116034,v1:172.21.15.169:6809/116034] conn(0x564264164a20 0x564264164e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:53.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.745+0000 7ff5f239a700 1 --2- 172.21.15.92:0/603957689 >> [v2:172.21.15.169:6808/116034,v1:172.21.15.169:6809/116034] conn(0x564264164a20 0x564264164e30 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:53.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.746+0000 7ff5f239a700 1 -- 172.21.15.92:0/603957689 <== osd.6 v2:172.21.15.169:6808/116034 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x7ff5e8016070 con 0x564264164a20 2023-11-28T11:22:53.748 INFO:teuthology.orchestra.run.smithi092.stderr:error getting xattr cephfs_data/10000000002.00000000/scrub_tag: (61) No data available 2023-11-28T11:22:53.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.746+0000 7ff60123f0c0 1 -- 172.21.15.92:0/603957689 >> [v2:172.21.15.169:6808/116034,v1:172.21.15.169:6809/116034] conn(0x564264164a20 msgr2=0x564264164e30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:53.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.746+0000 7ff60123f0c0 1 --2- 172.21.15.92:0/603957689 >> [v2:172.21.15.169:6808/116034,v1:172.21.15.169:6809/116034] conn(0x564264164a20 0x564264164e30 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:53.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.746+0000 7ff60123f0c0 1 -- 172.21.15.92:0/603957689 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff5e0066c10 msgr2=0x7ff5e00690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:53.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.746+0000 7ff60123f0c0 1 --2- 172.21.15.92:0/603957689 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff5e0066c10 0x7ff5e00690c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4000c00 tx=0x7ff5e400e000 comp rx=0 tx=0).stop 2023-11-28T11:22:53.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.746+0000 7ff60123f0c0 1 -- 172.21.15.92:0/603957689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56426416f4c0 msgr2=0x56426416c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:53.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.746+0000 7ff60123f0c0 1 --2- 172.21.15.92:0/603957689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56426416f4c0 0x56426416c0f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec006f70 tx=0x7ff5ec00d440 comp rx=0 tx=0).stop 2023-11-28T11:22:53.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.746+0000 7ff5f339c700 1 -- 172.21.15.92:0/603957689 reap_dead start 2023-11-28T11:22:53.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.746+0000 7ff60123f0c0 1 -- 172.21.15.92:0/603957689 shutdown_connections 2023-11-28T11:22:53.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.746+0000 7ff60123f0c0 1 -- 172.21.15.92:0/603957689 >> 172.21.15.92:0/603957689 conn(0x56426409a5b0 msgr2=0x5642640e1300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:53.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.746+0000 7ff60123f0c0 1 -- 172.21.15.92:0/603957689 shutdown_connections 2023-11-28T11:22:53.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:53.746+0000 7ff60123f0c0 1 -- 172.21.15.92:0/603957689 wait complete. 2023-11-28T11:22:53.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:22:53.751 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T11:22:53.751 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:22:53.835 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:22:53.935 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T11:22:53.935 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-28T11:22:53.935 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:22:53.983 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348288k,nr_inodes=4087072,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275616k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:22:53.983 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-28T11:22:53.983 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:22:54.031 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348296k,nr_inodes=4087074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275620k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:22:54.032 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-28T11:22:54.032 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:22:54.079 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348288k,nr_inodes=4087072,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275616k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:22:54.079 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-28T11:22:54.079 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:22:54.126 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348296k,nr_inodes=4087074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275620k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:22:54.126 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:22:54.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.321+0000 7f9d533ff700 1 Processor -- start 2023-11-28T11:22:54.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.321+0000 7f9d533ff700 1 -- start start 2023-11-28T11:22:54.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.322+0000 7f9d533ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d4c0d1d70 0x7f9d4c167270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:54.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.322+0000 7f9d533ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d4c06c790 con 0x7f9d4c0b8510 2023-11-28T11:22:54.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.323+0000 7f9d533ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d4c0b88e0 con 0x7f9d4c0cd460 2023-11-28T11:22:54.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.323+0000 7f9d533ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4c0cdbd0 con 0x7f9d4c0d1d70 2023-11-28T11:22:54.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.323+0000 7f9d5119b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f9d4c0b8510 0x7f9d4c06b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51066/0 (socket says 172.21.15.92:51066) 2023-11-28T11:22:54.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.323+0000 7f9d5119b700 1 -- 172.21.15.92:0/2714113160 learned_addr learned my addr 172.21.15.92:0/2714113160 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:22:54.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.323+0000 7f9d5199c700 1 --2- 172.21.15.92:0/2714113160 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d4c0d1d70 0x7f9d4c167270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:54.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.323+0000 7f9d43fff700 1 -- 172.21.15.92:0/2714113160 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4232925143 0 0) 0x7f9d4c06c790 con 0x7f9d4c0b8510 2023-11-28T11:22:54.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.323+0000 7f9d43fff700 1 -- 172.21.15.92:0/2714113160 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d380035b0 con 0x7f9d4c0b8510 2023-11-28T11:22:54.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.323+0000 7f9d43fff700 1 -- 172.21.15.92:0/2714113160 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3869296629 0 0) 0x7f9d4c0b88e0 con 0x7f9d4c0cd460 2023-11-28T11:22:54.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.323+0000 7f9d43fff700 1 -- 172.21.15.92:0/2714113160 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d4c06c790 con 0x7f9d4c0cd460 2023-11-28T11:22:54.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.323+0000 7f9d43fff700 1 -- 172.21.15.92:0/2714113160 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2981928055 0 0) 0x7f9d380035b0 con 0x7f9d4c0b8510 2023-11-28T11:22:54.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.323+0000 7f9d43fff700 1 -- 172.21.15.92:0/2714113160 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d4c0d1d70 msgr2=0x7f9d4c167270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:54.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.323+0000 7f9d43fff700 1 --2- 172.21.15.92:0/2714113160 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d4c0d1d70 0x7f9d4c167270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:54.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.323+0000 7f9d43fff700 1 -- 172.21.15.92:0/2714113160 >> v1:172.21.15.169:6789/0 conn(0x7f9d4c0cd460 legacy=0x7f9d4c06c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:54.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.323+0000 7f9d43fff700 1 -- 172.21.15.92:0/2714113160 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d4c0cd8f0 con 0x7f9d4c0b8510 2023-11-28T11:22:54.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.323+0000 7f9d43fff700 1 -- 172.21.15.92:0/2714113160 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532161587 0 0) 0x7f9d48001670 con 0x7f9d4c0b8510 2023-11-28T11:22:54.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.323+0000 7f9d5199c700 1 --2- 172.21.15.92:0/2714113160 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d4c0d1d70 0x7f9d4c167270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:22:54.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.324+0000 7f9d43fff700 1 -- 172.21.15.92:0/2714113160 >> v1:172.21.15.92:6789/0 conn(0x7f9d4c0b8510 legacy=0x7f9d4c06b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:54.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.324+0000 7f9d43fff700 1 --2- 172.21.15.92:0/2714113160 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d4c0d1d70 0x7f9d38003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:54.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.324+0000 7f9d43fff700 1 --2- 172.21.15.92:0/2714113160 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d38004150 0x7f9d38004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:54.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.324+0000 7f9d43fff700 1 --2- 172.21.15.92:0/2714113160 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d38004d90 0x7f9d38005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:54.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.324+0000 7f9d43fff700 1 -- 172.21.15.92:0/2714113160 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4c0cdbd0 con 0x7f9d38004d90 2023-11-28T11:22:54.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.324+0000 7f9d43fff700 1 -- 172.21.15.92:0/2714113160 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3800baf0 con 0x7f9d38004150 2023-11-28T11:22:54.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.324+0000 7f9d43fff700 1 -- 172.21.15.92:0/2714113160 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3800bcb0 con 0x7f9d4c0d1d70 2023-11-28T11:22:54.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.324+0000 7f9d5199c700 1 --2- 172.21.15.92:0/2714113160 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d38004d90 0x7f9d38005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:54.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.324+0000 7f9d5119b700 1 --2- 172.21.15.92:0/2714113160 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d4c0d1d70 0x7f9d38003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:54.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.324+0000 7f9d5099a700 1 --2- 172.21.15.92:0/2714113160 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d38004150 0x7f9d38004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:54.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.324+0000 7f9d5199c700 1 -- 172.21.15.92:0/2714113160 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d4c0d1d70 msgr2=0x7f9d38003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:54.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.324+0000 7f9d5199c700 1 --2- 172.21.15.92:0/2714113160 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d4c0d1d70 0x7f9d38003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:54.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.324+0000 7f9d5199c700 1 -- 172.21.15.92:0/2714113160 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d38004150 msgr2=0x7f9d38004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:54.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.324+0000 7f9d5199c700 1 --2- 172.21.15.92:0/2714113160 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d38004150 0x7f9d38004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:54.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.324+0000 7f9d5199c700 1 -- 172.21.15.92:0/2714113160 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d44009a70 con 0x7f9d38004d90 2023-11-28T11:22:54.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.325+0000 7f9d5199c700 1 --2- 172.21.15.92:0/2714113160 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d38004d90 0x7f9d38005200 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9d4400e350 tx=0x7f9d38005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80bf280488be2e68 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:54.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.325+0000 7f9d43fff700 1 -- 172.21.15.92:0/2714113160 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4400ee10 con 0x7f9d38004d90 2023-11-28T11:22:54.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.325+0000 7f9d43fff700 1 -- 172.21.15.92:0/2714113160 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d4400f040 con 0x7f9d38004d90 2023-11-28T11:22:54.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.325+0000 7f9d43fff700 1 -- 172.21.15.92:0/2714113160 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d44003c60 con 0x7f9d38004d90 2023-11-28T11:22:54.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.325+0000 7f9d533ff700 1 -- 172.21.15.92:0/2714113160 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d38004d90 msgr2=0x7f9d38005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:54.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.325+0000 7f9d533ff700 1 --2- 172.21.15.92:0/2714113160 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d38004d90 0x7f9d38005200 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9d4400e350 tx=0x7f9d38005880 comp rx=0 tx=0).stop 2023-11-28T11:22:54.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.325+0000 7f9d5199c700 1 -- 172.21.15.92:0/2714113160 reap_dead start 2023-11-28T11:22:54.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.325+0000 7f9d533ff700 1 -- 172.21.15.92:0/2714113160 shutdown_connections 2023-11-28T11:22:54.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.325+0000 7f9d533ff700 1 -- 172.21.15.92:0/2714113160 >> 172.21.15.92:0/2714113160 conn(0x7f9d4c0719a0 msgr2=0x7f9d4c071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:54.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.325+0000 7f9d533ff700 1 -- 172.21.15.92:0/2714113160 shutdown_connections 2023-11-28T11:22:54.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.325+0000 7f9d533ff700 1 -- 172.21.15.92:0/2714113160 wait complete. 2023-11-28T11:22:54.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.327+0000 7f9d533ff700 1 Processor -- start 2023-11-28T11:22:54.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.327+0000 7f9d533ff700 1 -- start start 2023-11-28T11:22:54.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.328+0000 7f9d533ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d38004d90 0x7f9d4c0c84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:54.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.328+0000 7f9d533ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d4c154840 0x7f9d4c153ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:54.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.328+0000 7f9d533ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d4c1553a0 0x7f9d4c155810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:54.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.328+0000 7f9d533ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4c155d50 con 0x7f9d4c154840 2023-11-28T11:22:54.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.328+0000 7f9d533ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4c0ca930 con 0x7f9d4c1553a0 2023-11-28T11:22:54.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.328+0000 7f9d533ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4c0c8af0 con 0x7f9d38004d90 2023-11-28T11:22:54.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.329+0000 7f9d5099a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d4c154840 0x7f9d4c153ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:54.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.329+0000 7f9d5099a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d4c154840 0x7f9d4c153ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50688/0 (socket says 172.21.15.92:50688) 2023-11-28T11:22:54.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.329+0000 7f9d5099a700 1 -- 172.21.15.92:0/958513254 learned_addr learned my addr 172.21.15.92:0/958513254 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:54.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.329+0000 7f9d5119b700 1 --2- 172.21.15.92:0/958513254 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d38004d90 0x7f9d4c0c84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:54.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.329+0000 7f9d5199c700 1 --2- 172.21.15.92:0/958513254 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d4c1553a0 0x7f9d4c155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:54.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.329+0000 7f9d5099a700 1 -- 172.21.15.92:0/958513254 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d38004d90 msgr2=0x7f9d4c0c84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:54.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.329+0000 7f9d5099a700 1 --2- 172.21.15.92:0/958513254 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d38004d90 0x7f9d4c0c84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:54.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.329+0000 7f9d5099a700 1 -- 172.21.15.92:0/958513254 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d4c1553a0 msgr2=0x7f9d4c155810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:54.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.329+0000 7f9d5099a700 1 --2- 172.21.15.92:0/958513254 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d4c1553a0 0x7f9d4c155810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:54.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.329+0000 7f9d5099a700 1 -- 172.21.15.92:0/958513254 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d4c0cfff0 con 0x7f9d4c154840 2023-11-28T11:22:54.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.329+0000 7f9d5099a700 1 --2- 172.21.15.92:0/958513254 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d4c154840 0x7f9d4c153ff0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c00f730 tx=0x7f9d3c00fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:54.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.329+0000 7f9d41ffb700 1 -- 172.21.15.92:0/958513254 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c0053e0 con 0x7f9d4c154840 2023-11-28T11:22:54.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.330+0000 7f9d41ffb700 1 -- 172.21.15.92:0/958513254 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d3c005540 con 0x7f9d4c154840 2023-11-28T11:22:54.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.330+0000 7f9d533ff700 1 -- 172.21.15.92:0/958513254 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d4c0cfd10 con 0x7f9d4c154840 2023-11-28T11:22:54.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.330+0000 7f9d41ffb700 1 -- 172.21.15.92:0/958513254 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c006aa0 con 0x7f9d4c154840 2023-11-28T11:22:54.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.330+0000 7f9d533ff700 1 -- 172.21.15.92:0/958513254 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d4c0d29c0 con 0x7f9d4c154840 2023-11-28T11:22:54.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.331+0000 7f9d533ff700 1 -- 172.21.15.92:0/958513254 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d4c0caaa0 con 0x7f9d4c154840 2023-11-28T11:22:54.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.332+0000 7f9d41ffb700 1 -- 172.21.15.92:0/958513254 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9d3c006c00 con 0x7f9d4c154840 2023-11-28T11:22:54.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.332+0000 7f9d41ffb700 1 --2- 172.21.15.92:0/958513254 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9d300682f0 0x7f9d3006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:54.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.332+0000 7f9d41ffb700 1 -- 172.21.15.92:0/958513254 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f9d3c085430 con 0x7f9d4c154840 2023-11-28T11:22:54.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.333+0000 7f9d5119b700 1 --2- 172.21.15.92:0/958513254 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9d300682f0 0x7f9d3006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:54.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.333+0000 7f9d5119b700 1 --2- 172.21.15.92:0/958513254 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9d300682f0 0x7f9d3006a7a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9d48001050 tx=0x7f9d48000f80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:54.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.337+0000 7f9d41ffb700 1 -- 172.21.15.92:0/958513254 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9d3c051a20 con 0x7f9d4c154840 2023-11-28T11:22:54.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.567+0000 7f9d533ff700 1 -- 172.21.15.92:0/958513254 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9d4c067880 con 0x7f9d4c154840 2023-11-28T11:22:54.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.568+0000 7f9d41ffb700 1 -- 172.21.15.92:0/958513254 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4645 (secure 0 0 0) 0x7f9d3c055040 con 0x7f9d4c154840 2023-11-28T11:22:54.568 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:22:54.569 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4976,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/3102603064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3102603064},{"type":"v1","addr":"172.21.15.169:6834","nonce":3102603064}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4979,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/79247121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":79247121},{"type":"v1","addr":"172.21.15.92:6837","nonce":79247121}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5221,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/219797192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":219797192},{"type":"v1","addr":"172.21.15.169:6835","nonce":219797192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:22:24.715447+0000","modified":"2023-11-28T11:22:27.733117+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5298},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5298":{"gid":5298,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.92:6835/1577294649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1577294649},{"type":"v1","addr":"172.21.15.92:6835","nonce":1577294649}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T11:22:54.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.570+0000 7f9d533ff700 1 -- 172.21.15.92:0/958513254 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9d300682f0 msgr2=0x7f9d3006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:54.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.570+0000 7f9d533ff700 1 --2- 172.21.15.92:0/958513254 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9d300682f0 0x7f9d3006a7a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9d48001050 tx=0x7f9d48000f80 comp rx=0 tx=0).stop 2023-11-28T11:22:54.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.570+0000 7f9d533ff700 1 -- 172.21.15.92:0/958513254 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d4c154840 msgr2=0x7f9d4c153ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:54.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.570+0000 7f9d533ff700 1 --2- 172.21.15.92:0/958513254 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d4c154840 0x7f9d4c153ff0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c00f730 tx=0x7f9d3c00fa40 comp rx=0 tx=0).stop 2023-11-28T11:22:54.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.570+0000 7f9d533ff700 1 -- 172.21.15.92:0/958513254 shutdown_connections 2023-11-28T11:22:54.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.570+0000 7f9d533ff700 1 --2- 172.21.15.92:0/958513254 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d4c1553a0 0x7f9d4c155810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:54.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.570+0000 7f9d533ff700 1 --2- 172.21.15.92:0/958513254 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d38004d90 0x7f9d4c0c84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:54.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.570+0000 7f9d533ff700 1 --2- 172.21.15.92:0/958513254 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9d300682f0 0x7f9d3006a7a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:54.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.570+0000 7f9d533ff700 1 --2- 172.21.15.92:0/958513254 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d4c154840 0x7f9d4c153ff0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:54.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.570+0000 7f9d533ff700 1 -- 172.21.15.92:0/958513254 >> 172.21.15.92:0/958513254 conn(0x7f9d4c0719a0 msgr2=0x7f9d4c0b1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:54.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.570+0000 7f9d533ff700 1 -- 172.21.15.92:0/958513254 shutdown_connections 2023-11-28T11:22:54.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.570+0000 7f9d533ff700 1 -- 172.21.15.92:0/958513254 wait complete. 2023-11-28T11:22:54.572 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 17 2023-11-28T11:22:54.590 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:22:54.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.785+0000 7f2f9d3ff700 1 Processor -- start 2023-11-28T11:22:54.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.786+0000 7f2f9d3ff700 1 -- start start 2023-11-28T11:22:54.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.787+0000 7f2f9d3ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2f980ca600 0x7f2f98164220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:54.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.787+0000 7f2f9d3ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f98159b10 con 0x7f2f9806a980 2023-11-28T11:22:54.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.787+0000 7f2f9d3ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f98143250 con 0x7f2f98142e80 2023-11-28T11:22:54.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.787+0000 7f2f9d3ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f980c9380 con 0x7f2f980ca600 2023-11-28T11:22:54.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.787+0000 7f2f967fc700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f2f9806a980 0x7f2f980c9ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51072/0 (socket says 172.21.15.92:51072) 2023-11-28T11:22:54.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.787+0000 7f2f967fc700 1 -- 172.21.15.92:0/565024719 learned_addr learned my addr 172.21.15.92:0/565024719 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:22:54.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.787+0000 7f2f977fe700 1 --2- 172.21.15.92:0/565024719 >> v2:172.21.15.92:3300/0 conn(0x7f2f980ca600 0x7f2f98164220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:54.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.788+0000 7f2f977fe700 1 -- 172.21.15.92:0/565024719 >> v1:172.21.15.169:6789/0 conn(0x7f2f98142e80 legacy=0x7f2f9806a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:54.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.788+0000 7f2f977fe700 1 -- 172.21.15.92:0/565024719 >> v1:172.21.15.92:6789/0 conn(0x7f2f9806a980 legacy=0x7f2f980c9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:54.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.788+0000 7f2f977fe700 1 -- 172.21.15.92:0/565024719 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f980cab00 con 0x7f2f980ca600 2023-11-28T11:22:54.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.788+0000 7f2f95ffb700 1 -- 172.21.15.92:0/565024719 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1922462966 0 0) 0x7f2f98159b10 con 0x7f2f9806a980 2023-11-28T11:22:54.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.788+0000 7f2f977fe700 1 --2- 172.21.15.92:0/565024719 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f980ca600 0x7f2f98164220 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f2f90009d90 tx=0x7f2f9000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=667ff5339b77ecc5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:54.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.788+0000 7f2f95ffb700 1 -- 172.21.15.92:0/565024719 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9000e070 con 0x7f2f980ca600 2023-11-28T11:22:54.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.788+0000 7f2f95ffb700 1 -- 172.21.15.92:0/565024719 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f9000bbd0 con 0x7f2f980ca600 2023-11-28T11:22:54.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.788+0000 7f2f95ffb700 1 -- 172.21.15.92:0/565024719 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f90012430 con 0x7f2f980ca600 2023-11-28T11:22:54.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.789+0000 7f2f9d3ff700 1 -- 172.21.15.92:0/565024719 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f980ca600 msgr2=0x7f2f98164220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:54.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.789+0000 7f2f9d3ff700 1 --2- 172.21.15.92:0/565024719 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f980ca600 0x7f2f98164220 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f2f90009d90 tx=0x7f2f9000b330 comp rx=0 tx=0).stop 2023-11-28T11:22:54.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.789+0000 7f2f9d3ff700 1 -- 172.21.15.92:0/565024719 shutdown_connections 2023-11-28T11:22:54.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.789+0000 7f2f9d3ff700 1 --2- 172.21.15.92:0/565024719 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f980ca600 0x7f2f98164220 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:54.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.789+0000 7f2f9d3ff700 1 -- 172.21.15.92:0/565024719 >> 172.21.15.92:0/565024719 conn(0x7f2f980cb490 msgr2=0x7f2f980cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:54.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.789+0000 7f2f9d3ff700 1 -- 172.21.15.92:0/565024719 shutdown_connections 2023-11-28T11:22:54.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.789+0000 7f2f9d3ff700 1 -- 172.21.15.92:0/565024719 wait complete. 2023-11-28T11:22:54.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.791+0000 7f2f9d3ff700 1 Processor -- start 2023-11-28T11:22:54.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.791+0000 7f2f9d3ff700 1 -- start start 2023-11-28T11:22:54.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.792+0000 7f2f9d3ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f9806a980 0x7f2f981f1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:54.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.792+0000 7f2f9d3ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f980ca600 0x7f2f98157f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:54.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.792+0000 7f2f9d3ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f98142e80 0x7f2f981584d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:54.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.792+0000 7f2f9d3ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f981567b0 con 0x7f2f98142e80 2023-11-28T11:22:54.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.792+0000 7f2f9d3ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f980b1840 con 0x7f2f9806a980 2023-11-28T11:22:54.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.792+0000 7f2f9d3ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f980ac250 con 0x7f2f980ca600 2023-11-28T11:22:54.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.792+0000 7f2f977fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f98142e80 0x7f2f981584d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:54.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.792+0000 7f2f977fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f98142e80 0x7f2f981584d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50696/0 (socket says 172.21.15.92:50696) 2023-11-28T11:22:54.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.792+0000 7f2f977fe700 1 -- 172.21.15.92:0/723578445 learned_addr learned my addr 172.21.15.92:0/723578445 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:54.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.792+0000 7f2f967fc700 1 --2- 172.21.15.92:0/723578445 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f980ca600 0x7f2f98157f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:54.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.792+0000 7f2f96ffd700 1 --2- 172.21.15.92:0/723578445 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f9806a980 0x7f2f981f1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:54.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.792+0000 7f2f977fe700 1 -- 172.21.15.92:0/723578445 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f980ca600 msgr2=0x7f2f98157f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:54.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.792+0000 7f2f977fe700 1 --2- 172.21.15.92:0/723578445 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f980ca600 0x7f2f98157f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:54.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.792+0000 7f2f977fe700 1 -- 172.21.15.92:0/723578445 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f9806a980 msgr2=0x7f2f981f1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:54.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.792+0000 7f2f977fe700 1 --2- 172.21.15.92:0/723578445 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f9806a980 0x7f2f981f1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:54.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.792+0000 7f2f977fe700 1 -- 172.21.15.92:0/723578445 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f90009a70 con 0x7f2f98142e80 2023-11-28T11:22:54.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.792+0000 7f2f96ffd700 1 --2- 172.21.15.92:0/723578445 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f9806a980 0x7f2f981f1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:22:54.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.793+0000 7f2f977fe700 1 --2- 172.21.15.92:0/723578445 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f98142e80 0x7f2f981584d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f2f90014040 tx=0x7f2f90004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:54.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.793+0000 7f2f7f7fe700 1 -- 172.21.15.92:0/723578445 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9001b810 con 0x7f2f98142e80 2023-11-28T11:22:54.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.793+0000 7f2f7f7fe700 1 -- 172.21.15.92:0/723578445 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f90020070 con 0x7f2f98142e80 2023-11-28T11:22:54.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.793+0000 7f2f9d3ff700 1 -- 172.21.15.92:0/723578445 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f980cc8d0 con 0x7f2f98142e80 2023-11-28T11:22:54.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.793+0000 7f2f7f7fe700 1 -- 172.21.15.92:0/723578445 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9001bbb0 con 0x7f2f98142e80 2023-11-28T11:22:54.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.793+0000 7f2f9d3ff700 1 -- 172.21.15.92:0/723578445 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f980cc5f0 con 0x7f2f98142e80 2023-11-28T11:22:54.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.794+0000 7f2f9d3ff700 1 -- 172.21.15.92:0/723578445 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f980ccbb0 con 0x7f2f98142e80 2023-11-28T11:22:54.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.795+0000 7f2f7f7fe700 1 -- 172.21.15.92:0/723578445 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2f9002e070 con 0x7f2f98142e80 2023-11-28T11:22:54.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.796+0000 7f2f7f7fe700 1 --2- 172.21.15.92:0/723578445 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f2f840682f0 0x7f2f8406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:54.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.796+0000 7f2f7f7fe700 1 -- 172.21.15.92:0/723578445 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f2f90022070 con 0x7f2f98142e80 2023-11-28T11:22:54.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.796+0000 7f2f96ffd700 1 --2- 172.21.15.92:0/723578445 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f2f840682f0 0x7f2f8406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:54.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.797+0000 7f2f96ffd700 1 --2- 172.21.15.92:0/723578445 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f2f840682f0 0x7f2f8406a7a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2f88000f60 tx=0x7f2f88009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:54.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.800+0000 7f2f7f7fe700 1 -- 172.21.15.92:0/723578445 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2f900545d0 con 0x7f2f98142e80 2023-11-28T11:22:54.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.992+0000 7f2f9d3ff700 1 -- 172.21.15.92:0/723578445 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2f9806b440 con 0x7f2f98142e80 2023-11-28T11:22:54.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.994+0000 7f2f7f7fe700 1 -- 172.21.15.92:0/723578445 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+18279 (secure 0 0 0) 0x7f2f9002c020 con 0x7f2f98142e80 2023-11-28T11:22:54.995 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:22:54.995 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":44,"fsid":"68956e0c-c16f-47cd-a220-f98364b0831d","created":"2023-11-28T11:20:28.697573+0000","modified":"2023-11-28T11:22:38.531199+0000","last_up_change":"2023-11-28T11:20:41.257011+0000","last_in_change":"2023-11-28T11:20:32.249767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:20:41.925237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:20:48.682497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:22:22.942125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-28T11:22:23.684528+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6abe483f-0e03-4a26-b9cf-62041291747e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":115823},{"type":"v1","addr":"172.21.15.92:6809","nonce":115823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":115823},{"type":"v1","addr":"172.21.15.92:6813","nonce":115823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":115823},{"type":"v1","addr":"172.21.15.92:6821","nonce":115823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":115823},{"type":"v1","addr":"172.21.15.92:6817","nonce":115823}]},"public_addr":"172.21.15.92:6809/115823","cluster_addr":"172.21.15.92:6813/115823","heartbeat_back_addr":"172.21.15.92:6821/115823","heartbeat_front_addr":"172.21.15.92:6817/115823","state":["exists","up"]},{"osd":1,"uuid":"13c1de73-1a8c-4d7e-8a28-9dae52dfe089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":115824},{"type":"v1","addr":"172.21.15.92:6812","nonce":115824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":115824},{"type":"v1","addr":"172.21.15.92:6816","nonce":115824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":115824},{"type":"v1","addr":"172.21.15.92:6823","nonce":115824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":115824},{"type":"v1","addr":"172.21.15.92:6820","nonce":115824}]},"public_addr":"172.21.15.92:6812/115824","cluster_addr":"172.21.15.92:6816/115824","heartbeat_back_addr":"172.21.15.92:6823/115824","heartbeat_front_addr":"172.21.15.92:6820/115824","state":["exists","up"]},{"osd":2,"uuid":"25788a55-112f-4f00-810d-2bc37cf76778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":115821},{"type":"v1","addr":"172.21.15.92:6801","nonce":115821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":115821},{"type":"v1","addr":"172.21.15.92:6803","nonce":115821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":115821},{"type":"v1","addr":"172.21.15.92:6807","nonce":115821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":115821},{"type":"v1","addr":"172.21.15.92:6805","nonce":115821}]},"public_addr":"172.21.15.92:6801/115821","cluster_addr":"172.21.15.92:6803/115821","heartbeat_back_addr":"172.21.15.92:6807/115821","heartbeat_front_addr":"172.21.15.92:6805/115821","state":["exists","up"]},{"osd":3,"uuid":"adf54e24-33a9-41fe-9ce3-98a5ab6c158e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":115825},{"type":"v1","addr":"172.21.15.92:6825","nonce":115825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":115825},{"type":"v1","addr":"172.21.15.92:6827","nonce":115825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":115825},{"type":"v1","addr":"172.21.15.92:6831","nonce":115825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":115825},{"type":"v1","addr":"172.21.15.92:6829","nonce":115825}]},"public_addr":"172.21.15.92:6825/115825","cluster_addr":"172.21.15.92:6827/115825","heartbeat_back_addr":"172.21.15.92:6831/115825","heartbeat_front_addr":"172.21.15.92:6829/115825","state":["exists","up"]},{"osd":4,"uuid":"ccb9b747-f399-44ea-9a18-9db3152227e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":116035},{"type":"v1","addr":"172.21.15.169:6817","nonce":116035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":116035},{"type":"v1","addr":"172.21.15.169:6819","nonce":116035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":116035},{"type":"v1","addr":"172.21.15.169:6826","nonce":116035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":116035},{"type":"v1","addr":"172.21.15.169:6822","nonce":116035}]},"public_addr":"172.21.15.169:6817/116035","cluster_addr":"172.21.15.169:6819/116035","heartbeat_back_addr":"172.21.15.169:6826/116035","heartbeat_front_addr":"172.21.15.169:6822/116035","state":["exists","up"]},{"osd":5,"uuid":"308a1e63-75aa-4d7d-8a7c-cecc24714fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":116033},{"type":"v1","addr":"172.21.15.169:6801","nonce":116033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":116033},{"type":"v1","addr":"172.21.15.169:6803","nonce":116033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":116033},{"type":"v1","addr":"172.21.15.169:6807","nonce":116033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":116033},{"type":"v1","addr":"172.21.15.169:6805","nonce":116033}]},"public_addr":"172.21.15.169:6801/116033","cluster_addr":"172.21.15.169:6803/116033","heartbeat_back_addr":"172.21.15.169:6807/116033","heartbeat_front_addr":"172.21.15.169:6805/116033","state":["exists","up"]},{"osd":6,"uuid":"9c459b7b-b821-4b4b-b91e-6aaae271457c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":116034},{"type":"v1","addr":"172.21.15.169:6809","nonce":116034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":116034},{"type":"v1","addr":"172.21.15.169:6811","nonce":116034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":116034},{"type":"v1","addr":"172.21.15.169:6815","nonce":116034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":116034},{"type":"v1","addr":"172.21.15.169:6813","nonce":116034}]},"public_addr":"172.21.15.169:6809/116034","cluster_addr":"172.21.15.169:6811/116034","heartbeat_back_addr":"172.21.15.169:6815/116034","heartbeat_front_addr":"172.21.15.169:6813/116034","state":["exists","up"]},{"osd":7,"uuid":"1ca9aa4a-755d-4e28-a937-4d8dccab4df3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":116044},{"type":"v1","addr":"172.21.15.169:6823","nonce":116044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6825","nonce":116044},{"type":"v1","addr":"172.21.15.169:6827","nonce":116044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":116044},{"type":"v1","addr":"172.21.15.169:6831","nonce":116044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":116044},{"type":"v1","addr":"172.21.15.169:6829","nonce":116044}]},"public_addr":"172.21.15.169:6823/116044","cluster_addr":"172.21.15.169:6827/116044","heartbeat_back_addr":"172.21.15.169:6831/116044","heartbeat_front_addr":"172.21.15.169:6829/116044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.586781+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.869778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.508594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.860493+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.872267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.746933+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.611520+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.871446+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.f","mappings":[{"from":0,"to":4}]},{"pgid":"7.3e","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:22:54.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.996+0000 7f2f9d3ff700 1 -- 172.21.15.92:0/723578445 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f2f840682f0 msgr2=0x7f2f8406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:54.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.996+0000 7f2f9d3ff700 1 --2- 172.21.15.92:0/723578445 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f2f840682f0 0x7f2f8406a7a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2f88000f60 tx=0x7f2f88009040 comp rx=0 tx=0).stop 2023-11-28T11:22:54.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.996+0000 7f2f9d3ff700 1 -- 172.21.15.92:0/723578445 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f98142e80 msgr2=0x7f2f981584d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:54.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.996+0000 7f2f9d3ff700 1 --2- 172.21.15.92:0/723578445 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f98142e80 0x7f2f981584d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f2f90014040 tx=0x7f2f90004ad0 comp rx=0 tx=0).stop 2023-11-28T11:22:54.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.996+0000 7f2f9d3ff700 1 -- 172.21.15.92:0/723578445 shutdown_connections 2023-11-28T11:22:54.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.996+0000 7f2f9d3ff700 1 --2- 172.21.15.92:0/723578445 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f2f840682f0 0x7f2f8406a7a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:54.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.996+0000 7f2f9d3ff700 1 --2- 172.21.15.92:0/723578445 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f98142e80 0x7f2f981584d0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:54.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.996+0000 7f2f9d3ff700 1 --2- 172.21.15.92:0/723578445 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2f9806a980 0x7f2f981f1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:54.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.996+0000 7f2f9d3ff700 1 --2- 172.21.15.92:0/723578445 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2f980ca600 0x7f2f98157f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:54.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.996+0000 7f2f9d3ff700 1 -- 172.21.15.92:0/723578445 >> 172.21.15.92:0/723578445 conn(0x7f2f980cb490 msgr2=0x7f2f980711d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:54.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.996+0000 7f2f9d3ff700 1 -- 172.21.15.92:0/723578445 shutdown_connections 2023-11-28T11:22:54.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:54.996+0000 7f2f9d3ff700 1 -- 172.21.15.92:0/723578445 wait complete. 2023-11-28T11:22:55.015 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:20:41.925237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:20:48.682497+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:22:22.942125+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:22:23.684528+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:22:55.016 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:22:55.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.213+0000 7f79039ff700 1 Processor -- start 2023-11-28T11:22:55.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.213+0000 7f79039ff700 1 -- start start 2023-11-28T11:22:55.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.214+0000 7f79039ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f78fc0cd460 0x7f78fc06c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:55.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.214+0000 7f79039ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78fc0d0d60 0x7f78fc0d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:55.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.214+0000 7f79039ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78fc0b88e0 con 0x7f78fc0b8510 2023-11-28T11:22:55.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.214+0000 7f79039ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78fc0cdbd0 con 0x7f78fc0cd460 2023-11-28T11:22:55.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.214+0000 7f79039ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc0d02d0 con 0x7f78fc0d0d60 2023-11-28T11:22:55.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.214+0000 7f790179b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f78fc0b8510 0x7f78fc06b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51086/0 (socket says 172.21.15.92:51086) 2023-11-28T11:22:55.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.214+0000 7f790179b700 1 -- 172.21.15.92:0/3841341488 learned_addr learned my addr 172.21.15.92:0/3841341488 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:22:55.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.215+0000 7f7900f9a700 1 --2- 172.21.15.92:0/3841341488 >> v2:172.21.15.169:3300/0 conn(0x7f78fc0cd460 0x7f78fc06c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:55.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.215+0000 7f7901f9c700 1 --2- 172.21.15.92:0/3841341488 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78fc0d0d60 0x7f78fc0d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:55.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.215+0000 7f78f3fff700 1 -- 172.21.15.92:0/3841341488 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2843090236 0 0) 0x7f78fc0b88e0 con 0x7f78fc0b8510 2023-11-28T11:22:55.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.215+0000 7f78f3fff700 1 -- 172.21.15.92:0/3841341488 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78e80035b0 con 0x7f78fc0b8510 2023-11-28T11:22:55.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.215+0000 7f7900f9a700 1 -- 172.21.15.92:0/3841341488 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78fc0d0d60 msgr2=0x7f78fc0d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:55.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.215+0000 7f7900f9a700 1 --2- 172.21.15.92:0/3841341488 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78fc0d0d60 0x7f78fc0d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:55.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.215+0000 7f7900f9a700 1 -- 172.21.15.92:0/3841341488 >> v1:172.21.15.92:6789/0 conn(0x7f78fc0b8510 legacy=0x7f78fc06b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:55.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.215+0000 7f7900f9a700 1 -- 172.21.15.92:0/3841341488 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78fc0cd8f0 con 0x7f78fc0cd460 2023-11-28T11:22:55.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.215+0000 7f7901f9c700 1 --2- 172.21.15.92:0/3841341488 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78fc0d0d60 0x7f78fc0d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:22:55.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.216+0000 7f7900f9a700 1 --2- 172.21.15.92:0/3841341488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78fc0cd460 0x7f78fc06c080 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f78ec009870 tx=0x7f78ec009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9deb187df37ffbf3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:55.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.216+0000 7f78f3fff700 1 -- 172.21.15.92:0/3841341488 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ec011830 con 0x7f78fc0cd460 2023-11-28T11:22:55.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.216+0000 7f78f3fff700 1 -- 172.21.15.92:0/3841341488 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78ec011990 con 0x7f78fc0cd460 2023-11-28T11:22:55.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.216+0000 7f78f3fff700 1 -- 172.21.15.92:0/3841341488 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ec011d00 con 0x7f78fc0cd460 2023-11-28T11:22:55.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.216+0000 7f79039ff700 1 -- 172.21.15.92:0/3841341488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78fc0cd460 msgr2=0x7f78fc06c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:55.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.216+0000 7f79039ff700 1 --2- 172.21.15.92:0/3841341488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78fc0cd460 0x7f78fc06c080 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f78ec009870 tx=0x7f78ec009b80 comp rx=0 tx=0).stop 2023-11-28T11:22:55.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.216+0000 7f79039ff700 1 -- 172.21.15.92:0/3841341488 shutdown_connections 2023-11-28T11:22:55.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.216+0000 7f79039ff700 1 --2- 172.21.15.92:0/3841341488 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78fc0d0d60 0x7f78fc0d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:55.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.216+0000 7f79039ff700 1 --2- 172.21.15.92:0/3841341488 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78fc0cd460 0x7f78fc06c080 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:55.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.216+0000 7f79039ff700 1 -- 172.21.15.92:0/3841341488 >> 172.21.15.92:0/3841341488 conn(0x7f78fc0719a0 msgr2=0x7f78fc071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:55.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.216+0000 7f79039ff700 1 -- 172.21.15.92:0/3841341488 shutdown_connections 2023-11-28T11:22:55.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.216+0000 7f79039ff700 1 -- 172.21.15.92:0/3841341488 wait complete. 2023-11-28T11:22:55.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.219+0000 7f79039ff700 1 Processor -- start 2023-11-28T11:22:55.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.219+0000 7f79039ff700 1 -- start start 2023-11-28T11:22:55.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.220+0000 7f79039ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78fc0b8510 0x7f78fc1f5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:55.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.220+0000 7f79039ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78fc0cd460 0x7f78fc157740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:55.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.220+0000 7f79039ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78fc0d0d60 0x7f78fc156600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:55.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.220+0000 7f79039ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc0cbb60 con 0x7f78fc0b8510 2023-11-28T11:22:55.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.220+0000 7f79039ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc157c80 con 0x7f78fc0d0d60 2023-11-28T11:22:55.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.220+0000 7f79039ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc157f50 con 0x7f78fc0cd460 2023-11-28T11:22:55.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.220+0000 7f790179b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78fc0b8510 0x7f78fc1f5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:55.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.220+0000 7f790179b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78fc0b8510 0x7f78fc1f5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50706/0 (socket says 172.21.15.92:50706) 2023-11-28T11:22:55.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.220+0000 7f790179b700 1 -- 172.21.15.92:0/788789068 learned_addr learned my addr 172.21.15.92:0/788789068 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:55.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.221+0000 7f7901f9c700 1 --2- 172.21.15.92:0/788789068 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78fc0d0d60 0x7f78fc156600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:55.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.221+0000 7f7900f9a700 1 --2- 172.21.15.92:0/788789068 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78fc0cd460 0x7f78fc157740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:55.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.221+0000 7f790179b700 1 -- 172.21.15.92:0/788789068 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78fc0cd460 msgr2=0x7f78fc157740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:55.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.221+0000 7f790179b700 1 --2- 172.21.15.92:0/788789068 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78fc0cd460 0x7f78fc157740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:55.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.221+0000 7f790179b700 1 -- 172.21.15.92:0/788789068 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78fc0d0d60 msgr2=0x7f78fc156600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:55.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.221+0000 7f790179b700 1 --2- 172.21.15.92:0/788789068 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78fc0d0d60 0x7f78fc156600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:55.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.221+0000 7f790179b700 1 -- 172.21.15.92:0/788789068 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f4009a70 con 0x7f78fc0b8510 2023-11-28T11:22:55.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.221+0000 7f7901f9c700 1 --2- 172.21.15.92:0/788789068 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78fc0d0d60 0x7f78fc156600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:22:55.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.221+0000 7f790179b700 1 --2- 172.21.15.92:0/788789068 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78fc0b8510 0x7f78fc1f5c60 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f78f800c4d0 tx=0x7f78f800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:55.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.221+0000 7f78f1ffb700 1 -- 172.21.15.92:0/788789068 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f8005d70 con 0x7f78fc0b8510 2023-11-28T11:22:55.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.221+0000 7f78f1ffb700 1 -- 172.21.15.92:0/788789068 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78f80070d0 con 0x7f78fc0b8510 2023-11-28T11:22:55.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.221+0000 7f79039ff700 1 -- 172.21.15.92:0/788789068 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78ec0094a0 con 0x7f78fc0b8510 2023-11-28T11:22:55.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.221+0000 7f78f1ffb700 1 -- 172.21.15.92:0/788789068 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f8007440 con 0x7f78fc0b8510 2023-11-28T11:22:55.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.222+0000 7f79039ff700 1 -- 172.21.15.92:0/788789068 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78fc0cfff0 con 0x7f78fc0b8510 2023-11-28T11:22:55.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.223+0000 7f79039ff700 1 -- 172.21.15.92:0/788789068 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78fc0cfd10 con 0x7f78fc0b8510 2023-11-28T11:22:55.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.223+0000 7f78f1ffb700 1 -- 172.21.15.92:0/788789068 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f78f8019030 con 0x7f78fc0b8510 2023-11-28T11:22:55.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.224+0000 7f78f1ffb700 1 --2- 172.21.15.92:0/788789068 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f78e00682f0 0x7f78e006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:55.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.224+0000 7f78f1ffb700 1 -- 172.21.15.92:0/788789068 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f78f8085030 con 0x7f78fc0b8510 2023-11-28T11:22:55.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.225+0000 7f7900f9a700 1 --2- 172.21.15.92:0/788789068 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f78e00682f0 0x7f78e006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:55.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.225+0000 7f7900f9a700 1 --2- 172.21.15.92:0/788789068 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f78e00682f0 0x7f78e006a7a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f78fc0b1de0 tx=0x7f78ec008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:55.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.228+0000 7f78f1ffb700 1 -- 172.21.15.92:0/788789068 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f78f804b5e0 con 0x7f78fc0b8510 2023-11-28T11:22:55.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.417+0000 7f79039ff700 1 -- 172.21.15.92:0/788789068 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f78fc067880 con 0x7f78fc0b8510 2023-11-28T11:22:55.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.418+0000 7f78f1ffb700 1 -- 172.21.15.92:0/788789068 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v44) v1 ==== 93+0+10 (secure 0 0 0) 0x7f78f8017020 con 0x7f78fc0b8510 2023-11-28T11:22:55.419 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-11-28T11:22:55.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.420+0000 7f79039ff700 1 -- 172.21.15.92:0/788789068 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f78e00682f0 msgr2=0x7f78e006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:55.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.420+0000 7f79039ff700 1 --2- 172.21.15.92:0/788789068 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f78e00682f0 0x7f78e006a7a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f78fc0b1de0 tx=0x7f78ec008030 comp rx=0 tx=0).stop 2023-11-28T11:22:55.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.420+0000 7f79039ff700 1 -- 172.21.15.92:0/788789068 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78fc0b8510 msgr2=0x7f78fc1f5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:55.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.420+0000 7f79039ff700 1 --2- 172.21.15.92:0/788789068 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78fc0b8510 0x7f78fc1f5c60 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f78f800c4d0 tx=0x7f78f800c7e0 comp rx=0 tx=0).stop 2023-11-28T11:22:55.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.420+0000 7f79039ff700 1 -- 172.21.15.92:0/788789068 shutdown_connections 2023-11-28T11:22:55.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.420+0000 7f79039ff700 1 --2- 172.21.15.92:0/788789068 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f78fc0d0d60 0x7f78fc156600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:55.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.420+0000 7f79039ff700 1 --2- 172.21.15.92:0/788789068 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f78e00682f0 0x7f78e006a7a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:55.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.420+0000 7f79039ff700 1 --2- 172.21.15.92:0/788789068 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78fc0b8510 0x7f78fc1f5c60 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:55.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.420+0000 7f79039ff700 1 --2- 172.21.15.92:0/788789068 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f78fc0cd460 0x7f78fc157740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:55.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.420+0000 7f79039ff700 1 -- 172.21.15.92:0/788789068 >> 172.21.15.92:0/788789068 conn(0x7f78fc0719a0 msgr2=0x7f78fc0b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:55.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.421+0000 7f79039ff700 1 -- 172.21.15.92:0/788789068 shutdown_connections 2023-11-28T11:22:55.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.421+0000 7f79039ff700 1 -- 172.21.15.92:0/788789068 wait complete. 2023-11-28T11:22:55.440 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:22:55.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.638+0000 7f2d7d1ff700 1 Processor -- start 2023-11-28T11:22:55.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.638+0000 7f2d7d1ff700 1 -- start start 2023-11-28T11:22:55.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.639+0000 7f2d7d1ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f2d780b8510 0x7f2d7806b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:55.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.639+0000 7f2d7d1ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d780d0d60 0x7f2d780d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:55.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.639+0000 7f2d7d1ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d780b88e0 con 0x7f2d780cd460 2023-11-28T11:22:55.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.639+0000 7f2d7d1ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d780cdbd0 con 0x7f2d780b8510 2023-11-28T11:22:55.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.639+0000 7f2d7d1ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d780d02d0 con 0x7f2d780d0d60 2023-11-28T11:22:55.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.640+0000 7f2d7759e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d780d0d60 0x7f2d780d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:55.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.640+0000 7f2d7759e700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d780d0d60 0x7f2d780d1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.92:56318/0 (socket says 172.21.15.92:56318) 2023-11-28T11:22:55.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.640+0000 7f2d7759e700 1 -- 172.21.15.92:0/595903122 learned_addr learned my addr 172.21.15.92:0/595903122 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:55.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.640+0000 7f2d76d9d700 1 --2- 172.21.15.92:0/595903122 >> v2:172.21.15.169:3300/0 conn(0x7f2d780b8510 0x7f2d7806b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:55.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.640+0000 7f2d7759e700 1 -- 172.21.15.92:0/595903122 >> v2:172.21.15.169:3300/0 conn(0x7f2d780b8510 msgr2=0x7f2d7806b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:55.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.640+0000 7f2d7759e700 1 --2- 172.21.15.92:0/595903122 >> v2:172.21.15.169:3300/0 conn(0x7f2d780b8510 0x7f2d7806b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:55.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.640+0000 7f2d7759e700 1 -- 172.21.15.92:0/595903122 >> v1:172.21.15.169:6789/0 conn(0x7f2d780cd460 legacy=0x7f2d7806beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:55.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.640+0000 7f2d7759e700 1 -- 172.21.15.92:0/595903122 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d780cd8f0 con 0x7f2d780d0d60 2023-11-28T11:22:55.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.640+0000 7f2d7759e700 1 --2- 172.21.15.92:0/595903122 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d780d0d60 0x7f2d780d1200 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2d70009e50 tx=0x7f2d7000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7bc5dcdc076f554 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:55.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.641+0000 7f2d75d9b700 1 -- 172.21.15.92:0/595903122 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7000e070 con 0x7f2d780d0d60 2023-11-28T11:22:55.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.641+0000 7f2d75d9b700 1 -- 172.21.15.92:0/595903122 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d7000b910 con 0x7f2d780d0d60 2023-11-28T11:22:55.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.641+0000 7f2d75d9b700 1 -- 172.21.15.92:0/595903122 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7000bcc0 con 0x7f2d780d0d60 2023-11-28T11:22:55.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.641+0000 7f2d7d1ff700 1 -- 172.21.15.92:0/595903122 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d780d0d60 msgr2=0x7f2d780d1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:55.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.641+0000 7f2d7d1ff700 1 --2- 172.21.15.92:0/595903122 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d780d0d60 0x7f2d780d1200 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2d70009e50 tx=0x7f2d7000b330 comp rx=0 tx=0).stop 2023-11-28T11:22:55.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.641+0000 7f2d7d1ff700 1 -- 172.21.15.92:0/595903122 shutdown_connections 2023-11-28T11:22:55.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.641+0000 7f2d7d1ff700 1 --2- 172.21.15.92:0/595903122 >> v2:172.21.15.169:3300/0 conn(0x7f2d780b8510 0x7f2d7806b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:55.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.641+0000 7f2d7d1ff700 1 --2- 172.21.15.92:0/595903122 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d780d0d60 0x7f2d780d1200 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:55.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.641+0000 7f2d7d1ff700 1 -- 172.21.15.92:0/595903122 >> 172.21.15.92:0/595903122 conn(0x7f2d780719a0 msgr2=0x7f2d78071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:55.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.641+0000 7f2d7d1ff700 1 -- 172.21.15.92:0/595903122 shutdown_connections 2023-11-28T11:22:55.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.641+0000 7f2d7d1ff700 1 -- 172.21.15.92:0/595903122 wait complete. 2023-11-28T11:22:55.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.643+0000 7f2d7d1ff700 1 Processor -- start 2023-11-28T11:22:55.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.643+0000 7f2d7d1ff700 1 -- start start 2023-11-28T11:22:55.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.644+0000 7f2d7d1ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d780b8510 0x7f2d781f27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:55.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.644+0000 7f2d7d1ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d780cd460 0x7f2d781f51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:55.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.644+0000 7f2d7d1ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d780d0d60 0x7f2d7815abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:55.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.644+0000 7f2d7d1ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d781f59e0 con 0x7f2d780b8510 2023-11-28T11:22:55.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.644+0000 7f2d7d1ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d781f5cc0 con 0x7f2d780cd460 2023-11-28T11:22:55.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.644+0000 7f2d7d1ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d780ca770 con 0x7f2d780d0d60 2023-11-28T11:22:55.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.644+0000 7f2d76d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d780b8510 0x7f2d781f27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:55.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.644+0000 7f2d76d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d780b8510 0x7f2d781f27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50708/0 (socket says 172.21.15.92:50708) 2023-11-28T11:22:55.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.644+0000 7f2d76d9d700 1 -- 172.21.15.92:0/1970019225 learned_addr learned my addr 172.21.15.92:0/1970019225 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:55.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.645+0000 7f2d7759e700 1 --2- 172.21.15.92:0/1970019225 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d780d0d60 0x7f2d7815abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:55.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.645+0000 7f2d7659c700 1 --2- 172.21.15.92:0/1970019225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d780cd460 0x7f2d781f51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:55.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.645+0000 7f2d76d9d700 1 -- 172.21.15.92:0/1970019225 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d780d0d60 msgr2=0x7f2d7815abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:55.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.645+0000 7f2d76d9d700 1 --2- 172.21.15.92:0/1970019225 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d780d0d60 0x7f2d7815abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:55.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.645+0000 7f2d76d9d700 1 -- 172.21.15.92:0/1970019225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d780cd460 msgr2=0x7f2d781f51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:55.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.645+0000 7f2d76d9d700 1 --2- 172.21.15.92:0/1970019225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d780cd460 0x7f2d781f51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:55.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.645+0000 7f2d76d9d700 1 -- 172.21.15.92:0/1970019225 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d70009b30 con 0x7f2d780b8510 2023-11-28T11:22:55.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.645+0000 7f2d7659c700 1 --2- 172.21.15.92:0/1970019225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d780cd460 0x7f2d781f51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:22:55.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.645+0000 7f2d76d9d700 1 --2- 172.21.15.92:0/1970019225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d780b8510 0x7f2d781f27b0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f2d6800e9b0 tx=0x7f2d6800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:55.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.645+0000 7f2d5f7fe700 1 -- 172.21.15.92:0/1970019225 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d68009710 con 0x7f2d780b8510 2023-11-28T11:22:55.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.645+0000 7f2d5f7fe700 1 -- 172.21.15.92:0/1970019225 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d6800c850 con 0x7f2d780b8510 2023-11-28T11:22:55.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.645+0000 7f2d7d1ff700 1 -- 172.21.15.92:0/1970019225 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d780cfff0 con 0x7f2d780b8510 2023-11-28T11:22:55.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.645+0000 7f2d5f7fe700 1 -- 172.21.15.92:0/1970019225 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6800cbc0 con 0x7f2d780b8510 2023-11-28T11:22:55.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.646+0000 7f2d7d1ff700 1 -- 172.21.15.92:0/1970019225 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d780cfd10 con 0x7f2d780b8510 2023-11-28T11:22:55.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.647+0000 7f2d7d1ff700 1 -- 172.21.15.92:0/1970019225 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d781f5750 con 0x7f2d780b8510 2023-11-28T11:22:55.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.648+0000 7f2d5f7fe700 1 -- 172.21.15.92:0/1970019225 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2d68019030 con 0x7f2d780b8510 2023-11-28T11:22:55.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.649+0000 7f2d5f7fe700 1 --2- 172.21.15.92:0/1970019225 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f2d640682f0 0x7f2d6406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:55.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.649+0000 7f2d5f7fe700 1 -- 172.21.15.92:0/1970019225 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f2d68085db0 con 0x7f2d780b8510 2023-11-28T11:22:55.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.649+0000 7f2d7659c700 1 --2- 172.21.15.92:0/1970019225 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f2d640682f0 0x7f2d6406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:55.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.650+0000 7f2d7659c700 1 --2- 172.21.15.92:0/1970019225 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f2d640682f0 0x7f2d6406a7a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2d781f5110 tx=0x7f2d6c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:55.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.653+0000 7f2d5f7fe700 1 -- 172.21.15.92:0/1970019225 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2d68052410 con 0x7f2d780b8510 2023-11-28T11:22:55.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.842+0000 7f2d7d1ff700 1 -- 172.21.15.92:0/1970019225 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2d78067880 con 0x7f2d780b8510 2023-11-28T11:22:55.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.842+0000 7f2d5f7fe700 1 -- 172.21.15.92:0/1970019225 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v44) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2d680170a0 con 0x7f2d780b8510 2023-11-28T11:22:55.844 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-11-28T11:22:55.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.844+0000 7f2d7d1ff700 1 -- 172.21.15.92:0/1970019225 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f2d640682f0 msgr2=0x7f2d6406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:55.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.844+0000 7f2d7d1ff700 1 --2- 172.21.15.92:0/1970019225 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f2d640682f0 0x7f2d6406a7a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2d781f5110 tx=0x7f2d6c009040 comp rx=0 tx=0).stop 2023-11-28T11:22:55.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.845+0000 7f2d7d1ff700 1 -- 172.21.15.92:0/1970019225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d780b8510 msgr2=0x7f2d781f27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:55.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.845+0000 7f2d7d1ff700 1 --2- 172.21.15.92:0/1970019225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d780b8510 0x7f2d781f27b0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f2d6800e9b0 tx=0x7f2d6800ecc0 comp rx=0 tx=0).stop 2023-11-28T11:22:55.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.845+0000 7f2d7d1ff700 1 -- 172.21.15.92:0/1970019225 shutdown_connections 2023-11-28T11:22:55.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.845+0000 7f2d7d1ff700 1 --2- 172.21.15.92:0/1970019225 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d780d0d60 0x7f2d7815abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:55.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.845+0000 7f2d7d1ff700 1 --2- 172.21.15.92:0/1970019225 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f2d640682f0 0x7f2d6406a7a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:55.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.845+0000 7f2d7d1ff700 1 --2- 172.21.15.92:0/1970019225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d780b8510 0x7f2d781f27b0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:55.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.845+0000 7f2d7d1ff700 1 --2- 172.21.15.92:0/1970019225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d780cd460 0x7f2d781f51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:55.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.845+0000 7f2d7d1ff700 1 -- 172.21.15.92:0/1970019225 >> 172.21.15.92:0/1970019225 conn(0x7f2d780719a0 msgr2=0x7f2d780b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:55.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.845+0000 7f2d7d1ff700 1 -- 172.21.15.92:0/1970019225 shutdown_connections 2023-11-28T11:22:55.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:55.845+0000 7f2d7d1ff700 1 -- 172.21.15.92:0/1970019225 wait complete. 2023-11-28T11:22:55.864 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:22:56.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.062+0000 7fc007fff700 1 Processor -- start 2023-11-28T11:22:56.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.062+0000 7fc007fff700 1 -- start start 2023-11-28T11:22:56.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.063+0000 7fc007fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc0000cd460 0x7fc00006c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:56.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.063+0000 7fc007fff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc0000d0d60 0x7fc0000d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:56.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.064+0000 7fc007fff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0000b88e0 con 0x7fc0000b8510 2023-11-28T11:22:56.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.064+0000 7fc007fff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0000cdbd0 con 0x7fc0000cd460 2023-11-28T11:22:56.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.064+0000 7fc007fff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0000d02d0 con 0x7fc0000d0d60 2023-11-28T11:22:56.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.064+0000 7fc005d9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc0000b8510 0x7fc00006b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51102/0 (socket says 172.21.15.92:51102) 2023-11-28T11:22:56.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.064+0000 7fc005d9b700 1 -- 172.21.15.92:0/1480122083 learned_addr learned my addr 172.21.15.92:0/1480122083 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:22:56.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.064+0000 7fc00559a700 1 --2- 172.21.15.92:0/1480122083 >> v2:172.21.15.92:3300/0 conn(0x7fc0000cd460 0x7fc00006c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:56.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.064+0000 7fc00659c700 1 --2- 172.21.15.92:0/1480122083 >> v2:172.21.15.169:3300/0 conn(0x7fc0000d0d60 0x7fc0000d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:56.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.064+0000 7fc004d99700 1 -- 172.21.15.92:0/1480122083 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 502857857 0 0) 0x7fc0000b88e0 con 0x7fc0000b8510 2023-11-28T11:22:56.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.064+0000 7fc004d99700 1 -- 172.21.15.92:0/1480122083 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfec0035b0 con 0x7fc0000b8510 2023-11-28T11:22:56.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.064+0000 7fc00559a700 1 -- 172.21.15.92:0/1480122083 >> v2:172.21.15.169:3300/0 conn(0x7fc0000d0d60 msgr2=0x7fc0000d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:56.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.064+0000 7fc00559a700 1 --2- 172.21.15.92:0/1480122083 >> v2:172.21.15.169:3300/0 conn(0x7fc0000d0d60 0x7fc0000d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:56.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.064+0000 7fc00559a700 1 -- 172.21.15.92:0/1480122083 >> v1:172.21.15.92:6789/0 conn(0x7fc0000b8510 legacy=0x7fc00006b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:56.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.064+0000 7fc00559a700 1 -- 172.21.15.92:0/1480122083 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0000cd8f0 con 0x7fc0000cd460 2023-11-28T11:22:56.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.065+0000 7fc00559a700 1 --2- 172.21.15.92:0/1480122083 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0000cd460 0x7fc00006c080 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc0000ce8c0 tx=0x7fbff0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8852c33341d1dd1e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:56.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.065+0000 7fc004d99700 1 -- 172.21.15.92:0/1480122083 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff0011830 con 0x7fc0000cd460 2023-11-28T11:22:56.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.065+0000 7fc004d99700 1 -- 172.21.15.92:0/1480122083 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff0011990 con 0x7fc0000cd460 2023-11-28T11:22:56.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.065+0000 7fc004d99700 1 -- 172.21.15.92:0/1480122083 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff0011d20 con 0x7fc0000cd460 2023-11-28T11:22:56.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.065+0000 7fc007fff700 1 -- 172.21.15.92:0/1480122083 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0000cd460 msgr2=0x7fc00006c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:56.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.065+0000 7fc007fff700 1 --2- 172.21.15.92:0/1480122083 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0000cd460 0x7fc00006c080 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc0000ce8c0 tx=0x7fbff0009b50 comp rx=0 tx=0).stop 2023-11-28T11:22:56.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.065+0000 7fc007fff700 1 -- 172.21.15.92:0/1480122083 shutdown_connections 2023-11-28T11:22:56.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.065+0000 7fc007fff700 1 --2- 172.21.15.92:0/1480122083 >> v2:172.21.15.169:3300/0 conn(0x7fc0000d0d60 0x7fc0000d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:56.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.065+0000 7fc007fff700 1 --2- 172.21.15.92:0/1480122083 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0000cd460 0x7fc00006c080 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:56.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.065+0000 7fc007fff700 1 -- 172.21.15.92:0/1480122083 >> 172.21.15.92:0/1480122083 conn(0x7fc0000719a0 msgr2=0x7fc000071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:56.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.066+0000 7fc007fff700 1 -- 172.21.15.92:0/1480122083 shutdown_connections 2023-11-28T11:22:56.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.066+0000 7fc007fff700 1 -- 172.21.15.92:0/1480122083 wait complete. 2023-11-28T11:22:56.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.067+0000 7fc007fff700 1 Processor -- start 2023-11-28T11:22:56.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.067+0000 7fc007fff700 1 -- start start 2023-11-28T11:22:56.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.068+0000 7fc007fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0000b8510 0x7fc000150630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:56.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.068+0000 7fc007fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0000cd460 0x7fc0000ca800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:56.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.068+0000 7fc007fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0000d0d60 0x7fc0000b3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:56.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.068+0000 7fc007fff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0001e4e00 con 0x7fc0000b8510 2023-11-28T11:22:56.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.068+0000 7fc007fff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0001e50e0 con 0x7fc0000cd460 2023-11-28T11:22:56.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.068+0000 7fc007fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0001e53c0 con 0x7fc0000d0d60 2023-11-28T11:22:56.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.069+0000 7fc005d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0000b8510 0x7fc000150630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:56.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.069+0000 7fc005d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0000b8510 0x7fc000150630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50716/0 (socket says 172.21.15.92:50716) 2023-11-28T11:22:56.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.069+0000 7fc005d9b700 1 -- 172.21.15.92:0/2720177492 learned_addr learned my addr 172.21.15.92:0/2720177492 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:56.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.069+0000 7fc00559a700 1 --2- 172.21.15.92:0/2720177492 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0000cd460 0x7fc0000ca800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:56.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.069+0000 7fc00659c700 1 --2- 172.21.15.92:0/2720177492 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0000d0d60 0x7fc0000b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:56.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.069+0000 7fc005d9b700 1 -- 172.21.15.92:0/2720177492 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0000d0d60 msgr2=0x7fc0000b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:56.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.069+0000 7fc005d9b700 1 --2- 172.21.15.92:0/2720177492 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0000d0d60 0x7fc0000b3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:56.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.069+0000 7fc005d9b700 1 -- 172.21.15.92:0/2720177492 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0000cd460 msgr2=0x7fc0000ca800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:56.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.069+0000 7fc005d9b700 1 --2- 172.21.15.92:0/2720177492 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0000cd460 0x7fc0000ca800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:56.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.069+0000 7fc005d9b700 1 -- 172.21.15.92:0/2720177492 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff00094a0 con 0x7fc0000b8510 2023-11-28T11:22:56.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.069+0000 7fc005d9b700 1 --2- 172.21.15.92:0/2720177492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0000b8510 0x7fc000150630 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fbffc00c700 tx=0x7fbffc00cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:56.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.069+0000 7fbff67fc700 1 -- 172.21.15.92:0/2720177492 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbffc005d70 con 0x7fc0000b8510 2023-11-28T11:22:56.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.069+0000 7fbff67fc700 1 -- 172.21.15.92:0/2720177492 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbffc0070d0 con 0x7fc0000b8510 2023-11-28T11:22:56.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.069+0000 7fc007fff700 1 -- 172.21.15.92:0/2720177492 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0000cfff0 con 0x7fc0000b8510 2023-11-28T11:22:56.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.069+0000 7fbff67fc700 1 -- 172.21.15.92:0/2720177492 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbffc007460 con 0x7fc0000b8510 2023-11-28T11:22:56.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.070+0000 7fc007fff700 1 -- 172.21.15.92:0/2720177492 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0000cfd10 con 0x7fc0000b8510 2023-11-28T11:22:56.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.071+0000 7fc007fff700 1 -- 172.21.15.92:0/2720177492 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc000067880 con 0x7fc0000b8510 2023-11-28T11:22:56.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.071+0000 7fbff67fc700 1 -- 172.21.15.92:0/2720177492 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbffc019030 con 0x7fc0000b8510 2023-11-28T11:22:56.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.073+0000 7fbff67fc700 1 --2- 172.21.15.92:0/2720177492 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fbfe40682f0 0x7fbfe406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:56.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.073+0000 7fbff67fc700 1 -- 172.21.15.92:0/2720177492 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fbffc085440 con 0x7fc0000b8510 2023-11-28T11:22:56.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.073+0000 7fc00559a700 1 --2- 172.21.15.92:0/2720177492 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fbfe40682f0 0x7fbfe406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:56.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.073+0000 7fc00559a700 1 --2- 172.21.15.92:0/2720177492 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fbfe40682f0 0x7fbfe406a7a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc0000b1de0 tx=0x7fbff001c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:56.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.077+0000 7fbff67fc700 1 -- 172.21.15.92:0/2720177492 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbffc0519b0 con 0x7fc0000b8510 2023-11-28T11:22:56.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.269+0000 7fc007fff700 1 -- 172.21.15.92:0/2720177492 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc0000d2940 con 0x7fc0000b8510 2023-11-28T11:22:56.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.269+0000 7fbff67fc700 1 -- 172.21.15.92:0/2720177492 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v44) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbffc054fd0 con 0x7fc0000b8510 2023-11-28T11:22:56.271 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-11-28T11:22:56.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.272+0000 7fc007fff700 1 -- 172.21.15.92:0/2720177492 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fbfe40682f0 msgr2=0x7fbfe406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:56.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.272+0000 7fc007fff700 1 --2- 172.21.15.92:0/2720177492 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fbfe40682f0 0x7fbfe406a7a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc0000b1de0 tx=0x7fbff001c040 comp rx=0 tx=0).stop 2023-11-28T11:22:56.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.272+0000 7fc007fff700 1 -- 172.21.15.92:0/2720177492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0000b8510 msgr2=0x7fc000150630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:56.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.272+0000 7fc007fff700 1 --2- 172.21.15.92:0/2720177492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0000b8510 0x7fc000150630 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fbffc00c700 tx=0x7fbffc00cac0 comp rx=0 tx=0).stop 2023-11-28T11:22:56.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.272+0000 7fc007fff700 1 -- 172.21.15.92:0/2720177492 shutdown_connections 2023-11-28T11:22:56.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.272+0000 7fc007fff700 1 --2- 172.21.15.92:0/2720177492 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc0000d0d60 0x7fc0000b3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:56.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.272+0000 7fc007fff700 1 --2- 172.21.15.92:0/2720177492 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fbfe40682f0 0x7fbfe406a7a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:56.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.272+0000 7fc007fff700 1 --2- 172.21.15.92:0/2720177492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0000b8510 0x7fc000150630 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:56.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.272+0000 7fc007fff700 1 --2- 172.21.15.92:0/2720177492 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc0000cd460 0x7fc0000ca800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:56.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.272+0000 7fc007fff700 1 -- 172.21.15.92:0/2720177492 >> 172.21.15.92:0/2720177492 conn(0x7fc0000719a0 msgr2=0x7fc0000b5d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:56.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.272+0000 7fc007fff700 1 -- 172.21.15.92:0/2720177492 shutdown_connections 2023-11-28T11:22:56.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.272+0000 7fc007fff700 1 -- 172.21.15.92:0/2720177492 wait complete. 2023-11-28T11:22:56.291 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:22:56.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.488+0000 7f9f20bff700 1 Processor -- start 2023-11-28T11:22:56.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.488+0000 7f9f20bff700 1 -- start start 2023-11-28T11:22:56.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.489+0000 7f9f20bff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9f1c0cd460 0x7f9f1c06c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:56.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.489+0000 7f9f20bff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f1c0d0d60 0x7f9f1c0d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:56.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.489+0000 7f9f20bff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f1c0b88e0 con 0x7f9f1c0b8510 2023-11-28T11:22:56.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.489+0000 7f9f20bff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f1c0cdbd0 con 0x7f9f1c0cd460 2023-11-28T11:22:56.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.489+0000 7f9f20bff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1c0d02d0 con 0x7f9f1c0d0d60 2023-11-28T11:22:56.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.490+0000 7f9f19d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9f1c0cd460 0x7f9f1c06c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:56.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.490+0000 7f9f19d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9f1c0cd460 0x7f9f1c06c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50720/0 (socket says 172.21.15.92:50720) 2023-11-28T11:22:56.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.490+0000 7f9f19d9b700 1 -- 172.21.15.92:0/1080764590 learned_addr learned my addr 172.21.15.92:0/1080764590 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:56.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.490+0000 7f9f1ad9d700 1 --2- 172.21.15.92:0/1080764590 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f1c0d0d60 0x7f9f1c0d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:56.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.490+0000 7f9f19d9b700 1 -- 172.21.15.92:0/1080764590 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f1c0d0d60 msgr2=0x7f9f1c0d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:56.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.490+0000 7f9f19d9b700 1 --2- 172.21.15.92:0/1080764590 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f1c0d0d60 0x7f9f1c0d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:56.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.490+0000 7f9f19d9b700 1 -- 172.21.15.92:0/1080764590 >> v1:172.21.15.169:6789/0 conn(0x7f9f1c0b8510 legacy=0x7f9f1c06b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:56.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.490+0000 7f9f19d9b700 1 -- 172.21.15.92:0/1080764590 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f1c0cd8f0 con 0x7f9f1c0cd460 2023-11-28T11:22:56.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.490+0000 7f9f19d9b700 1 --2- 172.21.15.92:0/1080764590 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f1c0cd460 0x7f9f1c06c080 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f9f10009920 tx=0x7f9f10009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82b6b7cac8e011a8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:56.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.491+0000 7f9f1959a700 1 -- 172.21.15.92:0/1080764590 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f10011830 con 0x7f9f1c0cd460 2023-11-28T11:22:56.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.491+0000 7f9f1959a700 1 -- 172.21.15.92:0/1080764590 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f10011990 con 0x7f9f1c0cd460 2023-11-28T11:22:56.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.491+0000 7f9f1959a700 1 -- 172.21.15.92:0/1080764590 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f10011d20 con 0x7f9f1c0cd460 2023-11-28T11:22:56.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.491+0000 7f9f20bff700 1 -- 172.21.15.92:0/1080764590 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f1c0cd460 msgr2=0x7f9f1c06c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:56.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.491+0000 7f9f20bff700 1 --2- 172.21.15.92:0/1080764590 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f1c0cd460 0x7f9f1c06c080 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f9f10009920 tx=0x7f9f10009c30 comp rx=0 tx=0).stop 2023-11-28T11:22:56.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.491+0000 7f9f20bff700 1 -- 172.21.15.92:0/1080764590 shutdown_connections 2023-11-28T11:22:56.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.491+0000 7f9f20bff700 1 --2- 172.21.15.92:0/1080764590 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f1c0d0d60 0x7f9f1c0d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:56.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.491+0000 7f9f20bff700 1 --2- 172.21.15.92:0/1080764590 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f1c0cd460 0x7f9f1c06c080 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:56.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.491+0000 7f9f20bff700 1 -- 172.21.15.92:0/1080764590 >> 172.21.15.92:0/1080764590 conn(0x7f9f1c0719a0 msgr2=0x7f9f1c071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:56.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.491+0000 7f9f20bff700 1 -- 172.21.15.92:0/1080764590 shutdown_connections 2023-11-28T11:22:56.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.491+0000 7f9f20bff700 1 -- 172.21.15.92:0/1080764590 wait complete. 2023-11-28T11:22:56.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.493+0000 7f9f20bff700 1 Processor -- start 2023-11-28T11:22:56.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.493+0000 7f9f20bff700 1 -- start start 2023-11-28T11:22:56.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.494+0000 7f9f20bff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f1c0b8510 0x7f9f1c1f2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:56.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.494+0000 7f9f20bff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f1c0cd460 0x7f9f1c1f4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:56.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.494+0000 7f9f20bff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f1c0d0d60 0x7f9f1c15a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:56.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.494+0000 7f9f20bff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1c15a0c0 con 0x7f9f1c0cd460 2023-11-28T11:22:56.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.494+0000 7f9f20bff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1c1e19a0 con 0x7f9f1c0d0d60 2023-11-28T11:22:56.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.494+0000 7f9f20bff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1c1e1c80 con 0x7f9f1c0b8510 2023-11-28T11:22:56.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.494+0000 7f9f19d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f1c0cd460 0x7f9f1c1f4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:56.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.494+0000 7f9f19d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f1c0cd460 0x7f9f1c1f4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50730/0 (socket says 172.21.15.92:50730) 2023-11-28T11:22:56.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.494+0000 7f9f19d9b700 1 -- 172.21.15.92:0/1477210559 learned_addr learned my addr 172.21.15.92:0/1477210559 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:56.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.495+0000 7f9f19d9b700 1 -- 172.21.15.92:0/1477210559 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f1c0b8510 msgr2=0x7f9f1c1f2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:56.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.495+0000 7f9f19d9b700 1 --2- 172.21.15.92:0/1477210559 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f1c0b8510 0x7f9f1c1f2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:56.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.495+0000 7f9f19d9b700 1 -- 172.21.15.92:0/1477210559 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f1c0d0d60 msgr2=0x7f9f1c15a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:56.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.495+0000 7f9f19d9b700 1 --2- 172.21.15.92:0/1477210559 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f1c0d0d60 0x7f9f1c15a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:56.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.495+0000 7f9f19d9b700 1 -- 172.21.15.92:0/1477210559 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f10009550 con 0x7f9f1c0cd460 2023-11-28T11:22:56.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.495+0000 7f9f19d9b700 1 --2- 172.21.15.92:0/1477210559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f1c0cd460 0x7f9f1c1f4c90 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f9f10000c00 tx=0x7f9f10004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:56.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.495+0000 7f9f0affd700 1 -- 172.21.15.92:0/1477210559 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f10004400 con 0x7f9f1c0cd460 2023-11-28T11:22:56.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.495+0000 7f9f0affd700 1 -- 172.21.15.92:0/1477210559 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f1001f070 con 0x7f9f1c0cd460 2023-11-28T11:22:56.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.495+0000 7f9f20bff700 1 -- 172.21.15.92:0/1477210559 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f1c0cfff0 con 0x7f9f1c0cd460 2023-11-28T11:22:56.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.495+0000 7f9f0affd700 1 -- 172.21.15.92:0/1477210559 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f100038e0 con 0x7f9f1c0cd460 2023-11-28T11:22:56.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.495+0000 7f9f20bff700 1 -- 172.21.15.92:0/1477210559 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f1c0cfd10 con 0x7f9f1c0cd460 2023-11-28T11:22:56.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.496+0000 7f9f20bff700 1 -- 172.21.15.92:0/1477210559 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f1c067880 con 0x7f9f1c0cd460 2023-11-28T11:22:56.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.497+0000 7f9f0affd700 1 -- 172.21.15.92:0/1477210559 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9f1002a020 con 0x7f9f1c0cd460 2023-11-28T11:22:56.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.498+0000 7f9f0affd700 1 --2- 172.21.15.92:0/1477210559 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9efc0682f0 0x7f9efc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:56.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.498+0000 7f9f0affd700 1 -- 172.21.15.92:0/1477210559 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f9f10087360 con 0x7f9f1c0cd460 2023-11-28T11:22:56.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.498+0000 7f9f1a59c700 1 --2- 172.21.15.92:0/1477210559 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9efc0682f0 0x7f9efc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:56.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.499+0000 7f9f1a59c700 1 --2- 172.21.15.92:0/1477210559 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9efc0682f0 0x7f9efc06a7a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c000f60 tx=0x7f9f0c007a40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:56.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.502+0000 7f9f0affd700 1 -- 172.21.15.92:0/1477210559 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9f10053950 con 0x7f9f1c0cd460 2023-11-28T11:22:56.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.695+0000 7f9f20bff700 1 -- 172.21.15.92:0/1477210559 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9f1c0d2940 con 0x7f9f1c0cd460 2023-11-28T11:22:56.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.695+0000 7f9f0affd700 1 -- 172.21.15.92:0/1477210559 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v44) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9f10056f70 con 0x7f9f1c0cd460 2023-11-28T11:22:56.696 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-11-28T11:22:56.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.697+0000 7f9f20bff700 1 -- 172.21.15.92:0/1477210559 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9efc0682f0 msgr2=0x7f9efc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:56.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.697+0000 7f9f20bff700 1 --2- 172.21.15.92:0/1477210559 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9efc0682f0 0x7f9efc06a7a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c000f60 tx=0x7f9f0c007a40 comp rx=0 tx=0).stop 2023-11-28T11:22:56.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.697+0000 7f9f20bff700 1 -- 172.21.15.92:0/1477210559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f1c0cd460 msgr2=0x7f9f1c1f4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:56.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.697+0000 7f9f20bff700 1 --2- 172.21.15.92:0/1477210559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f1c0cd460 0x7f9f1c1f4c90 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f9f10000c00 tx=0x7f9f10004750 comp rx=0 tx=0).stop 2023-11-28T11:22:56.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.698+0000 7f9f20bff700 1 -- 172.21.15.92:0/1477210559 shutdown_connections 2023-11-28T11:22:56.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.698+0000 7f9f20bff700 1 --2- 172.21.15.92:0/1477210559 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f1c0d0d60 0x7f9f1c15a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:56.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.698+0000 7f9f20bff700 1 --2- 172.21.15.92:0/1477210559 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f1c0b8510 0x7f9f1c1f2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:56.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.698+0000 7f9f20bff700 1 --2- 172.21.15.92:0/1477210559 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9efc0682f0 0x7f9efc06a7a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:56.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.698+0000 7f9f20bff700 1 --2- 172.21.15.92:0/1477210559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f1c0cd460 0x7f9f1c1f4c90 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:56.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.698+0000 7f9f20bff700 1 -- 172.21.15.92:0/1477210559 >> 172.21.15.92:0/1477210559 conn(0x7f9f1c0719a0 msgr2=0x7f9f1c0b5d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:56.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.698+0000 7f9f20bff700 1 -- 172.21.15.92:0/1477210559 shutdown_connections 2023-11-28T11:22:56.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.698+0000 7f9f20bff700 1 -- 172.21.15.92:0/1477210559 wait complete. 2023-11-28T11:22:56.717 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:22:56.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.915+0000 7f22fabff700 1 Processor -- start 2023-11-28T11:22:56.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.915+0000 7f22fabff700 1 -- start start 2023-11-28T11:22:56.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.916+0000 7f22fabff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f22f40b8510 0x7f22f406b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:56.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.916+0000 7f22fabff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f22f40cd460 0x7f22f406beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:56.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.916+0000 7f22fabff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22f40b88e0 con 0x7f22f406c3f0 2023-11-28T11:22:56.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.916+0000 7f22fabff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22f40cdbd0 con 0x7f22f40cd460 2023-11-28T11:22:56.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.916+0000 7f22fabff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22f40d02d0 con 0x7f22f40b8510 2023-11-28T11:22:56.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.916+0000 7f22f3fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f22f40cd460 0x7f22f406beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:56.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.916+0000 7f22f919c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f22f406c3f0 0x7f22f415cc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51104/0 (socket says 172.21.15.92:51104) 2023-11-28T11:22:56.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.917+0000 7f22f919c700 1 -- 172.21.15.92:0/766321263 learned_addr learned my addr 172.21.15.92:0/766321263 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:22:56.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.917+0000 7f22f899b700 1 --2- 172.21.15.92:0/766321263 >> v2:172.21.15.169:3300/0 conn(0x7f22f40b8510 0x7f22f406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:56.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.917+0000 7f22f3fff700 1 -- 172.21.15.92:0/766321263 >> v2:172.21.15.169:3300/0 conn(0x7f22f40b8510 msgr2=0x7f22f406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:56.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.917+0000 7f22f3fff700 1 --2- 172.21.15.92:0/766321263 >> v2:172.21.15.169:3300/0 conn(0x7f22f40b8510 0x7f22f406b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:56.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.917+0000 7f22f3fff700 1 -- 172.21.15.92:0/766321263 >> v1:172.21.15.92:6789/0 conn(0x7f22f406c3f0 legacy=0x7f22f415cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:56.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.917+0000 7f22f3fff700 1 -- 172.21.15.92:0/766321263 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22f40cd8f0 con 0x7f22f40cd460 2023-11-28T11:22:56.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.917+0000 7f22f37fe700 1 -- 172.21.15.92:0/766321263 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1832063838 0 0) 0x7f22f40b88e0 con 0x7f22f406c3f0 2023-11-28T11:22:56.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.917+0000 7f22f3fff700 1 --2- 172.21.15.92:0/766321263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22f40cd460 0x7f22f406beb0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f22e0009870 tx=0x7f22e0009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b713f61ea2b9894c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:56.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.917+0000 7f22f37fe700 1 -- 172.21.15.92:0/766321263 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e0011830 con 0x7f22f40cd460 2023-11-28T11:22:56.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.917+0000 7f22f37fe700 1 -- 172.21.15.92:0/766321263 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22e0011990 con 0x7f22f40cd460 2023-11-28T11:22:56.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.917+0000 7f22f37fe700 1 -- 172.21.15.92:0/766321263 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e0011d20 con 0x7f22f40cd460 2023-11-28T11:22:56.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.918+0000 7f22fabff700 1 -- 172.21.15.92:0/766321263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22f40cd460 msgr2=0x7f22f406beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:56.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.918+0000 7f22fabff700 1 --2- 172.21.15.92:0/766321263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22f40cd460 0x7f22f406beb0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f22e0009870 tx=0x7f22e0009b80 comp rx=0 tx=0).stop 2023-11-28T11:22:56.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.918+0000 7f22fabff700 1 -- 172.21.15.92:0/766321263 shutdown_connections 2023-11-28T11:22:56.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.918+0000 7f22fabff700 1 --2- 172.21.15.92:0/766321263 >> v2:172.21.15.169:3300/0 conn(0x7f22f40b8510 0x7f22f406b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:56.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.918+0000 7f22fabff700 1 --2- 172.21.15.92:0/766321263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22f40cd460 0x7f22f406beb0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:56.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.918+0000 7f22fabff700 1 -- 172.21.15.92:0/766321263 >> 172.21.15.92:0/766321263 conn(0x7f22f40719a0 msgr2=0x7f22f4071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:56.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.918+0000 7f22fabff700 1 -- 172.21.15.92:0/766321263 shutdown_connections 2023-11-28T11:22:56.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.918+0000 7f22fabff700 1 -- 172.21.15.92:0/766321263 wait complete. 2023-11-28T11:22:56.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.920+0000 7f22fabff700 1 Processor -- start 2023-11-28T11:22:56.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.920+0000 7f22fabff700 1 -- start start 2023-11-28T11:22:56.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.921+0000 7f22fabff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22f406c3f0 0x7f22f4150380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:56.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.921+0000 7f22fabff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22f40b8510 0x7f22f40d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:56.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.921+0000 7f22fabff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22f40cd460 0x7f22f40d1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:56.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.921+0000 7f22fabff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f41508c0 con 0x7f22f40b8510 2023-11-28T11:22:56.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.921+0000 7f22fabff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f40cbff0 con 0x7f22f406c3f0 2023-11-28T11:22:56.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.921+0000 7f22fabff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22f40cc2d0 con 0x7f22f40cd460 2023-11-28T11:22:56.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.921+0000 7f22f3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22f40b8510 0x7f22f40d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:56.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.921+0000 7f22f3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22f40b8510 0x7f22f40d0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50758/0 (socket says 172.21.15.92:50758) 2023-11-28T11:22:56.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.921+0000 7f22f3fff700 1 -- 172.21.15.92:0/3002772191 learned_addr learned my addr 172.21.15.92:0/3002772191 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:56.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.921+0000 7f22f3fff700 1 -- 172.21.15.92:0/3002772191 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22f40cd460 msgr2=0x7f22f40d1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:56.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.921+0000 7f22f3fff700 1 --2- 172.21.15.92:0/3002772191 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22f40cd460 0x7f22f40d1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:56.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.922+0000 7f22f3fff700 1 -- 172.21.15.92:0/3002772191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22f406c3f0 msgr2=0x7f22f4150380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:56.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.922+0000 7f22f3fff700 1 --2- 172.21.15.92:0/3002772191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22f406c3f0 0x7f22f4150380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:56.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.922+0000 7f22f3fff700 1 -- 172.21.15.92:0/3002772191 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e00094a0 con 0x7f22f40b8510 2023-11-28T11:22:56.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.922+0000 7f22f3fff700 1 --2- 172.21.15.92:0/3002772191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22f40b8510 0x7f22f40d0c10 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f22e00090b0 tx=0x7f22e0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:56.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.922+0000 7f22f17fa700 1 -- 172.21.15.92:0/3002772191 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e00038a0 con 0x7f22f40b8510 2023-11-28T11:22:56.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.922+0000 7f22f17fa700 1 -- 172.21.15.92:0/3002772191 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22e001e070 con 0x7f22f40b8510 2023-11-28T11:22:56.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.922+0000 7f22f17fa700 1 -- 172.21.15.92:0/3002772191 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e0003c80 con 0x7f22f40b8510 2023-11-28T11:22:56.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.922+0000 7f22fabff700 1 -- 172.21.15.92:0/3002772191 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22f40cfff0 con 0x7f22f40b8510 2023-11-28T11:22:56.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.922+0000 7f22fabff700 1 -- 172.21.15.92:0/3002772191 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22f40cfd10 con 0x7f22f40b8510 2023-11-28T11:22:56.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.923+0000 7f22fabff700 1 -- 172.21.15.92:0/3002772191 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22f4067880 con 0x7f22f40b8510 2023-11-28T11:22:56.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.924+0000 7f22f17fa700 1 -- 172.21.15.92:0/3002772191 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f22e000f050 con 0x7f22f40b8510 2023-11-28T11:22:56.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.925+0000 7f22f17fa700 1 --2- 172.21.15.92:0/3002772191 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f22d8068320 0x7f22d806a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:56.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.925+0000 7f22f17fa700 1 -- 172.21.15.92:0/3002772191 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f22e0086f40 con 0x7f22f40b8510 2023-11-28T11:22:56.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.925+0000 7f22f899b700 1 --2- 172.21.15.92:0/3002772191 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f22d8068320 0x7f22d806a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:56.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.926+0000 7f22f899b700 1 --2- 172.21.15.92:0/3002772191 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f22d8068320 0x7f22d806a7d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f22e8006b20 tx=0x7f22e8006ab0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:56.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:56.929+0000 7f22f17fa700 1 -- 172.21.15.92:0/3002772191 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f22e0053530 con 0x7f22f40b8510 2023-11-28T11:22:57.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.166+0000 7f22fabff700 1 -- 172.21.15.92:0/3002772191 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f22f40d2940 con 0x7f22f40b8510 2023-11-28T11:22:57.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.166+0000 7f22f17fa700 1 -- 172.21.15.92:0/3002772191 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4645 (secure 0 0 0) 0x7f22e0015020 con 0x7f22f40b8510 2023-11-28T11:22:57.167 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:22:57.167 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4976,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/3102603064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3102603064},{"type":"v1","addr":"172.21.15.169:6834","nonce":3102603064}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4979,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/79247121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":79247121},{"type":"v1","addr":"172.21.15.92:6837","nonce":79247121}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5221,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/219797192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":219797192},{"type":"v1","addr":"172.21.15.169:6835","nonce":219797192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:22:24.715447+0000","modified":"2023-11-28T11:22:27.733117+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5298},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5298":{"gid":5298,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.92:6835/1577294649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1577294649},{"type":"v1","addr":"172.21.15.92:6835","nonce":1577294649}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T11:22:57.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.168+0000 7f22fabff700 1 -- 172.21.15.92:0/3002772191 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f22d8068320 msgr2=0x7f22d806a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:57.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.168+0000 7f22fabff700 1 --2- 172.21.15.92:0/3002772191 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f22d8068320 0x7f22d806a7d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f22e8006b20 tx=0x7f22e8006ab0 comp rx=0 tx=0).stop 2023-11-28T11:22:57.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.168+0000 7f22fabff700 1 -- 172.21.15.92:0/3002772191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22f40b8510 msgr2=0x7f22f40d0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:57.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.168+0000 7f22fabff700 1 --2- 172.21.15.92:0/3002772191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22f40b8510 0x7f22f40d0c10 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f22e00090b0 tx=0x7f22e0004750 comp rx=0 tx=0).stop 2023-11-28T11:22:57.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.169+0000 7f22fabff700 1 -- 172.21.15.92:0/3002772191 shutdown_connections 2023-11-28T11:22:57.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.169+0000 7f22fabff700 1 --2- 172.21.15.92:0/3002772191 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22f40cd460 0x7f22f40d1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:57.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.169+0000 7f22fabff700 1 --2- 172.21.15.92:0/3002772191 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22f406c3f0 0x7f22f4150380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:57.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.169+0000 7f22fabff700 1 --2- 172.21.15.92:0/3002772191 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f22d8068320 0x7f22d806a7d0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:57.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.169+0000 7f22fabff700 1 --2- 172.21.15.92:0/3002772191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22f40b8510 0x7f22f40d0c10 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:57.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.169+0000 7f22fabff700 1 -- 172.21.15.92:0/3002772191 >> 172.21.15.92:0/3002772191 conn(0x7f22f40719a0 msgr2=0x7f22f40b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:57.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.169+0000 7f22fabff700 1 -- 172.21.15.92:0/3002772191 shutdown_connections 2023-11-28T11:22:57.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.169+0000 7f22fabff700 1 -- 172.21.15.92:0/3002772191 wait complete. 2023-11-28T11:22:57.171 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 17 2023-11-28T11:22:57.188 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:22:57.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.383+0000 7f377a7ff700 1 Processor -- start 2023-11-28T11:22:57.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.383+0000 7f377a7ff700 1 -- start start 2023-11-28T11:22:57.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.384+0000 7f377a7ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f37740cd460 0x7f377406c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:57.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.384+0000 7f377a7ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f37740d0d60 0x7f37740d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:57.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.384+0000 7f377a7ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37740b88e0 con 0x7f37740b8510 2023-11-28T11:22:57.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.384+0000 7f377a7ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37740cdbd0 con 0x7f37740d0d60 2023-11-28T11:22:57.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.384+0000 7f377a7ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37740d02d0 con 0x7f37740cd460 2023-11-28T11:22:57.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.384+0000 7f3773fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f37740b8510 0x7f377406b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51110/0 (socket says 172.21.15.92:51110) 2023-11-28T11:22:57.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.384+0000 7f3773fff700 1 -- 172.21.15.92:0/3263548629 learned_addr learned my addr 172.21.15.92:0/3263548629 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:22:57.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.384+0000 7f3778d9c700 1 --2- 172.21.15.92:0/3263548629 >> v2:172.21.15.92:3300/0 conn(0x7f37740d0d60 0x7f37740d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:57.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.385+0000 7f37737fe700 1 --2- 172.21.15.92:0/3263548629 >> v2:172.21.15.169:3300/0 conn(0x7f37740cd460 0x7f377406c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:57.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.385+0000 7f3772ffd700 1 -- 172.21.15.92:0/3263548629 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4146857710 0 0) 0x7f37740b88e0 con 0x7f37740b8510 2023-11-28T11:22:57.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.385+0000 7f3772ffd700 1 -- 172.21.15.92:0/3263548629 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f375c0035b0 con 0x7f37740b8510 2023-11-28T11:22:57.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.385+0000 7f3778d9c700 1 -- 172.21.15.92:0/3263548629 >> v2:172.21.15.169:3300/0 conn(0x7f37740cd460 msgr2=0x7f377406c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:57.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.385+0000 7f3778d9c700 1 --2- 172.21.15.92:0/3263548629 >> v2:172.21.15.169:3300/0 conn(0x7f37740cd460 0x7f377406c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:57.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.385+0000 7f3778d9c700 1 -- 172.21.15.92:0/3263548629 >> v1:172.21.15.92:6789/0 conn(0x7f37740b8510 legacy=0x7f377406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:57.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.385+0000 7f3778d9c700 1 -- 172.21.15.92:0/3263548629 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37740cd8f0 con 0x7f37740d0d60 2023-11-28T11:22:57.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.385+0000 7f3778d9c700 1 --2- 172.21.15.92:0/3263548629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37740d0d60 0x7f37740d1200 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f37740ceba0 tx=0x7f376c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c023f401704460dd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:57.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.385+0000 7f3772ffd700 1 -- 172.21.15.92:0/3263548629 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376c00e070 con 0x7f37740d0d60 2023-11-28T11:22:57.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.385+0000 7f3772ffd700 1 -- 172.21.15.92:0/3263548629 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f376c00bae0 con 0x7f37740d0d60 2023-11-28T11:22:57.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.385+0000 7f3772ffd700 1 -- 172.21.15.92:0/3263548629 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376c00be90 con 0x7f37740d0d60 2023-11-28T11:22:57.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.386+0000 7f377a7ff700 1 -- 172.21.15.92:0/3263548629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37740d0d60 msgr2=0x7f37740d1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:57.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.386+0000 7f377a7ff700 1 --2- 172.21.15.92:0/3263548629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37740d0d60 0x7f37740d1200 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f37740ceba0 tx=0x7f376c00b330 comp rx=0 tx=0).stop 2023-11-28T11:22:57.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.386+0000 7f377a7ff700 1 -- 172.21.15.92:0/3263548629 shutdown_connections 2023-11-28T11:22:57.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.386+0000 7f377a7ff700 1 --2- 172.21.15.92:0/3263548629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37740d0d60 0x7f37740d1200 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:57.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.386+0000 7f377a7ff700 1 --2- 172.21.15.92:0/3263548629 >> v2:172.21.15.169:3300/0 conn(0x7f37740cd460 0x7f377406c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:57.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.386+0000 7f377a7ff700 1 -- 172.21.15.92:0/3263548629 >> 172.21.15.92:0/3263548629 conn(0x7f37740719a0 msgr2=0x7f3774071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:57.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.386+0000 7f377a7ff700 1 -- 172.21.15.92:0/3263548629 shutdown_connections 2023-11-28T11:22:57.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.386+0000 7f377a7ff700 1 -- 172.21.15.92:0/3263548629 wait complete. 2023-11-28T11:22:57.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.388+0000 7f377a7ff700 1 Processor -- start 2023-11-28T11:22:57.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.388+0000 7f377a7ff700 1 -- start start 2023-11-28T11:22:57.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.389+0000 7f377a7ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37740b8510 0x7f37740ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:57.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.389+0000 7f377a7ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37740cd460 0x7f37740aa890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:57.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.389+0000 7f377a7ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37740d0d60 0x7f37740af5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:57.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.389+0000 7f377a7ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377415bc70 con 0x7f37740d0d60 2023-11-28T11:22:57.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.389+0000 7f377a7ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37740ac3f0 con 0x7f37740b8510 2023-11-28T11:22:57.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.389+0000 7f377a7ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37740abf60 con 0x7f37740cd460 2023-11-28T11:22:57.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.389+0000 7f3778d9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37740d0d60 0x7f37740af5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:57.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.389+0000 7f3778d9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37740d0d60 0x7f37740af5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50778/0 (socket says 172.21.15.92:50778) 2023-11-28T11:22:57.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.389+0000 7f3778d9c700 1 -- 172.21.15.92:0/2775155916 learned_addr learned my addr 172.21.15.92:0/2775155916 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:57.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.389+0000 7f3773fff700 1 --2- 172.21.15.92:0/2775155916 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37740b8510 0x7f37740ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:57.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.389+0000 7f3778d9c700 1 -- 172.21.15.92:0/2775155916 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37740cd460 msgr2=0x7f37740aa890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:57.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.389+0000 7f37737fe700 1 --2- 172.21.15.92:0/2775155916 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37740cd460 0x7f37740aa890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:57.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.389+0000 7f3778d9c700 1 --2- 172.21.15.92:0/2775155916 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37740cd460 0x7f37740aa890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:57.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.389+0000 7f3778d9c700 1 -- 172.21.15.92:0/2775155916 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37740b8510 msgr2=0x7f37740ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:57.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.389+0000 7f3778d9c700 1 --2- 172.21.15.92:0/2775155916 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37740b8510 0x7f37740ae680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:57.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.389+0000 7f3778d9c700 1 -- 172.21.15.92:0/2775155916 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f376c009a70 con 0x7f37740d0d60 2023-11-28T11:22:57.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.389+0000 7f3773fff700 1 --2- 172.21.15.92:0/2775155916 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37740b8510 0x7f37740ae680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:22:57.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.389+0000 7f37737fe700 1 --2- 172.21.15.92:0/2775155916 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37740cd460 0x7f37740aa890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:22:57.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.390+0000 7f3778d9c700 1 --2- 172.21.15.92:0/2775155916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37740d0d60 0x7f37740af5c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f376c014040 tx=0x7f376c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:57.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.390+0000 7f3770ff9700 1 -- 172.21.15.92:0/2775155916 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376c004680 con 0x7f37740d0d60 2023-11-28T11:22:57.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.390+0000 7f3770ff9700 1 -- 172.21.15.92:0/2775155916 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f376c020070 con 0x7f37740d0d60 2023-11-28T11:22:57.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.390+0000 7f3770ff9700 1 -- 172.21.15.92:0/2775155916 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376c027480 con 0x7f37740d0d60 2023-11-28T11:22:57.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.390+0000 7f377a7ff700 1 -- 172.21.15.92:0/2775155916 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37740cfff0 con 0x7f37740d0d60 2023-11-28T11:22:57.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.390+0000 7f377a7ff700 1 -- 172.21.15.92:0/2775155916 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37740cfd10 con 0x7f37740d0d60 2023-11-28T11:22:57.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.391+0000 7f377a7ff700 1 -- 172.21.15.92:0/2775155916 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37740ac110 con 0x7f37740d0d60 2023-11-28T11:22:57.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.392+0000 7f3770ff9700 1 -- 172.21.15.92:0/2775155916 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f376c012430 con 0x7f37740d0d60 2023-11-28T11:22:57.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.393+0000 7f3770ff9700 1 --2- 172.21.15.92:0/2775155916 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f37600682f0 0x7f376006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:57.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.393+0000 7f3770ff9700 1 -- 172.21.15.92:0/2775155916 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f376c022070 con 0x7f37740d0d60 2023-11-28T11:22:57.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.393+0000 7f3773fff700 1 --2- 172.21.15.92:0/2775155916 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f37600682f0 0x7f376006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:57.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.394+0000 7f3773fff700 1 --2- 172.21.15.92:0/2775155916 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f37600682f0 0x7f376006a7a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3764005910 tx=0x7f3764009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:57.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.397+0000 7f3770ff9700 1 -- 172.21.15.92:0/2775155916 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f37740ac110 con 0x7f37740d0d60 2023-11-28T11:22:57.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.589+0000 7f377a7ff700 1 -- 172.21.15.92:0/2775155916 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f37740ac0d0 con 0x7f37740d0d60 2023-11-28T11:22:57.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.591+0000 7f3770ff9700 1 -- 172.21.15.92:0/2775155916 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+18279 (secure 0 0 0) 0x7f376c0547d0 con 0x7f37740d0d60 2023-11-28T11:22:57.592 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:22:57.592 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":44,"fsid":"68956e0c-c16f-47cd-a220-f98364b0831d","created":"2023-11-28T11:20:28.697573+0000","modified":"2023-11-28T11:22:38.531199+0000","last_up_change":"2023-11-28T11:20:41.257011+0000","last_in_change":"2023-11-28T11:20:32.249767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:20:41.925237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:20:48.682497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:22:22.942125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-28T11:22:23.684528+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6abe483f-0e03-4a26-b9cf-62041291747e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":115823},{"type":"v1","addr":"172.21.15.92:6809","nonce":115823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":115823},{"type":"v1","addr":"172.21.15.92:6813","nonce":115823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":115823},{"type":"v1","addr":"172.21.15.92:6821","nonce":115823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":115823},{"type":"v1","addr":"172.21.15.92:6817","nonce":115823}]},"public_addr":"172.21.15.92:6809/115823","cluster_addr":"172.21.15.92:6813/115823","heartbeat_back_addr":"172.21.15.92:6821/115823","heartbeat_front_addr":"172.21.15.92:6817/115823","state":["exists","up"]},{"osd":1,"uuid":"13c1de73-1a8c-4d7e-8a28-9dae52dfe089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":115824},{"type":"v1","addr":"172.21.15.92:6812","nonce":115824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":115824},{"type":"v1","addr":"172.21.15.92:6816","nonce":115824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":115824},{"type":"v1","addr":"172.21.15.92:6823","nonce":115824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":115824},{"type":"v1","addr":"172.21.15.92:6820","nonce":115824}]},"public_addr":"172.21.15.92:6812/115824","cluster_addr":"172.21.15.92:6816/115824","heartbeat_back_addr":"172.21.15.92:6823/115824","heartbeat_front_addr":"172.21.15.92:6820/115824","state":["exists","up"]},{"osd":2,"uuid":"25788a55-112f-4f00-810d-2bc37cf76778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":115821},{"type":"v1","addr":"172.21.15.92:6801","nonce":115821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":115821},{"type":"v1","addr":"172.21.15.92:6803","nonce":115821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":115821},{"type":"v1","addr":"172.21.15.92:6807","nonce":115821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":115821},{"type":"v1","addr":"172.21.15.92:6805","nonce":115821}]},"public_addr":"172.21.15.92:6801/115821","cluster_addr":"172.21.15.92:6803/115821","heartbeat_back_addr":"172.21.15.92:6807/115821","heartbeat_front_addr":"172.21.15.92:6805/115821","state":["exists","up"]},{"osd":3,"uuid":"adf54e24-33a9-41fe-9ce3-98a5ab6c158e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":115825},{"type":"v1","addr":"172.21.15.92:6825","nonce":115825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":115825},{"type":"v1","addr":"172.21.15.92:6827","nonce":115825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":115825},{"type":"v1","addr":"172.21.15.92:6831","nonce":115825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":115825},{"type":"v1","addr":"172.21.15.92:6829","nonce":115825}]},"public_addr":"172.21.15.92:6825/115825","cluster_addr":"172.21.15.92:6827/115825","heartbeat_back_addr":"172.21.15.92:6831/115825","heartbeat_front_addr":"172.21.15.92:6829/115825","state":["exists","up"]},{"osd":4,"uuid":"ccb9b747-f399-44ea-9a18-9db3152227e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":116035},{"type":"v1","addr":"172.21.15.169:6817","nonce":116035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":116035},{"type":"v1","addr":"172.21.15.169:6819","nonce":116035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":116035},{"type":"v1","addr":"172.21.15.169:6826","nonce":116035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":116035},{"type":"v1","addr":"172.21.15.169:6822","nonce":116035}]},"public_addr":"172.21.15.169:6817/116035","cluster_addr":"172.21.15.169:6819/116035","heartbeat_back_addr":"172.21.15.169:6826/116035","heartbeat_front_addr":"172.21.15.169:6822/116035","state":["exists","up"]},{"osd":5,"uuid":"308a1e63-75aa-4d7d-8a7c-cecc24714fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":116033},{"type":"v1","addr":"172.21.15.169:6801","nonce":116033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":116033},{"type":"v1","addr":"172.21.15.169:6803","nonce":116033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":116033},{"type":"v1","addr":"172.21.15.169:6807","nonce":116033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":116033},{"type":"v1","addr":"172.21.15.169:6805","nonce":116033}]},"public_addr":"172.21.15.169:6801/116033","cluster_addr":"172.21.15.169:6803/116033","heartbeat_back_addr":"172.21.15.169:6807/116033","heartbeat_front_addr":"172.21.15.169:6805/116033","state":["exists","up"]},{"osd":6,"uuid":"9c459b7b-b821-4b4b-b91e-6aaae271457c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":116034},{"type":"v1","addr":"172.21.15.169:6809","nonce":116034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":116034},{"type":"v1","addr":"172.21.15.169:6811","nonce":116034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":116034},{"type":"v1","addr":"172.21.15.169:6815","nonce":116034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":116034},{"type":"v1","addr":"172.21.15.169:6813","nonce":116034}]},"public_addr":"172.21.15.169:6809/116034","cluster_addr":"172.21.15.169:6811/116034","heartbeat_back_addr":"172.21.15.169:6815/116034","heartbeat_front_addr":"172.21.15.169:6813/116034","state":["exists","up"]},{"osd":7,"uuid":"1ca9aa4a-755d-4e28-a937-4d8dccab4df3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":116044},{"type":"v1","addr":"172.21.15.169:6823","nonce":116044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6825","nonce":116044},{"type":"v1","addr":"172.21.15.169:6827","nonce":116044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":116044},{"type":"v1","addr":"172.21.15.169:6831","nonce":116044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":116044},{"type":"v1","addr":"172.21.15.169:6829","nonce":116044}]},"public_addr":"172.21.15.169:6823/116044","cluster_addr":"172.21.15.169:6827/116044","heartbeat_back_addr":"172.21.15.169:6831/116044","heartbeat_front_addr":"172.21.15.169:6829/116044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.586781+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.869778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.508594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.860493+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.872267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.746933+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.611520+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.871446+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.f","mappings":[{"from":0,"to":4}]},{"pgid":"7.3e","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:22:57.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.593+0000 7f377a7ff700 1 -- 172.21.15.92:0/2775155916 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f37600682f0 msgr2=0x7f376006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:57.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.593+0000 7f377a7ff700 1 --2- 172.21.15.92:0/2775155916 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f37600682f0 0x7f376006a7a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3764005910 tx=0x7f3764009040 comp rx=0 tx=0).stop 2023-11-28T11:22:57.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.593+0000 7f377a7ff700 1 -- 172.21.15.92:0/2775155916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37740d0d60 msgr2=0x7f37740af5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:57.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.593+0000 7f377a7ff700 1 --2- 172.21.15.92:0/2775155916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37740d0d60 0x7f37740af5c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f376c014040 tx=0x7f376c004ad0 comp rx=0 tx=0).stop 2023-11-28T11:22:57.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.593+0000 7f377a7ff700 1 -- 172.21.15.92:0/2775155916 shutdown_connections 2023-11-28T11:22:57.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.593+0000 7f377a7ff700 1 --2- 172.21.15.92:0/2775155916 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f37600682f0 0x7f376006a7a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:57.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.593+0000 7f377a7ff700 1 --2- 172.21.15.92:0/2775155916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37740d0d60 0x7f37740af5c0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:57.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.593+0000 7f377a7ff700 1 --2- 172.21.15.92:0/2775155916 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37740b8510 0x7f37740ae680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:57.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.593+0000 7f377a7ff700 1 --2- 172.21.15.92:0/2775155916 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37740cd460 0x7f37740aa890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:57.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.593+0000 7f377a7ff700 1 -- 172.21.15.92:0/2775155916 >> 172.21.15.92:0/2775155916 conn(0x7f37740719a0 msgr2=0x7f37740b3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:57.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.593+0000 7f377a7ff700 1 -- 172.21.15.92:0/2775155916 shutdown_connections 2023-11-28T11:22:57.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.593+0000 7f377a7ff700 1 -- 172.21.15.92:0/2775155916 wait complete. 2023-11-28T11:22:57.613 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T11:22:57.613 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:22:57.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.809+0000 7f91099ff700 1 Processor -- start 2023-11-28T11:22:57.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.809+0000 7f91099ff700 1 -- start start 2023-11-28T11:22:57.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.810+0000 7f91099ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f91040b8510 0x7f910406b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:57.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.810+0000 7f91099ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f91040cd460 0x7f910406beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:57.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.810+0000 7f91099ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f910406c3f0 0x7f910415cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:57.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.810+0000 7f91099ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91040cdbd0 con 0x7f91040cd460 2023-11-28T11:22:57.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.810+0000 7f91099ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91040d02d0 con 0x7f91040b8510 2023-11-28T11:22:57.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.810+0000 7f91099ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91040b88e0 con 0x7f910406c3f0 2023-11-28T11:22:57.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.810+0000 7f91027fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f91040cd460 0x7f910406beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:57.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.811+0000 7f91027fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f91040cd460 0x7f910406beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50782/0 (socket says 172.21.15.92:50782) 2023-11-28T11:22:57.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.811+0000 7f91027fc700 1 -- 172.21.15.92:0/1334274495 learned_addr learned my addr 172.21.15.92:0/1334274495 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:57.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.811+0000 7f91037fe700 1 --2- 172.21.15.92:0/1334274495 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f910406c3f0 0x7f910415cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:57.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.811+0000 7f9102ffd700 1 --2- 172.21.15.92:0/1334274495 >> v2:172.21.15.169:3300/0 conn(0x7f91040b8510 0x7f910406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:57.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.811+0000 7f91027fc700 1 -- 172.21.15.92:0/1334274495 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f910406c3f0 msgr2=0x7f910415cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:57.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.811+0000 7f91027fc700 1 --2- 172.21.15.92:0/1334274495 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f910406c3f0 0x7f910415cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:57.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.811+0000 7f91027fc700 1 -- 172.21.15.92:0/1334274495 >> v2:172.21.15.169:3300/0 conn(0x7f91040b8510 msgr2=0x7f910406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:57.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.811+0000 7f91027fc700 1 --2- 172.21.15.92:0/1334274495 >> v2:172.21.15.169:3300/0 conn(0x7f91040b8510 0x7f910406b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:57.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.811+0000 7f91027fc700 1 -- 172.21.15.92:0/1334274495 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91040cd8f0 con 0x7f91040cd460 2023-11-28T11:22:57.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.811+0000 7f91027fc700 1 --2- 172.21.15.92:0/1334274495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91040cd460 0x7f910406beb0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f90f8009920 tx=0x7f90f8009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf577bafb397f466 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:57.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.811+0000 7f9101ffb700 1 -- 172.21.15.92:0/1334274495 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f800b9e0 con 0x7f91040cd460 2023-11-28T11:22:57.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.811+0000 7f9101ffb700 1 -- 172.21.15.92:0/1334274495 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90f800bb40 con 0x7f91040cd460 2023-11-28T11:22:57.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.811+0000 7f9101ffb700 1 -- 172.21.15.92:0/1334274495 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f8009e70 con 0x7f91040cd460 2023-11-28T11:22:57.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.812+0000 7f91099ff700 1 -- 172.21.15.92:0/1334274495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91040cd460 msgr2=0x7f910406beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:57.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.812+0000 7f91099ff700 1 --2- 172.21.15.92:0/1334274495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91040cd460 0x7f910406beb0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f90f8009920 tx=0x7f90f8009c30 comp rx=0 tx=0).stop 2023-11-28T11:22:57.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.812+0000 7f91099ff700 1 -- 172.21.15.92:0/1334274495 shutdown_connections 2023-11-28T11:22:57.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.812+0000 7f91099ff700 1 --2- 172.21.15.92:0/1334274495 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f910406c3f0 0x7f910415cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:57.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.812+0000 7f91099ff700 1 --2- 172.21.15.92:0/1334274495 >> v2:172.21.15.169:3300/0 conn(0x7f91040b8510 0x7f910406b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:57.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.812+0000 7f91099ff700 1 --2- 172.21.15.92:0/1334274495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91040cd460 0x7f910406beb0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:57.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.812+0000 7f91099ff700 1 -- 172.21.15.92:0/1334274495 >> 172.21.15.92:0/1334274495 conn(0x7f91040719a0 msgr2=0x7f9104071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:57.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.812+0000 7f91099ff700 1 -- 172.21.15.92:0/1334274495 shutdown_connections 2023-11-28T11:22:57.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.812+0000 7f91099ff700 1 -- 172.21.15.92:0/1334274495 wait complete. 2023-11-28T11:22:57.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.814+0000 7f91099ff700 1 Processor -- start 2023-11-28T11:22:57.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.815+0000 7f91099ff700 1 -- start start 2023-11-28T11:22:57.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.816+0000 7f91099ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f910406c3f0 0x7f9104156690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:57.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.816+0000 7f91099ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91040b8510 0x7f91040b6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:57.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.816+0000 7f91099ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91040cd460 0x7f91040b6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:57.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.816+0000 7f91099ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9104157fb0 con 0x7f910406c3f0 2023-11-28T11:22:57.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.816+0000 7f91099ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9104156f10 con 0x7f91040cd460 2023-11-28T11:22:57.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.816+0000 7f91099ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91041571f0 con 0x7f91040b8510 2023-11-28T11:22:57.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.816+0000 7f9102ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f910406c3f0 0x7f9104156690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:57.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.816+0000 7f9102ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f910406c3f0 0x7f9104156690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50796/0 (socket says 172.21.15.92:50796) 2023-11-28T11:22:57.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.816+0000 7f9102ffd700 1 -- 172.21.15.92:0/766674037 learned_addr learned my addr 172.21.15.92:0/766674037 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:57.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.816+0000 7f91027fc700 1 --2- 172.21.15.92:0/766674037 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91040b8510 0x7f91040b6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:57.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.816+0000 7f91037fe700 1 --2- 172.21.15.92:0/766674037 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91040cd460 0x7f91040b6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:57.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.817+0000 7f9102ffd700 1 -- 172.21.15.92:0/766674037 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91040b8510 msgr2=0x7f91040b6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:57.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.817+0000 7f9102ffd700 1 --2- 172.21.15.92:0/766674037 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91040b8510 0x7f91040b6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:57.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.817+0000 7f9102ffd700 1 -- 172.21.15.92:0/766674037 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91040cd460 msgr2=0x7f91040b6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:57.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.817+0000 7f9102ffd700 1 --2- 172.21.15.92:0/766674037 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91040cd460 0x7f91040b6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:57.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.817+0000 7f9102ffd700 1 -- 172.21.15.92:0/766674037 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f8009550 con 0x7f910406c3f0 2023-11-28T11:22:57.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.817+0000 7f91027fc700 1 --2- 172.21.15.92:0/766674037 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91040b8510 0x7f91040b6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:22:57.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.817+0000 7f9102ffd700 1 --2- 172.21.15.92:0/766674037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f910406c3f0 0x7f9104156690 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f90f400e810 tx=0x7f90f400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:57.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.817+0000 7f90ebfff700 1 -- 172.21.15.92:0/766674037 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f4013610 con 0x7f910406c3f0 2023-11-28T11:22:57.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.817+0000 7f90ebfff700 1 -- 172.21.15.92:0/766674037 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90f4013770 con 0x7f910406c3f0 2023-11-28T11:22:57.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.817+0000 7f91099ff700 1 -- 172.21.15.92:0/766674037 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91040cfff0 con 0x7f910406c3f0 2023-11-28T11:22:57.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.817+0000 7f90ebfff700 1 -- 172.21.15.92:0/766674037 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f4013b20 con 0x7f910406c3f0 2023-11-28T11:22:57.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.817+0000 7f91099ff700 1 -- 172.21.15.92:0/766674037 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91040cfd10 con 0x7f910406c3f0 2023-11-28T11:22:57.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.818+0000 7f91099ff700 1 -- 172.21.15.92:0/766674037 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9104067880 con 0x7f910406c3f0 2023-11-28T11:22:57.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.819+0000 7f90ebfff700 1 -- 172.21.15.92:0/766674037 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f90f4023070 con 0x7f910406c3f0 2023-11-28T11:22:57.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.820+0000 7f90ebfff700 1 --2- 172.21.15.92:0/766674037 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f90f0068320 0x7f90f006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:57.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.820+0000 7f90ebfff700 1 -- 172.21.15.92:0/766674037 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f90f4085c20 con 0x7f910406c3f0 2023-11-28T11:22:57.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.820+0000 7f91027fc700 1 --2- 172.21.15.92:0/766674037 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f90f0068320 0x7f90f006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:57.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.821+0000 7f91027fc700 1 --2- 172.21.15.92:0/766674037 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f90f0068320 0x7f90f006a7d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f91040b1de0 tx=0x7f90f8008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:57.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:57.824+0000 7f90ebfff700 1 -- 172.21.15.92:0/766674037 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f90f4052190 con 0x7f910406c3f0 2023-11-28T11:22:58.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.057+0000 7f91099ff700 1 -- 172.21.15.92:0/766674037 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f91040d2940 con 0x7f910406c3f0 2023-11-28T11:22:58.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.057+0000 7f90ebfff700 1 -- 172.21.15.92:0/766674037 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4645 (secure 0 0 0) 0x7f90f40557b0 con 0x7f910406c3f0 2023-11-28T11:22:58.059 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:22:58.059 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4976,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/3102603064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3102603064},{"type":"v1","addr":"172.21.15.169:6834","nonce":3102603064}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4979,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/79247121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":79247121},{"type":"v1","addr":"172.21.15.92:6837","nonce":79247121}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5221,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/219797192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":219797192},{"type":"v1","addr":"172.21.15.169:6835","nonce":219797192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:22:24.715447+0000","modified":"2023-11-28T11:22:27.733117+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5298},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5298":{"gid":5298,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.92:6835/1577294649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1577294649},{"type":"v1","addr":"172.21.15.92:6835","nonce":1577294649}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T11:22:58.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.060+0000 7f91099ff700 1 -- 172.21.15.92:0/766674037 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f90f0068320 msgr2=0x7f90f006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:58.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.060+0000 7f91099ff700 1 --2- 172.21.15.92:0/766674037 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f90f0068320 0x7f90f006a7d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f91040b1de0 tx=0x7f90f8008030 comp rx=0 tx=0).stop 2023-11-28T11:22:58.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.060+0000 7f91099ff700 1 -- 172.21.15.92:0/766674037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f910406c3f0 msgr2=0x7f9104156690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:58.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.060+0000 7f91099ff700 1 --2- 172.21.15.92:0/766674037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f910406c3f0 0x7f9104156690 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f90f400e810 tx=0x7f90f400ebd0 comp rx=0 tx=0).stop 2023-11-28T11:22:58.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.060+0000 7f91099ff700 1 -- 172.21.15.92:0/766674037 shutdown_connections 2023-11-28T11:22:58.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.060+0000 7f91099ff700 1 --2- 172.21.15.92:0/766674037 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91040cd460 0x7f91040b6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:58.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.060+0000 7f91099ff700 1 --2- 172.21.15.92:0/766674037 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f90f0068320 0x7f90f006a7d0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:58.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.060+0000 7f91099ff700 1 --2- 172.21.15.92:0/766674037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f910406c3f0 0x7f9104156690 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:58.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.060+0000 7f91099ff700 1 --2- 172.21.15.92:0/766674037 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91040b8510 0x7f91040b6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:58.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.060+0000 7f91099ff700 1 -- 172.21.15.92:0/766674037 >> 172.21.15.92:0/766674037 conn(0x7f91040719a0 msgr2=0x7f91040b4070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:58.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.060+0000 7f91099ff700 1 -- 172.21.15.92:0/766674037 shutdown_connections 2023-11-28T11:22:58.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.060+0000 7f91099ff700 1 -- 172.21.15.92:0/766674037 wait complete. 2023-11-28T11:22:58.062 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 17 2023-11-28T11:22:58.080 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:22:58.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.275+0000 7f18135ff700 1 Processor -- start 2023-11-28T11:22:58.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.276+0000 7f18135ff700 1 -- start start 2023-11-28T11:22:58.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.277+0000 7f18135ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f180c0d1d70 0x7f180c167270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:58.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.277+0000 7f18135ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f180c06c790 con 0x7f180c0b8510 2023-11-28T11:22:58.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.277+0000 7f18135ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f180c0b88e0 con 0x7f180c0cd460 2023-11-28T11:22:58.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.277+0000 7f18135ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f180c0cdbd0 con 0x7f180c0d1d70 2023-11-28T11:22:58.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.277+0000 7f181139b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f180c0b8510 0x7f180c06b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51124/0 (socket says 172.21.15.92:51124) 2023-11-28T11:22:58.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.277+0000 7f181139b700 1 -- 172.21.15.92:0/3943830720 learned_addr learned my addr 172.21.15.92:0/3943830720 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:22:58.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.277+0000 7f1803fff700 1 -- 172.21.15.92:0/3943830720 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1668752175 0 0) 0x7f180c06c790 con 0x7f180c0b8510 2023-11-28T11:22:58.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.277+0000 7f1811b9c700 1 --2- 172.21.15.92:0/3943830720 >> v2:172.21.15.169:3300/0 conn(0x7f180c0d1d70 0x7f180c167270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:58.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.277+0000 7f1803fff700 1 -- 172.21.15.92:0/3943830720 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17f80035b0 con 0x7f180c0b8510 2023-11-28T11:22:58.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.278+0000 7f1803fff700 1 -- 172.21.15.92:0/3943830720 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 873869985 0 0) 0x7f17f80035b0 con 0x7f180c0b8510 2023-11-28T11:22:58.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.278+0000 7f1803fff700 1 -- 172.21.15.92:0/3943830720 >> v2:172.21.15.169:3300/0 conn(0x7f180c0d1d70 msgr2=0x7f180c167270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:58.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.278+0000 7f1803fff700 1 --2- 172.21.15.92:0/3943830720 >> v2:172.21.15.169:3300/0 conn(0x7f180c0d1d70 0x7f180c167270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:58.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.278+0000 7f1803fff700 1 -- 172.21.15.92:0/3943830720 >> v1:172.21.15.169:6789/0 conn(0x7f180c0cd460 legacy=0x7f180c06c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:58.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.278+0000 7f1803fff700 1 -- 172.21.15.92:0/3943830720 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f180c0cd8f0 con 0x7f180c0b8510 2023-11-28T11:22:58.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.278+0000 7f1803fff700 1 -- 172.21.15.92:0/3943830720 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532161587 0 0) 0x7f1808001670 con 0x7f180c0b8510 2023-11-28T11:22:58.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.278+0000 7f1803fff700 1 -- 172.21.15.92:0/3943830720 >> v1:172.21.15.92:6789/0 conn(0x7f180c0b8510 legacy=0x7f180c06b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:58.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.278+0000 7f1803fff700 1 --2- 172.21.15.92:0/3943830720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17f8003a50 0x7f17f8003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:58.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.278+0000 7f1803fff700 1 --2- 172.21.15.92:0/3943830720 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17f8004850 0x7f17f8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:58.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.278+0000 7f1803fff700 1 --2- 172.21.15.92:0/3943830720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17f8005200 0x7f17f800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:58.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.278+0000 7f1803fff700 1 -- 172.21.15.92:0/3943830720 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180c0cdbd0 con 0x7f17f8003a50 2023-11-28T11:22:58.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.278+0000 7f1803fff700 1 -- 172.21.15.92:0/3943830720 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f800c0e0 con 0x7f17f8005200 2023-11-28T11:22:58.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.278+0000 7f1803fff700 1 -- 172.21.15.92:0/3943830720 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17f800c2a0 con 0x7f17f8004850 2023-11-28T11:22:58.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.278+0000 7f181139b700 1 --2- 172.21.15.92:0/3943830720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17f8003a50 0x7f17f8003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:58.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.278+0000 7f1811b9c700 1 --2- 172.21.15.92:0/3943830720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17f8005200 0x7f17f800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:58.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.278+0000 7f1810b9a700 1 --2- 172.21.15.92:0/3943830720 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17f8004850 0x7f17f8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:58.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.278+0000 7f181139b700 1 -- 172.21.15.92:0/3943830720 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17f8004850 msgr2=0x7f17f8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:58.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.278+0000 7f181139b700 1 --2- 172.21.15.92:0/3943830720 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17f8004850 0x7f17f8004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:58.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.278+0000 7f181139b700 1 -- 172.21.15.92:0/3943830720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17f8005200 msgr2=0x7f17f800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:22:58.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.278+0000 7f181139b700 1 --2- 172.21.15.92:0/3943830720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f17f8005200 0x7f17f800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:58.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.279+0000 7f181139b700 1 -- 172.21.15.92:0/3943830720 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17f80035b0 con 0x7f17f8003a50 2023-11-28T11:22:58.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.279+0000 7f1811b9c700 1 -- 172.21.15.92:0/3943830720 reap_dead start 2023-11-28T11:22:58.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.279+0000 7f181139b700 1 --2- 172.21.15.92:0/3943830720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17f8003a50 0x7f17f8003e60 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f1808001eb0 tx=0x7f17f8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=983ae9e226b9f15 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:58.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.279+0000 7f1803fff700 1 -- 172.21.15.92:0/3943830720 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1808007b70 con 0x7f17f8003a50 2023-11-28T11:22:58.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.279+0000 7f1803fff700 1 -- 172.21.15.92:0/3943830720 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f180800cc50 con 0x7f17f8003a50 2023-11-28T11:22:58.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.279+0000 7f1803fff700 1 -- 172.21.15.92:0/3943830720 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1808005960 con 0x7f17f8003a50 2023-11-28T11:22:58.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.279+0000 7f18135ff700 1 -- 172.21.15.92:0/3943830720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17f8003a50 msgr2=0x7f17f8003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:58.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.279+0000 7f18135ff700 1 --2- 172.21.15.92:0/3943830720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17f8003a50 0x7f17f8003e60 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f1808001eb0 tx=0x7f17f8005880 comp rx=0 tx=0).stop 2023-11-28T11:22:58.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.280+0000 7f18135ff700 1 -- 172.21.15.92:0/3943830720 shutdown_connections 2023-11-28T11:22:58.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.280+0000 7f18135ff700 1 --2- 172.21.15.92:0/3943830720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17f8003a50 0x7f17f8003e60 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:58.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.280+0000 7f18135ff700 1 -- 172.21.15.92:0/3943830720 >> 172.21.15.92:0/3943830720 conn(0x7f180c0719a0 msgr2=0x7f180c071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:58.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.280+0000 7f18135ff700 1 -- 172.21.15.92:0/3943830720 shutdown_connections 2023-11-28T11:22:58.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.280+0000 7f18135ff700 1 -- 172.21.15.92:0/3943830720 wait complete. 2023-11-28T11:22:58.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.281+0000 7f18135ff700 1 Processor -- start 2023-11-28T11:22:58.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.281+0000 7f18135ff700 1 -- start start 2023-11-28T11:22:58.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.282+0000 7f18135ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17f8003a50 0x7f180c1f20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:58.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.282+0000 7f18135ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f180c14d250 0x7f180c0b6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:58.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.282+0000 7f18135ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f180c0b6860 0x7f180c159210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:58.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.282+0000 7f18135ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180c0b6cd0 con 0x7f180c14d250 2023-11-28T11:22:58.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.282+0000 7f18135ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180c157420 con 0x7f180c0b6860 2023-11-28T11:22:58.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.282+0000 7f18135ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f180c156810 con 0x7f17f8003a50 2023-11-28T11:22:58.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.283+0000 7f1810b9a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f180c14d250 0x7f180c0b6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:58.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.283+0000 7f1810b9a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f180c14d250 0x7f180c0b6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50802/0 (socket says 172.21.15.92:50802) 2023-11-28T11:22:58.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.283+0000 7f1810b9a700 1 -- 172.21.15.92:0/3342797376 learned_addr learned my addr 172.21.15.92:0/3342797376 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:58.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.283+0000 7f181139b700 1 --2- 172.21.15.92:0/3342797376 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17f8003a50 0x7f180c1f20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:58.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.283+0000 7f1811b9c700 1 --2- 172.21.15.92:0/3342797376 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f180c0b6860 0x7f180c159210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:58.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.283+0000 7f1810b9a700 1 -- 172.21.15.92:0/3342797376 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17f8003a50 msgr2=0x7f180c1f20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:58.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.283+0000 7f1810b9a700 1 --2- 172.21.15.92:0/3342797376 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17f8003a50 0x7f180c1f20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:58.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.283+0000 7f1810b9a700 1 -- 172.21.15.92:0/3342797376 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f180c0b6860 msgr2=0x7f180c159210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:58.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.283+0000 7f1810b9a700 1 --2- 172.21.15.92:0/3342797376 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f180c0b6860 0x7f180c159210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:58.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.283+0000 7f1810b9a700 1 -- 172.21.15.92:0/3342797376 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f180c0cfff0 con 0x7f180c14d250 2023-11-28T11:22:58.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.283+0000 7f1810b9a700 1 --2- 172.21.15.92:0/3342797376 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f180c14d250 0x7f180c0b6320 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f17fc000c00 tx=0x7f17fc00e740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:58.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.283+0000 7f1801ffb700 1 -- 172.21.15.92:0/3342797376 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17fc00ee80 con 0x7f180c14d250 2023-11-28T11:22:58.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.284+0000 7f1801ffb700 1 -- 172.21.15.92:0/3342797376 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17fc0052e0 con 0x7f180c14d250 2023-11-28T11:22:58.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.284+0000 7f18135ff700 1 -- 172.21.15.92:0/3342797376 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f180c0cfd10 con 0x7f180c14d250 2023-11-28T11:22:58.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.284+0000 7f1801ffb700 1 -- 172.21.15.92:0/3342797376 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17fc003680 con 0x7f180c14d250 2023-11-28T11:22:58.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.284+0000 7f18135ff700 1 -- 172.21.15.92:0/3342797376 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f180c0d29c0 con 0x7f180c14d250 2023-11-28T11:22:58.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.285+0000 7f18135ff700 1 -- 172.21.15.92:0/3342797376 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f180c067880 con 0x7f180c14d250 2023-11-28T11:22:58.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.286+0000 7f1801ffb700 1 -- 172.21.15.92:0/3342797376 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f17fc01b030 con 0x7f180c14d250 2023-11-28T11:22:58.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.286+0000 7f1801ffb700 1 --2- 172.21.15.92:0/3342797376 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f17f00682f0 0x7f17f006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:58.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.287+0000 7f1801ffb700 1 -- 172.21.15.92:0/3342797376 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f17fc085150 con 0x7f180c14d250 2023-11-28T11:22:58.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.287+0000 7f181139b700 1 --2- 172.21.15.92:0/3342797376 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f17f00682f0 0x7f17f006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:58.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.287+0000 7f181139b700 1 --2- 172.21.15.92:0/3342797376 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f17f00682f0 0x7f17f006a7a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1808000c00 tx=0x7f1808007000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:58.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.290+0000 7f1801ffb700 1 -- 172.21.15.92:0/3342797376 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f17fc04d5e0 con 0x7f180c14d250 2023-11-28T11:22:58.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.522+0000 7f18135ff700 1 -- 172.21.15.92:0/3342797376 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f180c157560 con 0x7f180c14d250 2023-11-28T11:22:58.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.523+0000 7f1801ffb700 1 -- 172.21.15.92:0/3342797376 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4645 (secure 0 0 0) 0x7f17fc0518e0 con 0x7f180c14d250 2023-11-28T11:22:58.525 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:22:58.525 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4976,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/3102603064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3102603064},{"type":"v1","addr":"172.21.15.169:6834","nonce":3102603064}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4979,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/79247121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":79247121},{"type":"v1","addr":"172.21.15.92:6837","nonce":79247121}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5221,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/219797192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":219797192},{"type":"v1","addr":"172.21.15.169:6835","nonce":219797192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:22:24.715447+0000","modified":"2023-11-28T11:22:27.733117+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5298},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5298":{"gid":5298,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.92:6835/1577294649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1577294649},{"type":"v1","addr":"172.21.15.92:6835","nonce":1577294649}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T11:22:58.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.525+0000 7f18135ff700 1 -- 172.21.15.92:0/3342797376 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f17f00682f0 msgr2=0x7f17f006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:58.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.525+0000 7f18135ff700 1 --2- 172.21.15.92:0/3342797376 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f17f00682f0 0x7f17f006a7a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1808000c00 tx=0x7f1808007000 comp rx=0 tx=0).stop 2023-11-28T11:22:58.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.525+0000 7f18135ff700 1 -- 172.21.15.92:0/3342797376 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f180c14d250 msgr2=0x7f180c0b6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:58.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.525+0000 7f18135ff700 1 --2- 172.21.15.92:0/3342797376 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f180c14d250 0x7f180c0b6320 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f17fc000c00 tx=0x7f17fc00e740 comp rx=0 tx=0).stop 2023-11-28T11:22:58.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.525+0000 7f18135ff700 1 -- 172.21.15.92:0/3342797376 shutdown_connections 2023-11-28T11:22:58.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.525+0000 7f18135ff700 1 --2- 172.21.15.92:0/3342797376 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f180c0b6860 0x7f180c159210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:58.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.525+0000 7f18135ff700 1 --2- 172.21.15.92:0/3342797376 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f17f8003a50 0x7f180c1f20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:58.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.525+0000 7f18135ff700 1 --2- 172.21.15.92:0/3342797376 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f17f00682f0 0x7f17f006a7a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:58.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.525+0000 7f18135ff700 1 --2- 172.21.15.92:0/3342797376 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f180c14d250 0x7f180c0b6320 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:58.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.525+0000 7f18135ff700 1 -- 172.21.15.92:0/3342797376 >> 172.21.15.92:0/3342797376 conn(0x7f180c0719a0 msgr2=0x7f180c0c9670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:58.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.525+0000 7f18135ff700 1 -- 172.21.15.92:0/3342797376 shutdown_connections 2023-11-28T11:22:58.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.525+0000 7f18135ff700 1 -- 172.21.15.92:0/3342797376 wait complete. 2023-11-28T11:22:58.528 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 17 2023-11-28T11:22:58.546 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:22:58.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.743+0000 7f6e87dff700 1 Processor -- start 2023-11-28T11:22:58.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.743+0000 7f6e87dff700 1 -- start start 2023-11-28T11:22:58.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.744+0000 7f6e87dff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e800cd460 0x7f6e8006c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:58.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.744+0000 7f6e87dff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e800b88e0 con 0x7f6e800d0d60 2023-11-28T11:22:58.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.744+0000 7f6e87dff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e800cd8f0 con 0x7f6e800b8510 2023-11-28T11:22:58.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.744+0000 7f6e87dff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e800cdbd0 con 0x7f6e800cd460 2023-11-28T11:22:58.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.744+0000 7f6e8639c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6e800d0d60 0x7f6e800d11e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51126/0 (socket says 172.21.15.92:51126) 2023-11-28T11:22:58.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.744+0000 7f6e8639c700 1 -- 172.21.15.92:0/3825158716 learned_addr learned my addr 172.21.15.92:0/3825158716 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:22:58.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.744+0000 7f6e8539a700 1 --2- 172.21.15.92:0/3825158716 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e800cd460 0x7f6e8006c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:58.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.745+0000 7f6e84b99700 1 -- 172.21.15.92:0/3825158716 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3222319757 0 0) 0x7f6e800b88e0 con 0x7f6e800d0d60 2023-11-28T11:22:58.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.745+0000 7f6e84b99700 1 -- 172.21.15.92:0/3825158716 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e6c0035b0 con 0x7f6e800d0d60 2023-11-28T11:22:58.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.745+0000 7f6e84b99700 1 -- 172.21.15.92:0/3825158716 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2089834843 0 0) 0x7f6e800cd8f0 con 0x7f6e800b8510 2023-11-28T11:22:58.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.745+0000 7f6e84b99700 1 -- 172.21.15.92:0/3825158716 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e800b88e0 con 0x7f6e800b8510 2023-11-28T11:22:58.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.745+0000 7f6e8539a700 1 -- 172.21.15.92:0/3825158716 >> v1:172.21.15.169:6789/0 conn(0x7f6e800b8510 legacy=0x7f6e8006b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:58.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.745+0000 7f6e8539a700 1 -- 172.21.15.92:0/3825158716 >> v1:172.21.15.92:6789/0 conn(0x7f6e800d0d60 legacy=0x7f6e800d11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:58.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.745+0000 7f6e84b99700 1 -- 172.21.15.92:0/3825158716 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 266044468 0 0) 0x7f6e6c0035b0 con 0x7f6e800d0d60 2023-11-28T11:22:58.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.745+0000 7f6e8539a700 1 -- 172.21.15.92:0/3825158716 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e800cfd10 con 0x7f6e800cd460 2023-11-28T11:22:58.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.745+0000 7f6e8539a700 1 --2- 172.21.15.92:0/3825158716 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e800cd460 0x7f6e8006c080 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6e70009870 tx=0x7f6e70009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee302be98068f474 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:58.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.746+0000 7f6e84b99700 1 -- 172.21.15.92:0/3825158716 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e780019f0 con 0x7f6e800cd460 2023-11-28T11:22:58.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.746+0000 7f6e84b99700 1 -- 172.21.15.92:0/3825158716 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e70004030 con 0x7f6e800cd460 2023-11-28T11:22:58.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.746+0000 7f6e84b99700 1 -- 172.21.15.92:0/3825158716 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e700043c0 con 0x7f6e800cd460 2023-11-28T11:22:58.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.746+0000 7f6e87dff700 1 -- 172.21.15.92:0/3825158716 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e800cd460 msgr2=0x7f6e8006c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:58.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.746+0000 7f6e87dff700 1 --2- 172.21.15.92:0/3825158716 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e800cd460 0x7f6e8006c080 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6e70009870 tx=0x7f6e70009b80 comp rx=0 tx=0).stop 2023-11-28T11:22:58.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.746+0000 7f6e87dff700 1 -- 172.21.15.92:0/3825158716 shutdown_connections 2023-11-28T11:22:58.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.746+0000 7f6e87dff700 1 --2- 172.21.15.92:0/3825158716 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e800cd460 0x7f6e8006c080 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:58.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.746+0000 7f6e87dff700 1 -- 172.21.15.92:0/3825158716 >> 172.21.15.92:0/3825158716 conn(0x7f6e800719a0 msgr2=0x7f6e80071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:58.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.746+0000 7f6e87dff700 1 -- 172.21.15.92:0/3825158716 shutdown_connections 2023-11-28T11:22:58.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.746+0000 7f6e87dff700 1 -- 172.21.15.92:0/3825158716 wait complete. 2023-11-28T11:22:58.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.748+0000 7f6e87dff700 1 Processor -- start 2023-11-28T11:22:58.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.748+0000 7f6e87dff700 1 -- start start 2023-11-28T11:22:58.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.749+0000 7f6e87dff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e800b8510 0x7f6e80147050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:58.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.749+0000 7f6e87dff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e800cd460 0x7f6e80159010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:58.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.749+0000 7f6e87dff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e800d0d60 0x7f6e800b27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:58.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.749+0000 7f6e87dff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8015a440 con 0x7f6e800cd460 2023-11-28T11:22:58.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.749+0000 7f6e87dff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e801f1be0 con 0x7f6e800b8510 2023-11-28T11:22:58.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.749+0000 7f6e87dff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e801598c0 con 0x7f6e800d0d60 2023-11-28T11:22:58.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.749+0000 7f6e8539a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e800cd460 0x7f6e80159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:58.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.749+0000 7f6e8539a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e800cd460 0x7f6e80159010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50806/0 (socket says 172.21.15.92:50806) 2023-11-28T11:22:58.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.749+0000 7f6e8539a700 1 -- 172.21.15.92:0/2111096121 learned_addr learned my addr 172.21.15.92:0/2111096121 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:58.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.750+0000 7f6e8639c700 1 --2- 172.21.15.92:0/2111096121 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e800d0d60 0x7f6e800b27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:58.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.750+0000 7f6e85b9b700 1 --2- 172.21.15.92:0/2111096121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e800b8510 0x7f6e80147050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:58.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.750+0000 7f6e8539a700 1 -- 172.21.15.92:0/2111096121 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e800d0d60 msgr2=0x7f6e800b27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:58.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.750+0000 7f6e8539a700 1 --2- 172.21.15.92:0/2111096121 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e800d0d60 0x7f6e800b27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:58.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.750+0000 7f6e8539a700 1 -- 172.21.15.92:0/2111096121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e800b8510 msgr2=0x7f6e80147050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:58.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.750+0000 7f6e8539a700 1 --2- 172.21.15.92:0/2111096121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e800b8510 0x7f6e80147050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:58.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.750+0000 7f6e8539a700 1 -- 172.21.15.92:0/2111096121 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e700094a0 con 0x7f6e800cd460 2023-11-28T11:22:58.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.750+0000 7f6e8639c700 1 --2- 172.21.15.92:0/2111096121 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e800d0d60 0x7f6e800b27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:22:58.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.750+0000 7f6e85b9b700 1 --2- 172.21.15.92:0/2111096121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e800b8510 0x7f6e80147050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:22:58.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.750+0000 7f6e8539a700 1 --2- 172.21.15.92:0/2111096121 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e800cd460 0x7f6e80159010 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f6e70000c00 tx=0x7f6e70004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:58.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.750+0000 7f6e767fc700 1 -- 172.21.15.92:0/2111096121 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e70011630 con 0x7f6e800cd460 2023-11-28T11:22:58.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.750+0000 7f6e767fc700 1 -- 172.21.15.92:0/2111096121 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e7001f070 con 0x7f6e800cd460 2023-11-28T11:22:58.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.750+0000 7f6e767fc700 1 -- 172.21.15.92:0/2111096121 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e700119f0 con 0x7f6e800cd460 2023-11-28T11:22:58.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.750+0000 7f6e87dff700 1 -- 172.21.15.92:0/2111096121 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e800d29c0 con 0x7f6e800cd460 2023-11-28T11:22:58.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.750+0000 7f6e87dff700 1 -- 172.21.15.92:0/2111096121 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e800cfff0 con 0x7f6e800cd460 2023-11-28T11:22:58.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.751+0000 7f6e87dff700 1 -- 172.21.15.92:0/2111096121 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e800d2b50 con 0x7f6e800cd460 2023-11-28T11:22:58.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.752+0000 7f6e767fc700 1 -- 172.21.15.92:0/2111096121 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6e70017020 con 0x7f6e800cd460 2023-11-28T11:22:58.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.753+0000 7f6e767fc700 1 --2- 172.21.15.92:0/2111096121 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6e640682f0 0x7f6e6406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:58.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.753+0000 7f6e767fc700 1 -- 172.21.15.92:0/2111096121 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f6e70021070 con 0x7f6e800cd460 2023-11-28T11:22:58.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.753+0000 7f6e85b9b700 1 --2- 172.21.15.92:0/2111096121 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6e640682f0 0x7f6e6406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:58.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.754+0000 7f6e85b9b700 1 --2- 172.21.15.92:0/2111096121 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6e640682f0 0x7f6e6406a7a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c0011e0 tx=0x7f6e7c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:58.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.758+0000 7f6e767fc700 1 -- 172.21.15.92:0/2111096121 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6e70053e40 con 0x7f6e800cd460 2023-11-28T11:22:58.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.952+0000 7f6e87dff700 1 -- 172.21.15.92:0/2111096121 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6e80067880 con 0x7f6e800cd460 2023-11-28T11:22:58.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.953+0000 7f6e767fc700 1 -- 172.21.15.92:0/2111096121 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+18279 (secure 0 0 0) 0x7f6e70057460 con 0x7f6e800cd460 2023-11-28T11:22:58.955 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:22:58.955 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":44,"fsid":"68956e0c-c16f-47cd-a220-f98364b0831d","created":"2023-11-28T11:20:28.697573+0000","modified":"2023-11-28T11:22:38.531199+0000","last_up_change":"2023-11-28T11:20:41.257011+0000","last_in_change":"2023-11-28T11:20:32.249767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:20:41.925237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:20:48.682497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:22:22.942125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-28T11:22:23.684528+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6abe483f-0e03-4a26-b9cf-62041291747e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":115823},{"type":"v1","addr":"172.21.15.92:6809","nonce":115823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":115823},{"type":"v1","addr":"172.21.15.92:6813","nonce":115823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":115823},{"type":"v1","addr":"172.21.15.92:6821","nonce":115823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":115823},{"type":"v1","addr":"172.21.15.92:6817","nonce":115823}]},"public_addr":"172.21.15.92:6809/115823","cluster_addr":"172.21.15.92:6813/115823","heartbeat_back_addr":"172.21.15.92:6821/115823","heartbeat_front_addr":"172.21.15.92:6817/115823","state":["exists","up"]},{"osd":1,"uuid":"13c1de73-1a8c-4d7e-8a28-9dae52dfe089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":115824},{"type":"v1","addr":"172.21.15.92:6812","nonce":115824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":115824},{"type":"v1","addr":"172.21.15.92:6816","nonce":115824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":115824},{"type":"v1","addr":"172.21.15.92:6823","nonce":115824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":115824},{"type":"v1","addr":"172.21.15.92:6820","nonce":115824}]},"public_addr":"172.21.15.92:6812/115824","cluster_addr":"172.21.15.92:6816/115824","heartbeat_back_addr":"172.21.15.92:6823/115824","heartbeat_front_addr":"172.21.15.92:6820/115824","state":["exists","up"]},{"osd":2,"uuid":"25788a55-112f-4f00-810d-2bc37cf76778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":115821},{"type":"v1","addr":"172.21.15.92:6801","nonce":115821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":115821},{"type":"v1","addr":"172.21.15.92:6803","nonce":115821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":115821},{"type":"v1","addr":"172.21.15.92:6807","nonce":115821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":115821},{"type":"v1","addr":"172.21.15.92:6805","nonce":115821}]},"public_addr":"172.21.15.92:6801/115821","cluster_addr":"172.21.15.92:6803/115821","heartbeat_back_addr":"172.21.15.92:6807/115821","heartbeat_front_addr":"172.21.15.92:6805/115821","state":["exists","up"]},{"osd":3,"uuid":"adf54e24-33a9-41fe-9ce3-98a5ab6c158e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":115825},{"type":"v1","addr":"172.21.15.92:6825","nonce":115825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":115825},{"type":"v1","addr":"172.21.15.92:6827","nonce":115825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":115825},{"type":"v1","addr":"172.21.15.92:6831","nonce":115825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":115825},{"type":"v1","addr":"172.21.15.92:6829","nonce":115825}]},"public_addr":"172.21.15.92:6825/115825","cluster_addr":"172.21.15.92:6827/115825","heartbeat_back_addr":"172.21.15.92:6831/115825","heartbeat_front_addr":"172.21.15.92:6829/115825","state":["exists","up"]},{"osd":4,"uuid":"ccb9b747-f399-44ea-9a18-9db3152227e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":116035},{"type":"v1","addr":"172.21.15.169:6817","nonce":116035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":116035},{"type":"v1","addr":"172.21.15.169:6819","nonce":116035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":116035},{"type":"v1","addr":"172.21.15.169:6826","nonce":116035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":116035},{"type":"v1","addr":"172.21.15.169:6822","nonce":116035}]},"public_addr":"172.21.15.169:6817/116035","cluster_addr":"172.21.15.169:6819/116035","heartbeat_back_addr":"172.21.15.169:6826/116035","heartbeat_front_addr":"172.21.15.169:6822/116035","state":["exists","up"]},{"osd":5,"uuid":"308a1e63-75aa-4d7d-8a7c-cecc24714fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":116033},{"type":"v1","addr":"172.21.15.169:6801","nonce":116033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":116033},{"type":"v1","addr":"172.21.15.169:6803","nonce":116033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":116033},{"type":"v1","addr":"172.21.15.169:6807","nonce":116033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":116033},{"type":"v1","addr":"172.21.15.169:6805","nonce":116033}]},"public_addr":"172.21.15.169:6801/116033","cluster_addr":"172.21.15.169:6803/116033","heartbeat_back_addr":"172.21.15.169:6807/116033","heartbeat_front_addr":"172.21.15.169:6805/116033","state":["exists","up"]},{"osd":6,"uuid":"9c459b7b-b821-4b4b-b91e-6aaae271457c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":116034},{"type":"v1","addr":"172.21.15.169:6809","nonce":116034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":116034},{"type":"v1","addr":"172.21.15.169:6811","nonce":116034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":116034},{"type":"v1","addr":"172.21.15.169:6815","nonce":116034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":116034},{"type":"v1","addr":"172.21.15.169:6813","nonce":116034}]},"public_addr":"172.21.15.169:6809/116034","cluster_addr":"172.21.15.169:6811/116034","heartbeat_back_addr":"172.21.15.169:6815/116034","heartbeat_front_addr":"172.21.15.169:6813/116034","state":["exists","up"]},{"osd":7,"uuid":"1ca9aa4a-755d-4e28-a937-4d8dccab4df3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":116044},{"type":"v1","addr":"172.21.15.169:6823","nonce":116044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6825","nonce":116044},{"type":"v1","addr":"172.21.15.169:6827","nonce":116044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":116044},{"type":"v1","addr":"172.21.15.169:6831","nonce":116044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":116044},{"type":"v1","addr":"172.21.15.169:6829","nonce":116044}]},"public_addr":"172.21.15.169:6823/116044","cluster_addr":"172.21.15.169:6827/116044","heartbeat_back_addr":"172.21.15.169:6831/116044","heartbeat_front_addr":"172.21.15.169:6829/116044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.586781+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.869778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.508594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.860493+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.872267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.746933+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.611520+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.871446+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.f","mappings":[{"from":0,"to":4}]},{"pgid":"7.3e","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:22:58.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.955+0000 7f6e87dff700 1 -- 172.21.15.92:0/2111096121 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6e640682f0 msgr2=0x7f6e6406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:58.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.955+0000 7f6e87dff700 1 --2- 172.21.15.92:0/2111096121 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6e640682f0 0x7f6e6406a7a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c0011e0 tx=0x7f6e7c009040 comp rx=0 tx=0).stop 2023-11-28T11:22:58.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.955+0000 7f6e87dff700 1 -- 172.21.15.92:0/2111096121 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e800cd460 msgr2=0x7f6e80159010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:58.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.955+0000 7f6e87dff700 1 --2- 172.21.15.92:0/2111096121 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e800cd460 0x7f6e80159010 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f6e70000c00 tx=0x7f6e70004750 comp rx=0 tx=0).stop 2023-11-28T11:22:58.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.955+0000 7f6e87dff700 1 -- 172.21.15.92:0/2111096121 shutdown_connections 2023-11-28T11:22:58.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.955+0000 7f6e87dff700 1 --2- 172.21.15.92:0/2111096121 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6e800d0d60 0x7f6e800b27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:58.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.955+0000 7f6e87dff700 1 --2- 172.21.15.92:0/2111096121 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6e800b8510 0x7f6e80147050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:58.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.955+0000 7f6e87dff700 1 --2- 172.21.15.92:0/2111096121 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6e640682f0 0x7f6e6406a7a0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:58.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.955+0000 7f6e87dff700 1 --2- 172.21.15.92:0/2111096121 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e800cd460 0x7f6e80159010 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:58.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.955+0000 7f6e87dff700 1 -- 172.21.15.92:0/2111096121 >> 172.21.15.92:0/2111096121 conn(0x7f6e800719a0 msgr2=0x7f6e800b37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:58.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.956+0000 7f6e87dff700 1 -- 172.21.15.92:0/2111096121 shutdown_connections 2023-11-28T11:22:58.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:58.956+0000 7f6e87dff700 1 -- 172.21.15.92:0/2111096121 wait complete. 2023-11-28T11:22:58.976 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T11:22:59.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.171+0000 7f86b4fff700 1 Processor -- start 2023-11-28T11:22:59.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.171+0000 7f86b4fff700 1 -- start start 2023-11-28T11:22:59.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.172+0000 7f86b4fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86b00d1d70 0x7f86b0167270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:59.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.172+0000 7f86b4fff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86b006c790 con 0x7f86b00cd460 2023-11-28T11:22:59.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.173+0000 7f86b4fff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86b00b88e0 con 0x7f86b00b8510 2023-11-28T11:22:59.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.173+0000 7f86b4fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86b00cdbd0 con 0x7f86b00d1d70 2023-11-28T11:22:59.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.173+0000 7f86a7fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f86b00cd460 0x7f86b006c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46066/0 (socket says 172.21.15.92:46066) 2023-11-28T11:22:59.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.173+0000 7f86a7fff700 1 -- 172.21.15.92:0/1740665957 learned_addr learned my addr 172.21.15.92:0/1740665957 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:22:59.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.173+0000 7f86aed9d700 1 --2- 172.21.15.92:0/1740665957 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86b00d1d70 0x7f86b0167270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:59.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.173+0000 7f86add9b700 1 -- 172.21.15.92:0/1740665957 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1056528341 0 0) 0x7f86b006c790 con 0x7f86b00cd460 2023-11-28T11:22:59.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.173+0000 7f86add9b700 1 -- 172.21.15.92:0/1740665957 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f869c0035b0 con 0x7f86b00cd460 2023-11-28T11:22:59.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.173+0000 7f86add9b700 1 -- 172.21.15.92:0/1740665957 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1761116326 0 0) 0x7f86b00b88e0 con 0x7f86b00b8510 2023-11-28T11:22:59.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.173+0000 7f86add9b700 1 -- 172.21.15.92:0/1740665957 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86b006c790 con 0x7f86b00b8510 2023-11-28T11:22:59.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.173+0000 7f86aed9d700 1 -- 172.21.15.92:0/1740665957 >> v1:172.21.15.169:6789/0 conn(0x7f86b00b8510 legacy=0x7f86b006b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:59.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.173+0000 7f86aed9d700 1 -- 172.21.15.92:0/1740665957 >> v1:172.21.15.92:6789/0 conn(0x7f86b00cd460 legacy=0x7f86b006c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:59.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.173+0000 7f86aed9d700 1 -- 172.21.15.92:0/1740665957 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86b00cd8f0 con 0x7f86b00d1d70 2023-11-28T11:22:59.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.174+0000 7f86aed9d700 1 --2- 172.21.15.92:0/1740665957 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86b00d1d70 0x7f86b0167270 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f86a8009dc0 tx=0x7f86a800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f97bf1f00aa5916b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:59.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.174+0000 7f86add9b700 1 -- 172.21.15.92:0/1740665957 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a800e070 con 0x7f86b00d1d70 2023-11-28T11:22:59.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.174+0000 7f86add9b700 1 -- 172.21.15.92:0/1740665957 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86a800bb20 con 0x7f86b00d1d70 2023-11-28T11:22:59.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.174+0000 7f86add9b700 1 -- 172.21.15.92:0/1740665957 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a8012430 con 0x7f86b00d1d70 2023-11-28T11:22:59.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.175+0000 7f86b4fff700 1 -- 172.21.15.92:0/1740665957 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86b00d1d70 msgr2=0x7f86b0167270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:59.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.175+0000 7f86b4fff700 1 --2- 172.21.15.92:0/1740665957 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86b00d1d70 0x7f86b0167270 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f86a8009dc0 tx=0x7f86a800b330 comp rx=0 tx=0).stop 2023-11-28T11:22:59.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.175+0000 7f86b4fff700 1 -- 172.21.15.92:0/1740665957 shutdown_connections 2023-11-28T11:22:59.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.175+0000 7f86b4fff700 1 --2- 172.21.15.92:0/1740665957 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86b00d1d70 0x7f86b0167270 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:59.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.175+0000 7f86b4fff700 1 -- 172.21.15.92:0/1740665957 >> 172.21.15.92:0/1740665957 conn(0x7f86b00719a0 msgr2=0x7f86b0071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:59.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.175+0000 7f86b4fff700 1 -- 172.21.15.92:0/1740665957 shutdown_connections 2023-11-28T11:22:59.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.175+0000 7f86b4fff700 1 -- 172.21.15.92:0/1740665957 wait complete. 2023-11-28T11:22:59.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.176+0000 7f86b4fff700 1 Processor -- start 2023-11-28T11:22:59.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.177+0000 7f86b4fff700 1 -- start start 2023-11-28T11:22:59.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.177+0000 7f86b4fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86b00b8510 0x7f86b01e82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:59.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.177+0000 7f86b4fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86b00cd460 0x7f86b0153f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:59.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.178+0000 7f86b4fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86b00d1d70 0x7f86b0157f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:59.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.178+0000 7f86b4fff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b01e2880 con 0x7f86b00b8510 2023-11-28T11:22:59.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.178+0000 7f86b4fff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b01e1c70 con 0x7f86b00cd460 2023-11-28T11:22:59.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.178+0000 7f86b4fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86b01e1f50 con 0x7f86b00d1d70 2023-11-28T11:22:59.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.178+0000 7f86ae59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86b00b8510 0x7f86b01e82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:59.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.178+0000 7f86ae59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86b00b8510 0x7f86b01e82e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57356/0 (socket says 172.21.15.92:57356) 2023-11-28T11:22:59.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.178+0000 7f86ae59c700 1 -- 172.21.15.92:0/4211757268 learned_addr learned my addr 172.21.15.92:0/4211757268 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:59.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.178+0000 7f86a7fff700 1 --2- 172.21.15.92:0/4211757268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86b00cd460 0x7f86b0153f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:59.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.178+0000 7f86aed9d700 1 --2- 172.21.15.92:0/4211757268 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86b00d1d70 0x7f86b0157f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:59.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.178+0000 7f86ae59c700 1 -- 172.21.15.92:0/4211757268 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86b00d1d70 msgr2=0x7f86b0157f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:59.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.178+0000 7f86ae59c700 1 --2- 172.21.15.92:0/4211757268 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86b00d1d70 0x7f86b0157f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:59.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.178+0000 7f86ae59c700 1 -- 172.21.15.92:0/4211757268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86b00cd460 msgr2=0x7f86b0153f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:59.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.178+0000 7f86ae59c700 1 --2- 172.21.15.92:0/4211757268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86b00cd460 0x7f86b0153f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:59.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.178+0000 7f86ae59c700 1 -- 172.21.15.92:0/4211757268 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86a8009a70 con 0x7f86b00b8510 2023-11-28T11:22:59.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.178+0000 7f86ae59c700 1 --2- 172.21.15.92:0/4211757268 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86b00b8510 0x7f86b01e82e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f86a000c520 tx=0x7f86a000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:59.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.179+0000 7f86a6ffd700 1 -- 172.21.15.92:0/4211757268 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a00052d0 con 0x7f86b00b8510 2023-11-28T11:22:59.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.179+0000 7f86a6ffd700 1 -- 172.21.15.92:0/4211757268 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86a0005430 con 0x7f86b00b8510 2023-11-28T11:22:59.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.179+0000 7f86b4fff700 1 -- 172.21.15.92:0/4211757268 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86b00cfff0 con 0x7f86b00b8510 2023-11-28T11:22:59.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.179+0000 7f86a6ffd700 1 -- 172.21.15.92:0/4211757268 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a00038c0 con 0x7f86b00b8510 2023-11-28T11:22:59.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.179+0000 7f86b4fff700 1 -- 172.21.15.92:0/4211757268 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86b00cfd10 con 0x7f86b00b8510 2023-11-28T11:22:59.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.180+0000 7f86b4fff700 1 -- 172.21.15.92:0/4211757268 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86b0067880 con 0x7f86b00b8510 2023-11-28T11:22:59.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.181+0000 7f86a6ffd700 1 -- 172.21.15.92:0/4211757268 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f86a0019030 con 0x7f86b00b8510 2023-11-28T11:22:59.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.181+0000 7f86a6ffd700 1 --2- 172.21.15.92:0/4211757268 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f86900682f0 0x7f869006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:59.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.182+0000 7f86a6ffd700 1 -- 172.21.15.92:0/4211757268 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f86a00851e0 con 0x7f86b00b8510 2023-11-28T11:22:59.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.182+0000 7f86a7fff700 1 --2- 172.21.15.92:0/4211757268 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f86900682f0 0x7f869006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:59.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.182+0000 7f86a7fff700 1 --2- 172.21.15.92:0/4211757268 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f86900682f0 0x7f869006a7a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f86b01e49a0 tx=0x7f8698007dc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:59.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.186+0000 7f86a6ffd700 1 -- 172.21.15.92:0/4211757268 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f86a00517d0 con 0x7f86b00b8510 2023-11-28T11:22:59.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.420+0000 7f86b4fff700 1 -- 172.21.15.92:0/4211757268 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f86b00d2940 con 0x7f86b00b8510 2023-11-28T11:22:59.602 INFO:tasks.ceph.mon.a.smithi092.stderr:2023-11-28T11:22:59.600+0000 7f086abf9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T11:22:59.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.612+0000 7f86a6ffd700 1 -- 172.21.15.92:0/4211757268 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) v1 ==== 161+0+0 (secure 0 0 0) 0x7f86a0054df0 con 0x7f86b00b8510 2023-11-28T11:22:59.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.614+0000 7f86b4fff700 1 -- 172.21.15.92:0/4211757268 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f86900682f0 msgr2=0x7f869006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:59.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.614+0000 7f86b4fff700 1 --2- 172.21.15.92:0/4211757268 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f86900682f0 0x7f869006a7a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f86b01e49a0 tx=0x7f8698007dc0 comp rx=0 tx=0).stop 2023-11-28T11:22:59.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.615+0000 7f86b4fff700 1 -- 172.21.15.92:0/4211757268 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86b00b8510 msgr2=0x7f86b01e82e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:59.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.615+0000 7f86b4fff700 1 --2- 172.21.15.92:0/4211757268 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86b00b8510 0x7f86b01e82e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f86a000c520 tx=0x7f86a000c830 comp rx=0 tx=0).stop 2023-11-28T11:22:59.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.615+0000 7f86b4fff700 1 -- 172.21.15.92:0/4211757268 shutdown_connections 2023-11-28T11:22:59.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.615+0000 7f86b4fff700 1 --2- 172.21.15.92:0/4211757268 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86b00d1d70 0x7f86b0157f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:59.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.615+0000 7f86b4fff700 1 --2- 172.21.15.92:0/4211757268 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f86900682f0 0x7f869006a7a0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:59.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.615+0000 7f86b4fff700 1 --2- 172.21.15.92:0/4211757268 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86b00b8510 0x7f86b01e82e0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:59.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.615+0000 7f86b4fff700 1 --2- 172.21.15.92:0/4211757268 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86b00cd460 0x7f86b0153f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:59.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.615+0000 7f86b4fff700 1 -- 172.21.15.92:0/4211757268 >> 172.21.15.92:0/4211757268 conn(0x7f86b00719a0 msgr2=0x7f86b006b970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:59.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.615+0000 7f86b4fff700 1 -- 172.21.15.92:0/4211757268 shutdown_connections 2023-11-28T11:22:59.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.615+0000 7f86b4fff700 1 -- 172.21.15.92:0/4211757268 wait complete. 2023-11-28T11:22:59.618 INFO:teuthology.orchestra.run.smithi092.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T11:22:59.626 INFO:tasks.ceph.mds.a.smithi092.stderr: -7388> 2023-11-28T11:22:26.897+0000 7fa25f9f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T11:22:59.636 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T11:22:59.682 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-11-28T11:22:59.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.836+0000 7f6d32dff700 1 Processor -- start 2023-11-28T11:22:59.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.836+0000 7f6d32dff700 1 -- start start 2023-11-28T11:22:59.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.837+0000 7f6d32dff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6d2c0cd460 0x7f6d2c06c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:59.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.837+0000 7f6d32dff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d2c0b88e0 con 0x7f6d2c0d0d60 2023-11-28T11:22:59.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.837+0000 7f6d32dff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d2c0cd8f0 con 0x7f6d2c0b8510 2023-11-28T11:22:59.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.837+0000 7f6d32dff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d2c0cdbd0 con 0x7f6d2c0cd460 2023-11-28T11:22:59.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.838+0000 7f6d3139c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6d2c0d0d60 0x7f6d2c0d1200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46086/0 (socket says 172.21.15.92:46086) 2023-11-28T11:22:59.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.838+0000 7f6d3139c700 1 -- 172.21.15.92:0/4038732229 learned_addr learned my addr 172.21.15.92:0/4038732229 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:22:59.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.838+0000 7f6d2bfff700 1 --2- 172.21.15.92:0/4038732229 >> v2:172.21.15.169:3300/0 conn(0x7f6d2c0cd460 0x7f6d2c06c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:59.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.838+0000 7f6d2b7fe700 1 -- 172.21.15.92:0/4038732229 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3598573567 0 0) 0x7f6d2c0b88e0 con 0x7f6d2c0d0d60 2023-11-28T11:22:59.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.838+0000 7f6d2b7fe700 1 -- 172.21.15.92:0/4038732229 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d1c0035b0 con 0x7f6d2c0d0d60 2023-11-28T11:22:59.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.838+0000 7f6d2bfff700 1 -- 172.21.15.92:0/4038732229 >> v1:172.21.15.169:6789/0 conn(0x7f6d2c0b8510 legacy=0x7f6d2c06b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:59.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.838+0000 7f6d2bfff700 1 -- 172.21.15.92:0/4038732229 >> v1:172.21.15.92:6789/0 conn(0x7f6d2c0d0d60 legacy=0x7f6d2c0d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:59.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.838+0000 7f6d2bfff700 1 -- 172.21.15.92:0/4038732229 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d2c0cfd10 con 0x7f6d2c0cd460 2023-11-28T11:22:59.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.839+0000 7f6d2bfff700 1 --2- 172.21.15.92:0/4038732229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d2c0cd460 0x7f6d2c06c080 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6d18009840 tx=0x7f6d18009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f88d401720a45e62 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:59.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.839+0000 7f6d2b7fe700 1 -- 172.21.15.92:0/4038732229 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d18011830 con 0x7f6d2c0cd460 2023-11-28T11:22:59.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.839+0000 7f6d2b7fe700 1 -- 172.21.15.92:0/4038732229 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d18011990 con 0x7f6d2c0cd460 2023-11-28T11:22:59.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.839+0000 7f6d2b7fe700 1 -- 172.21.15.92:0/4038732229 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d18011d00 con 0x7f6d2c0cd460 2023-11-28T11:22:59.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.839+0000 7f6d32dff700 1 -- 172.21.15.92:0/4038732229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d2c0cd460 msgr2=0x7f6d2c06c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:59.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.839+0000 7f6d32dff700 1 --2- 172.21.15.92:0/4038732229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d2c0cd460 0x7f6d2c06c080 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6d18009840 tx=0x7f6d18009b50 comp rx=0 tx=0).stop 2023-11-28T11:22:59.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.839+0000 7f6d32dff700 1 -- 172.21.15.92:0/4038732229 shutdown_connections 2023-11-28T11:22:59.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.839+0000 7f6d32dff700 1 --2- 172.21.15.92:0/4038732229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d2c0cd460 0x7f6d2c06c080 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:59.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.840+0000 7f6d32dff700 1 -- 172.21.15.92:0/4038732229 >> 172.21.15.92:0/4038732229 conn(0x7f6d2c0719a0 msgr2=0x7f6d2c071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:22:59.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.840+0000 7f6d32dff700 1 -- 172.21.15.92:0/4038732229 shutdown_connections 2023-11-28T11:22:59.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.840+0000 7f6d32dff700 1 -- 172.21.15.92:0/4038732229 wait complete. 2023-11-28T11:22:59.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.842+0000 7f6d32dff700 1 Processor -- start 2023-11-28T11:22:59.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.842+0000 7f6d32dff700 1 -- start start 2023-11-28T11:22:59.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.843+0000 7f6d32dff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d2c0b8510 0x7f6d2c0b1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:59.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.843+0000 7f6d32dff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d2c0cd460 0x7f6d2c06b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:59.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.843+0000 7f6d32dff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d2c0d0d60 0x7f6d2c06beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:59.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.843+0000 7f6d32dff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d2c15b8a0 con 0x7f6d2c0d0d60 2023-11-28T11:22:59.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.843+0000 7f6d32dff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d2c1496c0 con 0x7f6d2c0cd460 2023-11-28T11:22:59.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.843+0000 7f6d32dff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d2c148ab0 con 0x7f6d2c0b8510 2023-11-28T11:22:59.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.844+0000 7f6d3139c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d2c0d0d60 0x7f6d2c06beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:59.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.844+0000 7f6d3139c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d2c0d0d60 0x7f6d2c06beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57388/0 (socket says 172.21.15.92:57388) 2023-11-28T11:22:59.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.844+0000 7f6d3139c700 1 -- 172.21.15.92:0/184137828 learned_addr learned my addr 172.21.15.92:0/184137828 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:22:59.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.844+0000 7f6d2bfff700 1 --2- 172.21.15.92:0/184137828 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d2c0cd460 0x7f6d2c06b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:59.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.844+0000 7f6d30b9b700 1 --2- 172.21.15.92:0/184137828 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d2c0b8510 0x7f6d2c0b1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:59.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.844+0000 7f6d3139c700 1 -- 172.21.15.92:0/184137828 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d2c0b8510 msgr2=0x7f6d2c0b1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:59.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.844+0000 7f6d3139c700 1 --2- 172.21.15.92:0/184137828 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d2c0b8510 0x7f6d2c0b1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:59.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.844+0000 7f6d3139c700 1 -- 172.21.15.92:0/184137828 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d2c0cd460 msgr2=0x7f6d2c06b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:22:59.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.844+0000 7f6d3139c700 1 --2- 172.21.15.92:0/184137828 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d2c0cd460 0x7f6d2c06b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:22:59.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.844+0000 7f6d3139c700 1 -- 172.21.15.92:0/184137828 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d180094a0 con 0x7f6d2c0d0d60 2023-11-28T11:22:59.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.844+0000 7f6d2bfff700 1 --2- 172.21.15.92:0/184137828 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d2c0cd460 0x7f6d2c06b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:22:59.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.844+0000 7f6d3139c700 1 --2- 172.21.15.92:0/184137828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d2c0d0d60 0x7f6d2c06beb0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6d2400caa0 tx=0x7f6d2400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:59.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.844+0000 7f6d297fa700 1 -- 172.21.15.92:0/184137828 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2400aa80 con 0x7f6d2c0d0d60 2023-11-28T11:22:59.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.844+0000 7f6d297fa700 1 -- 172.21.15.92:0/184137828 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d2400abe0 con 0x7f6d2c0d0d60 2023-11-28T11:22:59.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.845+0000 7f6d32dff700 1 -- 172.21.15.92:0/184137828 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d2c0d29c0 con 0x7f6d2c0d0d60 2023-11-28T11:22:59.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.845+0000 7f6d297fa700 1 -- 172.21.15.92:0/184137828 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d24012400 con 0x7f6d2c0d0d60 2023-11-28T11:22:59.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.845+0000 7f6d32dff700 1 -- 172.21.15.92:0/184137828 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d2c0cfff0 con 0x7f6d2c0d0d60 2023-11-28T11:22:59.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.846+0000 7f6d32dff700 1 -- 172.21.15.92:0/184137828 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d2c0d2b50 con 0x7f6d2c0d0d60 2023-11-28T11:22:59.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.846+0000 7f6d297fa700 1 -- 172.21.15.92:0/184137828 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6d24019030 con 0x7f6d2c0d0d60 2023-11-28T11:22:59.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.848+0000 7f6d297fa700 1 --2- 172.21.15.92:0/184137828 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6d100682f0 0x7f6d1006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:22:59.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.848+0000 7f6d297fa700 1 -- 172.21.15.92:0/184137828 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6526+0+0 (secure 0 0 0) 0x7f6d24085b80 con 0x7f6d2c0d0d60 2023-11-28T11:22:59.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.848+0000 7f6d30b9b700 1 --2- 172.21.15.92:0/184137828 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6d100682f0 0x7f6d1006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:22:59.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.848+0000 7f6d30b9b700 1 --2- 172.21.15.92:0/184137828 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6d100682f0 0x7f6d1006a7a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c1e8310 tx=0x7f6d20009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:22:59.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:22:59.852+0000 7f6d297fa700 1 -- 172.21.15.92:0/184137828 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6d24052120 con 0x7f6d2c0d0d60 2023-11-28T11:23:00.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.084+0000 7f6d32dff700 1 -- 172.21.15.92:0/184137828 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f6d2c066d60 con 0x7f6d2c0d0d60 2023-11-28T11:23:00.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.614+0000 7f6d297fa700 1 -- 172.21.15.92:0/184137828 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6d24055740 con 0x7f6d2c0d0d60 2023-11-28T11:23:00.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.617+0000 7f6d32dff700 1 -- 172.21.15.92:0/184137828 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6d100682f0 msgr2=0x7f6d1006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:00.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.617+0000 7f6d32dff700 1 --2- 172.21.15.92:0/184137828 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6d100682f0 0x7f6d1006a7a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c1e8310 tx=0x7f6d20009040 comp rx=0 tx=0).stop 2023-11-28T11:23:00.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.617+0000 7f6d32dff700 1 -- 172.21.15.92:0/184137828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d2c0d0d60 msgr2=0x7f6d2c06beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:00.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.617+0000 7f6d32dff700 1 --2- 172.21.15.92:0/184137828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d2c0d0d60 0x7f6d2c06beb0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6d2400caa0 tx=0x7f6d2400ce60 comp rx=0 tx=0).stop 2023-11-28T11:23:00.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.617+0000 7f6d32dff700 1 -- 172.21.15.92:0/184137828 shutdown_connections 2023-11-28T11:23:00.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.617+0000 7f6d32dff700 1 --2- 172.21.15.92:0/184137828 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6d100682f0 0x7f6d1006a7a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:00.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.617+0000 7f6d32dff700 1 --2- 172.21.15.92:0/184137828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d2c0d0d60 0x7f6d2c06beb0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:00.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.617+0000 7f6d32dff700 1 --2- 172.21.15.92:0/184137828 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d2c0b8510 0x7f6d2c0b1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:00.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.617+0000 7f6d32dff700 1 --2- 172.21.15.92:0/184137828 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d2c0cd460 0x7f6d2c06b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:00.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.617+0000 7f6d32dff700 1 -- 172.21.15.92:0/184137828 >> 172.21.15.92:0/184137828 conn(0x7f6d2c0719a0 msgr2=0x7f6d2c0b6bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:00.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.617+0000 7f6d32dff700 1 -- 172.21.15.92:0/184137828 shutdown_connections 2023-11-28T11:23:00.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.617+0000 7f6d32dff700 1 -- 172.21.15.92:0/184137828 wait complete. 2023-11-28T11:23:00.639 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T11:23:00.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.880+0000 7fb0301ff700 1 Processor -- start 2023-11-28T11:23:00.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.881+0000 7fb0301ff700 1 -- start start 2023-11-28T11:23:00.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.882+0000 7fb0301ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb02815d180 0x7fb0280cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:00.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.882+0000 7fb0301ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb0280d3500 0x7fb0280d0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:00.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.882+0000 7fb0301ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0280cf230 con 0x7fb02815d550 2023-11-28T11:23:00.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.882+0000 7fb0301ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0280d0210 con 0x7fb0280d3500 2023-11-28T11:23:00.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.882+0000 7fb0301ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0280cd850 con 0x7fb02815d180 2023-11-28T11:23:00.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.882+0000 7fb02d79a700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb02815d550 0x7fb0280d2df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46090/0 (socket says 172.21.15.92:46090) 2023-11-28T11:23:00.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.882+0000 7fb02d79a700 1 -- 172.21.15.92:0/1356082026 learned_addr learned my addr 172.21.15.92:0/1356082026 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:23:00.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.882+0000 7fb02e79c700 1 --2- 172.21.15.92:0/1356082026 >> v2:172.21.15.92:3300/0 conn(0x7fb0280d3500 0x7fb0280d0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:00.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.882+0000 7fb02df9b700 1 --2- 172.21.15.92:0/1356082026 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb02815d180 0x7fb0280cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:00.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.882+0000 7fb02cf99700 1 -- 172.21.15.92:0/1356082026 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3442719426 0 0) 0x7fb0280cf230 con 0x7fb02815d550 2023-11-28T11:23:00.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.882+0000 7fb02cf99700 1 -- 172.21.15.92:0/1356082026 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0140035b0 con 0x7fb02815d550 2023-11-28T11:23:00.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.882+0000 7fb02e79c700 1 -- 172.21.15.92:0/1356082026 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb02815d180 msgr2=0x7fb0280cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:00.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.883+0000 7fb02e79c700 1 --2- 172.21.15.92:0/1356082026 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb02815d180 0x7fb0280cfcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:00.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.883+0000 7fb02e79c700 1 -- 172.21.15.92:0/1356082026 >> v1:172.21.15.92:6789/0 conn(0x7fb02815d550 legacy=0x7fb0280d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:00.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.883+0000 7fb02e79c700 1 -- 172.21.15.92:0/1356082026 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0280cf4d0 con 0x7fb0280d3500 2023-11-28T11:23:00.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.883+0000 7fb02e79c700 1 --2- 172.21.15.92:0/1356082026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0280d3500 0x7fb0280d0a30 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fb020009d90 tx=0x7fb02000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc653eb3445cc8d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:00.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.883+0000 7fb02cf99700 1 -- 172.21.15.92:0/1356082026 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02000e070 con 0x7fb0280d3500 2023-11-28T11:23:00.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.883+0000 7fb02cf99700 1 -- 172.21.15.92:0/1356082026 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb02000bb20 con 0x7fb0280d3500 2023-11-28T11:23:00.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.883+0000 7fb02cf99700 1 -- 172.21.15.92:0/1356082026 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb020012430 con 0x7fb0280d3500 2023-11-28T11:23:00.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.884+0000 7fb0301ff700 1 -- 172.21.15.92:0/1356082026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0280d3500 msgr2=0x7fb0280d0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:00.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.884+0000 7fb0301ff700 1 --2- 172.21.15.92:0/1356082026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0280d3500 0x7fb0280d0a30 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fb020009d90 tx=0x7fb02000b330 comp rx=0 tx=0).stop 2023-11-28T11:23:00.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.884+0000 7fb0301ff700 1 -- 172.21.15.92:0/1356082026 shutdown_connections 2023-11-28T11:23:00.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.884+0000 7fb0301ff700 1 --2- 172.21.15.92:0/1356082026 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb02815d180 0x7fb0280cfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:00.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.884+0000 7fb0301ff700 1 --2- 172.21.15.92:0/1356082026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0280d3500 0x7fb0280d0a30 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:00.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.884+0000 7fb0301ff700 1 -- 172.21.15.92:0/1356082026 >> 172.21.15.92:0/1356082026 conn(0x7fb0280719a0 msgr2=0x7fb028071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:00.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.884+0000 7fb0301ff700 1 -- 172.21.15.92:0/1356082026 shutdown_connections 2023-11-28T11:23:00.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.884+0000 7fb0301ff700 1 -- 172.21.15.92:0/1356082026 wait complete. 2023-11-28T11:23:00.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.885+0000 7fb0301ff700 1 Processor -- start 2023-11-28T11:23:00.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.886+0000 7fb0301ff700 1 -- start start 2023-11-28T11:23:00.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.886+0000 7fb0301ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0280d3500 0x7fb02814b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:00.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.886+0000 7fb0301ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb02815d180 0x7fb028150a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:00.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.886+0000 7fb0301ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb02815d550 0x7fb028150fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:00.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.886+0000 7fb0301ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0280c73a0 con 0x7fb0280d3500 2023-11-28T11:23:00.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.886+0000 7fb0301ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0280c7680 con 0x7fb02815d550 2023-11-28T11:23:00.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.886+0000 7fb0301ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0280c7960 con 0x7fb02815d180 2023-11-28T11:23:00.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.887+0000 7fb02df9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0280d3500 0x7fb02814b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:00.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.887+0000 7fb02df9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0280d3500 0x7fb02814b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57404/0 (socket says 172.21.15.92:57404) 2023-11-28T11:23:00.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.887+0000 7fb02df9b700 1 -- 172.21.15.92:0/3532592333 learned_addr learned my addr 172.21.15.92:0/3532592333 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:00.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.887+0000 7fb02e79c700 1 --2- 172.21.15.92:0/3532592333 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb02815d550 0x7fb028150fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:00.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.887+0000 7fb02d79a700 1 --2- 172.21.15.92:0/3532592333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb02815d180 0x7fb028150a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:00.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.887+0000 7fb02df9b700 1 -- 172.21.15.92:0/3532592333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb02815d180 msgr2=0x7fb028150a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:00.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.887+0000 7fb02df9b700 1 --2- 172.21.15.92:0/3532592333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb02815d180 0x7fb028150a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:00.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.887+0000 7fb02df9b700 1 -- 172.21.15.92:0/3532592333 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb02815d550 msgr2=0x7fb028150fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:00.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.887+0000 7fb02df9b700 1 --2- 172.21.15.92:0/3532592333 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb02815d550 0x7fb028150fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:00.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.887+0000 7fb02df9b700 1 -- 172.21.15.92:0/3532592333 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb020009a70 con 0x7fb0280d3500 2023-11-28T11:23:00.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.887+0000 7fb02e79c700 1 --2- 172.21.15.92:0/3532592333 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb02815d550 0x7fb028150fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:23:00.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.887+0000 7fb02d79a700 1 --2- 172.21.15.92:0/3532592333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb02815d180 0x7fb028150a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:23:00.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.887+0000 7fb02df9b700 1 --2- 172.21.15.92:0/3532592333 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0280d3500 0x7fb02814b610 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fb02400d800 tx=0x7fb02400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:00.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.888+0000 7fb01e7fc700 1 -- 172.21.15.92:0/3532592333 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0240125f0 con 0x7fb0280d3500 2023-11-28T11:23:00.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.888+0000 7fb01e7fc700 1 -- 172.21.15.92:0/3532592333 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb024012750 con 0x7fb0280d3500 2023-11-28T11:23:00.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.888+0000 7fb01e7fc700 1 -- 172.21.15.92:0/3532592333 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb024012b00 con 0x7fb0280d3500 2023-11-28T11:23:00.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.888+0000 7fb0301ff700 1 -- 172.21.15.92:0/3532592333 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb02815d9c0 con 0x7fb0280d3500 2023-11-28T11:23:00.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.888+0000 7fb0301ff700 1 -- 172.21.15.92:0/3532592333 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0280cf7b0 con 0x7fb0280d3500 2023-11-28T11:23:00.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.889+0000 7fb0301ff700 1 -- 172.21.15.92:0/3532592333 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0280cd4b0 con 0x7fb0280d3500 2023-11-28T11:23:00.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.890+0000 7fb01e7fc700 1 -- 172.21.15.92:0/3532592333 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb024012c60 con 0x7fb0280d3500 2023-11-28T11:23:00.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.891+0000 7fb01e7fc700 1 --2- 172.21.15.92:0/3532592333 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb00c0682f0 0x7fb00c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:00.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.891+0000 7fb01e7fc700 1 -- 172.21.15.92:0/3532592333 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6526+0+0 (secure 0 0 0) 0x7fb024086300 con 0x7fb0280d3500 2023-11-28T11:23:00.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.891+0000 7fb02d79a700 1 --2- 172.21.15.92:0/3532592333 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb00c0682f0 0x7fb00c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:00.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.891+0000 7fb02d79a700 1 --2- 172.21.15.92:0/3532592333 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb00c0682f0 0x7fb00c06a7a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb028153450 tx=0x7fb018009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:00.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:00.895+0000 7fb01e7fc700 1 -- 172.21.15.92:0/3532592333 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb024052990 con 0x7fb0280d3500 2023-11-28T11:23:01.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.087+0000 7fb0301ff700 1 -- 172.21.15.92:0/3532592333 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb028067170 con 0x7fb0280d3500 2023-11-28T11:23:01.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.696+0000 7fb01e7fc700 1 -- 172.21.15.92:0/3532592333 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v46) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb024055fb0 con 0x7fb0280d3500 2023-11-28T11:23:01.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.700+0000 7fb0301ff700 1 -- 172.21.15.92:0/3532592333 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb00c0682f0 msgr2=0x7fb00c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:01.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.700+0000 7fb0301ff700 1 --2- 172.21.15.92:0/3532592333 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb00c0682f0 0x7fb00c06a7a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb028153450 tx=0x7fb018009040 comp rx=0 tx=0).stop 2023-11-28T11:23:01.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.700+0000 7fb0301ff700 1 -- 172.21.15.92:0/3532592333 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0280d3500 msgr2=0x7fb02814b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:01.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.700+0000 7fb0301ff700 1 --2- 172.21.15.92:0/3532592333 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0280d3500 0x7fb02814b610 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fb02400d800 tx=0x7fb02400dbc0 comp rx=0 tx=0).stop 2023-11-28T11:23:01.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.700+0000 7fb0301ff700 1 -- 172.21.15.92:0/3532592333 shutdown_connections 2023-11-28T11:23:01.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.700+0000 7fb0301ff700 1 --2- 172.21.15.92:0/3532592333 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb02815d550 0x7fb028150fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:01.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.700+0000 7fb0301ff700 1 --2- 172.21.15.92:0/3532592333 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb00c0682f0 0x7fb00c06a7a0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:01.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.700+0000 7fb0301ff700 1 --2- 172.21.15.92:0/3532592333 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0280d3500 0x7fb02814b610 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:01.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.700+0000 7fb0301ff700 1 --2- 172.21.15.92:0/3532592333 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb02815d180 0x7fb028150a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:01.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.700+0000 7fb0301ff700 1 -- 172.21.15.92:0/3532592333 >> 172.21.15.92:0/3532592333 conn(0x7fb0280719a0 msgr2=0x7fb028152490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:01.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.700+0000 7fb0301ff700 1 -- 172.21.15.92:0/3532592333 shutdown_connections 2023-11-28T11:23:01.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.700+0000 7fb0301ff700 1 -- 172.21.15.92:0/3532592333 wait complete. 2023-11-28T11:23:01.703 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' removed 2023-11-28T11:23:01.727 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T11:23:01.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.929+0000 7f0571bff700 1 Processor -- start 2023-11-28T11:23:01.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.929+0000 7f0571bff700 1 -- start start 2023-11-28T11:23:01.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.930+0000 7f0571bff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f056c0b8790 0x7f056c0cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:01.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.930+0000 7f0571bff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f056c0d00b0 con 0x7f056c06c050 2023-11-28T11:23:01.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.930+0000 7f0571bff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f056c0b5240 con 0x7f056c0cd460 2023-11-28T11:23:01.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.930+0000 7f0571bff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f056c0b5520 con 0x7f056c0b8790 2023-11-28T11:23:01.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.930+0000 7f056bfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f056c06c050 0x7f056c0d0fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46102/0 (socket says 172.21.15.92:46102) 2023-11-28T11:23:01.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.931+0000 7f056bfff700 1 -- 172.21.15.92:0/3638491642 learned_addr learned my addr 172.21.15.92:0/3638491642 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:23:01.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.931+0000 7f056b7fe700 1 --2- 172.21.15.92:0/3638491642 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f056c0b8790 0x7f056c0cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:01.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.931+0000 7f056a7fc700 1 -- 172.21.15.92:0/3638491642 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1693473951 0 0) 0x7f056c0d00b0 con 0x7f056c06c050 2023-11-28T11:23:01.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.931+0000 7f056a7fc700 1 -- 172.21.15.92:0/3638491642 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05540035b0 con 0x7f056c06c050 2023-11-28T11:23:01.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.931+0000 7f056a7fc700 1 -- 172.21.15.92:0/3638491642 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3378456292 0 0) 0x7f056c0b5240 con 0x7f056c0cd460 2023-11-28T11:23:01.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.931+0000 7f056a7fc700 1 -- 172.21.15.92:0/3638491642 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f056c0d00b0 con 0x7f056c0cd460 2023-11-28T11:23:01.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.931+0000 7f056b7fe700 1 -- 172.21.15.92:0/3638491642 >> v1:172.21.15.169:6789/0 conn(0x7f056c0cd460 legacy=0x7f056c06b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:01.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.931+0000 7f056b7fe700 1 -- 172.21.15.92:0/3638491642 >> v1:172.21.15.92:6789/0 conn(0x7f056c06c050 legacy=0x7f056c0d0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:01.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.931+0000 7f056a7fc700 1 -- 172.21.15.92:0/3638491642 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4233846656 0 0) 0x7f05540035b0 con 0x7f056c06c050 2023-11-28T11:23:01.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.931+0000 7f056b7fe700 1 -- 172.21.15.92:0/3638491642 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f056c0d0a80 con 0x7f056c0b8790 2023-11-28T11:23:01.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.932+0000 7f056b7fe700 1 --2- 172.21.15.92:0/3638491642 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f056c0b8790 0x7f056c0cfb70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f055c009870 tx=0x7f055c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=105ecde93ec65471 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:01.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.932+0000 7f056a7fc700 1 -- 172.21.15.92:0/3638491642 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05640019f0 con 0x7f056c0b8790 2023-11-28T11:23:01.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.932+0000 7f056a7fc700 1 -- 172.21.15.92:0/3638491642 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f055c004030 con 0x7f056c0b8790 2023-11-28T11:23:01.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.932+0000 7f056a7fc700 1 -- 172.21.15.92:0/3638491642 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055c0043c0 con 0x7f056c0b8790 2023-11-28T11:23:01.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.932+0000 7f0571bff700 1 -- 172.21.15.92:0/3638491642 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f056c0b8790 msgr2=0x7f056c0cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:01.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.932+0000 7f0571bff700 1 --2- 172.21.15.92:0/3638491642 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f056c0b8790 0x7f056c0cfb70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f055c009870 tx=0x7f055c009b80 comp rx=0 tx=0).stop 2023-11-28T11:23:01.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.932+0000 7f0571bff700 1 -- 172.21.15.92:0/3638491642 shutdown_connections 2023-11-28T11:23:01.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.932+0000 7f0571bff700 1 --2- 172.21.15.92:0/3638491642 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f056c0b8790 0x7f056c0cfb70 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:01.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.932+0000 7f0571bff700 1 -- 172.21.15.92:0/3638491642 >> 172.21.15.92:0/3638491642 conn(0x7f056c0719a0 msgr2=0x7f056c071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:01.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.933+0000 7f0571bff700 1 -- 172.21.15.92:0/3638491642 shutdown_connections 2023-11-28T11:23:01.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.933+0000 7f0571bff700 1 -- 172.21.15.92:0/3638491642 wait complete. 2023-11-28T11:23:01.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.934+0000 7f0571bff700 1 Processor -- start 2023-11-28T11:23:01.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.934+0000 7f0571bff700 1 -- start start 2023-11-28T11:23:01.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.935+0000 7f0571bff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f056c06c050 0x7f056c15c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:01.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.935+0000 7f0571bff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f056c0b8790 0x7f056c149d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:01.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.935+0000 7f0571bff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f056c0cd460 0x7f056c14a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:01.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.935+0000 7f0571bff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056c155be0 con 0x7f056c0b8790 2023-11-28T11:23:01.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.935+0000 7f0571bff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056c154f60 con 0x7f056c06c050 2023-11-28T11:23:01.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.935+0000 7f0571bff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f056c155240 con 0x7f056c0cd460 2023-11-28T11:23:01.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.936+0000 7f056affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f056c0b8790 0x7f056c149d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:01.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.936+0000 7f056affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f056c0b8790 0x7f056c149d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57416/0 (socket says 172.21.15.92:57416) 2023-11-28T11:23:01.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.936+0000 7f056affd700 1 -- 172.21.15.92:0/2031809126 learned_addr learned my addr 172.21.15.92:0/2031809126 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:01.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.936+0000 7f056b7fe700 1 --2- 172.21.15.92:0/2031809126 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f056c06c050 0x7f056c15c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:01.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.936+0000 7f056bfff700 1 --2- 172.21.15.92:0/2031809126 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f056c0cd460 0x7f056c14a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:01.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.936+0000 7f056affd700 1 -- 172.21.15.92:0/2031809126 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f056c0cd460 msgr2=0x7f056c14a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:01.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.936+0000 7f056affd700 1 --2- 172.21.15.92:0/2031809126 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f056c0cd460 0x7f056c14a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:01.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.936+0000 7f056affd700 1 -- 172.21.15.92:0/2031809126 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f056c06c050 msgr2=0x7f056c15c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:01.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.936+0000 7f056affd700 1 --2- 172.21.15.92:0/2031809126 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f056c06c050 0x7f056c15c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:01.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.936+0000 7f056affd700 1 -- 172.21.15.92:0/2031809126 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f055c0094a0 con 0x7f056c0b8790 2023-11-28T11:23:01.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.936+0000 7f056bfff700 1 --2- 172.21.15.92:0/2031809126 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f056c0cd460 0x7f056c14a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:23:01.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.936+0000 7f056affd700 1 --2- 172.21.15.92:0/2031809126 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f056c0b8790 0x7f056c149d80 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f056000c500 tx=0x7f056000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:01.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.937+0000 7f0553fff700 1 -- 172.21.15.92:0/2031809126 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05600052d0 con 0x7f056c0b8790 2023-11-28T11:23:01.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.937+0000 7f0553fff700 1 -- 172.21.15.92:0/2031809126 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0560005450 con 0x7f056c0b8790 2023-11-28T11:23:01.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.937+0000 7f0553fff700 1 -- 172.21.15.92:0/2031809126 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05600038c0 con 0x7f056c0b8790 2023-11-28T11:23:01.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.937+0000 7f0571bff700 1 -- 172.21.15.92:0/2031809126 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f056c0d16b0 con 0x7f056c0b8790 2023-11-28T11:23:01.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.937+0000 7f0571bff700 1 -- 172.21.15.92:0/2031809126 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f056c0b5800 con 0x7f056c0b8790 2023-11-28T11:23:01.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.938+0000 7f0571bff700 1 -- 172.21.15.92:0/2031809126 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f056c0d1840 con 0x7f056c0b8790 2023-11-28T11:23:01.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.939+0000 7f0553fff700 1 -- 172.21.15.92:0/2031809126 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0560019030 con 0x7f056c0b8790 2023-11-28T11:23:01.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.939+0000 7f0553fff700 1 --2- 172.21.15.92:0/2031809126 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f05580682f0 0x7f055806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:01.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.939+0000 7f0553fff700 1 -- 172.21.15.92:0/2031809126 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6088+0+0 (secure 0 0 0) 0x7f0560085070 con 0x7f056c0b8790 2023-11-28T11:23:01.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.940+0000 7f056b7fe700 1 --2- 172.21.15.92:0/2031809126 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f05580682f0 0x7f055806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:01.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.940+0000 7f056b7fe700 1 --2- 172.21.15.92:0/2031809126 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f05580682f0 0x7f055806a7a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f055c0090b0 tx=0x7f055c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:01.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:01.943+0000 7f0553fff700 1 -- 172.21.15.92:0/2031809126 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f05600517c0 con 0x7f056c0b8790 2023-11-28T11:23:02.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.133+0000 7f0571bff700 1 -- 172.21.15.92:0/2031809126 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f056c155d20 con 0x7f056c0b8790 2023-11-28T11:23:02.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.705+0000 7f0553fff700 1 -- 172.21.15.92:0/2031809126 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v47) v1 ==== 169+0+0 (secure 0 0 0) 0x7f056c155d20 con 0x7f056c0b8790 2023-11-28T11:23:02.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.709+0000 7f0571bff700 1 -- 172.21.15.92:0/2031809126 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f05580682f0 msgr2=0x7f055806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:02.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.709+0000 7f0571bff700 1 --2- 172.21.15.92:0/2031809126 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f05580682f0 0x7f055806a7a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f055c0090b0 tx=0x7f055c008030 comp rx=0 tx=0).stop 2023-11-28T11:23:02.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.709+0000 7f0571bff700 1 -- 172.21.15.92:0/2031809126 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f056c0b8790 msgr2=0x7f056c149d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:02.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.709+0000 7f0571bff700 1 --2- 172.21.15.92:0/2031809126 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f056c0b8790 0x7f056c149d80 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f056000c500 tx=0x7f056000c810 comp rx=0 tx=0).stop 2023-11-28T11:23:02.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.709+0000 7f0571bff700 1 -- 172.21.15.92:0/2031809126 shutdown_connections 2023-11-28T11:23:02.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.709+0000 7f0571bff700 1 --2- 172.21.15.92:0/2031809126 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f056c0cd460 0x7f056c14a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:02.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.709+0000 7f0571bff700 1 --2- 172.21.15.92:0/2031809126 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f056c06c050 0x7f056c15c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:02.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.709+0000 7f0571bff700 1 --2- 172.21.15.92:0/2031809126 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f05580682f0 0x7f055806a7a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:02.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.709+0000 7f0571bff700 1 --2- 172.21.15.92:0/2031809126 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f056c0b8790 0x7f056c149d80 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:02.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.709+0000 7f0571bff700 1 -- 172.21.15.92:0/2031809126 >> 172.21.15.92:0/2031809126 conn(0x7f056c0719a0 msgr2=0x7f056c15b130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:02.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.709+0000 7f0571bff700 1 -- 172.21.15.92:0/2031809126 shutdown_connections 2023-11-28T11:23:02.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.709+0000 7f0571bff700 1 -- 172.21.15.92:0/2031809126 wait complete. 2023-11-28T11:23:02.713 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' removed 2023-11-28T11:23:02.731 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-28T11:23:02.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.938+0000 7f98961ff700 1 Processor -- start 2023-11-28T11:23:02.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.938+0000 7f98961ff700 1 -- start start 2023-11-28T11:23:02.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.940+0000 7f98961ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f98900d1f20 0x7f98900d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:02.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.940+0000 7f98961ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f989015b4e0 con 0x7f9890160fd0 2023-11-28T11:23:02.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.940+0000 7f98961ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98900d2970 con 0x7f98900cf8b0 2023-11-28T11:23:02.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.940+0000 7f98961ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f989006c1b0 con 0x7f98900d1f20 2023-11-28T11:23:02.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.940+0000 7f988f7fe700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f98900cf8b0 0x7f98900cfc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.92:55506/0 (socket says 172.21.15.92:55506) 2023-11-28T11:23:02.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.940+0000 7f988f7fe700 1 -- 172.21.15.92:0/2715400451 learned_addr learned my addr 172.21.15.92:0/2715400451 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:23:02.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.940+0000 7f988effd700 1 --2- 172.21.15.92:0/2715400451 >> v2:172.21.15.169:3300/0 conn(0x7f98900d1f20 0x7f98900d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:02.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.941+0000 7f988effd700 1 -- 172.21.15.92:0/2715400451 >> v1:172.21.15.169:6789/0 conn(0x7f98900cf8b0 legacy=0x7f98900cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:02.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.941+0000 7f988effd700 1 -- 172.21.15.92:0/2715400451 >> v1:172.21.15.92:6789/0 conn(0x7f9890160fd0 legacy=0x7f9890161470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:02.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.941+0000 7f988effd700 1 -- 172.21.15.92:0/2715400451 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f989006baf0 con 0x7f98900d1f20 2023-11-28T11:23:02.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.941+0000 7f988effd700 1 --2- 172.21.15.92:0/2715400451 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98900d1f20 0x7f98900d2370 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9878009840 tx=0x7f9878009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62527c1321574397 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:02.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.942+0000 7f988e7fc700 1 -- 172.21.15.92:0/2715400451 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9878011830 con 0x7f98900d1f20 2023-11-28T11:23:02.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.942+0000 7f988e7fc700 1 -- 172.21.15.92:0/2715400451 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9878011990 con 0x7f98900d1f20 2023-11-28T11:23:02.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.942+0000 7f988e7fc700 1 -- 172.21.15.92:0/2715400451 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9878011d20 con 0x7f98900d1f20 2023-11-28T11:23:02.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.942+0000 7f98961ff700 1 -- 172.21.15.92:0/2715400451 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98900d1f20 msgr2=0x7f98900d2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:02.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.942+0000 7f98961ff700 1 --2- 172.21.15.92:0/2715400451 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98900d1f20 0x7f98900d2370 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9878009840 tx=0x7f9878009b50 comp rx=0 tx=0).stop 2023-11-28T11:23:02.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.942+0000 7f98961ff700 1 -- 172.21.15.92:0/2715400451 shutdown_connections 2023-11-28T11:23:02.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.942+0000 7f98961ff700 1 --2- 172.21.15.92:0/2715400451 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f98900d1f20 0x7f98900d2370 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:02.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.942+0000 7f98961ff700 1 -- 172.21.15.92:0/2715400451 >> 172.21.15.92:0/2715400451 conn(0x7f98900719a0 msgr2=0x7f9890071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:02.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.943+0000 7f98961ff700 1 -- 172.21.15.92:0/2715400451 shutdown_connections 2023-11-28T11:23:02.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.943+0000 7f98961ff700 1 -- 172.21.15.92:0/2715400451 wait complete. 2023-11-28T11:23:02.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.944+0000 7f98961ff700 1 Processor -- start 2023-11-28T11:23:02.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.945+0000 7f98961ff700 1 -- start start 2023-11-28T11:23:02.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.946+0000 7f98961ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98900cf8b0 0x7f9890145d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:02.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.946+0000 7f98961ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98900d1f20 0x7f98900b7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:02.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.946+0000 7f98961ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9890160fd0 0x7f98901f9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:02.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.946+0000 7f98961ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98900caf30 con 0x7f98900cf8b0 2023-11-28T11:23:02.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.946+0000 7f98961ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98900b84d0 con 0x7f9890160fd0 2023-11-28T11:23:02.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.946+0000 7f98961ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98900b87b0 con 0x7f98900d1f20 2023-11-28T11:23:02.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.946+0000 7f988f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98900cf8b0 0x7f9890145d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:02.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.946+0000 7f988f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98900cf8b0 0x7f9890145d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57428/0 (socket says 172.21.15.92:57428) 2023-11-28T11:23:02.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.946+0000 7f988f7fe700 1 -- 172.21.15.92:0/2466871725 learned_addr learned my addr 172.21.15.92:0/2466871725 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:02.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.946+0000 7f988effd700 1 --2- 172.21.15.92:0/2466871725 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98900d1f20 0x7f98900b7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:02.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.946+0000 7f988ffff700 1 --2- 172.21.15.92:0/2466871725 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9890160fd0 0x7f98901f9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:02.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.946+0000 7f988f7fe700 1 -- 172.21.15.92:0/2466871725 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98900d1f20 msgr2=0x7f98900b7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:02.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.946+0000 7f988f7fe700 1 --2- 172.21.15.92:0/2466871725 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98900d1f20 0x7f98900b7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:02.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.946+0000 7f988f7fe700 1 -- 172.21.15.92:0/2466871725 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9890160fd0 msgr2=0x7f98901f9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:02.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.946+0000 7f988f7fe700 1 --2- 172.21.15.92:0/2466871725 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9890160fd0 0x7f98901f9980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:02.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.947+0000 7f988f7fe700 1 -- 172.21.15.92:0/2466871725 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98780094a0 con 0x7f98900cf8b0 2023-11-28T11:23:02.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.947+0000 7f988f7fe700 1 --2- 172.21.15.92:0/2466871725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98900cf8b0 0x7f9890145d40 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f988800c520 tx=0x7f988800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:02.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.947+0000 7f9887fff700 1 -- 172.21.15.92:0/2466871725 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9888005d70 con 0x7f98900cf8b0 2023-11-28T11:23:02.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.947+0000 7f9887fff700 1 -- 172.21.15.92:0/2466871725 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98880070d0 con 0x7f98900cf8b0 2023-11-28T11:23:02.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.947+0000 7f9887fff700 1 -- 172.21.15.92:0/2466871725 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9888007440 con 0x7f98900cf8b0 2023-11-28T11:23:02.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.947+0000 7f98961ff700 1 -- 172.21.15.92:0/2466871725 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98900b5270 con 0x7f98900cf8b0 2023-11-28T11:23:02.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.947+0000 7f98961ff700 1 -- 172.21.15.92:0/2466871725 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f989006bdd0 con 0x7f98900cf8b0 2023-11-28T11:23:02.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.948+0000 7f98961ff700 1 -- 172.21.15.92:0/2466871725 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98900b5400 con 0x7f98900cf8b0 2023-11-28T11:23:02.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.949+0000 7f9887fff700 1 -- 172.21.15.92:0/2466871725 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9888019030 con 0x7f98900cf8b0 2023-11-28T11:23:02.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.950+0000 7f9887fff700 1 --2- 172.21.15.92:0/2466871725 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f98700682f0 0x7f987006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:02.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.950+0000 7f9887fff700 1 -- 172.21.15.92:0/2466871725 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f9888084d10 con 0x7f98900cf8b0 2023-11-28T11:23:02.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.950+0000 7f988effd700 1 --2- 172.21.15.92:0/2466871725 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f98700682f0 0x7f987006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:02.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.951+0000 7f988effd700 1 --2- 172.21.15.92:0/2466871725 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f98700682f0 0x7f987006a7a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f98780090b0 tx=0x7f9878008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:02.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:02.954+0000 7f9887fff700 1 -- 172.21.15.92:0/2466871725 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f988804b5e0 con 0x7f98900cf8b0 2023-11-28T11:23:03.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.148+0000 7f98961ff700 1 -- 172.21.15.92:0/2466871725 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f9890067880 con 0x7f98900cf8b0 2023-11-28T11:23:03.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.152+0000 7f9887fff700 1 -- 172.21.15.92:0/2466871725 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f988808a020 con 0x7f98900cf8b0 2023-11-28T11:23:03.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.154+0000 7f98961ff700 1 -- 172.21.15.92:0/2466871725 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f98700682f0 msgr2=0x7f987006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:03.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.154+0000 7f98961ff700 1 --2- 172.21.15.92:0/2466871725 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f98700682f0 0x7f987006a7a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f98780090b0 tx=0x7f9878008030 comp rx=0 tx=0).stop 2023-11-28T11:23:03.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.154+0000 7f98961ff700 1 -- 172.21.15.92:0/2466871725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98900cf8b0 msgr2=0x7f9890145d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:03.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.154+0000 7f98961ff700 1 --2- 172.21.15.92:0/2466871725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98900cf8b0 0x7f9890145d40 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f988800c520 tx=0x7f988800c830 comp rx=0 tx=0).stop 2023-11-28T11:23:03.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.155+0000 7f98961ff700 1 -- 172.21.15.92:0/2466871725 shutdown_connections 2023-11-28T11:23:03.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.155+0000 7f98961ff700 1 --2- 172.21.15.92:0/2466871725 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9890160fd0 0x7f98901f9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:03.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.155+0000 7f98961ff700 1 --2- 172.21.15.92:0/2466871725 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f98700682f0 0x7f987006a7a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:03.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.155+0000 7f98961ff700 1 --2- 172.21.15.92:0/2466871725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98900cf8b0 0x7f9890145d40 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:03.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.155+0000 7f98961ff700 1 --2- 172.21.15.92:0/2466871725 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f98900d1f20 0x7f98900b7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:03.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.155+0000 7f98961ff700 1 -- 172.21.15.92:0/2466871725 >> 172.21.15.92:0/2466871725 conn(0x7f98900719a0 msgr2=0x7f98900cfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:03.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.155+0000 7f98961ff700 1 -- 172.21.15.92:0/2466871725 shutdown_connections 2023-11-28T11:23:03.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.155+0000 7f98961ff700 1 -- 172.21.15.92:0/2466871725 wait complete. 2023-11-28T11:23:03.226 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2023-11-28T11:23:03.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.421+0000 7f119a7ff700 1 Processor -- start 2023-11-28T11:23:03.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.421+0000 7f119a7ff700 1 -- start start 2023-11-28T11:23:03.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.422+0000 7f119a7ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11940b8510 0x7f119406b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:03.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.422+0000 7f119a7ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11940b88e0 con 0x7f11940cd460 2023-11-28T11:23:03.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.422+0000 7f119a7ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11940cd8f0 con 0x7f11940d0d60 2023-11-28T11:23:03.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.422+0000 7f119a7ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11940cdbd0 con 0x7f11940b8510 2023-11-28T11:23:03.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.422+0000 7f11937fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f11940cd460 0x7f119406beb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46108/0 (socket says 172.21.15.92:46108) 2023-11-28T11:23:03.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.423+0000 7f11937fe700 1 -- 172.21.15.92:0/2230470099 learned_addr learned my addr 172.21.15.92:0/2230470099 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:23:03.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.423+0000 7f1193fff700 1 --2- 172.21.15.92:0/2230470099 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11940b8510 0x7f119406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:03.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.423+0000 7f1192ffd700 1 -- 172.21.15.92:0/2230470099 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3289686050 0 0) 0x7f11940b88e0 con 0x7f11940cd460 2023-11-28T11:23:03.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.423+0000 7f1192ffd700 1 -- 172.21.15.92:0/2230470099 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f117c0035b0 con 0x7f11940cd460 2023-11-28T11:23:03.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.423+0000 7f1192ffd700 1 -- 172.21.15.92:0/2230470099 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4083536503 0 0) 0x7f11940cd8f0 con 0x7f11940d0d60 2023-11-28T11:23:03.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.423+0000 7f1192ffd700 1 -- 172.21.15.92:0/2230470099 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11940b88e0 con 0x7f11940d0d60 2023-11-28T11:23:03.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.423+0000 7f1193fff700 1 -- 172.21.15.92:0/2230470099 >> v1:172.21.15.169:6789/0 conn(0x7f11940d0d60 legacy=0x7f11940d11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:03.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.423+0000 7f1193fff700 1 -- 172.21.15.92:0/2230470099 >> v1:172.21.15.92:6789/0 conn(0x7f11940cd460 legacy=0x7f119406beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:03.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.423+0000 7f1193fff700 1 -- 172.21.15.92:0/2230470099 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11940cfd10 con 0x7f11940b8510 2023-11-28T11:23:03.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.424+0000 7f1193fff700 1 --2- 172.21.15.92:0/2230470099 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11940b8510 0x7f119406b970 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1184009870 tx=0x7f1184009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8306288006add6f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:03.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.424+0000 7f1192ffd700 1 -- 172.21.15.92:0/2230470099 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1184011830 con 0x7f11940b8510 2023-11-28T11:23:03.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.424+0000 7f1192ffd700 1 -- 172.21.15.92:0/2230470099 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1184011990 con 0x7f11940b8510 2023-11-28T11:23:03.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.424+0000 7f1192ffd700 1 -- 172.21.15.92:0/2230470099 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1184011d00 con 0x7f11940b8510 2023-11-28T11:23:03.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.424+0000 7f119a7ff700 1 -- 172.21.15.92:0/2230470099 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11940b8510 msgr2=0x7f119406b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:03.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.424+0000 7f119a7ff700 1 --2- 172.21.15.92:0/2230470099 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11940b8510 0x7f119406b970 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1184009870 tx=0x7f1184009b80 comp rx=0 tx=0).stop 2023-11-28T11:23:03.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.424+0000 7f119a7ff700 1 -- 172.21.15.92:0/2230470099 shutdown_connections 2023-11-28T11:23:03.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.424+0000 7f119a7ff700 1 --2- 172.21.15.92:0/2230470099 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11940b8510 0x7f119406b970 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:03.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.424+0000 7f119a7ff700 1 -- 172.21.15.92:0/2230470099 >> 172.21.15.92:0/2230470099 conn(0x7f11940719a0 msgr2=0x7f1194071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:03.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.424+0000 7f119a7ff700 1 -- 172.21.15.92:0/2230470099 shutdown_connections 2023-11-28T11:23:03.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.425+0000 7f119a7ff700 1 -- 172.21.15.92:0/2230470099 wait complete. 2023-11-28T11:23:03.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.426+0000 7f119a7ff700 1 Processor -- start 2023-11-28T11:23:03.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.426+0000 7f119a7ff700 1 -- start start 2023-11-28T11:23:03.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.427+0000 7f119a7ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11940b8510 0x7f119414a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:03.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.427+0000 7f119a7ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11940cd460 0x7f1194159fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:03.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.427+0000 7f119a7ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11940d0d60 0x7f1194158540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:03.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.427+0000 7f119a7ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1194159b80 con 0x7f11940d0d60 2023-11-28T11:23:03.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.427+0000 7f119a7ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119414a860 con 0x7f11940cd460 2023-11-28T11:23:03.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.427+0000 7f119a7ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1194158f00 con 0x7f11940b8510 2023-11-28T11:23:03.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.428+0000 7f1198d9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11940d0d60 0x7f1194158540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:03.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.428+0000 7f1193fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11940b8510 0x7f119414a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:03.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.428+0000 7f1193fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11940b8510 0x7f119414a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.92:34464/0 (socket says 172.21.15.92:34464) 2023-11-28T11:23:03.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.428+0000 7f1193fff700 1 -- 172.21.15.92:0/3150794093 learned_addr learned my addr 172.21.15.92:0/3150794093 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:03.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.428+0000 7f11937fe700 1 --2- 172.21.15.92:0/3150794093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11940cd460 0x7f1194159fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:03.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.428+0000 7f1193fff700 1 -- 172.21.15.92:0/3150794093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11940cd460 msgr2=0x7f1194159fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:03.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.428+0000 7f1193fff700 1 --2- 172.21.15.92:0/3150794093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11940cd460 0x7f1194159fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:03.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.428+0000 7f1193fff700 1 -- 172.21.15.92:0/3150794093 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11940d0d60 msgr2=0x7f1194158540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:03.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.428+0000 7f1193fff700 1 --2- 172.21.15.92:0/3150794093 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11940d0d60 0x7f1194158540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:03.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.428+0000 7f1193fff700 1 -- 172.21.15.92:0/3150794093 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11840094a0 con 0x7f11940b8510 2023-11-28T11:23:03.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.428+0000 7f1198d9c700 1 --2- 172.21.15.92:0/3150794093 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11940d0d60 0x7f1194158540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:23:03.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.428+0000 7f1193fff700 1 --2- 172.21.15.92:0/3150794093 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11940b8510 0x7f119414a320 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f11840097c0 tx=0x7f1184004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:03.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.429+0000 7f1190ff9700 1 -- 172.21.15.92:0/3150794093 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11840044d0 con 0x7f11940b8510 2023-11-28T11:23:03.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.429+0000 7f1190ff9700 1 -- 172.21.15.92:0/3150794093 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f118401f070 con 0x7f11940b8510 2023-11-28T11:23:03.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.429+0000 7f1190ff9700 1 -- 172.21.15.92:0/3150794093 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11840038e0 con 0x7f11940b8510 2023-11-28T11:23:03.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.429+0000 7f119a7ff700 1 -- 172.21.15.92:0/3150794093 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11940d29c0 con 0x7f11940b8510 2023-11-28T11:23:03.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.429+0000 7f119a7ff700 1 -- 172.21.15.92:0/3150794093 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11940cfff0 con 0x7f11940b8510 2023-11-28T11:23:03.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.430+0000 7f119a7ff700 1 -- 172.21.15.92:0/3150794093 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11940d2b50 con 0x7f11940b8510 2023-11-28T11:23:03.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.431+0000 7f1190ff9700 1 -- 172.21.15.92:0/3150794093 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f118400f050 con 0x7f11940b8510 2023-11-28T11:23:03.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.432+0000 7f1190ff9700 1 --2- 172.21.15.92:0/3150794093 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f11800682f0 0x7f118006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:03.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.432+0000 7f1190ff9700 1 -- 172.21.15.92:0/3150794093 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f1184021070 con 0x7f11940b8510 2023-11-28T11:23:03.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.432+0000 7f11937fe700 1 --2- 172.21.15.92:0/3150794093 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f11800682f0 0x7f118006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:03.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.432+0000 7f11937fe700 1 --2- 172.21.15.92:0/3150794093 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f11800682f0 0x7f118006a7a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1194154cb0 tx=0x7f1188007dc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:03.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.437+0000 7f1190ff9700 1 -- 172.21.15.92:0/3150794093 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1184053c50 con 0x7f11940b8510 2023-11-28T11:23:03.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:03.707+0000 7f119a7ff700 1 -- 172.21.15.92:0/3150794093 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) v1 -- 0x7f1194159df0 con 0x7f11940b8510 2023-11-28T11:23:04.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.715+0000 7f1190ff9700 1 -- 172.21.15.92:0/3150794093 <== mon.2 v2:172.21.15.169:3301/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v140) v1 ==== 142+0+0 (secure 0 0 0) 0x7f1184057270 con 0x7f11940b8510 2023-11-28T11:23:04.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.718+0000 7f119a7ff700 1 -- 172.21.15.92:0/3150794093 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f11800682f0 msgr2=0x7f118006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:04.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.718+0000 7f119a7ff700 1 --2- 172.21.15.92:0/3150794093 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f11800682f0 0x7f118006a7a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1194154cb0 tx=0x7f1188007dc0 comp rx=0 tx=0).stop 2023-11-28T11:23:04.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.718+0000 7f119a7ff700 1 -- 172.21.15.92:0/3150794093 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11940b8510 msgr2=0x7f119414a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:04.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.718+0000 7f119a7ff700 1 --2- 172.21.15.92:0/3150794093 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11940b8510 0x7f119414a320 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f11840097c0 tx=0x7f1184004750 comp rx=0 tx=0).stop 2023-11-28T11:23:04.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.718+0000 7f119a7ff700 1 -- 172.21.15.92:0/3150794093 shutdown_connections 2023-11-28T11:23:04.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.718+0000 7f119a7ff700 1 --2- 172.21.15.92:0/3150794093 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f11800682f0 0x7f118006a7a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:04.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.718+0000 7f119a7ff700 1 --2- 172.21.15.92:0/3150794093 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11940d0d60 0x7f1194158540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:04.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.718+0000 7f119a7ff700 1 --2- 172.21.15.92:0/3150794093 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f11940b8510 0x7f119414a320 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:04.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.718+0000 7f119a7ff700 1 --2- 172.21.15.92:0/3150794093 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f11940cd460 0x7f1194159fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:04.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.718+0000 7f119a7ff700 1 -- 172.21.15.92:0/3150794093 >> 172.21.15.92:0/3150794093 conn(0x7f11940719a0 msgr2=0x7f11940b3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:04.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.718+0000 7f119a7ff700 1 -- 172.21.15.92:0/3150794093 shutdown_connections 2023-11-28T11:23:04.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.718+0000 7f119a7ff700 1 -- 172.21.15.92:0/3150794093 wait complete. 2023-11-28T11:23:04.741 INFO:tasks.cephfs_test_runner:test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2023-11-28T11:23:04.742 INFO:tasks.cephfs_test_runner:Starting test: test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) That the MDS can repair an inodes backtrace in the data pool 2023-11-28T11:23:04.742 INFO:tasks.cephfs_test_runner:test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T11:23:04.742 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2023-11-28T11:23:04.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.940+0000 7f5b94dff700 1 Processor -- start 2023-11-28T11:23:04.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.940+0000 7f5b94dff700 1 -- start start 2023-11-28T11:23:04.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.941+0000 7f5b94dff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b900b8510 0x7f5b9006b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:04.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.942+0000 7f5b94dff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5b900cd460 0x7f5b9006beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:04.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.942+0000 7f5b94dff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b900b88e0 con 0x7f5b9006c3f0 2023-11-28T11:23:04.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.942+0000 7f5b94dff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b900cdbd0 con 0x7f5b900cd460 2023-11-28T11:23:04.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.942+0000 7f5b94dff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9006c830 con 0x7f5b900b8510 2023-11-28T11:23:04.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.942+0000 7f5b8ed9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f5b9006c3f0 0x7f5b9015cc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46120/0 (socket says 172.21.15.92:46120) 2023-11-28T11:23:04.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.942+0000 7f5b8ed9d700 1 -- 172.21.15.92:0/2441462645 learned_addr learned my addr 172.21.15.92:0/2441462645 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:23:04.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.942+0000 7f5b85d9b700 1 --2- 172.21.15.92:0/2441462645 >> v2:172.21.15.169:3300/0 conn(0x7f5b900cd460 0x7f5b9006beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:04.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.942+0000 7f5b8e59c700 1 --2- 172.21.15.92:0/2441462645 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b900b8510 0x7f5b9006b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:04.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.942+0000 7f5b8dd9b700 1 -- 172.21.15.92:0/2441462645 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4160954773 0 0) 0x7f5b900b88e0 con 0x7f5b9006c3f0 2023-11-28T11:23:04.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.942+0000 7f5b8dd9b700 1 -- 172.21.15.92:0/2441462645 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b7c0035b0 con 0x7f5b9006c3f0 2023-11-28T11:23:04.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.943+0000 7f5b85d9b700 1 -- 172.21.15.92:0/2441462645 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b900b8510 msgr2=0x7f5b9006b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:04.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.943+0000 7f5b85d9b700 1 --2- 172.21.15.92:0/2441462645 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b900b8510 0x7f5b9006b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:04.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.943+0000 7f5b85d9b700 1 -- 172.21.15.92:0/2441462645 >> v1:172.21.15.92:6789/0 conn(0x7f5b9006c3f0 legacy=0x7f5b9015cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:04.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.943+0000 7f5b85d9b700 1 -- 172.21.15.92:0/2441462645 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b900cd8f0 con 0x7f5b900cd460 2023-11-28T11:23:04.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.943+0000 7f5b85d9b700 1 --2- 172.21.15.92:0/2441462645 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b900cd460 0x7f5b9006beb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5b78009dc0 tx=0x7f5b7800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebf7f786995b248f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:04.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.943+0000 7f5b8dd9b700 1 -- 172.21.15.92:0/2441462645 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7800e070 con 0x7f5b900cd460 2023-11-28T11:23:04.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.943+0000 7f5b8dd9b700 1 -- 172.21.15.92:0/2441462645 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b7800bae0 con 0x7f5b900cd460 2023-11-28T11:23:04.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.943+0000 7f5b8dd9b700 1 -- 172.21.15.92:0/2441462645 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7800be90 con 0x7f5b900cd460 2023-11-28T11:23:04.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.944+0000 7f5b94dff700 1 -- 172.21.15.92:0/2441462645 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b900cd460 msgr2=0x7f5b9006beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:04.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.944+0000 7f5b94dff700 1 --2- 172.21.15.92:0/2441462645 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b900cd460 0x7f5b9006beb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5b78009dc0 tx=0x7f5b7800b330 comp rx=0 tx=0).stop 2023-11-28T11:23:04.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.944+0000 7f5b94dff700 1 -- 172.21.15.92:0/2441462645 shutdown_connections 2023-11-28T11:23:04.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.944+0000 7f5b94dff700 1 --2- 172.21.15.92:0/2441462645 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b900b8510 0x7f5b9006b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:04.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.944+0000 7f5b94dff700 1 --2- 172.21.15.92:0/2441462645 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b900cd460 0x7f5b9006beb0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:04.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.944+0000 7f5b94dff700 1 -- 172.21.15.92:0/2441462645 >> 172.21.15.92:0/2441462645 conn(0x7f5b900719a0 msgr2=0x7f5b90071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:04.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.944+0000 7f5b94dff700 1 -- 172.21.15.92:0/2441462645 shutdown_connections 2023-11-28T11:23:04.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.944+0000 7f5b94dff700 1 -- 172.21.15.92:0/2441462645 wait complete. 2023-11-28T11:23:04.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.946+0000 7f5b94dff700 1 Processor -- start 2023-11-28T11:23:04.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.946+0000 7f5b94dff700 1 -- start start 2023-11-28T11:23:04.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.947+0000 7f5b94dff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b9006c3f0 0x7f5b900ad320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:04.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.947+0000 7f5b94dff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b900b8510 0x7f5b900b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:04.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.947+0000 7f5b94dff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b900cd460 0x7f5b90152750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:04.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.947+0000 7f5b94dff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b901536c0 con 0x7f5b900b8510 2023-11-28T11:23:04.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.947+0000 7f5b94dff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b90152c90 con 0x7f5b9006c3f0 2023-11-28T11:23:04.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.947+0000 7f5b94dff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b900aed80 con 0x7f5b900cd460 2023-11-28T11:23:04.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.947+0000 7f5b85d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b900b8510 0x7f5b900b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:04.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.947+0000 7f5b85d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b900b8510 0x7f5b900b7a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57442/0 (socket says 172.21.15.92:57442) 2023-11-28T11:23:04.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.947+0000 7f5b85d9b700 1 -- 172.21.15.92:0/361385420 learned_addr learned my addr 172.21.15.92:0/361385420 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:04.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.947+0000 7f5b8e59c700 1 --2- 172.21.15.92:0/361385420 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b9006c3f0 0x7f5b900ad320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:04.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.947+0000 7f5b8ed9d700 1 --2- 172.21.15.92:0/361385420 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b900cd460 0x7f5b90152750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:04.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.947+0000 7f5b85d9b700 1 -- 172.21.15.92:0/361385420 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b900cd460 msgr2=0x7f5b90152750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:04.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.947+0000 7f5b85d9b700 1 --2- 172.21.15.92:0/361385420 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b900cd460 0x7f5b90152750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:04.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.947+0000 7f5b85d9b700 1 -- 172.21.15.92:0/361385420 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b9006c3f0 msgr2=0x7f5b900ad320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:04.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.947+0000 7f5b85d9b700 1 --2- 172.21.15.92:0/361385420 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b9006c3f0 0x7f5b900ad320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:04.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.947+0000 7f5b85d9b700 1 -- 172.21.15.92:0/361385420 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b78009a70 con 0x7f5b900b8510 2023-11-28T11:23:04.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.948+0000 7f5b85d9b700 1 --2- 172.21.15.92:0/361385420 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b900b8510 0x7f5b900b7a40 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f5b78000f80 tx=0x7f5b78004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:04.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.948+0000 7f5b877fe700 1 -- 172.21.15.92:0/361385420 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b78004550 con 0x7f5b900b8510 2023-11-28T11:23:04.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.948+0000 7f5b877fe700 1 -- 172.21.15.92:0/361385420 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b78020070 con 0x7f5b900b8510 2023-11-28T11:23:04.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.948+0000 7f5b94dff700 1 -- 172.21.15.92:0/361385420 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b900cfff0 con 0x7f5b900b8510 2023-11-28T11:23:04.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.948+0000 7f5b877fe700 1 -- 172.21.15.92:0/361385420 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b78004910 con 0x7f5b900b8510 2023-11-28T11:23:04.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.948+0000 7f5b94dff700 1 -- 172.21.15.92:0/361385420 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b900cfd10 con 0x7f5b900b8510 2023-11-28T11:23:04.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.949+0000 7f5b94dff700 1 -- 172.21.15.92:0/361385420 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b90067880 con 0x7f5b900b8510 2023-11-28T11:23:04.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.950+0000 7f5b877fe700 1 -- 172.21.15.92:0/361385420 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5b78027480 con 0x7f5b900b8510 2023-11-28T11:23:04.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.951+0000 7f5b877fe700 1 --2- 172.21.15.92:0/361385420 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5b700682f0 0x7f5b7006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:04.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.951+0000 7f5b877fe700 1 -- 172.21.15.92:0/361385420 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f5b78022070 con 0x7f5b900b8510 2023-11-28T11:23:04.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.951+0000 7f5b8e59c700 1 --2- 172.21.15.92:0/361385420 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5b700682f0 0x7f5b7006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:04.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.951+0000 7f5b8e59c700 1 --2- 172.21.15.92:0/361385420 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5b700682f0 0x7f5b7006a7a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5b80009be0 tx=0x7f5b80009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:04.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:04.955+0000 7f5b877fe700 1 -- 172.21.15.92:0/361385420 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5b78054aa0 con 0x7f5b900b8510 2023-11-28T11:23:05.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.220+0000 7f5b94dff700 1 -- 172.21.15.92:0/361385420 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) v1 -- 0x7f5b900d2940 con 0x7f5b900b8510 2023-11-28T11:23:05.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.713+0000 7f5b877fe700 1 -- 172.21.15.92:0/361385420 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v141) v1 ==== 152+0+0 (secure 0 0 0) 0x7f5b780580c0 con 0x7f5b900b8510 2023-11-28T11:23:05.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.715+0000 7f5b94dff700 1 -- 172.21.15.92:0/361385420 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5b700682f0 msgr2=0x7f5b7006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:05.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.715+0000 7f5b94dff700 1 --2- 172.21.15.92:0/361385420 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5b700682f0 0x7f5b7006a7a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5b80009be0 tx=0x7f5b80009210 comp rx=0 tx=0).stop 2023-11-28T11:23:05.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.715+0000 7f5b94dff700 1 -- 172.21.15.92:0/361385420 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b900b8510 msgr2=0x7f5b900b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:05.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.715+0000 7f5b94dff700 1 --2- 172.21.15.92:0/361385420 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b900b8510 0x7f5b900b7a40 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f5b78000f80 tx=0x7f5b78004ad0 comp rx=0 tx=0).stop 2023-11-28T11:23:05.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.716+0000 7f5b94dff700 1 -- 172.21.15.92:0/361385420 shutdown_connections 2023-11-28T11:23:05.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.716+0000 7f5b94dff700 1 --2- 172.21.15.92:0/361385420 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b900cd460 0x7f5b90152750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:05.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.716+0000 7f5b94dff700 1 --2- 172.21.15.92:0/361385420 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5b9006c3f0 0x7f5b900ad320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:05.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.716+0000 7f5b94dff700 1 --2- 172.21.15.92:0/361385420 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5b700682f0 0x7f5b7006a7a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:05.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.716+0000 7f5b94dff700 1 --2- 172.21.15.92:0/361385420 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b900b8510 0x7f5b900b7a40 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:05.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.716+0000 7f5b94dff700 1 -- 172.21.15.92:0/361385420 >> 172.21.15.92:0/361385420 conn(0x7f5b900719a0 msgr2=0x7f5b90071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:05.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.716+0000 7f5b94dff700 1 -- 172.21.15.92:0/361385420 shutdown_connections 2023-11-28T11:23:05.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.716+0000 7f5b94dff700 1 -- 172.21.15.92:0/361385420 wait complete. 2023-11-28T11:23:05.737 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-28T11:23:05.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.934+0000 7fd699dff700 1 Processor -- start 2023-11-28T11:23:05.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.934+0000 7fd699dff700 1 -- start start 2023-11-28T11:23:05.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.936+0000 7fd699dff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6940b8510 0x7fd69406b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:05.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.936+0000 7fd699dff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd6940cd460 0x7fd69406beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:05.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.936+0000 7fd699dff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd69406c3f0 0x7fd69415cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:05.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.936+0000 7fd699dff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6940cdbd0 con 0x7fd6940cd460 2023-11-28T11:23:05.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.936+0000 7fd699dff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd69406c830 con 0x7fd69406c3f0 2023-11-28T11:23:05.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.936+0000 7fd699dff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6940d02d0 con 0x7fd6940b8510 2023-11-28T11:23:05.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.936+0000 7fd68bfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd6940cd460 0x7fd69406beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:05.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.936+0000 7fd68bfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd6940cd460 0x7fd69406beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57452/0 (socket says 172.21.15.92:57452) 2023-11-28T11:23:05.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.936+0000 7fd68bfff700 1 -- 172.21.15.92:0/2319634255 learned_addr learned my addr 172.21.15.92:0/2319634255 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:05.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.936+0000 7fd693fff700 1 --2- 172.21.15.92:0/2319634255 >> v2:172.21.15.169:3300/0 conn(0x7fd69406c3f0 0x7fd69415cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:05.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.936+0000 7fd6937fe700 1 --2- 172.21.15.92:0/2319634255 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6940b8510 0x7fd69406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:05.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.936+0000 7fd68bfff700 1 -- 172.21.15.92:0/2319634255 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6940b8510 msgr2=0x7fd69406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:05.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.936+0000 7fd68bfff700 1 --2- 172.21.15.92:0/2319634255 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6940b8510 0x7fd69406b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:05.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.936+0000 7fd68bfff700 1 -- 172.21.15.92:0/2319634255 >> v2:172.21.15.169:3300/0 conn(0x7fd69406c3f0 msgr2=0x7fd69415cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:05.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.936+0000 7fd68bfff700 1 --2- 172.21.15.92:0/2319634255 >> v2:172.21.15.169:3300/0 conn(0x7fd69406c3f0 0x7fd69415cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:05.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.936+0000 7fd68bfff700 1 -- 172.21.15.92:0/2319634255 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6940b88e0 con 0x7fd6940cd460 2023-11-28T11:23:05.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.937+0000 7fd68bfff700 1 --2- 172.21.15.92:0/2319634255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6940cd460 0x7fd69406beb0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fd6940cdd40 tx=0x7fd67c009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48aae3d773fe73ee server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:05.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.937+0000 7fd692ffd700 1 -- 172.21.15.92:0/2319634255 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd67c00b9e0 con 0x7fd6940cd460 2023-11-28T11:23:05.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.937+0000 7fd692ffd700 1 -- 172.21.15.92:0/2319634255 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd67c00bb40 con 0x7fd6940cd460 2023-11-28T11:23:05.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.937+0000 7fd692ffd700 1 -- 172.21.15.92:0/2319634255 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd67c009e70 con 0x7fd6940cd460 2023-11-28T11:23:05.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.937+0000 7fd699dff700 1 -- 172.21.15.92:0/2319634255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6940cd460 msgr2=0x7fd69406beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:05.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.937+0000 7fd699dff700 1 --2- 172.21.15.92:0/2319634255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6940cd460 0x7fd69406beb0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fd6940cdd40 tx=0x7fd67c009c30 comp rx=0 tx=0).stop 2023-11-28T11:23:05.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.938+0000 7fd699dff700 1 -- 172.21.15.92:0/2319634255 shutdown_connections 2023-11-28T11:23:05.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.938+0000 7fd699dff700 1 --2- 172.21.15.92:0/2319634255 >> v2:172.21.15.169:3300/0 conn(0x7fd69406c3f0 0x7fd69415cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:05.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.938+0000 7fd699dff700 1 --2- 172.21.15.92:0/2319634255 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6940b8510 0x7fd69406b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:05.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.938+0000 7fd699dff700 1 --2- 172.21.15.92:0/2319634255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6940cd460 0x7fd69406beb0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:05.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.938+0000 7fd699dff700 1 -- 172.21.15.92:0/2319634255 >> 172.21.15.92:0/2319634255 conn(0x7fd6940719a0 msgr2=0x7fd694071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:05.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.938+0000 7fd699dff700 1 -- 172.21.15.92:0/2319634255 shutdown_connections 2023-11-28T11:23:05.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.938+0000 7fd699dff700 1 -- 172.21.15.92:0/2319634255 wait complete. 2023-11-28T11:23:05.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.939+0000 7fd699dff700 1 Processor -- start 2023-11-28T11:23:05.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.939+0000 7fd699dff700 1 -- start start 2023-11-28T11:23:05.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.940+0000 7fd699dff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd69406c3f0 0x7fd694155f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:05.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.940+0000 7fd699dff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6940b8510 0x7fd6941e1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:05.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.940+0000 7fd699dff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6940cd460 0x7fd69415a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:05.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.940+0000 7fd699dff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6941e2350 con 0x7fd69406c3f0 2023-11-28T11:23:05.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.940+0000 7fd699dff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6941e2630 con 0x7fd6940b8510 2023-11-28T11:23:05.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.940+0000 7fd699dff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6941e2910 con 0x7fd6940cd460 2023-11-28T11:23:05.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.941+0000 7fd6937fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd69406c3f0 0x7fd694155f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:05.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.941+0000 7fd6937fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd69406c3f0 0x7fd694155f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57458/0 (socket says 172.21.15.92:57458) 2023-11-28T11:23:05.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.941+0000 7fd6937fe700 1 -- 172.21.15.92:0/3817046541 learned_addr learned my addr 172.21.15.92:0/3817046541 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:05.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.941+0000 7fd68bfff700 1 --2- 172.21.15.92:0/3817046541 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6940b8510 0x7fd6941e1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:05.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.941+0000 7fd693fff700 1 --2- 172.21.15.92:0/3817046541 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6940cd460 0x7fd69415a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:05.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.941+0000 7fd6937fe700 1 -- 172.21.15.92:0/3817046541 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6940cd460 msgr2=0x7fd69415a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:05.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.941+0000 7fd6937fe700 1 --2- 172.21.15.92:0/3817046541 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6940cd460 0x7fd69415a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:05.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.941+0000 7fd6937fe700 1 -- 172.21.15.92:0/3817046541 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6940b8510 msgr2=0x7fd6941e1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:05.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.941+0000 7fd6937fe700 1 --2- 172.21.15.92:0/3817046541 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6940b8510 0x7fd6941e1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:05.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.941+0000 7fd6937fe700 1 -- 172.21.15.92:0/3817046541 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd67c009550 con 0x7fd69406c3f0 2023-11-28T11:23:05.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.941+0000 7fd693fff700 1 --2- 172.21.15.92:0/3817046541 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6940cd460 0x7fd69415a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:23:05.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.941+0000 7fd6937fe700 1 --2- 172.21.15.92:0/3817046541 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd69406c3f0 0x7fd694155f20 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fd68400e810 tx=0x7fd68400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:05.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.942+0000 7fd690ff9700 1 -- 172.21.15.92:0/3817046541 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd684013610 con 0x7fd69406c3f0 2023-11-28T11:23:05.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.942+0000 7fd690ff9700 1 -- 172.21.15.92:0/3817046541 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd684013770 con 0x7fd69406c3f0 2023-11-28T11:23:05.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.942+0000 7fd699dff700 1 -- 172.21.15.92:0/3817046541 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6940cfd10 con 0x7fd69406c3f0 2023-11-28T11:23:05.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.942+0000 7fd690ff9700 1 -- 172.21.15.92:0/3817046541 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd684013b20 con 0x7fd69406c3f0 2023-11-28T11:23:05.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.942+0000 7fd699dff700 1 -- 172.21.15.92:0/3817046541 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6940cd8f0 con 0x7fd69406c3f0 2023-11-28T11:23:05.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.943+0000 7fd699dff700 1 -- 172.21.15.92:0/3817046541 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6940cfff0 con 0x7fd69406c3f0 2023-11-28T11:23:05.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.944+0000 7fd690ff9700 1 -- 172.21.15.92:0/3817046541 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd684023070 con 0x7fd69406c3f0 2023-11-28T11:23:05.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.944+0000 7fd690ff9700 1 --2- 172.21.15.92:0/3817046541 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd6740682f0 0x7fd67406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:05.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.944+0000 7fd690ff9700 1 -- 172.21.15.92:0/3817046541 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fd684085800 con 0x7fd69406c3f0 2023-11-28T11:23:05.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.945+0000 7fd68bfff700 1 --2- 172.21.15.92:0/3817046541 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd6740682f0 0x7fd67406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:05.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.945+0000 7fd68bfff700 1 --2- 172.21.15.92:0/3817046541 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd6740682f0 0x7fd67406a7a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd67c009870 tx=0x7fd67c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:05.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:05.949+0000 7fd690ff9700 1 -- 172.21.15.92:0/3817046541 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd6840520d0 con 0x7fd69406c3f0 2023-11-28T11:23:06.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.137+0000 7fd699dff700 1 -- 172.21.15.92:0/3817046541 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fd6941e3240 con 0x7fd69406c3f0 2023-11-28T11:23:06.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.141+0000 7fd690ff9700 1 -- 172.21.15.92:0/3817046541 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd6840556f0 con 0x7fd69406c3f0 2023-11-28T11:23:06.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.144+0000 7fd699dff700 1 -- 172.21.15.92:0/3817046541 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd6740682f0 msgr2=0x7fd67406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:06.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.144+0000 7fd699dff700 1 --2- 172.21.15.92:0/3817046541 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd6740682f0 0x7fd67406a7a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd67c009870 tx=0x7fd67c008030 comp rx=0 tx=0).stop 2023-11-28T11:23:06.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.144+0000 7fd699dff700 1 -- 172.21.15.92:0/3817046541 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd69406c3f0 msgr2=0x7fd694155f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:06.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.144+0000 7fd699dff700 1 --2- 172.21.15.92:0/3817046541 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd69406c3f0 0x7fd694155f20 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fd68400e810 tx=0x7fd68400eb20 comp rx=0 tx=0).stop 2023-11-28T11:23:06.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.144+0000 7fd699dff700 1 -- 172.21.15.92:0/3817046541 shutdown_connections 2023-11-28T11:23:06.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.144+0000 7fd699dff700 1 --2- 172.21.15.92:0/3817046541 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6940cd460 0x7fd69415a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:06.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.144+0000 7fd699dff700 1 --2- 172.21.15.92:0/3817046541 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd6740682f0 0x7fd67406a7a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:06.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.144+0000 7fd699dff700 1 --2- 172.21.15.92:0/3817046541 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd69406c3f0 0x7fd694155f20 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:06.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.144+0000 7fd699dff700 1 --2- 172.21.15.92:0/3817046541 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6940b8510 0x7fd6941e1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:06.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.144+0000 7fd699dff700 1 -- 172.21.15.92:0/3817046541 >> 172.21.15.92:0/3817046541 conn(0x7fd6940719a0 msgr2=0x7fd6940b3fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:06.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.144+0000 7fd699dff700 1 -- 172.21.15.92:0/3817046541 shutdown_connections 2023-11-28T11:23:06.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.144+0000 7fd699dff700 1 -- 172.21.15.92:0/3817046541 wait complete. 2023-11-28T11:23:06.164 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T11:23:06.164 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:23:06.251 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:23:06.349 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T11:23:06.350 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-28T11:23:06.350 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:23:06.397 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348288k,nr_inodes=4087072,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275616k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:23:06.397 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-28T11:23:06.398 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:23:06.446 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348296k,nr_inodes=4087074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275620k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:23:06.446 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-28T11:23:06.446 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:23:06.493 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348288k,nr_inodes=4087072,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275616k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:23:06.494 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-28T11:23:06.494 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:23:06.541 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348296k,nr_inodes=4087074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275620k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:23:06.541 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-28T11:23:06.542 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:23:06.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.736+0000 7fae9d9ff700 1 Processor -- start 2023-11-28T11:23:06.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.736+0000 7fae9d9ff700 1 -- start start 2023-11-28T11:23:06.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.738+0000 7fae9d9ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae98159080 0x7fae980ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:06.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.738+0000 7fae9d9ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae9806d870 con 0x7fae98158cb0 2023-11-28T11:23:06.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.738+0000 7fae9d9ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae9806cc60 con 0x7fae98071020 2023-11-28T11:23:06.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.738+0000 7fae9d9ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae9806cf40 con 0x7fae98159080 2023-11-28T11:23:06.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.738+0000 7fae96ffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fae98158cb0 0x7fae98070910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46128/0 (socket says 172.21.15.92:46128) 2023-11-28T11:23:06.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.738+0000 7fae96ffd700 1 -- 172.21.15.92:0/2060551255 learned_addr learned my addr 172.21.15.92:0/2060551255 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:23:06.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.738+0000 7fae8ffff700 1 --2- 172.21.15.92:0/2060551255 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae98159080 0x7fae980ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:06.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.738+0000 7fae967fc700 1 -- 172.21.15.92:0/2060551255 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1082246318 0 0) 0x7fae9806d870 con 0x7fae98158cb0 2023-11-28T11:23:06.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.738+0000 7fae967fc700 1 -- 172.21.15.92:0/2060551255 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae840034d0 con 0x7fae98158cb0 2023-11-28T11:23:06.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.739+0000 7fae967fc700 1 -- 172.21.15.92:0/2060551255 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3568001140 0 0) 0x7fae9806cc60 con 0x7fae98071020 2023-11-28T11:23:06.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.739+0000 7fae967fc700 1 -- 172.21.15.92:0/2060551255 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae9806d870 con 0x7fae98071020 2023-11-28T11:23:06.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.739+0000 7fae967fc700 1 -- 172.21.15.92:0/2060551255 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1032345188 0 0) 0x7fae840034d0 con 0x7fae98158cb0 2023-11-28T11:23:06.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.739+0000 7fae967fc700 1 -- 172.21.15.92:0/2060551255 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae98159080 msgr2=0x7fae980ca1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:06.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.739+0000 7fae967fc700 1 --2- 172.21.15.92:0/2060551255 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae98159080 0x7fae980ca1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:06.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.739+0000 7fae967fc700 1 -- 172.21.15.92:0/2060551255 >> v1:172.21.15.169:6789/0 conn(0x7fae98071020 legacy=0x7fae980cc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:06.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.739+0000 7fae967fc700 1 -- 172.21.15.92:0/2060551255 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae9806d220 con 0x7fae98158cb0 2023-11-28T11:23:06.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.739+0000 7fae967fc700 1 -- 172.21.15.92:0/2060551255 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532161587 0 0) 0x7fae88001670 con 0x7fae98158cb0 2023-11-28T11:23:06.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.739+0000 7fae967fc700 1 -- 172.21.15.92:0/2060551255 >> v1:172.21.15.92:6789/0 conn(0x7fae98158cb0 legacy=0x7fae98070910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:06.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.739+0000 7fae967fc700 1 --2- 172.21.15.92:0/2060551255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae84003cb0 0x7fae840040a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:06.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.739+0000 7fae967fc700 1 --2- 172.21.15.92:0/2060551255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae84008a10 0x7fae8400ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:06.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.739+0000 7fae967fc700 1 --2- 172.21.15.92:0/2060551255 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae8400b500 0x7fae8400d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:06.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.739+0000 7fae967fc700 1 -- 172.21.15.92:0/2060551255 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae9806cf40 con 0x7fae84008a10 2023-11-28T11:23:06.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.739+0000 7fae967fc700 1 -- 172.21.15.92:0/2060551255 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae8400e080 con 0x7fae84003cb0 2023-11-28T11:23:06.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.739+0000 7fae967fc700 1 -- 172.21.15.92:0/2060551255 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae8400e240 con 0x7fae8400b500 2023-11-28T11:23:06.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.739+0000 7fae8ffff700 1 --2- 172.21.15.92:0/2060551255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae84008a10 0x7fae8400ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:06.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.739+0000 7fae96ffd700 1 --2- 172.21.15.92:0/2060551255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae84003cb0 0x7fae840040a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:06.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.739+0000 7fae977fe700 1 --2- 172.21.15.92:0/2060551255 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae8400b500 0x7fae8400d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:06.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.739+0000 7fae8ffff700 1 -- 172.21.15.92:0/2060551255 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae8400b500 msgr2=0x7fae8400d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:06.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.739+0000 7fae8ffff700 1 --2- 172.21.15.92:0/2060551255 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae8400b500 0x7fae8400d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:06.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.739+0000 7fae8ffff700 1 -- 172.21.15.92:0/2060551255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae84003cb0 msgr2=0x7fae840040a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:06.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.739+0000 7fae8ffff700 1 --2- 172.21.15.92:0/2060551255 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae84003cb0 0x7fae840040a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:06.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.739+0000 7fae8ffff700 1 -- 172.21.15.92:0/2060551255 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae840034d0 con 0x7fae84008a10 2023-11-28T11:23:06.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.740+0000 7fae8ffff700 1 --2- 172.21.15.92:0/2060551255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae84008a10 0x7fae8400ae90 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fae80009b60 tx=0x7fae84003780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=777f806dc9952c9e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:06.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.740+0000 7fae967fc700 1 -- 172.21.15.92:0/2060551255 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8000e6c0 con 0x7fae84008a10 2023-11-28T11:23:06.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.740+0000 7fae967fc700 1 -- 172.21.15.92:0/2060551255 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae8000e820 con 0x7fae84008a10 2023-11-28T11:23:06.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.740+0000 7fae967fc700 1 -- 172.21.15.92:0/2060551255 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8000eb90 con 0x7fae84008a10 2023-11-28T11:23:06.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.740+0000 7fae9d9ff700 1 -- 172.21.15.92:0/2060551255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae84008a10 msgr2=0x7fae8400ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:06.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.740+0000 7fae9d9ff700 1 --2- 172.21.15.92:0/2060551255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae84008a10 0x7fae8400ae90 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fae80009b60 tx=0x7fae84003780 comp rx=0 tx=0).stop 2023-11-28T11:23:06.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.740+0000 7fae977fe700 1 -- 172.21.15.92:0/2060551255 reap_dead start 2023-11-28T11:23:06.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.740+0000 7fae9d9ff700 1 -- 172.21.15.92:0/2060551255 shutdown_connections 2023-11-28T11:23:06.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.740+0000 7fae9d9ff700 1 -- 172.21.15.92:0/2060551255 >> 172.21.15.92:0/2060551255 conn(0x7fae980cb490 msgr2=0x7fae980cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:06.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.740+0000 7fae9d9ff700 1 -- 172.21.15.92:0/2060551255 shutdown_connections 2023-11-28T11:23:06.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.741+0000 7fae9d9ff700 1 -- 172.21.15.92:0/2060551255 wait complete. 2023-11-28T11:23:06.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.742+0000 7fae9d9ff700 1 Processor -- start 2023-11-28T11:23:06.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.742+0000 7fae9d9ff700 1 -- start start 2023-11-28T11:23:06.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.743+0000 7fae9d9ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae84008a10 0x7fae98156140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:06.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.743+0000 7fae9d9ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae980c7fd0 0x7fae980c8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:06.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.743+0000 7fae9d9ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae980c5ac0 0x7fae980c5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:06.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.743+0000 7fae9d9ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae980c74b0 con 0x7fae980c7fd0 2023-11-28T11:23:06.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.743+0000 7fae9d9ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae980c8980 con 0x7fae84008a10 2023-11-28T11:23:06.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.743+0000 7fae9d9ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae98156680 con 0x7fae980c5ac0 2023-11-28T11:23:06.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.743+0000 7fae8ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae980c7fd0 0x7fae980c8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:06.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.743+0000 7fae8ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae980c7fd0 0x7fae980c8440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57486/0 (socket says 172.21.15.92:57486) 2023-11-28T11:23:06.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.744+0000 7fae8ffff700 1 -- 172.21.15.92:0/810893798 learned_addr learned my addr 172.21.15.92:0/810893798 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:06.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.744+0000 7fae96ffd700 1 --2- 172.21.15.92:0/810893798 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae84008a10 0x7fae98156140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:06.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.744+0000 7fae977fe700 1 --2- 172.21.15.92:0/810893798 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae980c5ac0 0x7fae980c5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:06.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.744+0000 7fae8ffff700 1 -- 172.21.15.92:0/810893798 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae980c5ac0 msgr2=0x7fae980c5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:06.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.744+0000 7fae8ffff700 1 --2- 172.21.15.92:0/810893798 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae980c5ac0 0x7fae980c5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:06.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.744+0000 7fae8ffff700 1 -- 172.21.15.92:0/810893798 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae84008a10 msgr2=0x7fae98156140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:06.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.744+0000 7fae8ffff700 1 --2- 172.21.15.92:0/810893798 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae84008a10 0x7fae98156140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:06.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.744+0000 7fae8ffff700 1 -- 172.21.15.92:0/810893798 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae98071b20 con 0x7fae980c7fd0 2023-11-28T11:23:06.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.744+0000 7fae8ffff700 1 --2- 172.21.15.92:0/810893798 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae980c7fd0 0x7fae980c8440 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fae8400e050 tx=0x7fae800047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:06.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.744+0000 7fae94ff9700 1 -- 172.21.15.92:0/810893798 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8000e6c0 con 0x7fae980c7fd0 2023-11-28T11:23:06.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.744+0000 7fae94ff9700 1 -- 172.21.15.92:0/810893798 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae80003ab0 con 0x7fae980c7fd0 2023-11-28T11:23:06.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.744+0000 7fae9d9ff700 1 -- 172.21.15.92:0/810893798 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae9806d500 con 0x7fae980c7fd0 2023-11-28T11:23:06.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.744+0000 7fae94ff9700 1 -- 172.21.15.92:0/810893798 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae80003e60 con 0x7fae980c7fd0 2023-11-28T11:23:06.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.745+0000 7fae9d9ff700 1 -- 172.21.15.92:0/810893798 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae980c9300 con 0x7fae980c7fd0 2023-11-28T11:23:06.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.745+0000 7fae9d9ff700 1 -- 172.21.15.92:0/810893798 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae980c22e0 con 0x7fae980c7fd0 2023-11-28T11:23:06.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.746+0000 7fae94ff9700 1 -- 172.21.15.92:0/810893798 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fae80019030 con 0x7fae980c7fd0 2023-11-28T11:23:06.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.747+0000 7fae94ff9700 1 --2- 172.21.15.92:0/810893798 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fae780682f0 0x7fae7806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:06.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.747+0000 7fae94ff9700 1 -- 172.21.15.92:0/810893798 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fae800870d0 con 0x7fae980c7fd0 2023-11-28T11:23:06.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.747+0000 7fae96ffd700 1 --2- 172.21.15.92:0/810893798 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fae780682f0 0x7fae7806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:06.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.748+0000 7fae96ffd700 1 --2- 172.21.15.92:0/810893798 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fae780682f0 0x7fae7806a7a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fae88007cf0 tx=0x7fae88000ed0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:06.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.751+0000 7fae94ff9700 1 -- 172.21.15.92:0/810893798 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fae980c22e0 con 0x7fae980c7fd0 2023-11-28T11:23:06.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.987+0000 7fae9d9ff700 1 -- 172.21.15.92:0/810893798 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fae980c22e0 con 0x7fae980c7fd0 2023-11-28T11:23:06.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.988+0000 7fae94ff9700 1 -- 172.21.15.92:0/810893798 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+3526 (secure 0 0 0) 0x7fae80017de0 con 0x7fae980c7fd0 2023-11-28T11:23:06.990 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:23:06.990 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":19,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4976,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/3102603064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3102603064},{"type":"v1","addr":"172.21.15.169:6834","nonce":3102603064}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4979,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/79247121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":79247121},{"type":"v1","addr":"172.21.15.92:6837","nonce":79247121}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5221,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/219797192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":219797192},{"type":"v1","addr":"172.21.15.169:6835","nonce":219797192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5682,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2442221270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2442221270},{"type":"v1","addr":"172.21.15.92:6835","nonce":2442221270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[]} 2023-11-28T11:23:06.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.990+0000 7fae9d9ff700 1 -- 172.21.15.92:0/810893798 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fae780682f0 msgr2=0x7fae7806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:06.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.990+0000 7fae9d9ff700 1 --2- 172.21.15.92:0/810893798 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fae780682f0 0x7fae7806a7a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fae88007cf0 tx=0x7fae88000ed0 comp rx=0 tx=0).stop 2023-11-28T11:23:06.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.990+0000 7fae9d9ff700 1 -- 172.21.15.92:0/810893798 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae980c7fd0 msgr2=0x7fae980c8440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:06.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.990+0000 7fae9d9ff700 1 --2- 172.21.15.92:0/810893798 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae980c7fd0 0x7fae980c8440 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fae8400e050 tx=0x7fae800047c0 comp rx=0 tx=0).stop 2023-11-28T11:23:06.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.991+0000 7fae9d9ff700 1 -- 172.21.15.92:0/810893798 shutdown_connections 2023-11-28T11:23:06.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.991+0000 7fae9d9ff700 1 --2- 172.21.15.92:0/810893798 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fae980c5ac0 0x7fae980c5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:06.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.991+0000 7fae9d9ff700 1 --2- 172.21.15.92:0/810893798 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fae84008a10 0x7fae98156140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:06.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.991+0000 7fae9d9ff700 1 --2- 172.21.15.92:0/810893798 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fae780682f0 0x7fae7806a7a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:06.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.991+0000 7fae9d9ff700 1 --2- 172.21.15.92:0/810893798 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae980c7fd0 0x7fae980c8440 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:06.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.991+0000 7fae9d9ff700 1 -- 172.21.15.92:0/810893798 >> 172.21.15.92:0/810893798 conn(0x7fae980cb490 msgr2=0x7fae980cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:06.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.991+0000 7fae9d9ff700 1 -- 172.21.15.92:0/810893798 shutdown_connections 2023-11-28T11:23:06.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:06.991+0000 7fae9d9ff700 1 -- 172.21.15.92:0/810893798 wait complete. 2023-11-28T11:23:06.994 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 19 2023-11-28T11:23:07.011 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T11:23:07.011 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-28T11:23:07.011 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T11:23:07.011 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:23:07.011 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T11:23:07.011 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-28T11:23:07.012 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T11:23:07.012 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:23:07.012 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T11:23:07.012 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-28T11:23:07.012 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T11:23:07.012 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:23:07.012 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T11:23:07.013 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-28T11:23:07.013 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T11:23:07.013 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:23:07.025 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T11:23:07.026 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T11:23:07.027 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T11:23:07.027 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T11:23:07.028 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T11:23:07.028 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T11:23:07.029 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T11:23:07.029 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T11:23:07.070 INFO:tasks.ceph.mds.a:Started 2023-11-28T11:23:07.071 DEBUG:teuthology.parallel:result is None 2023-11-28T11:23:07.073 INFO:tasks.ceph.mds.b:Started 2023-11-28T11:23:07.073 DEBUG:teuthology.parallel:result is None 2023-11-28T11:23:07.074 INFO:tasks.ceph.mds.c:Started 2023-11-28T11:23:07.074 DEBUG:teuthology.parallel:result is None 2023-11-28T11:23:07.076 INFO:tasks.ceph.mds.d:Started 2023-11-28T11:23:07.076 DEBUG:teuthology.parallel:result is None 2023-11-28T11:23:07.076 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-28T11:23:07.219 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-11-28T11:23:07.222 INFO:tasks.ceph.mds.c.smithi092.stdout:starting mds.c at 2023-11-28T11:23:07.223 INFO:tasks.ceph.mds.b.smithi169.stdout:starting mds.b at 2023-11-28T11:23:07.223 INFO:tasks.ceph.mds.d.smithi169.stdout:starting mds.d at 2023-11-28T11:23:07.230 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2023-11-28T11:23:07.231 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-11-28T11:23:07.231 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2023-11-28T11:23:07.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.289+0000 7f585a3ff700 1 Processor -- start 2023-11-28T11:23:07.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.290+0000 7f585a3ff700 1 -- start start 2023-11-28T11:23:07.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.291+0000 7f585a3ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f585406a980 0x7f58540c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:07.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.291+0000 7f585a3ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f58540ca430 0x7f58540cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:07.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.291+0000 7f585a3ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5854159b10 con 0x7f5854142e80 2023-11-28T11:23:07.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.291+0000 7f585a3ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58540c9380 con 0x7f585406a980 2023-11-28T11:23:07.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.291+0000 7f585a3ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58540ca890 con 0x7f58540ca430 2023-11-28T11:23:07.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.291+0000 7f58537fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f585406a980 0x7f58540c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:07.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.291+0000 7f5853fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f5854142e80 0x7f585406a270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46148/0 (socket says 172.21.15.92:46148) 2023-11-28T11:23:07.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.291+0000 7f5853fff700 1 -- 172.21.15.92:0/1906144045 learned_addr learned my addr 172.21.15.92:0/1906144045 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:23:07.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.291+0000 7f58537fe700 1 --2- 172.21.15.92:0/1906144045 >> v2:172.21.15.92:3300/0 conn(0x7f585406a980 0x7f58540c9ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57546/0 (socket says 172.21.15.92:57546) 2023-11-28T11:23:07.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.291+0000 7f58537fe700 1 -- 172.21.15.92:0/1906144045 >> v2:172.21.15.169:3300/0 conn(0x7f58540ca430 msgr2=0x7f58540cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:07.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.291+0000 7f585899c700 1 --2- 172.21.15.92:0/1906144045 >> v2:172.21.15.169:3300/0 conn(0x7f58540ca430 0x7f58540cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:07.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.291+0000 7f58537fe700 1 --2- 172.21.15.92:0/1906144045 >> v2:172.21.15.169:3300/0 conn(0x7f58540ca430 0x7f58540cd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:07.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.291+0000 7f58537fe700 1 -- 172.21.15.92:0/1906144045 >> v1:172.21.15.92:6789/0 conn(0x7f5854142e80 legacy=0x7f585406a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:07.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.291+0000 7f5852ffd700 1 -- 172.21.15.92:0/1906144045 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1788968168 0 0) 0x7f5854159b10 con 0x7f5854142e80 2023-11-28T11:23:07.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.292+0000 7f58537fe700 1 -- 172.21.15.92:0/1906144045 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5854143250 con 0x7f585406a980 2023-11-28T11:23:07.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.292+0000 7f585899c700 1 --2- 172.21.15.92:0/1906144045 >> v2:172.21.15.169:3300/0 conn(0x7f58540ca430 0x7f58540cd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:23:07.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.292+0000 7f58537fe700 1 --2- 172.21.15.92:0/1906144045 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f585406a980 0x7f58540c9ef0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f58540c5980 tx=0x7f5848009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdb85ffc306c82b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:07.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.292+0000 7f5852ffd700 1 -- 172.21.15.92:0/1906144045 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5848011830 con 0x7f585406a980 2023-11-28T11:23:07.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.292+0000 7f5852ffd700 1 -- 172.21.15.92:0/1906144045 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5848011990 con 0x7f585406a980 2023-11-28T11:23:07.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.292+0000 7f5852ffd700 1 -- 172.21.15.92:0/1906144045 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5848011d20 con 0x7f585406a980 2023-11-28T11:23:07.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.293+0000 7f585a3ff700 1 -- 172.21.15.92:0/1906144045 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f585406a980 msgr2=0x7f58540c9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:07.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.293+0000 7f585a3ff700 1 --2- 172.21.15.92:0/1906144045 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f585406a980 0x7f58540c9ef0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f58540c5980 tx=0x7f5848009b20 comp rx=0 tx=0).stop 2023-11-28T11:23:07.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.293+0000 7f585a3ff700 1 -- 172.21.15.92:0/1906144045 shutdown_connections 2023-11-28T11:23:07.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.293+0000 7f585a3ff700 1 --2- 172.21.15.92:0/1906144045 >> v2:172.21.15.169:3300/0 conn(0x7f58540ca430 0x7f58540cd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:07.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.293+0000 7f585a3ff700 1 --2- 172.21.15.92:0/1906144045 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f585406a980 0x7f58540c9ef0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:07.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.293+0000 7f585a3ff700 1 -- 172.21.15.92:0/1906144045 >> 172.21.15.92:0/1906144045 conn(0x7f58540cb490 msgr2=0x7f58540cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:07.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.293+0000 7f585a3ff700 1 -- 172.21.15.92:0/1906144045 shutdown_connections 2023-11-28T11:23:07.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.293+0000 7f585a3ff700 1 -- 172.21.15.92:0/1906144045 wait complete. 2023-11-28T11:23:07.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.295+0000 7f585a3ff700 1 Processor -- start 2023-11-28T11:23:07.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.295+0000 7f585a3ff700 1 -- start start 2023-11-28T11:23:07.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.296+0000 7f585a3ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f585406a980 0x7f585415b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:07.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.296+0000 7f585a3ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58540ca430 0x7f58540b3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:07.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.297+0000 7f585a3ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5854142e80 0x7f58540b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:07.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.297+0000 7f585a3ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f585415a960 con 0x7f5854142e80 2023-11-28T11:23:07.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.297+0000 7f585a3ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58540c5b80 con 0x7f58540ca430 2023-11-28T11:23:07.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.297+0000 7f585a3ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58540c5e60 con 0x7f585406a980 2023-11-28T11:23:07.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.297+0000 7f585899c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5854142e80 0x7f58540b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:07.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.297+0000 7f585899c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5854142e80 0x7f58540b3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57562/0 (socket says 172.21.15.92:57562) 2023-11-28T11:23:07.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.297+0000 7f585899c700 1 -- 172.21.15.92:0/4258742819 learned_addr learned my addr 172.21.15.92:0/4258742819 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:07.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.297+0000 7f58537fe700 1 --2- 172.21.15.92:0/4258742819 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58540ca430 0x7f58540b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:07.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.297+0000 7f5853fff700 1 --2- 172.21.15.92:0/4258742819 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f585406a980 0x7f585415b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:07.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.297+0000 7f585899c700 1 -- 172.21.15.92:0/4258742819 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f585406a980 msgr2=0x7f585415b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:07.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.297+0000 7f585899c700 1 --2- 172.21.15.92:0/4258742819 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f585406a980 0x7f585415b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:07.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.297+0000 7f585899c700 1 -- 172.21.15.92:0/4258742819 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58540ca430 msgr2=0x7f58540b3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:07.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.297+0000 7f585899c700 1 --2- 172.21.15.92:0/4258742819 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58540ca430 0x7f58540b3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:07.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.297+0000 7f585899c700 1 -- 172.21.15.92:0/4258742819 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58480094a0 con 0x7f5854142e80 2023-11-28T11:23:07.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.297+0000 7f5853fff700 1 --2- 172.21.15.92:0/4258742819 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f585406a980 0x7f585415b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:23:07.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.297+0000 7f585899c700 1 --2- 172.21.15.92:0/4258742819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5854142e80 0x7f58540b3d90 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f584c00cef0 tx=0x7f584c00cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:07.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.298+0000 7f5850ff9700 1 -- 172.21.15.92:0/4258742819 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584c00adb0 con 0x7f5854142e80 2023-11-28T11:23:07.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.298+0000 7f5850ff9700 1 -- 172.21.15.92:0/4258742819 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f584c015440 con 0x7f5854142e80 2023-11-28T11:23:07.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.298+0000 7f585a3ff700 1 -- 172.21.15.92:0/4258742819 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58540cc5f0 con 0x7f5854142e80 2023-11-28T11:23:07.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.298+0000 7f5850ff9700 1 -- 172.21.15.92:0/4258742819 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584c0157f0 con 0x7f5854142e80 2023-11-28T11:23:07.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.298+0000 7f585a3ff700 1 -- 172.21.15.92:0/4258742819 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58540cab00 con 0x7f5854142e80 2023-11-28T11:23:07.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.299+0000 7f585a3ff700 1 -- 172.21.15.92:0/4258742819 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58540cc8d0 con 0x7f5854142e80 2023-11-28T11:23:07.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.300+0000 7f5850ff9700 1 -- 172.21.15.92:0/4258742819 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f584c01c030 con 0x7f5854142e80 2023-11-28T11:23:07.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.301+0000 7f5850ff9700 1 --2- 172.21.15.92:0/4258742819 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f58400682f0 0x7f584006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:07.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.301+0000 7f5850ff9700 1 -- 172.21.15.92:0/4258742819 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f584c085f60 con 0x7f5854142e80 2023-11-28T11:23:07.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.301+0000 7f5853fff700 1 --2- 172.21.15.92:0/4258742819 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f58400682f0 0x7f584006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:07.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.302+0000 7f5853fff700 1 --2- 172.21.15.92:0/4258742819 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f58400682f0 0x7f584006a7a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5844000f60 tx=0x7f5844007d30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:07.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.306+0000 7f5850ff9700 1 -- 172.21.15.92:0/4258742819 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f584c0528b0 con 0x7f5854142e80 2023-11-28T11:23:07.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:07.511+0000 7f585a3ff700 1 -- 172.21.15.92:0/4258742819 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f5854067880 con 0x7f5854142e80 2023-11-28T11:23:08.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.254+0000 7f5850ff9700 1 -- 172.21.15.92:0/4258742819 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v52) v1 ==== 97+0+0 (secure 0 0 0) 0x7f584c01a080 con 0x7f5854142e80 2023-11-28T11:23:08.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.257+0000 7f585a3ff700 1 -- 172.21.15.92:0/4258742819 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f58400682f0 msgr2=0x7f584006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:08.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.257+0000 7f585a3ff700 1 --2- 172.21.15.92:0/4258742819 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f58400682f0 0x7f584006a7a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5844000f60 tx=0x7f5844007d30 comp rx=0 tx=0).stop 2023-11-28T11:23:08.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.257+0000 7f585a3ff700 1 -- 172.21.15.92:0/4258742819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5854142e80 msgr2=0x7f58540b3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:08.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.257+0000 7f585a3ff700 1 --2- 172.21.15.92:0/4258742819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5854142e80 0x7f58540b3d90 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f584c00cef0 tx=0x7f584c00cf20 comp rx=0 tx=0).stop 2023-11-28T11:23:08.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.257+0000 7f585a3ff700 1 -- 172.21.15.92:0/4258742819 shutdown_connections 2023-11-28T11:23:08.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.257+0000 7f585a3ff700 1 --2- 172.21.15.92:0/4258742819 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f58400682f0 0x7f584006a7a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:08.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.257+0000 7f585a3ff700 1 --2- 172.21.15.92:0/4258742819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5854142e80 0x7f58540b3d90 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:08.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.257+0000 7f585a3ff700 1 --2- 172.21.15.92:0/4258742819 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f585406a980 0x7f585415b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:08.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.257+0000 7f585a3ff700 1 --2- 172.21.15.92:0/4258742819 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58540ca430 0x7f58540b3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:08.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.257+0000 7f585a3ff700 1 -- 172.21.15.92:0/4258742819 >> 172.21.15.92:0/4258742819 conn(0x7f58540cb490 msgr2=0x7f585406a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:08.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.257+0000 7f585a3ff700 1 -- 172.21.15.92:0/4258742819 shutdown_connections 2023-11-28T11:23:08.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.257+0000 7f585a3ff700 1 -- 172.21.15.92:0/4258742819 wait complete. 2023-11-28T11:23:08.262 INFO:teuthology.orchestra.run.smithi092.stderr: removed all blocklist entries 2023-11-28T11:23:08.278 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-28T11:23:08.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.474+0000 7f891c1ff700 1 Processor -- start 2023-11-28T11:23:08.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.474+0000 7f891c1ff700 1 -- start start 2023-11-28T11:23:08.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.475+0000 7f891c1ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f89140b8510 0x7f891406b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:08.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.475+0000 7f891c1ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f89140d0d60 0x7f89140d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:08.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.475+0000 7f891c1ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89140b88e0 con 0x7f89140cd460 2023-11-28T11:23:08.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.475+0000 7f891c1ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89140cdbd0 con 0x7f89140d0d60 2023-11-28T11:23:08.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.475+0000 7f891c1ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89140d02d0 con 0x7f89140b8510 2023-11-28T11:23:08.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.476+0000 7f891a79c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f89140d0d60 0x7f89140d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:08.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.476+0000 7f891979a700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f89140cd460 0x7f891406beb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46162/0 (socket says 172.21.15.92:46162) 2023-11-28T11:23:08.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.476+0000 7f891979a700 1 -- 172.21.15.92:0/4013711717 learned_addr learned my addr 172.21.15.92:0/4013711717 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:23:08.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.476+0000 7f8919f9b700 1 --2- 172.21.15.92:0/4013711717 >> v2:172.21.15.169:3300/0 conn(0x7f89140b8510 0x7f891406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:08.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.476+0000 7f891a79c700 1 -- 172.21.15.92:0/4013711717 >> v2:172.21.15.169:3300/0 conn(0x7f89140b8510 msgr2=0x7f891406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:08.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.476+0000 7f891a79c700 1 --2- 172.21.15.92:0/4013711717 >> v2:172.21.15.169:3300/0 conn(0x7f89140b8510 0x7f891406b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:08.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.476+0000 7f891a79c700 1 -- 172.21.15.92:0/4013711717 >> v1:172.21.15.92:6789/0 conn(0x7f89140cd460 legacy=0x7f891406beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:08.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.476+0000 7f891a79c700 1 -- 172.21.15.92:0/4013711717 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89140cd8f0 con 0x7f89140d0d60 2023-11-28T11:23:08.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.476+0000 7f8918f99700 1 -- 172.21.15.92:0/4013711717 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 313884821 0 0) 0x7f89140b88e0 con 0x7f89140cd460 2023-11-28T11:23:08.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.476+0000 7f891a79c700 1 --2- 172.21.15.92:0/4013711717 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89140d0d60 0x7f89140d1200 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f89140b5a50 tx=0x7f890c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a80e88a4842266a0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:08.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.477+0000 7f8918f99700 1 -- 172.21.15.92:0/4013711717 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f890c00e070 con 0x7f89140d0d60 2023-11-28T11:23:08.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.477+0000 7f8918f99700 1 -- 172.21.15.92:0/4013711717 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f890c00bb90 con 0x7f89140d0d60 2023-11-28T11:23:08.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.477+0000 7f8918f99700 1 -- 172.21.15.92:0/4013711717 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f890c012430 con 0x7f89140d0d60 2023-11-28T11:23:08.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.477+0000 7f891c1ff700 1 -- 172.21.15.92:0/4013711717 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89140d0d60 msgr2=0x7f89140d1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:08.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.477+0000 7f891c1ff700 1 --2- 172.21.15.92:0/4013711717 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89140d0d60 0x7f89140d1200 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f89140b5a50 tx=0x7f890c00b330 comp rx=0 tx=0).stop 2023-11-28T11:23:08.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.477+0000 7f891c1ff700 1 -- 172.21.15.92:0/4013711717 shutdown_connections 2023-11-28T11:23:08.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.477+0000 7f891c1ff700 1 --2- 172.21.15.92:0/4013711717 >> v2:172.21.15.169:3300/0 conn(0x7f89140b8510 0x7f891406b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:08.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.477+0000 7f891c1ff700 1 --2- 172.21.15.92:0/4013711717 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89140d0d60 0x7f89140d1200 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:08.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.477+0000 7f891c1ff700 1 -- 172.21.15.92:0/4013711717 >> 172.21.15.92:0/4013711717 conn(0x7f89140719a0 msgr2=0x7f8914071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:08.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.477+0000 7f891c1ff700 1 -- 172.21.15.92:0/4013711717 shutdown_connections 2023-11-28T11:23:08.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.477+0000 7f891c1ff700 1 -- 172.21.15.92:0/4013711717 wait complete. 2023-11-28T11:23:08.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.479+0000 7f891c1ff700 1 Processor -- start 2023-11-28T11:23:08.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.479+0000 7f891c1ff700 1 -- start start 2023-11-28T11:23:08.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.480+0000 7f891c1ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89140b8510 0x7f89141e6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:08.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.480+0000 7f891c1ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89140cd460 0x7f89141e7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:08.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.480+0000 7f891c1ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89140d0d60 0x7f89141e7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:08.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.480+0000 7f891c1ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8914151e90 con 0x7f89140b8510 2023-11-28T11:23:08.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.480+0000 7f891c1ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89141e6070 con 0x7f89140d0d60 2023-11-28T11:23:08.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.480+0000 7f891c1ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89141e5460 con 0x7f89140cd460 2023-11-28T11:23:08.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.480+0000 7f8919f9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89140b8510 0x7f89141e6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:08.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.480+0000 7f8919f9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89140b8510 0x7f89141e6500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57586/0 (socket says 172.21.15.92:57586) 2023-11-28T11:23:08.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.480+0000 7f8919f9b700 1 -- 172.21.15.92:0/2370554404 learned_addr learned my addr 172.21.15.92:0/2370554404 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:08.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.481+0000 7f891a79c700 1 --2- 172.21.15.92:0/2370554404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89140d0d60 0x7f89141e7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:08.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.481+0000 7f891979a700 1 --2- 172.21.15.92:0/2370554404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89140cd460 0x7f89141e7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:08.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.481+0000 7f8919f9b700 1 -- 172.21.15.92:0/2370554404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89140cd460 msgr2=0x7f89141e7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:08.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.481+0000 7f8919f9b700 1 --2- 172.21.15.92:0/2370554404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89140cd460 0x7f89141e7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:08.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.481+0000 7f8919f9b700 1 -- 172.21.15.92:0/2370554404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89140d0d60 msgr2=0x7f89141e7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:08.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.481+0000 7f8919f9b700 1 --2- 172.21.15.92:0/2370554404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89140d0d60 0x7f89141e7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:08.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.481+0000 7f8919f9b700 1 -- 172.21.15.92:0/2370554404 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f890c009a70 con 0x7f89140b8510 2023-11-28T11:23:08.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.481+0000 7f8919f9b700 1 --2- 172.21.15.92:0/2370554404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89140b8510 0x7f89141e6500 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f891000e810 tx=0x7f891000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:08.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.481+0000 7f890a7fc700 1 -- 172.21.15.92:0/2370554404 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8910013610 con 0x7f89140b8510 2023-11-28T11:23:08.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.481+0000 7f890a7fc700 1 -- 172.21.15.92:0/2370554404 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8910013770 con 0x7f89140b8510 2023-11-28T11:23:08.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.481+0000 7f891c1ff700 1 -- 172.21.15.92:0/2370554404 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89140cfff0 con 0x7f89140b8510 2023-11-28T11:23:08.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.481+0000 7f890a7fc700 1 -- 172.21.15.92:0/2370554404 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8910013b20 con 0x7f89140b8510 2023-11-28T11:23:08.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.481+0000 7f891c1ff700 1 -- 172.21.15.92:0/2370554404 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89140cfd10 con 0x7f89140b8510 2023-11-28T11:23:08.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.483+0000 7f891c1ff700 1 -- 172.21.15.92:0/2370554404 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89141e6350 con 0x7f89140b8510 2023-11-28T11:23:08.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.483+0000 7f890a7fc700 1 -- 172.21.15.92:0/2370554404 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8910023070 con 0x7f89140b8510 2023-11-28T11:23:08.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.485+0000 7f890a7fc700 1 --2- 172.21.15.92:0/2370554404 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f88f80682f0 0x7f88f806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:08.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.485+0000 7f890a7fc700 1 -- 172.21.15.92:0/2370554404 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f89100856e0 con 0x7f89140b8510 2023-11-28T11:23:08.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.485+0000 7f891979a700 1 --2- 172.21.15.92:0/2370554404 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f88f80682f0 0x7f88f806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:08.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.485+0000 7f891979a700 1 --2- 172.21.15.92:0/2370554404 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f88f80682f0 0x7f88f806a7a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f89140a9960 tx=0x7f8904007be0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:08.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.489+0000 7f890a7fc700 1 -- 172.21.15.92:0/2370554404 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f89100520d0 con 0x7f89140b8510 2023-11-28T11:23:08.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.752+0000 7f891c1ff700 1 -- 172.21.15.92:0/2370554404 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f8914067880 con 0x7f89140b8510 2023-11-28T11:23:08.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.753+0000 7f890a7fc700 1 -- 172.21.15.92:0/2370554404 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5298 (secure 0 0 0) 0x7f89100556f0 con 0x7f89140b8510 2023-11-28T11:23:08.755 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:23:08.755 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-28T11:23:08.755 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_dump": [ 2023-11-28T11:23:08.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:23:08.755 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.a", 2023-11-28T11:23:08.755 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDXzGVlH/PGCBAAor2D9Zz+LcAPQsi1BhIuQA==", 2023-11-28T11:23:08.755 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:23:08.755 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-11-28T11:23:08.755 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-11-28T11:23:08.755 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-11-28T11:23:08.756 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-28T11:23:08.756 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:23:08.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:23:08.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:23:08.756 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.b", 2023-11-28T11:23:08.756 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDXzGVlamQ0IBAAbJVb+gWu1RqSykFqIM7Hhg==", 2023-11-28T11:23:08.756 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:23:08.756 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-11-28T11:23:08.756 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-11-28T11:23:08.756 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-11-28T11:23:08.756 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-28T11:23:08.756 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:23:08.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:23:08.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:23:08.757 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.c", 2023-11-28T11:23:08.757 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDXzGVlpfloFBAAVSQxpfstP6SpWbWUMhB/hg==", 2023-11-28T11:23:08.757 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:23:08.757 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-11-28T11:23:08.757 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-11-28T11:23:08.757 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-11-28T11:23:08.757 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-28T11:23:08.757 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:23:08.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:23:08.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:23:08.758 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.d", 2023-11-28T11:23:08.758 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDXzGVlqQ8ALBAAESaHrVIhfnF5ocn44xdS5A==", 2023-11-28T11:23:08.758 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:23:08.758 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-11-28T11:23:08.758 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-11-28T11:23:08.758 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-11-28T11:23:08.758 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-28T11:23:08.758 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:23:08.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:23:08.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:23:08.758 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.0", 2023-11-28T11:23:08.758 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDbzGVl9MVvKxAA1Pd/AN+XdW47auUMbazqdw==", 2023-11-28T11:23:08.759 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:23:08.759 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-11-28T11:23:08.759 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-11-28T11:23:08.759 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-28T11:23:08.759 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:23:08.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:23:08.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:23:08.759 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.1", 2023-11-28T11:23:08.759 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDezGVldH9fLxAASVpF1+s0RIj4ObUNZoLDKg==", 2023-11-28T11:23:08.760 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:23:08.760 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-11-28T11:23:08.760 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-11-28T11:23:08.760 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-28T11:23:08.760 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:23:08.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:23:08.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:23:08.760 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.2", 2023-11-28T11:23:08.760 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDhzGVl+hnoMxAAXG0K3wm2UYF3BUeGjOkHhw==", 2023-11-28T11:23:08.760 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:23:08.760 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-11-28T11:23:08.760 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-11-28T11:23:08.761 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-28T11:23:08.761 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:23:08.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:23:08.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:23:08.761 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.3", 2023-11-28T11:23:08.761 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDkzGVlBB0EORAAQ3HJyXBI2AFfb/I5REmALA==", 2023-11-28T11:23:08.761 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:23:08.761 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-11-28T11:23:08.761 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-11-28T11:23:08.761 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-28T11:23:08.761 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:23:08.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:23:08.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:23:08.762 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.4", 2023-11-28T11:23:08.762 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDrzGVlgOjDGBAAqwSISduMc7UF8VWdiSNJXQ==", 2023-11-28T11:23:08.762 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:23:08.762 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-11-28T11:23:08.762 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-11-28T11:23:08.762 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-28T11:23:08.762 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:23:08.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:23:08.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:23:08.763 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.5", 2023-11-28T11:23:08.763 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDuzGVl9FMMHhAAohYzehTD1xpWI+/zxipY4Q==", 2023-11-28T11:23:08.763 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:23:08.763 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-11-28T11:23:08.763 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-11-28T11:23:08.763 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-28T11:23:08.763 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:23:08.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:23:08.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:23:08.764 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.6", 2023-11-28T11:23:08.764 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDxzGVl+gOMIhAA0FwY/JqNT12aDRet8sSTYw==", 2023-11-28T11:23:08.764 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:23:08.764 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-11-28T11:23:08.764 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-11-28T11:23:08.764 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-28T11:23:08.764 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:23:08.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:23:08.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:23:08.764 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.7", 2023-11-28T11:23:08.764 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQD0zGVlFPIiJhAATxpSqw+VG3u46b/IIb+yqA==", 2023-11-28T11:23:08.765 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:23:08.765 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-11-28T11:23:08.765 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-11-28T11:23:08.765 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-28T11:23:08.765 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:23:08.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:23:08.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:23:08.765 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.0", 2023-11-28T11:23:08.765 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDXzGVl8nA7NhAAGB5c8Mp6PgErGowSH4BBdw==", 2023-11-28T11:23:08.765 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:23:08.765 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-11-28T11:23:08.765 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow r", 2023-11-28T11:23:08.766 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rw pool=cephfs_data" 2023-11-28T11:23:08.766 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:23:08.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:23:08.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:23:08.766 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.1", 2023-11-28T11:23:08.766 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDYzGVlbl/KCRAA23f+Q5tU6l1e3gNiomDzqQ==", 2023-11-28T11:23:08.766 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:23:08.766 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-11-28T11:23:08.766 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-11-28T11:23:08.766 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-11-28T11:23:08.766 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-11-28T11:23:08.766 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:23:08.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:23:08.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:23:08.767 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.2", 2023-11-28T11:23:08.767 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDYzGVl1oM3AhAAdrqZFbX0CRVpZ1q531/+Dg==", 2023-11-28T11:23:08.767 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:23:08.767 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-11-28T11:23:08.767 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-11-28T11:23:08.767 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-11-28T11:23:08.767 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-11-28T11:23:08.767 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:23:08.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:23:08.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:23:08.768 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.3", 2023-11-28T11:23:08.768 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDYzGVlGZJjERAA6iq24nyDkkyLjaWb9NkrtA==", 2023-11-28T11:23:08.768 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:23:08.768 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-11-28T11:23:08.768 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-11-28T11:23:08.768 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-11-28T11:23:08.768 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-11-28T11:23:08.768 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:23:08.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:23:08.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:23:08.768 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.admin", 2023-11-28T11:23:08.768 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDWzGVlmiaaDBAAcA6vmjgBJaurjIyTEGXdLg==", 2023-11-28T11:23:08.769 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:23:08.769 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-11-28T11:23:08.769 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-11-28T11:23:08.769 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-11-28T11:23:08.769 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-28T11:23:08.769 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:23:08.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:23:08.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:23:08.769 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.x", 2023-11-28T11:23:08.769 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDXzGVlb4AkARAAxs1XCTncmt28cyu+vL2AYA==", 2023-11-28T11:23:08.770 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:23:08.770 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-11-28T11:23:08.770 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-11-28T11:23:08.770 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-28T11:23:08.770 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:23:08.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:23:08.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:23:08.770 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.y", 2023-11-28T11:23:08.770 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDWzGVldLQpNBAAhdEG2ocL6T0ByZkLnwic2g==", 2023-11-28T11:23:08.770 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:23:08.770 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-11-28T11:23:08.770 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-11-28T11:23:08.771 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-28T11:23:08.771 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:23:08.771 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:23:08.771 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-28T11:23:08.771 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-28T11:23:08.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.755+0000 7f891c1ff700 1 -- 172.21.15.92:0/2370554404 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f88f80682f0 msgr2=0x7f88f806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:08.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.755+0000 7f891c1ff700 1 --2- 172.21.15.92:0/2370554404 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f88f80682f0 0x7f88f806a7a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f89140a9960 tx=0x7f8904007be0 comp rx=0 tx=0).stop 2023-11-28T11:23:08.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.755+0000 7f891c1ff700 1 -- 172.21.15.92:0/2370554404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89140b8510 msgr2=0x7f89141e6500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:08.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.755+0000 7f891c1ff700 1 --2- 172.21.15.92:0/2370554404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89140b8510 0x7f89141e6500 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f891000e810 tx=0x7f891000eb20 comp rx=0 tx=0).stop 2023-11-28T11:23:08.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.755+0000 7f891c1ff700 1 -- 172.21.15.92:0/2370554404 shutdown_connections 2023-11-28T11:23:08.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.755+0000 7f891c1ff700 1 --2- 172.21.15.92:0/2370554404 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89140d0d60 0x7f89141e7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:08.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.755+0000 7f891c1ff700 1 --2- 172.21.15.92:0/2370554404 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f88f80682f0 0x7f88f806a7a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:08.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.755+0000 7f891c1ff700 1 --2- 172.21.15.92:0/2370554404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89140b8510 0x7f89141e6500 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:08.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.755+0000 7f891c1ff700 1 --2- 172.21.15.92:0/2370554404 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89140cd460 0x7f89141e7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:08.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.755+0000 7f891c1ff700 1 -- 172.21.15.92:0/2370554404 >> 172.21.15.92:0/2370554404 conn(0x7f89140719a0 msgr2=0x7f89140b3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:08.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.755+0000 7f891c1ff700 1 -- 172.21.15.92:0/2370554404 shutdown_connections 2023-11-28T11:23:08.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.755+0000 7f891c1ff700 1 -- 172.21.15.92:0/2370554404 wait complete. 2023-11-28T11:23:08.775 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:23:08.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.973+0000 7f89591ff700 1 Processor -- start 2023-11-28T11:23:08.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.974+0000 7f89591ff700 1 -- start start 2023-11-28T11:23:08.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.975+0000 7f89591ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89540d1d70 0x7f8954167270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:08.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.975+0000 7f89591ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f895406c790 con 0x7f89540cd460 2023-11-28T11:23:08.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.975+0000 7f89591ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89540b88e0 con 0x7f89540b8510 2023-11-28T11:23:08.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.975+0000 7f89591ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89540cdbd0 con 0x7f89540d1d70 2023-11-28T11:23:08.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.975+0000 7f895259c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f89540cd460 0x7f895406c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36900/0 (socket says 172.21.15.92:36900) 2023-11-28T11:23:08.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.975+0000 7f895259c700 1 -- 172.21.15.92:0/3888993657 learned_addr learned my addr 172.21.15.92:0/3888993657 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:23:08.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.975+0000 7f895359e700 1 --2- 172.21.15.92:0/3888993657 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89540d1d70 0x7f8954167270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:08.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.975+0000 7f8951d9b700 1 -- 172.21.15.92:0/3888993657 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1731400621 0 0) 0x7f895406c790 con 0x7f89540cd460 2023-11-28T11:23:08.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.975+0000 7f8951d9b700 1 -- 172.21.15.92:0/3888993657 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f893c0035b0 con 0x7f89540cd460 2023-11-28T11:23:08.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.976+0000 7f895359e700 1 -- 172.21.15.92:0/3888993657 >> v1:172.21.15.169:6789/0 conn(0x7f89540b8510 legacy=0x7f895406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:08.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.976+0000 7f8951d9b700 1 -- 172.21.15.92:0/3888993657 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1050499579 0 0) 0x7f89540b88e0 con 0x7f89540b8510 2023-11-28T11:23:08.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.976+0000 7f895359e700 1 -- 172.21.15.92:0/3888993657 >> v1:172.21.15.92:6789/0 conn(0x7f89540cd460 legacy=0x7f895406c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:08.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.976+0000 7f895359e700 1 -- 172.21.15.92:0/3888993657 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89540cd8f0 con 0x7f89540d1d70 2023-11-28T11:23:08.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.976+0000 7f895359e700 1 --2- 172.21.15.92:0/3888993657 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89540d1d70 0x7f8954167270 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f894c009dc0 tx=0x7f894c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69e443806a87f982 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:08.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.976+0000 7f8951d9b700 1 -- 172.21.15.92:0/3888993657 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c00e070 con 0x7f89540d1d70 2023-11-28T11:23:08.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.976+0000 7f8951d9b700 1 -- 172.21.15.92:0/3888993657 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f894c00bb20 con 0x7f89540d1d70 2023-11-28T11:23:08.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.976+0000 7f8951d9b700 1 -- 172.21.15.92:0/3888993657 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c00be90 con 0x7f89540d1d70 2023-11-28T11:23:08.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.977+0000 7f89591ff700 1 -- 172.21.15.92:0/3888993657 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89540d1d70 msgr2=0x7f8954167270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:08.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.977+0000 7f89591ff700 1 --2- 172.21.15.92:0/3888993657 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89540d1d70 0x7f8954167270 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f894c009dc0 tx=0x7f894c00b330 comp rx=0 tx=0).stop 2023-11-28T11:23:08.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.977+0000 7f89591ff700 1 -- 172.21.15.92:0/3888993657 shutdown_connections 2023-11-28T11:23:08.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.977+0000 7f89591ff700 1 --2- 172.21.15.92:0/3888993657 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89540d1d70 0x7f8954167270 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:08.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.977+0000 7f89591ff700 1 -- 172.21.15.92:0/3888993657 >> 172.21.15.92:0/3888993657 conn(0x7f89540719a0 msgr2=0x7f8954071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:08.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.977+0000 7f89591ff700 1 -- 172.21.15.92:0/3888993657 shutdown_connections 2023-11-28T11:23:08.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.977+0000 7f89591ff700 1 -- 172.21.15.92:0/3888993657 wait complete. 2023-11-28T11:23:08.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.979+0000 7f89591ff700 1 Processor -- start 2023-11-28T11:23:08.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.979+0000 7f89591ff700 1 -- start start 2023-11-28T11:23:08.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.980+0000 7f89591ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89540b8510 0x7f89540a9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:08.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.980+0000 7f89591ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89540cd460 0x7f89540c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:08.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.980+0000 7f89591ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89540d1d70 0x7f895415a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:08.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.980+0000 7f89591ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89541e64e0 con 0x7f89540d1d70 2023-11-28T11:23:08.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.980+0000 7f89591ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89540ca420 con 0x7f89540cd460 2023-11-28T11:23:08.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.980+0000 7f89591ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89540caef0 con 0x7f89540b8510 2023-11-28T11:23:08.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.980+0000 7f895359e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89540d1d70 0x7f895415a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:08.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.980+0000 7f895359e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89540d1d70 0x7f895415a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40132/0 (socket says 172.21.15.92:40132) 2023-11-28T11:23:08.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.980+0000 7f895359e700 1 -- 172.21.15.92:0/2506291574 learned_addr learned my addr 172.21.15.92:0/2506291574 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:08.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.981+0000 7f8952d9d700 1 --2- 172.21.15.92:0/2506291574 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89540b8510 0x7f89540a9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:08.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.981+0000 7f895259c700 1 --2- 172.21.15.92:0/2506291574 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89540cd460 0x7f89540c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:08.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.981+0000 7f895359e700 1 -- 172.21.15.92:0/2506291574 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89540b8510 msgr2=0x7f89540a9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:08.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.981+0000 7f895359e700 1 --2- 172.21.15.92:0/2506291574 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89540b8510 0x7f89540a9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:08.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.981+0000 7f895359e700 1 -- 172.21.15.92:0/2506291574 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89540cd460 msgr2=0x7f89540c9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:08.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.981+0000 7f895359e700 1 --2- 172.21.15.92:0/2506291574 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89540cd460 0x7f89540c9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:08.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.981+0000 7f895359e700 1 -- 172.21.15.92:0/2506291574 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f894c009a70 con 0x7f89540d1d70 2023-11-28T11:23:08.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.981+0000 7f8952d9d700 1 --2- 172.21.15.92:0/2506291574 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89540b8510 0x7f89540a9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:23:08.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.981+0000 7f895259c700 1 --2- 172.21.15.92:0/2506291574 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89540cd460 0x7f89540c9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:23:08.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.981+0000 7f895359e700 1 --2- 172.21.15.92:0/2506291574 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89540d1d70 0x7f895415a8e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f8954072610 tx=0x7f894c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:08.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.981+0000 7f893b7fe700 1 -- 172.21.15.92:0/2506291574 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c003c20 con 0x7f89540d1d70 2023-11-28T11:23:08.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.981+0000 7f89591ff700 1 -- 172.21.15.92:0/2506291574 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89540cfff0 con 0x7f89540d1d70 2023-11-28T11:23:08.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.981+0000 7f893b7fe700 1 -- 172.21.15.92:0/2506291574 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f894c003d80 con 0x7f89540d1d70 2023-11-28T11:23:08.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.981+0000 7f893b7fe700 1 -- 172.21.15.92:0/2506291574 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c012c00 con 0x7f89540d1d70 2023-11-28T11:23:08.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.982+0000 7f89591ff700 1 -- 172.21.15.92:0/2506291574 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89540cfd10 con 0x7f89540d1d70 2023-11-28T11:23:08.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.983+0000 7f89591ff700 1 -- 172.21.15.92:0/2506291574 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8954067880 con 0x7f89540d1d70 2023-11-28T11:23:08.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.983+0000 7f893b7fe700 1 -- 172.21.15.92:0/2506291574 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f894c012d60 con 0x7f89540d1d70 2023-11-28T11:23:08.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.984+0000 7f893b7fe700 1 --2- 172.21.15.92:0/2506291574 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8940068320 0x7f894006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:08.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.984+0000 7f893b7fe700 1 -- 172.21.15.92:0/2506291574 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f894c087710 con 0x7f89540d1d70 2023-11-28T11:23:08.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.985+0000 7f8952d9d700 1 --2- 172.21.15.92:0/2506291574 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8940068320 0x7f894006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:08.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.985+0000 7f8952d9d700 1 --2- 172.21.15.92:0/2506291574 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8940068320 0x7f894006a7d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8944000f60 tx=0x7f8944009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:08.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:08.993+0000 7f893b7fe700 1 -- 172.21.15.92:0/2506291574 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f894c054010 con 0x7f89540d1d70 2023-11-28T11:23:09.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.187+0000 7f89591ff700 1 -- 172.21.15.92:0/2506291574 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f89540d2940 con 0x7f89540d1d70 2023-11-28T11:23:09.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.188+0000 7f893b7fe700 1 -- 172.21.15.92:0/2506291574 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+14472 (secure 0 0 0) 0x7f894c057630 con 0x7f89540d1d70 2023-11-28T11:23:09.189 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:23:09.189 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":52,"fsid":"68956e0c-c16f-47cd-a220-f98364b0831d","created":"2023-11-28T11:20:28.697573+0000","modified":"2023-11-28T11:23:08.251054+0000","last_up_change":"2023-11-28T11:20:41.257011+0000","last_in_change":"2023-11-28T11:20:32.249767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:20:41.925237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:20:48.682497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6abe483f-0e03-4a26-b9cf-62041291747e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":115823},{"type":"v1","addr":"172.21.15.92:6809","nonce":115823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":115823},{"type":"v1","addr":"172.21.15.92:6813","nonce":115823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":115823},{"type":"v1","addr":"172.21.15.92:6821","nonce":115823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":115823},{"type":"v1","addr":"172.21.15.92:6817","nonce":115823}]},"public_addr":"172.21.15.92:6809/115823","cluster_addr":"172.21.15.92:6813/115823","heartbeat_back_addr":"172.21.15.92:6821/115823","heartbeat_front_addr":"172.21.15.92:6817/115823","state":["exists","up"]},{"osd":1,"uuid":"13c1de73-1a8c-4d7e-8a28-9dae52dfe089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":115824},{"type":"v1","addr":"172.21.15.92:6812","nonce":115824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":115824},{"type":"v1","addr":"172.21.15.92:6816","nonce":115824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":115824},{"type":"v1","addr":"172.21.15.92:6823","nonce":115824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":115824},{"type":"v1","addr":"172.21.15.92:6820","nonce":115824}]},"public_addr":"172.21.15.92:6812/115824","cluster_addr":"172.21.15.92:6816/115824","heartbeat_back_addr":"172.21.15.92:6823/115824","heartbeat_front_addr":"172.21.15.92:6820/115824","state":["exists","up"]},{"osd":2,"uuid":"25788a55-112f-4f00-810d-2bc37cf76778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":115821},{"type":"v1","addr":"172.21.15.92:6801","nonce":115821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":115821},{"type":"v1","addr":"172.21.15.92:6803","nonce":115821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":115821},{"type":"v1","addr":"172.21.15.92:6807","nonce":115821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":115821},{"type":"v1","addr":"172.21.15.92:6805","nonce":115821}]},"public_addr":"172.21.15.92:6801/115821","cluster_addr":"172.21.15.92:6803/115821","heartbeat_back_addr":"172.21.15.92:6807/115821","heartbeat_front_addr":"172.21.15.92:6805/115821","state":["exists","up"]},{"osd":3,"uuid":"adf54e24-33a9-41fe-9ce3-98a5ab6c158e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":115825},{"type":"v1","addr":"172.21.15.92:6825","nonce":115825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":115825},{"type":"v1","addr":"172.21.15.92:6827","nonce":115825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":115825},{"type":"v1","addr":"172.21.15.92:6831","nonce":115825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":115825},{"type":"v1","addr":"172.21.15.92:6829","nonce":115825}]},"public_addr":"172.21.15.92:6825/115825","cluster_addr":"172.21.15.92:6827/115825","heartbeat_back_addr":"172.21.15.92:6831/115825","heartbeat_front_addr":"172.21.15.92:6829/115825","state":["exists","up"]},{"osd":4,"uuid":"ccb9b747-f399-44ea-9a18-9db3152227e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":116035},{"type":"v1","addr":"172.21.15.169:6817","nonce":116035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":116035},{"type":"v1","addr":"172.21.15.169:6819","nonce":116035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":116035},{"type":"v1","addr":"172.21.15.169:6826","nonce":116035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":116035},{"type":"v1","addr":"172.21.15.169:6822","nonce":116035}]},"public_addr":"172.21.15.169:6817/116035","cluster_addr":"172.21.15.169:6819/116035","heartbeat_back_addr":"172.21.15.169:6826/116035","heartbeat_front_addr":"172.21.15.169:6822/116035","state":["exists","up"]},{"osd":5,"uuid":"308a1e63-75aa-4d7d-8a7c-cecc24714fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":116033},{"type":"v1","addr":"172.21.15.169:6801","nonce":116033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":116033},{"type":"v1","addr":"172.21.15.169:6803","nonce":116033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":116033},{"type":"v1","addr":"172.21.15.169:6807","nonce":116033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":116033},{"type":"v1","addr":"172.21.15.169:6805","nonce":116033}]},"public_addr":"172.21.15.169:6801/116033","cluster_addr":"172.21.15.169:6803/116033","heartbeat_back_addr":"172.21.15.169:6807/116033","heartbeat_front_addr":"172.21.15.169:6805/116033","state":["exists","up"]},{"osd":6,"uuid":"9c459b7b-b821-4b4b-b91e-6aaae271457c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":116034},{"type":"v1","addr":"172.21.15.169:6809","nonce":116034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":116034},{"type":"v1","addr":"172.21.15.169:6811","nonce":116034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":116034},{"type":"v1","addr":"172.21.15.169:6815","nonce":116034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":116034},{"type":"v1","addr":"172.21.15.169:6813","nonce":116034}]},"public_addr":"172.21.15.169:6809/116034","cluster_addr":"172.21.15.169:6811/116034","heartbeat_back_addr":"172.21.15.169:6815/116034","heartbeat_front_addr":"172.21.15.169:6813/116034","state":["exists","up"]},{"osd":7,"uuid":"1ca9aa4a-755d-4e28-a937-4d8dccab4df3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":116044},{"type":"v1","addr":"172.21.15.169:6823","nonce":116044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6825","nonce":116044},{"type":"v1","addr":"172.21.15.169:6827","nonce":116044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":116044},{"type":"v1","addr":"172.21.15.169:6831","nonce":116044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":116044},{"type":"v1","addr":"172.21.15.169:6829","nonce":116044}]},"public_addr":"172.21.15.169:6823/116044","cluster_addr":"172.21.15.169:6827/116044","heartbeat_back_addr":"172.21.15.169:6831/116044","heartbeat_front_addr":"172.21.15.169:6829/116044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.586781+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.869778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.508594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.860493+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.872267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.746933+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.611520+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.871446+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:23:09.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.191+0000 7f89591ff700 1 -- 172.21.15.92:0/2506291574 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8940068320 msgr2=0x7f894006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:09.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.191+0000 7f89591ff700 1 --2- 172.21.15.92:0/2506291574 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8940068320 0x7f894006a7d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8944000f60 tx=0x7f8944009040 comp rx=0 tx=0).stop 2023-11-28T11:23:09.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.191+0000 7f89591ff700 1 -- 172.21.15.92:0/2506291574 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89540d1d70 msgr2=0x7f895415a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:09.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.191+0000 7f89591ff700 1 --2- 172.21.15.92:0/2506291574 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89540d1d70 0x7f895415a8e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f8954072610 tx=0x7f894c004ad0 comp rx=0 tx=0).stop 2023-11-28T11:23:09.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.191+0000 7f89591ff700 1 -- 172.21.15.92:0/2506291574 shutdown_connections 2023-11-28T11:23:09.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.191+0000 7f89591ff700 1 --2- 172.21.15.92:0/2506291574 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8940068320 0x7f894006a7d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:09.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.191+0000 7f89591ff700 1 --2- 172.21.15.92:0/2506291574 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89540d1d70 0x7f895415a8e0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:09.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.191+0000 7f89591ff700 1 --2- 172.21.15.92:0/2506291574 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f89540b8510 0x7f89540a9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:09.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.191+0000 7f89591ff700 1 --2- 172.21.15.92:0/2506291574 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f89540cd460 0x7f89540c9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:09.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.191+0000 7f89591ff700 1 -- 172.21.15.92:0/2506291574 >> 172.21.15.92:0/2506291574 conn(0x7f89540719a0 msgr2=0x7f895406b970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:09.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.191+0000 7f89591ff700 1 -- 172.21.15.92:0/2506291574 shutdown_connections 2023-11-28T11:23:09.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.191+0000 7f89591ff700 1 -- 172.21.15.92:0/2506291574 wait complete. 2023-11-28T11:23:09.210 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:20:41.925237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:20:48.682497+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:23:09.210 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:23:09.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.408+0000 7f3eab1ff700 1 Processor -- start 2023-11-28T11:23:09.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.408+0000 7f3eab1ff700 1 -- start start 2023-11-28T11:23:09.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.409+0000 7f3eab1ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f3ea40b8510 0x7f3ea406b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:09.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.410+0000 7f3eab1ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ea40b88e0 con 0x7f3ea40cd460 2023-11-28T11:23:09.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.410+0000 7f3eab1ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ea40cd8f0 con 0x7f3ea40d0d60 2023-11-28T11:23:09.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.410+0000 7f3eab1ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ea40cdbd0 con 0x7f3ea40b8510 2023-11-28T11:23:09.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.410+0000 7f3ea3fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f3ea40cd460 0x7f3ea406beb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36904/0 (socket says 172.21.15.92:36904) 2023-11-28T11:23:09.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.410+0000 7f3ea3fff700 1 -- 172.21.15.92:0/269104714 learned_addr learned my addr 172.21.15.92:0/269104714 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:23:09.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.410+0000 7f3ea8f9b700 1 --2- 172.21.15.92:0/269104714 >> v2:172.21.15.169:3300/0 conn(0x7f3ea40b8510 0x7f3ea406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:09.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.410+0000 7f3ea37fe700 1 -- 172.21.15.92:0/269104714 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4130540976 0 0) 0x7f3ea40b88e0 con 0x7f3ea40cd460 2023-11-28T11:23:09.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.410+0000 7f3ea37fe700 1 -- 172.21.15.92:0/269104714 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e940035b0 con 0x7f3ea40cd460 2023-11-28T11:23:09.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.411+0000 7f3ea37fe700 1 -- 172.21.15.92:0/269104714 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2956727403 0 0) 0x7f3e940035b0 con 0x7f3ea40cd460 2023-11-28T11:23:09.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.411+0000 7f3ea37fe700 1 -- 172.21.15.92:0/269104714 >> v2:172.21.15.169:3300/0 conn(0x7f3ea40b8510 msgr2=0x7f3ea406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:09.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.411+0000 7f3ea37fe700 1 --2- 172.21.15.92:0/269104714 >> v2:172.21.15.169:3300/0 conn(0x7f3ea40b8510 0x7f3ea406b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:09.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.411+0000 7f3ea37fe700 1 -- 172.21.15.92:0/269104714 >> v1:172.21.15.169:6789/0 conn(0x7f3ea40d0d60 legacy=0x7f3ea40d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:09.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.411+0000 7f3ea37fe700 1 -- 172.21.15.92:0/269104714 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea40cfd10 con 0x7f3ea40cd460 2023-11-28T11:23:09.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.411+0000 7f3ea37fe700 1 -- 172.21.15.92:0/269104714 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532161587 0 0) 0x7f3e90001670 con 0x7f3ea40cd460 2023-11-28T11:23:09.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.411+0000 7f3ea37fe700 1 -- 172.21.15.92:0/269104714 >> v1:172.21.15.92:6789/0 conn(0x7f3ea40cd460 legacy=0x7f3ea406beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:09.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.411+0000 7f3ea37fe700 1 --2- 172.21.15.92:0/269104714 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e94003a00 0x7f3e94003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:09.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.411+0000 7f3ea37fe700 1 --2- 172.21.15.92:0/269104714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e94004850 0x7f3e94004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:09.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.411+0000 7f3ea37fe700 1 --2- 172.21.15.92:0/269104714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e94005200 0x7f3e9400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:09.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.411+0000 7f3ea37fe700 1 -- 172.21.15.92:0/269104714 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea40cdbd0 con 0x7f3e94004850 2023-11-28T11:23:09.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.411+0000 7f3ea37fe700 1 -- 172.21.15.92:0/269104714 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9400c080 con 0x7f3e94005200 2023-11-28T11:23:09.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.411+0000 7f3ea37fe700 1 -- 172.21.15.92:0/269104714 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9400c240 con 0x7f3e94003a00 2023-11-28T11:23:09.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.411+0000 7f3ea3fff700 1 --2- 172.21.15.92:0/269104714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e94004850 0x7f3e94004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:09.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.411+0000 7f3ea8f9b700 1 --2- 172.21.15.92:0/269104714 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e94003a00 0x7f3e94003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:09.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.411+0000 7f3ea979c700 1 --2- 172.21.15.92:0/269104714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e94005200 0x7f3e9400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:09.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.412+0000 7f3ea3fff700 1 -- 172.21.15.92:0/269104714 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e94003a00 msgr2=0x7f3e94003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:09.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.412+0000 7f3ea3fff700 1 --2- 172.21.15.92:0/269104714 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e94003a00 0x7f3e94003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:09.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.412+0000 7f3ea3fff700 1 -- 172.21.15.92:0/269104714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e94005200 msgr2=0x7f3e9400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:09.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.412+0000 7f3ea3fff700 1 --2- 172.21.15.92:0/269104714 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e94005200 0x7f3e9400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:09.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.412+0000 7f3ea3fff700 1 -- 172.21.15.92:0/269104714 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e940035b0 con 0x7f3e94004850 2023-11-28T11:23:09.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.412+0000 7f3ea979c700 1 -- 172.21.15.92:0/269104714 reap_dead start 2023-11-28T11:23:09.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.412+0000 7f3ea3fff700 1 --2- 172.21.15.92:0/269104714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e94004850 0x7f3e94004cc0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f3ea40b1de0 tx=0x7f3e94005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6747765b6c9814 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:09.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.412+0000 7f3ea37fe700 1 -- 172.21.15.92:0/269104714 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e90007c10 con 0x7f3e94004850 2023-11-28T11:23:09.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.412+0000 7f3ea37fe700 1 -- 172.21.15.92:0/269104714 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e9000cda0 con 0x7f3e94004850 2023-11-28T11:23:09.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.412+0000 7f3ea37fe700 1 -- 172.21.15.92:0/269104714 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e90005ba0 con 0x7f3e94004850 2023-11-28T11:23:09.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.412+0000 7f3eab1ff700 1 -- 172.21.15.92:0/269104714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e94004850 msgr2=0x7f3e94004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:09.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.412+0000 7f3eab1ff700 1 --2- 172.21.15.92:0/269104714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e94004850 0x7f3e94004cc0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f3ea40b1de0 tx=0x7f3e94005880 comp rx=0 tx=0).stop 2023-11-28T11:23:09.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.413+0000 7f3eab1ff700 1 -- 172.21.15.92:0/269104714 shutdown_connections 2023-11-28T11:23:09.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.413+0000 7f3eab1ff700 1 --2- 172.21.15.92:0/269104714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e94004850 0x7f3e94004cc0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:09.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.413+0000 7f3eab1ff700 1 -- 172.21.15.92:0/269104714 >> 172.21.15.92:0/269104714 conn(0x7f3ea40719a0 msgr2=0x7f3ea4071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:09.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.413+0000 7f3eab1ff700 1 -- 172.21.15.92:0/269104714 shutdown_connections 2023-11-28T11:23:09.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.413+0000 7f3eab1ff700 1 -- 172.21.15.92:0/269104714 wait complete. 2023-11-28T11:23:09.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.414+0000 7f3eab1ff700 1 Processor -- start 2023-11-28T11:23:09.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.415+0000 7f3eab1ff700 1 -- start start 2023-11-28T11:23:09.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.415+0000 7f3eab1ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e94004850 0x7f3ea4157e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:09.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.415+0000 7f3eab1ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ea4158760 0x7f3ea41543c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:09.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.415+0000 7f3eab1ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ea41f23d0 0x7f3ea40b3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:09.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.415+0000 7f3eab1ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea41583b0 con 0x7f3ea4158760 2023-11-28T11:23:09.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.415+0000 7f3eab1ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea4158c90 con 0x7f3e94004850 2023-11-28T11:23:09.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.415+0000 7f3eab1ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea4158f70 con 0x7f3ea41f23d0 2023-11-28T11:23:09.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.416+0000 7f3ea3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ea4158760 0x7f3ea41543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:09.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.416+0000 7f3ea3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ea4158760 0x7f3ea41543c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40152/0 (socket says 172.21.15.92:40152) 2023-11-28T11:23:09.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.416+0000 7f3ea3fff700 1 -- 172.21.15.92:0/699487758 learned_addr learned my addr 172.21.15.92:0/699487758 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:09.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.416+0000 7f3ea8f9b700 1 --2- 172.21.15.92:0/699487758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e94004850 0x7f3ea4157e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:09.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.416+0000 7f3ea979c700 1 --2- 172.21.15.92:0/699487758 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ea41f23d0 0x7f3ea40b3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:09.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.416+0000 7f3ea3fff700 1 -- 172.21.15.92:0/699487758 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ea41f23d0 msgr2=0x7f3ea40b3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:09.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.416+0000 7f3ea3fff700 1 --2- 172.21.15.92:0/699487758 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ea41f23d0 0x7f3ea40b3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:09.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.416+0000 7f3ea3fff700 1 -- 172.21.15.92:0/699487758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e94004850 msgr2=0x7f3ea4157e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:09.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.416+0000 7f3ea3fff700 1 --2- 172.21.15.92:0/699487758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e94004850 0x7f3ea4157e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:09.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.416+0000 7f3ea3fff700 1 -- 172.21.15.92:0/699487758 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea40d29c0 con 0x7f3ea4158760 2023-11-28T11:23:09.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.416+0000 7f3ea3fff700 1 --2- 172.21.15.92:0/699487758 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ea4158760 0x7f3ea41543c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f3e9400c050 tx=0x7f3e90001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:09.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.417+0000 7f3ea17fa700 1 -- 172.21.15.92:0/699487758 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e90001670 con 0x7f3ea4158760 2023-11-28T11:23:09.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.417+0000 7f3ea17fa700 1 -- 172.21.15.92:0/699487758 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e900070d0 con 0x7f3ea4158760 2023-11-28T11:23:09.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.417+0000 7f3ea17fa700 1 -- 172.21.15.92:0/699487758 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e90007480 con 0x7f3ea4158760 2023-11-28T11:23:09.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.417+0000 7f3eab1ff700 1 -- 172.21.15.92:0/699487758 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ea40cfff0 con 0x7f3ea4158760 2023-11-28T11:23:09.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.417+0000 7f3eab1ff700 1 -- 172.21.15.92:0/699487758 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ea40d2b50 con 0x7f3ea4158760 2023-11-28T11:23:09.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.418+0000 7f3eab1ff700 1 -- 172.21.15.92:0/699487758 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ea4067880 con 0x7f3ea4158760 2023-11-28T11:23:09.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.419+0000 7f3ea17fa700 1 -- 172.21.15.92:0/699487758 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3e90024070 con 0x7f3ea4158760 2023-11-28T11:23:09.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.420+0000 7f3ea17fa700 1 --2- 172.21.15.92:0/699487758 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3e88068320 0x7f3e8806a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:09.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.420+0000 7f3ea17fa700 1 -- 172.21.15.92:0/699487758 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f3e90086840 con 0x7f3ea4158760 2023-11-28T11:23:09.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.420+0000 7f3ea8f9b700 1 --2- 172.21.15.92:0/699487758 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3e88068320 0x7f3e8806a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:09.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.421+0000 7f3ea8f9b700 1 --2- 172.21.15.92:0/699487758 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3e88068320 0x7f3e8806a7d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3e98009ce0 tx=0x7f3e98009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:09.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.424+0000 7f3ea17fa700 1 -- 172.21.15.92:0/699487758 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3e900532b0 con 0x7f3ea4158760 2023-11-28T11:23:09.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.616+0000 7f3eab1ff700 1 -- 172.21.15.92:0/699487758 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3ea41583b0 con 0x7f3ea4158760 2023-11-28T11:23:09.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.617+0000 7f3ea17fa700 1 -- 172.21.15.92:0/699487758 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v52) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3e900568d0 con 0x7f3ea4158760 2023-11-28T11:23:09.618 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-11-28T11:23:09.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.619+0000 7f3eab1ff700 1 -- 172.21.15.92:0/699487758 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3e88068320 msgr2=0x7f3e8806a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:09.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.619+0000 7f3eab1ff700 1 --2- 172.21.15.92:0/699487758 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3e88068320 0x7f3e8806a7d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3e98009ce0 tx=0x7f3e98009210 comp rx=0 tx=0).stop 2023-11-28T11:23:09.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.619+0000 7f3eab1ff700 1 -- 172.21.15.92:0/699487758 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ea4158760 msgr2=0x7f3ea41543c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:09.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.619+0000 7f3eab1ff700 1 --2- 172.21.15.92:0/699487758 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ea4158760 0x7f3ea41543c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f3e9400c050 tx=0x7f3e90001a00 comp rx=0 tx=0).stop 2023-11-28T11:23:09.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.619+0000 7f3eab1ff700 1 -- 172.21.15.92:0/699487758 shutdown_connections 2023-11-28T11:23:09.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.619+0000 7f3eab1ff700 1 --2- 172.21.15.92:0/699487758 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3ea41f23d0 0x7f3ea40b3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:09.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.619+0000 7f3eab1ff700 1 --2- 172.21.15.92:0/699487758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e94004850 0x7f3ea4157e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:09.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.619+0000 7f3eab1ff700 1 --2- 172.21.15.92:0/699487758 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3e88068320 0x7f3e8806a7d0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:09.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.619+0000 7f3eab1ff700 1 --2- 172.21.15.92:0/699487758 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ea4158760 0x7f3ea41543c0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:09.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.619+0000 7f3eab1ff700 1 -- 172.21.15.92:0/699487758 >> 172.21.15.92:0/699487758 conn(0x7f3ea40719a0 msgr2=0x7f3ea406b970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:09.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.619+0000 7f3eab1ff700 1 -- 172.21.15.92:0/699487758 shutdown_connections 2023-11-28T11:23:09.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.619+0000 7f3eab1ff700 1 -- 172.21.15.92:0/699487758 wait complete. 2023-11-28T11:23:09.638 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:23:09.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.837+0000 7f2acbdff700 1 Processor -- start 2023-11-28T11:23:09.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.837+0000 7f2acbdff700 1 -- start start 2023-11-28T11:23:09.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.838+0000 7f2acbdff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ac40d1d70 0x7f2ac4167270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:09.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.838+0000 7f2acbdff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ac406c790 con 0x7f2ac40b8510 2023-11-28T11:23:09.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.839+0000 7f2acbdff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ac40b88e0 con 0x7f2ac40cd460 2023-11-28T11:23:09.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.839+0000 7f2acbdff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac40cdbd0 con 0x7f2ac40d1d70 2023-11-28T11:23:09.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.839+0000 7f2ac9b9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f2ac40b8510 0x7f2ac406b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36910/0 (socket says 172.21.15.92:36910) 2023-11-28T11:23:09.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.839+0000 7f2ac9b9b700 1 -- 172.21.15.92:0/1038431456 learned_addr learned my addr 172.21.15.92:0/1038431456 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:23:09.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.839+0000 7f2aca39c700 1 --2- 172.21.15.92:0/1038431456 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ac40d1d70 0x7f2ac4167270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:09.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.839+0000 7f2ac8b99700 1 -- 172.21.15.92:0/1038431456 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3602498312 0 0) 0x7f2ac406c790 con 0x7f2ac40b8510 2023-11-28T11:23:09.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.839+0000 7f2ac8b99700 1 -- 172.21.15.92:0/1038431456 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ab00035b0 con 0x7f2ac40b8510 2023-11-28T11:23:09.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.839+0000 7f2ac8b99700 1 -- 172.21.15.92:0/1038431456 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3188137510 0 0) 0x7f2ab00035b0 con 0x7f2ac40b8510 2023-11-28T11:23:09.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.839+0000 7f2ac8b99700 1 -- 172.21.15.92:0/1038431456 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ac40d1d70 msgr2=0x7f2ac4167270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:09.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.839+0000 7f2ac8b99700 1 --2- 172.21.15.92:0/1038431456 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ac40d1d70 0x7f2ac4167270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:09.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.839+0000 7f2ac8b99700 1 -- 172.21.15.92:0/1038431456 >> v1:172.21.15.169:6789/0 conn(0x7f2ac40cd460 legacy=0x7f2ac406c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:09.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.840+0000 7f2ac8b99700 1 -- 172.21.15.92:0/1038431456 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ac40cd8f0 con 0x7f2ac40b8510 2023-11-28T11:23:09.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.840+0000 7f2ac8b99700 1 -- 172.21.15.92:0/1038431456 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532161587 0 0) 0x7f2ac0001670 con 0x7f2ac40b8510 2023-11-28T11:23:09.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.840+0000 7f2ac8b99700 1 -- 172.21.15.92:0/1038431456 >> v1:172.21.15.92:6789/0 conn(0x7f2ac40b8510 legacy=0x7f2ac406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:09.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.840+0000 7f2ac8b99700 1 --2- 172.21.15.92:0/1038431456 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ac40d1d70 0x7f2ab0003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:09.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.840+0000 7f2ac8b99700 1 --2- 172.21.15.92:0/1038431456 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ab0003fb0 0x7f2ab0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:09.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.840+0000 7f2ac8b99700 1 --2- 172.21.15.92:0/1038431456 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ab0004d90 0x7f2ab0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:09.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.840+0000 7f2ac8b99700 1 -- 172.21.15.92:0/1038431456 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac40cdbd0 con 0x7f2ab0003fb0 2023-11-28T11:23:09.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.840+0000 7f2ac8b99700 1 -- 172.21.15.92:0/1038431456 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab000baf0 con 0x7f2ab0004d90 2023-11-28T11:23:09.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.840+0000 7f2ac8b99700 1 -- 172.21.15.92:0/1038431456 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab000bcb0 con 0x7f2ac40d1d70 2023-11-28T11:23:09.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.840+0000 7f2ac939a700 1 --2- 172.21.15.92:0/1038431456 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ab0003fb0 0x7f2ab0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:09.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.840+0000 7f2ac9b9b700 1 --2- 172.21.15.92:0/1038431456 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ac40d1d70 0x7f2ab0003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:09.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.840+0000 7f2aca39c700 1 --2- 172.21.15.92:0/1038431456 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ab0004d90 0x7f2ab0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:09.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.840+0000 7f2ac939a700 1 -- 172.21.15.92:0/1038431456 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ac40d1d70 msgr2=0x7f2ab0003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:09.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.840+0000 7f2ac939a700 1 --2- 172.21.15.92:0/1038431456 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ac40d1d70 0x7f2ab0003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:09.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.840+0000 7f2ac939a700 1 -- 172.21.15.92:0/1038431456 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ab0004d90 msgr2=0x7f2ab0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:09.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.840+0000 7f2ac939a700 1 --2- 172.21.15.92:0/1038431456 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ab0004d90 0x7f2ab0005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:09.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.840+0000 7f2ac939a700 1 -- 172.21.15.92:0/1038431456 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab00035b0 con 0x7f2ab0003fb0 2023-11-28T11:23:09.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.841+0000 7f2ac939a700 1 --2- 172.21.15.92:0/1038431456 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ab0003fb0 0x7f2ab0004850 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4007fd0 tx=0x7f2ab0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f10d13875ecb4f0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:09.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.841+0000 7f2ac8b99700 1 -- 172.21.15.92:0/1038431456 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab40052d0 con 0x7f2ab0003fb0 2023-11-28T11:23:09.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.841+0000 7f2ac8b99700 1 -- 172.21.15.92:0/1038431456 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ab4005430 con 0x7f2ab0003fb0 2023-11-28T11:23:09.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.841+0000 7f2ac8b99700 1 -- 172.21.15.92:0/1038431456 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab40038c0 con 0x7f2ab0003fb0 2023-11-28T11:23:09.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.841+0000 7f2acbdff700 1 -- 172.21.15.92:0/1038431456 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ab0003fb0 msgr2=0x7f2ab0004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:09.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.841+0000 7f2acbdff700 1 --2- 172.21.15.92:0/1038431456 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ab0003fb0 0x7f2ab0004850 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4007fd0 tx=0x7f2ab0005880 comp rx=0 tx=0).stop 2023-11-28T11:23:09.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.841+0000 7f2aca39c700 1 -- 172.21.15.92:0/1038431456 reap_dead start 2023-11-28T11:23:09.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.841+0000 7f2acbdff700 1 -- 172.21.15.92:0/1038431456 shutdown_connections 2023-11-28T11:23:09.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.842+0000 7f2acbdff700 1 -- 172.21.15.92:0/1038431456 >> 172.21.15.92:0/1038431456 conn(0x7f2ac40719a0 msgr2=0x7f2ac4071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:09.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.842+0000 7f2acbdff700 1 -- 172.21.15.92:0/1038431456 shutdown_connections 2023-11-28T11:23:09.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.842+0000 7f2acbdff700 1 -- 172.21.15.92:0/1038431456 wait complete. 2023-11-28T11:23:09.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.844+0000 7f2acbdff700 1 Processor -- start 2023-11-28T11:23:09.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.844+0000 7f2acbdff700 1 -- start start 2023-11-28T11:23:09.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.845+0000 7f2acbdff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ab0003fb0 0x7f2ac415ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:09.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.845+0000 7f2acbdff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ac41f22d0 0x7f2ac41e4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:09.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.845+0000 7f2acbdff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ac41e5040 0x7f2ac41e8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:09.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.845+0000 7f2acbdff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac41f2740 con 0x7f2ab0003fb0 2023-11-28T11:23:09.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.845+0000 7f2acbdff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac41e5560 con 0x7f2ac41f22d0 2023-11-28T11:23:09.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.845+0000 7f2acbdff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac41e5840 con 0x7f2ac41e5040 2023-11-28T11:23:09.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.846+0000 7f2ac9b9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ab0003fb0 0x7f2ac415ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:09.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.846+0000 7f2ac9b9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ab0003fb0 0x7f2ac415ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40158/0 (socket says 172.21.15.92:40158) 2023-11-28T11:23:09.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.846+0000 7f2ac9b9b700 1 -- 172.21.15.92:0/3385892225 learned_addr learned my addr 172.21.15.92:0/3385892225 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:09.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.846+0000 7f2aca39c700 1 --2- 172.21.15.92:0/3385892225 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ac41e5040 0x7f2ac41e8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:09.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.846+0000 7f2ac939a700 1 --2- 172.21.15.92:0/3385892225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ac41f22d0 0x7f2ac41e4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:09.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.846+0000 7f2ac9b9b700 1 -- 172.21.15.92:0/3385892225 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ac41e5040 msgr2=0x7f2ac41e8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:09.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.846+0000 7f2ac9b9b700 1 --2- 172.21.15.92:0/3385892225 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ac41e5040 0x7f2ac41e8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:09.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.846+0000 7f2ac9b9b700 1 -- 172.21.15.92:0/3385892225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ac41f22d0 msgr2=0x7f2ac41e4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:09.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.846+0000 7f2ac9b9b700 1 --2- 172.21.15.92:0/3385892225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ac41f22d0 0x7f2ac41e4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:09.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.846+0000 7f2ac9b9b700 1 -- 172.21.15.92:0/3385892225 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ac40cfff0 con 0x7f2ab0003fb0 2023-11-28T11:23:09.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.846+0000 7f2ac9b9b700 1 --2- 172.21.15.92:0/3385892225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ab0003fb0 0x7f2ac415ad40 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0007290 tx=0x7f2ac00075a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:09.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.846+0000 7f2aba7fc700 1 -- 172.21.15.92:0/3385892225 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac000f660 con 0x7f2ab0003fb0 2023-11-28T11:23:09.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.847+0000 7f2aba7fc700 1 -- 172.21.15.92:0/3385892225 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ac000f7c0 con 0x7f2ab0003fb0 2023-11-28T11:23:09.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.847+0000 7f2acbdff700 1 -- 172.21.15.92:0/3385892225 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ac40cfd10 con 0x7f2ab0003fb0 2023-11-28T11:23:09.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.847+0000 7f2aba7fc700 1 -- 172.21.15.92:0/3385892225 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac000fb70 con 0x7f2ab0003fb0 2023-11-28T11:23:09.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.847+0000 7f2acbdff700 1 -- 172.21.15.92:0/3385892225 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ac40d29c0 con 0x7f2ab0003fb0 2023-11-28T11:23:09.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.848+0000 7f2acbdff700 1 -- 172.21.15.92:0/3385892225 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ac4067880 con 0x7f2ab0003fb0 2023-11-28T11:23:09.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.848+0000 7f2aba7fc700 1 -- 172.21.15.92:0/3385892225 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2ac000fcd0 con 0x7f2ab0003fb0 2023-11-28T11:23:09.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.849+0000 7f2aba7fc700 1 --2- 172.21.15.92:0/3385892225 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f2aa80682f0 0x7f2aa806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:09.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.849+0000 7f2aba7fc700 1 -- 172.21.15.92:0/3385892225 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f2ac0085d20 con 0x7f2ab0003fb0 2023-11-28T11:23:09.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.850+0000 7f2ac939a700 1 --2- 172.21.15.92:0/3385892225 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f2aa80682f0 0x7f2aa806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:09.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.850+0000 7f2ac939a700 1 --2- 172.21.15.92:0/3385892225 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f2aa80682f0 0x7f2aa806a7a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4007fd0 tx=0x7f2ab4009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:09.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:09.853+0000 7f2aba7fc700 1 -- 172.21.15.92:0/3385892225 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2ac0052750 con 0x7f2ab0003fb0 2023-11-28T11:23:10.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.043+0000 7f2acbdff700 1 -- 172.21.15.92:0/3385892225 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2ac408aa50 con 0x7f2ab0003fb0 2023-11-28T11:23:10.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.044+0000 7f2aba7fc700 1 -- 172.21.15.92:0/3385892225 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v52) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2ac0055d70 con 0x7f2ab0003fb0 2023-11-28T11:23:10.046 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-11-28T11:23:10.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.046+0000 7f2acbdff700 1 -- 172.21.15.92:0/3385892225 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f2aa80682f0 msgr2=0x7f2aa806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:10.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.046+0000 7f2acbdff700 1 --2- 172.21.15.92:0/3385892225 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f2aa80682f0 0x7f2aa806a7a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4007fd0 tx=0x7f2ab4009030 comp rx=0 tx=0).stop 2023-11-28T11:23:10.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.046+0000 7f2acbdff700 1 -- 172.21.15.92:0/3385892225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ab0003fb0 msgr2=0x7f2ac415ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:10.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.046+0000 7f2acbdff700 1 --2- 172.21.15.92:0/3385892225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ab0003fb0 0x7f2ac415ad40 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0007290 tx=0x7f2ac00075a0 comp rx=0 tx=0).stop 2023-11-28T11:23:10.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.046+0000 7f2acbdff700 1 -- 172.21.15.92:0/3385892225 shutdown_connections 2023-11-28T11:23:10.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.046+0000 7f2acbdff700 1 --2- 172.21.15.92:0/3385892225 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2ac41e5040 0x7f2ac41e8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:10.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.046+0000 7f2acbdff700 1 --2- 172.21.15.92:0/3385892225 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f2aa80682f0 0x7f2aa806a7a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:10.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.046+0000 7f2acbdff700 1 --2- 172.21.15.92:0/3385892225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ab0003fb0 0x7f2ac415ad40 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:10.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.046+0000 7f2acbdff700 1 --2- 172.21.15.92:0/3385892225 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2ac41f22d0 0x7f2ac41e4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:10.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.046+0000 7f2acbdff700 1 -- 172.21.15.92:0/3385892225 >> 172.21.15.92:0/3385892225 conn(0x7f2ac40719a0 msgr2=0x7f2ac40b33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:10.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.046+0000 7f2acbdff700 1 -- 172.21.15.92:0/3385892225 shutdown_connections 2023-11-28T11:23:10.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.046+0000 7f2acbdff700 1 -- 172.21.15.92:0/3385892225 wait complete. 2023-11-28T11:23:10.066 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T11:23:10.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.262+0000 7fbfef5ff700 1 Processor -- start 2023-11-28T11:23:10.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.263+0000 7fbfef5ff700 1 -- start start 2023-11-28T11:23:10.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.264+0000 7fbfef5ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbfe80cd460 0x7fbfe806c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:10.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.264+0000 7fbfef5ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fbfe80d0d60 0x7fbfe80d11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:10.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.264+0000 7fbfef5ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfe80b88e0 con 0x7fbfe80b8510 2023-11-28T11:23:10.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.264+0000 7fbfef5ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfe80cdbd0 con 0x7fbfe80d0d60 2023-11-28T11:23:10.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.264+0000 7fbfef5ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe80d02d0 con 0x7fbfe80cd460 2023-11-28T11:23:10.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.264+0000 7fbfecb9a700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbfe80cd460 0x7fbfe806c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:10.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.264+0000 7fbfedb9c700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fbfe80d0d60 0x7fbfe80d11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:10.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.264+0000 7fbfecb9a700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbfe80cd460 0x7fbfe806c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.92:47358/0 (socket says 172.21.15.92:47358) 2023-11-28T11:23:10.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.264+0000 7fbfecb9a700 1 -- 172.21.15.92:0/387500978 learned_addr learned my addr 172.21.15.92:0/387500978 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:10.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.264+0000 7fbfed39b700 1 --1- 172.21.15.92:0/387500978 >> v1:172.21.15.169:6789/0 conn(0x7fbfe80b8510 0x7fbfe806b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.92:42834/0 (socket says 172.21.15.92:42834) 2023-11-28T11:23:10.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.265+0000 7fbfedb9c700 1 -- 172.21.15.92:0/387500978 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbfe80cd460 msgr2=0x7fbfe806c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:10.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.265+0000 7fbfedb9c700 1 --2- 172.21.15.92:0/387500978 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbfe80cd460 0x7fbfe806c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:10.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.265+0000 7fbfedb9c700 1 -- 172.21.15.92:0/387500978 >> v1:172.21.15.169:6789/0 conn(0x7fbfe80b8510 legacy=0x7fbfe806b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:10.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.265+0000 7fbfedb9c700 1 -- 172.21.15.92:0/387500978 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe80cd8f0 con 0x7fbfe80d0d60 2023-11-28T11:23:10.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.265+0000 7fbfecb9a700 1 --2- 172.21.15.92:0/387500978 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbfe80cd460 0x7fbfe806c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:23:10.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.265+0000 7fbfedb9c700 1 --2- 172.21.15.92:0/387500978 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfe80d0d60 0x7fbfe80d11e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0009dc0 tx=0x7fbfe000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb953c917725ba58 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:10.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.265+0000 7fbfdffff700 1 -- 172.21.15.92:0/387500978 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe000e070 con 0x7fbfe80d0d60 2023-11-28T11:23:10.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.265+0000 7fbfdffff700 1 -- 172.21.15.92:0/387500978 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfe000bae0 con 0x7fbfe80d0d60 2023-11-28T11:23:10.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.265+0000 7fbfdffff700 1 -- 172.21.15.92:0/387500978 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe000be90 con 0x7fbfe80d0d60 2023-11-28T11:23:10.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.266+0000 7fbfef5ff700 1 -- 172.21.15.92:0/387500978 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfe80d0d60 msgr2=0x7fbfe80d11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:10.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.266+0000 7fbfef5ff700 1 --2- 172.21.15.92:0/387500978 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfe80d0d60 0x7fbfe80d11e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0009dc0 tx=0x7fbfe000b330 comp rx=0 tx=0).stop 2023-11-28T11:23:10.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.266+0000 7fbfef5ff700 1 -- 172.21.15.92:0/387500978 shutdown_connections 2023-11-28T11:23:10.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.266+0000 7fbfef5ff700 1 --2- 172.21.15.92:0/387500978 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfe80d0d60 0x7fbfe80d11e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:10.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.266+0000 7fbfef5ff700 1 --2- 172.21.15.92:0/387500978 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbfe80cd460 0x7fbfe806c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:10.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.266+0000 7fbfef5ff700 1 -- 172.21.15.92:0/387500978 >> 172.21.15.92:0/387500978 conn(0x7fbfe80719a0 msgr2=0x7fbfe8071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:10.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.266+0000 7fbfef5ff700 1 -- 172.21.15.92:0/387500978 shutdown_connections 2023-11-28T11:23:10.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.266+0000 7fbfef5ff700 1 -- 172.21.15.92:0/387500978 wait complete. 2023-11-28T11:23:10.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.268+0000 7fbfef5ff700 1 Processor -- start 2023-11-28T11:23:10.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.268+0000 7fbfef5ff700 1 -- start start 2023-11-28T11:23:10.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.269+0000 7fbfef5ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfe80b8510 0x7fbfe8157680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:10.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.269+0000 7fbfef5ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbfe80cd460 0x7fbfe81f29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:10.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.269+0000 7fbfef5ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfe80d0d60 0x7fbfe815a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:10.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.269+0000 7fbfef5ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe81585b0 con 0x7fbfe80d0d60 2023-11-28T11:23:10.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.269+0000 7fbfef5ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe8157d10 con 0x7fbfe80b8510 2023-11-28T11:23:10.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.269+0000 7fbfef5ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe8157ff0 con 0x7fbfe80cd460 2023-11-28T11:23:10.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.269+0000 7fbfedb9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfe80d0d60 0x7fbfe815a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:10.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.269+0000 7fbfedb9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfe80d0d60 0x7fbfe815a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40174/0 (socket says 172.21.15.92:40174) 2023-11-28T11:23:10.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.269+0000 7fbfedb9c700 1 -- 172.21.15.92:0/44929521 learned_addr learned my addr 172.21.15.92:0/44929521 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:10.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.269+0000 7fbfecb9a700 1 --2- 172.21.15.92:0/44929521 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbfe80cd460 0x7fbfe81f29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:10.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.269+0000 7fbfed39b700 1 --2- 172.21.15.92:0/44929521 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfe80b8510 0x7fbfe8157680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:10.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.270+0000 7fbfedb9c700 1 -- 172.21.15.92:0/44929521 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbfe80cd460 msgr2=0x7fbfe81f29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:10.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.270+0000 7fbfedb9c700 1 --2- 172.21.15.92:0/44929521 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbfe80cd460 0x7fbfe81f29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:10.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.270+0000 7fbfedb9c700 1 -- 172.21.15.92:0/44929521 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfe80b8510 msgr2=0x7fbfe8157680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:10.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.270+0000 7fbfedb9c700 1 --2- 172.21.15.92:0/44929521 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfe80b8510 0x7fbfe8157680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:10.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.270+0000 7fbfedb9c700 1 -- 172.21.15.92:0/44929521 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd8009550 con 0x7fbfe80d0d60 2023-11-28T11:23:10.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.270+0000 7fbfedb9c700 1 --2- 172.21.15.92:0/44929521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfe80d0d60 0x7fbfe815a1f0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fbfe000b370 tx=0x7fbfe0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:10.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.270+0000 7fbfddffb700 1 -- 172.21.15.92:0/44929521 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe0004730 con 0x7fbfe80d0d60 2023-11-28T11:23:10.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.270+0000 7fbfddffb700 1 -- 172.21.15.92:0/44929521 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfe0020070 con 0x7fbfe80d0d60 2023-11-28T11:23:10.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.270+0000 7fbfddffb700 1 -- 172.21.15.92:0/44929521 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe0027480 con 0x7fbfe80d0d60 2023-11-28T11:23:10.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.270+0000 7fbfef5ff700 1 -- 172.21.15.92:0/44929521 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfe0009a70 con 0x7fbfe80d0d60 2023-11-28T11:23:10.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.270+0000 7fbfef5ff700 1 -- 172.21.15.92:0/44929521 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfe80cfff0 con 0x7fbfe80d0d60 2023-11-28T11:23:10.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.271+0000 7fbfef5ff700 1 -- 172.21.15.92:0/44929521 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfe80cfd10 con 0x7fbfe80d0d60 2023-11-28T11:23:10.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.272+0000 7fbfddffb700 1 -- 172.21.15.92:0/44929521 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbfe002e070 con 0x7fbfe80d0d60 2023-11-28T11:23:10.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.273+0000 7fbfddffb700 1 --2- 172.21.15.92:0/44929521 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fbfcc0682f0 0x7fbfcc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:10.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.273+0000 7fbfddffb700 1 -- 172.21.15.92:0/44929521 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fbfe0022070 con 0x7fbfe80d0d60 2023-11-28T11:23:10.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.273+0000 7fbfed39b700 1 --2- 172.21.15.92:0/44929521 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fbfcc0682f0 0x7fbfcc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:10.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.274+0000 7fbfed39b700 1 --2- 172.21.15.92:0/44929521 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fbfcc0682f0 0x7fbfcc06a7a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4000f60 tx=0x7fbfe4007d10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:10.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.277+0000 7fbfddffb700 1 -- 172.21.15.92:0/44929521 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbfe0054240 con 0x7fbfe80d0d60 2023-11-28T11:23:10.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.468+0000 7fbfef5ff700 1 -- 172.21.15.92:0/44929521 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fbfe80ce360 con 0x7fbfe80d0d60 2023-11-28T11:23:10.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.468+0000 7fbfddffb700 1 -- 172.21.15.92:0/44929521 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v52) v1 ==== 84+0+126 (secure 0 0 0) 0x7fbfe002c090 con 0x7fbfe80d0d60 2023-11-28T11:23:10.470 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:23:10.470 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-11-28T11:23:10.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:23:10.470 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 1, 2023-11-28T11:23:10.470 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": ".mgr" 2023-11-28T11:23:10.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:23:10.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:23:10.470 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 2, 2023-11-28T11:23:10.470 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": "rbd" 2023-11-28T11:23:10.471 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:23:10.471 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-11-28T11:23:10.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.471+0000 7fbfef5ff700 1 -- 172.21.15.92:0/44929521 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fbfcc0682f0 msgr2=0x7fbfcc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:10.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.471+0000 7fbfef5ff700 1 --2- 172.21.15.92:0/44929521 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fbfcc0682f0 0x7fbfcc06a7a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4000f60 tx=0x7fbfe4007d10 comp rx=0 tx=0).stop 2023-11-28T11:23:10.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.471+0000 7fbfef5ff700 1 -- 172.21.15.92:0/44929521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfe80d0d60 msgr2=0x7fbfe815a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:10.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.471+0000 7fbfef5ff700 1 --2- 172.21.15.92:0/44929521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfe80d0d60 0x7fbfe815a1f0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fbfe000b370 tx=0x7fbfe0004ad0 comp rx=0 tx=0).stop 2023-11-28T11:23:10.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.471+0000 7fbfef5ff700 1 -- 172.21.15.92:0/44929521 shutdown_connections 2023-11-28T11:23:10.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.471+0000 7fbfef5ff700 1 --2- 172.21.15.92:0/44929521 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fbfcc0682f0 0x7fbfcc06a7a0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:10.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.471+0000 7fbfef5ff700 1 --2- 172.21.15.92:0/44929521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbfe80d0d60 0x7fbfe815a1f0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:10.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.471+0000 7fbfef5ff700 1 --2- 172.21.15.92:0/44929521 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbfe80b8510 0x7fbfe8157680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:10.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.471+0000 7fbfef5ff700 1 --2- 172.21.15.92:0/44929521 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbfe80cd460 0x7fbfe81f29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:10.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.471+0000 7fbfef5ff700 1 -- 172.21.15.92:0/44929521 >> 172.21.15.92:0/44929521 conn(0x7fbfe80719a0 msgr2=0x7fbfe80b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:10.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.471+0000 7fbfef5ff700 1 -- 172.21.15.92:0/44929521 shutdown_connections 2023-11-28T11:23:10.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.471+0000 7fbfef5ff700 1 -- 172.21.15.92:0/44929521 wait complete. 2023-11-28T11:23:10.490 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T11:23:10.490 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-28T11:23:10.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.690+0000 7f22b9dff700 1 Processor -- start 2023-11-28T11:23:10.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.690+0000 7f22b9dff700 1 -- start start 2023-11-28T11:23:10.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.691+0000 7f22b9dff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22b40b8790 0x7f22b40cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:10.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.691+0000 7f22b9dff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f22b406c050 0x7f22b40d0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:10.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.691+0000 7f22b9dff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22b40d00b0 con 0x7f22b40cd460 2023-11-28T11:23:10.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.691+0000 7f22b9dff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22b40b5240 con 0x7f22b406c050 2023-11-28T11:23:10.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.691+0000 7f22b9dff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22b40b5520 con 0x7f22b40b8790 2023-11-28T11:23:10.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.692+0000 7f22b37fe700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22b40b8790 0x7f22b40cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:10.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.692+0000 7f22b3fff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f22b406c050 0x7f22b40d0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:10.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.692+0000 7f22b37fe700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22b40b8790 0x7f22b40cfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.92:47378/0 (socket says 172.21.15.92:47378) 2023-11-28T11:23:10.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.692+0000 7f22abfff700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f22b40cd460 0x7f22b406b940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.92:42846/0 (socket says 172.21.15.92:42846) 2023-11-28T11:23:10.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.692+0000 7f22b37fe700 1 -- 172.21.15.92:0/949408364 learned_addr learned my addr 172.21.15.92:0/949408364 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:10.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.692+0000 7f22b3fff700 1 -- 172.21.15.92:0/949408364 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22b40b8790 msgr2=0x7f22b40cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:10.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.692+0000 7f22b3fff700 1 --2- 172.21.15.92:0/949408364 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22b40b8790 0x7f22b40cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:10.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.692+0000 7f22b3fff700 1 -- 172.21.15.92:0/949408364 >> v1:172.21.15.169:6789/0 conn(0x7f22b40cd460 legacy=0x7f22b406b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:10.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.692+0000 7f22b3fff700 1 -- 172.21.15.92:0/949408364 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b40d0a80 con 0x7f22b406c050 2023-11-28T11:23:10.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.692+0000 7f22b2ffd700 1 -- 172.21.15.92:0/949408364 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2061846866 0 0) 0x7f22b40d00b0 con 0x7f22b40cd460 2023-11-28T11:23:10.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.692+0000 7f22b3fff700 1 --2- 172.21.15.92:0/949408364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22b406c050 0x7f22b40d0fa0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f22ac009d90 tx=0x7f22ac00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c4312f01e1600ab server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:10.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.693+0000 7f22b2ffd700 1 -- 172.21.15.92:0/949408364 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ac00e070 con 0x7f22b406c050 2023-11-28T11:23:10.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.693+0000 7f22b2ffd700 1 -- 172.21.15.92:0/949408364 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22ac00bb20 con 0x7f22b406c050 2023-11-28T11:23:10.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.693+0000 7f22b2ffd700 1 -- 172.21.15.92:0/949408364 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ac012430 con 0x7f22b406c050 2023-11-28T11:23:10.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.694+0000 7f22b9dff700 1 -- 172.21.15.92:0/949408364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22b406c050 msgr2=0x7f22b40d0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:10.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.694+0000 7f22b9dff700 1 --2- 172.21.15.92:0/949408364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22b406c050 0x7f22b40d0fa0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f22ac009d90 tx=0x7f22ac00b330 comp rx=0 tx=0).stop 2023-11-28T11:23:10.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.694+0000 7f22b9dff700 1 -- 172.21.15.92:0/949408364 shutdown_connections 2023-11-28T11:23:10.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.694+0000 7f22b9dff700 1 --2- 172.21.15.92:0/949408364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22b406c050 0x7f22b40d0fa0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:10.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.694+0000 7f22b9dff700 1 --2- 172.21.15.92:0/949408364 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22b40b8790 0x7f22b40cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:10.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.694+0000 7f22b9dff700 1 -- 172.21.15.92:0/949408364 >> 172.21.15.92:0/949408364 conn(0x7f22b40719a0 msgr2=0x7f22b4071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:10.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.694+0000 7f22b9dff700 1 -- 172.21.15.92:0/949408364 shutdown_connections 2023-11-28T11:23:10.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.694+0000 7f22b9dff700 1 -- 172.21.15.92:0/949408364 wait complete. 2023-11-28T11:23:10.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.696+0000 7f22b9dff700 1 Processor -- start 2023-11-28T11:23:10.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.696+0000 7f22b9dff700 1 -- start start 2023-11-28T11:23:10.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.697+0000 7f22b9dff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22b406c050 0x7f22b40cab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:10.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.697+0000 7f22b9dff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22b40b8790 0x7f22b40cc500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:10.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.697+0000 7f22b9dff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22b40cd460 0x7f22b415c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:10.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.697+0000 7f22b9dff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b40cb050 con 0x7f22b406c050 2023-11-28T11:23:10.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.697+0000 7f22b9dff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b40cca40 con 0x7f22b40b8790 2023-11-28T11:23:10.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.697+0000 7f22b9dff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22b4143c90 con 0x7f22b40cd460 2023-11-28T11:23:10.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.697+0000 7f22b37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22b406c050 0x7f22b40cab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:10.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.697+0000 7f22b37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22b406c050 0x7f22b40cab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40182/0 (socket says 172.21.15.92:40182) 2023-11-28T11:23:10.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.697+0000 7f22b37fe700 1 -- 172.21.15.92:0/2617409261 learned_addr learned my addr 172.21.15.92:0/2617409261 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:10.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.697+0000 7f22abfff700 1 --2- 172.21.15.92:0/2617409261 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22b40b8790 0x7f22b40cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:10.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.697+0000 7f22b3fff700 1 --2- 172.21.15.92:0/2617409261 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22b40cd460 0x7f22b415c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:10.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.697+0000 7f22b37fe700 1 -- 172.21.15.92:0/2617409261 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22b40cd460 msgr2=0x7f22b415c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:10.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.697+0000 7f22b37fe700 1 --2- 172.21.15.92:0/2617409261 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22b40cd460 0x7f22b415c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:10.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.697+0000 7f22b37fe700 1 -- 172.21.15.92:0/2617409261 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22b40b8790 msgr2=0x7f22b40cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:10.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.697+0000 7f22b37fe700 1 --2- 172.21.15.92:0/2617409261 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22b40b8790 0x7f22b40cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:10.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.697+0000 7f22b37fe700 1 -- 172.21.15.92:0/2617409261 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22ac009a70 con 0x7f22b406c050 2023-11-28T11:23:10.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.698+0000 7f22b37fe700 1 --2- 172.21.15.92:0/2617409261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22b406c050 0x7f22b40cab10 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f22a400e910 tx=0x7f22a400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:10.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.698+0000 7f22b0ff9700 1 -- 172.21.15.92:0/2617409261 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a40096a0 con 0x7f22b406c050 2023-11-28T11:23:10.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.698+0000 7f22b0ff9700 1 -- 172.21.15.92:0/2617409261 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22a400c850 con 0x7f22b406c050 2023-11-28T11:23:10.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.698+0000 7f22b0ff9700 1 -- 172.21.15.92:0/2617409261 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a400cc00 con 0x7f22b406c050 2023-11-28T11:23:10.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.698+0000 7f22b9dff700 1 -- 172.21.15.92:0/2617409261 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22b40d14e0 con 0x7f22b406c050 2023-11-28T11:23:10.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.698+0000 7f22b9dff700 1 -- 172.21.15.92:0/2617409261 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22b40b5800 con 0x7f22b406c050 2023-11-28T11:23:10.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.699+0000 7f22b9dff700 1 -- 172.21.15.92:0/2617409261 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22b40d1670 con 0x7f22b406c050 2023-11-28T11:23:10.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.700+0000 7f22b0ff9700 1 -- 172.21.15.92:0/2617409261 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f22a4019030 con 0x7f22b406c050 2023-11-28T11:23:10.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.701+0000 7f22b0ff9700 1 --2- 172.21.15.92:0/2617409261 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f22940682f0 0x7f229406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:10.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.701+0000 7f22b0ff9700 1 -- 172.21.15.92:0/2617409261 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f22a4085b70 con 0x7f22b406c050 2023-11-28T11:23:10.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.702+0000 7f22abfff700 1 --2- 172.21.15.92:0/2617409261 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f22940682f0 0x7f229406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:10.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.702+0000 7f22abfff700 1 --2- 172.21.15.92:0/2617409261 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f22940682f0 0x7f229406a7a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f22b4157f00 tx=0x7f229c007c90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:10.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.705+0000 7f22b0ff9700 1 -- 172.21.15.92:0/2617409261 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f22a4052460 con 0x7f22b406c050 2023-11-28T11:23:10.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:10.899+0000 7f22b9dff700 1 -- 172.21.15.92:0/2617409261 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f22b4067880 con 0x7f22b406c050 2023-11-28T11:23:11.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.744+0000 7f22b0ff9700 1 -- 172.21.15.92:0/2617409261 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v53) v1 ==== 138+0+0 (secure 0 0 0) 0x7f22a40210b0 con 0x7f22b406c050 2023-11-28T11:23:11.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.749+0000 7f22b9dff700 1 -- 172.21.15.92:0/2617409261 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f22940682f0 msgr2=0x7f229406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:11.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.749+0000 7f22b9dff700 1 --2- 172.21.15.92:0/2617409261 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f22940682f0 0x7f229406a7a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f22b4157f00 tx=0x7f229c007c90 comp rx=0 tx=0).stop 2023-11-28T11:23:11.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.749+0000 7f22b9dff700 1 -- 172.21.15.92:0/2617409261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22b406c050 msgr2=0x7f22b40cab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:11.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.749+0000 7f22b9dff700 1 --2- 172.21.15.92:0/2617409261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22b406c050 0x7f22b40cab10 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f22a400e910 tx=0x7f22a400ecd0 comp rx=0 tx=0).stop 2023-11-28T11:23:11.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.749+0000 7f22b9dff700 1 -- 172.21.15.92:0/2617409261 shutdown_connections 2023-11-28T11:23:11.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.749+0000 7f22b9dff700 1 --2- 172.21.15.92:0/2617409261 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f22b40cd460 0x7f22b415c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:11.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.749+0000 7f22b9dff700 1 --2- 172.21.15.92:0/2617409261 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f22940682f0 0x7f229406a7a0 secure :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f22b4157f00 tx=0x7f229c007c90 comp rx=0 tx=0).stop 2023-11-28T11:23:11.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.749+0000 7f22b9dff700 1 --2- 172.21.15.92:0/2617409261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22b406c050 0x7f22b40cab10 secure :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f22a400e910 tx=0x7f22a400ecd0 comp rx=0 tx=0).stop 2023-11-28T11:23:11.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.749+0000 7f22b9dff700 1 --2- 172.21.15.92:0/2617409261 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f22b40b8790 0x7f22b40cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:11.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.749+0000 7f22b9dff700 1 -- 172.21.15.92:0/2617409261 >> 172.21.15.92:0/2617409261 conn(0x7f22b40719a0 msgr2=0x7f22b406b940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:11.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.751+0000 7f22b9dff700 1 -- 172.21.15.92:0/2617409261 shutdown_connections 2023-11-28T11:23:11.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.751+0000 7f22b9dff700 1 -- 172.21.15.92:0/2617409261 wait complete. 2023-11-28T11:23:11.754 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' created 2023-11-28T11:23:11.774 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T11:23:11.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.982+0000 7f0c88dff700 1 Processor -- start 2023-11-28T11:23:11.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.982+0000 7f0c88dff700 1 -- start start 2023-11-28T11:23:11.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.983+0000 7f0c88dff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0c840b2280 0x7f0c840cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:11.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.983+0000 7f0c88dff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c840cd4d0 con 0x7f0c840d2e10 2023-11-28T11:23:11.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.983+0000 7f0c88dff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c840cf470 con 0x7f0c8415d590 2023-11-28T11:23:11.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.983+0000 7f0c88dff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c840cdce0 con 0x7f0c840b2280 2023-11-28T11:23:11.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.983+0000 7f0c8259c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f0c840d2e10 0x7f0c840d31e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36922/0 (socket says 172.21.15.92:36922) 2023-11-28T11:23:11.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.983+0000 7f0c8259c700 1 -- 172.21.15.92:0/1924589289 learned_addr learned my addr 172.21.15.92:0/1924589289 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:23:11.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.984+0000 7f0c82d9d700 1 --2- 172.21.15.92:0/1924589289 >> v2:172.21.15.169:3300/0 conn(0x7f0c840b2280 0x7f0c840cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:11.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.984+0000 7f0c8159a700 1 -- 172.21.15.92:0/1924589289 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2748844169 0 0) 0x7f0c840cd4d0 con 0x7f0c840d2e10 2023-11-28T11:23:11.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.984+0000 7f0c8159a700 1 -- 172.21.15.92:0/1924589289 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c6c0035b0 con 0x7f0c840d2e10 2023-11-28T11:23:11.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.984+0000 7f0c8159a700 1 -- 172.21.15.92:0/1924589289 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3983616562 0 0) 0x7f0c840cf470 con 0x7f0c8415d590 2023-11-28T11:23:11.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.984+0000 7f0c8159a700 1 -- 172.21.15.92:0/1924589289 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c840cd4d0 con 0x7f0c8415d590 2023-11-28T11:23:11.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.984+0000 7f0c8159a700 1 -- 172.21.15.92:0/1924589289 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1127924186 0 0) 0x7f0c6c0035b0 con 0x7f0c840d2e10 2023-11-28T11:23:11.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.984+0000 7f0c8159a700 1 -- 172.21.15.92:0/1924589289 >> v2:172.21.15.169:3300/0 conn(0x7f0c840b2280 msgr2=0x7f0c840cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:11.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.984+0000 7f0c8159a700 1 --2- 172.21.15.92:0/1924589289 >> v2:172.21.15.169:3300/0 conn(0x7f0c840b2280 0x7f0c840cf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:11.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.984+0000 7f0c8159a700 1 -- 172.21.15.92:0/1924589289 >> v1:172.21.15.169:6789/0 conn(0x7f0c8415d590 legacy=0x7f0c8415dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:11.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.984+0000 7f0c8159a700 1 -- 172.21.15.92:0/1924589289 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c84156fa0 con 0x7f0c840d2e10 2023-11-28T11:23:11.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.984+0000 7f0c8159a700 1 -- 172.21.15.92:0/1924589289 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532161587 0 0) 0x7f0c74001670 con 0x7f0c840d2e10 2023-11-28T11:23:11.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.984+0000 7f0c8159a700 1 -- 172.21.15.92:0/1924589289 >> v1:172.21.15.92:6789/0 conn(0x7f0c840d2e10 legacy=0x7f0c840d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:11.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.984+0000 7f0c8159a700 1 --2- 172.21.15.92:0/1924589289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c6c003b90 0x7f0c6c003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:11.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.984+0000 7f0c8159a700 1 --2- 172.21.15.92:0/1924589289 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c6c004850 0x7f0c6c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:11.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.984+0000 7f0c8159a700 1 --2- 172.21.15.92:0/1924589289 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c6c005200 0x7f0c6c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:11.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.984+0000 7f0c8159a700 1 -- 172.21.15.92:0/1924589289 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c840cdce0 con 0x7f0c6c005200 2023-11-28T11:23:11.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.984+0000 7f0c8159a700 1 -- 172.21.15.92:0/1924589289 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6c00c110 con 0x7f0c6c003b90 2023-11-28T11:23:11.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.984+0000 7f0c8159a700 1 -- 172.21.15.92:0/1924589289 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6c00c2d0 con 0x7f0c6c004850 2023-11-28T11:23:11.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.985+0000 7f0c82d9d700 1 --2- 172.21.15.92:0/1924589289 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c6c005200 0x7f0c6c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:11.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.985+0000 7f0c8259c700 1 --2- 172.21.15.92:0/1924589289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c6c003b90 0x7f0c6c003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:11.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.985+0000 7f0c81d9b700 1 --2- 172.21.15.92:0/1924589289 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c6c004850 0x7f0c6c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:11.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.985+0000 7f0c82d9d700 1 -- 172.21.15.92:0/1924589289 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c6c004850 msgr2=0x7f0c6c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:11.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.985+0000 7f0c82d9d700 1 --2- 172.21.15.92:0/1924589289 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c6c004850 0x7f0c6c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:11.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.985+0000 7f0c82d9d700 1 -- 172.21.15.92:0/1924589289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c6c003b90 msgr2=0x7f0c6c003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:11.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.985+0000 7f0c82d9d700 1 --2- 172.21.15.92:0/1924589289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c6c003b90 0x7f0c6c003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:11.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.985+0000 7f0c82d9d700 1 -- 172.21.15.92:0/1924589289 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c6c0035b0 con 0x7f0c6c005200 2023-11-28T11:23:11.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.985+0000 7f0c82d9d700 1 -- 172.21.15.92:0/1924589289 reap_dead start 2023-11-28T11:23:11.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.985+0000 7f0c82d9d700 1 --2- 172.21.15.92:0/1924589289 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c6c005200 0x7f0c6c00ba10 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c009f90 tx=0x7f0c6c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edb4ee0acc203625 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:11.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.985+0000 7f0c8159a700 1 -- 172.21.15.92:0/1924589289 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c003a20 con 0x7f0c6c005200 2023-11-28T11:23:11.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.986+0000 7f0c8159a700 1 -- 172.21.15.92:0/1924589289 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c7c00f040 con 0x7f0c6c005200 2023-11-28T11:23:11.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.986+0000 7f0c8159a700 1 -- 172.21.15.92:0/1924589289 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c003dc0 con 0x7f0c6c005200 2023-11-28T11:23:11.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.986+0000 7f0c88dff700 1 -- 172.21.15.92:0/1924589289 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c6c005200 msgr2=0x7f0c6c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:11.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.986+0000 7f0c88dff700 1 --2- 172.21.15.92:0/1924589289 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c6c005200 0x7f0c6c00ba10 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c009f90 tx=0x7f0c6c005880 comp rx=0 tx=0).stop 2023-11-28T11:23:11.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.986+0000 7f0c88dff700 1 -- 172.21.15.92:0/1924589289 shutdown_connections 2023-11-28T11:23:11.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.986+0000 7f0c88dff700 1 --2- 172.21.15.92:0/1924589289 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c6c005200 0x7f0c6c00ba10 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:11.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.986+0000 7f0c88dff700 1 -- 172.21.15.92:0/1924589289 >> 172.21.15.92:0/1924589289 conn(0x7f0c840719a0 msgr2=0x7f0c84071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:11.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.986+0000 7f0c88dff700 1 -- 172.21.15.92:0/1924589289 shutdown_connections 2023-11-28T11:23:11.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.986+0000 7f0c88dff700 1 -- 172.21.15.92:0/1924589289 wait complete. 2023-11-28T11:23:11.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.988+0000 7f0c88dff700 1 Processor -- start 2023-11-28T11:23:11.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.988+0000 7f0c88dff700 1 -- start start 2023-11-28T11:23:11.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.989+0000 7f0c88dff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c6c005200 0x7f0c8414cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:11.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.989+0000 7f0c88dff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c8414d2e0 0x7f0c840c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:11.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.989+0000 7f0c88dff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c840c8f80 0x7f0c840c6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:11.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.989+0000 7f0c88dff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c840c9f50 con 0x7f0c840c8f80 2023-11-28T11:23:11.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.989+0000 7f0c88dff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c840c9440 con 0x7f0c8414d2e0 2023-11-28T11:23:11.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.989+0000 7f0c88dff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c840c9720 con 0x7f0c6c005200 2023-11-28T11:23:11.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.989+0000 7f0c82d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c840c8f80 0x7f0c840c6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:11.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.989+0000 7f0c82d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c840c8f80 0x7f0c840c6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40200/0 (socket says 172.21.15.92:40200) 2023-11-28T11:23:11.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.989+0000 7f0c82d9d700 1 -- 172.21.15.92:0/3261421348 learned_addr learned my addr 172.21.15.92:0/3261421348 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:11.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.989+0000 7f0c8259c700 1 --2- 172.21.15.92:0/3261421348 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c6c005200 0x7f0c8414cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:11.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.989+0000 7f0c81d9b700 1 --2- 172.21.15.92:0/3261421348 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c8414d2e0 0x7f0c840c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:11.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.990+0000 7f0c82d9d700 1 -- 172.21.15.92:0/3261421348 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c6c005200 msgr2=0x7f0c8414cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:11.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.990+0000 7f0c82d9d700 1 --2- 172.21.15.92:0/3261421348 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c6c005200 0x7f0c8414cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:11.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.990+0000 7f0c82d9d700 1 -- 172.21.15.92:0/3261421348 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c8414d2e0 msgr2=0x7f0c840c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:11.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.990+0000 7f0c82d9d700 1 --2- 172.21.15.92:0/3261421348 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c8414d2e0 0x7f0c840c8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:11.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.990+0000 7f0c82d9d700 1 -- 172.21.15.92:0/3261421348 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c840cd720 con 0x7f0c840c8f80 2023-11-28T11:23:11.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.990+0000 7f0c8259c700 1 --2- 172.21.15.92:0/3261421348 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c6c005200 0x7f0c8414cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:23:11.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.990+0000 7f0c82d9d700 1 --2- 172.21.15.92:0/3261421348 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c840c8f80 0x7f0c840c6970 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f0c840d2d60 tx=0x7f0c7c004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:11.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.990+0000 7f0c72ffd700 1 -- 172.21.15.92:0/3261421348 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c004230 con 0x7f0c840c8f80 2023-11-28T11:23:11.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.990+0000 7f0c72ffd700 1 -- 172.21.15.92:0/3261421348 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c7c004390 con 0x7f0c840c8f80 2023-11-28T11:23:11.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.990+0000 7f0c72ffd700 1 -- 172.21.15.92:0/3261421348 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c004740 con 0x7f0c840c8f80 2023-11-28T11:23:11.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.990+0000 7f0c88dff700 1 -- 172.21.15.92:0/3261421348 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c840b2680 con 0x7f0c840c8f80 2023-11-28T11:23:11.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.990+0000 7f0c88dff700 1 -- 172.21.15.92:0/3261421348 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c840cda00 con 0x7f0c840c8f80 2023-11-28T11:23:11.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.991+0000 7f0c88dff700 1 -- 172.21.15.92:0/3261421348 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c84067880 con 0x7f0c840c8f80 2023-11-28T11:23:11.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.992+0000 7f0c72ffd700 1 -- 172.21.15.92:0/3261421348 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0c7c014480 con 0x7f0c840c8f80 2023-11-28T11:23:11.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.993+0000 7f0c72ffd700 1 --2- 172.21.15.92:0/3261421348 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f0c640682f0 0x7f0c6406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:11.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.993+0000 7f0c72ffd700 1 -- 172.21.15.92:0/3261421348 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5900+0+0 (secure 0 0 0) 0x7f0c7c087640 con 0x7f0c840c8f80 2023-11-28T11:23:11.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.993+0000 7f0c8259c700 1 --2- 172.21.15.92:0/3261421348 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f0c640682f0 0x7f0c6406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:11.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.994+0000 7f0c8259c700 1 --2- 172.21.15.92:0/3261421348 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f0c640682f0 0x7f0c6406a7a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0c8415c6e0 tx=0x7f0c74009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:11.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:11.998+0000 7f0c72ffd700 1 -- 172.21.15.92:0/3261421348 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0c7c053d70 con 0x7f0c840c8f80 2023-11-28T11:23:12.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:12.187+0000 7f0c88dff700 1 -- 172.21.15.92:0/3261421348 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f0c840b1900 con 0x7f0c840c8f80 2023-11-28T11:23:12.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:12.754+0000 7f0c72ffd700 1 -- 172.21.15.92:0/3261421348 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v54) v1 ==== 170+0+0 (secure 0 0 0) 0x7f0c7c057390 con 0x7f0c840c8f80 2023-11-28T11:23:12.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:12.760+0000 7f0c88dff700 1 -- 172.21.15.92:0/3261421348 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f0c640682f0 msgr2=0x7f0c6406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:12.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:12.760+0000 7f0c88dff700 1 --2- 172.21.15.92:0/3261421348 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f0c640682f0 0x7f0c6406a7a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0c8415c6e0 tx=0x7f0c74009040 comp rx=0 tx=0).stop 2023-11-28T11:23:12.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:12.760+0000 7f0c88dff700 1 -- 172.21.15.92:0/3261421348 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c840c8f80 msgr2=0x7f0c840c6970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:12.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:12.760+0000 7f0c88dff700 1 --2- 172.21.15.92:0/3261421348 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c840c8f80 0x7f0c840c6970 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f0c840d2d60 tx=0x7f0c7c004b40 comp rx=0 tx=0).stop 2023-11-28T11:23:12.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:12.760+0000 7f0c88dff700 1 -- 172.21.15.92:0/3261421348 shutdown_connections 2023-11-28T11:23:12.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:12.760+0000 7f0c88dff700 1 --2- 172.21.15.92:0/3261421348 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f0c640682f0 0x7f0c6406a7a0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:12.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:12.760+0000 7f0c88dff700 1 --2- 172.21.15.92:0/3261421348 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c840c8f80 0x7f0c840c6970 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:12.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:12.760+0000 7f0c88dff700 1 --2- 172.21.15.92:0/3261421348 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c6c005200 0x7f0c8414cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:12.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:12.760+0000 7f0c88dff700 1 --2- 172.21.15.92:0/3261421348 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c8414d2e0 0x7f0c840c8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:12.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:12.760+0000 7f0c88dff700 1 -- 172.21.15.92:0/3261421348 >> 172.21.15.92:0/3261421348 conn(0x7f0c840719a0 msgr2=0x7f0c84071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:12.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:12.761+0000 7f0c88dff700 1 -- 172.21.15.92:0/3261421348 shutdown_connections 2023-11-28T11:23:12.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:12.761+0000 7f0c88dff700 1 -- 172.21.15.92:0/3261421348 wait complete. 2023-11-28T11:23:12.767 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' created 2023-11-28T11:23:12.795 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T11:23:12.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:12.994+0000 7f95d59ff700 1 Processor -- start 2023-11-28T11:23:12.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:12.994+0000 7f95d59ff700 1 -- start start 2023-11-28T11:23:13.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:12.996+0000 7f95d59ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95d00b8510 0x7f95d006b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:13.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:12.996+0000 7f95d59ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f95d00d0d60 0x7f95d00d11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:13.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:12.996+0000 7f95d59ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95d00b88e0 con 0x7f95d00cd460 2023-11-28T11:23:13.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:12.996+0000 7f95d59ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95d00cdbd0 con 0x7f95d00d0d60 2023-11-28T11:23:13.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:12.996+0000 7f95d59ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95d00d02d0 con 0x7f95d00b8510 2023-11-28T11:23:13.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:12.996+0000 7f95c7fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f95d00cd460 0x7f95d006beb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36936/0 (socket says 172.21.15.92:36936) 2023-11-28T11:23:13.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:12.996+0000 7f95c7fff700 1 -- 172.21.15.92:0/1721425532 learned_addr learned my addr 172.21.15.92:0/1721425532 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:23:13.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:12.996+0000 7f95ceffd700 1 --2- 172.21.15.92:0/1721425532 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95d00b8510 0x7f95d006b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:13.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:12.996+0000 7f95cf7fe700 1 --2- 172.21.15.92:0/1721425532 >> v2:172.21.15.169:3300/0 conn(0x7f95d00d0d60 0x7f95d00d11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:13.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:12.996+0000 7f95ce7fc700 1 -- 172.21.15.92:0/1721425532 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 154718924 0 0) 0x7f95d00b88e0 con 0x7f95d00cd460 2023-11-28T11:23:13.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:12.997+0000 7f95ce7fc700 1 -- 172.21.15.92:0/1721425532 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95bc0035b0 con 0x7f95d00cd460 2023-11-28T11:23:13.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:12.997+0000 7f95ceffd700 1 -- 172.21.15.92:0/1721425532 >> v2:172.21.15.169:3300/0 conn(0x7f95d00d0d60 msgr2=0x7f95d00d11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:13.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:12.997+0000 7f95ceffd700 1 --2- 172.21.15.92:0/1721425532 >> v2:172.21.15.169:3300/0 conn(0x7f95d00d0d60 0x7f95d00d11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:13.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:12.997+0000 7f95ceffd700 1 -- 172.21.15.92:0/1721425532 >> v1:172.21.15.92:6789/0 conn(0x7f95d00cd460 legacy=0x7f95d006beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:13.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:12.997+0000 7f95ceffd700 1 -- 172.21.15.92:0/1721425532 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d00cd8f0 con 0x7f95d00b8510 2023-11-28T11:23:13.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:12.997+0000 7f95cf7fe700 1 --2- 172.21.15.92:0/1721425532 >> v2:172.21.15.169:3300/0 conn(0x7f95d00d0d60 0x7f95d00d11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:23:13.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:12.997+0000 7f95ceffd700 1 --2- 172.21.15.92:0/1721425532 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95d00b8510 0x7f95d006b970 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f95c0009840 tx=0x7f95c0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d16e4a42f9e527ae server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:13.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:12.998+0000 7f95ce7fc700 1 -- 172.21.15.92:0/1721425532 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c0011830 con 0x7f95d00b8510 2023-11-28T11:23:13.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:12.998+0000 7f95ce7fc700 1 -- 172.21.15.92:0/1721425532 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c0011990 con 0x7f95d00b8510 2023-11-28T11:23:13.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:12.998+0000 7f95ce7fc700 1 -- 172.21.15.92:0/1721425532 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c0011d20 con 0x7f95d00b8510 2023-11-28T11:23:13.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:12.998+0000 7f95d59ff700 1 -- 172.21.15.92:0/1721425532 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95d00b8510 msgr2=0x7f95d006b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:13.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:12.998+0000 7f95d59ff700 1 --2- 172.21.15.92:0/1721425532 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95d00b8510 0x7f95d006b970 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f95c0009840 tx=0x7f95c0009b50 comp rx=0 tx=0).stop 2023-11-28T11:23:13.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:12.998+0000 7f95d59ff700 1 -- 172.21.15.92:0/1721425532 shutdown_connections 2023-11-28T11:23:13.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:12.998+0000 7f95d59ff700 1 --2- 172.21.15.92:0/1721425532 >> v2:172.21.15.169:3300/0 conn(0x7f95d00d0d60 0x7f95d00d11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:13.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:12.998+0000 7f95d59ff700 1 --2- 172.21.15.92:0/1721425532 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95d00b8510 0x7f95d006b970 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:13.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:12.998+0000 7f95d59ff700 1 -- 172.21.15.92:0/1721425532 >> 172.21.15.92:0/1721425532 conn(0x7f95d00719a0 msgr2=0x7f95d0071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:13.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:12.998+0000 7f95d59ff700 1 -- 172.21.15.92:0/1721425532 shutdown_connections 2023-11-28T11:23:13.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:12.998+0000 7f95d59ff700 1 -- 172.21.15.92:0/1721425532 wait complete. 2023-11-28T11:23:13.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.000+0000 7f95d59ff700 1 Processor -- start 2023-11-28T11:23:13.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.000+0000 7f95d59ff700 1 -- start start 2023-11-28T11:23:13.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.001+0000 7f95d59ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95d00b8510 0x7f95d01f6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:13.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.001+0000 7f95d59ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95d00cd460 0x7f95d00ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:13.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.001+0000 7f95d59ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95d00d0d60 0x7f95d015a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:13.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.001+0000 7f95d59ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d01591a0 con 0x7f95d00cd460 2023-11-28T11:23:13.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.001+0000 7f95d59ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d015ab90 con 0x7f95d00b8510 2023-11-28T11:23:13.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.001+0000 7f95d59ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95d00cd030 con 0x7f95d00d0d60 2023-11-28T11:23:13.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.001+0000 7f95c7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95d00cd460 0x7f95d00ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:13.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.001+0000 7f95c7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95d00cd460 0x7f95d00ccaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40202/0 (socket says 172.21.15.92:40202) 2023-11-28T11:23:13.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.001+0000 7f95c7fff700 1 -- 172.21.15.92:0/4168627791 learned_addr learned my addr 172.21.15.92:0/4168627791 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:13.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.002+0000 7f95ceffd700 1 --2- 172.21.15.92:0/4168627791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95d00b8510 0x7f95d01f6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:13.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.002+0000 7f95cf7fe700 1 --2- 172.21.15.92:0/4168627791 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95d00d0d60 0x7f95d015a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:13.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.002+0000 7f95c7fff700 1 -- 172.21.15.92:0/4168627791 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95d00d0d60 msgr2=0x7f95d015a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:13.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.002+0000 7f95c7fff700 1 --2- 172.21.15.92:0/4168627791 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95d00d0d60 0x7f95d015a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:13.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.002+0000 7f95c7fff700 1 -- 172.21.15.92:0/4168627791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95d00b8510 msgr2=0x7f95d01f6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:13.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.002+0000 7f95c7fff700 1 --2- 172.21.15.92:0/4168627791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95d00b8510 0x7f95d01f6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:13.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.002+0000 7f95c7fff700 1 -- 172.21.15.92:0/4168627791 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c8009a70 con 0x7f95d00cd460 2023-11-28T11:23:13.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.002+0000 7f95c7fff700 1 --2- 172.21.15.92:0/4168627791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95d00cd460 0x7f95d00ccaf0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f95b800c730 tx=0x7f95b800caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:13.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.002+0000 7f95ccff9700 1 -- 172.21.15.92:0/4168627791 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b8005d70 con 0x7f95d00cd460 2023-11-28T11:23:13.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.002+0000 7f95ccff9700 1 -- 172.21.15.92:0/4168627791 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95b80070d0 con 0x7f95d00cd460 2023-11-28T11:23:13.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.002+0000 7f95d59ff700 1 -- 172.21.15.92:0/4168627791 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95c00094a0 con 0x7f95d00cd460 2023-11-28T11:23:13.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.002+0000 7f95ccff9700 1 -- 172.21.15.92:0/4168627791 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b8007440 con 0x7f95d00cd460 2023-11-28T11:23:13.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.002+0000 7f95d59ff700 1 -- 172.21.15.92:0/4168627791 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95d00cfff0 con 0x7f95d00cd460 2023-11-28T11:23:13.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.003+0000 7f95d59ff700 1 -- 172.21.15.92:0/4168627791 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95d00cfd10 con 0x7f95d00cd460 2023-11-28T11:23:13.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.004+0000 7f95ccff9700 1 -- 172.21.15.92:0/4168627791 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f95b8019030 con 0x7f95d00cd460 2023-11-28T11:23:13.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.005+0000 7f95ccff9700 1 --2- 172.21.15.92:0/4168627791 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f95b00682f0 0x7f95b006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:13.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.005+0000 7f95ccff9700 1 -- 172.21.15.92:0/4168627791 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6282+0+0 (secure 0 0 0) 0x7f95b8085180 con 0x7f95d00cd460 2023-11-28T11:23:13.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.006+0000 7f95ceffd700 1 --2- 172.21.15.92:0/4168627791 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f95b00682f0 0x7f95b006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:13.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.006+0000 7f95ceffd700 1 --2- 172.21.15.92:0/4168627791 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f95b00682f0 0x7f95b006a7a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f95c0009fd0 tx=0x7f95c0008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:13.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.010+0000 7f95ccff9700 1 -- 172.21.15.92:0/4168627791 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f95b804b5e0 con 0x7f95d00cd460 2023-11-28T11:23:13.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.245+0000 7f95d59ff700 1 -- 172.21.15.92:0/4168627791 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f95d00ceca0 con 0x7f95d00cd460 2023-11-28T11:23:13.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.255+0000 7f95ccff9700 1 -- 172.21.15.92:0/4168627791 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v21) v1 ==== 172+0+0 (secure 0 0 0) 0x7f95b80518c0 con 0x7f95d00cd460 2023-11-28T11:23:13.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.259+0000 7f95d59ff700 1 -- 172.21.15.92:0/4168627791 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f95b00682f0 msgr2=0x7f95b006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:13.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.259+0000 7f95d59ff700 1 --2- 172.21.15.92:0/4168627791 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f95b00682f0 0x7f95b006a7a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f95c0009fd0 tx=0x7f95c0008030 comp rx=0 tx=0).stop 2023-11-28T11:23:13.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.259+0000 7f95d59ff700 1 -- 172.21.15.92:0/4168627791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95d00cd460 msgr2=0x7f95d00ccaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:13.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.259+0000 7f95d59ff700 1 --2- 172.21.15.92:0/4168627791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95d00cd460 0x7f95d00ccaf0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f95b800c730 tx=0x7f95b800caf0 comp rx=0 tx=0).stop 2023-11-28T11:23:13.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.264+0000 7f95d59ff700 1 -- 172.21.15.92:0/4168627791 shutdown_connections 2023-11-28T11:23:13.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.264+0000 7f95d59ff700 1 --2- 172.21.15.92:0/4168627791 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f95d00d0d60 0x7f95d015a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:13.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.264+0000 7f95d59ff700 1 --2- 172.21.15.92:0/4168627791 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f95d00b8510 0x7f95d01f6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:13.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.264+0000 7f95d59ff700 1 --2- 172.21.15.92:0/4168627791 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f95b00682f0 0x7f95b006a7a0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:13.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.264+0000 7f95d59ff700 1 --2- 172.21.15.92:0/4168627791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95d00cd460 0x7f95d00ccaf0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:13.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.264+0000 7f95d59ff700 1 -- 172.21.15.92:0/4168627791 >> 172.21.15.92:0/4168627791 conn(0x7f95d00719a0 msgr2=0x7f95d00d1f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:13.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.265+0000 7f95d59ff700 1 -- 172.21.15.92:0/4168627791 shutdown_connections 2023-11-28T11:23:13.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.266+0000 7f95d59ff700 1 -- 172.21.15.92:0/4168627791 wait complete. 2023-11-28T11:23:13.270 INFO:teuthology.orchestra.run.smithi092.stderr:new fs with metadata pool 8 and data pool 9 2023-11-28T11:23:13.300 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:23:13.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.495+0000 7fc5f3bff700 1 Processor -- start 2023-11-28T11:23:13.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.495+0000 7fc5f3bff700 1 -- start start 2023-11-28T11:23:13.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.496+0000 7fc5f3bff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc5ec0cd460 0x7fc5ec06c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:13.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.496+0000 7fc5f3bff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc5ec0d0d60 0x7fc5ec0d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:13.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.496+0000 7fc5f3bff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5ec0b88e0 con 0x7fc5ec0b8510 2023-11-28T11:23:13.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.496+0000 7fc5f3bff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5ec0cdbd0 con 0x7fc5ec0cd460 2023-11-28T11:23:13.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.496+0000 7fc5f3bff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5ec0d02d0 con 0x7fc5ec0d0d60 2023-11-28T11:23:13.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.497+0000 7fc5f119a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc5ec0cd460 0x7fc5ec06c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:13.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.497+0000 7fc5f119a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc5ec0cd460 0x7fc5ec06c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40210/0 (socket says 172.21.15.92:40210) 2023-11-28T11:23:13.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.497+0000 7fc5f119a700 1 -- 172.21.15.92:0/382823469 learned_addr learned my addr 172.21.15.92:0/382823469 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:13.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.497+0000 7fc5f219c700 1 --2- 172.21.15.92:0/382823469 >> v2:172.21.15.169:3300/0 conn(0x7fc5ec0d0d60 0x7fc5ec0d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:13.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.497+0000 7fc5f119a700 1 -- 172.21.15.92:0/382823469 >> v2:172.21.15.169:3300/0 conn(0x7fc5ec0d0d60 msgr2=0x7fc5ec0d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:13.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.497+0000 7fc5f119a700 1 --2- 172.21.15.92:0/382823469 >> v2:172.21.15.169:3300/0 conn(0x7fc5ec0d0d60 0x7fc5ec0d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:13.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.497+0000 7fc5f119a700 1 -- 172.21.15.92:0/382823469 >> v1:172.21.15.169:6789/0 conn(0x7fc5ec0b8510 legacy=0x7fc5ec06b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:13.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.497+0000 7fc5f119a700 1 -- 172.21.15.92:0/382823469 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5ec0cd8f0 con 0x7fc5ec0cd460 2023-11-28T11:23:13.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.497+0000 7fc5f119a700 1 --2- 172.21.15.92:0/382823469 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5ec0cd460 0x7fc5ec06c080 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec0ce8c0 tx=0x7fc5dc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea832f89eb431e94 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:13.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.498+0000 7fc5f0999700 1 -- 172.21.15.92:0/382823469 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc00b860 con 0x7fc5ec0cd460 2023-11-28T11:23:13.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.498+0000 7fc5f0999700 1 -- 172.21.15.92:0/382823469 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5dc00b9c0 con 0x7fc5ec0cd460 2023-11-28T11:23:13.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.498+0000 7fc5f0999700 1 -- 172.21.15.92:0/382823469 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc00bd50 con 0x7fc5ec0cd460 2023-11-28T11:23:13.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.498+0000 7fc5f3bff700 1 -- 172.21.15.92:0/382823469 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5ec0cd460 msgr2=0x7fc5ec06c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:13.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.498+0000 7fc5f3bff700 1 --2- 172.21.15.92:0/382823469 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5ec0cd460 0x7fc5ec06c080 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec0ce8c0 tx=0x7fc5dc009c00 comp rx=0 tx=0).stop 2023-11-28T11:23:13.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.498+0000 7fc5f3bff700 1 -- 172.21.15.92:0/382823469 shutdown_connections 2023-11-28T11:23:13.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.498+0000 7fc5f3bff700 1 --2- 172.21.15.92:0/382823469 >> v2:172.21.15.169:3300/0 conn(0x7fc5ec0d0d60 0x7fc5ec0d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:13.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.498+0000 7fc5f3bff700 1 --2- 172.21.15.92:0/382823469 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5ec0cd460 0x7fc5ec06c080 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:13.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.498+0000 7fc5f3bff700 1 -- 172.21.15.92:0/382823469 >> 172.21.15.92:0/382823469 conn(0x7fc5ec0719a0 msgr2=0x7fc5ec071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:13.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.498+0000 7fc5f3bff700 1 -- 172.21.15.92:0/382823469 shutdown_connections 2023-11-28T11:23:13.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.498+0000 7fc5f3bff700 1 -- 172.21.15.92:0/382823469 wait complete. 2023-11-28T11:23:13.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.501+0000 7fc5f3bff700 1 Processor -- start 2023-11-28T11:23:13.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.501+0000 7fc5f3bff700 1 -- start start 2023-11-28T11:23:13.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.502+0000 7fc5f3bff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5ec0b8510 0x7fc5ec1e1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:13.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.502+0000 7fc5f3bff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5ec0cd460 0x7fc5ec1e4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:13.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.502+0000 7fc5f3bff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5ec0d0d60 0x7fc5ec14de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:13.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.502+0000 7fc5f3bff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ec1e5120 con 0x7fc5ec0d0d60 2023-11-28T11:23:13.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.502+0000 7fc5f3bff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ec1e2320 con 0x7fc5ec0cd460 2023-11-28T11:23:13.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.502+0000 7fc5f3bff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ec1e2600 con 0x7fc5ec0b8510 2023-11-28T11:23:13.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.502+0000 7fc5f219c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5ec0d0d60 0x7fc5ec14de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:13.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.502+0000 7fc5f219c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5ec0d0d60 0x7fc5ec14de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40222/0 (socket says 172.21.15.92:40222) 2023-11-28T11:23:13.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.502+0000 7fc5f219c700 1 -- 172.21.15.92:0/1845845316 learned_addr learned my addr 172.21.15.92:0/1845845316 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:13.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.502+0000 7fc5f199b700 1 --2- 172.21.15.92:0/1845845316 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5ec0b8510 0x7fc5ec1e1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:13.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.502+0000 7fc5f119a700 1 --2- 172.21.15.92:0/1845845316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5ec0cd460 0x7fc5ec1e4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:13.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.503+0000 7fc5f219c700 1 -- 172.21.15.92:0/1845845316 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5ec0b8510 msgr2=0x7fc5ec1e1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:13.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.503+0000 7fc5f219c700 1 --2- 172.21.15.92:0/1845845316 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5ec0b8510 0x7fc5ec1e1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:13.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.503+0000 7fc5f219c700 1 -- 172.21.15.92:0/1845845316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5ec0cd460 msgr2=0x7fc5ec1e4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:13.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.503+0000 7fc5f219c700 1 --2- 172.21.15.92:0/1845845316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5ec0cd460 0x7fc5ec1e4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:13.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.503+0000 7fc5f219c700 1 -- 172.21.15.92:0/1845845316 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5dc009550 con 0x7fc5ec0d0d60 2023-11-28T11:23:13.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.503+0000 7fc5f219c700 1 --2- 172.21.15.92:0/1845845316 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5ec0d0d60 0x7fc5ec14de50 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4007e60 tx=0x7fc5e400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:13.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.503+0000 7fc5e27fc700 1 -- 172.21.15.92:0/1845845316 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e400ed30 con 0x7fc5ec0d0d60 2023-11-28T11:23:13.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.503+0000 7fc5e27fc700 1 -- 172.21.15.92:0/1845845316 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5e4014420 con 0x7fc5ec0d0d60 2023-11-28T11:23:13.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.503+0000 7fc5e27fc700 1 -- 172.21.15.92:0/1845845316 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e40147d0 con 0x7fc5ec0d0d60 2023-11-28T11:23:13.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.503+0000 7fc5f3bff700 1 -- 172.21.15.92:0/1845845316 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5ec0cfff0 con 0x7fc5ec0d0d60 2023-11-28T11:23:13.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.503+0000 7fc5f3bff700 1 -- 172.21.15.92:0/1845845316 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5ec0cfd10 con 0x7fc5ec0d0d60 2023-11-28T11:23:13.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.504+0000 7fc5f3bff700 1 -- 172.21.15.92:0/1845845316 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5ec067880 con 0x7fc5ec0d0d60 2023-11-28T11:23:13.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.505+0000 7fc5e27fc700 1 -- 172.21.15.92:0/1845845316 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc5e4012050 con 0x7fc5ec0d0d60 2023-11-28T11:23:13.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.506+0000 7fc5e27fc700 1 --2- 172.21.15.92:0/1845845316 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc5d00682f0 0x7fc5d006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:13.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.506+0000 7fc5e27fc700 1 -- 172.21.15.92:0/1845845316 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fc5e4086380 con 0x7fc5ec0d0d60 2023-11-28T11:23:13.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.506+0000 7fc5f199b700 1 --2- 172.21.15.92:0/1845845316 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc5d00682f0 0x7fc5d006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:13.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.507+0000 7fc5f199b700 1 --2- 172.21.15.92:0/1845845316 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc5d00682f0 0x7fc5d006a7a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8007fd0 tx=0x7fc5e8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:13.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.511+0000 7fc5e27fc700 1 -- 172.21.15.92:0/1845845316 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc5e40529b0 con 0x7fc5ec0d0d60 2023-11-28T11:23:13.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.701+0000 7fc5f3bff700 1 -- 172.21.15.92:0/1845845316 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc5ec0d2940 con 0x7fc5ec0d0d60 2023-11-28T11:23:13.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.703+0000 7fc5e27fc700 1 -- 172.21.15.92:0/1845845316 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+18193 (secure 0 0 0) 0x7fc5e4055fd0 con 0x7fc5ec0d0d60 2023-11-28T11:23:13.704 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:23:13.704 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":55,"fsid":"68956e0c-c16f-47cd-a220-f98364b0831d","created":"2023-11-28T11:20:28.697573+0000","modified":"2023-11-28T11:23:13.246802+0000","last_up_change":"2023-11-28T11:20:41.257011+0000","last_in_change":"2023-11-28T11:20:32.249767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:20:41.925237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:20:48.682497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:23:10.939884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-28T11:23:12.227433+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6abe483f-0e03-4a26-b9cf-62041291747e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":115823},{"type":"v1","addr":"172.21.15.92:6809","nonce":115823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":115823},{"type":"v1","addr":"172.21.15.92:6813","nonce":115823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":115823},{"type":"v1","addr":"172.21.15.92:6821","nonce":115823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":115823},{"type":"v1","addr":"172.21.15.92:6817","nonce":115823}]},"public_addr":"172.21.15.92:6809/115823","cluster_addr":"172.21.15.92:6813/115823","heartbeat_back_addr":"172.21.15.92:6821/115823","heartbeat_front_addr":"172.21.15.92:6817/115823","state":["exists","up"]},{"osd":1,"uuid":"13c1de73-1a8c-4d7e-8a28-9dae52dfe089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":115824},{"type":"v1","addr":"172.21.15.92:6812","nonce":115824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":115824},{"type":"v1","addr":"172.21.15.92:6816","nonce":115824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":115824},{"type":"v1","addr":"172.21.15.92:6823","nonce":115824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":115824},{"type":"v1","addr":"172.21.15.92:6820","nonce":115824}]},"public_addr":"172.21.15.92:6812/115824","cluster_addr":"172.21.15.92:6816/115824","heartbeat_back_addr":"172.21.15.92:6823/115824","heartbeat_front_addr":"172.21.15.92:6820/115824","state":["exists","up"]},{"osd":2,"uuid":"25788a55-112f-4f00-810d-2bc37cf76778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":115821},{"type":"v1","addr":"172.21.15.92:6801","nonce":115821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":115821},{"type":"v1","addr":"172.21.15.92:6803","nonce":115821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":115821},{"type":"v1","addr":"172.21.15.92:6807","nonce":115821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":115821},{"type":"v1","addr":"172.21.15.92:6805","nonce":115821}]},"public_addr":"172.21.15.92:6801/115821","cluster_addr":"172.21.15.92:6803/115821","heartbeat_back_addr":"172.21.15.92:6807/115821","heartbeat_front_addr":"172.21.15.92:6805/115821","state":["exists","up"]},{"osd":3,"uuid":"adf54e24-33a9-41fe-9ce3-98a5ab6c158e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":115825},{"type":"v1","addr":"172.21.15.92:6825","nonce":115825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":115825},{"type":"v1","addr":"172.21.15.92:6827","nonce":115825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":115825},{"type":"v1","addr":"172.21.15.92:6831","nonce":115825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":115825},{"type":"v1","addr":"172.21.15.92:6829","nonce":115825}]},"public_addr":"172.21.15.92:6825/115825","cluster_addr":"172.21.15.92:6827/115825","heartbeat_back_addr":"172.21.15.92:6831/115825","heartbeat_front_addr":"172.21.15.92:6829/115825","state":["exists","up"]},{"osd":4,"uuid":"ccb9b747-f399-44ea-9a18-9db3152227e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":116035},{"type":"v1","addr":"172.21.15.169:6817","nonce":116035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":116035},{"type":"v1","addr":"172.21.15.169:6819","nonce":116035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":116035},{"type":"v1","addr":"172.21.15.169:6826","nonce":116035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":116035},{"type":"v1","addr":"172.21.15.169:6822","nonce":116035}]},"public_addr":"172.21.15.169:6817/116035","cluster_addr":"172.21.15.169:6819/116035","heartbeat_back_addr":"172.21.15.169:6826/116035","heartbeat_front_addr":"172.21.15.169:6822/116035","state":["exists","up"]},{"osd":5,"uuid":"308a1e63-75aa-4d7d-8a7c-cecc24714fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":116033},{"type":"v1","addr":"172.21.15.169:6801","nonce":116033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":116033},{"type":"v1","addr":"172.21.15.169:6803","nonce":116033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":116033},{"type":"v1","addr":"172.21.15.169:6807","nonce":116033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":116033},{"type":"v1","addr":"172.21.15.169:6805","nonce":116033}]},"public_addr":"172.21.15.169:6801/116033","cluster_addr":"172.21.15.169:6803/116033","heartbeat_back_addr":"172.21.15.169:6807/116033","heartbeat_front_addr":"172.21.15.169:6805/116033","state":["exists","up"]},{"osd":6,"uuid":"9c459b7b-b821-4b4b-b91e-6aaae271457c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":116034},{"type":"v1","addr":"172.21.15.169:6809","nonce":116034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":116034},{"type":"v1","addr":"172.21.15.169:6811","nonce":116034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":116034},{"type":"v1","addr":"172.21.15.169:6815","nonce":116034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":116034},{"type":"v1","addr":"172.21.15.169:6813","nonce":116034}]},"public_addr":"172.21.15.169:6809/116034","cluster_addr":"172.21.15.169:6811/116034","heartbeat_back_addr":"172.21.15.169:6815/116034","heartbeat_front_addr":"172.21.15.169:6813/116034","state":["exists","up"]},{"osd":7,"uuid":"1ca9aa4a-755d-4e28-a937-4d8dccab4df3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":116044},{"type":"v1","addr":"172.21.15.169:6823","nonce":116044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6825","nonce":116044},{"type":"v1","addr":"172.21.15.169:6827","nonce":116044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":116044},{"type":"v1","addr":"172.21.15.169:6831","nonce":116044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":116044},{"type":"v1","addr":"172.21.15.169:6829","nonce":116044}]},"public_addr":"172.21.15.169:6823/116044","cluster_addr":"172.21.15.169:6827/116044","heartbeat_back_addr":"172.21.15.169:6831/116044","heartbeat_front_addr":"172.21.15.169:6829/116044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.586781+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.869778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.508594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.860493+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.872267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.746933+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.611520+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.871446+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:23:13.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.705+0000 7fc5f3bff700 1 -- 172.21.15.92:0/1845845316 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc5d00682f0 msgr2=0x7fc5d006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:13.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.705+0000 7fc5f3bff700 1 --2- 172.21.15.92:0/1845845316 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc5d00682f0 0x7fc5d006a7a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8007fd0 tx=0x7fc5e8009040 comp rx=0 tx=0).stop 2023-11-28T11:23:13.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.705+0000 7fc5f3bff700 1 -- 172.21.15.92:0/1845845316 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5ec0d0d60 msgr2=0x7fc5ec14de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:13.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.705+0000 7fc5f3bff700 1 --2- 172.21.15.92:0/1845845316 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5ec0d0d60 0x7fc5ec14de50 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4007e60 tx=0x7fc5e400e3c0 comp rx=0 tx=0).stop 2023-11-28T11:23:13.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.705+0000 7fc5f3bff700 1 -- 172.21.15.92:0/1845845316 shutdown_connections 2023-11-28T11:23:13.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.705+0000 7fc5f3bff700 1 --2- 172.21.15.92:0/1845845316 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc5d00682f0 0x7fc5d006a7a0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:13.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.705+0000 7fc5f3bff700 1 --2- 172.21.15.92:0/1845845316 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5ec0d0d60 0x7fc5ec14de50 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:13.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.705+0000 7fc5f3bff700 1 --2- 172.21.15.92:0/1845845316 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5ec0b8510 0x7fc5ec1e1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:13.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.705+0000 7fc5f3bff700 1 --2- 172.21.15.92:0/1845845316 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc5ec0cd460 0x7fc5ec1e4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:13.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.705+0000 7fc5f3bff700 1 -- 172.21.15.92:0/1845845316 >> 172.21.15.92:0/1845845316 conn(0x7fc5ec0719a0 msgr2=0x7fc5ec0b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:13.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.705+0000 7fc5f3bff700 1 -- 172.21.15.92:0/1845845316 shutdown_connections 2023-11-28T11:23:13.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.705+0000 7fc5f3bff700 1 -- 172.21.15.92:0/1845845316 wait complete. 2023-11-28T11:23:13.725 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:23:13.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.923+0000 7f86e77ff700 1 Processor -- start 2023-11-28T11:23:13.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.923+0000 7f86e77ff700 1 -- start start 2023-11-28T11:23:13.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.924+0000 7f86e77ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86e00cd460 0x7f86e006c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:13.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.924+0000 7f86e77ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f86e00d0d60 0x7f86e00d11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:13.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.924+0000 7f86e77ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86e00b88e0 con 0x7f86e00b8510 2023-11-28T11:23:13.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.924+0000 7f86e77ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86e00cdbd0 con 0x7f86e00d0d60 2023-11-28T11:23:13.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.925+0000 7f86e77ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86e00d02d0 con 0x7f86e00cd460 2023-11-28T11:23:13.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.925+0000 7f86e4d9a700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86e00cd460 0x7f86e006c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:13.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.925+0000 7f86e5d9c700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f86e00d0d60 0x7f86e00d11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:13.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.925+0000 7f86e4d9a700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86e00cd460 0x7f86e006c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.92:47424/0 (socket says 172.21.15.92:47424) 2023-11-28T11:23:13.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.925+0000 7f86e4d9a700 1 -- 172.21.15.92:0/3316199744 learned_addr learned my addr 172.21.15.92:0/3316199744 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:13.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.925+0000 7f86e559b700 1 --1- 172.21.15.92:0/3316199744 >> v1:172.21.15.169:6789/0 conn(0x7f86e00b8510 0x7f86e006b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.92:42876/0 (socket says 172.21.15.92:42876) 2023-11-28T11:23:13.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.925+0000 7f86e4d9a700 1 -- 172.21.15.92:0/3316199744 >> v2:172.21.15.169:3300/0 conn(0x7f86e00d0d60 msgr2=0x7f86e00d11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:13.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.925+0000 7f86e4d9a700 1 --2- 172.21.15.92:0/3316199744 >> v2:172.21.15.169:3300/0 conn(0x7f86e00d0d60 0x7f86e00d11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:13.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.925+0000 7f86e4d9a700 1 -- 172.21.15.92:0/3316199744 >> v1:172.21.15.169:6789/0 conn(0x7f86e00b8510 legacy=0x7f86e006b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:13.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.925+0000 7f86e4d9a700 1 -- 172.21.15.92:0/3316199744 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e00cd8f0 con 0x7f86e00cd460 2023-11-28T11:23:13.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.926+0000 7f86e4d9a700 1 --2- 172.21.15.92:0/3316199744 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86e00cd460 0x7f86e006c080 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f86d00098f0 tx=0x7f86d0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcfcb6ccf29a8d4d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:13.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.926+0000 7f86d7fff700 1 -- 172.21.15.92:0/3316199744 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d0011830 con 0x7f86e00cd460 2023-11-28T11:23:13.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.926+0000 7f86d7fff700 1 -- 172.21.15.92:0/3316199744 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86d0011990 con 0x7f86e00cd460 2023-11-28T11:23:13.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.926+0000 7f86d7fff700 1 -- 172.21.15.92:0/3316199744 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d0011d20 con 0x7f86e00cd460 2023-11-28T11:23:13.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.927+0000 7f86e77ff700 1 -- 172.21.15.92:0/3316199744 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86e00cd460 msgr2=0x7f86e006c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:13.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.927+0000 7f86e77ff700 1 --2- 172.21.15.92:0/3316199744 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86e00cd460 0x7f86e006c080 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f86d00098f0 tx=0x7f86d0009c00 comp rx=0 tx=0).stop 2023-11-28T11:23:13.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.927+0000 7f86e77ff700 1 -- 172.21.15.92:0/3316199744 shutdown_connections 2023-11-28T11:23:13.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.927+0000 7f86e77ff700 1 --2- 172.21.15.92:0/3316199744 >> v2:172.21.15.169:3300/0 conn(0x7f86e00d0d60 0x7f86e00d11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:13.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.927+0000 7f86e77ff700 1 --2- 172.21.15.92:0/3316199744 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86e00cd460 0x7f86e006c080 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:13.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.927+0000 7f86e77ff700 1 -- 172.21.15.92:0/3316199744 >> 172.21.15.92:0/3316199744 conn(0x7f86e00719a0 msgr2=0x7f86e0071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:13.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.927+0000 7f86e77ff700 1 -- 172.21.15.92:0/3316199744 shutdown_connections 2023-11-28T11:23:13.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.927+0000 7f86e77ff700 1 -- 172.21.15.92:0/3316199744 wait complete. 2023-11-28T11:23:13.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.929+0000 7f86e77ff700 1 Processor -- start 2023-11-28T11:23:13.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.929+0000 7f86e77ff700 1 -- start start 2023-11-28T11:23:13.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.930+0000 7f86e77ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86e00b8510 0x7f86e0156830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:13.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.930+0000 7f86e77ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86e00cd460 0x7f86e00c8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:13.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.930+0000 7f86e77ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86e00d0d60 0x7f86e00c8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:13.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.930+0000 7f86e77ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e01f2c10 con 0x7f86e00b8510 2023-11-28T11:23:13.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.930+0000 7f86e77ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e015a060 con 0x7f86e00d0d60 2023-11-28T11:23:13.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.930+0000 7f86e77ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86e0156d70 con 0x7f86e00cd460 2023-11-28T11:23:13.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.930+0000 7f86e559b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86e00b8510 0x7f86e0156830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:13.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.930+0000 7f86e559b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86e00b8510 0x7f86e0156830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40238/0 (socket says 172.21.15.92:40238) 2023-11-28T11:23:13.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.930+0000 7f86e559b700 1 -- 172.21.15.92:0/1635074164 learned_addr learned my addr 172.21.15.92:0/1635074164 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:13.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.930+0000 7f86e4d9a700 1 --2- 172.21.15.92:0/1635074164 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86e00cd460 0x7f86e00c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:13.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.930+0000 7f86e5d9c700 1 --2- 172.21.15.92:0/1635074164 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86e00d0d60 0x7f86e00c8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:13.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.930+0000 7f86e559b700 1 -- 172.21.15.92:0/1635074164 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86e00cd460 msgr2=0x7f86e00c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:13.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.930+0000 7f86e559b700 1 --2- 172.21.15.92:0/1635074164 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86e00cd460 0x7f86e00c8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:13.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.930+0000 7f86e559b700 1 -- 172.21.15.92:0/1635074164 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86e00d0d60 msgr2=0x7f86e00c8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:13.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.930+0000 7f86e559b700 1 --2- 172.21.15.92:0/1635074164 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86e00d0d60 0x7f86e00c8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:13.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.930+0000 7f86e559b700 1 -- 172.21.15.92:0/1635074164 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86d0009550 con 0x7f86e00b8510 2023-11-28T11:23:13.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.931+0000 7f86e559b700 1 --2- 172.21.15.92:0/1635074164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86e00b8510 0x7f86e0156830 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f86dc00c550 tx=0x7f86dc00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:13.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.931+0000 7f86d5ffb700 1 -- 172.21.15.92:0/1635074164 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86dc005cb0 con 0x7f86e00b8510 2023-11-28T11:23:13.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.931+0000 7f86d5ffb700 1 -- 172.21.15.92:0/1635074164 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86dc005e10 con 0x7f86e00b8510 2023-11-28T11:23:13.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.931+0000 7f86d5ffb700 1 -- 172.21.15.92:0/1635074164 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86dc0072d0 con 0x7f86e00b8510 2023-11-28T11:23:13.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.931+0000 7f86e77ff700 1 -- 172.21.15.92:0/1635074164 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86e00cfff0 con 0x7f86e00b8510 2023-11-28T11:23:13.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.931+0000 7f86e77ff700 1 -- 172.21.15.92:0/1635074164 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86e00cfd10 con 0x7f86e00b8510 2023-11-28T11:23:13.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.932+0000 7f86e77ff700 1 -- 172.21.15.92:0/1635074164 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86e0067880 con 0x7f86e00b8510 2023-11-28T11:23:13.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.933+0000 7f86d5ffb700 1 -- 172.21.15.92:0/1635074164 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f86dc019030 con 0x7f86e00b8510 2023-11-28T11:23:13.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.934+0000 7f86d5ffb700 1 --2- 172.21.15.92:0/1635074164 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f86c40682f0 0x7f86c406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:13.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.934+0000 7f86d5ffb700 1 -- 172.21.15.92:0/1635074164 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f86dc084fe0 con 0x7f86e00b8510 2023-11-28T11:23:13.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.934+0000 7f86e4d9a700 1 --2- 172.21.15.92:0/1635074164 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f86c40682f0 0x7f86c406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:13.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.934+0000 7f86e4d9a700 1 --2- 172.21.15.92:0/1635074164 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f86c40682f0 0x7f86c406a7a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f86d0000c00 tx=0x7f86d0008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:13.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:13.938+0000 7f86d5ffb700 1 -- 172.21.15.92:0/1635074164 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f86dc04b5e0 con 0x7f86e00b8510 2023-11-28T11:23:13.942 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2023-11-28T11:23:13.942 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2023-11-28T11:23:13.942 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2023-11-28T11:23:14.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.130+0000 7f86e77ff700 1 -- 172.21.15.92:0/1635074164 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f86e00d2940 con 0x7f86e00b8510 2023-11-28T11:23:14.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.131+0000 7f86d5ffb700 1 -- 172.21.15.92:0/1635074164 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+18193 (secure 0 0 0) 0x7f86dc017020 con 0x7f86e00b8510 2023-11-28T11:23:14.133 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:23:14.134 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":55,"fsid":"68956e0c-c16f-47cd-a220-f98364b0831d","created":"2023-11-28T11:20:28.697573+0000","modified":"2023-11-28T11:23:13.246802+0000","last_up_change":"2023-11-28T11:20:41.257011+0000","last_in_change":"2023-11-28T11:20:32.249767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:20:41.925237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:20:48.682497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:23:10.939884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-28T11:23:12.227433+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6abe483f-0e03-4a26-b9cf-62041291747e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":115823},{"type":"v1","addr":"172.21.15.92:6809","nonce":115823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":115823},{"type":"v1","addr":"172.21.15.92:6813","nonce":115823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":115823},{"type":"v1","addr":"172.21.15.92:6821","nonce":115823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":115823},{"type":"v1","addr":"172.21.15.92:6817","nonce":115823}]},"public_addr":"172.21.15.92:6809/115823","cluster_addr":"172.21.15.92:6813/115823","heartbeat_back_addr":"172.21.15.92:6821/115823","heartbeat_front_addr":"172.21.15.92:6817/115823","state":["exists","up"]},{"osd":1,"uuid":"13c1de73-1a8c-4d7e-8a28-9dae52dfe089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":115824},{"type":"v1","addr":"172.21.15.92:6812","nonce":115824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":115824},{"type":"v1","addr":"172.21.15.92:6816","nonce":115824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":115824},{"type":"v1","addr":"172.21.15.92:6823","nonce":115824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":115824},{"type":"v1","addr":"172.21.15.92:6820","nonce":115824}]},"public_addr":"172.21.15.92:6812/115824","cluster_addr":"172.21.15.92:6816/115824","heartbeat_back_addr":"172.21.15.92:6823/115824","heartbeat_front_addr":"172.21.15.92:6820/115824","state":["exists","up"]},{"osd":2,"uuid":"25788a55-112f-4f00-810d-2bc37cf76778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":115821},{"type":"v1","addr":"172.21.15.92:6801","nonce":115821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":115821},{"type":"v1","addr":"172.21.15.92:6803","nonce":115821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":115821},{"type":"v1","addr":"172.21.15.92:6807","nonce":115821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":115821},{"type":"v1","addr":"172.21.15.92:6805","nonce":115821}]},"public_addr":"172.21.15.92:6801/115821","cluster_addr":"172.21.15.92:6803/115821","heartbeat_back_addr":"172.21.15.92:6807/115821","heartbeat_front_addr":"172.21.15.92:6805/115821","state":["exists","up"]},{"osd":3,"uuid":"adf54e24-33a9-41fe-9ce3-98a5ab6c158e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":115825},{"type":"v1","addr":"172.21.15.92:6825","nonce":115825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":115825},{"type":"v1","addr":"172.21.15.92:6827","nonce":115825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":115825},{"type":"v1","addr":"172.21.15.92:6831","nonce":115825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":115825},{"type":"v1","addr":"172.21.15.92:6829","nonce":115825}]},"public_addr":"172.21.15.92:6825/115825","cluster_addr":"172.21.15.92:6827/115825","heartbeat_back_addr":"172.21.15.92:6831/115825","heartbeat_front_addr":"172.21.15.92:6829/115825","state":["exists","up"]},{"osd":4,"uuid":"ccb9b747-f399-44ea-9a18-9db3152227e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":116035},{"type":"v1","addr":"172.21.15.169:6817","nonce":116035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":116035},{"type":"v1","addr":"172.21.15.169:6819","nonce":116035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":116035},{"type":"v1","addr":"172.21.15.169:6826","nonce":116035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":116035},{"type":"v1","addr":"172.21.15.169:6822","nonce":116035}]},"public_addr":"172.21.15.169:6817/116035","cluster_addr":"172.21.15.169:6819/116035","heartbeat_back_addr":"172.21.15.169:6826/116035","heartbeat_front_addr":"172.21.15.169:6822/116035","state":["exists","up"]},{"osd":5,"uuid":"308a1e63-75aa-4d7d-8a7c-cecc24714fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":116033},{"type":"v1","addr":"172.21.15.169:6801","nonce":116033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":116033},{"type":"v1","addr":"172.21.15.169:6803","nonce":116033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":116033},{"type":"v1","addr":"172.21.15.169:6807","nonce":116033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":116033},{"type":"v1","addr":"172.21.15.169:6805","nonce":116033}]},"public_addr":"172.21.15.169:6801/116033","cluster_addr":"172.21.15.169:6803/116033","heartbeat_back_addr":"172.21.15.169:6807/116033","heartbeat_front_addr":"172.21.15.169:6805/116033","state":["exists","up"]},{"osd":6,"uuid":"9c459b7b-b821-4b4b-b91e-6aaae271457c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":116034},{"type":"v1","addr":"172.21.15.169:6809","nonce":116034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":116034},{"type":"v1","addr":"172.21.15.169:6811","nonce":116034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":116034},{"type":"v1","addr":"172.21.15.169:6815","nonce":116034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":116034},{"type":"v1","addr":"172.21.15.169:6813","nonce":116034}]},"public_addr":"172.21.15.169:6809/116034","cluster_addr":"172.21.15.169:6811/116034","heartbeat_back_addr":"172.21.15.169:6815/116034","heartbeat_front_addr":"172.21.15.169:6813/116034","state":["exists","up"]},{"osd":7,"uuid":"1ca9aa4a-755d-4e28-a937-4d8dccab4df3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":116044},{"type":"v1","addr":"172.21.15.169:6823","nonce":116044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6825","nonce":116044},{"type":"v1","addr":"172.21.15.169:6827","nonce":116044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":116044},{"type":"v1","addr":"172.21.15.169:6831","nonce":116044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":116044},{"type":"v1","addr":"172.21.15.169:6829","nonce":116044}]},"public_addr":"172.21.15.169:6823/116044","cluster_addr":"172.21.15.169:6827/116044","heartbeat_back_addr":"172.21.15.169:6831/116044","heartbeat_front_addr":"172.21.15.169:6829/116044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.586781+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.869778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.508594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.860493+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.872267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.746933+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.611520+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.871446+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:23:14.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.133+0000 7f86e77ff700 1 -- 172.21.15.92:0/1635074164 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f86c40682f0 msgr2=0x7f86c406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:14.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.133+0000 7f86e77ff700 1 --2- 172.21.15.92:0/1635074164 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f86c40682f0 0x7f86c406a7a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f86d0000c00 tx=0x7f86d0008030 comp rx=0 tx=0).stop 2023-11-28T11:23:14.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.133+0000 7f86e77ff700 1 -- 172.21.15.92:0/1635074164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86e00b8510 msgr2=0x7f86e0156830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:14.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.133+0000 7f86e77ff700 1 --2- 172.21.15.92:0/1635074164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86e00b8510 0x7f86e0156830 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f86dc00c550 tx=0x7f86dc00c860 comp rx=0 tx=0).stop 2023-11-28T11:23:14.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.133+0000 7f86e77ff700 1 -- 172.21.15.92:0/1635074164 shutdown_connections 2023-11-28T11:23:14.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.133+0000 7f86e77ff700 1 --2- 172.21.15.92:0/1635074164 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86e00d0d60 0x7f86e00c8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:14.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.133+0000 7f86e77ff700 1 --2- 172.21.15.92:0/1635074164 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f86c40682f0 0x7f86c406a7a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:14.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.133+0000 7f86e77ff700 1 --2- 172.21.15.92:0/1635074164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86e00b8510 0x7f86e0156830 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:14.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.133+0000 7f86e77ff700 1 --2- 172.21.15.92:0/1635074164 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86e00cd460 0x7f86e00c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:14.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.133+0000 7f86e77ff700 1 -- 172.21.15.92:0/1635074164 >> 172.21.15.92:0/1635074164 conn(0x7f86e00719a0 msgr2=0x7f86e00b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:14.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.134+0000 7f86e77ff700 1 -- 172.21.15.92:0/1635074164 shutdown_connections 2023-11-28T11:23:14.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.134+0000 7f86e77ff700 1 -- 172.21.15.92:0/1635074164 wait complete. 2023-11-28T11:23:14.153 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T11:23:14.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.352+0000 7fc1c79ff700 1 Processor -- start 2023-11-28T11:23:14.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.352+0000 7fc1c79ff700 1 -- start start 2023-11-28T11:23:14.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.353+0000 7fc1c79ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1c00b8510 0x7fc1c006b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:14.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.353+0000 7fc1c79ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc1c00d0d60 0x7fc1c00d11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:14.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.354+0000 7fc1c79ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1c00b88e0 con 0x7fc1c00cd460 2023-11-28T11:23:14.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.354+0000 7fc1c79ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1c00cdbd0 con 0x7fc1c00d0d60 2023-11-28T11:23:14.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.354+0000 7fc1c79ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c00d02d0 con 0x7fc1c00b8510 2023-11-28T11:23:14.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.354+0000 7fc1c4f9a700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc1c00cd460 0x7fc1c006beb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36940/0 (socket says 172.21.15.92:36940) 2023-11-28T11:23:14.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.354+0000 7fc1c4f9a700 1 -- 172.21.15.92:0/61857846 learned_addr learned my addr 172.21.15.92:0/61857846 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:23:14.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.354+0000 7fc1c579b700 1 --2- 172.21.15.92:0/61857846 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1c00b8510 0x7fc1c006b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:14.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.354+0000 7fc1c5f9c700 1 --2- 172.21.15.92:0/61857846 >> v2:172.21.15.169:3300/0 conn(0x7fc1c00d0d60 0x7fc1c00d11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:14.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.354+0000 7fc1b7fff700 1 -- 172.21.15.92:0/61857846 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3793433793 0 0) 0x7fc1c00b88e0 con 0x7fc1c00cd460 2023-11-28T11:23:14.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.354+0000 7fc1b7fff700 1 -- 172.21.15.92:0/61857846 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1ac0035b0 con 0x7fc1c00cd460 2023-11-28T11:23:14.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.354+0000 7fc1c579b700 1 -- 172.21.15.92:0/61857846 >> v2:172.21.15.169:3300/0 conn(0x7fc1c00d0d60 msgr2=0x7fc1c00d11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:14.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.354+0000 7fc1c579b700 1 --2- 172.21.15.92:0/61857846 >> v2:172.21.15.169:3300/0 conn(0x7fc1c00d0d60 0x7fc1c00d11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:14.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.354+0000 7fc1c579b700 1 -- 172.21.15.92:0/61857846 >> v1:172.21.15.92:6789/0 conn(0x7fc1c00cd460 legacy=0x7fc1c006beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:14.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.354+0000 7fc1c579b700 1 -- 172.21.15.92:0/61857846 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1c00cd8f0 con 0x7fc1c00b8510 2023-11-28T11:23:14.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.354+0000 7fc1c5f9c700 1 --2- 172.21.15.92:0/61857846 >> v2:172.21.15.169:3300/0 conn(0x7fc1c00d0d60 0x7fc1c00d11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:23:14.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.355+0000 7fc1c579b700 1 --2- 172.21.15.92:0/61857846 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1c00b8510 0x7fc1c006b970 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc009840 tx=0x7fc1bc009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18257519a5c081f0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:14.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.355+0000 7fc1b7fff700 1 -- 172.21.15.92:0/61857846 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc011830 con 0x7fc1c00b8510 2023-11-28T11:23:14.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.355+0000 7fc1b7fff700 1 -- 172.21.15.92:0/61857846 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1bc011990 con 0x7fc1c00b8510 2023-11-28T11:23:14.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.355+0000 7fc1b7fff700 1 -- 172.21.15.92:0/61857846 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc011d20 con 0x7fc1c00b8510 2023-11-28T11:23:14.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.356+0000 7fc1c79ff700 1 -- 172.21.15.92:0/61857846 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1c00b8510 msgr2=0x7fc1c006b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:14.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.356+0000 7fc1c79ff700 1 --2- 172.21.15.92:0/61857846 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1c00b8510 0x7fc1c006b970 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc009840 tx=0x7fc1bc009b50 comp rx=0 tx=0).stop 2023-11-28T11:23:14.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.356+0000 7fc1c79ff700 1 -- 172.21.15.92:0/61857846 shutdown_connections 2023-11-28T11:23:14.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.356+0000 7fc1c79ff700 1 --2- 172.21.15.92:0/61857846 >> v2:172.21.15.169:3300/0 conn(0x7fc1c00d0d60 0x7fc1c00d11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:14.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.356+0000 7fc1c79ff700 1 --2- 172.21.15.92:0/61857846 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1c00b8510 0x7fc1c006b970 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:14.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.356+0000 7fc1c79ff700 1 -- 172.21.15.92:0/61857846 >> 172.21.15.92:0/61857846 conn(0x7fc1c00719a0 msgr2=0x7fc1c0071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:14.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.356+0000 7fc1c79ff700 1 -- 172.21.15.92:0/61857846 shutdown_connections 2023-11-28T11:23:14.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.356+0000 7fc1c79ff700 1 -- 172.21.15.92:0/61857846 wait complete. 2023-11-28T11:23:14.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.357+0000 7fc1c79ff700 1 Processor -- start 2023-11-28T11:23:14.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.358+0000 7fc1c79ff700 1 -- start start 2023-11-28T11:23:14.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.358+0000 7fc1c79ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1c00b8510 0x7fc1c01f6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:14.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.358+0000 7fc1c79ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1c00cd460 0x7fc1c00ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:14.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.358+0000 7fc1c79ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1c00d0d60 0x7fc1c015a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:14.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.359+0000 7fc1c79ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c01591a0 con 0x7fc1c00b8510 2023-11-28T11:23:14.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.359+0000 7fc1c79ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c015ab90 con 0x7fc1c00cd460 2023-11-28T11:23:14.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.359+0000 7fc1c79ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c00cd030 con 0x7fc1c00d0d60 2023-11-28T11:23:14.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.359+0000 7fc1c579b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1c00b8510 0x7fc1c01f6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:14.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.359+0000 7fc1c579b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1c00b8510 0x7fc1c01f6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40250/0 (socket says 172.21.15.92:40250) 2023-11-28T11:23:14.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.359+0000 7fc1c579b700 1 -- 172.21.15.92:0/168891512 learned_addr learned my addr 172.21.15.92:0/168891512 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:14.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.359+0000 7fc1c5f9c700 1 --2- 172.21.15.92:0/168891512 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1c00d0d60 0x7fc1c015a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:14.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.359+0000 7fc1c4f9a700 1 --2- 172.21.15.92:0/168891512 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1c00cd460 0x7fc1c00ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:14.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.359+0000 7fc1c579b700 1 -- 172.21.15.92:0/168891512 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1c00d0d60 msgr2=0x7fc1c015a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:14.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.359+0000 7fc1c579b700 1 --2- 172.21.15.92:0/168891512 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1c00d0d60 0x7fc1c015a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:14.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.359+0000 7fc1c579b700 1 -- 172.21.15.92:0/168891512 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1c00cd460 msgr2=0x7fc1c00ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:14.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.359+0000 7fc1c579b700 1 --2- 172.21.15.92:0/168891512 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1c00cd460 0x7fc1c00ccaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:14.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.359+0000 7fc1c579b700 1 -- 172.21.15.92:0/168891512 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b8009a70 con 0x7fc1c00b8510 2023-11-28T11:23:14.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.359+0000 7fc1c579b700 1 --2- 172.21.15.92:0/168891512 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1c00b8510 0x7fc1c01f6420 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc0090b0 tx=0x7fc1bc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:14.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.360+0000 7fc1b5ffb700 1 -- 172.21.15.92:0/168891512 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc0044d0 con 0x7fc1c00b8510 2023-11-28T11:23:14.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.360+0000 7fc1b5ffb700 1 -- 172.21.15.92:0/168891512 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1bc01f070 con 0x7fc1c00b8510 2023-11-28T11:23:14.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.360+0000 7fc1b5ffb700 1 -- 172.21.15.92:0/168891512 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc0038c0 con 0x7fc1c00b8510 2023-11-28T11:23:14.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.360+0000 7fc1c79ff700 1 -- 172.21.15.92:0/168891512 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1bc0094a0 con 0x7fc1c00b8510 2023-11-28T11:23:14.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.360+0000 7fc1c79ff700 1 -- 172.21.15.92:0/168891512 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1c00cfff0 con 0x7fc1c00b8510 2023-11-28T11:23:14.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.361+0000 7fc1c79ff700 1 -- 172.21.15.92:0/168891512 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1c00cfd10 con 0x7fc1c00b8510 2023-11-28T11:23:14.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.362+0000 7fc1b5ffb700 1 -- 172.21.15.92:0/168891512 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc1bc00f050 con 0x7fc1c00b8510 2023-11-28T11:23:14.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.362+0000 7fc1b5ffb700 1 --2- 172.21.15.92:0/168891512 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc1a40682f0 0x7fc1a406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:14.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.363+0000 7fc1b5ffb700 1 -- 172.21.15.92:0/168891512 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fc1bc021070 con 0x7fc1c00b8510 2023-11-28T11:23:14.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.363+0000 7fc1c4f9a700 1 --2- 172.21.15.92:0/168891512 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc1a40682f0 0x7fc1a406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:14.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.363+0000 7fc1c4f9a700 1 --2- 172.21.15.92:0/168891512 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc1a40682f0 0x7fc1a406a7a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc1c00cc8b0 tx=0x7fc1b0007dc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:14.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.367+0000 7fc1b5ffb700 1 -- 172.21.15.92:0/168891512 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc1bc053c80 con 0x7fc1c00b8510 2023-11-28T11:23:14.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:14.602+0000 7fc1c79ff700 1 -- 172.21.15.92:0/168891512 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc1c00ceca0 con 0x7fc1c00b8510 2023-11-28T11:23:15.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.255+0000 7fc1b5ffb700 1 -- 172.21.15.92:0/168891512 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v23) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc1bc0572a0 con 0x7fc1c00b8510 2023-11-28T11:23:15.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.258+0000 7fc1c79ff700 1 -- 172.21.15.92:0/168891512 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc1a40682f0 msgr2=0x7fc1a406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:15.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.258+0000 7fc1c79ff700 1 --2- 172.21.15.92:0/168891512 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc1a40682f0 0x7fc1a406a7a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc1c00cc8b0 tx=0x7fc1b0007dc0 comp rx=0 tx=0).stop 2023-11-28T11:23:15.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.258+0000 7fc1c79ff700 1 -- 172.21.15.92:0/168891512 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1c00b8510 msgr2=0x7fc1c01f6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:15.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.258+0000 7fc1c79ff700 1 --2- 172.21.15.92:0/168891512 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1c00b8510 0x7fc1c01f6420 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc0090b0 tx=0x7fc1bc004750 comp rx=0 tx=0).stop 2023-11-28T11:23:15.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.258+0000 7fc1c79ff700 1 -- 172.21.15.92:0/168891512 shutdown_connections 2023-11-28T11:23:15.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.258+0000 7fc1c79ff700 1 --2- 172.21.15.92:0/168891512 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc1c00d0d60 0x7fc1c015a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:15.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.258+0000 7fc1c79ff700 1 --2- 172.21.15.92:0/168891512 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc1a40682f0 0x7fc1a406a7a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:15.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.258+0000 7fc1c79ff700 1 --2- 172.21.15.92:0/168891512 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1c00b8510 0x7fc1c01f6420 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:15.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.258+0000 7fc1c79ff700 1 --2- 172.21.15.92:0/168891512 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc1c00cd460 0x7fc1c00ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:15.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.258+0000 7fc1c79ff700 1 -- 172.21.15.92:0/168891512 >> 172.21.15.92:0/168891512 conn(0x7fc1c00719a0 msgr2=0x7fc1c00d1f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:15.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.258+0000 7fc1c79ff700 1 -- 172.21.15.92:0/168891512 shutdown_connections 2023-11-28T11:23:15.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.258+0000 7fc1c79ff700 1 -- 172.21.15.92:0/168891512 wait complete. 2023-11-28T11:23:15.278 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-28T11:23:15.279 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T11:23:15.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.478+0000 7f6017dff700 1 Processor -- start 2023-11-28T11:23:15.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.478+0000 7f6017dff700 1 -- start start 2023-11-28T11:23:15.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.479+0000 7f6017dff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f60100b8510 0x7f601006b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:15.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.480+0000 7f6017dff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f60100cd460 0x7f601006beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:15.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.480+0000 7f6017dff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60100b88e0 con 0x7f601006c3f0 2023-11-28T11:23:15.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.480+0000 7f6017dff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60100cdbd0 con 0x7f60100cd460 2023-11-28T11:23:15.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.480+0000 7f6017dff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60100d02d0 con 0x7f60100b8510 2023-11-28T11:23:15.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.480+0000 7f601539a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f60100cd460 0x7f601006beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:15.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.480+0000 7f601639c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f601006c3f0 0x7f601015cc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36948/0 (socket says 172.21.15.92:36948) 2023-11-28T11:23:15.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.480+0000 7f601539a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f60100cd460 0x7f601006beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40266/0 (socket says 172.21.15.92:40266) 2023-11-28T11:23:15.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.480+0000 7f601639c700 1 -- 172.21.15.92:0/718976425 learned_addr learned my addr 172.21.15.92:0/718976425 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:23:15.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.480+0000 7f6015b9b700 1 --2- 172.21.15.92:0/718976425 >> v2:172.21.15.169:3300/0 conn(0x7f60100b8510 0x7f601006b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:15.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.480+0000 7f601539a700 1 -- 172.21.15.92:0/718976425 >> v2:172.21.15.169:3300/0 conn(0x7f60100b8510 msgr2=0x7f601006b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:15.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.480+0000 7f601539a700 1 --2- 172.21.15.92:0/718976425 >> v2:172.21.15.169:3300/0 conn(0x7f60100b8510 0x7f601006b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:15.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.480+0000 7f601539a700 1 -- 172.21.15.92:0/718976425 >> v1:172.21.15.92:6789/0 conn(0x7f601006c3f0 legacy=0x7f601015cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:15.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.480+0000 7f601539a700 1 -- 172.21.15.92:0/718976425 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60100cd8f0 con 0x7f60100cd460 2023-11-28T11:23:15.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.481+0000 7f601539a700 1 --2- 172.21.15.92:0/718976425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60100cd460 0x7f601006beb0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f6000009870 tx=0x7f6000009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=820962a864371c6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:15.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.481+0000 7f6014b99700 1 -- 172.21.15.92:0/718976425 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6000011830 con 0x7f60100cd460 2023-11-28T11:23:15.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.481+0000 7f6014b99700 1 -- 172.21.15.92:0/718976425 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6000011990 con 0x7f60100cd460 2023-11-28T11:23:15.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.481+0000 7f6014b99700 1 -- 172.21.15.92:0/718976425 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6000011d20 con 0x7f60100cd460 2023-11-28T11:23:15.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.481+0000 7f6017dff700 1 -- 172.21.15.92:0/718976425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60100cd460 msgr2=0x7f601006beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:15.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.481+0000 7f6017dff700 1 --2- 172.21.15.92:0/718976425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60100cd460 0x7f601006beb0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f6000009870 tx=0x7f6000009b80 comp rx=0 tx=0).stop 2023-11-28T11:23:15.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.481+0000 7f6017dff700 1 -- 172.21.15.92:0/718976425 shutdown_connections 2023-11-28T11:23:15.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.481+0000 7f6017dff700 1 --2- 172.21.15.92:0/718976425 >> v2:172.21.15.169:3300/0 conn(0x7f60100b8510 0x7f601006b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:15.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.481+0000 7f6017dff700 1 --2- 172.21.15.92:0/718976425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60100cd460 0x7f601006beb0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:15.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.481+0000 7f6017dff700 1 -- 172.21.15.92:0/718976425 >> 172.21.15.92:0/718976425 conn(0x7f60100719a0 msgr2=0x7f6010071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:15.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.482+0000 7f6017dff700 1 -- 172.21.15.92:0/718976425 shutdown_connections 2023-11-28T11:23:15.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.482+0000 7f6017dff700 1 -- 172.21.15.92:0/718976425 wait complete. 2023-11-28T11:23:15.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.483+0000 7f6017dff700 1 Processor -- start 2023-11-28T11:23:15.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.483+0000 7f6017dff700 1 -- start start 2023-11-28T11:23:15.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.484+0000 7f6017dff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f601006c3f0 0x7f60100b3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:15.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.484+0000 7f6017dff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60100b8510 0x7f6010157a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:15.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.484+0000 7f6017dff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60100cd460 0x7f6010143ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:15.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.484+0000 7f6017dff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6010142f50 con 0x7f60100b8510 2023-11-28T11:23:15.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.484+0000 7f6017dff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6010143230 con 0x7f601006c3f0 2023-11-28T11:23:15.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.484+0000 7f6017dff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6010143510 con 0x7f60100cd460 2023-11-28T11:23:15.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.484+0000 7f601539a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60100b8510 0x7f6010157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:15.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.485+0000 7f601539a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60100b8510 0x7f6010157a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40282/0 (socket says 172.21.15.92:40282) 2023-11-28T11:23:15.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.485+0000 7f601539a700 1 -- 172.21.15.92:0/693129650 learned_addr learned my addr 172.21.15.92:0/693129650 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:15.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.485+0000 7f6015b9b700 1 --2- 172.21.15.92:0/693129650 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f601006c3f0 0x7f60100b3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:15.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.485+0000 7f601639c700 1 --2- 172.21.15.92:0/693129650 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60100cd460 0x7f6010143ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:15.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.485+0000 7f601539a700 1 -- 172.21.15.92:0/693129650 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60100cd460 msgr2=0x7f6010143ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:15.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.485+0000 7f601539a700 1 --2- 172.21.15.92:0/693129650 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60100cd460 0x7f6010143ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:15.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.485+0000 7f601539a700 1 -- 172.21.15.92:0/693129650 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f601006c3f0 msgr2=0x7f60100b3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:15.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.485+0000 7f601539a700 1 --2- 172.21.15.92:0/693129650 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f601006c3f0 0x7f60100b3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:15.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.485+0000 7f601539a700 1 -- 172.21.15.92:0/693129650 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60000094a0 con 0x7f60100b8510 2023-11-28T11:23:15.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.485+0000 7f6015b9b700 1 --2- 172.21.15.92:0/693129650 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f601006c3f0 0x7f60100b3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:23:15.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.485+0000 7f601539a700 1 --2- 172.21.15.92:0/693129650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60100b8510 0x7f6010157a40 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f6000009fd0 tx=0x7f6000004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:15.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.485+0000 7f60067fc700 1 -- 172.21.15.92:0/693129650 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60000038a0 con 0x7f60100b8510 2023-11-28T11:23:15.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.485+0000 7f60067fc700 1 -- 172.21.15.92:0/693129650 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f600001e070 con 0x7f60100b8510 2023-11-28T11:23:15.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.485+0000 7f6017dff700 1 -- 172.21.15.92:0/693129650 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60100cfff0 con 0x7f60100b8510 2023-11-28T11:23:15.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.485+0000 7f60067fc700 1 -- 172.21.15.92:0/693129650 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6000003c60 con 0x7f60100b8510 2023-11-28T11:23:15.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.486+0000 7f6017dff700 1 -- 172.21.15.92:0/693129650 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60100cfd10 con 0x7f60100b8510 2023-11-28T11:23:15.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.486+0000 7f6017dff700 1 -- 172.21.15.92:0/693129650 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6010067880 con 0x7f60100b8510 2023-11-28T11:23:15.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.487+0000 7f60067fc700 1 -- 172.21.15.92:0/693129650 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f600000f050 con 0x7f60100b8510 2023-11-28T11:23:15.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.488+0000 7f60067fc700 1 --2- 172.21.15.92:0/693129650 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5ff40682f0 0x7f5ff406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:15.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.488+0000 7f60067fc700 1 -- 172.21.15.92:0/693129650 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f6000086ff0 con 0x7f60100b8510 2023-11-28T11:23:15.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.488+0000 7f6015b9b700 1 --2- 172.21.15.92:0/693129650 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5ff40682f0 0x7f5ff406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:15.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.489+0000 7f6015b9b700 1 --2- 172.21.15.92:0/693129650 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5ff40682f0 0x7f5ff406a7a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f600c00aa50 tx=0x7f600c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:15.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.493+0000 7f60067fc700 1 -- 172.21.15.92:0/693129650 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6000053680 con 0x7f60100b8510 2023-11-28T11:23:15.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:15.724+0000 7f6017dff700 1 -- 172.21.15.92:0/693129650 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f601015c9d0 con 0x7f60100b8510 2023-11-28T11:23:16.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.260+0000 7f60067fc700 1 -- 172.21.15.92:0/693129650 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7f60000150c0 con 0x7f60100b8510 2023-11-28T11:23:16.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.262+0000 7f6017dff700 1 -- 172.21.15.92:0/693129650 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5ff40682f0 msgr2=0x7f5ff406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:16.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.262+0000 7f6017dff700 1 --2- 172.21.15.92:0/693129650 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5ff40682f0 0x7f5ff406a7a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f600c00aa50 tx=0x7f600c008040 comp rx=0 tx=0).stop 2023-11-28T11:23:16.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.262+0000 7f6017dff700 1 -- 172.21.15.92:0/693129650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60100b8510 msgr2=0x7f6010157a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:16.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.262+0000 7f6017dff700 1 --2- 172.21.15.92:0/693129650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60100b8510 0x7f6010157a40 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f6000009fd0 tx=0x7f6000004750 comp rx=0 tx=0).stop 2023-11-28T11:23:16.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.262+0000 7f6017dff700 1 -- 172.21.15.92:0/693129650 shutdown_connections 2023-11-28T11:23:16.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.262+0000 7f6017dff700 1 --2- 172.21.15.92:0/693129650 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f60100cd460 0x7f6010143ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:16.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.262+0000 7f6017dff700 1 --2- 172.21.15.92:0/693129650 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f601006c3f0 0x7f60100b3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:16.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.262+0000 7f6017dff700 1 --2- 172.21.15.92:0/693129650 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5ff40682f0 0x7f5ff406a7a0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:16.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.262+0000 7f6017dff700 1 --2- 172.21.15.92:0/693129650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60100b8510 0x7f6010157a40 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:16.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.262+0000 7f6017dff700 1 -- 172.21.15.92:0/693129650 >> 172.21.15.92:0/693129650 conn(0x7f60100719a0 msgr2=0x7f60100b6930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:16.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.263+0000 7f6017dff700 1 -- 172.21.15.92:0/693129650 shutdown_connections 2023-11-28T11:23:16.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.263+0000 7f6017dff700 1 -- 172.21.15.92:0/693129650 wait complete. 2023-11-28T11:23:16.283 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T11:23:16.283 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:23:16.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.478+0000 7fe5f99ff700 1 Processor -- start 2023-11-28T11:23:16.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.478+0000 7fe5f99ff700 1 -- start start 2023-11-28T11:23:16.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.479+0000 7fe5f99ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5f40b8510 0x7fe5f406b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:16.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.480+0000 7fe5f99ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe5f40cd460 0x7fe5f406beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:16.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.480+0000 7fe5f99ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe5f406c3f0 0x7fe5f415cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:16.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.480+0000 7fe5f99ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5f40cdbd0 con 0x7fe5f406c3f0 2023-11-28T11:23:16.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.480+0000 7fe5f99ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5f406c830 con 0x7fe5f40cd460 2023-11-28T11:23:16.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.480+0000 7fe5f99ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f40d02d0 con 0x7fe5f40b8510 2023-11-28T11:23:16.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.480+0000 7fe5f37fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe5f406c3f0 0x7fe5f415cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:16.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.480+0000 7fe5f37fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe5f406c3f0 0x7fe5f415cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40298/0 (socket says 172.21.15.92:40298) 2023-11-28T11:23:16.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.480+0000 7fe5f37fe700 1 -- 172.21.15.92:0/796649762 learned_addr learned my addr 172.21.15.92:0/796649762 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:16.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.480+0000 7fe5f2ffd700 1 --2- 172.21.15.92:0/796649762 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5f40b8510 0x7fe5f406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:16.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.480+0000 7fe5ebfff700 1 --2- 172.21.15.92:0/796649762 >> v2:172.21.15.169:3300/0 conn(0x7fe5f40cd460 0x7fe5f406beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:16.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.480+0000 7fe5f37fe700 1 -- 172.21.15.92:0/796649762 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5f40b8510 msgr2=0x7fe5f406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:16.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.480+0000 7fe5f37fe700 1 --2- 172.21.15.92:0/796649762 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5f40b8510 0x7fe5f406b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:16.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.480+0000 7fe5f37fe700 1 -- 172.21.15.92:0/796649762 >> v2:172.21.15.169:3300/0 conn(0x7fe5f40cd460 msgr2=0x7fe5f406beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:16.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.480+0000 7fe5f37fe700 1 --2- 172.21.15.92:0/796649762 >> v2:172.21.15.169:3300/0 conn(0x7fe5f40cd460 0x7fe5f406beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:16.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.480+0000 7fe5f37fe700 1 -- 172.21.15.92:0/796649762 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f40b88e0 con 0x7fe5f406c3f0 2023-11-28T11:23:16.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.481+0000 7fe5f37fe700 1 --2- 172.21.15.92:0/796649762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5f406c3f0 0x7fe5f415cc60 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fe5f408ad80 tx=0x7fe5ec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=909920c9280cb4f7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:16.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.481+0000 7fe5f27fc700 1 -- 172.21.15.92:0/796649762 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec00e070 con 0x7fe5f406c3f0 2023-11-28T11:23:16.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.481+0000 7fe5f27fc700 1 -- 172.21.15.92:0/796649762 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5ec00b940 con 0x7fe5f406c3f0 2023-11-28T11:23:16.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.481+0000 7fe5f27fc700 1 -- 172.21.15.92:0/796649762 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec00bcf0 con 0x7fe5f406c3f0 2023-11-28T11:23:16.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.481+0000 7fe5f99ff700 1 -- 172.21.15.92:0/796649762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5f406c3f0 msgr2=0x7fe5f415cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:16.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.481+0000 7fe5f99ff700 1 --2- 172.21.15.92:0/796649762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5f406c3f0 0x7fe5f415cc60 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fe5f408ad80 tx=0x7fe5ec00b330 comp rx=0 tx=0).stop 2023-11-28T11:23:16.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.481+0000 7fe5f99ff700 1 -- 172.21.15.92:0/796649762 shutdown_connections 2023-11-28T11:23:16.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.481+0000 7fe5f99ff700 1 --2- 172.21.15.92:0/796649762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5f406c3f0 0x7fe5f415cc60 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:16.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.481+0000 7fe5f99ff700 1 --2- 172.21.15.92:0/796649762 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5f40b8510 0x7fe5f406b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:16.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.481+0000 7fe5f99ff700 1 --2- 172.21.15.92:0/796649762 >> v2:172.21.15.169:3300/0 conn(0x7fe5f40cd460 0x7fe5f406beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:16.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.481+0000 7fe5f99ff700 1 -- 172.21.15.92:0/796649762 >> 172.21.15.92:0/796649762 conn(0x7fe5f40719a0 msgr2=0x7fe5f4071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:16.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.482+0000 7fe5f99ff700 1 -- 172.21.15.92:0/796649762 shutdown_connections 2023-11-28T11:23:16.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.482+0000 7fe5f99ff700 1 -- 172.21.15.92:0/796649762 wait complete. 2023-11-28T11:23:16.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.484+0000 7fe5f99ff700 1 Processor -- start 2023-11-28T11:23:16.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.484+0000 7fe5f99ff700 1 -- start start 2023-11-28T11:23:16.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.485+0000 7fe5f99ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5f406c3f0 0x7fe5f40c7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:16.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.485+0000 7fe5f99ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5f40b8510 0x7fe5f41e3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:16.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.485+0000 7fe5f99ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5f40cd460 0x7fe5f41e1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:16.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.485+0000 7fe5f99ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f40c8ed0 con 0x7fe5f406c3f0 2023-11-28T11:23:16.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.485+0000 7fe5f99ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f40c82c0 con 0x7fe5f40b8510 2023-11-28T11:23:16.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.485+0000 7fe5f99ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f40c85a0 con 0x7fe5f40cd460 2023-11-28T11:23:16.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.486+0000 7fe5f2ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5f406c3f0 0x7fe5f40c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:16.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.486+0000 7fe5f2ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5f406c3f0 0x7fe5f40c7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40312/0 (socket says 172.21.15.92:40312) 2023-11-28T11:23:16.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.486+0000 7fe5f2ffd700 1 -- 172.21.15.92:0/1152239487 learned_addr learned my addr 172.21.15.92:0/1152239487 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:16.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.486+0000 7fe5ebfff700 1 --2- 172.21.15.92:0/1152239487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5f40b8510 0x7fe5f41e3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:16.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.486+0000 7fe5f37fe700 1 --2- 172.21.15.92:0/1152239487 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5f40cd460 0x7fe5f41e1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:16.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.486+0000 7fe5f2ffd700 1 -- 172.21.15.92:0/1152239487 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5f40cd460 msgr2=0x7fe5f41e1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:16.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.486+0000 7fe5f2ffd700 1 --2- 172.21.15.92:0/1152239487 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5f40cd460 0x7fe5f41e1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:16.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.486+0000 7fe5f2ffd700 1 -- 172.21.15.92:0/1152239487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5f40b8510 msgr2=0x7fe5f41e3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:16.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.486+0000 7fe5f2ffd700 1 --2- 172.21.15.92:0/1152239487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5f40b8510 0x7fe5f41e3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:16.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.486+0000 7fe5f2ffd700 1 -- 172.21.15.92:0/1152239487 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5ec009b30 con 0x7fe5f406c3f0 2023-11-28T11:23:16.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.486+0000 7fe5f2ffd700 1 --2- 172.21.15.92:0/1152239487 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5f406c3f0 0x7fe5f40c7990 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fe5e400e810 tx=0x7fe5e400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:16.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.487+0000 7fe5f0ff9700 1 -- 172.21.15.92:0/1152239487 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e400c070 con 0x7fe5f406c3f0 2023-11-28T11:23:16.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.487+0000 7fe5f0ff9700 1 -- 172.21.15.92:0/1152239487 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5e4013610 con 0x7fe5f406c3f0 2023-11-28T11:23:16.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.487+0000 7fe5f0ff9700 1 -- 172.21.15.92:0/1152239487 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e40139a0 con 0x7fe5f406c3f0 2023-11-28T11:23:16.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.487+0000 7fe5f99ff700 1 -- 172.21.15.92:0/1152239487 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5f40cfd10 con 0x7fe5f406c3f0 2023-11-28T11:23:16.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.487+0000 7fe5f99ff700 1 -- 172.21.15.92:0/1152239487 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5f40cd8f0 con 0x7fe5f406c3f0 2023-11-28T11:23:16.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.488+0000 7fe5f99ff700 1 -- 172.21.15.92:0/1152239487 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5f40cfff0 con 0x7fe5f406c3f0 2023-11-28T11:23:16.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.489+0000 7fe5f0ff9700 1 -- 172.21.15.92:0/1152239487 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe5e4023070 con 0x7fe5f406c3f0 2023-11-28T11:23:16.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.489+0000 7fe5f0ff9700 1 --2- 172.21.15.92:0/1152239487 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe5d40682f0 0x7fe5d406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:16.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.490+0000 7fe5f0ff9700 1 -- 172.21.15.92:0/1152239487 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fe5e40859d0 con 0x7fe5f406c3f0 2023-11-28T11:23:16.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.490+0000 7fe5ebfff700 1 --2- 172.21.15.92:0/1152239487 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe5d40682f0 0x7fe5d406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:16.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.490+0000 7fe5ebfff700 1 --2- 172.21.15.92:0/1152239487 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe5d40682f0 0x7fe5d406a7a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe5f415bb70 tx=0x7fe5dc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:16.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.493+0000 7fe5f0ff9700 1 -- 172.21.15.92:0/1152239487 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe5e40520f0 con 0x7fe5f406c3f0 2023-11-28T11:23:16.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.725+0000 7fe5f99ff700 1 -- 172.21.15.92:0/1152239487 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe5f4067880 con 0x7fe5f406c3f0 2023-11-28T11:23:16.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.726+0000 7fe5f0ff9700 1 -- 172.21.15.92:0/1152239487 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4651 (secure 0 0 0) 0x7fe5e4055710 con 0x7fe5f406c3f0 2023-11-28T11:23:16.727 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:23:16.727 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5593,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/3937189339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3937189339},{"type":"v1","addr":"172.21.15.169:6834","nonce":3937189339}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5596,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2020201016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2020201016},{"type":"v1","addr":"172.21.15.169:6835","nonce":2020201016}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5751,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2423750324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2423750324},{"type":"v1","addr":"172.21.15.92:6835","nonce":2423750324}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:23:13.247233+0000","modified":"2023-11-28T11:23:16.254256+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5754},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5754":{"gid":5754,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/950691230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":950691230},{"type":"v1","addr":"172.21.15.92:6837","nonce":950691230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T11:23:16.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.728+0000 7fe5f99ff700 1 -- 172.21.15.92:0/1152239487 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe5d40682f0 msgr2=0x7fe5d406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:16.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.728+0000 7fe5f99ff700 1 --2- 172.21.15.92:0/1152239487 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe5d40682f0 0x7fe5d406a7a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe5f415bb70 tx=0x7fe5dc008040 comp rx=0 tx=0).stop 2023-11-28T11:23:16.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.728+0000 7fe5f99ff700 1 -- 172.21.15.92:0/1152239487 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5f406c3f0 msgr2=0x7fe5f40c7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:16.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.728+0000 7fe5f99ff700 1 --2- 172.21.15.92:0/1152239487 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5f406c3f0 0x7fe5f40c7990 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fe5e400e810 tx=0x7fe5e400eb20 comp rx=0 tx=0).stop 2023-11-28T11:23:16.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.728+0000 7fe5f99ff700 1 -- 172.21.15.92:0/1152239487 shutdown_connections 2023-11-28T11:23:16.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.728+0000 7fe5f99ff700 1 --2- 172.21.15.92:0/1152239487 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe5f40cd460 0x7fe5f41e1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:16.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.728+0000 7fe5f99ff700 1 --2- 172.21.15.92:0/1152239487 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe5d40682f0 0x7fe5d406a7a0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:16.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.728+0000 7fe5f99ff700 1 --2- 172.21.15.92:0/1152239487 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5f406c3f0 0x7fe5f40c7990 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:16.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.728+0000 7fe5f99ff700 1 --2- 172.21.15.92:0/1152239487 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe5f40b8510 0x7fe5f41e3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:16.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.728+0000 7fe5f99ff700 1 -- 172.21.15.92:0/1152239487 >> 172.21.15.92:0/1152239487 conn(0x7fe5f40719a0 msgr2=0x7fe5f40d2130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:16.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.728+0000 7fe5f99ff700 1 -- 172.21.15.92:0/1152239487 shutdown_connections 2023-11-28T11:23:16.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.728+0000 7fe5f99ff700 1 -- 172.21.15.92:0/1152239487 wait complete. 2023-11-28T11:23:16.731 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 24 2023-11-28T11:23:16.747 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:23:16.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.944+0000 7feee9bff700 1 Processor -- start 2023-11-28T11:23:16.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.944+0000 7feee9bff700 1 -- start start 2023-11-28T11:23:16.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.945+0000 7feee9bff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7feee4158cb0 0x7feee40ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:16.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.945+0000 7feee9bff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feee4159080 0x7feee406dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:16.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.945+0000 7feee9bff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feee406d870 con 0x7feee40c9300 2023-11-28T11:23:16.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.945+0000 7feee9bff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feee406cc60 con 0x7feee4158cb0 2023-11-28T11:23:16.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.945+0000 7feee9bff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feee406cf40 con 0x7feee4159080 2023-11-28T11:23:16.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.946+0000 7feee2ffd700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feee4159080 0x7feee406dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:16.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.946+0000 7feee2ffd700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feee4159080 0x7feee406dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.92:47468/0 (socket says 172.21.15.92:47468) 2023-11-28T11:23:16.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.946+0000 7feee2ffd700 1 -- 172.21.15.92:0/1975433843 learned_addr learned my addr 172.21.15.92:0/1975433843 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:16.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.946+0000 7feee37fe700 1 --2- 172.21.15.92:0/1975433843 >> v2:172.21.15.169:3300/0 conn(0x7feee4158cb0 0x7feee40ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:16.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.946+0000 7feee2ffd700 1 -- 172.21.15.92:0/1975433843 >> v2:172.21.15.169:3300/0 conn(0x7feee4158cb0 msgr2=0x7feee40ca1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:16.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.946+0000 7feee2ffd700 1 --2- 172.21.15.92:0/1975433843 >> v2:172.21.15.169:3300/0 conn(0x7feee4158cb0 0x7feee40ca1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:16.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.946+0000 7feee2ffd700 1 -- 172.21.15.92:0/1975433843 >> v1:172.21.15.169:6789/0 conn(0x7feee40c9300 legacy=0x7feee4070910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:16.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.946+0000 7feee2ffd700 1 -- 172.21.15.92:0/1975433843 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee406d220 con 0x7feee4159080 2023-11-28T11:23:16.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.946+0000 7feee27fc700 1 -- 172.21.15.92:0/1975433843 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2450084784 0 0) 0x7feee406d870 con 0x7feee40c9300 2023-11-28T11:23:16.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.946+0000 7feee2ffd700 1 --2- 172.21.15.92:0/1975433843 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feee4159080 0x7feee406dd00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7feed80098f0 tx=0x7feed8009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcded5049f6d6e2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:16.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.947+0000 7feee27fc700 1 -- 172.21.15.92:0/1975433843 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed8011830 con 0x7feee4159080 2023-11-28T11:23:16.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.947+0000 7feee27fc700 1 -- 172.21.15.92:0/1975433843 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feed8011990 con 0x7feee4159080 2023-11-28T11:23:16.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.947+0000 7feee27fc700 1 -- 172.21.15.92:0/1975433843 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed8011d20 con 0x7feee4159080 2023-11-28T11:23:16.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.947+0000 7feee9bff700 1 -- 172.21.15.92:0/1975433843 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feee4159080 msgr2=0x7feee406dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:16.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.947+0000 7feee9bff700 1 --2- 172.21.15.92:0/1975433843 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feee4159080 0x7feee406dd00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7feed80098f0 tx=0x7feed8009c00 comp rx=0 tx=0).stop 2023-11-28T11:23:16.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.947+0000 7feee9bff700 1 -- 172.21.15.92:0/1975433843 shutdown_connections 2023-11-28T11:23:16.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.947+0000 7feee9bff700 1 --2- 172.21.15.92:0/1975433843 >> v2:172.21.15.169:3300/0 conn(0x7feee4158cb0 0x7feee40ca1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:16.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.947+0000 7feee9bff700 1 --2- 172.21.15.92:0/1975433843 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feee4159080 0x7feee406dd00 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:16.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.947+0000 7feee9bff700 1 -- 172.21.15.92:0/1975433843 >> 172.21.15.92:0/1975433843 conn(0x7feee40cb490 msgr2=0x7feee40cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:16.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.947+0000 7feee9bff700 1 -- 172.21.15.92:0/1975433843 shutdown_connections 2023-11-28T11:23:16.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.948+0000 7feee9bff700 1 -- 172.21.15.92:0/1975433843 wait complete. 2023-11-28T11:23:16.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.950+0000 7feee9bff700 1 Processor -- start 2023-11-28T11:23:16.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.950+0000 7feee9bff700 1 -- start start 2023-11-28T11:23:16.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.951+0000 7feee9bff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feee40c9300 0x7feee415b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:16.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.951+0000 7feee9bff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feee4158cb0 0x7feee40c6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:16.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.951+0000 7feee9bff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feee415bb30 0x7feee40c7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:16.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.951+0000 7feee9bff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee415a550 con 0x7feee40c9300 2023-11-28T11:23:16.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.951+0000 7feee9bff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee415a830 con 0x7feee4158cb0 2023-11-28T11:23:16.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.951+0000 7feee9bff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feee415ab10 con 0x7feee415bb30 2023-11-28T11:23:16.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.951+0000 7feee37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feee40c9300 0x7feee415b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:16.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.951+0000 7feee37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feee40c9300 0x7feee415b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40318/0 (socket says 172.21.15.92:40318) 2023-11-28T11:23:16.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.952+0000 7feee37fe700 1 -- 172.21.15.92:0/3943882022 learned_addr learned my addr 172.21.15.92:0/3943882022 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:16.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.952+0000 7feee2ffd700 1 --2- 172.21.15.92:0/3943882022 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feee4158cb0 0x7feee40c6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:16.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.952+0000 7feee3fff700 1 --2- 172.21.15.92:0/3943882022 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feee415bb30 0x7feee40c7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:16.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.952+0000 7feee37fe700 1 -- 172.21.15.92:0/3943882022 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feee415bb30 msgr2=0x7feee40c7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:16.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.952+0000 7feee37fe700 1 --2- 172.21.15.92:0/3943882022 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feee415bb30 0x7feee40c7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:16.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.952+0000 7feee37fe700 1 -- 172.21.15.92:0/3943882022 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feee4158cb0 msgr2=0x7feee40c6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:16.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.952+0000 7feee37fe700 1 --2- 172.21.15.92:0/3943882022 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feee4158cb0 0x7feee40c6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:16.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.952+0000 7feee37fe700 1 -- 172.21.15.92:0/3943882022 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed8009550 con 0x7feee40c9300 2023-11-28T11:23:16.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.952+0000 7feee37fe700 1 --2- 172.21.15.92:0/3943882022 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feee40c9300 0x7feee415b5f0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7feed400e9b0 tx=0x7feed400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:16.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.952+0000 7feecbfff700 1 -- 172.21.15.92:0/3943882022 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed4009710 con 0x7feee40c9300 2023-11-28T11:23:16.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.952+0000 7feecbfff700 1 -- 172.21.15.92:0/3943882022 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feed400ee50 con 0x7feee40c9300 2023-11-28T11:23:16.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.952+0000 7feecbfff700 1 -- 172.21.15.92:0/3943882022 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed400ca50 con 0x7feee40c9300 2023-11-28T11:23:16.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.953+0000 7feee9bff700 1 -- 172.21.15.92:0/3943882022 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feee4071b20 con 0x7feee40c9300 2023-11-28T11:23:16.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.953+0000 7feee9bff700 1 -- 172.21.15.92:0/3943882022 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feee406d500 con 0x7feee40c9300 2023-11-28T11:23:16.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.954+0000 7feee9bff700 1 -- 172.21.15.92:0/3943882022 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feee4071020 con 0x7feee40c9300 2023-11-28T11:23:16.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.954+0000 7feecbfff700 1 -- 172.21.15.92:0/3943882022 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7feed4019030 con 0x7feee40c9300 2023-11-28T11:23:16.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.955+0000 7feecbfff700 1 --2- 172.21.15.92:0/3943882022 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7feed00682f0 0x7feed006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:16.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.955+0000 7feecbfff700 1 -- 172.21.15.92:0/3943882022 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6382+0+0 (secure 0 0 0) 0x7feed4085dc0 con 0x7feee40c9300 2023-11-28T11:23:16.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.956+0000 7feee2ffd700 1 --2- 172.21.15.92:0/3943882022 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7feed00682f0 0x7feed006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:16.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.956+0000 7feee2ffd700 1 --2- 172.21.15.92:0/3943882022 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7feed00682f0 0x7feed006a7a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7feed8014040 tx=0x7feed8008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:16.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:16.960+0000 7feecbfff700 1 -- 172.21.15.92:0/3943882022 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7feed4052390 con 0x7feee40c9300 2023-11-28T11:23:17.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.153+0000 7feee9bff700 1 -- 172.21.15.92:0/3943882022 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feee4067880 con 0x7feee40c9300 2023-11-28T11:23:17.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.154+0000 7feecbfff700 1 -- 172.21.15.92:0/3943882022 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+18180 (secure 0 0 0) 0x7feed4017090 con 0x7feee40c9300 2023-11-28T11:23:17.155 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:23:17.156 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":56,"fsid":"68956e0c-c16f-47cd-a220-f98364b0831d","created":"2023-11-28T11:20:28.697573+0000","modified":"2023-11-28T11:23:14.249429+0000","last_up_change":"2023-11-28T11:20:41.257011+0000","last_in_change":"2023-11-28T11:20:32.249767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:20:41.925237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:20:48.682497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:23:10.939884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-28T11:23:12.227433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6abe483f-0e03-4a26-b9cf-62041291747e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":115823},{"type":"v1","addr":"172.21.15.92:6809","nonce":115823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":115823},{"type":"v1","addr":"172.21.15.92:6813","nonce":115823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":115823},{"type":"v1","addr":"172.21.15.92:6821","nonce":115823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":115823},{"type":"v1","addr":"172.21.15.92:6817","nonce":115823}]},"public_addr":"172.21.15.92:6809/115823","cluster_addr":"172.21.15.92:6813/115823","heartbeat_back_addr":"172.21.15.92:6821/115823","heartbeat_front_addr":"172.21.15.92:6817/115823","state":["exists","up"]},{"osd":1,"uuid":"13c1de73-1a8c-4d7e-8a28-9dae52dfe089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":115824},{"type":"v1","addr":"172.21.15.92:6812","nonce":115824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":115824},{"type":"v1","addr":"172.21.15.92:6816","nonce":115824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":115824},{"type":"v1","addr":"172.21.15.92:6823","nonce":115824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":115824},{"type":"v1","addr":"172.21.15.92:6820","nonce":115824}]},"public_addr":"172.21.15.92:6812/115824","cluster_addr":"172.21.15.92:6816/115824","heartbeat_back_addr":"172.21.15.92:6823/115824","heartbeat_front_addr":"172.21.15.92:6820/115824","state":["exists","up"]},{"osd":2,"uuid":"25788a55-112f-4f00-810d-2bc37cf76778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":115821},{"type":"v1","addr":"172.21.15.92:6801","nonce":115821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":115821},{"type":"v1","addr":"172.21.15.92:6803","nonce":115821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":115821},{"type":"v1","addr":"172.21.15.92:6807","nonce":115821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":115821},{"type":"v1","addr":"172.21.15.92:6805","nonce":115821}]},"public_addr":"172.21.15.92:6801/115821","cluster_addr":"172.21.15.92:6803/115821","heartbeat_back_addr":"172.21.15.92:6807/115821","heartbeat_front_addr":"172.21.15.92:6805/115821","state":["exists","up"]},{"osd":3,"uuid":"adf54e24-33a9-41fe-9ce3-98a5ab6c158e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":115825},{"type":"v1","addr":"172.21.15.92:6825","nonce":115825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":115825},{"type":"v1","addr":"172.21.15.92:6827","nonce":115825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":115825},{"type":"v1","addr":"172.21.15.92:6831","nonce":115825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":115825},{"type":"v1","addr":"172.21.15.92:6829","nonce":115825}]},"public_addr":"172.21.15.92:6825/115825","cluster_addr":"172.21.15.92:6827/115825","heartbeat_back_addr":"172.21.15.92:6831/115825","heartbeat_front_addr":"172.21.15.92:6829/115825","state":["exists","up"]},{"osd":4,"uuid":"ccb9b747-f399-44ea-9a18-9db3152227e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":116035},{"type":"v1","addr":"172.21.15.169:6817","nonce":116035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":116035},{"type":"v1","addr":"172.21.15.169:6819","nonce":116035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":116035},{"type":"v1","addr":"172.21.15.169:6826","nonce":116035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":116035},{"type":"v1","addr":"172.21.15.169:6822","nonce":116035}]},"public_addr":"172.21.15.169:6817/116035","cluster_addr":"172.21.15.169:6819/116035","heartbeat_back_addr":"172.21.15.169:6826/116035","heartbeat_front_addr":"172.21.15.169:6822/116035","state":["exists","up"]},{"osd":5,"uuid":"308a1e63-75aa-4d7d-8a7c-cecc24714fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":116033},{"type":"v1","addr":"172.21.15.169:6801","nonce":116033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":116033},{"type":"v1","addr":"172.21.15.169:6803","nonce":116033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":116033},{"type":"v1","addr":"172.21.15.169:6807","nonce":116033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":116033},{"type":"v1","addr":"172.21.15.169:6805","nonce":116033}]},"public_addr":"172.21.15.169:6801/116033","cluster_addr":"172.21.15.169:6803/116033","heartbeat_back_addr":"172.21.15.169:6807/116033","heartbeat_front_addr":"172.21.15.169:6805/116033","state":["exists","up"]},{"osd":6,"uuid":"9c459b7b-b821-4b4b-b91e-6aaae271457c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":116034},{"type":"v1","addr":"172.21.15.169:6809","nonce":116034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":116034},{"type":"v1","addr":"172.21.15.169:6811","nonce":116034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":116034},{"type":"v1","addr":"172.21.15.169:6815","nonce":116034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":116034},{"type":"v1","addr":"172.21.15.169:6813","nonce":116034}]},"public_addr":"172.21.15.169:6809/116034","cluster_addr":"172.21.15.169:6811/116034","heartbeat_back_addr":"172.21.15.169:6815/116034","heartbeat_front_addr":"172.21.15.169:6813/116034","state":["exists","up"]},{"osd":7,"uuid":"1ca9aa4a-755d-4e28-a937-4d8dccab4df3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":116044},{"type":"v1","addr":"172.21.15.169:6823","nonce":116044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6825","nonce":116044},{"type":"v1","addr":"172.21.15.169:6827","nonce":116044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":116044},{"type":"v1","addr":"172.21.15.169:6831","nonce":116044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":116044},{"type":"v1","addr":"172.21.15.169:6829","nonce":116044}]},"public_addr":"172.21.15.169:6823/116044","cluster_addr":"172.21.15.169:6827/116044","heartbeat_back_addr":"172.21.15.169:6831/116044","heartbeat_front_addr":"172.21.15.169:6829/116044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.586781+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.869778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.508594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.860493+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.872267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.746933+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.611520+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.871446+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:23:17.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.156+0000 7feee9bff700 1 -- 172.21.15.92:0/3943882022 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7feed00682f0 msgr2=0x7feed006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:17.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.156+0000 7feee9bff700 1 --2- 172.21.15.92:0/3943882022 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7feed00682f0 0x7feed006a7a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7feed8014040 tx=0x7feed8008030 comp rx=0 tx=0).stop 2023-11-28T11:23:17.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.156+0000 7feee9bff700 1 -- 172.21.15.92:0/3943882022 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feee40c9300 msgr2=0x7feee415b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:17.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.156+0000 7feee9bff700 1 --2- 172.21.15.92:0/3943882022 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feee40c9300 0x7feee415b5f0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7feed400e9b0 tx=0x7feed400ecc0 comp rx=0 tx=0).stop 2023-11-28T11:23:17.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.157+0000 7feee9bff700 1 -- 172.21.15.92:0/3943882022 shutdown_connections 2023-11-28T11:23:17.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.157+0000 7feee9bff700 1 --2- 172.21.15.92:0/3943882022 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feee415bb30 0x7feee40c7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:17.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.157+0000 7feee9bff700 1 --2- 172.21.15.92:0/3943882022 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7feed00682f0 0x7feed006a7a0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:17.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.157+0000 7feee9bff700 1 --2- 172.21.15.92:0/3943882022 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feee40c9300 0x7feee415b5f0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:17.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.157+0000 7feee9bff700 1 --2- 172.21.15.92:0/3943882022 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feee4158cb0 0x7feee40c6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:17.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.157+0000 7feee9bff700 1 -- 172.21.15.92:0/3943882022 >> 172.21.15.92:0/3943882022 conn(0x7feee40cb490 msgr2=0x7feee40b3850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:17.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.157+0000 7feee9bff700 1 -- 172.21.15.92:0/3943882022 shutdown_connections 2023-11-28T11:23:17.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.157+0000 7feee9bff700 1 -- 172.21.15.92:0/3943882022 wait complete. 2023-11-28T11:23:17.176 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T11:23:17.176 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:23:17.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.373+0000 7f27277ff700 1 Processor -- start 2023-11-28T11:23:17.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.374+0000 7f27277ff700 1 -- start start 2023-11-28T11:23:17.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.375+0000 7f27277ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27200b8510 0x7f272006b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:17.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.375+0000 7f27277ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f27200cd460 0x7f272006beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:17.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.375+0000 7f27277ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f272006c3f0 0x7f272015cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:17.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.375+0000 7f27277ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27200cdbd0 con 0x7f27200cd460 2023-11-28T11:23:17.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.375+0000 7f27277ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f272006c830 con 0x7f272006c3f0 2023-11-28T11:23:17.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.375+0000 7f27277ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27200d02d0 con 0x7f27200b8510 2023-11-28T11:23:17.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.375+0000 7f2724d9a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f27200cd460 0x7f272006beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:17.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.375+0000 7f2724d9a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f27200cd460 0x7f272006beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40334/0 (socket says 172.21.15.92:40334) 2023-11-28T11:23:17.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.375+0000 7f2724d9a700 1 -- 172.21.15.92:0/2544735192 learned_addr learned my addr 172.21.15.92:0/2544735192 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:17.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.376+0000 7f2724d9a700 1 -- 172.21.15.92:0/2544735192 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27200b8510 msgr2=0x7f272006b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:17.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.376+0000 7f2725d9c700 1 --2- 172.21.15.92:0/2544735192 >> v2:172.21.15.169:3300/0 conn(0x7f272006c3f0 0x7f272015cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:17.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.376+0000 7f272559b700 1 --2- 172.21.15.92:0/2544735192 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27200b8510 0x7f272006b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:17.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.376+0000 7f2724d9a700 1 --2- 172.21.15.92:0/2544735192 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27200b8510 0x7f272006b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:17.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.376+0000 7f2724d9a700 1 -- 172.21.15.92:0/2544735192 >> v2:172.21.15.169:3300/0 conn(0x7f272006c3f0 msgr2=0x7f272015cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:17.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.376+0000 7f2724d9a700 1 --2- 172.21.15.92:0/2544735192 >> v2:172.21.15.169:3300/0 conn(0x7f272006c3f0 0x7f272015cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:17.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.376+0000 7f2724d9a700 1 -- 172.21.15.92:0/2544735192 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27200b88e0 con 0x7f27200cd460 2023-11-28T11:23:17.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.376+0000 7f2725d9c700 1 --2- 172.21.15.92:0/2544735192 >> v2:172.21.15.169:3300/0 conn(0x7f272006c3f0 0x7f272015cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:23:17.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.376+0000 7f272559b700 1 --2- 172.21.15.92:0/2544735192 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27200b8510 0x7f272006b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:23:17.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.376+0000 7f2724d9a700 1 --2- 172.21.15.92:0/2544735192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27200cd460 0x7f272006beb0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f27200cdd40 tx=0x7f2710009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a3f33b96dbf49e2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:17.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.376+0000 7f2717fff700 1 -- 172.21.15.92:0/2544735192 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f271000b9e0 con 0x7f27200cd460 2023-11-28T11:23:17.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.376+0000 7f2717fff700 1 -- 172.21.15.92:0/2544735192 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f271000bb40 con 0x7f27200cd460 2023-11-28T11:23:17.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.376+0000 7f2717fff700 1 -- 172.21.15.92:0/2544735192 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2710009e70 con 0x7f27200cd460 2023-11-28T11:23:17.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.377+0000 7f27277ff700 1 -- 172.21.15.92:0/2544735192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27200cd460 msgr2=0x7f272006beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:17.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.377+0000 7f27277ff700 1 --2- 172.21.15.92:0/2544735192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27200cd460 0x7f272006beb0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f27200cdd40 tx=0x7f2710009c30 comp rx=0 tx=0).stop 2023-11-28T11:23:17.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.377+0000 7f27277ff700 1 -- 172.21.15.92:0/2544735192 shutdown_connections 2023-11-28T11:23:17.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.377+0000 7f27277ff700 1 --2- 172.21.15.92:0/2544735192 >> v2:172.21.15.169:3300/0 conn(0x7f272006c3f0 0x7f272015cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:17.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.377+0000 7f27277ff700 1 --2- 172.21.15.92:0/2544735192 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27200b8510 0x7f272006b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:17.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.377+0000 7f27277ff700 1 --2- 172.21.15.92:0/2544735192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27200cd460 0x7f272006beb0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:17.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.377+0000 7f27277ff700 1 -- 172.21.15.92:0/2544735192 >> 172.21.15.92:0/2544735192 conn(0x7f27200719a0 msgr2=0x7f2720071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:17.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.377+0000 7f27277ff700 1 -- 172.21.15.92:0/2544735192 shutdown_connections 2023-11-28T11:23:17.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.377+0000 7f27277ff700 1 -- 172.21.15.92:0/2544735192 wait complete. 2023-11-28T11:23:17.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.378+0000 7f27277ff700 1 Processor -- start 2023-11-28T11:23:17.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.379+0000 7f27277ff700 1 -- start start 2023-11-28T11:23:17.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.379+0000 7f27277ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f272006c3f0 0x7f27200c8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:17.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.379+0000 7f27277ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27200b8510 0x7f2720159a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:17.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.379+0000 7f27277ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27200cd460 0x7f2720159f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:17.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.379+0000 7f27277ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27201585a0 con 0x7f27200cd460 2023-11-28T11:23:17.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.379+0000 7f27277ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2720157d00 con 0x7f272006c3f0 2023-11-28T11:23:17.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.380+0000 7f27277ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2720157fe0 con 0x7f27200b8510 2023-11-28T11:23:17.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.380+0000 7f272559b700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f272006c3f0 0x7f27200c8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:17.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.380+0000 7f272559b700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f272006c3f0 0x7f27200c8fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.92:60072/0 (socket says 172.21.15.92:60072) 2023-11-28T11:23:17.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.380+0000 7f2725d9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27200cd460 0x7f2720159f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:17.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.380+0000 7f272559b700 1 -- 172.21.15.92:0/1352438933 learned_addr learned my addr 172.21.15.92:0/1352438933 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:17.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.380+0000 7f2724d9a700 1 --2- 172.21.15.92:0/1352438933 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27200b8510 0x7f2720159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:17.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.380+0000 7f2724d9a700 1 -- 172.21.15.92:0/1352438933 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f272006c3f0 msgr2=0x7f27200c8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:17.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.380+0000 7f2724d9a700 1 --2- 172.21.15.92:0/1352438933 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f272006c3f0 0x7f27200c8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:17.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.380+0000 7f2724d9a700 1 -- 172.21.15.92:0/1352438933 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27200cd460 msgr2=0x7f2720159f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:17.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.380+0000 7f2724d9a700 1 --2- 172.21.15.92:0/1352438933 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27200cd460 0x7f2720159f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:17.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.380+0000 7f2724d9a700 1 -- 172.21.15.92:0/1352438933 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2710009550 con 0x7f27200b8510 2023-11-28T11:23:17.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.380+0000 7f2725d9c700 1 --2- 172.21.15.92:0/1352438933 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27200cd460 0x7f2720159f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:23:17.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.380+0000 7f272559b700 1 --2- 172.21.15.92:0/1352438933 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f272006c3f0 0x7f27200c8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:23:17.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.381+0000 7f2724d9a700 1 --2- 172.21.15.92:0/1352438933 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27200b8510 0x7f2720159a20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2710016000 tx=0x7f2710004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:17.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.381+0000 7f2715ffb700 1 -- 172.21.15.92:0/1352438933 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2710004590 con 0x7f27200b8510 2023-11-28T11:23:17.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.381+0000 7f2715ffb700 1 -- 172.21.15.92:0/1352438933 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f271001e070 con 0x7f27200b8510 2023-11-28T11:23:17.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.381+0000 7f27277ff700 1 -- 172.21.15.92:0/1352438933 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27200cfd10 con 0x7f27200b8510 2023-11-28T11:23:17.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.381+0000 7f2715ffb700 1 -- 172.21.15.92:0/1352438933 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27100038a0 con 0x7f27200b8510 2023-11-28T11:23:17.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.381+0000 7f27277ff700 1 -- 172.21.15.92:0/1352438933 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27200cd8f0 con 0x7f27200b8510 2023-11-28T11:23:17.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.382+0000 7f27277ff700 1 -- 172.21.15.92:0/1352438933 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27200cfff0 con 0x7f27200b8510 2023-11-28T11:23:17.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.383+0000 7f2715ffb700 1 -- 172.21.15.92:0/1352438933 <== mon.2 v2:172.21.15.169:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f271000f050 con 0x7f27200b8510 2023-11-28T11:23:17.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.384+0000 7f2715ffb700 1 --2- 172.21.15.92:0/1352438933 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f27040682f0 0x7f270406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:17.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.384+0000 7f2715ffb700 1 -- 172.21.15.92:0/1352438933 <== mon.2 v2:172.21.15.169:3301/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f2710020070 con 0x7f27200b8510 2023-11-28T11:23:17.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.384+0000 7f272559b700 1 --2- 172.21.15.92:0/1352438933 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f27040682f0 0x7f270406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:17.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.385+0000 7f272559b700 1 --2- 172.21.15.92:0/1352438933 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f27040682f0 0x7f270406a7a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f271c006fd0 tx=0x7f271c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:17.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.389+0000 7f2715ffb700 1 -- 172.21.15.92:0/1352438933 <== mon.2 v2:172.21.15.169:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f27100538a0 con 0x7f27200b8510 2023-11-28T11:23:17.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.576+0000 7f27277ff700 1 -- 172.21.15.92:0/1352438933 --> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2720067880 con 0x7f27040682f0 2023-11-28T11:23:17.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.585+0000 7f2715ffb700 1 -- 172.21.15.92:0/1352438933 <== mgr.4103 v2:172.21.15.92:6832/115482 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+300347 (secure 0 0 0) 0x7f2720067880 con 0x7f27040682f0 2023-11-28T11:23:17.592 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:23:17.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.589+0000 7f27277ff700 1 -- 172.21.15.92:0/1352438933 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f27040682f0 msgr2=0x7f270406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:17.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.589+0000 7f27277ff700 1 --2- 172.21.15.92:0/1352438933 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f27040682f0 0x7f270406a7a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f271c006fd0 tx=0x7f271c008040 comp rx=0 tx=0).stop 2023-11-28T11:23:17.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.589+0000 7f27277ff700 1 -- 172.21.15.92:0/1352438933 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27200b8510 msgr2=0x7f2720159a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:17.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.589+0000 7f27277ff700 1 --2- 172.21.15.92:0/1352438933 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27200b8510 0x7f2720159a20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2710016000 tx=0x7f2710004750 comp rx=0 tx=0).stop 2023-11-28T11:23:17.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.589+0000 7f27277ff700 1 -- 172.21.15.92:0/1352438933 shutdown_connections 2023-11-28T11:23:17.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.589+0000 7f27277ff700 1 --2- 172.21.15.92:0/1352438933 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f27040682f0 0x7f270406a7a0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:17.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.589+0000 7f27277ff700 1 --2- 172.21.15.92:0/1352438933 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27200cd460 0x7f2720159f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:17.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.589+0000 7f27277ff700 1 --2- 172.21.15.92:0/1352438933 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f272006c3f0 0x7f27200c8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:17.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.589+0000 7f27277ff700 1 --2- 172.21.15.92:0/1352438933 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f27200b8510 0x7f2720159a20 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:17.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.589+0000 7f27277ff700 1 -- 172.21.15.92:0/1352438933 >> 172.21.15.92:0/1352438933 conn(0x7f27200719a0 msgr2=0x7f27200b27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:17.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.589+0000 7f27277ff700 1 -- 172.21.15.92:0/1352438933 shutdown_connections 2023-11-28T11:23:17.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.589+0000 7f27277ff700 1 -- 172.21.15.92:0/1352438933 wait complete. 2023-11-28T11:23:17.596 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-11-28T11:23:17.608 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":126,"stamp":"2023-11-28T11:23:15.889106+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":74660,"kb_used_data":5988,"kb_used_omap":0,"kb_used_meta":68608,"kb_avail":754900060,"statfs":{"total":773094113280,"available":773017661440,"internally_reserved":0,"allocated":6131712,"data_stored":3429602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":70254592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.143289"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2023-11-28T11:23:11.749176+0000","last_change":"2023-11-28T11:23:11.749176+0000","last_active":"2023-11-28T11:23:11.735760+0000","last_peered":"2023-11-28T11:23:11.735760+0000","last_clean":"2023-11-28T11:23:11.735760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:11.749176+0000","last_undegraded":"2023-11-28T11:23:11.749176+0000","last_fullsized":"2023-11-28T11:23:11.749176+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:18:12.361339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2023-11-28T11:23:11.751475+0000","last_change":"2023-11-28T11:23:11.751475+0000","last_active":"2023-11-28T11:23:11.735760+0000","last_peered":"2023-11-28T11:23:11.735760+0000","last_clean":"2023-11-28T11:23:11.735760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:11.751475+0000","last_undegraded":"2023-11-28T11:23:11.751475+0000","last_fullsized":"2023-11-28T11:23:11.751475+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:36:26.866427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2023-11-28T11:23:11.749642+0000","last_change":"2023-11-28T11:23:11.749642+0000","last_active":"2023-11-28T11:23:11.735760+0000","last_peered":"2023-11-28T11:23:11.735760+0000","last_clean":"2023-11-28T11:23:11.735760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:11.749642+0000","last_undegraded":"2023-11-28T11:23:11.749642+0000","last_fullsized":"2023-11-28T11:23:11.749642+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:11:36.215172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2023-11-28T11:23:11.750236+0000","last_change":"2023-11-28T11:23:11.750236+0000","last_active":"2023-11-28T11:23:11.735760+0000","last_peered":"2023-11-28T11:23:11.735760+0000","last_clean":"2023-11-28T11:23:11.735760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:11.750236+0000","last_undegraded":"2023-11-28T11:23:11.750236+0000","last_fullsized":"2023-11-28T11:23:11.750236+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:42:40.152252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2023-11-28T11:23:11.749244+0000","last_change":"2023-11-28T11:23:11.749244+0000","last_active":"2023-11-28T11:23:11.735760+0000","last_peered":"2023-11-28T11:23:11.735760+0000","last_clean":"2023-11-28T11:23:11.735760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:11.749244+0000","last_undegraded":"2023-11-28T11:23:11.749244+0000","last_fullsized":"2023-11-28T11:23:11.749244+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:07:05.101057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2023-11-28T11:23:11.750394+0000","last_change":"2023-11-28T11:23:11.750394+0000","last_active":"2023-11-28T11:23:11.735760+0000","last_peered":"2023-11-28T11:23:11.735760+0000","last_clean":"2023-11-28T11:23:11.735760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:11.750394+0000","last_undegraded":"2023-11-28T11:23:11.750394+0000","last_fullsized":"2023-11-28T11:23:11.750394+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:46:14.315147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2023-11-28T11:23:11.749672+0000","last_change":"2023-11-28T11:23:11.749672+0000","last_active":"2023-11-28T11:23:11.735760+0000","last_peered":"2023-11-28T11:23:11.735760+0000","last_clean":"2023-11-28T11:23:11.735760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:11.749672+0000","last_undegraded":"2023-11-28T11:23:11.749672+0000","last_fullsized":"2023-11-28T11:23:11.749672+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:50:37.417549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2023-11-28T11:23:11.748775+0000","last_change":"2023-11-28T11:23:11.748775+0000","last_active":"2023-11-28T11:23:11.735760+0000","last_peered":"2023-11-28T11:23:11.735760+0000","last_clean":"2023-11-28T11:23:11.735760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:11.748775+0000","last_undegraded":"2023-11-28T11:23:11.748775+0000","last_fullsized":"2023-11-28T11:23:11.748775+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:18:12.361339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2023-11-28T11:23:11.753199+0000","last_change":"2023-11-28T11:23:11.753199+0000","last_active":"2023-11-28T11:23:11.735760+0000","last_peered":"2023-11-28T11:23:11.735760+0000","last_clean":"2023-11-28T11:23:11.735760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:11.753199+0000","last_undegraded":"2023-11-28T11:23:11.753199+0000","last_fullsized":"2023-11-28T11:23:11.753199+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:19:39.112941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2023-11-28T11:23:11.749465+0000","last_change":"2023-11-28T11:23:11.749465+0000","last_active":"2023-11-28T11:23:11.735760+0000","last_peered":"2023-11-28T11:23:11.735760+0000","last_clean":"2023-11-28T11:23:11.735760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:11.749465+0000","last_undegraded":"2023-11-28T11:23:11.749465+0000","last_fullsized":"2023-11-28T11:23:11.749465+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:13:55.826943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2023-11-28T11:23:11.753187+0000","last_change":"2023-11-28T11:23:11.753187+0000","last_active":"2023-11-28T11:23:11.735760+0000","last_peered":"2023-11-28T11:23:11.735760+0000","last_clean":"2023-11-28T11:23:11.735760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:11.753187+0000","last_undegraded":"2023-11-28T11:23:11.753187+0000","last_fullsized":"2023-11-28T11:23:11.753187+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:37:08.575542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2023-11-28T11:23:11.750151+0000","last_change":"2023-11-28T11:23:11.750151+0000","last_active":"2023-11-28T11:23:11.735760+0000","last_peered":"2023-11-28T11:23:11.735760+0000","last_clean":"2023-11-28T11:23:11.735760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:11.750151+0000","last_undegraded":"2023-11-28T11:23:11.750151+0000","last_fullsized":"2023-11-28T11:23:11.750151+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:11:36.215172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2023-11-28T11:23:11.750009+0000","last_change":"2023-11-28T11:23:11.750009+0000","last_active":"2023-11-28T11:23:11.735760+0000","last_peered":"2023-11-28T11:23:11.735760+0000","last_clean":"2023-11-28T11:23:11.735760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:11.750009+0000","last_undegraded":"2023-11-28T11:23:11.750009+0000","last_fullsized":"2023-11-28T11:23:11.750009+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:15:10.937464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2023-11-28T11:23:11.755379+0000","last_change":"2023-11-28T11:23:11.755379+0000","last_active":"2023-11-28T11:23:11.735760+0000","last_peered":"2023-11-28T11:23:11.735760+0000","last_clean":"2023-11-28T11:23:11.735760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:11.755379+0000","last_undegraded":"2023-11-28T11:23:11.755379+0000","last_fullsized":"2023-11-28T11:23:11.755379+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:52:07.798680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2023-11-28T11:23:11.748728+0000","last_change":"2023-11-28T11:23:11.748728+0000","last_active":"2023-11-28T11:23:11.735760+0000","last_peered":"2023-11-28T11:23:11.735760+0000","last_clean":"2023-11-28T11:23:11.735760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:11.748728+0000","last_undegraded":"2023-11-28T11:23:11.748728+0000","last_fullsized":"2023-11-28T11:23:11.748728+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:30:37.672629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2023-11-28T11:23:11.753163+0000","last_change":"2023-11-28T11:23:11.753163+0000","last_active":"2023-11-28T11:23:11.735760+0000","last_peered":"2023-11-28T11:23:11.735760+0000","last_clean":"2023-11-28T11:23:11.735760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:11.753163+0000","last_undegraded":"2023-11-28T11:23:11.753163+0000","last_fullsized":"2023-11-28T11:23:11.753163+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:04:55.693431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2023-11-28T11:23:11.745965+0000","last_change":"2023-11-28T11:23:11.745965+0000","last_active":"2023-11-28T11:23:11.735760+0000","last_peered":"2023-11-28T11:23:11.735760+0000","last_clean":"2023-11-28T11:23:11.735760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:11.745965+0000","last_undegraded":"2023-11-28T11:23:11.745965+0000","last_fullsized":"2023-11-28T11:23:11.745965+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:40:06.308424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2023-11-28T11:23:11.749348+0000","last_change":"2023-11-28T11:23:11.749348+0000","last_active":"2023-11-28T11:23:11.735760+0000","last_peered":"2023-11-28T11:23:11.735760+0000","last_clean":"2023-11-28T11:23:11.735760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:11.749348+0000","last_undegraded":"2023-11-28T11:23:11.749348+0000","last_fullsized":"2023-11-28T11:23:11.749348+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:43:12.367309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2023-11-28T11:23:11.755479+0000","last_change":"2023-11-28T11:23:11.755479+0000","last_active":"2023-11-28T11:23:11.735760+0000","last_peered":"2023-11-28T11:23:11.735760+0000","last_clean":"2023-11-28T11:23:11.735760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:11.755479+0000","last_undegraded":"2023-11-28T11:23:11.755479+0000","last_fullsized":"2023-11-28T11:23:11.755479+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:53:02.731439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2023-11-28T11:23:11.748725+0000","last_change":"2023-11-28T11:23:11.748725+0000","last_active":"2023-11-28T11:23:11.735760+0000","last_peered":"2023-11-28T11:23:11.735760+0000","last_clean":"2023-11-28T11:23:11.735760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:11.748725+0000","last_undegraded":"2023-11-28T11:23:11.748725+0000","last_fullsized":"2023-11-28T11:23:11.748725+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:49:02.897556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2023-11-28T11:23:11.746002+0000","last_change":"2023-11-28T11:23:11.746002+0000","last_active":"2023-11-28T11:23:11.735760+0000","last_peered":"2023-11-28T11:23:11.735760+0000","last_clean":"2023-11-28T11:23:11.735760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:11.746002+0000","last_undegraded":"2023-11-28T11:23:11.746002+0000","last_fullsized":"2023-11-28T11:23:11.746002+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:34:11.467849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2023-11-28T11:23:11.748820+0000","last_change":"2023-11-28T11:23:11.748820+0000","last_active":"2023-11-28T11:23:11.735760+0000","last_peered":"2023-11-28T11:23:11.735760+0000","last_clean":"2023-11-28T11:23:11.735760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:11.748820+0000","last_undegraded":"2023-11-28T11:23:11.748820+0000","last_fullsized":"2023-11-28T11:23:11.748820+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:37:14.105378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2023-11-28T11:23:11.745966+0000","last_change":"2023-11-28T11:23:11.745966+0000","last_active":"2023-11-28T11:23:11.735760+0000","last_peered":"2023-11-28T11:23:11.735760+0000","last_clean":"2023-11-28T11:23:11.735760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:11.745966+0000","last_undegraded":"2023-11-28T11:23:11.745966+0000","last_fullsized":"2023-11-28T11:23:11.745966+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:49:29.966143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2023-11-28T11:23:11.750662+0000","last_change":"2023-11-28T11:23:11.750662+0000","last_active":"2023-11-28T11:23:11.735760+0000","last_peered":"2023-11-28T11:23:11.735760+0000","last_clean":"2023-11-28T11:23:11.735760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:11.750662+0000","last_undegraded":"2023-11-28T11:23:11.750662+0000","last_fullsized":"2023-11-28T11:23:11.750662+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:01:14.097702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2023-11-28T11:23:11.750720+0000","last_change":"2023-11-28T11:23:11.750720+0000","last_active":"2023-11-28T11:23:11.735760+0000","last_peered":"2023-11-28T11:23:11.735760+0000","last_clean":"2023-11-28T11:23:11.735760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:11.750720+0000","last_undegraded":"2023-11-28T11:23:11.750720+0000","last_fullsized":"2023-11-28T11:23:11.750720+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:07:05.101057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":41,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T11:23:08.256169+0000","last_change":"2023-11-28T11:20:51.020016+0000","last_active":"2023-11-28T11:23:08.256169+0000","last_peered":"2023-11-28T11:23:08.256169+0000","last_clean":"2023-11-28T11:23:08.256169+0000","last_became_active":"2023-11-28T11:20:51.019492+0000","last_became_peered":"2023-11-28T11:20:51.019492+0000","last_unstale":"2023-11-28T11:23:08.256169+0000","last_undegraded":"2023-11-28T11:23:08.256169+0000","last_fullsized":"2023-11-28T11:23:08.256169+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:27.884538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2023-11-28T11:23:11.748716+0000","last_change":"2023-11-28T11:23:11.748716+0000","last_active":"2023-11-28T11:23:11.735760+0000","last_peered":"2023-11-28T11:23:11.735760+0000","last_clean":"2023-11-28T11:23:11.735760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:11.748716+0000","last_undegraded":"2023-11-28T11:23:11.748716+0000","last_fullsized":"2023-11-28T11:23:11.748716+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:15:52.711267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":43,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T11:23:08.256412+0000","last_change":"2023-11-28T11:20:50.521908+0000","last_active":"2023-11-28T11:23:08.256412+0000","last_peered":"2023-11-28T11:23:08.256412+0000","last_clean":"2023-11-28T11:23:08.256412+0000","last_became_active":"2023-11-28T11:20:50.521059+0000","last_became_peered":"2023-11-28T11:20:50.521059+0000","last_unstale":"2023-11-28T11:23:08.256412+0000","last_undegraded":"2023-11-28T11:23:08.256412+0000","last_fullsized":"2023-11-28T11:23:08.256412+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:05.302498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2023-11-28T11:23:11.749449+0000","last_change":"2023-11-28T11:23:11.749449+0000","last_active":"2023-11-28T11:23:11.735760+0000","last_peered":"2023-11-28T11:23:11.735760+0000","last_clean":"2023-11-28T11:23:11.735760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:11.749449+0000","last_undegraded":"2023-11-28T11:23:11.749449+0000","last_fullsized":"2023-11-28T11:23:11.749449+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:35:56.057089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":38,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T11:23:08.436383+0000","last_change":"2023-11-28T11:20:50.619004+0000","last_active":"2023-11-28T11:23:08.436383+0000","last_peered":"2023-11-28T11:23:08.436383+0000","last_clean":"2023-11-28T11:23:08.436383+0000","last_became_active":"2023-11-28T11:20:50.618315+0000","last_became_peered":"2023-11-28T11:20:50.618315+0000","last_unstale":"2023-11-28T11:23:08.436383+0000","last_undegraded":"2023-11-28T11:23:08.436383+0000","last_fullsized":"2023-11-28T11:23:08.436383+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:27.884538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2023-11-28T11:23:11.755363+0000","last_change":"2023-11-28T11:23:11.755363+0000","last_active":"2023-11-28T11:23:11.735760+0000","last_peered":"2023-11-28T11:23:11.735760+0000","last_clean":"2023-11-28T11:23:11.735760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:11.755363+0000","last_undegraded":"2023-11-28T11:23:11.755363+0000","last_fullsized":"2023-11-28T11:23:11.755363+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:10:46.424470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":43,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T11:23:08.256405+0000","last_change":"2023-11-28T11:20:50.522582+0000","last_active":"2023-11-28T11:23:08.256405+0000","last_peered":"2023-11-28T11:23:08.256405+0000","last_clean":"2023-11-28T11:23:08.256405+0000","last_became_active":"2023-11-28T11:20:50.522157+0000","last_became_peered":"2023-11-28T11:20:50.522157+0000","last_unstale":"2023-11-28T11:23:08.256405+0000","last_undegraded":"2023-11-28T11:23:08.256405+0000","last_fullsized":"2023-11-28T11:23:08.256405+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:27.884538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":192,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T11:23:09.537521+0000","last_change":"2023-11-28T11:20:43.294845+0000","last_active":"2023-11-28T11:23:09.537521+0000","last_peered":"2023-11-28T11:23:09.537521+0000","last_clean":"2023-11-28T11:23:09.537521+0000","last_became_active":"2023-11-28T11:20:43.294483+0000","last_became_peered":"2023-11-28T11:20:43.294483+0000","last_unstale":"2023-11-28T11:23:09.537521+0000","last_undegraded":"2023-11-28T11:23:09.537521+0000","last_fullsized":"2023-11-28T11:23:09.537521+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:42.270512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:42.270512+0000","last_clean_scrub_stamp":"2023-11-28T11:20:42.270512+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:17:03.376680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2023-11-28T11:23:11.745968+0000","last_change":"2023-11-28T11:23:11.745968+0000","last_active":"2023-11-28T11:23:11.735760+0000","last_peered":"2023-11-28T11:23:11.735760+0000","last_clean":"2023-11-28T11:23:11.735760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:11.745968+0000","last_undegraded":"2023-11-28T11:23:11.745968+0000","last_fullsized":"2023-11-28T11:23:11.745968+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:00:26.159761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":39,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T11:23:09.537535+0000","last_change":"2023-11-28T11:20:50.526865+0000","last_active":"2023-11-28T11:23:09.537535+0000","last_peered":"2023-11-28T11:23:09.537535+0000","last_clean":"2023-11-28T11:23:09.537535+0000","last_became_active":"2023-11-28T11:20:50.526576+0000","last_became_peered":"2023-11-28T11:20:50.526576+0000","last_unstale":"2023-11-28T11:23:09.537535+0000","last_undegraded":"2023-11-28T11:23:09.537535+0000","last_fullsized":"2023-11-28T11:23:09.537535+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:09:18.460112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2023-11-28T11:23:11.749828+0000","last_change":"2023-11-28T11:23:11.749828+0000","last_active":"2023-11-28T11:23:11.735760+0000","last_peered":"2023-11-28T11:23:11.735760+0000","last_clean":"2023-11-28T11:23:11.735760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:11.749828+0000","last_undegraded":"2023-11-28T11:23:11.749828+0000","last_fullsized":"2023-11-28T11:23:11.749828+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:08:59.922691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":39,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T11:23:09.537523+0000","last_change":"2023-11-28T11:20:51.020043+0000","last_active":"2023-11-28T11:23:09.537523+0000","last_peered":"2023-11-28T11:23:09.537523+0000","last_clean":"2023-11-28T11:23:09.537523+0000","last_became_active":"2023-11-28T11:20:51.019702+0000","last_became_peered":"2023-11-28T11:20:51.019702+0000","last_unstale":"2023-11-28T11:23:09.537523+0000","last_undegraded":"2023-11-28T11:23:09.537523+0000","last_fullsized":"2023-11-28T11:23:09.537523+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:26:20.246873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2023-11-28T11:23:11.750348+0000","last_change":"2023-11-28T11:23:11.750348+0000","last_active":"2023-11-28T11:23:11.735760+0000","last_peered":"2023-11-28T11:23:11.735760+0000","last_clean":"2023-11-28T11:23:11.735760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:11.750348+0000","last_undegraded":"2023-11-28T11:23:11.750348+0000","last_fullsized":"2023-11-28T11:23:11.750348+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:37:14.105378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":41,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T11:23:08.256177+0000","last_change":"2023-11-28T11:20:50.524251+0000","last_active":"2023-11-28T11:23:08.256177+0000","last_peered":"2023-11-28T11:23:08.256177+0000","last_clean":"2023-11-28T11:23:08.256177+0000","last_became_active":"2023-11-28T11:20:50.524012+0000","last_became_peered":"2023-11-28T11:20:50.524012+0000","last_unstale":"2023-11-28T11:23:08.256177+0000","last_undegraded":"2023-11-28T11:23:08.256177+0000","last_fullsized":"2023-11-28T11:23:08.256177+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:05.302498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:23:12.743291+0000","last_change":"2023-11-28T11:23:12.743291+0000","last_active":"2023-11-28T11:23:12.743291+0000","last_peered":"2023-11-28T11:23:12.743291+0000","last_clean":"2023-11-28T11:23:12.743291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:12.743291+0000","last_undegraded":"2023-11-28T11:23:12.743291+0000","last_fullsized":"2023-11-28T11:23:12.743291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2023-11-28T11:23:11.748812+0000","last_change":"2023-11-28T11:23:11.748812+0000","last_active":"2023-11-28T11:23:11.735760+0000","last_peered":"2023-11-28T11:23:11.735760+0000","last_clean":"2023-11-28T11:23:11.735760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:23:11.748812+0000","last_undegraded":"2023-11-28T11:23:11.748812+0000","last_fullsized":"2023-11-28T11:23:11.748812+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:37:14.105378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":39,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T11:23:08.620902+0000","last_change":"2023-11-28T11:20:50.524838+0000","last_active":"2023-11-28T11:23:08.620902+0000","last_peered":"2023-11-28T11:23:08.620902+0000","last_clean":"2023-11-28T11:23:08.620902+0000","last_became_active":"2023-11-28T11:20:50.524402+0000","last_became_peered":"2023-11-28T11:20:50.524402+0000","last_unstale":"2023-11-28T11:23:08.620902+0000","last_undegraded":"2023-11-28T11:23:08.620902+0000","last_fullsized":"2023-11-28T11:23:08.620902+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:22.699447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574882,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10148,"kb_used_data":1180,"kb_used_omap":0,"kb_used_meta":8960,"kb_avail":94361692,"statfs":{"total":96636764160,"available":96626372608,"internally_reserved":0,"allocated":1208320,"data_stored":870932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9175040},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:22:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.61199999999999999,"15min":0.62},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.63700000000000001,"15min":0.64500000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78000000000000003,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.82999999999999996}]},{"osd":1,"last update":"Tue Nov 28 11:22:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.83199999999999996,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.624,"5min":0.66800000000000004,"15min":0.67600000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.88300000000000001,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.69599999999999995}]},{"osd":2,"last update":"Tue Nov 28 11:22:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.64600000000000002,"15min":0.65200000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.78500000000000003,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.71299999999999997,"15min":0.71999999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.84899999999999998,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.88500000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:22:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.65300000000000002,"15min":0.66200000000000003},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.78900000000000003,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.69499999999999995,"15min":0.70299999999999996},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.84099999999999997,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.80700000000000005}]},{"osd":4,"last update":"Tue Nov 28 11:22:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.65700000000000003,"15min":0.67000000000000004},"min":{"1min":0.37,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.86699999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.755},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.70299999999999996,"15min":0.70399999999999996},"min":{"1min":0.57799999999999996,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.90900000000000003,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.63200000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:22:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.67900000000000005,"15min":0.68500000000000005},"min":{"1min":0.38,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.89200000000000002,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.71099999999999997,"15min":0.71699999999999997},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0209999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.78900000000000003}]},{"osd":6,"last update":"Tue Nov 28 11:22:46 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.77100000000000002,"15min":0.78200000000000003},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.97199999999999998,"5min":1.633,"15min":1.633},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.748,"15min":0.75900000000000001},"min":{"1min":0.501,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.86699999999999999,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.84999999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574882,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8996,"kb_used_data":604,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362844,"statfs":{"total":96636764160,"available":96627552256,"internally_reserved":0,"allocated":618496,"data_stored":281186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:22:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.63,"15min":0.63800000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.81799999999999995,"5min":1.409,"15min":1.409},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.628,"15min":0.63600000000000001},"min":{"1min":0.42599999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.82099999999999995,"5min":1.244,"15min":1.244},"last":0.64400000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:22:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.622,"15min":0.625},"min":{"1min":0.45400000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.82199999999999995,"5min":1.038,"15min":1.038},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.67200000000000004,"15min":0.68300000000000005},"min":{"1min":0.434,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.78200000000000003,"5min":1.038,"15min":1.038},"last":0.74299999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:22:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.623,"15min":0.627},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.83799999999999997,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.76600000000000001,"15min":0.78400000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.84899999999999998,"5min":1.996,"15min":1.996},"last":0.61399999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:22:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.68899999999999995,"15min":0.69699999999999995},"min":{"1min":0.54600000000000004,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.77000000000000002,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.67400000000000004,"15min":0.68400000000000005},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.76800000000000002,"5min":1.008,"15min":1.008},"last":0.66400000000000003}]},{"osd":4,"last update":"Tue Nov 28 11:22:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.71599999999999997,"15min":0.73499999999999999},"min":{"1min":0.438,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.83499999999999996,"5min":2.048,"15min":2.048},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.71799999999999997,"15min":0.72799999999999998},"min":{"1min":0.44800000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.81899999999999995,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.54700000000000004}]},{"osd":5,"last update":"Tue Nov 28 11:22:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.69199999999999995,"15min":0.70399999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.91800000000000004,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.80600000000000005,"15min":0.82499999999999996},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.90300000000000002,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":0.69499999999999995}]},{"osd":7,"last update":"Tue Nov 28 11:22:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.77400000000000002,"15min":0.78200000000000003},"min":{"1min":0.57899999999999996,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.873,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.79300000000000004,"15min":0.80300000000000005},"min":{"1min":0.54300000000000004,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.98599999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.81999999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574882,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9060,"kb_used_data":604,"kb_used_omap":0,"kb_used_meta":8448,"kb_avail":94362780,"statfs":{"total":96636764160,"available":96627486720,"internally_reserved":0,"allocated":618496,"data_stored":281186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8650752},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:22:45 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.45800000000000002,"15min":0.46700000000000003},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.53000000000000003,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.45400000000000001,"15min":0.46000000000000002},"min":{"1min":0.311,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.57899999999999996,"5min":0.999,"15min":0.999},"last":0.497}]},{"osd":2,"last update":"Tue Nov 28 11:22:45 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.50700000000000001,"15min":0.51700000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.60699999999999998,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.442,"5min":0.499,"15min":0.50900000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.621,"5min":1.002,"15min":1.002},"last":0.38700000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:22:45 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.54600000000000004,"15min":0.55600000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.59199999999999997,"5min":1.198,"15min":1.198},"last":0.441},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.51600000000000001,"15min":0.52200000000000002},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.70899999999999996,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.46000000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:22:45 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.751,"15min":0.75900000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.0169999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.74199999999999999,"15min":0.75600000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.86799999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.58399999999999996}]},{"osd":5,"last update":"Tue Nov 28 11:22:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.75900000000000001,"15min":0.77100000000000002},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.93700000000000006,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.76300000000000001,"15min":0.77300000000000002},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.94599999999999995,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.61399999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:22:45 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.77400000000000002,"15min":0.78400000000000003},"min":{"1min":0.58499999999999996,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.91400000000000003,"5min":1.177,"15min":1.177},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.79800000000000004,"15min":0.80900000000000005},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.018,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.65900000000000003}]},{"osd":7,"last update":"Tue Nov 28 11:22:45 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.753,"15min":0.76100000000000001},"min":{"1min":0.55200000000000005,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.88600000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.79500000000000004,"15min":0.80300000000000005},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.98399999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.72199999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10152,"kb_used_data":1184,"kb_used_omap":0,"kb_used_meta":8960,"kb_avail":94361688,"statfs":{"total":96636764160,"available":96626368512,"internally_reserved":0,"allocated":1212416,"data_stored":871554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9175040},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:22:44 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.44,"15min":0.44600000000000001},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.60399999999999998,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.61499999999999999,"15min":0.64700000000000002},"min":{"1min":0.30099999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.89000000000000001,"5min":3.8420000000000001,"15min":3.8420000000000001},"last":0.50800000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:22:44 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.496,"15min":0.50600000000000001},"min":{"1min":0.308,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.71199999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.65700000000000003,"15min":0.68899999999999995},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.63900000000000001,"5min":3.8940000000000001,"15min":3.8940000000000001},"last":0.435}]},{"osd":3,"last update":"Tue Nov 28 11:22:44 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.61699999999999999,"15min":0.64700000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.65900000000000003,"5min":3.681,"15min":3.681},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.57799999999999996,"15min":0.59299999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.60099999999999998,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.52800000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:22:44 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.90500000000000003,"15min":0.93700000000000006},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.97999999999999998,"5min":3.9790000000000001,"15min":3.9790000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.92700000000000005,"15min":0.96099999999999997},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.98899999999999999,"5min":3.5779999999999998,"15min":3.5779999999999998},"last":0.78500000000000003}]},{"osd":5,"last update":"Tue Nov 28 11:22:44 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.81299999999999994,"15min":0.82699999999999996},"min":{"1min":0.53900000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.0069999999999999,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.94199999999999995,"15min":0.97899999999999998},"min":{"1min":0.50900000000000001,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.97499999999999998,"5min":3.9409999999999998,"15min":3.9409999999999998},"last":0.75900000000000001}]},{"osd":6,"last update":"Tue Nov 28 11:22:44 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.92600000000000005,"15min":0.96099999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.92800000000000005,"5min":3.6160000000000001,"15min":3.6160000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.83099999999999996,"15min":0.84899999999999998},"min":{"1min":0.55900000000000005,"5min":0.501,"15min":0.501},"max":{"1min":0.95699999999999996,"5min":1.363,"15min":1.363},"last":0.85399999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:22:44 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":1.012,"15min":1.0549999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.94399999999999995,"5min":4.0099999999999998,"15min":4.0099999999999998},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.97199999999999998,"15min":1.01},"min":{"1min":0.58899999999999997,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.98999999999999999,"5min":3.653,"15min":3.653},"last":0.70099999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8932,"kb_used_data":604,"kb_used_omap":0,"kb_used_meta":8320,"kb_avail":94362908,"statfs":{"total":96636764160,"available":96627617792,"internally_reserved":0,"allocated":618496,"data_stored":281186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8519680},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:22:48 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.54100000000000004,"15min":0.56799999999999995},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.51200000000000001,"5min":4.1109999999999998,"15min":4.1109999999999998},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.58099999999999996,"15min":0.60399999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.96699999999999997,"5min":3.6230000000000002,"15min":3.6230000000000002},"last":0.432}]},{"osd":1,"last update":"Tue Nov 28 11:22:48 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.66100000000000003,"15min":0.68300000000000005},"min":{"1min":0.38400000000000001,"5min":0.371,"15min":0.371},"max":{"1min":1.008,"5min":3.6640000000000001,"15min":3.6640000000000001},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.66900000000000004,"15min":0.70299999999999996},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.66100000000000003,"5min":4.0830000000000002,"15min":4.0830000000000002},"last":0.54000000000000004}]},{"osd":3,"last update":"Tue Nov 28 11:22:48 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.64100000000000001,"15min":0.66600000000000004},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.89300000000000002,"5min":4.1529999999999996,"15min":4.1529999999999996},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.57899999999999996,"15min":0.58899999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.056,"5min":1.304,"15min":1.304},"last":0.48699999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:22:48 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.80200000000000005,"15min":0.81399999999999995},"min":{"1min":0.54900000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.1910000000000001,"5min":1.649,"15min":1.649},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.89300000000000002,"15min":0.92900000000000005},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.96899999999999997,"5min":4.0060000000000002,"15min":4.0060000000000002},"last":0.75}]},{"osd":5,"last update":"Tue Nov 28 11:22:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.90600000000000003,"15min":0.93999999999999995},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0309999999999999,"5min":4.0529999999999999,"15min":4.0529999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.80800000000000005,"15min":0.81599999999999995},"min":{"1min":0.57199999999999995,"5min":0.497,"15min":0.497},"max":{"1min":1.0780000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.83299999999999996}]},{"osd":6,"last update":"Tue Nov 28 11:22:48 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.83499999999999996,"15min":0.84799999999999998},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.155,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.873},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.84899999999999998,"15min":0.86099999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.161,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.84799999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:22:48 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.80400000000000005,"15min":0.81299999999999994},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.123,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.82499999999999996,"15min":0.83499999999999996},"min":{"1min":0.59699999999999998,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.125,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.73899999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9124,"kb_used_data":604,"kb_used_omap":0,"kb_used_meta":8512,"kb_avail":94362716,"statfs":{"total":96636764160,"available":96627421184,"internally_reserved":0,"allocated":618496,"data_stored":281186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8716288},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:22:45 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.39900000000000002,"15min":0.40400000000000003},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.51200000000000001,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.46000000000000002,"15min":0.46200000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.65500000000000003,"5min":1.157,"15min":1.157},"last":0.47499999999999998}]},{"osd":1,"last update":"Tue Nov 28 11:22:45 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.56999999999999995,"15min":0.59199999999999997},"min":{"1min":0.315,"5min":0.313,"15min":0.313},"max":{"1min":0.54200000000000004,"5min":4.8019999999999996,"15min":4.8019999999999996},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.58099999999999996,"15min":0.60299999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.63800000000000001,"5min":4.7539999999999996,"15min":4.7539999999999996},"last":0.435}]},{"osd":2,"last update":"Tue Nov 28 11:22:45 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.64900000000000002,"15min":0.67300000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.67300000000000004,"5min":4.8419999999999996,"15min":4.8419999999999996},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.64000000000000001,"15min":0.66600000000000004},"min":{"1min":0.35999999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76800000000000002,"5min":4.7919999999999998,"15min":4.7919999999999998},"last":0.57599999999999996}]},{"osd":4,"last update":"Tue Nov 28 11:22:45 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.82999999999999996,"15min":0.85099999999999998},"min":{"1min":0.53100000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.94799999999999995,"5min":4.8869999999999996,"15min":4.8869999999999996},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.88700000000000001,"15min":0.91700000000000004},"min":{"1min":0.54900000000000004,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.85999999999999999,"5min":5.2460000000000004,"15min":5.2460000000000004},"last":0.64500000000000002}]},{"osd":5,"last update":"Tue Nov 28 11:22:45 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.89700000000000002,"15min":0.92700000000000005},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.88900000000000001,"5min":5.0759999999999996,"15min":5.0759999999999996},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.85899999999999999,"15min":0.88500000000000001},"min":{"1min":0.52800000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.99099999999999999,"5min":5.3010000000000002,"15min":5.3010000000000002},"last":0.68000000000000005}]},{"osd":6,"last update":"Tue Nov 28 11:22:45 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.875,"15min":0.89800000000000002},"min":{"1min":0.56200000000000006,"5min":0.501,"15min":0.501},"max":{"1min":1.0269999999999999,"5min":5.3399999999999999,"15min":5.3399999999999999},"last":0.627},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.85999999999999999,"15min":0.88100000000000001},"min":{"1min":0.497,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.93100000000000005,"5min":4.8330000000000002,"15min":4.8330000000000002},"last":0.54500000000000004}]},{"osd":7,"last update":"Tue Nov 28 11:22:45 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.86599999999999999,"15min":0.88900000000000001},"min":{"1min":0.54800000000000004,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.93300000000000005,"5min":4.8600000000000003,"15min":4.8600000000000003},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.749,"5min":0.92400000000000004,"15min":0.95299999999999996},"min":{"1min":0.56000000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.01,"5min":5.1909999999999998,"15min":5.1909999999999998},"last":0.60699999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9252,"kb_used_data":604,"kb_used_omap":0,"kb_used_meta":8640,"kb_avail":94362588,"statfs":{"total":96636764160,"available":96627290112,"internally_reserved":0,"allocated":618496,"data_stored":281186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8847360},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.67600000000000005,"15min":0.68400000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.85799999999999998,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.65600000000000003,"15min":0.65900000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.79100000000000004,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.53500000000000003}]},{"osd":1,"last update":"Tue Nov 28 11:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.64700000000000002,"15min":0.64300000000000002},"min":{"1min":0.36799999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.89400000000000002,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.628},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.69999999999999996,"15min":0.70699999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.85999999999999999,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.58899999999999997}]},{"osd":2,"last update":"Tue Nov 28 11:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.73699999999999999,"15min":0.74399999999999999},"min":{"1min":0.51900000000000002,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.88300000000000001,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.90800000000000003,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.54800000000000004}]},{"osd":3,"last update":"Tue Nov 28 11:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.67100000000000004,"15min":0.67900000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88700000000000001,"5min":1.179,"15min":1.179},"last":0.498},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.77400000000000002,"15min":0.78200000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.97399999999999998,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.51300000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69299999999999995,"15min":0.69199999999999995},"min":{"1min":0.46700000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.91000000000000003,"5min":1.161,"15min":1.161},"last":0.373},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.48499999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.82099999999999995,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.41299999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.72899999999999998,"15min":0.73599999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.92600000000000005,"5min":1.514,"15min":1.514},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.44800000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.70599999999999996,"15min":0.70399999999999996},"min":{"1min":0.51200000000000001,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.94399999999999995,"5min":1.226,"15min":1.226},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.82899999999999996,"15min":0.84099999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0149999999999999,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":0.60299999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8996,"kb_used_data":604,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362844,"statfs":{"total":96636764160,"available":96627552256,"internally_reserved":0,"allocated":618496,"data_stored":281186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:22:46 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.622,"15min":0.622},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84899999999999998,"5min":1.032,"15min":1.032},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.69299999999999995,"15min":0.69699999999999995},"min":{"1min":0.47299999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.877,"5min":1.02,"15min":1.02},"last":0.56000000000000005}]},{"osd":1,"last update":"Tue Nov 28 11:22:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.65600000000000003,"15min":0.65500000000000003},"min":{"1min":0.51200000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.88100000000000001,"5min":1.355,"15min":1.355},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.68700000000000006,"15min":0.69399999999999995},"min":{"1min":0.498,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.86399999999999999,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.71699999999999997}]},{"osd":2,"last update":"Tue Nov 28 11:22:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.68799999999999994,"15min":0.69599999999999995},"min":{"1min":0.44700000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.88500000000000001,"5min":1.198,"15min":1.198},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.72599999999999998,"15min":0.73299999999999998},"min":{"1min":0.50600000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.97199999999999998,"5min":1.387,"15min":1.387},"last":0.58899999999999997}]},{"osd":3,"last update":"Tue Nov 28 11:22:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.74299999999999999,"15min":0.754},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.89600000000000002,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.72099999999999997,"15min":0.72499999999999998},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.63900000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:22:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.71599999999999997,"15min":0.72699999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.90400000000000003,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.74099999999999999,"15min":0.74199999999999999},"min":{"1min":0.48899999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.97999999999999998,"5min":1.198,"15min":1.198},"last":0.65800000000000003}]},{"osd":6,"last update":"Tue Nov 28 11:22:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.499,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0189999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72899999999999998,"15min":0.72999999999999998},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.96099999999999997,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.81399999999999995}]},{"osd":7,"last update":"Tue Nov 28 11:22:46 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.82099999999999995,"15min":0.82899999999999996},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.97499999999999998,"5min":1.371,"15min":1.371},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.82399999999999995,"15min":0.83099999999999996},"min":{"1min":0.54300000000000004,"5min":0.496,"15min":0.496},"max":{"1min":1.0509999999999999,"5min":1.298,"15min":1.298},"last":0.79700000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:23:17.615 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:23:17.728 INFO:tasks.ceph.mds.c.smithi092.stderr:2023-11-28T11:23:17.727+0000 7f54499f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T11:23:17.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.843+0000 7fc41a9ff700 1 Processor -- start 2023-11-28T11:23:17.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.843+0000 7fc41a9ff700 1 -- start start 2023-11-28T11:23:17.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.844+0000 7fc41a9ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4140b8510 0x7fc41406b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:17.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.844+0000 7fc41a9ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc4140cd460 0x7fc41406beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:17.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.845+0000 7fc41a9ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4140b88e0 con 0x7fc41406c3f0 2023-11-28T11:23:17.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.845+0000 7fc41a9ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4140cdbd0 con 0x7fc4140cd460 2023-11-28T11:23:17.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.845+0000 7fc41a9ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc41406c830 con 0x7fc4140b8510 2023-11-28T11:23:17.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.845+0000 7fc4137fe700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc4140cd460 0x7fc41406beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:17.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.845+0000 7fc413fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4140b8510 0x7fc41406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:17.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.845+0000 7fc418f9c700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fc41406c3f0 0x7fc41415cc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.92:42892/0 (socket says 172.21.15.92:42892) 2023-11-28T11:23:17.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.845+0000 7fc418f9c700 1 -- 172.21.15.92:0/113268956 learned_addr learned my addr 172.21.15.92:0/113268956 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:23:17.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.846+0000 7fc413fff700 1 -- 172.21.15.92:0/113268956 >> v2:172.21.15.169:3300/0 conn(0x7fc4140cd460 msgr2=0x7fc41406beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:17.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.846+0000 7fc413fff700 1 --2- 172.21.15.92:0/113268956 >> v2:172.21.15.169:3300/0 conn(0x7fc4140cd460 0x7fc41406beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:17.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.846+0000 7fc413fff700 1 -- 172.21.15.92:0/113268956 >> v1:172.21.15.169:6789/0 conn(0x7fc41406c3f0 legacy=0x7fc41415cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:17.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.846+0000 7fc413fff700 1 -- 172.21.15.92:0/113268956 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4140cd8f0 con 0x7fc4140b8510 2023-11-28T11:23:17.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.846+0000 7fc4137fe700 1 --2- 172.21.15.92:0/113268956 >> v2:172.21.15.169:3300/0 conn(0x7fc4140cd460 0x7fc41406beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:23:17.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.846+0000 7fc413fff700 1 --2- 172.21.15.92:0/113268956 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4140b8510 0x7fc41406b970 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc404009840 tx=0x7fc404009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e576acfc0ee135f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:17.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.846+0000 7fc412ffd700 1 -- 172.21.15.92:0/113268956 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc404011830 con 0x7fc4140b8510 2023-11-28T11:23:17.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.846+0000 7fc412ffd700 1 -- 172.21.15.92:0/113268956 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc404011990 con 0x7fc4140b8510 2023-11-28T11:23:17.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.846+0000 7fc412ffd700 1 -- 172.21.15.92:0/113268956 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc404011d20 con 0x7fc4140b8510 2023-11-28T11:23:17.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.847+0000 7fc41a9ff700 1 -- 172.21.15.92:0/113268956 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4140b8510 msgr2=0x7fc41406b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:17.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.847+0000 7fc41a9ff700 1 --2- 172.21.15.92:0/113268956 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4140b8510 0x7fc41406b970 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc404009840 tx=0x7fc404009b50 comp rx=0 tx=0).stop 2023-11-28T11:23:17.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.847+0000 7fc41a9ff700 1 -- 172.21.15.92:0/113268956 shutdown_connections 2023-11-28T11:23:17.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.847+0000 7fc41a9ff700 1 --2- 172.21.15.92:0/113268956 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4140b8510 0x7fc41406b970 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:17.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.847+0000 7fc41a9ff700 1 --2- 172.21.15.92:0/113268956 >> v2:172.21.15.169:3300/0 conn(0x7fc4140cd460 0x7fc41406beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:17.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.847+0000 7fc41a9ff700 1 -- 172.21.15.92:0/113268956 >> 172.21.15.92:0/113268956 conn(0x7fc4140719a0 msgr2=0x7fc414071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:17.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.847+0000 7fc41a9ff700 1 -- 172.21.15.92:0/113268956 shutdown_connections 2023-11-28T11:23:17.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.847+0000 7fc41a9ff700 1 -- 172.21.15.92:0/113268956 wait complete. 2023-11-28T11:23:17.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.849+0000 7fc41a9ff700 1 Processor -- start 2023-11-28T11:23:17.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.849+0000 7fc41a9ff700 1 -- start start 2023-11-28T11:23:17.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.850+0000 7fc41a9ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc41406c3f0 0x7fc41415b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:17.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.850+0000 7fc41a9ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4140b8510 0x7fc41415a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:17.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.850+0000 7fc41a9ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4140cd460 0x7fc41415aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:17.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.850+0000 7fc41a9ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4141e7e10 con 0x7fc4140b8510 2023-11-28T11:23:17.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.850+0000 7fc41a9ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4141e80f0 con 0x7fc41406c3f0 2023-11-28T11:23:17.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.850+0000 7fc41a9ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4141f22d0 con 0x7fc4140cd460 2023-11-28T11:23:17.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.850+0000 7fc4137fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4140b8510 0x7fc41415a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:17.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.850+0000 7fc4137fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4140b8510 0x7fc41415a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40356/0 (socket says 172.21.15.92:40356) 2023-11-28T11:23:17.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.850+0000 7fc4137fe700 1 -- 172.21.15.92:0/413663644 learned_addr learned my addr 172.21.15.92:0/413663644 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:17.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.850+0000 7fc413fff700 1 --2- 172.21.15.92:0/413663644 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc41406c3f0 0x7fc41415b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:17.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.850+0000 7fc418f9c700 1 --2- 172.21.15.92:0/413663644 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4140cd460 0x7fc41415aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:17.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.850+0000 7fc4137fe700 1 -- 172.21.15.92:0/413663644 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4140cd460 msgr2=0x7fc41415aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:17.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.850+0000 7fc4137fe700 1 --2- 172.21.15.92:0/413663644 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4140cd460 0x7fc41415aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:17.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.850+0000 7fc4137fe700 1 -- 172.21.15.92:0/413663644 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc41406c3f0 msgr2=0x7fc41415b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:17.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.850+0000 7fc4137fe700 1 --2- 172.21.15.92:0/413663644 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc41406c3f0 0x7fc41415b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:17.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.850+0000 7fc4137fe700 1 -- 172.21.15.92:0/413663644 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4080094a0 con 0x7fc4140b8510 2023-11-28T11:23:17.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.850+0000 7fc418f9c700 1 --2- 172.21.15.92:0/413663644 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4140cd460 0x7fc41415aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:23:17.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.851+0000 7fc4137fe700 1 --2- 172.21.15.92:0/413663644 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4140b8510 0x7fc41415a560 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc40800ebe0 tx=0x7fc40800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:17.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.851+0000 7fc410ff9700 1 -- 172.21.15.92:0/413663644 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40800c850 con 0x7fc4140b8510 2023-11-28T11:23:17.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.851+0000 7fc410ff9700 1 -- 172.21.15.92:0/413663644 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc40800c9b0 con 0x7fc4140b8510 2023-11-28T11:23:17.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.851+0000 7fc410ff9700 1 -- 172.21.15.92:0/413663644 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40800cd40 con 0x7fc4140b8510 2023-11-28T11:23:17.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.851+0000 7fc41a9ff700 1 -- 172.21.15.92:0/413663644 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4040094a0 con 0x7fc4140b8510 2023-11-28T11:23:17.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.851+0000 7fc41a9ff700 1 -- 172.21.15.92:0/413663644 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4140cfff0 con 0x7fc4140b8510 2023-11-28T11:23:17.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.852+0000 7fc41a9ff700 1 -- 172.21.15.92:0/413663644 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4140cfd10 con 0x7fc4140b8510 2023-11-28T11:23:17.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.853+0000 7fc410ff9700 1 -- 172.21.15.92:0/413663644 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc408019030 con 0x7fc4140b8510 2023-11-28T11:23:17.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.854+0000 7fc410ff9700 1 --2- 172.21.15.92:0/413663644 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc4000682f0 0x7fc40006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:17.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.854+0000 7fc410ff9700 1 -- 172.21.15.92:0/413663644 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fc408085d50 con 0x7fc4140b8510 2023-11-28T11:23:17.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.854+0000 7fc413fff700 1 --2- 172.21.15.92:0/413663644 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc4000682f0 0x7fc40006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:17.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.855+0000 7fc413fff700 1 --2- 172.21.15.92:0/413663644 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc4000682f0 0x7fc40006a7a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc404009fd0 tx=0x7fc404008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:17.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:17.858+0000 7fc410ff9700 1 -- 172.21.15.92:0/413663644 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc408052470 con 0x7fc4140b8510 2023-11-28T11:23:18.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.044+0000 7fc41a9ff700 1 -- 172.21.15.92:0/413663644 --> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc414067880 con 0x7fc4000682f0 2023-11-28T11:23:18.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.053+0000 7fc410ff9700 1 -- 172.21.15.92:0/413663644 <== mgr.4103 v2:172.21.15.92:6832/115482 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311269 (secure 0 0 0) 0x7fc414067880 con 0x7fc4000682f0 2023-11-28T11:23:18.058 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:23:18.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.057+0000 7fc41a9ff700 1 -- 172.21.15.92:0/413663644 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc4000682f0 msgr2=0x7fc40006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:18.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.057+0000 7fc41a9ff700 1 --2- 172.21.15.92:0/413663644 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc4000682f0 0x7fc40006a7a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc404009fd0 tx=0x7fc404008030 comp rx=0 tx=0).stop 2023-11-28T11:23:18.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.057+0000 7fc41a9ff700 1 -- 172.21.15.92:0/413663644 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4140b8510 msgr2=0x7fc41415a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:18.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.057+0000 7fc41a9ff700 1 --2- 172.21.15.92:0/413663644 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4140b8510 0x7fc41415a560 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc40800ebe0 tx=0x7fc40800eef0 comp rx=0 tx=0).stop 2023-11-28T11:23:18.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.057+0000 7fc41a9ff700 1 -- 172.21.15.92:0/413663644 shutdown_connections 2023-11-28T11:23:18.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.057+0000 7fc41a9ff700 1 --2- 172.21.15.92:0/413663644 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4140cd460 0x7fc41415aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:18.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.057+0000 7fc41a9ff700 1 --2- 172.21.15.92:0/413663644 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc41406c3f0 0x7fc41415b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:18.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.057+0000 7fc41a9ff700 1 --2- 172.21.15.92:0/413663644 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc4000682f0 0x7fc40006a7a0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:18.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.057+0000 7fc41a9ff700 1 --2- 172.21.15.92:0/413663644 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4140b8510 0x7fc41415a560 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:18.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.057+0000 7fc41a9ff700 1 -- 172.21.15.92:0/413663644 >> 172.21.15.92:0/413663644 conn(0x7fc4140719a0 msgr2=0x7fc4140b5e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:18.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.058+0000 7fc41a9ff700 1 -- 172.21.15.92:0/413663644 shutdown_connections 2023-11-28T11:23:18.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.058+0000 7fc41a9ff700 1 -- 172.21.15.92:0/413663644 wait complete. 2023-11-28T11:23:18.061 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-11-28T11:23:18.077 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":127,"stamp":"2023-11-28T11:23:17.889426+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":172,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":76388,"kb_used_data":6436,"kb_used_omap":0,"kb_used_meta":69888,"kb_avail":754898332,"statfs":{"total":773094113280,"available":773015891968,"internally_reserved":0,"allocated":6590464,"data_stored":3657070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":71565312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000931"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:15.027811+0000","last_change":"2023-11-28T11:23:13.273831+0000","last_active":"2023-11-28T11:23:15.027811+0000","last_peered":"2023-11-28T11:23:15.027811+0000","last_clean":"2023-11-28T11:23:15.027811+0000","last_became_active":"2023-11-28T11:23:13.273558+0000","last_became_peered":"2023-11-28T11:23:13.273558+0000","last_unstale":"2023-11-28T11:23:15.027811+0000","last_undegraded":"2023-11-28T11:23:15.027811+0000","last_fullsized":"2023-11-28T11:23:15.027811+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:34:12.475380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:15.028409+0000","last_change":"2023-11-28T11:23:13.284677+0000","last_active":"2023-11-28T11:23:15.028409+0000","last_peered":"2023-11-28T11:23:15.028409+0000","last_clean":"2023-11-28T11:23:15.028409+0000","last_became_active":"2023-11-28T11:23:13.284437+0000","last_became_peered":"2023-11-28T11:23:13.284437+0000","last_unstale":"2023-11-28T11:23:15.028409+0000","last_undegraded":"2023-11-28T11:23:15.028409+0000","last_fullsized":"2023-11-28T11:23:15.028409+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:35:16.468653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:14.949605+0000","last_change":"2023-11-28T11:23:13.273638+0000","last_active":"2023-11-28T11:23:14.949605+0000","last_peered":"2023-11-28T11:23:14.949605+0000","last_clean":"2023-11-28T11:23:14.949605+0000","last_became_active":"2023-11-28T11:23:13.273332+0000","last_became_peered":"2023-11-28T11:23:13.273332+0000","last_unstale":"2023-11-28T11:23:14.949605+0000","last_undegraded":"2023-11-28T11:23:14.949605+0000","last_fullsized":"2023-11-28T11:23:14.949605+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:46:15.322678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:14.256237+0000","last_change":"2023-11-28T11:23:13.281099+0000","last_active":"2023-11-28T11:23:14.256237+0000","last_peered":"2023-11-28T11:23:14.256237+0000","last_clean":"2023-11-28T11:23:14.256237+0000","last_became_active":"2023-11-28T11:23:13.280584+0000","last_became_peered":"2023-11-28T11:23:13.280584+0000","last_unstale":"2023-11-28T11:23:14.256237+0000","last_undegraded":"2023-11-28T11:23:14.256237+0000","last_fullsized":"2023-11-28T11:23:14.256237+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:48:56.822376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:15.027806+0000","last_change":"2023-11-28T11:23:13.273449+0000","last_active":"2023-11-28T11:23:15.027806+0000","last_peered":"2023-11-28T11:23:15.027806+0000","last_clean":"2023-11-28T11:23:15.027806+0000","last_became_active":"2023-11-28T11:23:13.273188+0000","last_became_peered":"2023-11-28T11:23:13.273188+0000","last_unstale":"2023-11-28T11:23:15.027806+0000","last_undegraded":"2023-11-28T11:23:15.027806+0000","last_fullsized":"2023-11-28T11:23:15.027806+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:53:03.738971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:14.527037+0000","last_change":"2023-11-28T11:23:13.277523+0000","last_active":"2023-11-28T11:23:14.527037+0000","last_peered":"2023-11-28T11:23:14.527037+0000","last_clean":"2023-11-28T11:23:14.527037+0000","last_became_active":"2023-11-28T11:23:13.273521+0000","last_became_peered":"2023-11-28T11:23:13.273521+0000","last_unstale":"2023-11-28T11:23:14.527037+0000","last_undegraded":"2023-11-28T11:23:14.527037+0000","last_fullsized":"2023-11-28T11:23:14.527037+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:14:46.302096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:14.256395+0000","last_change":"2023-11-28T11:23:13.267451+0000","last_active":"2023-11-28T11:23:14.256395+0000","last_peered":"2023-11-28T11:23:14.256395+0000","last_clean":"2023-11-28T11:23:14.256395+0000","last_became_active":"2023-11-28T11:23:13.267172+0000","last_became_peered":"2023-11-28T11:23:13.267172+0000","last_unstale":"2023-11-28T11:23:14.256395+0000","last_undegraded":"2023-11-28T11:23:14.256395+0000","last_fullsized":"2023-11-28T11:23:14.256395+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:03:28.900297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:15.028162+0000","last_change":"2023-11-28T11:23:13.260145+0000","last_active":"2023-11-28T11:23:15.028162+0000","last_peered":"2023-11-28T11:23:15.028162+0000","last_clean":"2023-11-28T11:23:15.028162+0000","last_became_active":"2023-11-28T11:23:13.259925+0000","last_became_peered":"2023-11-28T11:23:13.259925+0000","last_unstale":"2023-11-28T11:23:15.028162+0000","last_undegraded":"2023-11-28T11:23:15.028162+0000","last_fullsized":"2023-11-28T11:23:15.028162+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:21:05.495401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:14.256370+0000","last_change":"2023-11-28T11:23:13.268036+0000","last_active":"2023-11-28T11:23:14.256370+0000","last_peered":"2023-11-28T11:23:14.256370+0000","last_clean":"2023-11-28T11:23:14.256370+0000","last_became_active":"2023-11-28T11:23:13.267441+0000","last_became_peered":"2023-11-28T11:23:13.267441+0000","last_unstale":"2023-11-28T11:23:14.256370+0000","last_undegraded":"2023-11-28T11:23:14.256370+0000","last_fullsized":"2023-11-28T11:23:14.256370+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:13:56.834474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:14.254258+0000","last_change":"2023-11-28T11:23:13.283071+0000","last_active":"2023-11-28T11:23:14.254258+0000","last_peered":"2023-11-28T11:23:14.254258+0000","last_clean":"2023-11-28T11:23:14.254258+0000","last_became_active":"2023-11-28T11:23:13.282799+0000","last_became_peered":"2023-11-28T11:23:13.282799+0000","last_unstale":"2023-11-28T11:23:14.254258+0000","last_undegraded":"2023-11-28T11:23:14.254258+0000","last_fullsized":"2023-11-28T11:23:14.254258+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:02:48.594501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:14.254205+0000","last_change":"2023-11-28T11:23:13.276016+0000","last_active":"2023-11-28T11:23:14.254205+0000","last_peered":"2023-11-28T11:23:14.254205+0000","last_clean":"2023-11-28T11:23:14.254205+0000","last_became_active":"2023-11-28T11:23:13.275787+0000","last_became_peered":"2023-11-28T11:23:13.275787+0000","last_unstale":"2023-11-28T11:23:14.254205+0000","last_undegraded":"2023-11-28T11:23:14.254205+0000","last_fullsized":"2023-11-28T11:23:14.254205+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:50:13.769630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:14.256551+0000","last_change":"2023-11-28T11:23:13.271532+0000","last_active":"2023-11-28T11:23:14.256551+0000","last_peered":"2023-11-28T11:23:14.256551+0000","last_clean":"2023-11-28T11:23:14.256551+0000","last_became_active":"2023-11-28T11:23:13.271205+0000","last_became_peered":"2023-11-28T11:23:13.271205+0000","last_unstale":"2023-11-28T11:23:14.256551+0000","last_undegraded":"2023-11-28T11:23:14.256551+0000","last_fullsized":"2023-11-28T11:23:14.256551+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:36:27.873958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:14.709952+0000","last_change":"2023-11-28T11:23:13.278775+0000","last_active":"2023-11-28T11:23:14.709952+0000","last_peered":"2023-11-28T11:23:14.709952+0000","last_clean":"2023-11-28T11:23:14.709952+0000","last_became_active":"2023-11-28T11:23:13.278463+0000","last_became_peered":"2023-11-28T11:23:13.278463+0000","last_unstale":"2023-11-28T11:23:14.709952+0000","last_undegraded":"2023-11-28T11:23:14.709952+0000","last_fullsized":"2023-11-28T11:23:14.709952+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:40:07.315955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:14.256389+0000","last_change":"2023-11-28T11:23:13.275562+0000","last_active":"2023-11-28T11:23:14.256389+0000","last_peered":"2023-11-28T11:23:14.256389+0000","last_clean":"2023-11-28T11:23:14.256389+0000","last_became_active":"2023-11-28T11:23:13.275285+0000","last_became_peered":"2023-11-28T11:23:13.275285+0000","last_unstale":"2023-11-28T11:23:14.256389+0000","last_undegraded":"2023-11-28T11:23:14.256389+0000","last_fullsized":"2023-11-28T11:23:14.256389+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:48:56.822376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:14.256347+0000","last_change":"2023-11-28T11:23:13.278375+0000","last_active":"2023-11-28T11:23:14.256347+0000","last_peered":"2023-11-28T11:23:14.256347+0000","last_clean":"2023-11-28T11:23:14.256347+0000","last_became_active":"2023-11-28T11:23:13.277846+0000","last_became_peered":"2023-11-28T11:23:13.277846+0000","last_unstale":"2023-11-28T11:23:14.256347+0000","last_undegraded":"2023-11-28T11:23:14.256347+0000","last_fullsized":"2023-11-28T11:23:14.256347+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:14:46.302096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:14.526978+0000","last_change":"2023-11-28T11:23:13.269154+0000","last_active":"2023-11-28T11:23:14.526978+0000","last_peered":"2023-11-28T11:23:14.526978+0000","last_clean":"2023-11-28T11:23:14.526978+0000","last_became_active":"2023-11-28T11:23:13.268868+0000","last_became_peered":"2023-11-28T11:23:13.268868+0000","last_unstale":"2023-11-28T11:23:14.526978+0000","last_undegraded":"2023-11-28T11:23:14.526978+0000","last_fullsized":"2023-11-28T11:23:14.526978+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:11:13.461708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:15.028576+0000","last_change":"2023-11-28T11:23:13.266865+0000","last_active":"2023-11-28T11:23:15.028576+0000","last_peered":"2023-11-28T11:23:15.028576+0000","last_clean":"2023-11-28T11:23:15.028576+0000","last_became_active":"2023-11-28T11:23:13.266610+0000","last_became_peered":"2023-11-28T11:23:13.266610+0000","last_unstale":"2023-11-28T11:23:15.028576+0000","last_undegraded":"2023-11-28T11:23:15.028576+0000","last_fullsized":"2023-11-28T11:23:15.028576+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:11:47.333120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:14.949866+0000","last_change":"2023-11-28T11:23:13.269745+0000","last_active":"2023-11-28T11:23:14.949866+0000","last_peered":"2023-11-28T11:23:14.949866+0000","last_clean":"2023-11-28T11:23:14.949866+0000","last_became_active":"2023-11-28T11:23:13.269415+0000","last_became_peered":"2023-11-28T11:23:13.269415+0000","last_unstale":"2023-11-28T11:23:14.949866+0000","last_undegraded":"2023-11-28T11:23:14.949866+0000","last_fullsized":"2023-11-28T11:23:14.949866+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:44:20.532570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:15.027739+0000","last_change":"2023-11-28T11:23:13.281139+0000","last_active":"2023-11-28T11:23:15.027739+0000","last_peered":"2023-11-28T11:23:15.027739+0000","last_clean":"2023-11-28T11:23:15.027739+0000","last_became_active":"2023-11-28T11:23:13.280869+0000","last_became_peered":"2023-11-28T11:23:13.280869+0000","last_unstale":"2023-11-28T11:23:15.027739+0000","last_undegraded":"2023-11-28T11:23:15.027739+0000","last_fullsized":"2023-11-28T11:23:15.027739+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:00:27.167292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:15.028344+0000","last_change":"2023-11-28T11:23:13.280821+0000","last_active":"2023-11-28T11:23:15.028344+0000","last_peered":"2023-11-28T11:23:15.028344+0000","last_clean":"2023-11-28T11:23:15.028344+0000","last_became_active":"2023-11-28T11:23:13.274289+0000","last_became_peered":"2023-11-28T11:23:13.274289+0000","last_unstale":"2023-11-28T11:23:15.028344+0000","last_undegraded":"2023-11-28T11:23:15.028344+0000","last_fullsized":"2023-11-28T11:23:15.028344+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:24:10.722257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:14.949391+0000","last_change":"2023-11-28T11:23:13.267787+0000","last_active":"2023-11-28T11:23:14.949391+0000","last_peered":"2023-11-28T11:23:14.949391+0000","last_clean":"2023-11-28T11:23:14.949391+0000","last_became_active":"2023-11-28T11:23:13.267501+0000","last_became_peered":"2023-11-28T11:23:13.267501+0000","last_unstale":"2023-11-28T11:23:14.949391+0000","last_undegraded":"2023-11-28T11:23:14.949391+0000","last_fullsized":"2023-11-28T11:23:14.949391+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:43:13.374840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:15.028402+0000","last_change":"2023-11-28T11:23:13.282765+0000","last_active":"2023-11-28T11:23:15.028402+0000","last_peered":"2023-11-28T11:23:15.028402+0000","last_clean":"2023-11-28T11:23:15.028402+0000","last_became_active":"2023-11-28T11:23:13.281275+0000","last_became_peered":"2023-11-28T11:23:13.281275+0000","last_unstale":"2023-11-28T11:23:15.028402+0000","last_undegraded":"2023-11-28T11:23:15.028402+0000","last_fullsized":"2023-11-28T11:23:15.028402+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:08:21.174092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:14.949309+0000","last_change":"2023-11-28T11:23:13.263388+0000","last_active":"2023-11-28T11:23:14.949309+0000","last_peered":"2023-11-28T11:23:14.949309+0000","last_clean":"2023-11-28T11:23:14.949309+0000","last_became_active":"2023-11-28T11:23:13.262922+0000","last_became_peered":"2023-11-28T11:23:13.262922+0000","last_unstale":"2023-11-28T11:23:14.949309+0000","last_undegraded":"2023-11-28T11:23:14.949309+0000","last_fullsized":"2023-11-28T11:23:14.949309+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:07:06.108589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:14.709757+0000","last_change":"2023-11-28T11:23:13.282206+0000","last_active":"2023-11-28T11:23:14.709757+0000","last_peered":"2023-11-28T11:23:14.709757+0000","last_clean":"2023-11-28T11:23:14.709757+0000","last_became_active":"2023-11-28T11:23:13.281946+0000","last_became_peered":"2023-11-28T11:23:13.281946+0000","last_unstale":"2023-11-28T11:23:14.709757+0000","last_undegraded":"2023-11-28T11:23:14.709757+0000","last_fullsized":"2023-11-28T11:23:14.709757+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:04:56.700962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:15.027913+0000","last_change":"2023-11-28T11:23:13.275769+0000","last_active":"2023-11-28T11:23:15.027913+0000","last_peered":"2023-11-28T11:23:15.027913+0000","last_clean":"2023-11-28T11:23:15.027913+0000","last_became_active":"2023-11-28T11:23:13.275496+0000","last_became_peered":"2023-11-28T11:23:13.275496+0000","last_unstale":"2023-11-28T11:23:15.027913+0000","last_undegraded":"2023-11-28T11:23:15.027913+0000","last_fullsized":"2023-11-28T11:23:15.027913+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:57:50.843246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:14.256565+0000","last_change":"2023-11-28T11:23:13.281666+0000","last_active":"2023-11-28T11:23:14.256565+0000","last_peered":"2023-11-28T11:23:14.256565+0000","last_clean":"2023-11-28T11:23:14.256565+0000","last_became_active":"2023-11-28T11:23:13.281258+0000","last_became_peered":"2023-11-28T11:23:13.281258+0000","last_unstale":"2023-11-28T11:23:14.256565+0000","last_undegraded":"2023-11-28T11:23:14.256565+0000","last_fullsized":"2023-11-28T11:23:14.256565+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:52.321611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:14.527166+0000","last_change":"2023-11-28T11:23:13.278224+0000","last_active":"2023-11-28T11:23:14.527166+0000","last_peered":"2023-11-28T11:23:14.527166+0000","last_clean":"2023-11-28T11:23:14.527166+0000","last_became_active":"2023-11-28T11:23:13.278028+0000","last_became_peered":"2023-11-28T11:23:13.278028+0000","last_unstale":"2023-11-28T11:23:14.527166+0000","last_undegraded":"2023-11-28T11:23:14.527166+0000","last_fullsized":"2023-11-28T11:23:14.527166+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:19:28.562401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:15.027777+0000","last_change":"2023-11-28T11:23:13.275545+0000","last_active":"2023-11-28T11:23:15.027777+0000","last_peered":"2023-11-28T11:23:15.027777+0000","last_clean":"2023-11-28T11:23:15.027777+0000","last_became_active":"2023-11-28T11:23:13.273487+0000","last_became_peered":"2023-11-28T11:23:13.273487+0000","last_unstale":"2023-11-28T11:23:15.027777+0000","last_undegraded":"2023-11-28T11:23:15.027777+0000","last_fullsized":"2023-11-28T11:23:15.027777+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:52:08.806212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:15.028234+0000","last_change":"2023-11-28T11:23:13.264595+0000","last_active":"2023-11-28T11:23:15.028234+0000","last_peered":"2023-11-28T11:23:15.028234+0000","last_clean":"2023-11-28T11:23:15.028234+0000","last_became_active":"2023-11-28T11:23:13.264354+0000","last_became_peered":"2023-11-28T11:23:13.264354+0000","last_unstale":"2023-11-28T11:23:15.028234+0000","last_undegraded":"2023-11-28T11:23:15.028234+0000","last_fullsized":"2023-11-28T11:23:15.028234+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:21:14.099054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:14.949438+0000","last_change":"2023-11-28T11:23:13.274180+0000","last_active":"2023-11-28T11:23:14.949438+0000","last_peered":"2023-11-28T11:23:14.949438+0000","last_clean":"2023-11-28T11:23:14.949438+0000","last_became_active":"2023-11-28T11:23:13.273481+0000","last_became_peered":"2023-11-28T11:23:13.273481+0000","last_unstale":"2023-11-28T11:23:14.949438+0000","last_undegraded":"2023-11-28T11:23:14.949438+0000","last_fullsized":"2023-11-28T11:23:14.949438+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:42:41.159783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:15.027523+0000","last_change":"2023-11-28T11:23:13.274470+0000","last_active":"2023-11-28T11:23:15.027523+0000","last_peered":"2023-11-28T11:23:15.027523+0000","last_clean":"2023-11-28T11:23:15.027523+0000","last_became_active":"2023-11-28T11:23:13.274225+0000","last_became_peered":"2023-11-28T11:23:13.274225+0000","last_unstale":"2023-11-28T11:23:15.027523+0000","last_undegraded":"2023-11-28T11:23:15.027523+0000","last_fullsized":"2023-11-28T11:23:15.027523+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:50:00.184296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:14.527053+0000","last_change":"2023-11-28T11:23:13.272978+0000","last_active":"2023-11-28T11:23:14.527053+0000","last_peered":"2023-11-28T11:23:14.527053+0000","last_clean":"2023-11-28T11:23:14.527053+0000","last_became_active":"2023-11-28T11:23:13.272741+0000","last_became_peered":"2023-11-28T11:23:13.272741+0000","last_unstale":"2023-11-28T11:23:14.527053+0000","last_undegraded":"2023-11-28T11:23:14.527053+0000","last_fullsized":"2023-11-28T11:23:14.527053+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:14:53.616457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:14.527300+0000","last_change":"2023-11-28T11:23:13.278409+0000","last_active":"2023-11-28T11:23:14.527300+0000","last_peered":"2023-11-28T11:23:14.527300+0000","last_clean":"2023-11-28T11:23:14.527300+0000","last_became_active":"2023-11-28T11:23:13.273357+0000","last_became_peered":"2023-11-28T11:23:13.273357+0000","last_unstale":"2023-11-28T11:23:14.527300+0000","last_undegraded":"2023-11-28T11:23:14.527300+0000","last_fullsized":"2023-11-28T11:23:14.527300+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:55:22.937238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"55'4","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-28T11:23:13.292739+0000","last_change":"2023-11-28T11:23:12.776355+0000","last_active":"2023-11-28T11:23:13.292739+0000","last_peered":"2023-11-28T11:23:13.292739+0000","last_clean":"2023-11-28T11:23:13.292739+0000","last_became_active":"2023-11-28T11:23:12.776119+0000","last_became_peered":"2023-11-28T11:23:12.776119+0000","last_unstale":"2023-11-28T11:23:13.292739+0000","last_undegraded":"2023-11-28T11:23:13.292739+0000","last_fullsized":"2023-11-28T11:23:13.292739+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:18:12.361339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:14.254537+0000","last_change":"2023-11-28T11:23:13.286660+0000","last_active":"2023-11-28T11:23:14.254537+0000","last_peered":"2023-11-28T11:23:14.254537+0000","last_clean":"2023-11-28T11:23:14.254537+0000","last_became_active":"2023-11-28T11:23:13.286320+0000","last_became_peered":"2023-11-28T11:23:13.286320+0000","last_unstale":"2023-11-28T11:23:14.254537+0000","last_undegraded":"2023-11-28T11:23:14.254537+0000","last_fullsized":"2023-11-28T11:23:14.254537+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:56:08.242546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"55'1","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-28T11:23:13.291854+0000","last_change":"2023-11-28T11:23:12.776680+0000","last_active":"2023-11-28T11:23:13.291854+0000","last_peered":"2023-11-28T11:23:13.291854+0000","last_clean":"2023-11-28T11:23:13.291854+0000","last_became_active":"2023-11-28T11:23:12.775633+0000","last_became_peered":"2023-11-28T11:23:12.775633+0000","last_unstale":"2023-11-28T11:23:13.291854+0000","last_undegraded":"2023-11-28T11:23:13.291854+0000","last_fullsized":"2023-11-28T11:23:13.291854+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:36:26.866427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:14.527335+0000","last_change":"2023-11-28T11:23:13.289834+0000","last_active":"2023-11-28T11:23:14.527335+0000","last_peered":"2023-11-28T11:23:14.527335+0000","last_clean":"2023-11-28T11:23:14.527335+0000","last_became_active":"2023-11-28T11:23:13.279311+0000","last_became_peered":"2023-11-28T11:23:13.279311+0000","last_unstale":"2023-11-28T11:23:14.527335+0000","last_undegraded":"2023-11-28T11:23:14.527335+0000","last_fullsized":"2023-11-28T11:23:14.527335+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:48:56.822376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"55'2","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-28T11:23:13.293130+0000","last_change":"2023-11-28T11:23:12.778882+0000","last_active":"2023-11-28T11:23:13.293130+0000","last_peered":"2023-11-28T11:23:13.293130+0000","last_clean":"2023-11-28T11:23:13.293130+0000","last_became_active":"2023-11-28T11:23:12.776530+0000","last_became_peered":"2023-11-28T11:23:12.776530+0000","last_unstale":"2023-11-28T11:23:13.293130+0000","last_undegraded":"2023-11-28T11:23:13.293130+0000","last_fullsized":"2023-11-28T11:23:13.293130+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:11:36.215172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:14.527279+0000","last_change":"2023-11-28T11:23:13.289682+0000","last_active":"2023-11-28T11:23:14.527279+0000","last_peered":"2023-11-28T11:23:14.527279+0000","last_clean":"2023-11-28T11:23:14.527279+0000","last_became_active":"2023-11-28T11:23:13.289383+0000","last_became_peered":"2023-11-28T11:23:13.289383+0000","last_unstale":"2023-11-28T11:23:14.527279+0000","last_undegraded":"2023-11-28T11:23:14.527279+0000","last_fullsized":"2023-11-28T11:23:14.527279+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:09:59.359823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"55'2","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-28T11:23:13.285561+0000","last_change":"2023-11-28T11:23:12.772541+0000","last_active":"2023-11-28T11:23:13.285561+0000","last_peered":"2023-11-28T11:23:13.285561+0000","last_clean":"2023-11-28T11:23:13.285561+0000","last_became_active":"2023-11-28T11:23:12.772007+0000","last_became_peered":"2023-11-28T11:23:12.772007+0000","last_unstale":"2023-11-28T11:23:13.285561+0000","last_undegraded":"2023-11-28T11:23:13.285561+0000","last_fullsized":"2023-11-28T11:23:13.285561+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:42:40.152252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:15.028321+0000","last_change":"2023-11-28T11:23:13.270292+0000","last_active":"2023-11-28T11:23:15.028321+0000","last_peered":"2023-11-28T11:23:15.028321+0000","last_clean":"2023-11-28T11:23:15.028321+0000","last_became_active":"2023-11-28T11:23:13.270039+0000","last_became_peered":"2023-11-28T11:23:13.270039+0000","last_unstale":"2023-11-28T11:23:15.028321+0000","last_undegraded":"2023-11-28T11:23:15.028321+0000","last_fullsized":"2023-11-28T11:23:15.028321+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:19:09.833950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"55'1","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-28T11:23:13.287759+0000","last_change":"2023-11-28T11:23:12.781541+0000","last_active":"2023-11-28T11:23:13.287759+0000","last_peered":"2023-11-28T11:23:13.287759+0000","last_clean":"2023-11-28T11:23:13.287759+0000","last_became_active":"2023-11-28T11:23:12.781232+0000","last_became_peered":"2023-11-28T11:23:12.781232+0000","last_unstale":"2023-11-28T11:23:13.287759+0000","last_undegraded":"2023-11-28T11:23:13.287759+0000","last_fullsized":"2023-11-28T11:23:13.287759+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:07:05.101057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:15.028351+0000","last_change":"2023-11-28T11:23:13.276585+0000","last_active":"2023-11-28T11:23:15.028351+0000","last_peered":"2023-11-28T11:23:15.028351+0000","last_clean":"2023-11-28T11:23:15.028351+0000","last_became_active":"2023-11-28T11:23:13.276411+0000","last_became_peered":"2023-11-28T11:23:13.276411+0000","last_unstale":"2023-11-28T11:23:15.028351+0000","last_undegraded":"2023-11-28T11:23:15.028351+0000","last_fullsized":"2023-11-28T11:23:15.028351+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:34:49.912487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"55'1","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-28T11:23:13.289182+0000","last_change":"2023-11-28T11:23:12.775319+0000","last_active":"2023-11-28T11:23:13.289182+0000","last_peered":"2023-11-28T11:23:13.289182+0000","last_clean":"2023-11-28T11:23:13.289182+0000","last_became_active":"2023-11-28T11:23:12.775028+0000","last_became_peered":"2023-11-28T11:23:12.775028+0000","last_unstale":"2023-11-28T11:23:13.289182+0000","last_undegraded":"2023-11-28T11:23:13.289182+0000","last_fullsized":"2023-11-28T11:23:13.289182+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:46:14.315147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:14.709903+0000","last_change":"2023-11-28T11:23:13.273494+0000","last_active":"2023-11-28T11:23:14.709903+0000","last_peered":"2023-11-28T11:23:14.709903+0000","last_clean":"2023-11-28T11:23:14.709903+0000","last_became_active":"2023-11-28T11:23:13.273182+0000","last_became_peered":"2023-11-28T11:23:13.273182+0000","last_unstale":"2023-11-28T11:23:14.709903+0000","last_undegraded":"2023-11-28T11:23:14.709903+0000","last_fullsized":"2023-11-28T11:23:14.709903+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:49:30.973674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-28T11:23:13.266069+0000","last_change":"2023-11-28T11:23:12.774635+0000","last_active":"2023-11-28T11:23:13.266069+0000","last_peered":"2023-11-28T11:23:13.266069+0000","last_clean":"2023-11-28T11:23:13.266069+0000","last_became_active":"2023-11-28T11:23:12.774379+0000","last_became_peered":"2023-11-28T11:23:12.774379+0000","last_unstale":"2023-11-28T11:23:13.266069+0000","last_undegraded":"2023-11-28T11:23:13.266069+0000","last_fullsized":"2023-11-28T11:23:13.266069+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:50:37.417549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:15.028249+0000","last_change":"2023-11-28T11:23:13.278499+0000","last_active":"2023-11-28T11:23:15.028249+0000","last_peered":"2023-11-28T11:23:15.028249+0000","last_clean":"2023-11-28T11:23:15.028249+0000","last_became_active":"2023-11-28T11:23:13.272662+0000","last_became_peered":"2023-11-28T11:23:13.272662+0000","last_unstale":"2023-11-28T11:23:15.028249+0000","last_undegraded":"2023-11-28T11:23:15.028249+0000","last_fullsized":"2023-11-28T11:23:15.028249+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:40:17.623208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-28T11:23:13.255156+0000","last_change":"2023-11-28T11:23:12.783252+0000","last_active":"2023-11-28T11:23:13.255156+0000","last_peered":"2023-11-28T11:23:13.255156+0000","last_clean":"2023-11-28T11:23:13.255156+0000","last_became_active":"2023-11-28T11:23:12.783063+0000","last_became_peered":"2023-11-28T11:23:12.783063+0000","last_unstale":"2023-11-28T11:23:13.255156+0000","last_undegraded":"2023-11-28T11:23:13.255156+0000","last_fullsized":"2023-11-28T11:23:13.255156+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:18:12.361339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:14.256680+0000","last_change":"2023-11-28T11:23:13.277488+0000","last_active":"2023-11-28T11:23:14.256680+0000","last_peered":"2023-11-28T11:23:14.256680+0000","last_clean":"2023-11-28T11:23:14.256680+0000","last_became_active":"2023-11-28T11:23:13.276057+0000","last_became_peered":"2023-11-28T11:23:13.276057+0000","last_unstale":"2023-11-28T11:23:14.256680+0000","last_undegraded":"2023-11-28T11:23:14.256680+0000","last_fullsized":"2023-11-28T11:23:14.256680+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:36:27.873958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-28T11:23:13.257554+0000","last_change":"2023-11-28T11:23:12.779266+0000","last_active":"2023-11-28T11:23:13.257554+0000","last_peered":"2023-11-28T11:23:13.257554+0000","last_clean":"2023-11-28T11:23:13.257554+0000","last_became_active":"2023-11-28T11:23:12.778941+0000","last_became_peered":"2023-11-28T11:23:12.778941+0000","last_unstale":"2023-11-28T11:23:13.257554+0000","last_undegraded":"2023-11-28T11:23:13.257554+0000","last_fullsized":"2023-11-28T11:23:13.257554+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:19:39.112941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:14.949911+0000","last_change":"2023-11-28T11:23:13.278420+0000","last_active":"2023-11-28T11:23:14.949911+0000","last_peered":"2023-11-28T11:23:14.949911+0000","last_clean":"2023-11-28T11:23:14.949911+0000","last_became_active":"2023-11-28T11:23:13.278078+0000","last_became_peered":"2023-11-28T11:23:13.278078+0000","last_unstale":"2023-11-28T11:23:14.949911+0000","last_undegraded":"2023-11-28T11:23:14.949911+0000","last_fullsized":"2023-11-28T11:23:14.949911+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:50:38.425080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-28T11:23:13.255006+0000","last_change":"2023-11-28T11:23:12.781298+0000","last_active":"2023-11-28T11:23:13.255006+0000","last_peered":"2023-11-28T11:23:13.255006+0000","last_clean":"2023-11-28T11:23:13.255006+0000","last_became_active":"2023-11-28T11:23:12.781051+0000","last_became_peered":"2023-11-28T11:23:12.781051+0000","last_unstale":"2023-11-28T11:23:13.255006+0000","last_undegraded":"2023-11-28T11:23:13.255006+0000","last_fullsized":"2023-11-28T11:23:13.255006+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:13:55.826943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:14.253870+0000","last_change":"2023-11-28T11:23:13.274725+0000","last_active":"2023-11-28T11:23:14.253870+0000","last_peered":"2023-11-28T11:23:14.253870+0000","last_clean":"2023-11-28T11:23:14.253870+0000","last_became_active":"2023-11-28T11:23:13.274384+0000","last_became_peered":"2023-11-28T11:23:13.274384+0000","last_unstale":"2023-11-28T11:23:14.253870+0000","last_undegraded":"2023-11-28T11:23:14.253870+0000","last_fullsized":"2023-11-28T11:23:14.253870+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:29:43.202838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"55'3","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-28T11:23:13.291730+0000","last_change":"2023-11-28T11:23:12.779375+0000","last_active":"2023-11-28T11:23:13.291730+0000","last_peered":"2023-11-28T11:23:13.291730+0000","last_clean":"2023-11-28T11:23:13.291730+0000","last_became_active":"2023-11-28T11:23:12.779068+0000","last_became_peered":"2023-11-28T11:23:12.779068+0000","last_unstale":"2023-11-28T11:23:13.291730+0000","last_undegraded":"2023-11-28T11:23:13.291730+0000","last_fullsized":"2023-11-28T11:23:13.291730+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:37:08.575542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:14.949847+0000","last_change":"2023-11-28T11:23:13.280123+0000","last_active":"2023-11-28T11:23:14.949847+0000","last_peered":"2023-11-28T11:23:14.949847+0000","last_clean":"2023-11-28T11:23:14.949847+0000","last_became_active":"2023-11-28T11:23:13.278297+0000","last_became_peered":"2023-11-28T11:23:13.278297+0000","last_unstale":"2023-11-28T11:23:14.949847+0000","last_undegraded":"2023-11-28T11:23:14.949847+0000","last_fullsized":"2023-11-28T11:23:14.949847+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:15:53.718798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"55'6","reported_seq":17,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-28T11:23:13.298252+0000","last_change":"2023-11-28T11:23:12.777410+0000","last_active":"2023-11-28T11:23:13.298252+0000","last_peered":"2023-11-28T11:23:13.298252+0000","last_clean":"2023-11-28T11:23:13.298252+0000","last_became_active":"2023-11-28T11:23:12.777082+0000","last_became_peered":"2023-11-28T11:23:12.777082+0000","last_unstale":"2023-11-28T11:23:13.298252+0000","last_undegraded":"2023-11-28T11:23:13.298252+0000","last_fullsized":"2023-11-28T11:23:13.298252+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:11:36.215172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:14.255905+0000","last_change":"2023-11-28T11:23:13.282517+0000","last_active":"2023-11-28T11:23:14.255905+0000","last_peered":"2023-11-28T11:23:14.255905+0000","last_clean":"2023-11-28T11:23:14.255905+0000","last_became_active":"2023-11-28T11:23:13.276614+0000","last_became_peered":"2023-11-28T11:23:13.276614+0000","last_unstale":"2023-11-28T11:23:14.255905+0000","last_undegraded":"2023-11-28T11:23:14.255905+0000","last_fullsized":"2023-11-28T11:23:14.255905+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:15:11.944995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"55'2","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-28T11:23:13.290575+0000","last_change":"2023-11-28T11:23:12.779359+0000","last_active":"2023-11-28T11:23:13.290575+0000","last_peered":"2023-11-28T11:23:13.290575+0000","last_clean":"2023-11-28T11:23:13.290575+0000","last_became_active":"2023-11-28T11:23:12.779097+0000","last_became_peered":"2023-11-28T11:23:12.779097+0000","last_unstale":"2023-11-28T11:23:13.290575+0000","last_undegraded":"2023-11-28T11:23:13.290575+0000","last_fullsized":"2023-11-28T11:23:13.290575+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:15:10.937464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:14.949558+0000","last_change":"2023-11-28T11:23:13.270583+0000","last_active":"2023-11-28T11:23:14.949558+0000","last_peered":"2023-11-28T11:23:14.949558+0000","last_clean":"2023-11-28T11:23:14.949558+0000","last_became_active":"2023-11-28T11:23:13.270306+0000","last_became_peered":"2023-11-28T11:23:13.270306+0000","last_unstale":"2023-11-28T11:23:14.949558+0000","last_undegraded":"2023-11-28T11:23:14.949558+0000","last_fullsized":"2023-11-28T11:23:14.949558+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:49:03.905087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-28T11:23:13.263822+0000","last_change":"2023-11-28T11:23:12.775481+0000","last_active":"2023-11-28T11:23:13.263822+0000","last_peered":"2023-11-28T11:23:13.263822+0000","last_clean":"2023-11-28T11:23:13.263822+0000","last_became_active":"2023-11-28T11:23:12.775181+0000","last_became_peered":"2023-11-28T11:23:12.775181+0000","last_unstale":"2023-11-28T11:23:13.263822+0000","last_undegraded":"2023-11-28T11:23:13.263822+0000","last_fullsized":"2023-11-28T11:23:13.263822+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:52:07.798680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:14.255796+0000","last_change":"2023-11-28T11:23:13.270689+0000","last_active":"2023-11-28T11:23:14.255796+0000","last_peered":"2023-11-28T11:23:14.255796+0000","last_clean":"2023-11-28T11:23:14.255796+0000","last_became_active":"2023-11-28T11:23:13.270400+0000","last_became_peered":"2023-11-28T11:23:13.270400+0000","last_unstale":"2023-11-28T11:23:14.255796+0000","last_undegraded":"2023-11-28T11:23:14.255796+0000","last_fullsized":"2023-11-28T11:23:14.255796+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:09:00.930222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-28T11:23:13.266064+0000","last_change":"2023-11-28T11:23:12.771534+0000","last_active":"2023-11-28T11:23:13.266064+0000","last_peered":"2023-11-28T11:23:13.266064+0000","last_clean":"2023-11-28T11:23:13.266064+0000","last_became_active":"2023-11-28T11:23:12.771299+0000","last_became_peered":"2023-11-28T11:23:12.771299+0000","last_unstale":"2023-11-28T11:23:13.266064+0000","last_undegraded":"2023-11-28T11:23:13.266064+0000","last_fullsized":"2023-11-28T11:23:13.266064+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:30:37.672629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:14.949509+0000","last_change":"2023-11-28T11:23:13.272827+0000","last_active":"2023-11-28T11:23:14.949509+0000","last_peered":"2023-11-28T11:23:14.949509+0000","last_clean":"2023-11-28T11:23:14.949509+0000","last_became_active":"2023-11-28T11:23:13.272405+0000","last_became_peered":"2023-11-28T11:23:13.272405+0000","last_unstale":"2023-11-28T11:23:14.949509+0000","last_undegraded":"2023-11-28T11:23:14.949509+0000","last_fullsized":"2023-11-28T11:23:14.949509+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:11:37.222704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"55'8","reported_seq":19,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-28T11:23:13.291760+0000","last_change":"2023-11-28T11:23:12.777246+0000","last_active":"2023-11-28T11:23:13.291760+0000","last_peered":"2023-11-28T11:23:13.291760+0000","last_clean":"2023-11-28T11:23:13.291760+0000","last_became_active":"2023-11-28T11:23:12.777071+0000","last_became_peered":"2023-11-28T11:23:12.777071+0000","last_unstale":"2023-11-28T11:23:13.291760+0000","last_undegraded":"2023-11-28T11:23:13.291760+0000","last_fullsized":"2023-11-28T11:23:13.291760+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:04:55.693431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:14.256882+0000","last_change":"2023-11-28T11:23:13.276293+0000","last_active":"2023-11-28T11:23:14.256882+0000","last_peered":"2023-11-28T11:23:14.256882+0000","last_clean":"2023-11-28T11:23:14.256882+0000","last_became_active":"2023-11-28T11:23:13.275985+0000","last_became_peered":"2023-11-28T11:23:13.275985+0000","last_unstale":"2023-11-28T11:23:14.256882+0000","last_undegraded":"2023-11-28T11:23:14.256882+0000","last_fullsized":"2023-11-28T11:23:14.256882+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:03:28.900297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"55'2","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-28T11:23:13.289328+0000","last_change":"2023-11-28T11:23:12.772891+0000","last_active":"2023-11-28T11:23:13.289328+0000","last_peered":"2023-11-28T11:23:13.289328+0000","last_clean":"2023-11-28T11:23:13.289328+0000","last_became_active":"2023-11-28T11:23:12.772593+0000","last_became_peered":"2023-11-28T11:23:12.772593+0000","last_unstale":"2023-11-28T11:23:13.289328+0000","last_undegraded":"2023-11-28T11:23:13.289328+0000","last_fullsized":"2023-11-28T11:23:13.289328+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:40:06.308424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:14.257001+0000","last_change":"2023-11-28T11:23:13.274620+0000","last_active":"2023-11-28T11:23:14.257001+0000","last_peered":"2023-11-28T11:23:14.257001+0000","last_clean":"2023-11-28T11:23:14.257001+0000","last_became_active":"2023-11-28T11:23:13.274370+0000","last_became_peered":"2023-11-28T11:23:13.274370+0000","last_unstale":"2023-11-28T11:23:14.257001+0000","last_undegraded":"2023-11-28T11:23:14.257001+0000","last_fullsized":"2023-11-28T11:23:14.257001+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:14:46.302096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-28T11:23:13.266356+0000","last_change":"2023-11-28T11:23:12.774864+0000","last_active":"2023-11-28T11:23:13.266356+0000","last_peered":"2023-11-28T11:23:13.266356+0000","last_clean":"2023-11-28T11:23:13.266356+0000","last_became_active":"2023-11-28T11:23:12.774541+0000","last_became_peered":"2023-11-28T11:23:12.774541+0000","last_unstale":"2023-11-28T11:23:13.266356+0000","last_undegraded":"2023-11-28T11:23:13.266356+0000","last_fullsized":"2023-11-28T11:23:13.266356+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:43:12.367309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:14.253915+0000","last_change":"2023-11-28T11:23:13.271747+0000","last_active":"2023-11-28T11:23:14.253915+0000","last_peered":"2023-11-28T11:23:14.253915+0000","last_clean":"2023-11-28T11:23:14.253915+0000","last_became_active":"2023-11-28T11:23:13.271363+0000","last_became_peered":"2023-11-28T11:23:13.271363+0000","last_unstale":"2023-11-28T11:23:14.253915+0000","last_undegraded":"2023-11-28T11:23:14.253915+0000","last_fullsized":"2023-11-28T11:23:14.253915+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:07:50.410643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-28T11:23:13.264977+0000","last_change":"2023-11-28T11:23:12.774295+0000","last_active":"2023-11-28T11:23:13.264977+0000","last_peered":"2023-11-28T11:23:13.264977+0000","last_clean":"2023-11-28T11:23:13.264977+0000","last_became_active":"2023-11-28T11:23:12.773943+0000","last_became_peered":"2023-11-28T11:23:12.773943+0000","last_unstale":"2023-11-28T11:23:13.264977+0000","last_undegraded":"2023-11-28T11:23:13.264977+0000","last_fullsized":"2023-11-28T11:23:13.264977+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:53:02.731439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:14.527382+0000","last_change":"2023-11-28T11:23:13.289662+0000","last_active":"2023-11-28T11:23:14.527382+0000","last_peered":"2023-11-28T11:23:14.527382+0000","last_clean":"2023-11-28T11:23:14.527382+0000","last_became_active":"2023-11-28T11:23:13.289312+0000","last_became_peered":"2023-11-28T11:23:13.289312+0000","last_unstale":"2023-11-28T11:23:14.527382+0000","last_undegraded":"2023-11-28T11:23:14.527382+0000","last_fullsized":"2023-11-28T11:23:14.527382+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:52.321611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-28T11:23:13.264660+0000","last_change":"2023-11-28T11:23:12.774564+0000","last_active":"2023-11-28T11:23:13.264660+0000","last_peered":"2023-11-28T11:23:13.264660+0000","last_clean":"2023-11-28T11:23:13.264660+0000","last_became_active":"2023-11-28T11:23:12.774275+0000","last_became_peered":"2023-11-28T11:23:12.774275+0000","last_unstale":"2023-11-28T11:23:13.264660+0000","last_undegraded":"2023-11-28T11:23:13.264660+0000","last_fullsized":"2023-11-28T11:23:13.264660+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:49:02.897556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:15.028441+0000","last_change":"2023-11-28T11:23:13.282068+0000","last_active":"2023-11-28T11:23:15.028441+0000","last_peered":"2023-11-28T11:23:15.028441+0000","last_clean":"2023-11-28T11:23:15.028441+0000","last_became_active":"2023-11-28T11:23:13.281589+0000","last_became_peered":"2023-11-28T11:23:13.281589+0000","last_unstale":"2023-11-28T11:23:15.028441+0000","last_undegraded":"2023-11-28T11:23:15.028441+0000","last_fullsized":"2023-11-28T11:23:15.028441+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:23:22.853707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-28T11:23:13.267828+0000","last_change":"2023-11-28T11:23:12.765073+0000","last_active":"2023-11-28T11:23:13.267828+0000","last_peered":"2023-11-28T11:23:13.267828+0000","last_clean":"2023-11-28T11:23:13.267828+0000","last_became_active":"2023-11-28T11:23:12.764814+0000","last_became_peered":"2023-11-28T11:23:12.764814+0000","last_unstale":"2023-11-28T11:23:13.267828+0000","last_undegraded":"2023-11-28T11:23:13.267828+0000","last_fullsized":"2023-11-28T11:23:13.267828+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:34:11.467849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:14.949567+0000","last_change":"2023-11-28T11:23:13.275153+0000","last_active":"2023-11-28T11:23:14.949567+0000","last_peered":"2023-11-28T11:23:14.949567+0000","last_clean":"2023-11-28T11:23:14.949567+0000","last_became_active":"2023-11-28T11:23:13.274834+0000","last_became_peered":"2023-11-28T11:23:13.274834+0000","last_unstale":"2023-11-28T11:23:14.949567+0000","last_undegraded":"2023-11-28T11:23:14.949567+0000","last_fullsized":"2023-11-28T11:23:14.949567+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:18:13.368870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-28T11:23:13.257449+0000","last_change":"2023-11-28T11:23:12.776666+0000","last_active":"2023-11-28T11:23:13.257449+0000","last_peered":"2023-11-28T11:23:13.257449+0000","last_clean":"2023-11-28T11:23:13.257449+0000","last_became_active":"2023-11-28T11:23:12.776405+0000","last_became_peered":"2023-11-28T11:23:12.776405+0000","last_unstale":"2023-11-28T11:23:13.257449+0000","last_undegraded":"2023-11-28T11:23:13.257449+0000","last_fullsized":"2023-11-28T11:23:13.257449+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:37:14.105378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:14.253602+0000","last_change":"2023-11-28T11:23:13.278119+0000","last_active":"2023-11-28T11:23:14.253602+0000","last_peered":"2023-11-28T11:23:14.253602+0000","last_clean":"2023-11-28T11:23:14.253602+0000","last_became_active":"2023-11-28T11:23:13.277970+0000","last_became_peered":"2023-11-28T11:23:13.277970+0000","last_unstale":"2023-11-28T11:23:14.253602+0000","last_undegraded":"2023-11-28T11:23:14.253602+0000","last_fullsized":"2023-11-28T11:23:14.253602+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:43:17.336243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-28T11:23:13.266776+0000","last_change":"2023-11-28T11:23:12.776563+0000","last_active":"2023-11-28T11:23:13.266776+0000","last_peered":"2023-11-28T11:23:13.266776+0000","last_clean":"2023-11-28T11:23:13.266776+0000","last_became_active":"2023-11-28T11:23:12.776298+0000","last_became_peered":"2023-11-28T11:23:12.776298+0000","last_unstale":"2023-11-28T11:23:13.266776+0000","last_undegraded":"2023-11-28T11:23:13.266776+0000","last_fullsized":"2023-11-28T11:23:13.266776+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:49:29.966143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:15.028497+0000","last_change":"2023-11-28T11:23:13.274526+0000","last_active":"2023-11-28T11:23:15.028497+0000","last_peered":"2023-11-28T11:23:15.028497+0000","last_clean":"2023-11-28T11:23:15.028497+0000","last_became_active":"2023-11-28T11:23:13.272364+0000","last_became_peered":"2023-11-28T11:23:13.272364+0000","last_unstale":"2023-11-28T11:23:15.028497+0000","last_undegraded":"2023-11-28T11:23:15.028497+0000","last_fullsized":"2023-11-28T11:23:15.028497+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:37:20.810455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-28T11:23:13.254176+0000","last_change":"2023-11-28T11:23:12.774380+0000","last_active":"2023-11-28T11:23:13.254176+0000","last_peered":"2023-11-28T11:23:13.254176+0000","last_clean":"2023-11-28T11:23:13.254176+0000","last_became_active":"2023-11-28T11:23:12.774090+0000","last_became_peered":"2023-11-28T11:23:12.774090+0000","last_unstale":"2023-11-28T11:23:13.254176+0000","last_undegraded":"2023-11-28T11:23:13.254176+0000","last_fullsized":"2023-11-28T11:23:13.254176+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:01:14.097702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:14.256988+0000","last_change":"2023-11-28T11:23:13.282312+0000","last_active":"2023-11-28T11:23:14.256988+0000","last_peered":"2023-11-28T11:23:14.256988+0000","last_clean":"2023-11-28T11:23:14.256988+0000","last_became_active":"2023-11-28T11:23:13.280017+0000","last_became_peered":"2023-11-28T11:23:13.280017+0000","last_unstale":"2023-11-28T11:23:14.256988+0000","last_undegraded":"2023-11-28T11:23:14.256988+0000","last_fullsized":"2023-11-28T11:23:14.256988+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:13:56.834474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"55'2","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-28T11:23:13.294048+0000","last_change":"2023-11-28T11:23:12.781395+0000","last_active":"2023-11-28T11:23:13.294048+0000","last_peered":"2023-11-28T11:23:13.294048+0000","last_clean":"2023-11-28T11:23:13.294048+0000","last_became_active":"2023-11-28T11:23:12.781016+0000","last_became_peered":"2023-11-28T11:23:12.781016+0000","last_unstale":"2023-11-28T11:23:13.294048+0000","last_undegraded":"2023-11-28T11:23:13.294048+0000","last_fullsized":"2023-11-28T11:23:13.294048+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:07:05.101057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":44,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-28T11:23:13.266729+0000","last_change":"2023-11-28T11:20:51.020016+0000","last_active":"2023-11-28T11:23:13.266729+0000","last_peered":"2023-11-28T11:23:13.266729+0000","last_clean":"2023-11-28T11:23:13.266729+0000","last_became_active":"2023-11-28T11:20:51.019492+0000","last_became_peered":"2023-11-28T11:20:51.019492+0000","last_unstale":"2023-11-28T11:23:13.266729+0000","last_undegraded":"2023-11-28T11:23:13.266729+0000","last_fullsized":"2023-11-28T11:23:13.266729+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:27.884538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:14.527012+0000","last_change":"2023-11-28T11:23:13.272372+0000","last_active":"2023-11-28T11:23:14.527012+0000","last_peered":"2023-11-28T11:23:14.527012+0000","last_clean":"2023-11-28T11:23:14.527012+0000","last_became_active":"2023-11-28T11:23:13.272069+0000","last_became_peered":"2023-11-28T11:23:13.272069+0000","last_unstale":"2023-11-28T11:23:14.527012+0000","last_undegraded":"2023-11-28T11:23:14.527012+0000","last_fullsized":"2023-11-28T11:23:14.527012+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:34:50.895049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-28T11:23:13.266186+0000","last_change":"2023-11-28T11:23:12.776157+0000","last_active":"2023-11-28T11:23:13.266186+0000","last_peered":"2023-11-28T11:23:13.266186+0000","last_clean":"2023-11-28T11:23:13.266186+0000","last_became_active":"2023-11-28T11:23:12.775046+0000","last_became_peered":"2023-11-28T11:23:12.775046+0000","last_unstale":"2023-11-28T11:23:13.266186+0000","last_undegraded":"2023-11-28T11:23:13.266186+0000","last_fullsized":"2023-11-28T11:23:13.266186+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:15:52.711267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":46,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-28T11:23:13.266322+0000","last_change":"2023-11-28T11:20:50.521908+0000","last_active":"2023-11-28T11:23:13.266322+0000","last_peered":"2023-11-28T11:23:13.266322+0000","last_clean":"2023-11-28T11:23:13.266322+0000","last_became_active":"2023-11-28T11:20:50.521059+0000","last_became_peered":"2023-11-28T11:20:50.521059+0000","last_unstale":"2023-11-28T11:23:13.266322+0000","last_undegraded":"2023-11-28T11:23:13.266322+0000","last_fullsized":"2023-11-28T11:23:13.266322+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:05.302498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:14.256021+0000","last_change":"2023-11-28T11:23:13.284131+0000","last_active":"2023-11-28T11:23:14.256021+0000","last_peered":"2023-11-28T11:23:14.256021+0000","last_clean":"2023-11-28T11:23:14.256021+0000","last_became_active":"2023-11-28T11:23:13.282657+0000","last_became_peered":"2023-11-28T11:23:13.282657+0000","last_unstale":"2023-11-28T11:23:14.256021+0000","last_undegraded":"2023-11-28T11:23:14.256021+0000","last_fullsized":"2023-11-28T11:23:14.256021+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:09:00.930222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"55'2","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-28T11:23:13.290716+0000","last_change":"2023-11-28T11:23:12.774528+0000","last_active":"2023-11-28T11:23:13.290716+0000","last_peered":"2023-11-28T11:23:13.290716+0000","last_clean":"2023-11-28T11:23:13.290716+0000","last_became_active":"2023-11-28T11:23:12.772337+0000","last_became_peered":"2023-11-28T11:23:12.772337+0000","last_unstale":"2023-11-28T11:23:13.290716+0000","last_undegraded":"2023-11-28T11:23:13.290716+0000","last_fullsized":"2023-11-28T11:23:13.290716+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:35:56.057089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":41,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-28T11:23:13.263864+0000","last_change":"2023-11-28T11:20:50.619004+0000","last_active":"2023-11-28T11:23:13.263864+0000","last_peered":"2023-11-28T11:23:13.263864+0000","last_clean":"2023-11-28T11:23:13.263864+0000","last_became_active":"2023-11-28T11:20:50.618315+0000","last_became_peered":"2023-11-28T11:20:50.618315+0000","last_unstale":"2023-11-28T11:23:13.263864+0000","last_undegraded":"2023-11-28T11:23:13.263864+0000","last_fullsized":"2023-11-28T11:23:13.263864+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:27.884538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:15.028274+0000","last_change":"2023-11-28T11:23:13.275962+0000","last_active":"2023-11-28T11:23:15.028274+0000","last_peered":"2023-11-28T11:23:15.028274+0000","last_clean":"2023-11-28T11:23:15.028274+0000","last_became_active":"2023-11-28T11:23:13.275229+0000","last_became_peered":"2023-11-28T11:23:13.275229+0000","last_unstale":"2023-11-28T11:23:15.028274+0000","last_undegraded":"2023-11-28T11:23:15.028274+0000","last_fullsized":"2023-11-28T11:23:15.028274+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:17:20.178905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-28T11:23:13.263754+0000","last_change":"2023-11-28T11:23:12.777525+0000","last_active":"2023-11-28T11:23:13.263754+0000","last_peered":"2023-11-28T11:23:13.263754+0000","last_clean":"2023-11-28T11:23:13.263754+0000","last_became_active":"2023-11-28T11:23:12.777144+0000","last_became_peered":"2023-11-28T11:23:12.777144+0000","last_unstale":"2023-11-28T11:23:13.263754+0000","last_undegraded":"2023-11-28T11:23:13.263754+0000","last_fullsized":"2023-11-28T11:23:13.263754+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:10:46.424470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":46,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-28T11:23:13.264188+0000","last_change":"2023-11-28T11:20:50.522582+0000","last_active":"2023-11-28T11:23:13.264188+0000","last_peered":"2023-11-28T11:23:13.264188+0000","last_clean":"2023-11-28T11:23:13.264188+0000","last_became_active":"2023-11-28T11:20:50.522157+0000","last_became_peered":"2023-11-28T11:20:50.522157+0000","last_unstale":"2023-11-28T11:23:13.264188+0000","last_undegraded":"2023-11-28T11:23:13.264188+0000","last_fullsized":"2023-11-28T11:23:13.264188+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:27.884538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":195,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-28T11:23:13.253971+0000","last_change":"2023-11-28T11:20:43.294845+0000","last_active":"2023-11-28T11:23:13.253971+0000","last_peered":"2023-11-28T11:23:13.253971+0000","last_clean":"2023-11-28T11:23:13.253971+0000","last_became_active":"2023-11-28T11:20:43.294483+0000","last_became_peered":"2023-11-28T11:20:43.294483+0000","last_unstale":"2023-11-28T11:23:13.253971+0000","last_undegraded":"2023-11-28T11:23:13.253971+0000","last_fullsized":"2023-11-28T11:23:13.253971+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:42.270512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:42.270512+0000","last_clean_scrub_stamp":"2023-11-28T11:20:42.270512+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:17:03.376680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:14.527188+0000","last_change":"2023-11-28T11:23:13.290291+0000","last_active":"2023-11-28T11:23:14.527188+0000","last_peered":"2023-11-28T11:23:14.527188+0000","last_clean":"2023-11-28T11:23:14.527188+0000","last_became_active":"2023-11-28T11:23:13.290002+0000","last_became_peered":"2023-11-28T11:23:13.290002+0000","last_unstale":"2023-11-28T11:23:14.527188+0000","last_undegraded":"2023-11-28T11:23:14.527188+0000","last_fullsized":"2023-11-28T11:23:14.527188+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:01:28.172800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-28T11:23:13.272805+0000","last_change":"2023-11-28T11:23:12.777743+0000","last_active":"2023-11-28T11:23:13.272805+0000","last_peered":"2023-11-28T11:23:13.272805+0000","last_clean":"2023-11-28T11:23:13.272805+0000","last_became_active":"2023-11-28T11:23:12.777447+0000","last_became_peered":"2023-11-28T11:23:12.777447+0000","last_unstale":"2023-11-28T11:23:13.272805+0000","last_undegraded":"2023-11-28T11:23:13.272805+0000","last_fullsized":"2023-11-28T11:23:13.272805+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:00:26.159761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":42,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-28T11:23:13.254077+0000","last_change":"2023-11-28T11:20:50.526865+0000","last_active":"2023-11-28T11:23:13.254077+0000","last_peered":"2023-11-28T11:23:13.254077+0000","last_clean":"2023-11-28T11:23:13.254077+0000","last_became_active":"2023-11-28T11:20:50.526576+0000","last_became_peered":"2023-11-28T11:20:50.526576+0000","last_unstale":"2023-11-28T11:23:13.254077+0000","last_undegraded":"2023-11-28T11:23:13.254077+0000","last_fullsized":"2023-11-28T11:23:13.254077+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:09:18.460112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:14.949850+0000","last_change":"2023-11-28T11:23:13.274145+0000","last_active":"2023-11-28T11:23:14.949850+0000","last_peered":"2023-11-28T11:23:14.949850+0000","last_clean":"2023-11-28T11:23:14.949850+0000","last_became_active":"2023-11-28T11:23:13.273860+0000","last_became_peered":"2023-11-28T11:23:13.273860+0000","last_unstale":"2023-11-28T11:23:14.949850+0000","last_undegraded":"2023-11-28T11:23:14.949850+0000","last_fullsized":"2023-11-28T11:23:14.949850+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:30:38.680160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"55'1","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-28T11:23:13.290500+0000","last_change":"2023-11-28T11:23:12.781941+0000","last_active":"2023-11-28T11:23:13.290500+0000","last_peered":"2023-11-28T11:23:13.290500+0000","last_clean":"2023-11-28T11:23:13.290500+0000","last_became_active":"2023-11-28T11:23:12.781688+0000","last_became_peered":"2023-11-28T11:23:12.781688+0000","last_unstale":"2023-11-28T11:23:13.290500+0000","last_undegraded":"2023-11-28T11:23:13.290500+0000","last_fullsized":"2023-11-28T11:23:13.290500+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:08:59.922691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":42,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-28T11:23:13.257081+0000","last_change":"2023-11-28T11:20:51.020043+0000","last_active":"2023-11-28T11:23:13.257081+0000","last_peered":"2023-11-28T11:23:13.257081+0000","last_clean":"2023-11-28T11:23:13.257081+0000","last_became_active":"2023-11-28T11:20:51.019702+0000","last_became_peered":"2023-11-28T11:20:51.019702+0000","last_unstale":"2023-11-28T11:23:13.257081+0000","last_undegraded":"2023-11-28T11:23:13.257081+0000","last_fullsized":"2023-11-28T11:23:13.257081+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:26:20.246873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:14.949700+0000","last_change":"2023-11-28T11:23:13.278631+0000","last_active":"2023-11-28T11:23:14.949700+0000","last_peered":"2023-11-28T11:23:14.949700+0000","last_clean":"2023-11-28T11:23:14.949700+0000","last_became_active":"2023-11-28T11:23:13.277923+0000","last_became_peered":"2023-11-28T11:23:13.277923+0000","last_unstale":"2023-11-28T11:23:14.949700+0000","last_undegraded":"2023-11-28T11:23:14.949700+0000","last_fullsized":"2023-11-28T11:23:14.949700+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:37:15.112909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-28T11:23:13.262533+0000","last_change":"2023-11-28T11:23:12.783667+0000","last_active":"2023-11-28T11:23:13.262533+0000","last_peered":"2023-11-28T11:23:13.262533+0000","last_clean":"2023-11-28T11:23:13.262533+0000","last_became_active":"2023-11-28T11:23:12.783197+0000","last_became_peered":"2023-11-28T11:23:12.783197+0000","last_unstale":"2023-11-28T11:23:13.262533+0000","last_undegraded":"2023-11-28T11:23:13.262533+0000","last_fullsized":"2023-11-28T11:23:13.262533+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:37:14.105378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":44,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-28T11:23:13.270293+0000","last_change":"2023-11-28T11:20:50.524251+0000","last_active":"2023-11-28T11:23:13.270293+0000","last_peered":"2023-11-28T11:23:13.270293+0000","last_clean":"2023-11-28T11:23:13.270293+0000","last_became_active":"2023-11-28T11:20:50.524012+0000","last_became_peered":"2023-11-28T11:20:50.524012+0000","last_unstale":"2023-11-28T11:23:13.270293+0000","last_undegraded":"2023-11-28T11:23:13.270293+0000","last_fullsized":"2023-11-28T11:23:13.270293+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:05.302498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-28T11:23:15.028491+0000","last_change":"2023-11-28T11:23:13.262031+0000","last_active":"2023-11-28T11:23:15.028491+0000","last_peered":"2023-11-28T11:23:15.028491+0000","last_clean":"2023-11-28T11:23:15.028491+0000","last_became_active":"2023-11-28T11:23:13.261783+0000","last_became_peered":"2023-11-28T11:23:13.261783+0000","last_unstale":"2023-11-28T11:23:15.028491+0000","last_undegraded":"2023-11-28T11:23:15.028491+0000","last_fullsized":"2023-11-28T11:23:15.028491+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:12.743291+0000","last_clean_scrub_stamp":"2023-11-28T11:23:12.743291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:18:19.985151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"55'1","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-28T11:23:13.285689+0000","last_change":"2023-11-28T11:23:12.774718+0000","last_active":"2023-11-28T11:23:13.285689+0000","last_peered":"2023-11-28T11:23:13.285689+0000","last_clean":"2023-11-28T11:23:13.285689+0000","last_became_active":"2023-11-28T11:23:12.773640+0000","last_became_peered":"2023-11-28T11:23:12.773640+0000","last_unstale":"2023-11-28T11:23:13.285689+0000","last_undegraded":"2023-11-28T11:23:13.285689+0000","last_fullsized":"2023-11-28T11:23:13.285689+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:23:11.735760+0000","last_clean_scrub_stamp":"2023-11-28T11:23:11.735760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:37:14.105378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":42,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-28T11:23:13.265257+0000","last_change":"2023-11-28T11:20:50.524838+0000","last_active":"2023-11-28T11:23:13.265257+0000","last_peered":"2023-11-28T11:23:13.265257+0000","last_clean":"2023-11-28T11:23:13.265257+0000","last_became_active":"2023-11-28T11:20:50.524402+0000","last_became_peered":"2023-11-28T11:20:50.524402+0000","last_unstale":"2023-11-28T11:23:13.265257+0000","last_undegraded":"2023-11-28T11:23:13.265257+0000","last_fullsized":"2023-11-28T11:23:13.265257+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:22.699447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574883,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10396,"kb_used_data":1236,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361444,"statfs":{"total":96636764160,"available":96626118656,"internally_reserved":0,"allocated":1265664,"data_stored":898943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:22:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.61199999999999999,"15min":0.62},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.63700000000000001,"15min":0.64500000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78000000000000003,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.70399999999999996}]},{"osd":1,"last update":"Tue Nov 28 11:22:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.83199999999999996,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.66800000000000004,"15min":0.67600000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.88300000000000001,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.623}]},{"osd":2,"last update":"Tue Nov 28 11:22:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.64600000000000002,"15min":0.65200000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.78500000000000003,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.71299999999999997,"15min":0.71999999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.84899999999999998,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.73999999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:22:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.65300000000000002,"15min":0.66200000000000003},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.78900000000000003,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.69499999999999995,"15min":0.70299999999999996},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.84099999999999997,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.78200000000000003}]},{"osd":4,"last update":"Tue Nov 28 11:22:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.65700000000000003,"15min":0.67000000000000004},"min":{"1min":0.37,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.86699999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.70299999999999996,"15min":0.70399999999999996},"min":{"1min":0.57799999999999996,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.90900000000000003,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.72199999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:22:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.67900000000000005,"15min":0.68500000000000005},"min":{"1min":0.38,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.89200000000000002,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.71099999999999997,"15min":0.71699999999999997},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0209999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.54400000000000004}]},{"osd":6,"last update":"Tue Nov 28 11:22:46 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.77100000000000002,"15min":0.78200000000000003},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.97199999999999998,"5min":1.633,"15min":1.633},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.748,"15min":0.75900000000000001},"min":{"1min":0.501,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.86699999999999999,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.76300000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574883,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9172,"kb_used_data":652,"kb_used_omap":0,"kb_used_meta":8512,"kb_avail":94362668,"statfs":{"total":96636764160,"available":96627372032,"internally_reserved":0,"allocated":667648,"data_stored":309037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8716288},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:22:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.63,"15min":0.63800000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.81799999999999995,"5min":1.409,"15min":1.409},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.628,"15min":0.63600000000000001},"min":{"1min":0.42599999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.82099999999999995,"5min":1.244,"15min":1.244},"last":0.83799999999999997}]},{"osd":1,"last update":"Tue Nov 28 11:22:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.622,"15min":0.625},"min":{"1min":0.45400000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.82199999999999995,"5min":1.038,"15min":1.038},"last":0.625},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.67200000000000004,"15min":0.68300000000000005},"min":{"1min":0.434,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.78200000000000003,"5min":1.038,"15min":1.038},"last":0.70499999999999996}]},{"osd":2,"last update":"Tue Nov 28 11:22:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.623,"15min":0.627},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.83799999999999997,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.76600000000000001,"15min":0.78400000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.84899999999999998,"5min":1.996,"15min":1.996},"last":0.68300000000000005}]},{"osd":3,"last update":"Tue Nov 28 11:22:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.68899999999999995,"15min":0.69699999999999995},"min":{"1min":0.54600000000000004,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.77000000000000002,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.746},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.67400000000000004,"15min":0.68400000000000005},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.76800000000000002,"5min":1.008,"15min":1.008},"last":0.72699999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:22:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.71599999999999997,"15min":0.73499999999999999},"min":{"1min":0.438,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.83499999999999996,"5min":2.048,"15min":2.048},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.71799999999999997,"15min":0.72799999999999998},"min":{"1min":0.44800000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.81899999999999995,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.79900000000000004}]},{"osd":5,"last update":"Tue Nov 28 11:22:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.69199999999999995,"15min":0.70399999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.91800000000000004,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.80600000000000005,"15min":0.82499999999999996},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.90300000000000002,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":0.81499999999999995}]},{"osd":7,"last update":"Tue Nov 28 11:22:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.77400000000000002,"15min":0.78200000000000003},"min":{"1min":0.57899999999999996,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.873,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.79300000000000004,"15min":0.80300000000000005},"min":{"1min":0.54300000000000004,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.98599999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.85499999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574883,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9240,"kb_used_data":656,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94362600,"statfs":{"total":96636764160,"available":96627302400,"internally_reserved":0,"allocated":671744,"data_stored":309127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:22:45 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.45800000000000002,"15min":0.46700000000000003},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.53000000000000003,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.45400000000000001,"15min":0.46000000000000002},"min":{"1min":0.311,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.57899999999999996,"5min":0.999,"15min":0.999},"last":0.42399999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:22:45 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.50700000000000001,"15min":0.51700000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.60699999999999998,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.442,"5min":0.499,"15min":0.50900000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.621,"5min":1.002,"15min":1.002},"last":0.48199999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:22:45 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.54600000000000004,"15min":0.55600000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.59199999999999997,"5min":1.198,"15min":1.198},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.51600000000000001,"15min":0.52200000000000002},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.70899999999999996,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.45700000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:22:45 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.751,"15min":0.75900000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.0169999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.74199999999999999,"15min":0.75600000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.86799999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.77900000000000003}]},{"osd":5,"last update":"Tue Nov 28 11:22:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.75900000000000001,"15min":0.77100000000000002},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.93700000000000006,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.76300000000000001,"15min":0.77300000000000002},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.94599999999999995,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.81100000000000005}]},{"osd":6,"last update":"Tue Nov 28 11:22:45 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.77400000000000002,"15min":0.78400000000000003},"min":{"1min":0.58499999999999996,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.91400000000000003,"5min":1.177,"15min":1.177},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.79800000000000004,"15min":0.80900000000000005},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.018,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.79000000000000004}]},{"osd":7,"last update":"Tue Nov 28 11:22:45 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.753,"15min":0.76100000000000001},"min":{"1min":0.55200000000000005,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.88600000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.746,"5min":0.79500000000000004,"15min":0.80300000000000005},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.98399999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.746}]}]},{"osd":0,"up_from":13,"seq":55834574883,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10408,"kb_used_data":1248,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361432,"statfs":{"total":96636764160,"available":96626106368,"internally_reserved":0,"allocated":1277952,"data_stored":901519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:22:44 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.44,"15min":0.44600000000000001},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.60399999999999998,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.61499999999999999,"15min":0.64700000000000002},"min":{"1min":0.30099999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.89000000000000001,"5min":3.8420000000000001,"15min":3.8420000000000001},"last":0.56299999999999994}]},{"osd":2,"last update":"Tue Nov 28 11:22:44 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.496,"15min":0.50600000000000001},"min":{"1min":0.308,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.71199999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.65700000000000003,"15min":0.68899999999999995},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.63900000000000001,"5min":3.8940000000000001,"15min":3.8940000000000001},"last":0.70699999999999996}]},{"osd":3,"last update":"Tue Nov 28 11:22:44 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.61699999999999999,"15min":0.64700000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.65900000000000003,"5min":3.681,"15min":3.681},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.57799999999999996,"15min":0.59299999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.60099999999999998,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.72699999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:22:44 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.90500000000000003,"15min":0.93700000000000006},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.97999999999999998,"5min":3.9790000000000001,"15min":3.9790000000000001},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.92700000000000005,"15min":0.96099999999999997},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.98899999999999999,"5min":3.5779999999999998,"15min":3.5779999999999998},"last":0.86799999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:22:44 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.81299999999999994,"15min":0.82699999999999996},"min":{"1min":0.53900000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.0069999999999999,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.94199999999999995,"15min":0.97899999999999998},"min":{"1min":0.50900000000000001,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.97499999999999998,"5min":3.9409999999999998,"15min":3.9409999999999998},"last":0.88700000000000001}]},{"osd":6,"last update":"Tue Nov 28 11:22:44 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.92600000000000005,"15min":0.96099999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.92800000000000005,"5min":3.6160000000000001,"15min":3.6160000000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.83099999999999996,"15min":0.84899999999999998},"min":{"1min":0.55900000000000005,"5min":0.501,"15min":0.501},"max":{"1min":0.95699999999999996,"5min":1.363,"15min":1.363},"last":0.81000000000000005}]},{"osd":7,"last update":"Tue Nov 28 11:22:44 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":1.012,"15min":1.0549999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.94399999999999995,"5min":4.0099999999999998,"15min":4.0099999999999998},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.97199999999999998,"15min":1.01},"min":{"1min":0.58899999999999997,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.98999999999999999,"5min":3.653,"15min":3.653},"last":0.997}]}]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9116,"kb_used_data":660,"kb_used_omap":0,"kb_used_meta":8448,"kb_avail":94362724,"statfs":{"total":96636764160,"available":96627429376,"internally_reserved":0,"allocated":675840,"data_stored":309093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8650752},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:22:48 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.54100000000000004,"15min":0.56799999999999995},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.51200000000000001,"5min":4.1109999999999998,"15min":4.1109999999999998},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.58099999999999996,"15min":0.60399999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.96699999999999997,"5min":3.6230000000000002,"15min":3.6230000000000002},"last":0.32300000000000001}]},{"osd":1,"last update":"Tue Nov 28 11:22:48 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.66100000000000003,"15min":0.68300000000000005},"min":{"1min":0.38400000000000001,"5min":0.371,"15min":0.371},"max":{"1min":1.008,"5min":3.6640000000000001,"15min":3.6640000000000001},"last":0.442},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.66900000000000004,"15min":0.70299999999999996},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.66100000000000003,"5min":4.0830000000000002,"15min":4.0830000000000002},"last":0.52700000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:22:48 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.64100000000000001,"15min":0.66600000000000004},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.89300000000000002,"5min":4.1529999999999996,"15min":4.1529999999999996},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.57899999999999996,"15min":0.58899999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.056,"5min":1.304,"15min":1.304},"last":0.49199999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:22:48 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.80200000000000005,"15min":0.81399999999999995},"min":{"1min":0.54900000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.1910000000000001,"5min":1.649,"15min":1.649},"last":0.87},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.89300000000000002,"15min":0.92900000000000005},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.96899999999999997,"5min":4.0060000000000002,"15min":4.0060000000000002},"last":0.88500000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:22:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.90600000000000003,"15min":0.93999999999999995},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0309999999999999,"5min":4.0529999999999999,"15min":4.0529999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.80800000000000005,"15min":0.81599999999999995},"min":{"1min":0.57199999999999995,"5min":0.497,"15min":0.497},"max":{"1min":1.0780000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.91000000000000003}]},{"osd":6,"last update":"Tue Nov 28 11:22:48 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.83499999999999996,"15min":0.84799999999999998},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.155,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.84899999999999998,"15min":0.86099999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.161,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.90000000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:22:48 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.80400000000000005,"15min":0.81299999999999994},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.123,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.82499999999999996,"15min":0.83499999999999996},"min":{"1min":0.59699999999999998,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.125,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.92000000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9372,"kb_used_data":660,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362468,"statfs":{"total":96636764160,"available":96627167232,"internally_reserved":0,"allocated":675840,"data_stored":309093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:22:45 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.39900000000000002,"15min":0.40400000000000003},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.51200000000000001,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.503},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.46000000000000002,"15min":0.46200000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.65500000000000003,"5min":1.157,"15min":1.157},"last":0.36699999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:22:45 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.56999999999999995,"15min":0.59199999999999997},"min":{"1min":0.315,"5min":0.313,"15min":0.313},"max":{"1min":0.54200000000000004,"5min":4.8019999999999996,"15min":4.8019999999999996},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.58099999999999996,"15min":0.60299999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.63800000000000001,"5min":4.7539999999999996,"15min":4.7539999999999996},"last":0.45800000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:22:45 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.64900000000000002,"15min":0.67300000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.67300000000000004,"5min":4.8419999999999996,"15min":4.8419999999999996},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.64000000000000001,"15min":0.66600000000000004},"min":{"1min":0.35999999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76800000000000002,"5min":4.7919999999999998,"15min":4.7919999999999998},"last":0.55100000000000005}]},{"osd":4,"last update":"Tue Nov 28 11:22:45 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.82999999999999996,"15min":0.85099999999999998},"min":{"1min":0.53100000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.94799999999999995,"5min":4.8869999999999996,"15min":4.8869999999999996},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.88700000000000001,"15min":0.91700000000000004},"min":{"1min":0.54900000000000004,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.85999999999999999,"5min":5.2460000000000004,"15min":5.2460000000000004},"last":0.82799999999999996}]},{"osd":5,"last update":"Tue Nov 28 11:22:45 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.89700000000000002,"15min":0.92700000000000005},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.88900000000000001,"5min":5.0759999999999996,"15min":5.0759999999999996},"last":0.875},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.85899999999999999,"15min":0.88500000000000001},"min":{"1min":0.52800000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.99099999999999999,"5min":5.3010000000000002,"15min":5.3010000000000002},"last":0.94299999999999995}]},{"osd":6,"last update":"Tue Nov 28 11:22:45 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.875,"15min":0.89800000000000002},"min":{"1min":0.56200000000000006,"5min":0.501,"15min":0.501},"max":{"1min":1.0269999999999999,"5min":5.3399999999999999,"15min":5.3399999999999999},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.85999999999999999,"15min":0.88100000000000001},"min":{"1min":0.497,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.93100000000000005,"5min":4.8330000000000002,"15min":4.8330000000000002},"last":0.88800000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:22:45 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.86599999999999999,"15min":0.88900000000000001},"min":{"1min":0.54800000000000004,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.93300000000000005,"5min":4.8600000000000003,"15min":4.8600000000000003},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.749,"5min":0.92400000000000004,"15min":0.95299999999999996},"min":{"1min":0.56000000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.01,"5min":5.1909999999999998,"15min":5.1909999999999998},"last":0.76300000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574882,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9440,"kb_used_data":664,"kb_used_omap":0,"kb_used_meta":8768,"kb_avail":94362400,"statfs":{"total":96636764160,"available":96627097600,"internally_reserved":0,"allocated":679936,"data_stored":310585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8978432},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.67600000000000005,"15min":0.68400000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.85799999999999998,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.65600000000000003,"15min":0.65900000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.79100000000000004,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.63600000000000001}]},{"osd":1,"last update":"Tue Nov 28 11:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.64700000000000002,"15min":0.64300000000000002},"min":{"1min":0.36799999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.89400000000000002,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.69999999999999996,"15min":0.70699999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.85999999999999999,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.50800000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.73699999999999999,"15min":0.74399999999999999},"min":{"1min":0.51900000000000002,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.88300000000000001,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.90800000000000003,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.58799999999999997}]},{"osd":3,"last update":"Tue Nov 28 11:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.67100000000000004,"15min":0.67900000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88700000000000001,"5min":1.179,"15min":1.179},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.77400000000000002,"15min":0.78200000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.97399999999999998,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.60199999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69299999999999995,"15min":0.69199999999999995},"min":{"1min":0.46700000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.91000000000000003,"5min":1.161,"15min":1.161},"last":0.753},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.48499999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.82099999999999995,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.80900000000000005}]},{"osd":6,"last update":"Tue Nov 28 11:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.72899999999999998,"15min":0.73599999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.92600000000000005,"5min":1.514,"15min":1.514},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.68500000000000005}]},{"osd":7,"last update":"Tue Nov 28 11:22:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.70599999999999996,"15min":0.70399999999999996},"min":{"1min":0.51200000000000001,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.94399999999999995,"5min":1.226,"15min":1.226},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.82899999999999996,"15min":0.84099999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0149999999999999,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":0.81599999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574882,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9244,"kb_used_data":660,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94362596,"statfs":{"total":96636764160,"available":96627298304,"internally_reserved":0,"allocated":675840,"data_stored":309673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:22:46 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.622,"15min":0.622},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84899999999999998,"5min":1.032,"15min":1.032},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.69299999999999995,"15min":0.69699999999999995},"min":{"1min":0.47299999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.877,"5min":1.02,"15min":1.02},"last":0.69899999999999995}]},{"osd":1,"last update":"Tue Nov 28 11:22:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.65600000000000003,"15min":0.65500000000000003},"min":{"1min":0.51200000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.88100000000000001,"5min":1.355,"15min":1.355},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.68700000000000006,"15min":0.69399999999999995},"min":{"1min":0.498,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.86399999999999999,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.84699999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:22:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.68799999999999994,"15min":0.69599999999999995},"min":{"1min":0.44700000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.88500000000000001,"5min":1.198,"15min":1.198},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.72599999999999998,"15min":0.73299999999999998},"min":{"1min":0.50600000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.97199999999999998,"5min":1.387,"15min":1.387},"last":0.86299999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:22:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.74299999999999999,"15min":0.754},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.89600000000000002,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.72099999999999997,"15min":0.72499999999999998},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.81699999999999995}]},{"osd":4,"last update":"Tue Nov 28 11:22:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.71599999999999997,"15min":0.72699999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.90400000000000003,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.74099999999999999,"15min":0.74199999999999999},"min":{"1min":0.48899999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.97999999999999998,"5min":1.198,"15min":1.198},"last":0.497}]},{"osd":6,"last update":"Tue Nov 28 11:22:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.499,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0189999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72899999999999998,"15min":0.72999999999999998},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.96099999999999997,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.61799999999999999}]},{"osd":7,"last update":"Tue Nov 28 11:22:46 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.82099999999999995,"15min":0.82899999999999996},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.97499999999999998,"5min":1.371,"15min":1.371},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.82399999999999995,"15min":0.83099999999999996},"min":{"1min":0.54300000000000004,"5min":0.496,"15min":0.496},"max":{"1min":1.0509999999999999,"5min":1.298,"15min":1.298},"last":0.57799999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:23:18.083 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T11:23:18.083 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2023-11-28T11:23:18.299 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:23:18.305 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:23:18.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.325+0000 7f533bdff700 1 Processor -- start 2023-11-28T11:23:18.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.326+0000 7f533bdff700 1 -- start start 2023-11-28T11:23:18.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.327+0000 7f533bdff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f533415a200 0x7f533415d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:18.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.327+0000 7f533bdff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53340cf420 con 0x7f53340b22a0 2023-11-28T11:23:18.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.328+0000 7f533bdff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53340b2740 con 0x7f53340cf8b0 2023-11-28T11:23:18.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.328+0000 7f533bdff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f533415db40 con 0x7f533415a200 2023-11-28T11:23:18.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.328+0000 7f533939a700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f53340b22a0 0x7f53340d0a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36958/0 (socket says 172.21.15.92:36958) 2023-11-28T11:23:18.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.328+0000 7f533939a700 1 -- 172.21.15.92:0/2096026693 learned_addr learned my addr 172.21.15.92:0/2096026693 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:23:18.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.328+0000 7f5339b9b700 1 --2- 172.21.15.92:0/2096026693 >> v2:172.21.15.169:3300/0 conn(0x7f533415a200 0x7f533415d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:18.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.328+0000 7f5338b99700 1 -- 172.21.15.92:0/2096026693 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2498354244 0 0) 0x7f53340cf420 con 0x7f53340b22a0 2023-11-28T11:23:18.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.328+0000 7f5338b99700 1 -- 172.21.15.92:0/2096026693 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53200035b0 con 0x7f53340b22a0 2023-11-28T11:23:18.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.328+0000 7f5338b99700 1 -- 172.21.15.92:0/2096026693 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2286799379 0 0) 0x7f53340b2740 con 0x7f53340cf8b0 2023-11-28T11:23:18.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.328+0000 7f5338b99700 1 -- 172.21.15.92:0/2096026693 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53340cf420 con 0x7f53340cf8b0 2023-11-28T11:23:18.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.328+0000 7f5339b9b700 1 -- 172.21.15.92:0/2096026693 >> v1:172.21.15.169:6789/0 conn(0x7f53340cf8b0 legacy=0x7f53340cfcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:18.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.328+0000 7f5339b9b700 1 -- 172.21.15.92:0/2096026693 >> v1:172.21.15.92:6789/0 conn(0x7f53340b22a0 legacy=0x7f53340d0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:18.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.328+0000 7f5339b9b700 1 -- 172.21.15.92:0/2096026693 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53340cd680 con 0x7f533415a200 2023-11-28T11:23:18.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.329+0000 7f5339b9b700 1 --2- 172.21.15.92:0/2096026693 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f533415a200 0x7f533415d560 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5330009840 tx=0x7f5330009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a19f22eea6e2d76a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:18.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.329+0000 7f5338b99700 1 -- 172.21.15.92:0/2096026693 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5330011830 con 0x7f533415a200 2023-11-28T11:23:18.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.329+0000 7f5338b99700 1 -- 172.21.15.92:0/2096026693 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5330011990 con 0x7f533415a200 2023-11-28T11:23:18.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.329+0000 7f5338b99700 1 -- 172.21.15.92:0/2096026693 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5330011d20 con 0x7f533415a200 2023-11-28T11:23:18.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.330+0000 7f533bdff700 1 -- 172.21.15.92:0/2096026693 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f533415a200 msgr2=0x7f533415d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:18.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.330+0000 7f533bdff700 1 --2- 172.21.15.92:0/2096026693 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f533415a200 0x7f533415d560 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5330009840 tx=0x7f5330009b50 comp rx=0 tx=0).stop 2023-11-28T11:23:18.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.330+0000 7f533bdff700 1 -- 172.21.15.92:0/2096026693 shutdown_connections 2023-11-28T11:23:18.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.330+0000 7f533bdff700 1 --2- 172.21.15.92:0/2096026693 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f533415a200 0x7f533415d560 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:18.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.330+0000 7f533bdff700 1 -- 172.21.15.92:0/2096026693 >> 172.21.15.92:0/2096026693 conn(0x7f53340719a0 msgr2=0x7f5334071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:18.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.330+0000 7f533bdff700 1 -- 172.21.15.92:0/2096026693 shutdown_connections 2023-11-28T11:23:18.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.330+0000 7f533bdff700 1 -- 172.21.15.92:0/2096026693 wait complete. 2023-11-28T11:23:18.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.332+0000 7f533bdff700 1 Processor -- start 2023-11-28T11:23:18.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.332+0000 7f533bdff700 1 -- start start 2023-11-28T11:23:18.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.333+0000 7f533bdff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53340b22a0 0x7f53341542b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:18.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.333+0000 7f533bdff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53340cf8b0 0x7f533415b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:18.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.333+0000 7f533bdff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f533415a200 0x7f533415b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:18.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.333+0000 7f533bdff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53340b6d10 con 0x7f53340cf8b0 2023-11-28T11:23:18.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.333+0000 7f533bdff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53340b6f60 con 0x7f53340b22a0 2023-11-28T11:23:18.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.333+0000 7f533bdff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53340b7240 con 0x7f533415a200 2023-11-28T11:23:18.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.333+0000 7f533939a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53340cf8b0 0x7f533415b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:18.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.333+0000 7f533939a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53340cf8b0 0x7f533415b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40364/0 (socket says 172.21.15.92:40364) 2023-11-28T11:23:18.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.333+0000 7f533939a700 1 -- 172.21.15.92:0/2539279912 learned_addr learned my addr 172.21.15.92:0/2539279912 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:18.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.333+0000 7f5339b9b700 1 --2- 172.21.15.92:0/2539279912 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53340b22a0 0x7f53341542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:18.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.333+0000 7f533a39c700 1 --2- 172.21.15.92:0/2539279912 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f533415a200 0x7f533415b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:18.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.334+0000 7f533939a700 1 -- 172.21.15.92:0/2539279912 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f533415a200 msgr2=0x7f533415b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:18.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.334+0000 7f533939a700 1 --2- 172.21.15.92:0/2539279912 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f533415a200 0x7f533415b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:18.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.334+0000 7f533939a700 1 -- 172.21.15.92:0/2539279912 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53340b22a0 msgr2=0x7f53341542b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:18.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.334+0000 7f533939a700 1 --2- 172.21.15.92:0/2539279912 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53340b22a0 0x7f53341542b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:18.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.334+0000 7f533939a700 1 -- 172.21.15.92:0/2539279912 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53300094a0 con 0x7f53340cf8b0 2023-11-28T11:23:18.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.334+0000 7f533939a700 1 --2- 172.21.15.92:0/2539279912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53340cf8b0 0x7f533415b080 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f5324001eb0 tx=0x7f532400c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:18.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.334+0000 7f532a7fc700 1 -- 172.21.15.92:0/2539279912 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5324005d70 con 0x7f53340cf8b0 2023-11-28T11:23:18.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.334+0000 7f532a7fc700 1 -- 172.21.15.92:0/2539279912 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53240070d0 con 0x7f53340cf8b0 2023-11-28T11:23:18.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.334+0000 7f533bdff700 1 -- 172.21.15.92:0/2539279912 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53340d1170 con 0x7f53340cf8b0 2023-11-28T11:23:18.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.334+0000 7f532a7fc700 1 -- 172.21.15.92:0/2539279912 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5324007460 con 0x7f53340cf8b0 2023-11-28T11:23:18.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.334+0000 7f533bdff700 1 -- 172.21.15.92:0/2539279912 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53340cd960 con 0x7f53340cf8b0 2023-11-28T11:23:18.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.336+0000 7f533bdff700 1 -- 172.21.15.92:0/2539279912 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53340cf040 con 0x7f53340cf8b0 2023-11-28T11:23:18.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.336+0000 7f532a7fc700 1 -- 172.21.15.92:0/2539279912 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5324019030 con 0x7f53340cf8b0 2023-11-28T11:23:18.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.337+0000 7f532a7fc700 1 --2- 172.21.15.92:0/2539279912 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f53180682f0 0x7f531806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:18.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.337+0000 7f532a7fc700 1 -- 172.21.15.92:0/2539279912 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f5324085140 con 0x7f53340cf8b0 2023-11-28T11:23:18.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.337+0000 7f5339b9b700 1 --2- 172.21.15.92:0/2539279912 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f53180682f0 0x7f531806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:18.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.338+0000 7f5339b9b700 1 --2- 172.21.15.92:0/2539279912 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f53180682f0 0x7f531806a7a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5330009fd0 tx=0x7f533001c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:18.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.341+0000 7f532a7fc700 1 -- 172.21.15.92:0/2539279912 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5324051770 con 0x7f53340cf8b0 2023-11-28T11:23:18.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.615+0000 7f533bdff700 1 -- 172.21.15.92:0/2539279912 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7f5334067170 con 0x7f53340cf8b0 2023-11-28T11:23:18.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.619+0000 7f532a7fc700 1 -- 172.21.15.92:0/2539279912 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 184+0+0 (secure 0 0 0) 0x7f5324054d90 con 0x7f53340cf8b0 2023-11-28T11:23:18.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.622+0000 7f533bdff700 1 -- 172.21.15.92:0/2539279912 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f53180682f0 msgr2=0x7f531806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:18.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.622+0000 7f533bdff700 1 --2- 172.21.15.92:0/2539279912 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f53180682f0 0x7f531806a7a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5330009fd0 tx=0x7f533001c040 comp rx=0 tx=0).stop 2023-11-28T11:23:18.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.622+0000 7f533bdff700 1 -- 172.21.15.92:0/2539279912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53340cf8b0 msgr2=0x7f533415b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:18.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.622+0000 7f533bdff700 1 --2- 172.21.15.92:0/2539279912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53340cf8b0 0x7f533415b080 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f5324001eb0 tx=0x7f532400c7b0 comp rx=0 tx=0).stop 2023-11-28T11:23:18.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.622+0000 7f533bdff700 1 -- 172.21.15.92:0/2539279912 shutdown_connections 2023-11-28T11:23:18.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.622+0000 7f533bdff700 1 --2- 172.21.15.92:0/2539279912 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f533415a200 0x7f533415b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:18.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.622+0000 7f533bdff700 1 --2- 172.21.15.92:0/2539279912 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f53340b22a0 0x7f53341542b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:18.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.622+0000 7f533bdff700 1 --2- 172.21.15.92:0/2539279912 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f53180682f0 0x7f531806a7a0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:18.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.622+0000 7f533bdff700 1 --2- 172.21.15.92:0/2539279912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53340cf8b0 0x7f533415b080 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:18.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.622+0000 7f533bdff700 1 -- 172.21.15.92:0/2539279912 >> 172.21.15.92:0/2539279912 conn(0x7f53340719a0 msgr2=0x7f533415f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:18.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.622+0000 7f533bdff700 1 -- 172.21.15.92:0/2539279912 shutdown_connections 2023-11-28T11:23:18.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.622+0000 7f533bdff700 1 -- 172.21.15.92:0/2539279912 wait complete. 2023-11-28T11:23:18.624 INFO:teuthology.orchestra.run.smithi092.stderr:updated caps for client.0 2023-11-28T11:23:18.641 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:23:18.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.837+0000 7f58c13ff700 1 Processor -- start 2023-11-28T11:23:18.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.837+0000 7f58c13ff700 1 -- start start 2023-11-28T11:23:18.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.838+0000 7f58c13ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58bc0d1d70 0x7f58bc167270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:18.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.838+0000 7f58c13ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58bc06c790 con 0x7f58bc0cd460 2023-11-28T11:23:18.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.838+0000 7f58c13ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58bc0b88e0 con 0x7f58bc0b8510 2023-11-28T11:23:18.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.838+0000 7f58c13ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc0cdbd0 con 0x7f58bc0d1d70 2023-11-28T11:23:18.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.838+0000 7f58ba7fc700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f58bc0cd460 0x7f58bc06c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36960/0 (socket says 172.21.15.92:36960) 2023-11-28T11:23:18.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.838+0000 7f58ba7fc700 1 -- 172.21.15.92:0/3289722640 learned_addr learned my addr 172.21.15.92:0/3289722640 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:23:18.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.839+0000 7f58bb7fe700 1 --2- 172.21.15.92:0/3289722640 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58bc0d1d70 0x7f58bc167270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:18.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.839+0000 7f58b9ffb700 1 -- 172.21.15.92:0/3289722640 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3555930413 0 0) 0x7f58bc06c790 con 0x7f58bc0cd460 2023-11-28T11:23:18.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.839+0000 7f58b9ffb700 1 -- 172.21.15.92:0/3289722640 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58a40035b0 con 0x7f58bc0cd460 2023-11-28T11:23:18.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.839+0000 7f58b9ffb700 1 -- 172.21.15.92:0/3289722640 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2161441787 0 0) 0x7f58a40035b0 con 0x7f58bc0cd460 2023-11-28T11:23:18.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.839+0000 7f58b9ffb700 1 -- 172.21.15.92:0/3289722640 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58bc0d1d70 msgr2=0x7f58bc167270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:18.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.839+0000 7f58b9ffb700 1 --2- 172.21.15.92:0/3289722640 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58bc0d1d70 0x7f58bc167270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:18.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.839+0000 7f58b9ffb700 1 -- 172.21.15.92:0/3289722640 >> v1:172.21.15.169:6789/0 conn(0x7f58bc0b8510 legacy=0x7f58bc06b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:18.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.839+0000 7f58b9ffb700 1 -- 172.21.15.92:0/3289722640 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58bc0cd8f0 con 0x7f58bc0cd460 2023-11-28T11:23:18.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.839+0000 7f58b9ffb700 1 -- 172.21.15.92:0/3289722640 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532161587 0 0) 0x7f58b0001670 con 0x7f58bc0cd460 2023-11-28T11:23:18.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.839+0000 7f58bb7fe700 1 --2- 172.21.15.92:0/3289722640 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58bc0d1d70 0x7f58bc167270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:23:18.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.839+0000 7f58b9ffb700 1 -- 172.21.15.92:0/3289722640 >> v1:172.21.15.92:6789/0 conn(0x7f58bc0cd460 legacy=0x7f58bc06c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:18.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.839+0000 7f58b9ffb700 1 --2- 172.21.15.92:0/3289722640 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58a4003a30 0x7f58a4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:18.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.839+0000 7f58b9ffb700 1 --2- 172.21.15.92:0/3289722640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58a40043d0 0x7f58a4004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:18.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.839+0000 7f58b9ffb700 1 --2- 172.21.15.92:0/3289722640 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58bc0d1d70 0x7f58a40050b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:18.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.840+0000 7f58b9ffb700 1 -- 172.21.15.92:0/3289722640 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc0cdbd0 con 0x7f58a40043d0 2023-11-28T11:23:18.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.840+0000 7f58b9ffb700 1 -- 172.21.15.92:0/3289722640 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a400ba10 con 0x7f58a4003a30 2023-11-28T11:23:18.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.840+0000 7f58b9ffb700 1 -- 172.21.15.92:0/3289722640 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58a400bba0 con 0x7f58bc0d1d70 2023-11-28T11:23:18.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.840+0000 7f58ba7fc700 1 --2- 172.21.15.92:0/3289722640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58a40043d0 0x7f58a4004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:18.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.840+0000 7f58baffd700 1 --2- 172.21.15.92:0/3289722640 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58a4003a30 0x7f58a4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:18.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.840+0000 7f58bb7fe700 1 --2- 172.21.15.92:0/3289722640 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58bc0d1d70 0x7f58a40050b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:18.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.840+0000 7f58ba7fc700 1 -- 172.21.15.92:0/3289722640 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58bc0d1d70 msgr2=0x7f58a40050b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:18.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.840+0000 7f58ba7fc700 1 --2- 172.21.15.92:0/3289722640 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58bc0d1d70 0x7f58a40050b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:18.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.840+0000 7f58ba7fc700 1 -- 172.21.15.92:0/3289722640 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58a4003a30 msgr2=0x7f58a4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:18.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.840+0000 7f58ba7fc700 1 --2- 172.21.15.92:0/3289722640 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58a4003a30 0x7f58a4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:18.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.840+0000 7f58ba7fc700 1 -- 172.21.15.92:0/3289722640 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58b4009a70 con 0x7f58a40043d0 2023-11-28T11:23:18.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.840+0000 7f58ba7fc700 1 --2- 172.21.15.92:0/3289722640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58a40043d0 0x7f58a4004890 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f58b000c210 tx=0x7f58a4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1616183630c73b90 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:18.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.841+0000 7f58b9ffb700 1 -- 172.21.15.92:0/3289722640 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b000cd10 con 0x7f58a40043d0 2023-11-28T11:23:18.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.841+0000 7f58b9ffb700 1 -- 172.21.15.92:0/3289722640 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58b000ce70 con 0x7f58a40043d0 2023-11-28T11:23:18.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.841+0000 7f58b9ffb700 1 -- 172.21.15.92:0/3289722640 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b0005b80 con 0x7f58a40043d0 2023-11-28T11:23:18.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.841+0000 7f58c13ff700 1 -- 172.21.15.92:0/3289722640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58a40043d0 msgr2=0x7f58a4004890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:18.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.841+0000 7f58c13ff700 1 --2- 172.21.15.92:0/3289722640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58a40043d0 0x7f58a4004890 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f58b000c210 tx=0x7f58a4005880 comp rx=0 tx=0).stop 2023-11-28T11:23:18.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.841+0000 7f58bb7fe700 1 -- 172.21.15.92:0/3289722640 reap_dead start 2023-11-28T11:23:18.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.841+0000 7f58c13ff700 1 -- 172.21.15.92:0/3289722640 shutdown_connections 2023-11-28T11:23:18.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.841+0000 7f58c13ff700 1 -- 172.21.15.92:0/3289722640 >> 172.21.15.92:0/3289722640 conn(0x7f58bc0719a0 msgr2=0x7f58bc071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:18.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.841+0000 7f58c13ff700 1 -- 172.21.15.92:0/3289722640 shutdown_connections 2023-11-28T11:23:18.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.841+0000 7f58c13ff700 1 -- 172.21.15.92:0/3289722640 wait complete. 2023-11-28T11:23:18.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.843+0000 7f58c13ff700 1 Processor -- start 2023-11-28T11:23:18.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.843+0000 7f58c13ff700 1 -- start start 2023-11-28T11:23:18.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.844+0000 7f58c13ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58a40043d0 0x7f58bc154190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:18.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.844+0000 7f58c13ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58bc159bc0 0x7f58bc15a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:18.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.844+0000 7f58c13ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58bc154f50 0x7f58bc1553c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:18.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.844+0000 7f58c13ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc15a570 con 0x7f58a40043d0 2023-11-28T11:23:18.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.844+0000 7f58c13ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc1546d0 con 0x7f58bc154f50 2023-11-28T11:23:18.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.844+0000 7f58c13ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc155c10 con 0x7f58bc159bc0 2023-11-28T11:23:18.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.844+0000 7f58baffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58a40043d0 0x7f58bc154190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:18.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.844+0000 7f58baffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58a40043d0 0x7f58bc154190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40376/0 (socket says 172.21.15.92:40376) 2023-11-28T11:23:18.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.844+0000 7f58baffd700 1 -- 172.21.15.92:0/2806920074 learned_addr learned my addr 172.21.15.92:0/2806920074 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:18.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.845+0000 7f58baffd700 1 -- 172.21.15.92:0/2806920074 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58bc159bc0 msgr2=0x7f58bc15a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:18.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.845+0000 7f58baffd700 1 --2- 172.21.15.92:0/2806920074 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58bc159bc0 0x7f58bc15a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:18.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.845+0000 7f58baffd700 1 -- 172.21.15.92:0/2806920074 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58bc154f50 msgr2=0x7f58bc1553c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:18.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.845+0000 7f58baffd700 1 --2- 172.21.15.92:0/2806920074 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58bc154f50 0x7f58bc1553c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:18.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.845+0000 7f58baffd700 1 -- 172.21.15.92:0/2806920074 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58bc0cfff0 con 0x7f58a40043d0 2023-11-28T11:23:18.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.845+0000 7f58baffd700 1 --2- 172.21.15.92:0/2806920074 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58a40043d0 0x7f58bc154190 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f58ac00f780 tx=0x7f58ac00fb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:18.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.845+0000 7f58a37fe700 1 -- 172.21.15.92:0/2806920074 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac0052e0 con 0x7f58a40043d0 2023-11-28T11:23:18.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.845+0000 7f58a37fe700 1 -- 172.21.15.92:0/2806920074 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58ac005440 con 0x7f58a40043d0 2023-11-28T11:23:18.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.845+0000 7f58c13ff700 1 -- 172.21.15.92:0/2806920074 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58bc0cfd10 con 0x7f58a40043d0 2023-11-28T11:23:18.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.845+0000 7f58a37fe700 1 -- 172.21.15.92:0/2806920074 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac0038c0 con 0x7f58a40043d0 2023-11-28T11:23:18.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.845+0000 7f58c13ff700 1 -- 172.21.15.92:0/2806920074 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58bc0d29c0 con 0x7f58a40043d0 2023-11-28T11:23:18.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.846+0000 7f58c13ff700 1 -- 172.21.15.92:0/2806920074 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58bc0ca0e0 con 0x7f58a40043d0 2023-11-28T11:23:18.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.847+0000 7f58a37fe700 1 -- 172.21.15.92:0/2806920074 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f58ac003a20 con 0x7f58a40043d0 2023-11-28T11:23:18.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.848+0000 7f58a37fe700 1 --2- 172.21.15.92:0/2806920074 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f58a80682f0 0x7f58a806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:18.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.848+0000 7f58a37fe700 1 -- 172.21.15.92:0/2806920074 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f58ac085ea0 con 0x7f58a40043d0 2023-11-28T11:23:18.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.848+0000 7f58ba7fc700 1 --2- 172.21.15.92:0/2806920074 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f58a80682f0 0x7f58a806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:18.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.849+0000 7f58ba7fc700 1 --2- 172.21.15.92:0/2806920074 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f58a80682f0 0x7f58a806a7a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f58bc0c8f30 tx=0x7f58b0007f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:18.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:18.852+0000 7f58a37fe700 1 -- 172.21.15.92:0/2806920074 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f58ac052450 con 0x7f58a40043d0 2023-11-28T11:23:19.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.084+0000 7f58c13ff700 1 -- 172.21.15.92:0/2806920074 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f58bc067880 con 0x7f58a40043d0 2023-11-28T11:23:19.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.085+0000 7f58a37fe700 1 -- 172.21.15.92:0/2806920074 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4651 (secure 0 0 0) 0x7f58ac003cb0 con 0x7f58a40043d0 2023-11-28T11:23:19.086 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:23:19.086 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5593,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/3937189339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3937189339},{"type":"v1","addr":"172.21.15.169:6834","nonce":3937189339}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5596,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2020201016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2020201016},{"type":"v1","addr":"172.21.15.169:6835","nonce":2020201016}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5751,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2423750324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2423750324},{"type":"v1","addr":"172.21.15.92:6835","nonce":2423750324}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:23:13.247233+0000","modified":"2023-11-28T11:23:16.254256+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5754},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5754":{"gid":5754,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/950691230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":950691230},{"type":"v1","addr":"172.21.15.92:6837","nonce":950691230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T11:23:19.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.087+0000 7f58c13ff700 1 -- 172.21.15.92:0/2806920074 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f58a80682f0 msgr2=0x7f58a806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:19.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.087+0000 7f58c13ff700 1 --2- 172.21.15.92:0/2806920074 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f58a80682f0 0x7f58a806a7a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f58bc0c8f30 tx=0x7f58b0007f90 comp rx=0 tx=0).stop 2023-11-28T11:23:19.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.087+0000 7f58c13ff700 1 -- 172.21.15.92:0/2806920074 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58a40043d0 msgr2=0x7f58bc154190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:19.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.087+0000 7f58c13ff700 1 --2- 172.21.15.92:0/2806920074 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58a40043d0 0x7f58bc154190 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f58ac00f780 tx=0x7f58ac00fb40 comp rx=0 tx=0).stop 2023-11-28T11:23:19.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.087+0000 7f58c13ff700 1 -- 172.21.15.92:0/2806920074 shutdown_connections 2023-11-28T11:23:19.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.087+0000 7f58c13ff700 1 --2- 172.21.15.92:0/2806920074 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58bc154f50 0x7f58bc1553c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:19.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.087+0000 7f58c13ff700 1 --2- 172.21.15.92:0/2806920074 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f58a80682f0 0x7f58a806a7a0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:19.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.087+0000 7f58c13ff700 1 --2- 172.21.15.92:0/2806920074 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58a40043d0 0x7f58bc154190 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:19.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.087+0000 7f58c13ff700 1 --2- 172.21.15.92:0/2806920074 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58bc159bc0 0x7f58bc15a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:19.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.087+0000 7f58c13ff700 1 -- 172.21.15.92:0/2806920074 >> 172.21.15.92:0/2806920074 conn(0x7f58bc0719a0 msgr2=0x7f58bc15d560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:19.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.087+0000 7f58c13ff700 1 -- 172.21.15.92:0/2806920074 shutdown_connections 2023-11-28T11:23:19.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.087+0000 7f58c13ff700 1 -- 172.21.15.92:0/2806920074 wait complete. 2023-11-28T11:23:19.089 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 24 2023-11-28T11:23:19.510 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T11:23:13.247233+0000', 'modified': '2023-11-28T11:23:16.254256+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5754}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5754': {'gid': 5754, 'name': 'c', 'rank': 0, 'incarnation': 21, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.92:6837/950691230', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6836', 'nonce': 950691230}, {'type': 'v1', 'addr': '172.21.15.92:6837', 'nonce': 950691230}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T11:23:19.510 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T11:23:19.511 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-28T11:23:19.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.711+0000 7fb8915ff700 1 Processor -- start 2023-11-28T11:23:19.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.711+0000 7fb8915ff700 1 -- start start 2023-11-28T11:23:19.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.712+0000 7fb8915ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb88c0b8510 0x7fb88c06b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:19.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.712+0000 7fb8915ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb88c0cd460 0x7fb88c06beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:19.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.712+0000 7fb8915ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb88c0b88e0 con 0x7fb88c06c3f0 2023-11-28T11:23:19.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.712+0000 7fb8915ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb88c0cdbd0 con 0x7fb88c0cd460 2023-11-28T11:23:19.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.712+0000 7fb8915ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb88c06c830 con 0x7fb88c0b8510 2023-11-28T11:23:19.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.713+0000 7fb8827fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb88c0cd460 0x7fb88c06beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:19.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.713+0000 7fb8827fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb88c0cd460 0x7fb88c06beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58402/0 (socket says 172.21.15.92:58402) 2023-11-28T11:23:19.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.713+0000 7fb8827fc700 1 -- 172.21.15.92:0/1871789276 learned_addr learned my addr 172.21.15.92:0/1871789276 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:19.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.713+0000 7fb88affd700 1 --2- 172.21.15.92:0/1871789276 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb88c0b8510 0x7fb88c06b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:19.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.713+0000 7fb8827fc700 1 -- 172.21.15.92:0/1871789276 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb88c0b8510 msgr2=0x7fb88c06b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:19.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.713+0000 7fb8827fc700 1 --2- 172.21.15.92:0/1871789276 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb88c0b8510 0x7fb88c06b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:19.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.713+0000 7fb8827fc700 1 -- 172.21.15.92:0/1871789276 >> v1:172.21.15.169:6789/0 conn(0x7fb88c06c3f0 legacy=0x7fb88c15cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:19.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.713+0000 7fb8827fc700 1 -- 172.21.15.92:0/1871789276 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb88c0cd8f0 con 0x7fb88c0cd460 2023-11-28T11:23:19.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.713+0000 7fb88affd700 1 --2- 172.21.15.92:0/1871789276 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb88c0b8510 0x7fb88c06b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:23:19.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.713+0000 7fb8827fc700 1 --2- 172.21.15.92:0/1871789276 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb88c0cd460 0x7fb88c06beb0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fb874009920 tx=0x7fb874009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baf7ffc57394cc90 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:19.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.714+0000 7fb88a7fc700 1 -- 172.21.15.92:0/1871789276 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87400ba20 con 0x7fb88c0cd460 2023-11-28T11:23:19.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.714+0000 7fb88a7fc700 1 -- 172.21.15.92:0/1871789276 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb87400bb80 con 0x7fb88c0cd460 2023-11-28T11:23:19.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.714+0000 7fb88a7fc700 1 -- 172.21.15.92:0/1871789276 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb874009e70 con 0x7fb88c0cd460 2023-11-28T11:23:19.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.714+0000 7fb8915ff700 1 -- 172.21.15.92:0/1871789276 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb88c0cd460 msgr2=0x7fb88c06beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:19.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.714+0000 7fb8915ff700 1 --2- 172.21.15.92:0/1871789276 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb88c0cd460 0x7fb88c06beb0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fb874009920 tx=0x7fb874009c30 comp rx=0 tx=0).stop 2023-11-28T11:23:19.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.714+0000 7fb8915ff700 1 -- 172.21.15.92:0/1871789276 shutdown_connections 2023-11-28T11:23:19.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.714+0000 7fb8915ff700 1 --2- 172.21.15.92:0/1871789276 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb88c0b8510 0x7fb88c06b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:19.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.714+0000 7fb8915ff700 1 --2- 172.21.15.92:0/1871789276 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb88c0cd460 0x7fb88c06beb0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:19.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.714+0000 7fb8915ff700 1 -- 172.21.15.92:0/1871789276 >> 172.21.15.92:0/1871789276 conn(0x7fb88c0719a0 msgr2=0x7fb88c071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:19.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.714+0000 7fb8915ff700 1 -- 172.21.15.92:0/1871789276 shutdown_connections 2023-11-28T11:23:19.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.714+0000 7fb8915ff700 1 -- 172.21.15.92:0/1871789276 wait complete. 2023-11-28T11:23:19.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.716+0000 7fb8915ff700 1 Processor -- start 2023-11-28T11:23:19.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.716+0000 7fb8915ff700 1 -- start start 2023-11-28T11:23:19.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.717+0000 7fb8915ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb88c06c3f0 0x7fb88c1f1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:19.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.717+0000 7fb8915ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb88c0b8510 0x7fb88c149af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:19.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.717+0000 7fb8915ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb88c0cd460 0x7fb88c14a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:19.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.717+0000 7fb8915ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb88c1e4570 con 0x7fb88c0cd460 2023-11-28T11:23:19.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.717+0000 7fb8915ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb88c14af90 con 0x7fb88c0b8510 2023-11-28T11:23:19.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.717+0000 7fb8915ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb88c148290 con 0x7fb88c06c3f0 2023-11-28T11:23:19.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.717+0000 7fb88b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb88c0cd460 0x7fb88c14a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:19.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.718+0000 7fb88b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb88c0cd460 0x7fb88c14a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58414/0 (socket says 172.21.15.92:58414) 2023-11-28T11:23:19.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.718+0000 7fb88b7fe700 1 -- 172.21.15.92:0/3139572417 learned_addr learned my addr 172.21.15.92:0/3139572417 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:19.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.718+0000 7fb8827fc700 1 --2- 172.21.15.92:0/3139572417 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb88c0b8510 0x7fb88c149af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:19.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.718+0000 7fb88affd700 1 --2- 172.21.15.92:0/3139572417 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb88c06c3f0 0x7fb88c1f1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:19.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.718+0000 7fb88b7fe700 1 -- 172.21.15.92:0/3139572417 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb88c06c3f0 msgr2=0x7fb88c1f1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:19.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.718+0000 7fb88b7fe700 1 --2- 172.21.15.92:0/3139572417 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb88c06c3f0 0x7fb88c1f1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:19.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.718+0000 7fb88b7fe700 1 -- 172.21.15.92:0/3139572417 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb88c0b8510 msgr2=0x7fb88c149af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:19.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.718+0000 7fb88b7fe700 1 --2- 172.21.15.92:0/3139572417 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb88c0b8510 0x7fb88c149af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:19.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.718+0000 7fb88b7fe700 1 -- 172.21.15.92:0/3139572417 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb874009550 con 0x7fb88c0cd460 2023-11-28T11:23:19.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.718+0000 7fb88b7fe700 1 --2- 172.21.15.92:0/3139572417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb88c0cd460 0x7fb88c14a030 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fb88400c500 tx=0x7fb88400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:19.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.718+0000 7fb883fff700 1 -- 172.21.15.92:0/3139572417 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8840052d0 con 0x7fb88c0cd460 2023-11-28T11:23:19.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.718+0000 7fb883fff700 1 -- 172.21.15.92:0/3139572417 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb884005430 con 0x7fb88c0cd460 2023-11-28T11:23:19.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.718+0000 7fb8915ff700 1 -- 172.21.15.92:0/3139572417 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb88c0cfff0 con 0x7fb88c0cd460 2023-11-28T11:23:19.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.719+0000 7fb883fff700 1 -- 172.21.15.92:0/3139572417 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8840038c0 con 0x7fb88c0cd460 2023-11-28T11:23:19.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.719+0000 7fb8915ff700 1 -- 172.21.15.92:0/3139572417 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb88c0cfd10 con 0x7fb88c0cd460 2023-11-28T11:23:19.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.721+0000 7fb883fff700 1 -- 172.21.15.92:0/3139572417 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb884019030 con 0x7fb88c0cd460 2023-11-28T11:23:19.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.722+0000 7fb883fff700 1 --2- 172.21.15.92:0/3139572417 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb86c0682f0 0x7fb86c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:19.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.722+0000 7fb883fff700 1 -- 172.21.15.92:0/3139572417 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fb884084fe0 con 0x7fb88c0cd460 2023-11-28T11:23:19.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.722+0000 7fb88affd700 1 --2- 172.21.15.92:0/3139572417 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb86c0682f0 0x7fb86c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:19.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.723+0000 7fb88affd700 1 --2- 172.21.15.92:0/3139572417 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb86c0682f0 0x7fb86c06a7a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb87c00d620 tx=0x7fb87c007810 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:19.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.724+0000 7fb8a4ebeb80 1 Processor -- start 2023-11-28T11:23:19.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.724+0000 7fb8a4ebeb80 1 -- start start 2023-11-28T11:23:19.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.725+0000 7fb8a4ebeb80 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fba42ccf50 0x55fba42869f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:19.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.725+0000 7fb8a4ebeb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fba42c3ac0 0x55fba42cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:19.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.725+0000 7fb8a4ebeb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fba42af660 0x55fba4287b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:19.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.725+0000 7fb8a4ebeb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fba4204de0 con 0x55fba42af660 2023-11-28T11:23:19.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.725+0000 7fb8a4ebeb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fba4171140 con 0x55fba42c3ac0 2023-11-28T11:23:19.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.725+0000 7fb8a4ebeb80 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fba41dfdb0 con 0x55fba42ccf50 2023-11-28T11:23:19.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.725+0000 7fb88b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fba42af660 0x55fba4287b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:19.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.725+0000 7fb88b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fba42af660 0x55fba4287b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58428/0 (socket says 172.21.15.92:58428) 2023-11-28T11:23:19.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.725+0000 7fb88b7fe700 1 -- 172.21.15.92:0/3787603814 learned_addr learned my addr 172.21.15.92:0/3787603814 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:19.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.725+0000 7fb8827fc700 1 --2- 172.21.15.92:0/3787603814 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fba42c3ac0 0x55fba42cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:19.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.725+0000 7fb88affd700 1 --2- 172.21.15.92:0/3787603814 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fba42ccf50 0x55fba42869f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:19.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.726+0000 7fb88b7fe700 1 -- 172.21.15.92:0/3787603814 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fba42ccf50 msgr2=0x55fba42869f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:19.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.726+0000 7fb88b7fe700 1 --2- 172.21.15.92:0/3787603814 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fba42ccf50 0x55fba42869f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:19.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.726+0000 7fb88b7fe700 1 -- 172.21.15.92:0/3787603814 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fba42c3ac0 msgr2=0x55fba42cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:19.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.726+0000 7fb88b7fe700 1 --2- 172.21.15.92:0/3787603814 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fba42c3ac0 0x55fba42cd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:19.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.726+0000 7fb88b7fe700 1 -- 172.21.15.92:0/3787603814 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fba3fcd790 con 0x55fba42af660 2023-11-28T11:23:19.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.726+0000 7fb88b7fe700 1 --2- 172.21.15.92:0/3787603814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fba42af660 0x55fba4287b60 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fb884054d40 tx=0x7fb884054d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a08a769742cc3f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:19.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.726+0000 7fb880ff9700 1 -- 172.21.15.92:0/3787603814 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88405d110 con 0x55fba42af660 2023-11-28T11:23:19.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.726+0000 7fb880ff9700 1 -- 172.21.15.92:0/3787603814 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb884053510 con 0x55fba42af660 2023-11-28T11:23:19.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.726+0000 7fb880ff9700 1 -- 172.21.15.92:0/3787603814 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb884051340 con 0x55fba42af660 2023-11-28T11:23:19.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.727+0000 7fb8a4ebeb80 1 -- 172.21.15.92:0/3787603814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fba42af660 msgr2=0x55fba4287b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:19.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.727+0000 7fb8a4ebeb80 1 --2- 172.21.15.92:0/3787603814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fba42af660 0x55fba4287b60 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fb884054d40 tx=0x7fb884054d70 comp rx=0 tx=0).stop 2023-11-28T11:23:19.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.727+0000 7fb8a4ebeb80 1 -- 172.21.15.92:0/3787603814 shutdown_connections 2023-11-28T11:23:19.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.727+0000 7fb8a4ebeb80 1 --2- 172.21.15.92:0/3787603814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fba42af660 0x55fba4287b60 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:19.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.727+0000 7fb8a4ebeb80 1 --2- 172.21.15.92:0/3787603814 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fba42ccf50 0x55fba42869f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:19.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.727+0000 7fb8a4ebeb80 1 --2- 172.21.15.92:0/3787603814 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fba42c3ac0 0x55fba42cd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:19.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.727+0000 7fb8a4ebeb80 1 -- 172.21.15.92:0/3787603814 >> 172.21.15.92:0/3787603814 conn(0x55fba427cd20 msgr2=0x55fba4285f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:19.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.727+0000 7fb8a4ebeb80 1 -- 172.21.15.92:0/3787603814 shutdown_connections 2023-11-28T11:23:19.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.727+0000 7fb8a4ebeb80 1 -- 172.21.15.92:0/3787603814 wait complete. 2023-11-28T11:23:19.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.728+0000 7fb8a4ebeb80 1 -- start start 2023-11-28T11:23:19.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.728+0000 7fb8a4ebeb80 1 Processor -- start 2023-11-28T11:23:19.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.729+0000 7fb8a4ebeb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fba42af660 0x55fba42a9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:19.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.729+0000 7fb8a4ebeb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fba42c3ac0 0x55fba42c7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:19.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.729+0000 7fb8a4ebeb80 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fba42ccf50 0x55fba42c7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:19.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.729+0000 7fb8a4ebeb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fba41d9120 con 0x55fba42c3ac0 2023-11-28T11:23:19.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.729+0000 7fb8a4ebeb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fba3fce390 con 0x55fba42af660 2023-11-28T11:23:19.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.729+0000 7fb8a4ebeb80 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fba4171340 con 0x55fba42ccf50 2023-11-28T11:23:19.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.729+0000 7fb8a4ebeb80 10 client.0 _ll_register_callbacks cb 0x55fba4177400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:23:19.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.729+0000 7fb8827fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fba42c3ac0 0x55fba42c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:19.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.729+0000 7fb8827fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fba42c3ac0 0x55fba42c7250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58430/0 (socket says 172.21.15.92:58430) 2023-11-28T11:23:19.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.729+0000 7fb8827fc700 1 -- 172.21.15.92:0/337891178 learned_addr learned my addr 172.21.15.92:0/337891178 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:19.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.729+0000 7fb88affd700 1 --2- 172.21.15.92:0/337891178 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fba42af660 0x55fba42a9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:19.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.729+0000 7fb88b7fe700 1 --2- 172.21.15.92:0/337891178 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fba42ccf50 0x55fba42c7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:19.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.729+0000 7fb8827fc700 1 -- 172.21.15.92:0/337891178 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fba42ccf50 msgr2=0x55fba42c7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:19.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.729+0000 7fb8827fc700 1 --2- 172.21.15.92:0/337891178 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fba42ccf50 0x55fba42c7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:19.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.729+0000 7fb8827fc700 1 -- 172.21.15.92:0/337891178 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fba42af660 msgr2=0x55fba42a9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:19.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.729+0000 7fb8827fc700 1 --2- 172.21.15.92:0/337891178 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fba42af660 0x55fba42a9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:19.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.729+0000 7fb8827fc700 1 -- 172.21.15.92:0/337891178 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb88c0cfd10 con 0x55fba42c3ac0 2023-11-28T11:23:19.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.730+0000 7fb8827fc700 1 --2- 172.21.15.92:0/337891178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fba42c3ac0 0x55fba42c7250 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fb88c072610 tx=0x7fb87400e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c39f2cb30006b0f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:19.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.730+0000 7fb869ffb700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-11-28T11:23:19.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.730+0000 7fb869ffb700 1 -- 172.21.15.92:0/337891178 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87401f040 con 0x55fba42c3ac0 2023-11-28T11:23:19.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.730+0000 7fb869ffb700 1 -- 172.21.15.92:0/337891178 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(56..56 src has 1..56) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fb874004450 con 0x55fba42c3ac0 2023-11-28T11:23:19.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.730+0000 7fb8a4ebeb80 1 -- 172.21.15.92:0/337891178 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fba4028ef0 con 0x55fba42c3ac0 2023-11-28T11:23:19.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.730+0000 7fb869ffb700 1 -- 172.21.15.92:0/337891178 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb87400b9e0 con 0x55fba42c3ac0 2023-11-28T11:23:19.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.730+0000 7fb8a4ebeb80 10 client.5877 fetch_fsmap learned FSMap version 24 2023-11-28T11:23:19.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.730+0000 7fb8a4ebeb80 1 -- 172.21.15.92:0/337891178 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55fba40ba0a0 con 0x55fba42c3ac0 2023-11-28T11:23:19.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.730+0000 7fb869ffb700 1 -- 172.21.15.92:0/337891178 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fba4028ef0 con 0x55fba42c3ac0 2023-11-28T11:23:19.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.730+0000 7fb869ffb700 1 -- 172.21.15.92:0/337891178 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 24) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fb874022030 con 0x55fba42c3ac0 2023-11-28T11:23:19.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.731+0000 7fb8a4ebeb80 10 client.5877 fetch_fsmap finished waiting for FSMap version 24 2023-11-28T11:23:19.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.731+0000 7fb8a4ebeb80 10 client.5877 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T11:23:19.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.731+0000 7fb8a4ebeb80 20 client.5877 populate_metadata read hostname 'smithi092' 2023-11-28T11:23:19.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.731+0000 7fb8a4ebeb80 1 --2- 172.21.15.92:0/337891178 >> [v2:172.21.15.92:6836/950691230,v1:172.21.15.92:6837/950691230] conn(0x55fba42c4b20 0x55fba42c4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:19.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.731+0000 7fb8a4ebeb80 4 client.5877 mds_command: new command op to 5754 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:23:19.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.731+0000 7fb8a4ebeb80 1 -- 172.21.15.92:0/337891178 --> [v2:172.21.15.92:6836/950691230,v1:172.21.15.92:6837/950691230] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fba405e040 con 0x55fba42c4b20 2023-11-28T11:23:19.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.731+0000 7fb88affd700 1 --2- 172.21.15.92:0/337891178 >> [v2:172.21.15.92:6836/950691230,v1:172.21.15.92:6837/950691230] conn(0x55fba42c4b20 0x55fba42c4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:19.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.731+0000 7fb869ffb700 10 client.5877 ms_handle_connect on v2:172.21.15.92:6836/950691230 2023-11-28T11:23:19.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.731+0000 7fb88affd700 1 --2- 172.21.15.92:0/337891178 >> [v2:172.21.15.92:6836/950691230,v1:172.21.15.92:6837/950691230] conn(0x55fba42c4b20 0x55fba42c4ef0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6357353ad774008d server_cookie=b706f6a0f83a7678 in_seq=0 out_seq=0 2023-11-28T11:23:19.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.732+0000 7fb869ffb700 1 -- 172.21.15.92:0/337891178 <== mds.0 v2:172.21.15.92:6836/950691230 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x55fba405e040 con 0x55fba42c4b20 2023-11-28T11:23:19.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.732+0000 7fb869ffb700 10 client.5877 handle_command_reply: tid=0 2023-11-28T11:23:19.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.732+0000 7fb8a4ebeb80 1 client.5877 shutdown 2023-11-28T11:23:19.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.732+0000 7fb8a4ebeb80 1 -- 172.21.15.92:0/337891178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fba42c3ac0 msgr2=0x55fba42c7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:19.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.732+0000 7fb8a4ebeb80 1 --2- 172.21.15.92:0/337891178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fba42c3ac0 0x55fba42c7250 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fb88c072610 tx=0x7fb87400e000 comp rx=0 tx=0).stop 2023-11-28T11:23:19.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.732+0000 7fb8a4ebeb80 1 -- 172.21.15.92:0/337891178 shutdown_connections 2023-11-28T11:23:19.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.732+0000 7fb8a4ebeb80 1 --2- 172.21.15.92:0/337891178 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fba42ccf50 0x55fba42c7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:19.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.732+0000 7fb8a4ebeb80 1 --2- 172.21.15.92:0/337891178 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fba42af660 0x55fba42a9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:19.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.732+0000 7fb8a4ebeb80 1 --2- 172.21.15.92:0/337891178 >> [v2:172.21.15.92:6836/950691230,v1:172.21.15.92:6837/950691230] conn(0x55fba42c4b20 0x55fba42c4ef0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:19.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.732+0000 7fb8a4ebeb80 1 --2- 172.21.15.92:0/337891178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fba42c3ac0 0x55fba42c7250 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:19.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.732+0000 7fb8a4ebeb80 1 -- 172.21.15.92:0/337891178 >> 172.21.15.92:0/337891178 conn(0x55fba427cd20 msgr2=0x55fba4285f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:19.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.732+0000 7fb869ffb700 0 client.5877 ms_handle_reset on v2:172.21.15.92:6836/950691230 2023-11-28T11:23:19.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.733+0000 7fb8a4ebeb80 1 -- 172.21.15.92:0/337891178 shutdown_connections 2023-11-28T11:23:19.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.733+0000 7fb8a4ebeb80 1 -- 172.21.15.92:0/337891178 wait complete. 2023-11-28T11:23:19.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.733+0000 7fb8a4ebeb80 20 client.5877 trim_cache size 0 max 16384 2023-11-28T11:23:19.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.750+0000 7fb8a4ebeb80 1 Processor -- start 2023-11-28T11:23:19.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.751+0000 7fb8a4ebeb80 1 -- start start 2023-11-28T11:23:19.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.751+0000 7fb8a4ebeb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fba42af660 0x55fba42aeba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:19.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.751+0000 7fb8a4ebeb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fba42c3ac0 0x55fba4284280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:19.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.751+0000 7fb8a4ebeb80 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fba42ccf50 0x55fba42c6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:19.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.751+0000 7fb8a4ebeb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fba42834a0 con 0x55fba42c3ac0 2023-11-28T11:23:19.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.751+0000 7fb8a4ebeb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fba42c7b70 con 0x55fba42af660 2023-11-28T11:23:19.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.751+0000 7fb8a4ebeb80 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fba4283290 con 0x55fba42ccf50 2023-11-28T11:23:19.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.752+0000 7fb8827fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fba42c3ac0 0x55fba4284280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:19.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.752+0000 7fb8827fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fba42c3ac0 0x55fba4284280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58446/0 (socket says 172.21.15.92:58446) 2023-11-28T11:23:19.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.752+0000 7fb8827fc700 1 -- 172.21.15.92:0/3476993831 learned_addr learned my addr 172.21.15.92:0/3476993831 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:19.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.752+0000 7fb88affd700 1 --2- 172.21.15.92:0/3476993831 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fba42af660 0x55fba42aeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:19.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.752+0000 7fb8827fc700 1 -- 172.21.15.92:0/3476993831 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fba42ccf50 msgr2=0x55fba42c6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:19.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.752+0000 7fb8827fc700 1 --2- 172.21.15.92:0/3476993831 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fba42ccf50 0x55fba42c6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:19.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.752+0000 7fb8827fc700 1 -- 172.21.15.92:0/3476993831 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fba42af660 msgr2=0x55fba42aeba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:19.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.752+0000 7fb8827fc700 1 --2- 172.21.15.92:0/3476993831 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fba42af660 0x55fba42aeba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:19.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.752+0000 7fb8827fc700 1 -- 172.21.15.92:0/3476993831 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fba41a02e0 con 0x55fba42c3ac0 2023-11-28T11:23:19.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.752+0000 7fb88affd700 1 --2- 172.21.15.92:0/3476993831 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fba42af660 0x55fba42aeba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:23:19.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.752+0000 7fb8827fc700 1 --2- 172.21.15.92:0/3476993831 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fba42c3ac0 0x55fba4284280 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb874003db0 tx=0x7fb874011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b94b9d5f4168764 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:19.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.752+0000 7fb8817fa700 1 -- 172.21.15.92:0/3476993831 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb874006b20 con 0x55fba42c3ac0 2023-11-28T11:23:19.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.752+0000 7fb8817fa700 1 -- 172.21.15.92:0/3476993831 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb874004450 con 0x55fba42c3ac0 2023-11-28T11:23:19.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.752+0000 7fb8817fa700 1 -- 172.21.15.92:0/3476993831 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb874003a20 con 0x55fba42c3ac0 2023-11-28T11:23:19.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.753+0000 7fb8a4ebeb80 1 -- 172.21.15.92:0/3476993831 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fba42c3ac0 msgr2=0x55fba4284280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:19.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.753+0000 7fb8a4ebeb80 1 --2- 172.21.15.92:0/3476993831 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fba42c3ac0 0x55fba4284280 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb874003db0 tx=0x7fb874011000 comp rx=0 tx=0).stop 2023-11-28T11:23:19.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.753+0000 7fb8a4ebeb80 1 -- 172.21.15.92:0/3476993831 shutdown_connections 2023-11-28T11:23:19.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.753+0000 7fb8a4ebeb80 1 --2- 172.21.15.92:0/3476993831 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fba42ccf50 0x55fba42c6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:19.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.753+0000 7fb8a4ebeb80 1 --2- 172.21.15.92:0/3476993831 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fba42af660 0x55fba42aeba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:19.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.753+0000 7fb8a4ebeb80 1 --2- 172.21.15.92:0/3476993831 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fba42c3ac0 0x55fba4284280 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:19.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.753+0000 7fb8a4ebeb80 1 -- 172.21.15.92:0/3476993831 >> 172.21.15.92:0/3476993831 conn(0x55fba427cd20 msgr2=0x55fba42a9230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:19.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.753+0000 7fb8a4ebeb80 1 -- 172.21.15.92:0/3476993831 shutdown_connections 2023-11-28T11:23:19.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.753+0000 7fb8a4ebeb80 1 -- 172.21.15.92:0/3476993831 wait complete. 2023-11-28T11:23:19.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.754+0000 7fb8a4ebeb80 1 -- start start 2023-11-28T11:23:19.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.754+0000 7fb8a4ebeb80 1 Processor -- start 2023-11-28T11:23:19.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.755+0000 7fb8a4ebeb80 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fba42af660 0x55fba431b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:19.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.755+0000 7fb8a4ebeb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fba42c3ac0 0x55fba4331e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:19.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.755+0000 7fb8a4ebeb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fba42ccf50 0x55fba42836f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:19.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.755+0000 7fb8a4ebeb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fba42ade00 con 0x55fba42c3ac0 2023-11-28T11:23:19.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.755+0000 7fb8a4ebeb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fba4332690 con 0x55fba42ccf50 2023-11-28T11:23:19.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.755+0000 7fb8a4ebeb80 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fba42a7670 con 0x55fba42af660 2023-11-28T11:23:19.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.755+0000 7fb8a4ebeb80 10 client.0 _ll_register_callbacks cb 0x55fba4177400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:23:19.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.755+0000 7fb8827fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fba42c3ac0 0x55fba4331e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:19.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.755+0000 7fb8827fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fba42c3ac0 0x55fba4331e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58456/0 (socket says 172.21.15.92:58456) 2023-11-28T11:23:19.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.755+0000 7fb8827fc700 1 -- 172.21.15.92:0/3074524994 learned_addr learned my addr 172.21.15.92:0/3074524994 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:19.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.755+0000 7fb88affd700 1 --2- 172.21.15.92:0/3074524994 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fba42af660 0x55fba431b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:19.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.755+0000 7fb88b7fe700 1 --2- 172.21.15.92:0/3074524994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fba42ccf50 0x55fba42836f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:19.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.756+0000 7fb8827fc700 1 -- 172.21.15.92:0/3074524994 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fba42af660 msgr2=0x55fba431b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:19.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.756+0000 7fb8827fc700 1 --2- 172.21.15.92:0/3074524994 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fba42af660 0x55fba431b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:19.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.756+0000 7fb8827fc700 1 -- 172.21.15.92:0/3074524994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fba42ccf50 msgr2=0x55fba42836f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:19.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.756+0000 7fb8827fc700 1 --2- 172.21.15.92:0/3074524994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fba42ccf50 0x55fba42836f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:19.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.756+0000 7fb8827fc700 1 -- 172.21.15.92:0/3074524994 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fba40ba0a0 con 0x55fba42c3ac0 2023-11-28T11:23:19.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.756+0000 7fb8827fc700 1 --2- 172.21.15.92:0/3074524994 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fba42c3ac0 0x55fba4331e90 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb874037040 tx=0x7fb8740047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=601fffec29382e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:19.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.756+0000 7fb869ffb700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-11-28T11:23:19.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.756+0000 7fb869ffb700 1 -- 172.21.15.92:0/3074524994 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb874003b30 con 0x55fba42c3ac0 2023-11-28T11:23:19.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.756+0000 7fb869ffb700 1 -- 172.21.15.92:0/3074524994 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(56..56 src has 1..56) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fb874000e40 con 0x55fba42c3ac0 2023-11-28T11:23:19.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.756+0000 7fb8a4ebeb80 1 -- 172.21.15.92:0/3074524994 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fba41dd030 con 0x55fba42c3ac0 2023-11-28T11:23:19.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.756+0000 7fb869ffb700 1 -- 172.21.15.92:0/3074524994 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb87400be70 con 0x55fba42c3ac0 2023-11-28T11:23:19.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.756+0000 7fb8a4ebeb80 10 client.5883 fetch_fsmap learned FSMap version 24 2023-11-28T11:23:19.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.756+0000 7fb8a4ebeb80 1 -- 172.21.15.92:0/3074524994 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55fba402e1d0 con 0x55fba42c3ac0 2023-11-28T11:23:19.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.756+0000 7fb869ffb700 1 -- 172.21.15.92:0/3074524994 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fba41dd030 con 0x55fba42c3ac0 2023-11-28T11:23:19.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.757+0000 7fb869ffb700 1 -- 172.21.15.92:0/3074524994 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 24) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fb874022050 con 0x55fba42c3ac0 2023-11-28T11:23:19.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.757+0000 7fb8a4ebeb80 10 client.5883 fetch_fsmap finished waiting for FSMap version 24 2023-11-28T11:23:19.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.757+0000 7fb8a4ebeb80 10 client.5883 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T11:23:19.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.757+0000 7fb8a4ebeb80 20 client.5883 populate_metadata read hostname 'smithi092' 2023-11-28T11:23:19.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.757+0000 7fb8a4ebeb80 1 --2- 172.21.15.92:0/3074524994 >> [v2:172.21.15.92:6836/950691230,v1:172.21.15.92:6837/950691230] conn(0x55fba42e5cb0 0x55fba42c5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:19.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.757+0000 7fb8a4ebeb80 4 client.5883 mds_command: new command op to 5754 tid=0[{"prefix": "status"}] 2023-11-28T11:23:19.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.757+0000 7fb8a4ebeb80 1 -- 172.21.15.92:0/3074524994 --> [v2:172.21.15.92:6836/950691230,v1:172.21.15.92:6837/950691230] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55fba411b3c0 con 0x55fba42e5cb0 2023-11-28T11:23:19.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.757+0000 7fb88affd700 1 --2- 172.21.15.92:0/3074524994 >> [v2:172.21.15.92:6836/950691230,v1:172.21.15.92:6837/950691230] conn(0x55fba42e5cb0 0x55fba42c5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:19.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.757+0000 7fb869ffb700 10 client.5883 ms_handle_connect on v2:172.21.15.92:6836/950691230 2023-11-28T11:23:19.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.757+0000 7fb88affd700 1 --2- 172.21.15.92:0/3074524994 >> [v2:172.21.15.92:6836/950691230,v1:172.21.15.92:6837/950691230] conn(0x55fba42e5cb0 0x55fba42c5590 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0a5ef2ad95c93c2 server_cookie=c284e747bcc7c92f in_seq=0 out_seq=0 2023-11-28T11:23:19.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.758+0000 7fb869ffb700 1 -- 172.21.15.92:0/3074524994 <== mds.0 v2:172.21.15.92:6836/950691230 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55fba411b3c0 con 0x55fba42e5cb0 2023-11-28T11:23:19.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.758+0000 7fb869ffb700 10 client.5883 handle_command_reply: tid=0 2023-11-28T11:23:19.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.758+0000 7fb8a4ebeb80 1 client.5883 shutdown 2023-11-28T11:23:19.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.758+0000 7fb8a4ebeb80 1 -- 172.21.15.92:0/3074524994 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fba42c3ac0 msgr2=0x55fba4331e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:19.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.758+0000 7fb8a4ebeb80 1 --2- 172.21.15.92:0/3074524994 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fba42c3ac0 0x55fba4331e90 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb874037040 tx=0x7fb8740047c0 comp rx=0 tx=0).stop 2023-11-28T11:23:19.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.758+0000 7fb8a4ebeb80 1 -- 172.21.15.92:0/3074524994 shutdown_connections 2023-11-28T11:23:19.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.758+0000 7fb8a4ebeb80 1 --2- 172.21.15.92:0/3074524994 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55fba42ccf50 0x55fba42836f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:19.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.758+0000 7fb8a4ebeb80 1 --2- 172.21.15.92:0/3074524994 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55fba42af660 0x55fba431b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:19.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.758+0000 7fb8a4ebeb80 1 --2- 172.21.15.92:0/3074524994 >> [v2:172.21.15.92:6836/950691230,v1:172.21.15.92:6837/950691230] conn(0x55fba42e5cb0 0x55fba42c5590 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:19.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.758+0000 7fb8a4ebeb80 1 --2- 172.21.15.92:0/3074524994 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fba42c3ac0 0x55fba4331e90 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:19.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.758+0000 7fb8a4ebeb80 1 -- 172.21.15.92:0/3074524994 >> 172.21.15.92:0/3074524994 conn(0x55fba427cd20 msgr2=0x55fba4284280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:19.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.758+0000 7fb869ffb700 0 client.5883 ms_handle_reset on v2:172.21.15.92:6836/950691230 2023-11-28T11:23:19.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.758+0000 7fb8a4ebeb80 1 -- 172.21.15.92:0/3074524994 shutdown_connections 2023-11-28T11:23:19.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.758+0000 7fb8a4ebeb80 1 -- 172.21.15.92:0/3074524994 wait complete. 2023-11-28T11:23:19.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.758+0000 7fb8a4ebeb80 20 client.5883 trim_cache size 0 max 16384 2023-11-28T11:23:19.766 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-28T11:23:19.766 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "68956e0c-c16f-47cd-a220-f98364b0831d", 2023-11-28T11:23:19.766 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 0, 2023-11-28T11:23:19.766 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 5754, 2023-11-28T11:23:19.766 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-11-28T11:23:19.766 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-11-28T11:23:19.766 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-11-28T11:23:19.766 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 6.5087797690000002, 2023-11-28T11:23:19.767 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 24, 2023-11-28T11:23:19.767 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 56, 2023-11-28T11:23:19.767 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 56, 2023-11-28T11:23:19.767 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 12.531628674 2023-11-28T11:23:19.767 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-28T11:23:19.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.759+0000 7fb8915ff700 1 -- 172.21.15.92:0/3139572417 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb86c0682f0 msgr2=0x7fb86c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:19.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.759+0000 7fb8915ff700 1 --2- 172.21.15.92:0/3139572417 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb86c0682f0 0x7fb86c06a7a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb87c00d620 tx=0x7fb87c007810 comp rx=0 tx=0).stop 2023-11-28T11:23:19.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.759+0000 7fb8915ff700 1 -- 172.21.15.92:0/3139572417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb88c0cd460 msgr2=0x7fb88c14a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:19.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.759+0000 7fb8915ff700 1 --2- 172.21.15.92:0/3139572417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb88c0cd460 0x7fb88c14a030 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fb88400c500 tx=0x7fb88400c810 comp rx=0 tx=0).stop 2023-11-28T11:23:19.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.759+0000 7fb8915ff700 1 -- 172.21.15.92:0/3139572417 shutdown_connections 2023-11-28T11:23:19.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.759+0000 7fb8915ff700 1 --2- 172.21.15.92:0/3139572417 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb86c0682f0 0x7fb86c06a7a0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:19.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.759+0000 7fb8915ff700 1 --2- 172.21.15.92:0/3139572417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb88c0cd460 0x7fb88c14a030 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:19.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.759+0000 7fb8915ff700 1 --2- 172.21.15.92:0/3139572417 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb88c06c3f0 0x7fb88c1f1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:19.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.759+0000 7fb8915ff700 1 --2- 172.21.15.92:0/3139572417 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb88c0b8510 0x7fb88c149af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:19.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.759+0000 7fb8915ff700 1 -- 172.21.15.92:0/3139572417 >> 172.21.15.92:0/3139572417 conn(0x7fb88c0719a0 msgr2=0x7fb88c0b5d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:19.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.759+0000 7fb8915ff700 1 -- 172.21.15.92:0/3139572417 shutdown_connections 2023-11-28T11:23:19.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.759+0000 7fb8915ff700 1 -- 172.21.15.92:0/3139572417 wait complete. 2023-11-28T11:23:19.778 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T11:23:19.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.974+0000 7fd23adff700 1 Processor -- start 2023-11-28T11:23:19.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.974+0000 7fd23adff700 1 -- start start 2023-11-28T11:23:19.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.975+0000 7fd23adff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd2340cd460 0x7fd23406c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:19.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.975+0000 7fd23adff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2340d0d60 0x7fd2340d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:19.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.975+0000 7fd23adff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2340b88e0 con 0x7fd2340b8510 2023-11-28T11:23:19.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.975+0000 7fd23adff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2340cdbd0 con 0x7fd2340cd460 2023-11-28T11:23:19.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.975+0000 7fd23adff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2340d02d0 con 0x7fd2340d0d60 2023-11-28T11:23:19.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.976+0000 7fd23939c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2340d0d60 0x7fd2340d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:19.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.976+0000 7fd23939c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2340d0d60 0x7fd2340d1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.92:35762/0 (socket says 172.21.15.92:35762) 2023-11-28T11:23:19.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.976+0000 7fd23939c700 1 -- 172.21.15.92:0/619734968 learned_addr learned my addr 172.21.15.92:0/619734968 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:19.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.976+0000 7fd233fff700 1 --2- 172.21.15.92:0/619734968 >> v2:172.21.15.169:3300/0 conn(0x7fd2340cd460 0x7fd23406c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:19.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.976+0000 7fd23939c700 1 -- 172.21.15.92:0/619734968 >> v2:172.21.15.169:3300/0 conn(0x7fd2340cd460 msgr2=0x7fd23406c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:19.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.976+0000 7fd23939c700 1 --2- 172.21.15.92:0/619734968 >> v2:172.21.15.169:3300/0 conn(0x7fd2340cd460 0x7fd23406c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:19.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.976+0000 7fd23939c700 1 -- 172.21.15.92:0/619734968 >> v1:172.21.15.169:6789/0 conn(0x7fd2340b8510 legacy=0x7fd23406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:19.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.976+0000 7fd23939c700 1 -- 172.21.15.92:0/619734968 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2340cd8f0 con 0x7fd2340d0d60 2023-11-28T11:23:19.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.976+0000 7fd233fff700 1 --2- 172.21.15.92:0/619734968 >> v2:172.21.15.169:3300/0 conn(0x7fd2340cd460 0x7fd23406c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:23:19.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.976+0000 7fd23939c700 1 --2- 172.21.15.92:0/619734968 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2340d0d60 0x7fd2340d1200 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd22c009e50 tx=0x7fd22c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43a9151e94b00251 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:19.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.977+0000 7fd2337fe700 1 -- 172.21.15.92:0/619734968 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd22c00e070 con 0x7fd2340d0d60 2023-11-28T11:23:19.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.977+0000 7fd2337fe700 1 -- 172.21.15.92:0/619734968 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd22c00b9c0 con 0x7fd2340d0d60 2023-11-28T11:23:19.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.977+0000 7fd2337fe700 1 -- 172.21.15.92:0/619734968 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd22c00bd70 con 0x7fd2340d0d60 2023-11-28T11:23:19.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.977+0000 7fd23adff700 1 -- 172.21.15.92:0/619734968 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2340d0d60 msgr2=0x7fd2340d1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:19.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.977+0000 7fd23adff700 1 --2- 172.21.15.92:0/619734968 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2340d0d60 0x7fd2340d1200 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd22c009e50 tx=0x7fd22c00b330 comp rx=0 tx=0).stop 2023-11-28T11:23:19.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.977+0000 7fd23adff700 1 -- 172.21.15.92:0/619734968 shutdown_connections 2023-11-28T11:23:19.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.977+0000 7fd23adff700 1 --2- 172.21.15.92:0/619734968 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2340d0d60 0x7fd2340d1200 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:19.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.977+0000 7fd23adff700 1 --2- 172.21.15.92:0/619734968 >> v2:172.21.15.169:3300/0 conn(0x7fd2340cd460 0x7fd23406c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:19.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.977+0000 7fd23adff700 1 -- 172.21.15.92:0/619734968 >> 172.21.15.92:0/619734968 conn(0x7fd2340719a0 msgr2=0x7fd234071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:19.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.977+0000 7fd23adff700 1 -- 172.21.15.92:0/619734968 shutdown_connections 2023-11-28T11:23:19.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.977+0000 7fd23adff700 1 -- 172.21.15.92:0/619734968 wait complete. 2023-11-28T11:23:19.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.979+0000 7fd23adff700 1 Processor -- start 2023-11-28T11:23:19.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.979+0000 7fd23adff700 1 -- start start 2023-11-28T11:23:19.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.980+0000 7fd23adff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2340b8510 0x7fd2340b6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:19.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.980+0000 7fd23adff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2340cd460 0x7fd234155fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:19.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.980+0000 7fd23adff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2340d0d60 0x7fd234156510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:19.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.980+0000 7fd23adff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2341e2060 con 0x7fd2340b8510 2023-11-28T11:23:19.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.980+0000 7fd23adff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2341e3030 con 0x7fd2340d0d60 2023-11-28T11:23:19.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.980+0000 7fd23adff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd234156a50 con 0x7fd2340cd460 2023-11-28T11:23:19.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.980+0000 7fd238b9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2340b8510 0x7fd2340b6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:19.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.980+0000 7fd238b9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2340b8510 0x7fd2340b6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58470/0 (socket says 172.21.15.92:58470) 2023-11-28T11:23:19.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.980+0000 7fd238b9b700 1 -- 172.21.15.92:0/2991749657 learned_addr learned my addr 172.21.15.92:0/2991749657 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:19.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.981+0000 7fd23939c700 1 --2- 172.21.15.92:0/2991749657 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2340d0d60 0x7fd234156510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:19.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.981+0000 7fd233fff700 1 --2- 172.21.15.92:0/2991749657 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2340cd460 0x7fd234155fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:19.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.981+0000 7fd238b9b700 1 -- 172.21.15.92:0/2991749657 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2340cd460 msgr2=0x7fd234155fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:19.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.981+0000 7fd238b9b700 1 --2- 172.21.15.92:0/2991749657 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2340cd460 0x7fd234155fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:19.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.981+0000 7fd238b9b700 1 -- 172.21.15.92:0/2991749657 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2340d0d60 msgr2=0x7fd234156510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:19.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.981+0000 7fd238b9b700 1 --2- 172.21.15.92:0/2991749657 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2340d0d60 0x7fd234156510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:19.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.981+0000 7fd238b9b700 1 -- 172.21.15.92:0/2991749657 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd22c009b30 con 0x7fd2340b8510 2023-11-28T11:23:19.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.981+0000 7fd23939c700 1 --2- 172.21.15.92:0/2991749657 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2340d0d60 0x7fd234156510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:23:19.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.981+0000 7fd233fff700 1 --2- 172.21.15.92:0/2991749657 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2340cd460 0x7fd234155fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:23:19.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.981+0000 7fd238b9b700 1 --2- 172.21.15.92:0/2991749657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2340b8510 0x7fd2340b6f00 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fd22800c510 tx=0x7fd22800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:19.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.981+0000 7fd2317fa700 1 -- 172.21.15.92:0/2991749657 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2280052d0 con 0x7fd2340b8510 2023-11-28T11:23:19.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.981+0000 7fd2317fa700 1 -- 172.21.15.92:0/2991749657 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd228005430 con 0x7fd2340b8510 2023-11-28T11:23:19.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.981+0000 7fd23adff700 1 -- 172.21.15.92:0/2991749657 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2340cfff0 con 0x7fd2340b8510 2023-11-28T11:23:19.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.981+0000 7fd2317fa700 1 -- 172.21.15.92:0/2991749657 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2280038c0 con 0x7fd2340b8510 2023-11-28T11:23:19.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.981+0000 7fd23adff700 1 -- 172.21.15.92:0/2991749657 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2340cfd10 con 0x7fd2340b8510 2023-11-28T11:23:19.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.982+0000 7fd23adff700 1 -- 172.21.15.92:0/2991749657 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd234067880 con 0x7fd2340b8510 2023-11-28T11:23:19.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.983+0000 7fd2317fa700 1 -- 172.21.15.92:0/2991749657 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd228019030 con 0x7fd2340b8510 2023-11-28T11:23:19.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.984+0000 7fd2317fa700 1 --2- 172.21.15.92:0/2991749657 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd2180682f0 0x7fd21806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:19.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.985+0000 7fd2317fa700 1 -- 172.21.15.92:0/2991749657 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fd228085270 con 0x7fd2340b8510 2023-11-28T11:23:19.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.985+0000 7fd233fff700 1 --2- 172.21.15.92:0/2991749657 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd2180682f0 0x7fd21806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:19.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.985+0000 7fd233fff700 1 --2- 172.21.15.92:0/2991749657 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd2180682f0 0x7fd21806a7a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd2341e1e20 tx=0x7fd220008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:19.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:19.988+0000 7fd2317fa700 1 -- 172.21.15.92:0/2991749657 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd2280518a0 con 0x7fd2340b8510 2023-11-28T11:23:20.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:20.221+0000 7fd23adff700 1 -- 172.21.15.92:0/2991749657 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd2340cf1a0 con 0x7fd2340b8510 2023-11-28T11:23:20.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:20.222+0000 7fd2317fa700 1 -- 172.21.15.92:0/2991749657 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd228054ec0 con 0x7fd2340b8510 2023-11-28T11:23:20.223 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T11:23:20.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:20.224+0000 7fd23adff700 1 -- 172.21.15.92:0/2991749657 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd2180682f0 msgr2=0x7fd21806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:20.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:20.224+0000 7fd23adff700 1 --2- 172.21.15.92:0/2991749657 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd2180682f0 0x7fd21806a7a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd2341e1e20 tx=0x7fd220008040 comp rx=0 tx=0).stop 2023-11-28T11:23:20.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:20.224+0000 7fd23adff700 1 -- 172.21.15.92:0/2991749657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2340b8510 msgr2=0x7fd2340b6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:20.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:20.224+0000 7fd23adff700 1 --2- 172.21.15.92:0/2991749657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2340b8510 0x7fd2340b6f00 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fd22800c510 tx=0x7fd22800c8d0 comp rx=0 tx=0).stop 2023-11-28T11:23:20.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:20.224+0000 7fd23adff700 1 -- 172.21.15.92:0/2991749657 shutdown_connections 2023-11-28T11:23:20.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:20.224+0000 7fd23adff700 1 --2- 172.21.15.92:0/2991749657 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd2340d0d60 0x7fd234156510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:20.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:20.224+0000 7fd23adff700 1 --2- 172.21.15.92:0/2991749657 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd2180682f0 0x7fd21806a7a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:20.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:20.224+0000 7fd23adff700 1 --2- 172.21.15.92:0/2991749657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2340b8510 0x7fd2340b6f00 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:20.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:20.224+0000 7fd23adff700 1 --2- 172.21.15.92:0/2991749657 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd2340cd460 0x7fd234155fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:20.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:20.224+0000 7fd23adff700 1 -- 172.21.15.92:0/2991749657 >> 172.21.15.92:0/2991749657 conn(0x7fd2340719a0 msgr2=0x7fd2340b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:20.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:20.224+0000 7fd23adff700 1 -- 172.21.15.92:0/2991749657 shutdown_connections 2023-11-28T11:23:20.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:20.224+0000 7fd23adff700 1 -- 172.21.15.92:0/2991749657 wait complete. 2023-11-28T11:23:20.243 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-28T11:23:20.243 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T11:23:20.243 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi092.front.sepia.ceph.com 2023-11-28T11:23:20.243 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T11:23:20.243 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T11:23:20.243 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-28T11:23:20.243 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-28T11:23:20.243 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T11:23:20.243 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:23:20.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:23:20.244 DEBUG:teuthology.orchestra.run.smithi092:> ip addr 2023-11-28T11:23:20.292 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:23:20.292 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:23:20.292 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:23:20.292 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:23:20.292 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-11-28T11:23:20.293 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:23:20.293 INFO:teuthology.orchestra.run.smithi092.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:23:20.293 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b2 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:23:20.293 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f0 2023-11-28T11:23:20.293 INFO:teuthology.orchestra.run.smithi092.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:23:20.293 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b3 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:23:20.293 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f1 2023-11-28T11:23:20.294 INFO:teuthology.orchestra.run.smithi092.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:23:20.294 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:44 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:23:20.294 INFO:teuthology.orchestra.run.smithi092.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T11:23:20.294 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:45 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:23:20.294 INFO:teuthology.orchestra.run.smithi092.stdout: inet 172.21.15.92/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T11:23:20.294 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft 41854sec preferred_lft 41854sec 2023-11-28T11:23:20.295 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::ec4:7aff:fe8f:cd45/64 scope link 2023-11-28T11:23:20.295 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:23:20.295 INFO:teuthology.orchestra.run.smithi092.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T11:23:20.295 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 02:f9:a8:fe:83:2e brd ff:ff:ff:ff:ff:ff 2023-11-28T11:23:20.295 INFO:teuthology.orchestra.run.smithi092.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T11:23:20.295 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:23:20.296 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::3ca6:3fff:feb6:2c13/64 scope link 2023-11-28T11:23:20.296 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:23:20.296 INFO:teuthology.orchestra.run.smithi092.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T11:23:20.296 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 4e:8f:e4:5b:18:ce brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T11:23:20.296 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::4c8f:e4ff:fe5b:18ce/64 scope link 2023-11-28T11:23:20.296 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:23:20.296 INFO:teuthology.orchestra.run.smithi092.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T11:23:20.297 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 02:f9:a8:fe:83:2e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T11:23:20.297 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::f9:a8ff:fefe:832e/64 scope link 2023-11-28T11:23:20.297 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:23:20.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:23:20.298 DEBUG:teuthology.orchestra.run.smithi092:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T11:23:20.399 INFO:teuthology.orchestra.run.smithi092.stdout:1 2023-11-28T11:23:20.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:23:20.402 DEBUG:teuthology.orchestra.run.smithi092:> route 2023-11-28T11:23:20.451 INFO:teuthology.orchestra.run.smithi092.stdout:Kernel IP routing table 2023-11-28T11:23:20.451 INFO:teuthology.orchestra.run.smithi092.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T11:23:20.451 INFO:teuthology.orchestra.run.smithi092.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T11:23:20.451 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T11:23:20.451 INFO:teuthology.orchestra.run.smithi092.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T11:23:20.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:23:20.453 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-11-28T11:23:20.453 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-11-28T11:23:20.453 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T11:23:20.453 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T11:23:20.453 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T11:23:20.453 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-11-28T11:23:20.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:23:20.623 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-11-28T11:23:20.671 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T11:23:20.672 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T11:23:20.672 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:23:20.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:23:20.672 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set brx.0 up 2023-11-28T11:23:20.783 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:23:20.783 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-28T11:23:20.830 INFO:teuthology.orchestra.run.smithi092.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:23:20.830 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:23:20.831 DEBUG:teuthology.orchestra.run.smithi092:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-28T11:23:20.916 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-11-28T11:23:20.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:23:20.917 DEBUG:teuthology.orchestra.run.smithi092:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:23:21.062 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:23:21.062 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T11:23:21.062 INFO:teuthology.orchestra.run.smithi092.stdout:invalid new device string format 2023-11-28T11:23:21.078 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:23:21.079 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T11:23:21.079 INFO:teuthology.orchestra.run.smithi092.stdout:invalid new device string format 2023-11-28T11:23:21.079 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.169:3300,172.21.15.169:3301" 2023-11-28T11:23:21.079 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:23:21.101 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:23:21.101 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T11:23:21.102 INFO:teuthology.orchestra.run.smithi092.stdout:invalid new device string format 2023-11-28T11:23:21.102 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.169:3300,172.21.15.169:3301" 2023-11-28T11:23:21.102 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:23:21.102 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:6789,172.21.15.169:6789,172.21.15.169:6790" 2023-11-28T11:23:21.103 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.92:6789,172.21.15.169:6789,172.21.15.169:6790:/ 2023-11-28T11:23:21.103 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=68956e0c-c16f-47cd-a220-f98364b0831d" will pass to kernel 2023-11-28T11:23:21.107 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T11:23:21.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:23:21.109 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T11:23:21.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:23:21.181 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:23:21.284 INFO:teuthology.orchestra.run.smithi092.stdout:client5889 2023-11-28T11:23:21.284 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-28T11:23:21.284 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:23:21.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:23:21.331 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent_a) 2023-11-28T11:23:21.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:23:21.426 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parent_a/alpha) 2023-11-28T11:23:21.474 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T11:23:21.474 DEBUG:teuthology.orchestra.run.smithi092:> import os 2023-11-28T11:23:21.474 DEBUG:teuthology.orchestra.run.smithi092:> import stat 2023-11-28T11:23:21.475 DEBUG:teuthology.orchestra.run.smithi092:> 2023-11-28T11:23:21.475 DEBUG:teuthology.orchestra.run.smithi092:> print(os.stat("/home/ubuntu/cephtest/mnt.0/parent_a/alpha").st_ino) 2023-11-28T11:23:21.475 DEBUG:teuthology.orchestra.run.smithi092:> ' 2023-11-28T11:23:21.589 INFO:teuthology.orchestra.run.smithi092.stdout:1099511627777 2023-11-28T11:23:21.774 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:23:22.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.011+0000 7f4aaa9ff700 1 Processor -- start 2023-11-28T11:23:22.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.011+0000 7f4aaa9ff700 1 -- start start 2023-11-28T11:23:22.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.013+0000 7f4aaa9ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4aa40b8510 0x7f4aa406b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:22.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.013+0000 7f4aaa9ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4aa40cd460 0x7f4aa406beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:22.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.013+0000 7f4aaa9ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4aa40b88e0 con 0x7f4aa406c3f0 2023-11-28T11:23:22.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.013+0000 7f4aaa9ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4aa40cdbd0 con 0x7f4aa40b8510 2023-11-28T11:23:22.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.013+0000 7f4aaa9ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa40d02d0 con 0x7f4aa40cd460 2023-11-28T11:23:22.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.013+0000 7f4aa3fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4aa40b8510 0x7f4aa406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:22.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.013+0000 7f4aa3fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4aa40b8510 0x7f4aa406b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58484/0 (socket says 172.21.15.92:58484) 2023-11-28T11:23:22.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.013+0000 7f4aa3fff700 1 -- 172.21.15.92:0/704749141 learned_addr learned my addr 172.21.15.92:0/704749141 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:22.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.013+0000 7f4aa8f9c700 1 --1- 172.21.15.92:0/704749141 >> v1:172.21.15.92:6789/0 conn(0x7f4aa406c3f0 0x7f4aa415cc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52438/0 (socket says 172.21.15.92:52438) 2023-11-28T11:23:22.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.013+0000 7f4aa37fe700 1 --2- 172.21.15.92:0/704749141 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4aa40cd460 0x7f4aa406beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:22.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.014+0000 7f4aa3fff700 1 -- 172.21.15.92:0/704749141 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4aa40cd460 msgr2=0x7f4aa406beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:22.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.014+0000 7f4aa3fff700 1 --2- 172.21.15.92:0/704749141 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4aa40cd460 0x7f4aa406beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:22.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.014+0000 7f4aa3fff700 1 -- 172.21.15.92:0/704749141 >> v1:172.21.15.92:6789/0 conn(0x7f4aa406c3f0 legacy=0x7f4aa415cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:22.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.014+0000 7f4aa3fff700 1 -- 172.21.15.92:0/704749141 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aa40cd8f0 con 0x7f4aa40b8510 2023-11-28T11:23:22.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.014+0000 7f4aa3fff700 1 --2- 172.21.15.92:0/704749141 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4aa40b8510 0x7f4aa406b970 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f4a94009920 tx=0x7f4a94009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5efc0a646e87da server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:22.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.014+0000 7f4aa2ffd700 1 -- 172.21.15.92:0/704749141 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9400ba20 con 0x7f4aa40b8510 2023-11-28T11:23:22.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.014+0000 7f4aa2ffd700 1 -- 172.21.15.92:0/704749141 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a9400bb80 con 0x7f4aa40b8510 2023-11-28T11:23:22.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.014+0000 7f4aa2ffd700 1 -- 172.21.15.92:0/704749141 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a94009e70 con 0x7f4aa40b8510 2023-11-28T11:23:22.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.014+0000 7f4aaa9ff700 1 -- 172.21.15.92:0/704749141 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4aa40b8510 msgr2=0x7f4aa406b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:22.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.014+0000 7f4aaa9ff700 1 --2- 172.21.15.92:0/704749141 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4aa40b8510 0x7f4aa406b970 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f4a94009920 tx=0x7f4a94009c30 comp rx=0 tx=0).stop 2023-11-28T11:23:22.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.015+0000 7f4aaa9ff700 1 -- 172.21.15.92:0/704749141 shutdown_connections 2023-11-28T11:23:22.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.015+0000 7f4aaa9ff700 1 --2- 172.21.15.92:0/704749141 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4aa40b8510 0x7f4aa406b970 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:22.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.015+0000 7f4aaa9ff700 1 --2- 172.21.15.92:0/704749141 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4aa40cd460 0x7f4aa406beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:22.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.015+0000 7f4aaa9ff700 1 -- 172.21.15.92:0/704749141 >> 172.21.15.92:0/704749141 conn(0x7f4aa40719a0 msgr2=0x7f4aa4071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:22.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.015+0000 7f4aaa9ff700 1 -- 172.21.15.92:0/704749141 shutdown_connections 2023-11-28T11:23:22.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.015+0000 7f4aaa9ff700 1 -- 172.21.15.92:0/704749141 wait complete. 2023-11-28T11:23:22.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.017+0000 7f4aaa9ff700 1 Processor -- start 2023-11-28T11:23:22.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.017+0000 7f4aaa9ff700 1 -- start start 2023-11-28T11:23:22.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.018+0000 7f4aaa9ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4aa406c3f0 0x7f4aa415ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:22.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.018+0000 7f4aaa9ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4aa40b8510 0x7f4aa41e7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:22.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.018+0000 7f4aaa9ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4aa40cd460 0x7f4aa41e7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:22.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.018+0000 7f4aaa9ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa41f53b0 con 0x7f4aa40cd460 2023-11-28T11:23:22.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.018+0000 7f4aaa9ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa41e5d30 con 0x7f4aa406c3f0 2023-11-28T11:23:22.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.018+0000 7f4aaa9ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa41e6010 con 0x7f4aa40b8510 2023-11-28T11:23:22.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.019+0000 7f4aa8f9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4aa40cd460 0x7f4aa41e7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:22.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.019+0000 7f4aa8f9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4aa40cd460 0x7f4aa41e7b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58500/0 (socket says 172.21.15.92:58500) 2023-11-28T11:23:22.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.019+0000 7f4aa8f9c700 1 -- 172.21.15.92:0/372595696 learned_addr learned my addr 172.21.15.92:0/372595696 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:22.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.019+0000 7f4aa3fff700 1 --2- 172.21.15.92:0/372595696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4aa406c3f0 0x7f4aa415ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:22.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.019+0000 7f4aa37fe700 1 --2- 172.21.15.92:0/372595696 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4aa40b8510 0x7f4aa41e7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:22.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.019+0000 7f4aa8f9c700 1 -- 172.21.15.92:0/372595696 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4aa40b8510 msgr2=0x7f4aa41e7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:22.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.019+0000 7f4aa8f9c700 1 --2- 172.21.15.92:0/372595696 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4aa40b8510 0x7f4aa41e7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:22.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.019+0000 7f4aa8f9c700 1 -- 172.21.15.92:0/372595696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4aa406c3f0 msgr2=0x7f4aa415ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:22.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.019+0000 7f4aa8f9c700 1 --2- 172.21.15.92:0/372595696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4aa406c3f0 0x7f4aa415ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:22.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.019+0000 7f4aa8f9c700 1 -- 172.21.15.92:0/372595696 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a94009550 con 0x7f4aa40cd460 2023-11-28T11:23:22.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.019+0000 7f4aa8f9c700 1 --2- 172.21.15.92:0/372595696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4aa40cd460 0x7f4aa41e7b50 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c00cbd0 tx=0x7f4a9c00cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:22.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.020+0000 7f4aa0ff9700 1 -- 172.21.15.92:0/372595696 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c00aaa0 con 0x7f4aa40cd460 2023-11-28T11:23:22.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.020+0000 7f4aa0ff9700 1 -- 172.21.15.92:0/372595696 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a9c00ac00 con 0x7f4aa40cd460 2023-11-28T11:23:22.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.020+0000 7f4aaa9ff700 1 -- 172.21.15.92:0/372595696 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aa40cfff0 con 0x7f4aa40cd460 2023-11-28T11:23:22.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.020+0000 7f4aa0ff9700 1 -- 172.21.15.92:0/372595696 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c007070 con 0x7f4aa40cd460 2023-11-28T11:23:22.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.020+0000 7f4aaa9ff700 1 -- 172.21.15.92:0/372595696 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aa40cfd10 con 0x7f4aa40cd460 2023-11-28T11:23:22.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.021+0000 7f4aaa9ff700 1 -- 172.21.15.92:0/372595696 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4aa40b4320 con 0x7f4aa40cd460 2023-11-28T11:23:22.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.022+0000 7f4aa0ff9700 1 -- 172.21.15.92:0/372595696 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4a9c019030 con 0x7f4aa40cd460 2023-11-28T11:23:22.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.023+0000 7f4aa0ff9700 1 --2- 172.21.15.92:0/372595696 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f4a900682f0 0x7f4a9006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:22.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.023+0000 7f4aa0ff9700 1 -- 172.21.15.92:0/372595696 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f4a9c085950 con 0x7f4aa40cd460 2023-11-28T11:23:22.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.023+0000 7f4aa3fff700 1 --2- 172.21.15.92:0/372595696 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f4a900682f0 0x7f4a9006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:22.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.024+0000 7f4aa3fff700 1 --2- 172.21.15.92:0/372595696 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f4a900682f0 0x7f4a9006a7a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4a94009c70 tx=0x7f4a94008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:22.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.027+0000 7f4aa0ff9700 1 -- 172.21.15.92:0/372595696 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4a9c04b5e0 con 0x7f4aa40cd460 2023-11-28T11:23:22.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.259+0000 7f4aaa9ff700 1 -- 172.21.15.92:0/372595696 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4aa4067880 con 0x7f4aa40cd460 2023-11-28T11:23:22.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.259+0000 7f4aa0ff9700 1 -- 172.21.15.92:0/372595696 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4651 (secure 0 0 0) 0x7f4a9c017090 con 0x7f4aa40cd460 2023-11-28T11:23:22.260 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:23:22.260 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5593,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/3937189339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3937189339},{"type":"v1","addr":"172.21.15.169:6834","nonce":3937189339}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5596,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2020201016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2020201016},{"type":"v1","addr":"172.21.15.169:6835","nonce":2020201016}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5751,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2423750324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2423750324},{"type":"v1","addr":"172.21.15.92:6835","nonce":2423750324}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:23:13.247233+0000","modified":"2023-11-28T11:23:16.254256+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5754},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5754":{"gid":5754,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/950691230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":950691230},{"type":"v1","addr":"172.21.15.92:6837","nonce":950691230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T11:23:22.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.261+0000 7f4aaa9ff700 1 -- 172.21.15.92:0/372595696 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f4a900682f0 msgr2=0x7f4a9006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:22.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.261+0000 7f4aaa9ff700 1 --2- 172.21.15.92:0/372595696 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f4a900682f0 0x7f4a9006a7a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4a94009c70 tx=0x7f4a94008030 comp rx=0 tx=0).stop 2023-11-28T11:23:22.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.261+0000 7f4aaa9ff700 1 -- 172.21.15.92:0/372595696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4aa40cd460 msgr2=0x7f4aa41e7b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:22.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.261+0000 7f4aaa9ff700 1 --2- 172.21.15.92:0/372595696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4aa40cd460 0x7f4aa41e7b50 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c00cbd0 tx=0x7f4a9c00cee0 comp rx=0 tx=0).stop 2023-11-28T11:23:22.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.261+0000 7f4aaa9ff700 1 -- 172.21.15.92:0/372595696 shutdown_connections 2023-11-28T11:23:22.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.261+0000 7f4aaa9ff700 1 --2- 172.21.15.92:0/372595696 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f4a900682f0 0x7f4a9006a7a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:22.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.261+0000 7f4aaa9ff700 1 --2- 172.21.15.92:0/372595696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4aa40cd460 0x7f4aa41e7b50 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:22.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.261+0000 7f4aaa9ff700 1 --2- 172.21.15.92:0/372595696 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4aa406c3f0 0x7f4aa415ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:22.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.261+0000 7f4aaa9ff700 1 --2- 172.21.15.92:0/372595696 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4aa40b8510 0x7f4aa41e7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:22.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.261+0000 7f4aaa9ff700 1 -- 172.21.15.92:0/372595696 >> 172.21.15.92:0/372595696 conn(0x7f4aa40719a0 msgr2=0x7f4aa40cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:22.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.262+0000 7f4aaa9ff700 1 -- 172.21.15.92:0/372595696 shutdown_connections 2023-11-28T11:23:22.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.262+0000 7f4aaa9ff700 1 -- 172.21.15.92:0/372595696 wait complete. 2023-11-28T11:23:22.264 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 24 2023-11-28T11:23:22.281 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2023-11-28T11:23:22.513 INFO:teuthology.orchestra.run.smithi092.stdout:{"message":"","return_code":0} 2023-11-28T11:23:22.514 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-28T11:23:22.514 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2023-11-28T11:23:22.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.602+0000 7f6d65e640c0 1 Processor -- start 2023-11-28T11:23:22.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.602+0000 7f6d65e640c0 1 -- start start 2023-11-28T11:23:22.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.602+0000 7f6d65e640c0 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55659511f520 0x55659514b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:22.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.603+0000 7f6d65e640c0 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5565951373d0 0x5565951f1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:22.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.603+0000 7f6d65e640c0 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556595138030 0x5565951f2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:22.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.603+0000 7f6d65e640c0 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x556595059fe0 con 0x55659511f520 2023-11-28T11:23:22.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.603+0000 7f6d65e640c0 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55659514ff80 con 0x556595138030 2023-11-28T11:23:22.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.603+0000 7f6d65e640c0 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5565951504d0 con 0x5565951373d0 2023-11-28T11:23:22.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.603+0000 7f6d5759b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55659511f520 0x55659514b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:22.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.603+0000 7f6d5759b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55659511f520 0x55659514b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58512/0 (socket says 172.21.15.92:58512) 2023-11-28T11:23:22.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.603+0000 7f6d5759b700 1 -- 172.21.15.92:0/3229535823 learned_addr learned my addr 172.21.15.92:0/3229535823 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:22.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.603+0000 7f6d56d9a700 1 --2- 172.21.15.92:0/3229535823 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5565951373d0 0x5565951f1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:22.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.603+0000 7f6d57d9c700 1 --2- 172.21.15.92:0/3229535823 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556595138030 0x5565951f2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:22.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.603+0000 7f6d5759b700 1 -- 172.21.15.92:0/3229535823 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5565951373d0 msgr2=0x5565951f1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:22.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.603+0000 7f6d5759b700 1 --2- 172.21.15.92:0/3229535823 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5565951373d0 0x5565951f1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:22.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.603+0000 7f6d5759b700 1 -- 172.21.15.92:0/3229535823 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556595138030 msgr2=0x5565951f2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:22.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.603+0000 7f6d5759b700 1 --2- 172.21.15.92:0/3229535823 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556595138030 0x5565951f2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:22.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.603+0000 7f6d5759b700 1 -- 172.21.15.92:0/3229535823 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d5000a100 con 0x55659511f520 2023-11-28T11:23:22.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.603+0000 7f6d5759b700 1 --2- 172.21.15.92:0/3229535823 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55659511f520 0x55659514b170 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f6d48001d30 tx=0x7f6d4800b510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=545928fb81bfe851 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:22.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.604+0000 7f6d55d98700 1 -- 172.21.15.92:0/3229535823 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d48014070 con 0x55659511f520 2023-11-28T11:23:22.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.604+0000 7f6d55d98700 1 -- 172.21.15.92:0/3229535823 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d4800bd00 con 0x55659511f520 2023-11-28T11:23:22.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.604+0000 7f6d55d98700 1 -- 172.21.15.92:0/3229535823 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4800f690 con 0x55659511f520 2023-11-28T11:23:22.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.604+0000 7f6d65e640c0 1 -- 172.21.15.92:0/3229535823 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55659511f520 msgr2=0x55659514b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:22.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.604+0000 7f6d65e640c0 1 --2- 172.21.15.92:0/3229535823 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55659511f520 0x55659514b170 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f6d48001d30 tx=0x7f6d4800b510 comp rx=0 tx=0).stop 2023-11-28T11:23:22.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.604+0000 7f6d65e640c0 1 -- 172.21.15.92:0/3229535823 shutdown_connections 2023-11-28T11:23:22.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.604+0000 7f6d65e640c0 1 --2- 172.21.15.92:0/3229535823 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x556595138030 0x5565951f2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:22.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.604+0000 7f6d65e640c0 1 --2- 172.21.15.92:0/3229535823 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55659511f520 0x55659514b170 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:22.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.604+0000 7f6d65e640c0 1 --2- 172.21.15.92:0/3229535823 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5565951373d0 0x5565951f1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:22.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.604+0000 7f6d65e640c0 1 -- 172.21.15.92:0/3229535823 >> 172.21.15.92:0/3229535823 conn(0x55659516a670 msgr2=0x556595121ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:22.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.604+0000 7f6d65e640c0 1 -- 172.21.15.92:0/3229535823 shutdown_connections 2023-11-28T11:23:22.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.604+0000 7f6d65e640c0 1 -- 172.21.15.92:0/3229535823 wait complete. 2023-11-28T11:23:22.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.605+0000 7f6d65e640c0 1 Processor -- start 2023-11-28T11:23:22.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.605+0000 7f6d65e640c0 1 -- start start 2023-11-28T11:23:22.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.606+0000 7f6d65e640c0 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55659511f520 0x5565951ee320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:22.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.606+0000 7f6d65e640c0 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565951373d0 0x5565951efdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:22.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.606+0000 7f6d65e640c0 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556595138030 0x55659511ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:22.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.606+0000 7f6d65e640c0 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565951ef930 con 0x5565951373d0 2023-11-28T11:23:22.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.606+0000 7f6d65e640c0 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565951eea40 con 0x55659511f520 2023-11-28T11:23:22.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.606+0000 7f6d65e640c0 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x556595288360 con 0x556595138030 2023-11-28T11:23:22.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.606+0000 7f6d56d9a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565951373d0 0x5565951efdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:22.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.606+0000 7f6d56d9a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565951373d0 0x5565951efdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58522/0 (socket says 172.21.15.92:58522) 2023-11-28T11:23:22.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.606+0000 7f6d56d9a700 1 -- 172.21.15.92:0/1646386070 learned_addr learned my addr 172.21.15.92:0/1646386070 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:22.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.607+0000 7f6d56d9a700 1 -- 172.21.15.92:0/1646386070 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556595138030 msgr2=0x55659511ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:22.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.607+0000 7f6d56d9a700 1 --2- 172.21.15.92:0/1646386070 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x556595138030 0x55659511ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:22.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.607+0000 7f6d56d9a700 1 -- 172.21.15.92:0/1646386070 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55659511f520 msgr2=0x5565951ee320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:22.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.607+0000 7f6d56d9a700 1 --2- 172.21.15.92:0/1646386070 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55659511f520 0x5565951ee320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:22.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.607+0000 7f6d56d9a700 1 -- 172.21.15.92:0/1646386070 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565950654a0 con 0x5565951373d0 2023-11-28T11:23:22.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.607+0000 7f6d56d9a700 1 --2- 172.21.15.92:0/1646386070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565951373d0 0x5565951efdc0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c011000 tx=0x7f6d4c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:22.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.607+0000 7f6d54d96700 1 -- 172.21.15.92:0/1646386070 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c012820 con 0x5565951373d0 2023-11-28T11:23:22.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.607+0000 7f6d54d96700 1 -- 172.21.15.92:0/1646386070 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d4c012980 con 0x5565951373d0 2023-11-28T11:23:22.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.607+0000 7f6d54d96700 1 -- 172.21.15.92:0/1646386070 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c012d30 con 0x5565951373d0 2023-11-28T11:23:22.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.607+0000 7f6d65e640c0 1 -- 172.21.15.92:0/1646386070 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556595065830 con 0x5565951373d0 2023-11-28T11:23:22.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.607+0000 7f6d65e640c0 1 -- 172.21.15.92:0/1646386070 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d4c005de0 con 0x5565951373d0 2023-11-28T11:23:22.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.609+0000 7f6d54d96700 1 -- 172.21.15.92:0/1646386070 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6d4c027410 con 0x5565951373d0 2023-11-28T11:23:22.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.610+0000 7f6d54d96700 1 --2- 172.21.15.92:0/1646386070 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6d44066c10 0x7f6d440690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:22.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.610+0000 7f6d54d96700 1 -- 172.21.15.92:0/1646386070 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f6d4c086700 con 0x5565951373d0 2023-11-28T11:23:22.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.610+0000 7f6d5759b700 1 --2- 172.21.15.92:0/1646386070 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6d44066c10 0x7f6d440690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:22.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.610+0000 7f6d65e640c0 1 --2- 172.21.15.92:0/1646386070 >> [v2:172.21.15.169:6800/116033,v1:172.21.15.169:6801/116033] conn(0x556595287ef0 0x556595285530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:22.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.610+0000 7f6d65e640c0 1 -- 172.21.15.92:0/1646386070 --> [v2:172.21.15.169:6800/116033,v1:172.21.15.169:6801/116033] -- osd_op(unknown.0.0:1 9.7 9:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56) v8 -- 0x556595285a70 con 0x556595287ef0 2023-11-28T11:23:22.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.611+0000 7f6d5759b700 1 --2- 172.21.15.92:0/1646386070 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6d44066c10 0x7f6d440690c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f6d48001d30 tx=0x7f6d48006000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:22.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.611+0000 7f6d57d9c700 1 --2- 172.21.15.92:0/1646386070 >> [v2:172.21.15.169:6800/116033,v1:172.21.15.169:6801/116033] conn(0x556595287ef0 0x556595285530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:22.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.611+0000 7f6d57d9c700 1 --2- 172.21.15.92:0/1646386070 >> [v2:172.21.15.169:6800/116033,v1:172.21.15.169:6801/116033] conn(0x556595287ef0 0x556595285530 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:22.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.612+0000 7f6d57d9c700 1 -- 172.21.15.92:0/1646386070 <== osd.5 v2:172.21.15.169:6800/116033 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x7f6d5000a920 con 0x556595287ef0 2023-11-28T11:23:22.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.612+0000 7f6d65e640c0 1 -- 172.21.15.92:0/1646386070 >> [v2:172.21.15.169:6800/116033,v1:172.21.15.169:6801/116033] conn(0x556595287ef0 msgr2=0x556595285530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:22.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.612+0000 7f6d65e640c0 1 --2- 172.21.15.92:0/1646386070 >> [v2:172.21.15.169:6800/116033,v1:172.21.15.169:6801/116033] conn(0x556595287ef0 0x556595285530 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:22.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.612+0000 7f6d65e640c0 1 -- 172.21.15.92:0/1646386070 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6d44066c10 msgr2=0x7f6d440690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:22.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.612+0000 7f6d65e640c0 1 --2- 172.21.15.92:0/1646386070 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6d44066c10 0x7f6d440690c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f6d48001d30 tx=0x7f6d48006000 comp rx=0 tx=0).stop 2023-11-28T11:23:22.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.612+0000 7f6d65e640c0 1 -- 172.21.15.92:0/1646386070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565951373d0 msgr2=0x5565951efdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:22.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.612+0000 7f6d65e640c0 1 --2- 172.21.15.92:0/1646386070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5565951373d0 0x5565951efdc0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c011000 tx=0x7f6d4c002fb0 comp rx=0 tx=0).stop 2023-11-28T11:23:22.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.612+0000 7f6d57d9c700 1 -- 172.21.15.92:0/1646386070 reap_dead start 2023-11-28T11:23:22.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.612+0000 7f6d65e640c0 1 -- 172.21.15.92:0/1646386070 shutdown_connections 2023-11-28T11:23:22.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.612+0000 7f6d65e640c0 1 -- 172.21.15.92:0/1646386070 >> 172.21.15.92:0/1646386070 conn(0x55659516a670 msgr2=0x5565951f2170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:22.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.612+0000 7f6d65e640c0 1 -- 172.21.15.92:0/1646386070 shutdown_connections 2023-11-28T11:23:22.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.612+0000 7f6d65e640c0 1 -- 172.21.15.92:0/1646386070 wait complete. 2023-11-28T11:23:22.623 INFO:teuthology.orchestra.run.smithi092.stdout:Yalphaparent_a 2023-11-28T11:23:22.624 DEBUG:teuthology.orchestra.run.smithi092:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2023-11-28T11:23:22.788 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-28T11:23:22.789 INFO:teuthology.orchestra.run.smithi092.stdout: "ino": 1099511627777, 2023-11-28T11:23:22.789 INFO:teuthology.orchestra.run.smithi092.stdout: "ancestors": [ 2023-11-28T11:23:22.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:23:22.789 INFO:teuthology.orchestra.run.smithi092.stdout: "dirino": 1099511627776, 2023-11-28T11:23:22.789 INFO:teuthology.orchestra.run.smithi092.stdout: "dname": "alpha", 2023-11-28T11:23:22.789 INFO:teuthology.orchestra.run.smithi092.stdout: "version": 2 2023-11-28T11:23:22.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:23:22.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:23:22.789 INFO:teuthology.orchestra.run.smithi092.stdout: "dirino": 1, 2023-11-28T11:23:22.789 INFO:teuthology.orchestra.run.smithi092.stdout: "dname": "parent_a", 2023-11-28T11:23:22.789 INFO:teuthology.orchestra.run.smithi092.stdout: "version": 4 2023-11-28T11:23:22.790 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:23:22.790 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2023-11-28T11:23:22.790 INFO:teuthology.orchestra.run.smithi092.stdout: "pool": 9, 2023-11-28T11:23:22.790 INFO:teuthology.orchestra.run.smithi092.stdout: "old_pools": [] 2023-11-28T11:23:22.790 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-28T11:23:22.790 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:23:22.793 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'oh i'"'"'m sorry did i overwrite your xattr?' 2023-11-28T11:23:22.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.881+0000 7f66124bb0c0 1 Processor -- start 2023-11-28T11:23:22.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.881+0000 7f66124bb0c0 1 -- start start 2023-11-28T11:23:22.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.882+0000 7f66124bb0c0 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560c7ee10b20 0x560c7ee3a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:22.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.882+0000 7f66124bb0c0 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560c7ee3cf90 0x560c7eede380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:22.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.882+0000 7f66124bb0c0 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c7eedf870 0x560c7ee0c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:22.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.882+0000 7f66124bb0c0 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c7edb5000 con 0x560c7eedf870 2023-11-28T11:23:22.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.882+0000 7f66124bb0c0 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c7ed47fe0 con 0x560c7ee3cf90 2023-11-28T11:23:22.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.882+0000 7f66124bb0c0 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x560c7ee3e1e0 con 0x560c7ee10b20 2023-11-28T11:23:22.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.882+0000 7f660439c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c7eedf870 0x560c7ee0c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:22.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.882+0000 7f660439c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c7eedf870 0x560c7ee0c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58544/0 (socket says 172.21.15.92:58544) 2023-11-28T11:23:22.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.882+0000 7f660439c700 1 -- 172.21.15.92:0/1372611144 learned_addr learned my addr 172.21.15.92:0/1372611144 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:22.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.883+0000 7f660439c700 1 -- 172.21.15.92:0/1372611144 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560c7ee10b20 msgr2=0x560c7ee3a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:22.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.883+0000 7f660439c700 1 --2- 172.21.15.92:0/1372611144 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560c7ee10b20 0x560c7ee3a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:22.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.883+0000 7f660439c700 1 -- 172.21.15.92:0/1372611144 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560c7ee3cf90 msgr2=0x560c7eede380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:22.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.883+0000 7f660439c700 1 --2- 172.21.15.92:0/1372611144 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560c7ee3cf90 0x560c7eede380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:22.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.883+0000 7f660439c700 1 -- 172.21.15.92:0/1372611144 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65fc00a100 con 0x560c7eedf870 2023-11-28T11:23:22.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.883+0000 7f660439c700 1 --2- 172.21.15.92:0/1372611144 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c7eedf870 0x560c7ee0c6d0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f65fc012040 tx=0x7f65fc00f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edf8ad4e8a950a6d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:22.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.883+0000 7f6602398700 1 -- 172.21.15.92:0/1372611144 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65fc00fa60 con 0x560c7eedf870 2023-11-28T11:23:22.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.883+0000 7f6602398700 1 -- 172.21.15.92:0/1372611144 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65fc00fbc0 con 0x560c7eedf870 2023-11-28T11:23:22.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.883+0000 7f6602398700 1 -- 172.21.15.92:0/1372611144 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65fc022440 con 0x560c7eedf870 2023-11-28T11:23:22.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.883+0000 7f66124bb0c0 1 -- 172.21.15.92:0/1372611144 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c7eedf870 msgr2=0x560c7ee0c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:22.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.883+0000 7f66124bb0c0 1 --2- 172.21.15.92:0/1372611144 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c7eedf870 0x560c7ee0c6d0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f65fc012040 tx=0x7f65fc00f000 comp rx=0 tx=0).stop 2023-11-28T11:23:22.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.884+0000 7f66124bb0c0 1 -- 172.21.15.92:0/1372611144 shutdown_connections 2023-11-28T11:23:22.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.884+0000 7f66124bb0c0 1 --2- 172.21.15.92:0/1372611144 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c7eedf870 0x560c7ee0c6d0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:22.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.884+0000 7f66124bb0c0 1 --2- 172.21.15.92:0/1372611144 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560c7ee10b20 0x560c7ee3a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:22.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.884+0000 7f66124bb0c0 1 --2- 172.21.15.92:0/1372611144 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560c7ee3cf90 0x560c7eede380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:22.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.884+0000 7f66124bb0c0 1 -- 172.21.15.92:0/1372611144 >> 172.21.15.92:0/1372611144 conn(0x560c7ee56f30 msgr2=0x560c7ee59ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:22.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.884+0000 7f66124bb0c0 1 -- 172.21.15.92:0/1372611144 shutdown_connections 2023-11-28T11:23:22.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.884+0000 7f66124bb0c0 1 -- 172.21.15.92:0/1372611144 wait complete. 2023-11-28T11:23:22.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.884+0000 7f66124bb0c0 1 Processor -- start 2023-11-28T11:23:22.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.885+0000 7f66124bb0c0 1 -- start start 2023-11-28T11:23:22.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.885+0000 7f66124bb0c0 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c7ee10b20 0x560c7ee59310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:22.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.885+0000 7f66124bb0c0 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560c7ee3cf90 0x560c7eedcff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:22.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.885+0000 7f66124bb0c0 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560c7eedf870 0x560c7eed1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:22.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.885+0000 7f66124bb0c0 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c7ee59850 con 0x560c7ee10b20 2023-11-28T11:23:22.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.885+0000 7f66124bb0c0 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c7eedba20 con 0x560c7eedf870 2023-11-28T11:23:22.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.885+0000 7f66124bb0c0 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x560c7eed5a10 con 0x560c7ee3cf90 2023-11-28T11:23:22.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.886+0000 7f6603b9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c7ee10b20 0x560c7ee59310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:22.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.886+0000 7f6603b9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c7ee10b20 0x560c7ee59310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58546/0 (socket says 172.21.15.92:58546) 2023-11-28T11:23:22.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.886+0000 7f6603b9b700 1 -- 172.21.15.92:0/1954564995 learned_addr learned my addr 172.21.15.92:0/1954564995 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:22.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.886+0000 7f660439c700 1 --2- 172.21.15.92:0/1954564995 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560c7eedf870 0x560c7eed1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:22.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.886+0000 7f660339a700 1 --2- 172.21.15.92:0/1954564995 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560c7ee3cf90 0x560c7eedcff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:22.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.886+0000 7f6603b9b700 1 -- 172.21.15.92:0/1954564995 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560c7ee3cf90 msgr2=0x560c7eedcff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:22.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.886+0000 7f6603b9b700 1 --2- 172.21.15.92:0/1954564995 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560c7ee3cf90 0x560c7eedcff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:22.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.886+0000 7f6603b9b700 1 -- 172.21.15.92:0/1954564995 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560c7eedf870 msgr2=0x560c7eed1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:22.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.886+0000 7f6603b9b700 1 --2- 172.21.15.92:0/1954564995 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560c7eedf870 0x560c7eed1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:22.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.886+0000 7f6603b9b700 1 -- 172.21.15.92:0/1954564995 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c7ed60120 con 0x560c7ee10b20 2023-11-28T11:23:22.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.886+0000 7f660339a700 1 --2- 172.21.15.92:0/1954564995 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560c7ee3cf90 0x560c7eedcff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:23:22.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.886+0000 7f6603b9b700 1 --2- 172.21.15.92:0/1954564995 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c7ee10b20 0x560c7ee59310 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f65f400ab80 tx=0x7f65f400ae90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:22.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.887+0000 7f6601396700 1 -- 172.21.15.92:0/1954564995 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f4008da0 con 0x560c7ee10b20 2023-11-28T11:23:22.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.887+0000 7f6601396700 1 -- 172.21.15.92:0/1954564995 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65f4014470 con 0x560c7ee10b20 2023-11-28T11:23:22.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.887+0000 7f66124bb0c0 1 -- 172.21.15.92:0/1954564995 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560c7ed53830 con 0x560c7ee10b20 2023-11-28T11:23:22.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.887+0000 7f6601396700 1 -- 172.21.15.92:0/1954564995 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f4014820 con 0x560c7ee10b20 2023-11-28T11:23:22.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.887+0000 7f66124bb0c0 1 -- 172.21.15.92:0/1954564995 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560c7ed60360 con 0x560c7ee10b20 2023-11-28T11:23:22.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.888+0000 7f6601396700 1 -- 172.21.15.92:0/1954564995 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f65f401e3f0 con 0x560c7ee10b20 2023-11-28T11:23:22.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.889+0000 7f6601396700 1 --2- 172.21.15.92:0/1954564995 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f65f0066c10 0x7f65f00690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:22.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.889+0000 7f6601396700 1 -- 172.21.15.92:0/1954564995 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f65f40846b0 con 0x560c7ee10b20 2023-11-28T11:23:22.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.890+0000 7f660339a700 1 --2- 172.21.15.92:0/1954564995 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f65f0066c10 0x7f65f00690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:22.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.890+0000 7f66124bb0c0 1 --2- 172.21.15.92:0/1954564995 >> [v2:172.21.15.169:6800/116033,v1:172.21.15.169:6801/116033] conn(0x560c7eedd530 0x560c7ef87e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:22.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.890+0000 7f66124bb0c0 1 -- 172.21.15.92:0/1954564995 --> [v2:172.21.15.169:6800/116033,v1:172.21.15.169:6801/116033] -- osd_op(unknown.0.0:1 9.7 9:e2e07930:::10000000001.00000000:head [setxattr parent (40) in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56) v8 -- 0x560c7eed6ee0 con 0x560c7eedd530 2023-11-28T11:23:22.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.890+0000 7f660439c700 1 --2- 172.21.15.92:0/1954564995 >> [v2:172.21.15.169:6800/116033,v1:172.21.15.169:6801/116033] conn(0x560c7eedd530 0x560c7ef87e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:22.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.890+0000 7f660339a700 1 --2- 172.21.15.92:0/1954564995 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f65f0066c10 0x7f65f00690c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x560c7ee3a520 tx=0x7f65f80015d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:22.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.891+0000 7f660439c700 1 --2- 172.21.15.92:0/1954564995 >> [v2:172.21.15.169:6800/116033,v1:172.21.15.169:6801/116033] conn(0x560c7eedd530 0x560c7ef87e00 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:22.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.892+0000 7f660439c700 1 -- 172.21.15.92:0/1954564995 <== osd.5 v2:172.21.15.169:6800/116033 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (40)] v56'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f65fc016020 con 0x560c7eedd530 2023-11-28T11:23:22.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.892+0000 7f66124bb0c0 1 -- 172.21.15.92:0/1954564995 >> [v2:172.21.15.169:6800/116033,v1:172.21.15.169:6801/116033] conn(0x560c7eedd530 msgr2=0x560c7ef87e00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:22.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.892+0000 7f66124bb0c0 1 --2- 172.21.15.92:0/1954564995 >> [v2:172.21.15.169:6800/116033,v1:172.21.15.169:6801/116033] conn(0x560c7eedd530 0x560c7ef87e00 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:22.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.892+0000 7f66124bb0c0 1 -- 172.21.15.92:0/1954564995 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f65f0066c10 msgr2=0x7f65f00690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:22.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.893+0000 7f66124bb0c0 1 --2- 172.21.15.92:0/1954564995 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f65f0066c10 0x7f65f00690c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x560c7ee3a520 tx=0x7f65f80015d0 comp rx=0 tx=0).stop 2023-11-28T11:23:22.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.893+0000 7f66124bb0c0 1 -- 172.21.15.92:0/1954564995 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c7ee10b20 msgr2=0x560c7ee59310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:22.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.893+0000 7f66124bb0c0 1 --2- 172.21.15.92:0/1954564995 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c7ee10b20 0x560c7ee59310 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f65f400ab80 tx=0x7f65f400ae90 comp rx=0 tx=0).stop 2023-11-28T11:23:22.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.893+0000 7f660439c700 1 -- 172.21.15.92:0/1954564995 reap_dead start 2023-11-28T11:23:22.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.893+0000 7f66124bb0c0 1 -- 172.21.15.92:0/1954564995 shutdown_connections 2023-11-28T11:23:22.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.893+0000 7f66124bb0c0 1 -- 172.21.15.92:0/1954564995 >> 172.21.15.92:0/1954564995 conn(0x560c7ee56f30 msgr2=0x560c7ee0e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:22.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.893+0000 7f66124bb0c0 1 -- 172.21.15.92:0/1954564995 shutdown_connections 2023-11-28T11:23:22.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:22.893+0000 7f66124bb0c0 1 -- 172.21.15.92:0/1954564995 wait complete. 2023-11-28T11:23:22.898 DEBUG:teuthology.orchestra.run.smithi092:> sudo daemon-helper kill ceph --cluster ceph -w 2023-11-28T11:23:22.902 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start / repair,recursive 2023-11-28T11:23:23.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.105+0000 7f9f4f5ff700 1 Processor -- start 2023-11-28T11:23:23.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.106+0000 7f9f4f5ff700 1 -- start start 2023-11-28T11:23:23.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.107+0000 7f9f4f5ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9f480cd460 0x7f9f4806b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:23.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.107+0000 7f9f4f5ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f4806be80 0x7f9f480d0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:23.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.107+0000 7f9f4f5ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f480b5240 con 0x7f9f480b8790 2023-11-28T11:23:23.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.107+0000 7f9f4f5ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f4806c300 con 0x7f9f480cd460 2023-11-28T11:23:23.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.107+0000 7f9f4f5ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f480b5520 con 0x7f9f4806be80 2023-11-28T11:23:23.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.107+0000 7f9f4db9c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f4806be80 0x7f9f480d0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:23.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.107+0000 7f9f4db9c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f4806be80 0x7f9f480d0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.92:35840/0 (socket says 172.21.15.92:35840) 2023-11-28T11:23:23.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.107+0000 7f9f4db9c700 1 -- 172.21.15.92:0/1740875717 learned_addr learned my addr 172.21.15.92:0/1740875717 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:23.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.107+0000 7f9f4cb9a700 1 --2- 172.21.15.92:0/1740875717 >> v2:172.21.15.169:3300/0 conn(0x7f9f480cd460 0x7f9f4806b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:23.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.108+0000 7f9f3ffff700 1 -- 172.21.15.92:0/1740875717 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1772563403 0 0) 0x7f9f480b5240 con 0x7f9f480b8790 2023-11-28T11:23:23.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.108+0000 7f9f4db9c700 1 -- 172.21.15.92:0/1740875717 >> v2:172.21.15.169:3300/0 conn(0x7f9f480cd460 msgr2=0x7f9f4806b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:23.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.108+0000 7f9f4db9c700 1 --2- 172.21.15.92:0/1740875717 >> v2:172.21.15.169:3300/0 conn(0x7f9f480cd460 0x7f9f4806b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.108+0000 7f9f4db9c700 1 -- 172.21.15.92:0/1740875717 >> v1:172.21.15.169:6789/0 conn(0x7f9f480b8790 legacy=0x7f9f480cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:23.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.108+0000 7f9f4db9c700 1 -- 172.21.15.92:0/1740875717 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f480d0a80 con 0x7f9f4806be80 2023-11-28T11:23:23.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.108+0000 7f9f4db9c700 1 --2- 172.21.15.92:0/1740875717 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f4806be80 0x7f9f480d0fa0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9f40009e80 tx=0x7f9f4000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb99494daad5eec3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:23.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.108+0000 7f9f3ffff700 1 -- 172.21.15.92:0/1740875717 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4000e070 con 0x7f9f4806be80 2023-11-28T11:23:23.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.108+0000 7f9f3ffff700 1 -- 172.21.15.92:0/1740875717 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f4000b8d0 con 0x7f9f4806be80 2023-11-28T11:23:23.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.108+0000 7f9f3ffff700 1 -- 172.21.15.92:0/1740875717 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4000bc80 con 0x7f9f4806be80 2023-11-28T11:23:23.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.109+0000 7f9f4f5ff700 1 -- 172.21.15.92:0/1740875717 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f4806be80 msgr2=0x7f9f480d0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:23.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.109+0000 7f9f4f5ff700 1 --2- 172.21.15.92:0/1740875717 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f4806be80 0x7f9f480d0fa0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9f40009e80 tx=0x7f9f4000b330 comp rx=0 tx=0).stop 2023-11-28T11:23:23.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.109+0000 7f9f4f5ff700 1 -- 172.21.15.92:0/1740875717 shutdown_connections 2023-11-28T11:23:23.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.109+0000 7f9f4f5ff700 1 --2- 172.21.15.92:0/1740875717 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f4806be80 0x7f9f480d0fa0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.109+0000 7f9f4f5ff700 1 --2- 172.21.15.92:0/1740875717 >> v2:172.21.15.169:3300/0 conn(0x7f9f480cd460 0x7f9f4806b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.109+0000 7f9f4f5ff700 1 -- 172.21.15.92:0/1740875717 >> 172.21.15.92:0/1740875717 conn(0x7f9f480719a0 msgr2=0x7f9f48071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:23.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.109+0000 7f9f4f5ff700 1 -- 172.21.15.92:0/1740875717 shutdown_connections 2023-11-28T11:23:23.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.109+0000 7f9f4f5ff700 1 -- 172.21.15.92:0/1740875717 wait complete. 2023-11-28T11:23:23.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.111+0000 7f9f4f5ff700 1 Processor -- start 2023-11-28T11:23:23.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.111+0000 7f9f4f5ff700 1 -- start start 2023-11-28T11:23:23.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.112+0000 7f9f4f5ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f4806be80 0x7f9f481588f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:23.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.112+0000 7f9f4f5ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f480b8790 0x7f9f48156fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:23.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.112+0000 7f9f4f5ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f480cd460 0x7f9f480d30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:23.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.112+0000 7f9f4f5ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f48155920 con 0x7f9f480cd460 2023-11-28T11:23:23.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.112+0000 7f9f4f5ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f48158e30 con 0x7f9f4806be80 2023-11-28T11:23:23.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.112+0000 7f9f4f5ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f48157850 con 0x7f9f480b8790 2023-11-28T11:23:23.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.113+0000 7f9f4db9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f480cd460 0x7f9f480d30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:23.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.113+0000 7f9f4cb9a700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f480b8790 0x7f9f48156fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:23.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.113+0000 7f9f4d39b700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f4806be80 0x7f9f481588f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:23.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.113+0000 7f9f4cb9a700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f480b8790 0x7f9f48156fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.92:35842/0 (socket says 172.21.15.92:35842) 2023-11-28T11:23:23.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.113+0000 7f9f4cb9a700 1 -- 172.21.15.92:0/1098786969 learned_addr learned my addr 172.21.15.92:0/1098786969 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:23.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.113+0000 7f9f4db9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f480cd460 0x7f9f480d30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58558/0 (socket says 172.21.15.92:58558) 2023-11-28T11:23:23.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.113+0000 7f9f4db9c700 1 -- 172.21.15.92:0/1098786969 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f480b8790 msgr2=0x7f9f48156fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:23.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.113+0000 7f9f4db9c700 1 --2- 172.21.15.92:0/1098786969 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f480b8790 0x7f9f48156fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.113+0000 7f9f4db9c700 1 -- 172.21.15.92:0/1098786969 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f4806be80 msgr2=0x7f9f481588f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:23.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.113+0000 7f9f4db9c700 1 --2- 172.21.15.92:0/1098786969 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f4806be80 0x7f9f481588f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.113+0000 7f9f4db9c700 1 -- 172.21.15.92:0/1098786969 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f40009b30 con 0x7f9f480cd460 2023-11-28T11:23:23.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.113+0000 7f9f4db9c700 1 --2- 172.21.15.92:0/1098786969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f480cd460 0x7f9f480d30a0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f9f40015000 tx=0x7f9f40004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:23.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.114+0000 7f9f3dffb700 1 -- 172.21.15.92:0/1098786969 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f40004880 con 0x7f9f480cd460 2023-11-28T11:23:23.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.114+0000 7f9f3dffb700 1 -- 172.21.15.92:0/1098786969 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f400099b0 con 0x7f9f480cd460 2023-11-28T11:23:23.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.114+0000 7f9f3dffb700 1 -- 172.21.15.92:0/1098786969 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4001c680 con 0x7f9f480cd460 2023-11-28T11:23:23.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.114+0000 7f9f4f5ff700 1 -- 172.21.15.92:0/1098786969 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f480d14e0 con 0x7f9f480cd460 2023-11-28T11:23:23.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.114+0000 7f9f4f5ff700 1 -- 172.21.15.92:0/1098786969 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f480b5800 con 0x7f9f480cd460 2023-11-28T11:23:23.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.116+0000 7f9f3dffb700 1 -- 172.21.15.92:0/1098786969 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9f40022030 con 0x7f9f480cd460 2023-11-28T11:23:23.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.117+0000 7f9f3dffb700 1 --2- 172.21.15.92:0/1098786969 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9f2c0682f0 0x7f9f2c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:23.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.117+0000 7f9f3dffb700 1 -- 172.21.15.92:0/1098786969 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f9f40087630 con 0x7f9f480cd460 2023-11-28T11:23:23.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.117+0000 7f9f4d39b700 1 --2- 172.21.15.92:0/1098786969 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9f2c0682f0 0x7f9f2c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:23.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.118+0000 7f9f4d39b700 1 --2- 172.21.15.92:0/1098786969 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9f2c0682f0 0x7f9f2c06a7a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9f44007fd0 tx=0x7f9f44009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:23.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.119+0000 7f9f62ea2b80 1 Processor -- start 2023-11-28T11:23:23.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.119+0000 7f9f62ea2b80 1 -- start start 2023-11-28T11:23:23.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.120+0000 7f9f62ea2b80 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e387612020 0x55e3875cbac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:23.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.120+0000 7f9f62ea2b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e387608b90 0x55e3876123f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:23.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.120+0000 7f9f62ea2b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e3875f4730 0x55e3875ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:23.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.120+0000 7f9f62ea2b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e387549eb0 con 0x55e3875f4730 2023-11-28T11:23:23.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.120+0000 7f9f62ea2b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3874b6210 con 0x55e387608b90 2023-11-28T11:23:23.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.120+0000 7f9f62ea2b80 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e387524e80 con 0x55e387612020 2023-11-28T11:23:23.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.120+0000 7f9f4db9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e3875f4730 0x55e3875ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:23.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.120+0000 7f9f4db9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e3875f4730 0x55e3875ccc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58562/0 (socket says 172.21.15.92:58562) 2023-11-28T11:23:23.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.120+0000 7f9f4db9c700 1 -- 172.21.15.92:0/1421250867 learned_addr learned my addr 172.21.15.92:0/1421250867 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:23.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.121+0000 7f9f4d39b700 1 --2- 172.21.15.92:0/1421250867 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e387612020 0x55e3875cbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:23.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.121+0000 7f9f4cb9a700 1 --2- 172.21.15.92:0/1421250867 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e387608b90 0x55e3876123f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:23.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.121+0000 7f9f4db9c700 1 -- 172.21.15.92:0/1421250867 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e387612020 msgr2=0x55e3875cbac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:23.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.121+0000 7f9f4db9c700 1 --2- 172.21.15.92:0/1421250867 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e387612020 0x55e3875cbac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.121+0000 7f9f4db9c700 1 -- 172.21.15.92:0/1421250867 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e387608b90 msgr2=0x55e3876123f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:23.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.121+0000 7f9f4db9c700 1 --2- 172.21.15.92:0/1421250867 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e387608b90 0x55e3876123f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.121+0000 7f9f4db9c700 1 -- 172.21.15.92:0/1421250867 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e387312860 con 0x55e3875f4730 2023-11-28T11:23:23.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.121+0000 7f9f4d39b700 1 --2- 172.21.15.92:0/1421250867 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e387612020 0x55e3875cbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:23:23.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.121+0000 7f9f4cb9a700 1 --2- 172.21.15.92:0/1421250867 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e387608b90 0x55e3876123f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:23:23.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.121+0000 7f9f4db9c700 1 --2- 172.21.15.92:0/1421250867 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e3875f4730 0x55e3875ccc30 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f9f40004aa0 tx=0x7f9f4005f9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26635b3770666f38 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:23.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.121+0000 7f9f2affd700 1 -- 172.21.15.92:0/1421250867 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f40053fd0 con 0x55e3875f4730 2023-11-28T11:23:23.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.121+0000 7f9f2affd700 1 -- 172.21.15.92:0/1421250867 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f40054130 con 0x55e3875f4730 2023-11-28T11:23:23.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.121+0000 7f9f2affd700 1 -- 172.21.15.92:0/1421250867 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f40056fb0 con 0x55e3875f4730 2023-11-28T11:23:23.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.122+0000 7f9f62ea2b80 1 -- 172.21.15.92:0/1421250867 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e3875f4730 msgr2=0x55e3875ccc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:23.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.122+0000 7f9f62ea2b80 1 --2- 172.21.15.92:0/1421250867 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e3875f4730 0x55e3875ccc30 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f9f40004aa0 tx=0x7f9f4005f9b0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.122+0000 7f9f62ea2b80 1 -- 172.21.15.92:0/1421250867 shutdown_connections 2023-11-28T11:23:23.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.122+0000 7f9f62ea2b80 1 --2- 172.21.15.92:0/1421250867 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e3875f4730 0x55e3875ccc30 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.122+0000 7f9f62ea2b80 1 --2- 172.21.15.92:0/1421250867 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e387612020 0x55e3875cbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.122+0000 7f9f62ea2b80 1 --2- 172.21.15.92:0/1421250867 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e387608b90 0x55e3876123f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.122+0000 7f9f62ea2b80 1 -- 172.21.15.92:0/1421250867 >> 172.21.15.92:0/1421250867 conn(0x55e3875c1df0 msgr2=0x55e3875caff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:23.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.122+0000 7f9f62ea2b80 1 -- 172.21.15.92:0/1421250867 shutdown_connections 2023-11-28T11:23:23.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.122+0000 7f9f62ea2b80 1 -- 172.21.15.92:0/1421250867 wait complete. 2023-11-28T11:23:23.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.123+0000 7f9f62ea2b80 1 -- start start 2023-11-28T11:23:23.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.123+0000 7f9f62ea2b80 1 Processor -- start 2023-11-28T11:23:23.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.125+0000 7f9f62ea2b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e3875f4730 0x55e3876099d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:23.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.125+0000 7f9f62ea2b80 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e387608b90 0x55e3875ef370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:23.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.125+0000 7f9f62ea2b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e387612020 0x55e38760fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:23.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.125+0000 7f9f62ea2b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e38751e1f0 con 0x55e3875f4730 2023-11-28T11:23:23.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.125+0000 7f9f62ea2b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e387313460 con 0x55e387612020 2023-11-28T11:23:23.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.125+0000 7f9f62ea2b80 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e3874b6410 con 0x55e387608b90 2023-11-28T11:23:23.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.125+0000 7f9f62ea2b80 10 client.0 _ll_register_callbacks cb 0x55e3874bc4d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:23:23.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.125+0000 7f9f4d39b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e3875f4730 0x55e3876099d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:23.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.125+0000 7f9f4d39b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e3875f4730 0x55e3876099d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58574/0 (socket says 172.21.15.92:58574) 2023-11-28T11:23:23.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.125+0000 7f9f4d39b700 1 -- 172.21.15.92:0/1649900261 learned_addr learned my addr 172.21.15.92:0/1649900261 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:23.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.125+0000 7f9f4d39b700 1 -- 172.21.15.92:0/1649900261 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e387608b90 msgr2=0x55e3875ef370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:23.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.125+0000 7f9f4d39b700 1 --2- 172.21.15.92:0/1649900261 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e387608b90 0x55e3875ef370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.125+0000 7f9f4d39b700 1 -- 172.21.15.92:0/1649900261 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e387612020 msgr2=0x55e38760fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:23.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.125+0000 7f9f4d39b700 1 --2- 172.21.15.92:0/1649900261 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e387612020 0x55e38760fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.125+0000 7f9f4d39b700 1 -- 172.21.15.92:0/1649900261 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f480b5800 con 0x55e3875f4730 2023-11-28T11:23:23.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.126+0000 7f9f4d39b700 1 --2- 172.21.15.92:0/1649900261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e3875f4730 0x55e3876099d0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f9f44001700 tx=0x7f9f44001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9db05fab55f2d75e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:23.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.126+0000 7f9f23fff700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-11-28T11:23:23.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.126+0000 7f9f23fff700 1 -- 172.21.15.92:0/1649900261 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f44014c50 con 0x55e3875f4730 2023-11-28T11:23:23.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.126+0000 7f9f23fff700 1 -- 172.21.15.92:0/1649900261 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(56..56 src has 1..56) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f9f44014db0 con 0x55e3875f4730 2023-11-28T11:23:23.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.126+0000 7f9f62ea2b80 1 -- 172.21.15.92:0/1649900261 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e387522100 con 0x55e3875f4730 2023-11-28T11:23:23.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.126+0000 7f9f23fff700 1 -- 172.21.15.92:0/1649900261 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f44020420 con 0x55e3875f4730 2023-11-28T11:23:23.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.126+0000 7f9f62ea2b80 10 client.5925 fetch_fsmap learned FSMap version 24 2023-11-28T11:23:23.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.126+0000 7f9f62ea2b80 1 -- 172.21.15.92:0/1649900261 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55e3873ff170 con 0x55e3875f4730 2023-11-28T11:23:23.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.127+0000 7f9f23fff700 1 -- 172.21.15.92:0/1649900261 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e387522100 con 0x55e3875f4730 2023-11-28T11:23:23.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.127+0000 7f9f23fff700 1 -- 172.21.15.92:0/1649900261 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 24) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f9f44028350 con 0x55e3875f4730 2023-11-28T11:23:23.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.127+0000 7f9f62ea2b80 10 client.5925 fetch_fsmap finished waiting for FSMap version 24 2023-11-28T11:23:23.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.127+0000 7f9f62ea2b80 10 client.5925 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-28T11:23:23.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.127+0000 7f9f62ea2b80 20 client.5925 populate_metadata read hostname 'smithi092' 2023-11-28T11:23:23.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.127+0000 7f9f62ea2b80 1 --2- 172.21.15.92:0/1649900261 >> [v2:172.21.15.92:6836/950691230,v1:172.21.15.92:6837/950691230] conn(0x55e38760b1f0 0x55e387624440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:23.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.127+0000 7f9f62ea2b80 4 client.5925 mds_command: new command op to 5754 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:23:23.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.127+0000 7f9f62ea2b80 1 -- 172.21.15.92:0/1649900261 --> [v2:172.21.15.92:6836/950691230,v1:172.21.15.92:6837/950691230] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e38736dfc0 con 0x55e38760b1f0 2023-11-28T11:23:23.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.127+0000 7f9f4cb9a700 1 --2- 172.21.15.92:0/1649900261 >> [v2:172.21.15.92:6836/950691230,v1:172.21.15.92:6837/950691230] conn(0x55e38760b1f0 0x55e387624440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:23.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.128+0000 7f9f4cb9a700 1 --2- 172.21.15.92:0/1649900261 >> [v2:172.21.15.92:6836/950691230,v1:172.21.15.92:6837/950691230] conn(0x55e38760b1f0 0x55e387624440 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df449aafa5003c62 server_cookie=b80d875ca03081de in_seq=0 out_seq=0 2023-11-28T11:23:23.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.128+0000 7f9f23fff700 10 client.5925 ms_handle_connect on v2:172.21.15.92:6836/950691230 2023-11-28T11:23:23.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.128+0000 7f9f23fff700 1 -- 172.21.15.92:0/1649900261 <== mds.0 v2:172.21.15.92:6836/950691230 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x55e38736dfc0 con 0x55e38760b1f0 2023-11-28T11:23:23.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.128+0000 7f9f23fff700 10 client.5925 handle_command_reply: tid=0 2023-11-28T11:23:23.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.128+0000 7f9f62ea2b80 1 client.5925 shutdown 2023-11-28T11:23:23.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.129+0000 7f9f62ea2b80 1 -- 172.21.15.92:0/1649900261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e3875f4730 msgr2=0x55e3876099d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:23.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.129+0000 7f9f62ea2b80 1 --2- 172.21.15.92:0/1649900261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e3875f4730 0x55e3876099d0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f9f44001700 tx=0x7f9f44001080 comp rx=0 tx=0).stop 2023-11-28T11:23:23.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.129+0000 7f9f62ea2b80 1 -- 172.21.15.92:0/1649900261 shutdown_connections 2023-11-28T11:23:23.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.129+0000 7f9f62ea2b80 1 --2- 172.21.15.92:0/1649900261 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e387612020 0x55e38760fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.129+0000 7f9f62ea2b80 1 --2- 172.21.15.92:0/1649900261 >> [v2:172.21.15.92:6836/950691230,v1:172.21.15.92:6837/950691230] conn(0x55e38760b1f0 0x55e387624440 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.129+0000 7f9f62ea2b80 1 --2- 172.21.15.92:0/1649900261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e3875f4730 0x55e3876099d0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.129+0000 7f9f62ea2b80 1 --2- 172.21.15.92:0/1649900261 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e387608b90 0x55e3875ef370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.129+0000 7f9f62ea2b80 1 -- 172.21.15.92:0/1649900261 >> 172.21.15.92:0/1649900261 conn(0x55e3875c1df0 msgr2=0x55e3875caff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:23.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.129+0000 7f9f23fff700 0 client.5925 ms_handle_reset on v2:172.21.15.92:6836/950691230 2023-11-28T11:23:23.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.129+0000 7f9f62ea2b80 1 -- 172.21.15.92:0/1649900261 shutdown_connections 2023-11-28T11:23:23.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.129+0000 7f9f62ea2b80 1 -- 172.21.15.92:0/1649900261 wait complete. 2023-11-28T11:23:23.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.129+0000 7f9f62ea2b80 20 client.5925 trim_cache size 0 max 16384 2023-11-28T11:23:23.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.135+0000 7fed569ff700 1 Processor -- start 2023-11-28T11:23:23.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.135+0000 7fed569ff700 1 -- start start 2023-11-28T11:23:23.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.137+0000 7fed569ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fed500cf2b0 0x7fed500cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:23.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.137+0000 7fed569ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fed500cfc90 0x7fed500d1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:23.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.137+0000 7fed569ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed500cd550 con 0x7fed5015c740 2023-11-28T11:23:23.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.137+0000 7fed569ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed500cd830 con 0x7fed500cf2b0 2023-11-28T11:23:23.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.137+0000 7fed569ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed500cdb10 con 0x7fed500cfc90 2023-11-28T11:23:23.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.137+0000 7fed4f7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fed500cf2b0 0x7fed500cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:23.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.137+0000 7fed4f7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fed500cf2b0 0x7fed500cf750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58582/0 (socket says 172.21.15.92:58582) 2023-11-28T11:23:23.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.137+0000 7fed4f7fe700 1 -- 172.21.15.92:0/441787860 learned_addr learned my addr 172.21.15.92:0/441787860 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:23.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.137+0000 7fed4ffff700 1 --1- 172.21.15.92:0/441787860 >> v1:172.21.15.169:6789/0 conn(0x7fed5015c740 0x7fed500d3670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.92:42032/0 (socket says 172.21.15.92:42032) 2023-11-28T11:23:23.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.137+0000 7fed54f9c700 1 --2- 172.21.15.92:0/441787860 >> v2:172.21.15.169:3300/0 conn(0x7fed500cfc90 0x7fed500d1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:23.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.138+0000 7fed4f7fe700 1 -- 172.21.15.92:0/441787860 >> v2:172.21.15.169:3300/0 conn(0x7fed500cfc90 msgr2=0x7fed500d1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:23.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.138+0000 7fed4f7fe700 1 --2- 172.21.15.92:0/441787860 >> v2:172.21.15.169:3300/0 conn(0x7fed500cfc90 0x7fed500d1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.138+0000 7fed4f7fe700 1 -- 172.21.15.92:0/441787860 >> v1:172.21.15.169:6789/0 conn(0x7fed5015c740 legacy=0x7fed500d3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:23.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.138+0000 7fed4f7fe700 1 -- 172.21.15.92:0/441787860 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed500d0a30 con 0x7fed500cf2b0 2023-11-28T11:23:23.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.138+0000 7fed4effd700 1 -- 172.21.15.92:0/441787860 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 668177997 0 0) 0x7fed500cd550 con 0x7fed5015c740 2023-11-28T11:23:23.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.138+0000 7fed4f7fe700 1 --2- 172.21.15.92:0/441787860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed500cf2b0 0x7fed500cf750 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fed440098f0 tx=0x7fed44009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b9a09a24da95e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:23.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.138+0000 7fed4effd700 1 -- 172.21.15.92:0/441787860 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed44011830 con 0x7fed500cf2b0 2023-11-28T11:23:23.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.138+0000 7fed4effd700 1 -- 172.21.15.92:0/441787860 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed44011990 con 0x7fed500cf2b0 2023-11-28T11:23:23.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.138+0000 7fed4effd700 1 -- 172.21.15.92:0/441787860 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed44011d20 con 0x7fed500cf2b0 2023-11-28T11:23:23.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.139+0000 7fed569ff700 1 -- 172.21.15.92:0/441787860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed500cf2b0 msgr2=0x7fed500cf750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:23.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.139+0000 7fed569ff700 1 --2- 172.21.15.92:0/441787860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed500cf2b0 0x7fed500cf750 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fed440098f0 tx=0x7fed44009c00 comp rx=0 tx=0).stop 2023-11-28T11:23:23.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.139+0000 7fed569ff700 1 -- 172.21.15.92:0/441787860 shutdown_connections 2023-11-28T11:23:23.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.139+0000 7fed569ff700 1 --2- 172.21.15.92:0/441787860 >> v2:172.21.15.169:3300/0 conn(0x7fed500cfc90 0x7fed500d1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.139+0000 7fed569ff700 1 --2- 172.21.15.92:0/441787860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed500cf2b0 0x7fed500cf750 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.139+0000 7fed569ff700 1 -- 172.21.15.92:0/441787860 >> 172.21.15.92:0/441787860 conn(0x7fed500719a0 msgr2=0x7fed50071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:23.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.139+0000 7fed569ff700 1 -- 172.21.15.92:0/441787860 shutdown_connections 2023-11-28T11:23:23.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.139+0000 7fed569ff700 1 -- 172.21.15.92:0/441787860 wait complete. 2023-11-28T11:23:23.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.141+0000 7fed569ff700 1 Processor -- start 2023-11-28T11:23:23.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.141+0000 7fed569ff700 1 -- start start 2023-11-28T11:23:23.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.142+0000 7fed569ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed500cf2b0 0x7fed50154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:23.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.142+0000 7fed569ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed500cfc90 0x7fed501f5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:23.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.142+0000 7fed569ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed5015c740 0x7fed500b3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:23.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.142+0000 7fed569ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed500b71f0 con 0x7fed5015c740 2023-11-28T11:23:23.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.142+0000 7fed569ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed50154a50 con 0x7fed500cfc90 2023-11-28T11:23:23.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.142+0000 7fed569ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed501f64d0 con 0x7fed500cf2b0 2023-11-28T11:23:23.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.142+0000 7fed54f9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed5015c740 0x7fed500b3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:23.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.142+0000 7fed54f9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed5015c740 0x7fed500b3060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58588/0 (socket says 172.21.15.92:58588) 2023-11-28T11:23:23.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.142+0000 7fed54f9c700 1 -- 172.21.15.92:0/3166003025 learned_addr learned my addr 172.21.15.92:0/3166003025 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:23.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.142+0000 7fed54f9c700 1 -- 172.21.15.92:0/3166003025 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed500cf2b0 msgr2=0x7fed50154510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:23.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.142+0000 7fed54f9c700 1 --2- 172.21.15.92:0/3166003025 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed500cf2b0 0x7fed50154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.142+0000 7fed4f7fe700 1 --2- 172.21.15.92:0/3166003025 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed500cfc90 0x7fed501f5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:23.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.142+0000 7fed54f9c700 1 -- 172.21.15.92:0/3166003025 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed500cfc90 msgr2=0x7fed501f5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:23.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.142+0000 7fed54f9c700 1 --2- 172.21.15.92:0/3166003025 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed500cfc90 0x7fed501f5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.142+0000 7fed54f9c700 1 -- 172.21.15.92:0/3166003025 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed44009550 con 0x7fed5015c740 2023-11-28T11:23:23.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.142+0000 7fed4f7fe700 1 --2- 172.21.15.92:0/3166003025 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed500cfc90 0x7fed501f5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:23:23.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.143+0000 7fed54f9c700 1 --2- 172.21.15.92:0/3166003025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed5015c740 0x7fed500b3060 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fed48009dd0 tx=0x7fed4800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:23.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.143+0000 7fed4cff9700 1 -- 172.21.15.92:0/3166003025 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed4800cd40 con 0x7fed5015c740 2023-11-28T11:23:23.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.143+0000 7fed4cff9700 1 -- 172.21.15.92:0/3166003025 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed4800f040 con 0x7fed5015c740 2023-11-28T11:23:23.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.143+0000 7fed569ff700 1 -- 172.21.15.92:0/3166003025 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed500d00b0 con 0x7fed5015c740 2023-11-28T11:23:23.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.143+0000 7fed4cff9700 1 -- 172.21.15.92:0/3166003025 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed480146c0 con 0x7fed5015c740 2023-11-28T11:23:23.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.143+0000 7fed569ff700 1 -- 172.21.15.92:0/3166003025 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed500d0d10 con 0x7fed5015c740 2023-11-28T11:23:23.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.144+0000 7fed569ff700 1 -- 172.21.15.92:0/3166003025 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fed500d1770 con 0x7fed5015c740 2023-11-28T11:23:23.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.145+0000 7fed4cff9700 1 -- 172.21.15.92:0/3166003025 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fed48014820 con 0x7fed5015c740 2023-11-28T11:23:23.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.146+0000 7fed4cff9700 1 --2- 172.21.15.92:0/3166003025 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fed3c0682f0 0x7fed3c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:23.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.146+0000 7fed4cff9700 1 -- 172.21.15.92:0/3166003025 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fed480869b0 con 0x7fed5015c740 2023-11-28T11:23:23.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.146+0000 7fed4ffff700 1 --2- 172.21.15.92:0/3166003025 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fed3c0682f0 0x7fed3c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:23.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.146+0000 7fed4cff9700 1 -- 172.21.15.92:0/3166003025 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+502 (secure 0 0 0) 0x7fed4808b220 con 0x7fed5015c740 2023-11-28T11:23:23.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.146+0000 7fed569ff700 1 -- 172.21.15.92:0/3166003025 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7fed500b7330 con 0x7fed5015c740 2023-11-28T11:23:23.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.147+0000 7fed4ffff700 1 --2- 172.21.15.92:0/3166003025 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fed3c0682f0 0x7fed3c06a7a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fed40001610 tx=0x7fed400079f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:23.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.149+0000 7f9f62ea2b80 1 Processor -- start 2023-11-28T11:23:23.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.149+0000 7f9f62ea2b80 1 -- start start 2023-11-28T11:23:23.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.150+0000 7f9f62ea2b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e3875f4730 0x55e38760a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:23.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.150+0000 7f9f62ea2b80 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e387608b90 0x55e3875c93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:23.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.150+0000 7f9f62ea2b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e387612020 0x55e3875ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:23.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.150+0000 7f9f62ea2b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e387612fb0 con 0x55e3875f4730 2023-11-28T11:23:23.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.150+0000 7f9f62ea2b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e38760c090 con 0x55e387612020 2023-11-28T11:23:23.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.150+0000 7f9f62ea2b80 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e387612870 con 0x55e387608b90 2023-11-28T11:23:23.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.150+0000 7f9f4d39b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e3875f4730 0x55e38760a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:23.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.150+0000 7f9f4d39b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e3875f4730 0x55e38760a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58600/0 (socket says 172.21.15.92:58600) 2023-11-28T11:23:23.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.150+0000 7f9f4d39b700 1 -- 172.21.15.92:0/4227309939 learned_addr learned my addr 172.21.15.92:0/4227309939 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:23.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.151+0000 7f9f4cb9a700 1 --2- 172.21.15.92:0/4227309939 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e387608b90 0x55e3875c93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:23.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.151+0000 7f9f4db9c700 1 --2- 172.21.15.92:0/4227309939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e387612020 0x55e3875ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:23.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.151+0000 7f9f4d39b700 1 -- 172.21.15.92:0/4227309939 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e387608b90 msgr2=0x55e3875c93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:23.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.151+0000 7f9f4d39b700 1 --2- 172.21.15.92:0/4227309939 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e387608b90 0x55e3875c93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.151+0000 7f9f4d39b700 1 -- 172.21.15.92:0/4227309939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e387612020 msgr2=0x55e3875ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:23.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.151+0000 7f9f4d39b700 1 --2- 172.21.15.92:0/4227309939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e387612020 0x55e3875ccce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.151+0000 7f9f4d39b700 1 -- 172.21.15.92:0/4227309939 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f480b3a90 con 0x55e3875f4730 2023-11-28T11:23:23.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.151+0000 7f9f4cb9a700 1 --2- 172.21.15.92:0/4227309939 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e387608b90 0x55e3875c93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:23:23.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.151+0000 7f9f4d39b700 1 --2- 172.21.15.92:0/4227309939 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e3875f4730 0x55e38760a270 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f9f4400f600 tx=0x7f9f44016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27e6cd70a3d8a03f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:23.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.151+0000 7f9f2b7fe700 1 -- 172.21.15.92:0/4227309939 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f44014890 con 0x55e3875f4730 2023-11-28T11:23:23.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.151+0000 7f9f2b7fe700 1 -- 172.21.15.92:0/4227309939 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f440288b0 con 0x55e3875f4730 2023-11-28T11:23:23.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.151+0000 7f9f2b7fe700 1 -- 172.21.15.92:0/4227309939 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f44014e50 con 0x55e3875f4730 2023-11-28T11:23:23.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.152+0000 7f9f62ea2b80 1 -- 172.21.15.92:0/4227309939 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e3875f4730 msgr2=0x55e38760a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:23.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.152+0000 7f9f62ea2b80 1 --2- 172.21.15.92:0/4227309939 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e3875f4730 0x55e38760a270 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f9f4400f600 tx=0x7f9f44016000 comp rx=0 tx=0).stop 2023-11-28T11:23:23.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.152+0000 7f9f62ea2b80 1 -- 172.21.15.92:0/4227309939 shutdown_connections 2023-11-28T11:23:23.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.152+0000 7f9f62ea2b80 1 --2- 172.21.15.92:0/4227309939 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e387612020 0x55e3875ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.152+0000 7f9f62ea2b80 1 --2- 172.21.15.92:0/4227309939 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e3875f4730 0x55e38760a270 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.152+0000 7f9f62ea2b80 1 --2- 172.21.15.92:0/4227309939 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e387608b90 0x55e3875c93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.152+0000 7f9f62ea2b80 1 -- 172.21.15.92:0/4227309939 >> 172.21.15.92:0/4227309939 conn(0x55e3875c1df0 msgr2=0x55e3875ce590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:23.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.152+0000 7f9f62ea2b80 1 -- 172.21.15.92:0/4227309939 shutdown_connections 2023-11-28T11:23:23.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.152+0000 7f9f62ea2b80 1 -- 172.21.15.92:0/4227309939 wait complete. 2023-11-28T11:23:23.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.153+0000 7f9f62ea2b80 1 -- start start 2023-11-28T11:23:23.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.153+0000 7f9f62ea2b80 1 Processor -- start 2023-11-28T11:23:23.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.154+0000 7f9f62ea2b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e3875f4730 0x55e3875ee7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:23.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.154+0000 7f9f62ea2b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e387608b90 0x55e3876691a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:23.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.154+0000 7f9f62ea2b80 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e387612020 0x55e3876696e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:23.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.154+0000 7f9f62ea2b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e38766d3f0 con 0x55e3875f4730 2023-11-28T11:23:23.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.154+0000 7f9f62ea2b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e387609690 con 0x55e387608b90 2023-11-28T11:23:23.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.154+0000 7f9f62ea2b80 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e3876098e0 con 0x55e387612020 2023-11-28T11:23:23.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.154+0000 7f9f62ea2b80 10 client.0 _ll_register_callbacks cb 0x55e3874bc4d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:23:23.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.154+0000 7f9f4d39b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e3875f4730 0x55e3875ee7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:23.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.154+0000 7f9f4d39b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e3875f4730 0x55e3875ee7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58616/0 (socket says 172.21.15.92:58616) 2023-11-28T11:23:23.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.154+0000 7f9f4d39b700 1 -- 172.21.15.92:0/776873230 learned_addr learned my addr 172.21.15.92:0/776873230 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:23.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.154+0000 7f9f4d39b700 1 -- 172.21.15.92:0/776873230 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e387612020 msgr2=0x55e3876696e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:23.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.154+0000 7f9f4d39b700 1 --2- 172.21.15.92:0/776873230 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e387612020 0x55e3876696e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.154+0000 7f9f4d39b700 1 -- 172.21.15.92:0/776873230 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e387608b90 msgr2=0x55e3876691a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:23.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.154+0000 7f9f4d39b700 1 --2- 172.21.15.92:0/776873230 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e387608b90 0x55e3876691a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.154+0000 7f9f4d39b700 1 -- 172.21.15.92:0/776873230 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e3873ff170 con 0x55e3875f4730 2023-11-28T11:23:23.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.155+0000 7f9f4d39b700 1 --2- 172.21.15.92:0/776873230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e3875f4730 0x55e3875ee7a0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f9f44001700 tx=0x7f9f44014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6d6cc28c492c797 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:23.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.155+0000 7f9f23fff700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-11-28T11:23:23.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.155+0000 7f9f23fff700 1 -- 172.21.15.92:0/776873230 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4403a630 con 0x55e3875f4730 2023-11-28T11:23:23.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.155+0000 7f9f23fff700 1 -- 172.21.15.92:0/776873230 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(56..56 src has 1..56) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f9f44022980 con 0x55e3875f4730 2023-11-28T11:23:23.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.155+0000 7f9f62ea2b80 1 -- 172.21.15.92:0/776873230 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9f40054130 con 0x55e3875f4730 2023-11-28T11:23:23.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.155+0000 7f9f23fff700 1 -- 172.21.15.92:0/776873230 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f44014e60 con 0x55e3875f4730 2023-11-28T11:23:23.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.155+0000 7f9f62ea2b80 10 client.5937 fetch_fsmap learned FSMap version 24 2023-11-28T11:23:23.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.155+0000 7f9f62ea2b80 1 -- 172.21.15.92:0/776873230 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55e3874cbbe0 con 0x55e3875f4730 2023-11-28T11:23:23.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.155+0000 7f9f23fff700 1 -- 172.21.15.92:0/776873230 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f40054130 con 0x55e3875f4730 2023-11-28T11:23:23.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.156+0000 7f9f23fff700 1 -- 172.21.15.92:0/776873230 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 24) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f9f44038070 con 0x55e3875f4730 2023-11-28T11:23:23.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.156+0000 7f9f62ea2b80 10 client.5937 fetch_fsmap finished waiting for FSMap version 24 2023-11-28T11:23:23.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.156+0000 7f9f62ea2b80 10 client.5937 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-28T11:23:23.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.156+0000 7f9f62ea2b80 20 client.5937 populate_metadata read hostname 'smithi092' 2023-11-28T11:23:23.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.156+0000 7f9f62ea2b80 1 --2- 172.21.15.92:0/776873230 >> [v2:172.21.15.92:6836/950691230,v1:172.21.15.92:6837/950691230] conn(0x55e387623d70 0x55e387607910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:23.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.156+0000 7f9f62ea2b80 4 client.5937 mds_command: new command op to 5754 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2023-11-28T11:23:23.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.156+0000 7f9f62ea2b80 1 -- 172.21.15.92:0/776873230 --> [v2:172.21.15.92:6836/950691230,v1:172.21.15.92:6837/950691230] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x55e3873a3110 con 0x55e387623d70 2023-11-28T11:23:23.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.156+0000 7f9f4cb9a700 1 --2- 172.21.15.92:0/776873230 >> [v2:172.21.15.92:6836/950691230,v1:172.21.15.92:6837/950691230] conn(0x55e387623d70 0x55e387607910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:23.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.156+0000 7f9f4cb9a700 1 --2- 172.21.15.92:0/776873230 >> [v2:172.21.15.92:6836/950691230,v1:172.21.15.92:6837/950691230] conn(0x55e387623d70 0x55e387607910 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93061d70a0d7765f server_cookie=cbd62e9e05c7b53f in_seq=0 out_seq=0 2023-11-28T11:23:23.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.156+0000 7f9f23fff700 10 client.5937 ms_handle_connect on v2:172.21.15.92:6836/950691230 2023-11-28T11:23:23.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.157+0000 7f9f23fff700 1 -- 172.21.15.92:0/776873230 <== mds.0 v2:172.21.15.92:6836/950691230 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e3873a3110 con 0x55e387623d70 2023-11-28T11:23:23.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.157+0000 7f9f23fff700 10 client.5937 handle_command_reply: tid=0 2023-11-28T11:23:23.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.157+0000 7f9f62ea2b80 1 client.5937 shutdown 2023-11-28T11:23:23.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.157+0000 7f9f62ea2b80 1 -- 172.21.15.92:0/776873230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e3875f4730 msgr2=0x55e3875ee7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:23.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.157+0000 7f9f62ea2b80 1 --2- 172.21.15.92:0/776873230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e3875f4730 0x55e3875ee7a0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f9f44001700 tx=0x7f9f44014000 comp rx=0 tx=0).stop 2023-11-28T11:23:23.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.157+0000 7f9f62ea2b80 1 -- 172.21.15.92:0/776873230 shutdown_connections 2023-11-28T11:23:23.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.157+0000 7f9f62ea2b80 1 --2- 172.21.15.92:0/776873230 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55e387612020 0x55e3876696e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.157+0000 7f9f62ea2b80 1 --2- 172.21.15.92:0/776873230 >> [v2:172.21.15.92:6836/950691230,v1:172.21.15.92:6837/950691230] conn(0x55e387623d70 0x55e387607910 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.157+0000 7f9f62ea2b80 1 --2- 172.21.15.92:0/776873230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e3875f4730 0x55e3875ee7a0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.157+0000 7f9f62ea2b80 1 --2- 172.21.15.92:0/776873230 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55e387608b90 0x55e3876691a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.157+0000 7f9f23fff700 0 client.5937 ms_handle_reset on v2:172.21.15.92:6836/950691230 2023-11-28T11:23:23.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.157+0000 7f9f62ea2b80 1 -- 172.21.15.92:0/776873230 >> 172.21.15.92:0/776873230 conn(0x55e3875c1df0 msgr2=0x55e3875c9350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:23.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.158+0000 7f9f62ea2b80 1 -- 172.21.15.92:0/776873230 shutdown_connections 2023-11-28T11:23:23.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.158+0000 7f9f62ea2b80 1 -- 172.21.15.92:0/776873230 wait complete. 2023-11-28T11:23:23.170 INFO:tasks.ceph.mds.c.smithi092.stderr:2023-11-28T11:23:23.158+0000 7f54461ea700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={5889=0-4194304@1} caps={5889=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5889 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x562cdbfd9b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-28T11:23:23.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.158+0000 7f9f62ea2b80 20 client.5937 trim_cache size 0 max 16384 2023-11-28T11:23:23.171 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-28T11:23:23.171 INFO:teuthology.orchestra.run.smithi092.stdout: "return_code": 0, 2023-11-28T11:23:23.171 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_tag": "1ac37aa7-e916-4500-afd8-79713c734769", 2023-11-28T11:23:23.171 INFO:teuthology.orchestra.run.smithi092.stdout: "mode": "asynchronous" 2023-11-28T11:23:23.171 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-28T11:23:23.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.158+0000 7f9f4f5ff700 1 -- 172.21.15.92:0/1098786969 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9f2c0682f0 msgr2=0x7f9f2c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:23.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.158+0000 7f9f4f5ff700 1 --2- 172.21.15.92:0/1098786969 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9f2c0682f0 0x7f9f2c06a7a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9f44007fd0 tx=0x7f9f44009040 comp rx=0 tx=0).stop 2023-11-28T11:23:23.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.158+0000 7f9f4f5ff700 1 -- 172.21.15.92:0/1098786969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f480cd460 msgr2=0x7f9f480d30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:23.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.158+0000 7f9f4f5ff700 1 --2- 172.21.15.92:0/1098786969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f480cd460 0x7f9f480d30a0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f9f40015000 tx=0x7f9f40004ad0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.158+0000 7f9f4f5ff700 1 -- 172.21.15.92:0/1098786969 shutdown_connections 2023-11-28T11:23:23.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.158+0000 7f9f4f5ff700 1 --2- 172.21.15.92:0/1098786969 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9f2c0682f0 0x7f9f2c06a7a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.158+0000 7f9f4f5ff700 1 --2- 172.21.15.92:0/1098786969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f480cd460 0x7f9f480d30a0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.158+0000 7f9f4f5ff700 1 --2- 172.21.15.92:0/1098786969 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9f4806be80 0x7f9f481588f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.158+0000 7f9f4f5ff700 1 --2- 172.21.15.92:0/1098786969 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9f480b8790 0x7f9f48156fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.158+0000 7f9f4f5ff700 1 -- 172.21.15.92:0/1098786969 >> 172.21.15.92:0/1098786969 conn(0x7f9f480719a0 msgr2=0x7f9f480b7640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:23.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.159+0000 7f9f4f5ff700 1 -- 172.21.15.92:0/1098786969 shutdown_connections 2023-11-28T11:23:23.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.159+0000 7f9f4f5ff700 1 -- 172.21.15.92:0/1098786969 wait complete. 2023-11-28T11:23:23.177 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub status 2023-11-28T11:23:23.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.376+0000 7fed2fdff700 1 Processor -- start 2023-11-28T11:23:23.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.376+0000 7fed2fdff700 1 -- start start 2023-11-28T11:23:23.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.377+0000 7fed2fdff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fed280b8510 0x7fed2806b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:23.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.377+0000 7fed2fdff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed280b88e0 con 0x7fed280d0d60 2023-11-28T11:23:23.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.377+0000 7fed2fdff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed280cd8f0 con 0x7fed280cd460 2023-11-28T11:23:23.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.377+0000 7fed2fdff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed280cdbd0 con 0x7fed280b8510 2023-11-28T11:23:23.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.377+0000 7fed2db9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fed280b8510 0x7fed2806b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:23.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.377+0000 7fed2e39c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fed280d0d60 0x7fed280d1200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52450/0 (socket says 172.21.15.92:52450) 2023-11-28T11:23:23.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.377+0000 7fed2e39c700 1 -- 172.21.15.92:0/1971618414 learned_addr learned my addr 172.21.15.92:0/1971618414 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:23:23.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.378+0000 7fed2db9b700 1 -- 172.21.15.92:0/1971618414 >> v1:172.21.15.169:6789/0 conn(0x7fed280cd460 legacy=0x7fed2806beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:23.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.378+0000 7fed2db9b700 1 -- 172.21.15.92:0/1971618414 >> v1:172.21.15.92:6789/0 conn(0x7fed280d0d60 legacy=0x7fed280d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:23.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.378+0000 7fed2db9b700 1 -- 172.21.15.92:0/1971618414 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed280cfd10 con 0x7fed280b8510 2023-11-28T11:23:23.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.378+0000 7fed2cb99700 1 -- 172.21.15.92:0/1971618414 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2374551759 0 0) 0x7fed280b88e0 con 0x7fed280d0d60 2023-11-28T11:23:23.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.378+0000 7fed2db9b700 1 --2- 172.21.15.92:0/1971618414 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed280b8510 0x7fed2806b970 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fed24009840 tx=0x7fed24009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10ea8af068e58e06 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:23.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.378+0000 7fed2cb99700 1 -- 172.21.15.92:0/1971618414 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed24011830 con 0x7fed280b8510 2023-11-28T11:23:23.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.378+0000 7fed2cb99700 1 -- 172.21.15.92:0/1971618414 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed24011990 con 0x7fed280b8510 2023-11-28T11:23:23.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.378+0000 7fed2cb99700 1 -- 172.21.15.92:0/1971618414 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed24011d20 con 0x7fed280b8510 2023-11-28T11:23:23.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.379+0000 7fed2fdff700 1 -- 172.21.15.92:0/1971618414 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed280b8510 msgr2=0x7fed2806b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:23.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.379+0000 7fed2fdff700 1 --2- 172.21.15.92:0/1971618414 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed280b8510 0x7fed2806b970 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fed24009840 tx=0x7fed24009b50 comp rx=0 tx=0).stop 2023-11-28T11:23:23.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.379+0000 7fed2fdff700 1 -- 172.21.15.92:0/1971618414 shutdown_connections 2023-11-28T11:23:23.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.379+0000 7fed2fdff700 1 --2- 172.21.15.92:0/1971618414 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed280b8510 0x7fed2806b970 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.379+0000 7fed2fdff700 1 -- 172.21.15.92:0/1971618414 >> 172.21.15.92:0/1971618414 conn(0x7fed280719a0 msgr2=0x7fed28071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:23.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.379+0000 7fed2fdff700 1 -- 172.21.15.92:0/1971618414 shutdown_connections 2023-11-28T11:23:23.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.379+0000 7fed2fdff700 1 -- 172.21.15.92:0/1971618414 wait complete. 2023-11-28T11:23:23.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.381+0000 7fed2fdff700 1 Processor -- start 2023-11-28T11:23:23.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.381+0000 7fed2fdff700 1 -- start start 2023-11-28T11:23:23.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.382+0000 7fed2fdff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed280b8510 0x7fed2815a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:23.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.382+0000 7fed2fdff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed280cd460 0x7fed281e5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:23.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.382+0000 7fed2fdff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed280d0d60 0x7fed281e6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:23.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.382+0000 7fed2fdff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2815acf0 con 0x7fed280cd460 2023-11-28T11:23:23.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.382+0000 7fed2fdff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed281e4560 con 0x7fed280d0d60 2023-11-28T11:23:23.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.382+0000 7fed2fdff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed281e4840 con 0x7fed280b8510 2023-11-28T11:23:23.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.382+0000 7fed2d39a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed280cd460 0x7fed281e5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:23.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.382+0000 7fed2d39a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed280cd460 0x7fed281e5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58628/0 (socket says 172.21.15.92:58628) 2023-11-28T11:23:23.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.382+0000 7fed2d39a700 1 -- 172.21.15.92:0/3102941578 learned_addr learned my addr 172.21.15.92:0/3102941578 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:23.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.382+0000 7fed2db9b700 1 --2- 172.21.15.92:0/3102941578 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed280b8510 0x7fed2815a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:23.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.382+0000 7fed2e39c700 1 --2- 172.21.15.92:0/3102941578 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed280d0d60 0x7fed281e6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:23.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.382+0000 7fed2d39a700 1 -- 172.21.15.92:0/3102941578 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed280b8510 msgr2=0x7fed2815a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:23.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.382+0000 7fed2d39a700 1 --2- 172.21.15.92:0/3102941578 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed280b8510 0x7fed2815a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.382+0000 7fed2d39a700 1 -- 172.21.15.92:0/3102941578 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed280d0d60 msgr2=0x7fed281e6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:23.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.382+0000 7fed2d39a700 1 --2- 172.21.15.92:0/3102941578 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed280d0d60 0x7fed281e6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.382+0000 7fed2d39a700 1 -- 172.21.15.92:0/3102941578 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed240094a0 con 0x7fed280cd460 2023-11-28T11:23:23.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.382+0000 7fed2e39c700 1 --2- 172.21.15.92:0/3102941578 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed280d0d60 0x7fed281e6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:23:23.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.383+0000 7fed2d39a700 1 --2- 172.21.15.92:0/3102941578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed280cd460 0x7fed281e5600 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fed1800c480 tx=0x7fed1800c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:23.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.383+0000 7fed1e7fc700 1 -- 172.21.15.92:0/3102941578 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed180052d0 con 0x7fed280cd460 2023-11-28T11:23:23.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.383+0000 7fed1e7fc700 1 -- 172.21.15.92:0/3102941578 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed18005430 con 0x7fed280cd460 2023-11-28T11:23:23.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.383+0000 7fed1e7fc700 1 -- 172.21.15.92:0/3102941578 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed180038c0 con 0x7fed280cd460 2023-11-28T11:23:23.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.383+0000 7fed2fdff700 1 -- 172.21.15.92:0/3102941578 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed280d29c0 con 0x7fed280cd460 2023-11-28T11:23:23.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.383+0000 7fed2fdff700 1 -- 172.21.15.92:0/3102941578 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed280cfff0 con 0x7fed280cd460 2023-11-28T11:23:23.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.385+0000 7fed1e7fc700 1 -- 172.21.15.92:0/3102941578 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fed18019030 con 0x7fed280cd460 2023-11-28T11:23:23.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.386+0000 7fed1e7fc700 1 --2- 172.21.15.92:0/3102941578 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fed0c0682f0 0x7fed0c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:23.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.386+0000 7fed1e7fc700 1 -- 172.21.15.92:0/3102941578 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fed18085150 con 0x7fed280cd460 2023-11-28T11:23:23.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.386+0000 7fed2db9b700 1 --2- 172.21.15.92:0/3102941578 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fed0c0682f0 0x7fed0c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:23.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.387+0000 7fed2db9b700 1 --2- 172.21.15.92:0/3102941578 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fed0c0682f0 0x7fed0c06a7a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fed240090b0 tx=0x7fed2401c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:23.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.388+0000 7fed4372ab80 1 Processor -- start 2023-11-28T11:23:23.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.388+0000 7fed4372ab80 1 -- start start 2023-11-28T11:23:23.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.389+0000 7fed4372ab80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557059af2fa0 0x557059aaca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:23.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.389+0000 7fed4372ab80 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557059ae9b10 0x557059af3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:23.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.389+0000 7fed4372ab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557059ad56b0 0x557059aadbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:23.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.389+0000 7fed4372ab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x557059a2ae30 con 0x557059ad56b0 2023-11-28T11:23:23.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.389+0000 7fed4372ab80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x557059997190 con 0x557059af2fa0 2023-11-28T11:23:23.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.389+0000 7fed4372ab80 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x557059a05e00 con 0x557059ae9b10 2023-11-28T11:23:23.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.389+0000 7fed2e39c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557059ad56b0 0x557059aadbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:23.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.389+0000 7fed2e39c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557059ad56b0 0x557059aadbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58638/0 (socket says 172.21.15.92:58638) 2023-11-28T11:23:23.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.389+0000 7fed2e39c700 1 -- 172.21.15.92:0/1588458991 learned_addr learned my addr 172.21.15.92:0/1588458991 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:23.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.390+0000 7fed2e39c700 1 -- 172.21.15.92:0/1588458991 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557059ae9b10 msgr2=0x557059af3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:23.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.390+0000 7fed2e39c700 1 --2- 172.21.15.92:0/1588458991 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557059ae9b10 0x557059af3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.390+0000 7fed2e39c700 1 -- 172.21.15.92:0/1588458991 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557059af2fa0 msgr2=0x557059aaca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:23.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.390+0000 7fed2e39c700 1 --2- 172.21.15.92:0/1588458991 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557059af2fa0 0x557059aaca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.390+0000 7fed2e39c700 1 -- 172.21.15.92:0/1588458991 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5570597f37e0 con 0x557059ad56b0 2023-11-28T11:23:23.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.390+0000 7fed2e39c700 1 --2- 172.21.15.92:0/1588458991 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557059ad56b0 0x557059aadbb0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fed20007050 tx=0x7fed20007360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2276256aeb4cf6ee server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:23.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.390+0000 7fed0b7fe700 1 -- 172.21.15.92:0/1588458991 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed20014480 con 0x557059ad56b0 2023-11-28T11:23:23.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.390+0000 7fed0b7fe700 1 -- 172.21.15.92:0/1588458991 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed200145e0 con 0x557059ad56b0 2023-11-28T11:23:23.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.390+0000 7fed0b7fe700 1 -- 172.21.15.92:0/1588458991 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed20014950 con 0x557059ad56b0 2023-11-28T11:23:23.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.391+0000 7fed4372ab80 1 -- 172.21.15.92:0/1588458991 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557059ad56b0 msgr2=0x557059aadbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:23.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.391+0000 7fed4372ab80 1 --2- 172.21.15.92:0/1588458991 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557059ad56b0 0x557059aadbb0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fed20007050 tx=0x7fed20007360 comp rx=0 tx=0).stop 2023-11-28T11:23:23.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.391+0000 7fed4372ab80 1 -- 172.21.15.92:0/1588458991 shutdown_connections 2023-11-28T11:23:23.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.391+0000 7fed4372ab80 1 --2- 172.21.15.92:0/1588458991 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557059ad56b0 0x557059aadbb0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.391+0000 7fed4372ab80 1 --2- 172.21.15.92:0/1588458991 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557059af2fa0 0x557059aaca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.391+0000 7fed4372ab80 1 --2- 172.21.15.92:0/1588458991 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557059ae9b10 0x557059af3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.391+0000 7fed4372ab80 1 -- 172.21.15.92:0/1588458991 >> 172.21.15.92:0/1588458991 conn(0x557059aa2d70 msgr2=0x557059aabf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:23.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.391+0000 7fed4372ab80 1 -- 172.21.15.92:0/1588458991 shutdown_connections 2023-11-28T11:23:23.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.391+0000 7fed4372ab80 1 -- 172.21.15.92:0/1588458991 wait complete. 2023-11-28T11:23:23.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.392+0000 7fed4372ab80 1 -- start start 2023-11-28T11:23:23.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.392+0000 7fed4372ab80 1 Processor -- start 2023-11-28T11:23:23.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.393+0000 7fed4372ab80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557059ad56b0 0x557059ad0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:23.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.393+0000 7fed4372ab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557059ae9b10 0x557059aee8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:23.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.393+0000 7fed4372ab80 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557059af2fa0 0x557059ad7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:23.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.393+0000 7fed4372ab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570599ff170 con 0x557059ae9b10 2023-11-28T11:23:23.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.393+0000 7fed4372ab80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570597f43e0 con 0x557059ad56b0 2023-11-28T11:23:23.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.393+0000 7fed4372ab80 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x557059997390 con 0x557059af2fa0 2023-11-28T11:23:23.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.393+0000 7fed4372ab80 10 client.0 _ll_register_callbacks cb 0x55705999d450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:23:23.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.393+0000 7fed2d39a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557059ae9b10 0x557059aee8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:23.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.393+0000 7fed2d39a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557059ae9b10 0x557059aee8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58644/0 (socket says 172.21.15.92:58644) 2023-11-28T11:23:23.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.393+0000 7fed2d39a700 1 -- 172.21.15.92:0/331830146 learned_addr learned my addr 172.21.15.92:0/331830146 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:23.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.393+0000 7fed2e39c700 1 --2- 172.21.15.92:0/331830146 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557059af2fa0 0x557059ad7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:23.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.393+0000 7fed2db9b700 1 --2- 172.21.15.92:0/331830146 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557059ad56b0 0x557059ad0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:23.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.393+0000 7fed2d39a700 1 -- 172.21.15.92:0/331830146 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557059af2fa0 msgr2=0x557059ad7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:23.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.393+0000 7fed2d39a700 1 --2- 172.21.15.92:0/331830146 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557059af2fa0 0x557059ad7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.393+0000 7fed2d39a700 1 -- 172.21.15.92:0/331830146 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557059ad56b0 msgr2=0x557059ad0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:23.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.393+0000 7fed2d39a700 1 --2- 172.21.15.92:0/331830146 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557059ad56b0 0x557059ad0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.394+0000 7fed2d39a700 1 -- 172.21.15.92:0/331830146 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed2000f040 con 0x557059ae9b10 2023-11-28T11:23:23.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.394+0000 7fed2d39a700 1 --2- 172.21.15.92:0/331830146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557059ae9b10 0x557059aee8a0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fed18055430 tx=0x7fed1805d640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44693cdca091ba02 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:23.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.394+0000 7fed08ff9700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-11-28T11:23:23.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.394+0000 7fed08ff9700 1 -- 172.21.15.92:0/331830146 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed1805d380 con 0x557059ae9b10 2023-11-28T11:23:23.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.394+0000 7fed08ff9700 1 -- 172.21.15.92:0/331830146 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(56..56 src has 1..56) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fed180512f0 con 0x557059ae9b10 2023-11-28T11:23:23.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.394+0000 7fed4372ab80 1 -- 172.21.15.92:0/331830146 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557059a03080 con 0x557059ae9b10 2023-11-28T11:23:23.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.394+0000 7fed08ff9700 1 -- 172.21.15.92:0/331830146 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed18060030 con 0x557059ae9b10 2023-11-28T11:23:23.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.394+0000 7fed4372ab80 10 client.5952 fetch_fsmap learned FSMap version 24 2023-11-28T11:23:23.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.394+0000 7fed4372ab80 1 -- 172.21.15.92:0/331830146 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x5570598e00f0 con 0x557059ae9b10 2023-11-28T11:23:23.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.394+0000 7fed08ff9700 1 -- 172.21.15.92:0/331830146 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557059a03080 con 0x557059ae9b10 2023-11-28T11:23:23.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.394+0000 7fed08ff9700 1 -- 172.21.15.92:0/331830146 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 24) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fed180514b0 con 0x557059ae9b10 2023-11-28T11:23:23.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.394+0000 7fed4372ab80 10 client.5952 fetch_fsmap finished waiting for FSMap version 24 2023-11-28T11:23:23.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.394+0000 7fed4372ab80 10 client.5952 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-28T11:23:23.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.394+0000 7fed4372ab80 20 client.5952 populate_metadata read hostname 'smithi092' 2023-11-28T11:23:23.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.394+0000 7fed4372ab80 1 --2- 172.21.15.92:0/331830146 >> [v2:172.21.15.92:6836/950691230,v1:172.21.15.92:6837/950691230] conn(0x557059aed2a0 0x557059aed6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:23.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.395+0000 7fed4372ab80 4 client.5952 mds_command: new command op to 5754 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:23:23.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.395+0000 7fed4372ab80 1 -- 172.21.15.92:0/331830146 --> [v2:172.21.15.92:6836/950691230,v1:172.21.15.92:6837/950691230] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55705984ef40 con 0x557059aed2a0 2023-11-28T11:23:23.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.395+0000 7fed2db9b700 1 --2- 172.21.15.92:0/331830146 >> [v2:172.21.15.92:6836/950691230,v1:172.21.15.92:6837/950691230] conn(0x557059aed2a0 0x557059aed6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:23.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.395+0000 7fed2db9b700 1 --2- 172.21.15.92:0/331830146 >> [v2:172.21.15.92:6836/950691230,v1:172.21.15.92:6837/950691230] conn(0x557059aed2a0 0x557059aed6a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92e7fc90ad4f5580 server_cookie=c06d2e97fe6a95cf in_seq=0 out_seq=0 2023-11-28T11:23:23.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.395+0000 7fed08ff9700 10 client.5952 ms_handle_connect on v2:172.21.15.92:6836/950691230 2023-11-28T11:23:23.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.396+0000 7fed08ff9700 1 -- 172.21.15.92:0/331830146 <== mds.0 v2:172.21.15.92:6836/950691230 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x55705984ef40 con 0x557059aed2a0 2023-11-28T11:23:23.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.396+0000 7fed08ff9700 10 client.5952 handle_command_reply: tid=0 2023-11-28T11:23:23.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.396+0000 7fed4372ab80 1 client.5952 shutdown 2023-11-28T11:23:23.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.396+0000 7fed4372ab80 1 -- 172.21.15.92:0/331830146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557059ae9b10 msgr2=0x557059aee8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:23.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.396+0000 7fed4372ab80 1 --2- 172.21.15.92:0/331830146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557059ae9b10 0x557059aee8a0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fed18055430 tx=0x7fed1805d640 comp rx=0 tx=0).stop 2023-11-28T11:23:23.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.396+0000 7fed4372ab80 1 -- 172.21.15.92:0/331830146 shutdown_connections 2023-11-28T11:23:23.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.396+0000 7fed4372ab80 1 --2- 172.21.15.92:0/331830146 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557059af2fa0 0x557059ad7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.396+0000 7fed4372ab80 1 --2- 172.21.15.92:0/331830146 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557059ad56b0 0x557059ad0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.396+0000 7fed4372ab80 1 --2- 172.21.15.92:0/331830146 >> [v2:172.21.15.92:6836/950691230,v1:172.21.15.92:6837/950691230] conn(0x557059aed2a0 0x557059aed6a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.396+0000 7fed4372ab80 1 --2- 172.21.15.92:0/331830146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557059ae9b10 0x557059aee8a0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.396+0000 7fed08ff9700 0 client.5952 ms_handle_reset on v2:172.21.15.92:6836/950691230 2023-11-28T11:23:23.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.396+0000 7fed4372ab80 1 -- 172.21.15.92:0/331830146 >> 172.21.15.92:0/331830146 conn(0x557059aa2d70 msgr2=0x557059ad3770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:23.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.396+0000 7fed4372ab80 1 -- 172.21.15.92:0/331830146 shutdown_connections 2023-11-28T11:23:23.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.396+0000 7fed4372ab80 1 -- 172.21.15.92:0/331830146 wait complete. 2023-11-28T11:23:23.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.397+0000 7fed4372ab80 20 client.5952 trim_cache size 0 max 16384 2023-11-28T11:23:23.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.413+0000 7fed4372ab80 1 Processor -- start 2023-11-28T11:23:23.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.413+0000 7fed4372ab80 1 -- start start 2023-11-28T11:23:23.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.414+0000 7fed4372ab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557059ad56b0 0x557059af0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:23.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.414+0000 7fed4372ab80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557059ae9b10 0x557059ad4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:23.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.414+0000 7fed4372ab80 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557059af2fa0 0x557059ad3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:23.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.414+0000 7fed4372ab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x557059afbc00 con 0x557059ad56b0 2023-11-28T11:23:23.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.414+0000 7fed4372ab80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x557059af0240 con 0x557059ae9b10 2023-11-28T11:23:23.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.414+0000 7fed4372ab80 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x557059aefcd0 con 0x557059af2fa0 2023-11-28T11:23:23.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.414+0000 7fed2db9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557059ad56b0 0x557059af0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:23.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.414+0000 7fed2db9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557059ad56b0 0x557059af0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58648/0 (socket says 172.21.15.92:58648) 2023-11-28T11:23:23.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.414+0000 7fed2db9b700 1 -- 172.21.15.92:0/1870768085 learned_addr learned my addr 172.21.15.92:0/1870768085 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:23.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.415+0000 7fed2e39c700 1 --2- 172.21.15.92:0/1870768085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557059af2fa0 0x557059ad3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:23.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.415+0000 7fed2d39a700 1 --2- 172.21.15.92:0/1870768085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557059ae9b10 0x557059ad4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:23.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.415+0000 7fed2db9b700 1 -- 172.21.15.92:0/1870768085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557059af2fa0 msgr2=0x557059ad3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:23.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.415+0000 7fed2db9b700 1 --2- 172.21.15.92:0/1870768085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557059af2fa0 0x557059ad3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.415+0000 7fed2db9b700 1 -- 172.21.15.92:0/1870768085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557059ae9b10 msgr2=0x557059ad4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:23.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.415+0000 7fed2db9b700 1 --2- 172.21.15.92:0/1870768085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557059ae9b10 0x557059ad4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.415+0000 7fed2db9b700 1 -- 172.21.15.92:0/1870768085 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5570599c6330 con 0x557059ad56b0 2023-11-28T11:23:23.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.415+0000 7fed2db9b700 1 --2- 172.21.15.92:0/1870768085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557059ad56b0 0x557059af0ab0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fed24009250 tx=0x7fed24010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8df5f8d467710816 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:23.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.415+0000 7fed0bfff700 1 -- 172.21.15.92:0/1870768085 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed24034040 con 0x557059ad56b0 2023-11-28T11:23:23.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.415+0000 7fed0bfff700 1 -- 172.21.15.92:0/1870768085 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed24038070 con 0x557059ad56b0 2023-11-28T11:23:23.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.415+0000 7fed0bfff700 1 -- 172.21.15.92:0/1870768085 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed240343a0 con 0x557059ad56b0 2023-11-28T11:23:23.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.416+0000 7fed4372ab80 1 -- 172.21.15.92:0/1870768085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557059ad56b0 msgr2=0x557059af0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:23.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.416+0000 7fed4372ab80 1 --2- 172.21.15.92:0/1870768085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557059ad56b0 0x557059af0ab0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fed24009250 tx=0x7fed24010000 comp rx=0 tx=0).stop 2023-11-28T11:23:23.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.416+0000 7fed4372ab80 1 -- 172.21.15.92:0/1870768085 shutdown_connections 2023-11-28T11:23:23.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.416+0000 7fed4372ab80 1 --2- 172.21.15.92:0/1870768085 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557059af2fa0 0x557059ad3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.416+0000 7fed4372ab80 1 --2- 172.21.15.92:0/1870768085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557059ad56b0 0x557059af0ab0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.416+0000 7fed4372ab80 1 --2- 172.21.15.92:0/1870768085 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557059ae9b10 0x557059ad4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.416+0000 7fed4372ab80 1 -- 172.21.15.92:0/1870768085 >> 172.21.15.92:0/1870768085 conn(0x557059aa2d70 msgr2=0x557059af3370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:23.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.416+0000 7fed4372ab80 1 -- 172.21.15.92:0/1870768085 shutdown_connections 2023-11-28T11:23:23.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.416+0000 7fed4372ab80 1 -- 172.21.15.92:0/1870768085 wait complete. 2023-11-28T11:23:23.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.417+0000 7fed4372ab80 1 -- start start 2023-11-28T11:23:23.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.417+0000 7fed4372ab80 1 Processor -- start 2023-11-28T11:23:23.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.418+0000 7fed4372ab80 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557059ad56b0 0x557059ad3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:23.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.418+0000 7fed4372ab80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557059ae9b10 0x557059b40d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:23.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.418+0000 7fed4372ab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557059af2fa0 0x557059b02760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:23.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.418+0000 7fed4372ab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x557059aee8e0 con 0x557059af2fa0 2023-11-28T11:23:23.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.418+0000 7fed4372ab80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x557059b5bc80 con 0x557059ae9b10 2023-11-28T11:23:23.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.418+0000 7fed4372ab80 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x557059ad0c50 con 0x557059ad56b0 2023-11-28T11:23:23.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.418+0000 7fed4372ab80 10 client.0 _ll_register_callbacks cb 0x55705999d450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:23:23.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.418+0000 7fed2e39c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557059af2fa0 0x557059b02760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:23.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.418+0000 7fed2e39c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557059af2fa0 0x557059b02760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58660/0 (socket says 172.21.15.92:58660) 2023-11-28T11:23:23.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.418+0000 7fed2e39c700 1 -- 172.21.15.92:0/2854806899 learned_addr learned my addr 172.21.15.92:0/2854806899 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:23.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.418+0000 7fed2db9b700 1 --2- 172.21.15.92:0/2854806899 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557059ad56b0 0x557059ad3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:23.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.418+0000 7fed2d39a700 1 --2- 172.21.15.92:0/2854806899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557059ae9b10 0x557059b40d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:23.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.418+0000 7fed2e39c700 1 -- 172.21.15.92:0/2854806899 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557059ad56b0 msgr2=0x557059ad3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:23.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.419+0000 7fed2e39c700 1 --2- 172.21.15.92:0/2854806899 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557059ad56b0 0x557059ad3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.419+0000 7fed2e39c700 1 -- 172.21.15.92:0/2854806899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557059ae9b10 msgr2=0x557059b40d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:23.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.419+0000 7fed2e39c700 1 --2- 172.21.15.92:0/2854806899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557059ae9b10 0x557059b40d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.419+0000 7fed2e39c700 1 -- 172.21.15.92:0/2854806899 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed280cfd10 con 0x557059af2fa0 2023-11-28T11:23:23.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.419+0000 7fed2d39a700 1 --2- 172.21.15.92:0/2854806899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557059ae9b10 0x557059b40d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:23:23.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.419+0000 7fed2e39c700 1 --2- 172.21.15.92:0/2854806899 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557059af2fa0 0x557059b02760 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fed20017040 tx=0x7fed20001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86187358c619f62c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:23.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.419+0000 7fed08ff9700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-11-28T11:23:23.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.419+0000 7fed08ff9700 1 -- 172.21.15.92:0/2854806899 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed200018f0 con 0x557059af2fa0 2023-11-28T11:23:23.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.419+0000 7fed08ff9700 1 -- 172.21.15.92:0/2854806899 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(56..56 src has 1..56) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fed20014df0 con 0x557059af2fa0 2023-11-28T11:23:23.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.419+0000 7fed4372ab80 1 -- 172.21.15.92:0/2854806899 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fed200145e0 con 0x557059af2fa0 2023-11-28T11:23:23.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.419+0000 7fed08ff9700 1 -- 172.21.15.92:0/2854806899 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed20029460 con 0x557059af2fa0 2023-11-28T11:23:23.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.419+0000 7fed4372ab80 10 client.5958 fetch_fsmap learned FSMap version 24 2023-11-28T11:23:23.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.419+0000 7fed4372ab80 1 -- 172.21.15.92:0/2854806899 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x557059854220 con 0x557059af2fa0 2023-11-28T11:23:23.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.420+0000 7fed08ff9700 1 -- 172.21.15.92:0/2854806899 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed200145e0 con 0x557059af2fa0 2023-11-28T11:23:23.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.420+0000 7fed08ff9700 1 -- 172.21.15.92:0/2854806899 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 24) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fed200295c0 con 0x557059af2fa0 2023-11-28T11:23:23.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.420+0000 7fed4372ab80 10 client.5958 fetch_fsmap finished waiting for FSMap version 24 2023-11-28T11:23:23.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.420+0000 7fed4372ab80 10 client.5958 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-28T11:23:23.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.420+0000 7fed4372ab80 20 client.5958 populate_metadata read hostname 'smithi092' 2023-11-28T11:23:23.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.420+0000 7fed4372ab80 1 --2- 172.21.15.92:0/2854806899 >> [v2:172.21.15.92:6836/950691230,v1:172.21.15.92:6837/950691230] conn(0x557059b5b4a0 0x557059b02ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:23.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.420+0000 7fed4372ab80 4 client.5958 mds_command: new command op to 5754 tid=0[{"prefix": "scrub status"}] 2023-11-28T11:23:23.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.420+0000 7fed4372ab80 1 -- 172.21.15.92:0/2854806899 --> [v2:172.21.15.92:6836/950691230,v1:172.21.15.92:6837/950691230] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557059884090 con 0x557059b5b4a0 2023-11-28T11:23:23.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.420+0000 7fed2db9b700 1 --2- 172.21.15.92:0/2854806899 >> [v2:172.21.15.92:6836/950691230,v1:172.21.15.92:6837/950691230] conn(0x557059b5b4a0 0x557059b02ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:23.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.420+0000 7fed2db9b700 1 --2- 172.21.15.92:0/2854806899 >> [v2:172.21.15.92:6836/950691230,v1:172.21.15.92:6837/950691230] conn(0x557059b5b4a0 0x557059b02ca0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea7777eae8e9ad92 server_cookie=3b7f01fbb0bb472c in_seq=0 out_seq=0 2023-11-28T11:23:23.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.420+0000 7fed08ff9700 10 client.5958 ms_handle_connect on v2:172.21.15.92:6836/950691230 2023-11-28T11:23:23.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.421+0000 7fed08ff9700 1 -- 172.21.15.92:0/2854806899 <== mds.0 v2:172.21.15.92:6836/950691230 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x557059884090 con 0x557059b5b4a0 2023-11-28T11:23:23.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.421+0000 7fed08ff9700 10 client.5958 handle_command_reply: tid=0 2023-11-28T11:23:23.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.421+0000 7fed4372ab80 1 client.5958 shutdown 2023-11-28T11:23:23.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.421+0000 7fed4372ab80 1 -- 172.21.15.92:0/2854806899 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557059af2fa0 msgr2=0x557059b02760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:23.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.421+0000 7fed4372ab80 1 --2- 172.21.15.92:0/2854806899 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557059af2fa0 0x557059b02760 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fed20017040 tx=0x7fed20001250 comp rx=0 tx=0).stop 2023-11-28T11:23:23.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.421+0000 7fed4372ab80 1 -- 172.21.15.92:0/2854806899 shutdown_connections 2023-11-28T11:23:23.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.421+0000 7fed4372ab80 1 --2- 172.21.15.92:0/2854806899 >> [v2:172.21.15.92:6836/950691230,v1:172.21.15.92:6837/950691230] conn(0x557059b5b4a0 0x557059b02ca0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.421+0000 7fed4372ab80 1 --2- 172.21.15.92:0/2854806899 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557059af2fa0 0x557059b02760 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.421+0000 7fed4372ab80 1 --2- 172.21.15.92:0/2854806899 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x557059ad56b0 0x557059ad3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.421+0000 7fed08ff9700 0 client.5958 ms_handle_reset on v2:172.21.15.92:6836/950691230 2023-11-28T11:23:23.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.421+0000 7fed4372ab80 1 --2- 172.21.15.92:0/2854806899 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x557059ae9b10 0x557059b40d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.421+0000 7fed4372ab80 1 -- 172.21.15.92:0/2854806899 >> 172.21.15.92:0/2854806899 conn(0x557059aa2d70 msgr2=0x557059af3370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:23.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.421+0000 7fed4372ab80 1 -- 172.21.15.92:0/2854806899 shutdown_connections 2023-11-28T11:23:23.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.421+0000 7fed4372ab80 1 -- 172.21.15.92:0/2854806899 wait complete. 2023-11-28T11:23:23.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.421+0000 7fed4372ab80 20 client.5958 trim_cache size 0 max 16384 2023-11-28T11:23:23.427 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-28T11:23:23.428 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-28T11:23:23.428 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": { 2023-11-28T11:23:23.428 INFO:teuthology.orchestra.run.smithi092.stdout: "1ac37aa7-e916-4500-afd8-79713c734769": { 2023-11-28T11:23:23.428 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2023-11-28T11:23:23.428 INFO:teuthology.orchestra.run.smithi092.stdout: "tag": "1ac37aa7-e916-4500-afd8-79713c734769", 2023-11-28T11:23:23.428 INFO:teuthology.orchestra.run.smithi092.stdout: "options": "recursive,repair" 2023-11-28T11:23:23.429 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:23:23.429 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:23:23.429 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-28T11:23:23.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.422+0000 7fed2fdff700 1 -- 172.21.15.92:0/3102941578 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fed0c0682f0 msgr2=0x7fed0c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:23.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.422+0000 7fed2fdff700 1 --2- 172.21.15.92:0/3102941578 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fed0c0682f0 0x7fed0c06a7a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fed240090b0 tx=0x7fed2401c040 comp rx=0 tx=0).stop 2023-11-28T11:23:23.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.422+0000 7fed2fdff700 1 -- 172.21.15.92:0/3102941578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed280cd460 msgr2=0x7fed281e5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:23.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.422+0000 7fed2fdff700 1 --2- 172.21.15.92:0/3102941578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed280cd460 0x7fed281e5600 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fed1800c480 tx=0x7fed1800c790 comp rx=0 tx=0).stop 2023-11-28T11:23:23.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.422+0000 7fed2fdff700 1 -- 172.21.15.92:0/3102941578 shutdown_connections 2023-11-28T11:23:23.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.422+0000 7fed2fdff700 1 --2- 172.21.15.92:0/3102941578 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fed280d0d60 0x7fed281e6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.422+0000 7fed2fdff700 1 --2- 172.21.15.92:0/3102941578 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fed280b8510 0x7fed2815a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.422+0000 7fed2fdff700 1 --2- 172.21.15.92:0/3102941578 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fed0c0682f0 0x7fed0c06a7a0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.422+0000 7fed2fdff700 1 --2- 172.21.15.92:0/3102941578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed280cd460 0x7fed281e5600 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:23.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.422+0000 7fed2fdff700 1 -- 172.21.15.92:0/3102941578 >> 172.21.15.92:0/3102941578 conn(0x7fed280719a0 msgr2=0x7fed280c9680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:23.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.422+0000 7fed2fdff700 1 -- 172.21.15.92:0/3102941578 shutdown_connections 2023-11-28T11:23:23.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.422+0000 7fed2fdff700 1 -- 172.21.15.92:0/3102941578 wait complete. 2023-11-28T11:23:23.440 INFO:tasks.cephfs.filesystem:scrub status for tag:1ac37aa7-e916-4500-afd8-79713c734769 - {'path': '/', 'tag': '1ac37aa7-e916-4500-afd8-79713c734769', 'options': 'recursive,repair'} 2023-11-28T11:23:23.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:23.547+0000 7fed4cff9700 1 -- 172.21.15.92:0/3166003025 <== mon.0 v2:172.21.15.92:3300/0 7 ==== log(6 entries from seq 1 at 2023-11-28T11:23:23.157688+0000) v1 ==== 1078+0+0 (secure 0 0 0) 0x7fed4805ed30 con 0x7fed5015c740 2023-11-28T11:23:23.548 INFO:teuthology.orchestra.run.smithi092.stdout: cluster: 2023-11-28T11:23:23.548 INFO:teuthology.orchestra.run.smithi092.stdout: id: 68956e0c-c16f-47cd-a220-f98364b0831d 2023-11-28T11:23:23.549 INFO:teuthology.orchestra.run.smithi092.stdout: health: HEALTH_OK 2023-11-28T11:23:23.549 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:23:23.549 INFO:teuthology.orchestra.run.smithi092.stdout: services: 2023-11-28T11:23:23.549 INFO:teuthology.orchestra.run.smithi092.stdout: mon: 3 daemons, quorum a,b,c (age 2m) 2023-11-28T11:23:23.549 INFO:teuthology.orchestra.run.smithi092.stdout: mgr: y(active, since 2m), standbys: x 2023-11-28T11:23:23.549 INFO:teuthology.orchestra.run.smithi092.stdout: mds: 1/1 daemons up, 3 standby 2023-11-28T11:23:23.549 INFO:teuthology.orchestra.run.smithi092.stdout: osd: 8 osds: 8 up (since 2m), 8 in (since 2m) 2023-11-28T11:23:23.549 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:23:23.549 INFO:teuthology.orchestra.run.smithi092.stdout: data: 2023-11-28T11:23:23.549 INFO:teuthology.orchestra.run.smithi092.stdout: volumes: 1/1 healthy 2023-11-28T11:23:23.549 INFO:teuthology.orchestra.run.smithi092.stdout: pools: 4 pools, 105 pgs 2023-11-28T11:23:23.550 INFO:teuthology.orchestra.run.smithi092.stdout: objects: 23 objects, 579 KiB 2023-11-28T11:23:23.550 INFO:teuthology.orchestra.run.smithi092.stdout: usage: 75 MiB used, 720 GiB / 720 GiB avail 2023-11-28T11:23:23.550 INFO:teuthology.orchestra.run.smithi092.stdout: pgs: 105 active+clean 2023-11-28T11:23:23.550 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:23:23.550 INFO:teuthology.orchestra.run.smithi092.stdout: io: 2023-11-28T11:23:23.550 INFO:teuthology.orchestra.run.smithi092.stdout: client: 1.6 KiB/s wr, 0 op/s rd, 5 op/s wr 2023-11-28T11:23:23.550 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:23:23.550 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:23:23.550 INFO:teuthology.orchestra.run.smithi092.stdout:2023-11-28T11:23:23.157688+0000 mds.c [INF] scrub queued for path: / 2023-11-28T11:23:23.550 INFO:teuthology.orchestra.run.smithi092.stdout:2023-11-28T11:23:23.157692+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2023-11-28T11:23:23.550 INFO:teuthology.orchestra.run.smithi092.stdout:2023-11-28T11:23:23.157696+0000 mds.c [INF] scrub summary: active paths [/] 2023-11-28T11:23:23.550 INFO:teuthology.orchestra.run.smithi092.stdout:2023-11-28T11:23:23.158865+0000 mds.c [WRN] bad backtrace on inode 0x10000000001(/parent_a/alpha), rewriting it 2023-11-28T11:23:23.551 INFO:teuthology.orchestra.run.smithi092.stdout:2023-11-28T11:23:23.158880+0000 mds.c [INF] Scrub repaired inode 0x10000000001 (/parent_a/alpha) 2023-11-28T11:23:23.551 INFO:teuthology.orchestra.run.smithi092.stdout:2023-11-28T11:23:23.159378+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2023-11-28T11:23:28.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:28.557+0000 7fed4cff9700 1 -- 172.21.15.92:0/3166003025 <== mon.0 v2:172.21.15.92:3300/0 8 ==== log(1 entries from seq 7 at 2023-11-28T11:23:27.230848+0000) v1 ==== 189+0+0 (secure 0 0 0) 0x7fed48054de0 con 0x7fed5015c740 2023-11-28T11:23:28.559 INFO:teuthology.orchestra.run.smithi092.stdout:2023-11-28T11:23:27.230848+0000 mds.c [INF] scrub summary: idle 2023-11-28T11:23:48.420 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:23:48.426 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:23:53.441 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub status 2023-11-28T11:23:53.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.647+0000 7f30b03ff700 1 Processor -- start 2023-11-28T11:23:53.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.647+0000 7f30b03ff700 1 -- start start 2023-11-28T11:23:53.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.648+0000 7f30b03ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f30a80b8510 0x7f30a806b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:53.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.648+0000 7f30b03ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30a80b88e0 con 0x7f30a80d0d60 2023-11-28T11:23:53.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.648+0000 7f30b03ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30a80cd8f0 con 0x7f30a80cd460 2023-11-28T11:23:53.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.648+0000 7f30b03ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30a80cdbd0 con 0x7f30a80b8510 2023-11-28T11:23:53.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.648+0000 7f30ae19b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f30a80b8510 0x7f30a806b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:53.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.648+0000 7f30ae99c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f30a80d0d60 0x7f30a80d1200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36404/0 (socket says 172.21.15.92:36404) 2023-11-28T11:23:53.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.648+0000 7f30ae99c700 1 -- 172.21.15.92:0/3474067287 learned_addr learned my addr 172.21.15.92:0/3474067287 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:23:53.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.649+0000 7f30ae19b700 1 -- 172.21.15.92:0/3474067287 >> v1:172.21.15.169:6789/0 conn(0x7f30a80cd460 legacy=0x7f30a806beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:53.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.649+0000 7f30ae19b700 1 -- 172.21.15.92:0/3474067287 >> v1:172.21.15.92:6789/0 conn(0x7f30a80d0d60 legacy=0x7f30a80d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:53.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.649+0000 7f30ae19b700 1 -- 172.21.15.92:0/3474067287 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30a80cfd10 con 0x7f30a80b8510 2023-11-28T11:23:53.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.649+0000 7f30ad199700 1 -- 172.21.15.92:0/3474067287 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1108776267 0 0) 0x7f30a80b88e0 con 0x7f30a80d0d60 2023-11-28T11:23:53.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.649+0000 7f30ae19b700 1 --2- 172.21.15.92:0/3474067287 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30a80b8510 0x7f30a806b970 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f30a4009840 tx=0x7f30a4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23ffdbbbd5671eec server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:53.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.649+0000 7f30ad199700 1 -- 172.21.15.92:0/3474067287 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a4011830 con 0x7f30a80b8510 2023-11-28T11:23:53.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.649+0000 7f30ad199700 1 -- 172.21.15.92:0/3474067287 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30a4011990 con 0x7f30a80b8510 2023-11-28T11:23:53.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.649+0000 7f30ad199700 1 -- 172.21.15.92:0/3474067287 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a4011d20 con 0x7f30a80b8510 2023-11-28T11:23:53.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.650+0000 7f30b03ff700 1 -- 172.21.15.92:0/3474067287 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30a80b8510 msgr2=0x7f30a806b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:53.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.650+0000 7f30b03ff700 1 --2- 172.21.15.92:0/3474067287 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30a80b8510 0x7f30a806b970 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f30a4009840 tx=0x7f30a4009b50 comp rx=0 tx=0).stop 2023-11-28T11:23:53.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.650+0000 7f30b03ff700 1 -- 172.21.15.92:0/3474067287 shutdown_connections 2023-11-28T11:23:53.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.650+0000 7f30b03ff700 1 --2- 172.21.15.92:0/3474067287 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30a80b8510 0x7f30a806b970 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:53.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.650+0000 7f30b03ff700 1 -- 172.21.15.92:0/3474067287 >> 172.21.15.92:0/3474067287 conn(0x7f30a80719a0 msgr2=0x7f30a8071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:53.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.650+0000 7f30b03ff700 1 -- 172.21.15.92:0/3474067287 shutdown_connections 2023-11-28T11:23:53.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.650+0000 7f30b03ff700 1 -- 172.21.15.92:0/3474067287 wait complete. 2023-11-28T11:23:53.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.652+0000 7f30b03ff700 1 Processor -- start 2023-11-28T11:23:53.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.652+0000 7f30b03ff700 1 -- start start 2023-11-28T11:23:53.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.654+0000 7f30b03ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30a80b8510 0x7f30a815a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:53.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.654+0000 7f30b03ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30a80cd460 0x7f30a81e5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:53.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.654+0000 7f30b03ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30a80d0d60 0x7f30a81e6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:53.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.654+0000 7f30b03ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a815acf0 con 0x7f30a80d0d60 2023-11-28T11:23:53.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.654+0000 7f30b03ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a81e4560 con 0x7f30a80b8510 2023-11-28T11:23:53.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.654+0000 7f30b03ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30a81e4840 con 0x7f30a80cd460 2023-11-28T11:23:53.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.654+0000 7f30ae99c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30a80d0d60 0x7f30a81e6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:53.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.654+0000 7f30ae99c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30a80d0d60 0x7f30a81e6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52080/0 (socket says 172.21.15.92:52080) 2023-11-28T11:23:53.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.654+0000 7f30ae99c700 1 -- 172.21.15.92:0/3599219950 learned_addr learned my addr 172.21.15.92:0/3599219950 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:53.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.654+0000 7f30ae19b700 1 --2- 172.21.15.92:0/3599219950 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30a80b8510 0x7f30a815a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:53.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.654+0000 7f30ae99c700 1 -- 172.21.15.92:0/3599219950 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30a80cd460 msgr2=0x7f30a81e5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:53.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.654+0000 7f30ad99a700 1 --2- 172.21.15.92:0/3599219950 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30a80cd460 0x7f30a81e5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:53.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.654+0000 7f30ae99c700 1 --2- 172.21.15.92:0/3599219950 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30a80cd460 0x7f30a81e5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:53.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.654+0000 7f30ae99c700 1 -- 172.21.15.92:0/3599219950 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30a80b8510 msgr2=0x7f30a815a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:53.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.654+0000 7f30ae99c700 1 --2- 172.21.15.92:0/3599219950 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30a80b8510 0x7f30a815a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:53.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.654+0000 7f30ae99c700 1 -- 172.21.15.92:0/3599219950 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30a40094a0 con 0x7f30a80d0d60 2023-11-28T11:23:53.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.654+0000 7f30ad99a700 1 --2- 172.21.15.92:0/3599219950 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30a80cd460 0x7f30a81e5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:23:53.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.655+0000 7f30ae99c700 1 --2- 172.21.15.92:0/3599219950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30a80d0d60 0x7f30a81e6840 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f30a000cba0 tx=0x7f30a000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:53.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.655+0000 7f309effd700 1 -- 172.21.15.92:0/3599219950 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a000ab80 con 0x7f30a80d0d60 2023-11-28T11:23:53.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.655+0000 7f309effd700 1 -- 172.21.15.92:0/3599219950 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30a000ad00 con 0x7f30a80d0d60 2023-11-28T11:23:53.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.655+0000 7f30b03ff700 1 -- 172.21.15.92:0/3599219950 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30a80d29c0 con 0x7f30a80d0d60 2023-11-28T11:23:53.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.655+0000 7f309effd700 1 -- 172.21.15.92:0/3599219950 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a0007310 con 0x7f30a80d0d60 2023-11-28T11:23:53.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.655+0000 7f30b03ff700 1 -- 172.21.15.92:0/3599219950 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30a80cfff0 con 0x7f30a80d0d60 2023-11-28T11:23:53.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.657+0000 7f309effd700 1 -- 172.21.15.92:0/3599219950 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f30a0010070 con 0x7f30a80d0d60 2023-11-28T11:23:53.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.658+0000 7f309effd700 1 --2- 172.21.15.92:0/3599219950 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f308c0682f0 0x7f308c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:53.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.658+0000 7f309effd700 1 -- 172.21.15.92:0/3599219950 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f30a0085890 con 0x7f30a80d0d60 2023-11-28T11:23:53.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.658+0000 7f30ae19b700 1 --2- 172.21.15.92:0/3599219950 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f308c0682f0 0x7f308c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:53.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.659+0000 7f30ae19b700 1 --2- 172.21.15.92:0/3599219950 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f308c0682f0 0x7f308c06a7a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f30a4000c00 tx=0x7f30a401c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:53.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.660+0000 7f30c3cfeb80 1 Processor -- start 2023-11-28T11:23:53.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.661+0000 7f30c3cfeb80 1 -- start start 2023-11-28T11:23:53.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.662+0000 7f30c3cfeb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560831039fa0 0x560830ff3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:53.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.662+0000 7f30c3cfeb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560831030b10 0x56083103a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:53.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.662+0000 7f30c3cfeb80 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56083101c6b0 0x560830ff4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:53.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.662+0000 7f30c3cfeb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x560830f71e30 con 0x560831030b10 2023-11-28T11:23:53.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.662+0000 7f30c3cfeb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x560830ede190 con 0x560831039fa0 2023-11-28T11:23:53.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.662+0000 7f30c3cfeb80 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x560830f4ce00 con 0x56083101c6b0 2023-11-28T11:23:53.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.662+0000 7f30ad99a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560831030b10 0x56083103a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:53.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.662+0000 7f30ad99a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560831030b10 0x56083103a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52082/0 (socket says 172.21.15.92:52082) 2023-11-28T11:23:53.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.662+0000 7f30ad99a700 1 -- 172.21.15.92:0/512548942 learned_addr learned my addr 172.21.15.92:0/512548942 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:53.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.662+0000 7f30ae19b700 1 --2- 172.21.15.92:0/512548942 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560831039fa0 0x560830ff3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:53.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.662+0000 7f30ae99c700 1 --2- 172.21.15.92:0/512548942 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56083101c6b0 0x560830ff4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:53.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.662+0000 7f30ad99a700 1 -- 172.21.15.92:0/512548942 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56083101c6b0 msgr2=0x560830ff4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:53.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.662+0000 7f30ad99a700 1 --2- 172.21.15.92:0/512548942 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56083101c6b0 0x560830ff4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:53.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.662+0000 7f30ad99a700 1 -- 172.21.15.92:0/512548942 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560831039fa0 msgr2=0x560830ff3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:53.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.662+0000 7f30ad99a700 1 --2- 172.21.15.92:0/512548942 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560831039fa0 0x560830ff3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:53.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.662+0000 7f30ad99a700 1 -- 172.21.15.92:0/512548942 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560830d3a7e0 con 0x560831030b10 2023-11-28T11:23:53.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.663+0000 7f30ad99a700 1 --2- 172.21.15.92:0/512548942 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560831030b10 0x56083103a370 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f30980016c0 tx=0x7f30980019d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ab3d846ffa99909 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:53.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.663+0000 7f308bfff700 1 -- 172.21.15.92:0/512548942 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309800b070 con 0x560831030b10 2023-11-28T11:23:53.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.663+0000 7f308bfff700 1 -- 172.21.15.92:0/512548942 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30980054e0 con 0x560831030b10 2023-11-28T11:23:53.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.663+0000 7f308bfff700 1 -- 172.21.15.92:0/512548942 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30980038a0 con 0x560831030b10 2023-11-28T11:23:53.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.663+0000 7f30c3cfeb80 1 -- 172.21.15.92:0/512548942 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560831030b10 msgr2=0x56083103a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:53.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.663+0000 7f30c3cfeb80 1 --2- 172.21.15.92:0/512548942 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560831030b10 0x56083103a370 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f30980016c0 tx=0x7f30980019d0 comp rx=0 tx=0).stop 2023-11-28T11:23:53.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.664+0000 7f30c3cfeb80 1 -- 172.21.15.92:0/512548942 shutdown_connections 2023-11-28T11:23:53.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.664+0000 7f30c3cfeb80 1 --2- 172.21.15.92:0/512548942 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56083101c6b0 0x560830ff4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:53.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.664+0000 7f30c3cfeb80 1 --2- 172.21.15.92:0/512548942 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560831039fa0 0x560830ff3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:53.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.664+0000 7f30c3cfeb80 1 --2- 172.21.15.92:0/512548942 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560831030b10 0x56083103a370 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:53.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.664+0000 7f30c3cfeb80 1 -- 172.21.15.92:0/512548942 >> 172.21.15.92:0/512548942 conn(0x560830fe9d70 msgr2=0x560830ff2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:53.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.664+0000 7f30c3cfeb80 1 -- 172.21.15.92:0/512548942 shutdown_connections 2023-11-28T11:23:53.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.664+0000 7f30c3cfeb80 1 -- 172.21.15.92:0/512548942 wait complete. 2023-11-28T11:23:53.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.664+0000 7f30c3cfeb80 1 -- start start 2023-11-28T11:23:53.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.665+0000 7f30c3cfeb80 1 Processor -- start 2023-11-28T11:23:53.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.665+0000 7f30c3cfeb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56083101c6b0 0x56083101a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:53.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.665+0000 7f30c3cfeb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560831030b10 0x560831034cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:53.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.666+0000 7f30c3cfeb80 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560831039fa0 0x560831038df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:53.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.666+0000 7f30c3cfeb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x560830f46170 con 0x560831030b10 2023-11-28T11:23:53.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.666+0000 7f30c3cfeb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x560830d3b3e0 con 0x56083101c6b0 2023-11-28T11:23:53.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.666+0000 7f30c3cfeb80 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x560830ede390 con 0x560831039fa0 2023-11-28T11:23:53.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.666+0000 7f30c3cfeb80 10 client.0 _ll_register_callbacks cb 0x560830ee4450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:23:53.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.666+0000 7f30ad99a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560831030b10 0x560831034cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:53.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.666+0000 7f30ad99a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560831030b10 0x560831034cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52096/0 (socket says 172.21.15.92:52096) 2023-11-28T11:23:53.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.666+0000 7f30ad99a700 1 -- 172.21.15.92:0/1942571150 learned_addr learned my addr 172.21.15.92:0/1942571150 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:53.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.666+0000 7f30ae19b700 1 --2- 172.21.15.92:0/1942571150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56083101c6b0 0x56083101a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:53.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.666+0000 7f30ae99c700 1 --2- 172.21.15.92:0/1942571150 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560831039fa0 0x560831038df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:53.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.666+0000 7f30ad99a700 1 -- 172.21.15.92:0/1942571150 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560831039fa0 msgr2=0x560831038df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:53.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.666+0000 7f30ad99a700 1 --2- 172.21.15.92:0/1942571150 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560831039fa0 0x560831038df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:53.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.666+0000 7f30ad99a700 1 -- 172.21.15.92:0/1942571150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56083101c6b0 msgr2=0x56083101a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:53.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.666+0000 7f30ad99a700 1 --2- 172.21.15.92:0/1942571150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56083101c6b0 0x56083101a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:53.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.666+0000 7f30ad99a700 1 -- 172.21.15.92:0/1942571150 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30a80cd8f0 con 0x560831030b10 2023-11-28T11:23:53.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.666+0000 7f30ae19b700 1 --2- 172.21.15.92:0/1942571150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56083101c6b0 0x56083101a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:23:53.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.666+0000 7f30ae99c700 1 --2- 172.21.15.92:0/1942571150 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560831039fa0 0x560831038df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:23:53.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.666+0000 7f30ad99a700 1 --2- 172.21.15.92:0/1942571150 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560831030b10 0x560831034cb0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f30980015b0 tx=0x7f3098000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=138c453e6dfbd8a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:53.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.666+0000 7f30897fa700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-11-28T11:23:53.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.667+0000 7f30897fa700 1 -- 172.21.15.92:0/1942571150 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3098007ae0 con 0x560831030b10 2023-11-28T11:23:53.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.667+0000 7f30897fa700 1 -- 172.21.15.92:0/1942571150 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(56..56 src has 1..56) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f30980052e0 con 0x560831030b10 2023-11-28T11:23:53.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.667+0000 7f30c3cfeb80 1 -- 172.21.15.92:0/1942571150 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560830f4a080 con 0x560831030b10 2023-11-28T11:23:53.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.667+0000 7f30897fa700 1 -- 172.21.15.92:0/1942571150 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3098001240 con 0x560831030b10 2023-11-28T11:23:53.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.667+0000 7f30c3cfeb80 10 client.5973 fetch_fsmap learned FSMap version 24 2023-11-28T11:23:53.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.667+0000 7f30c3cfeb80 1 -- 172.21.15.92:0/1942571150 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x560830e270f0 con 0x560831030b10 2023-11-28T11:23:53.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.667+0000 7f30897fa700 1 -- 172.21.15.92:0/1942571150 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560830f4a080 con 0x560831030b10 2023-11-28T11:23:53.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.667+0000 7f30897fa700 1 -- 172.21.15.92:0/1942571150 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 24) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f3098013680 con 0x560831030b10 2023-11-28T11:23:53.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.667+0000 7f30c3cfeb80 10 client.5973 fetch_fsmap finished waiting for FSMap version 24 2023-11-28T11:23:53.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.667+0000 7f30c3cfeb80 10 client.5973 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-28T11:23:53.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.667+0000 7f30c3cfeb80 20 client.5973 populate_metadata read hostname 'smithi092' 2023-11-28T11:23:53.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.667+0000 7f30c3cfeb80 1 --2- 172.21.15.92:0/1942571150 >> [v2:172.21.15.92:6836/950691230,v1:172.21.15.92:6837/950691230] conn(0x56083104c6d0 0x56083104cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:53.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.667+0000 7f30c3cfeb80 4 client.5973 mds_command: new command op to 5754 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:23:53.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.667+0000 7f30c3cfeb80 1 -- 172.21.15.92:0/1942571150 --> [v2:172.21.15.92:6836/950691230,v1:172.21.15.92:6837/950691230] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560830d95f40 con 0x56083104c6d0 2023-11-28T11:23:53.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.668+0000 7f30ae19b700 1 --2- 172.21.15.92:0/1942571150 >> [v2:172.21.15.92:6836/950691230,v1:172.21.15.92:6837/950691230] conn(0x56083104c6d0 0x56083104cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:53.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.668+0000 7f30ae19b700 1 --2- 172.21.15.92:0/1942571150 >> [v2:172.21.15.92:6836/950691230,v1:172.21.15.92:6837/950691230] conn(0x56083104c6d0 0x56083104cad0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a76f5fc6644a378 server_cookie=3cce4cdda4f893d8 in_seq=0 out_seq=0 2023-11-28T11:23:53.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.668+0000 7f30897fa700 10 client.5973 ms_handle_connect on v2:172.21.15.92:6836/950691230 2023-11-28T11:23:53.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.669+0000 7f30897fa700 1 -- 172.21.15.92:0/1942571150 <== mds.0 v2:172.21.15.92:6836/950691230 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x560830d95f40 con 0x56083104c6d0 2023-11-28T11:23:53.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.669+0000 7f30897fa700 10 client.5973 handle_command_reply: tid=0 2023-11-28T11:23:53.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.669+0000 7f30c3cfeb80 1 client.5973 shutdown 2023-11-28T11:23:53.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.669+0000 7f30c3cfeb80 1 -- 172.21.15.92:0/1942571150 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560831030b10 msgr2=0x560831034cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:53.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.669+0000 7f30c3cfeb80 1 --2- 172.21.15.92:0/1942571150 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560831030b10 0x560831034cb0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f30980015b0 tx=0x7f3098000ed0 comp rx=0 tx=0).stop 2023-11-28T11:23:53.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.669+0000 7f30c3cfeb80 1 -- 172.21.15.92:0/1942571150 shutdown_connections 2023-11-28T11:23:53.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.669+0000 7f30c3cfeb80 1 --2- 172.21.15.92:0/1942571150 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560831039fa0 0x560831038df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:53.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.669+0000 7f30c3cfeb80 1 --2- 172.21.15.92:0/1942571150 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56083101c6b0 0x56083101a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:53.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.669+0000 7f30c3cfeb80 1 --2- 172.21.15.92:0/1942571150 >> [v2:172.21.15.92:6836/950691230,v1:172.21.15.92:6837/950691230] conn(0x56083104c6d0 0x56083104cad0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:53.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.669+0000 7f30c3cfeb80 1 --2- 172.21.15.92:0/1942571150 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560831030b10 0x560831034cb0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:53.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.669+0000 7f30897fa700 0 client.5973 ms_handle_reset on v2:172.21.15.92:6836/950691230 2023-11-28T11:23:53.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.669+0000 7f30c3cfeb80 1 -- 172.21.15.92:0/1942571150 >> 172.21.15.92:0/1942571150 conn(0x560830fe9d70 msgr2=0x560830ff2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:53.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.669+0000 7f30c3cfeb80 1 -- 172.21.15.92:0/1942571150 shutdown_connections 2023-11-28T11:23:53.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.669+0000 7f30c3cfeb80 1 -- 172.21.15.92:0/1942571150 wait complete. 2023-11-28T11:23:53.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.669+0000 7f30c3cfeb80 20 client.5973 trim_cache size 0 max 16384 2023-11-28T11:23:53.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.686+0000 7f30c3cfeb80 1 Processor -- start 2023-11-28T11:23:53.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.686+0000 7f30c3cfeb80 1 -- start start 2023-11-28T11:23:53.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.687+0000 7f30c3cfeb80 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56083101c6b0 0x560830ff08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:53.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.687+0000 7f30c3cfeb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560831030b10 0x5608310342a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:53.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.687+0000 7f30c3cfeb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560831039fa0 0x56083101ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:53.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.687+0000 7f30c3cfeb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x560830ff2ae0 con 0x560831030b10 2023-11-28T11:23:53.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.687+0000 7f30c3cfeb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x560831022f50 con 0x560831039fa0 2023-11-28T11:23:53.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.687+0000 7f30c3cfeb80 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x560831034cb0 con 0x56083101c6b0 2023-11-28T11:23:53.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.687+0000 7f30ad99a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560831030b10 0x5608310342a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:53.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.687+0000 7f30ad99a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560831030b10 0x5608310342a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52098/0 (socket says 172.21.15.92:52098) 2023-11-28T11:23:53.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.687+0000 7f30ad99a700 1 -- 172.21.15.92:0/2370933006 learned_addr learned my addr 172.21.15.92:0/2370933006 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:53.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.687+0000 7f30ae19b700 1 --2- 172.21.15.92:0/2370933006 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56083101c6b0 0x560830ff08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:53.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.687+0000 7f30ae99c700 1 --2- 172.21.15.92:0/2370933006 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560831039fa0 0x56083101ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:53.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.687+0000 7f30ad99a700 1 -- 172.21.15.92:0/2370933006 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56083101c6b0 msgr2=0x560830ff08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:53.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.687+0000 7f30ad99a700 1 --2- 172.21.15.92:0/2370933006 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56083101c6b0 0x560830ff08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:53.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.687+0000 7f30ad99a700 1 -- 172.21.15.92:0/2370933006 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560831039fa0 msgr2=0x56083101ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:53.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.688+0000 7f30ad99a700 1 --2- 172.21.15.92:0/2370933006 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560831039fa0 0x56083101ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:53.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.688+0000 7f30ad99a700 1 -- 172.21.15.92:0/2370933006 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560830f0d330 con 0x560831030b10 2023-11-28T11:23:53.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.688+0000 7f30ad99a700 1 --2- 172.21.15.92:0/2370933006 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560831030b10 0x5608310342a0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f3098008fd0 tx=0x7f309801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55c5654f4de8560f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:53.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.688+0000 7f309cbff700 1 -- 172.21.15.92:0/2370933006 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3098013be0 con 0x560831030b10 2023-11-28T11:23:53.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.688+0000 7f309cbff700 1 -- 172.21.15.92:0/2370933006 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f309800d040 con 0x560831030b10 2023-11-28T11:23:53.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.688+0000 7f309cbff700 1 -- 172.21.15.92:0/2370933006 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3098001240 con 0x560831030b10 2023-11-28T11:23:53.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.688+0000 7f30c3cfeb80 1 -- 172.21.15.92:0/2370933006 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560831030b10 msgr2=0x5608310342a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:53.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.688+0000 7f30c3cfeb80 1 --2- 172.21.15.92:0/2370933006 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560831030b10 0x5608310342a0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f3098008fd0 tx=0x7f309801d000 comp rx=0 tx=0).stop 2023-11-28T11:23:53.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.689+0000 7f30c3cfeb80 1 -- 172.21.15.92:0/2370933006 shutdown_connections 2023-11-28T11:23:53.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.689+0000 7f30c3cfeb80 1 --2- 172.21.15.92:0/2370933006 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x560831039fa0 0x56083101ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:53.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.689+0000 7f30c3cfeb80 1 --2- 172.21.15.92:0/2370933006 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56083101c6b0 0x560830ff08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:53.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.689+0000 7f30c3cfeb80 1 --2- 172.21.15.92:0/2370933006 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560831030b10 0x5608310342a0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:53.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.689+0000 7f30c3cfeb80 1 -- 172.21.15.92:0/2370933006 >> 172.21.15.92:0/2370933006 conn(0x560830fe9d70 msgr2=0x560830ff12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:53.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.689+0000 7f30c3cfeb80 1 -- 172.21.15.92:0/2370933006 shutdown_connections 2023-11-28T11:23:53.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.689+0000 7f30c3cfeb80 1 -- 172.21.15.92:0/2370933006 wait complete. 2023-11-28T11:23:53.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.689+0000 7f30c3cfeb80 1 -- start start 2023-11-28T11:23:53.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.689+0000 7f30c3cfeb80 1 Processor -- start 2023-11-28T11:23:53.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.690+0000 7f30c3cfeb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56083101c6b0 0x560831056120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:53.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.690+0000 7f30c3cfeb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560831030b10 0x56083109f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:53.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.690+0000 7f30c3cfeb80 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560831039fa0 0x56083109f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:53.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.690+0000 7f30c3cfeb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56083101b4e0 con 0x560831030b10 2023-11-28T11:23:53.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.690+0000 7f30c3cfeb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x56083101b120 con 0x56083101c6b0 2023-11-28T11:23:53.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.690+0000 7f30c3cfeb80 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5608310a0040 con 0x560831039fa0 2023-11-28T11:23:53.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.690+0000 7f30c3cfeb80 10 client.0 _ll_register_callbacks cb 0x560830ee4450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:23:53.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.691+0000 7f30ad99a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560831030b10 0x56083109f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:53.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.691+0000 7f30ad99a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560831030b10 0x56083109f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52108/0 (socket says 172.21.15.92:52108) 2023-11-28T11:23:53.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.691+0000 7f30ad99a700 1 -- 172.21.15.92:0/693624637 learned_addr learned my addr 172.21.15.92:0/693624637 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:53.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.691+0000 7f30ae19b700 1 --2- 172.21.15.92:0/693624637 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56083101c6b0 0x560831056120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:53.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.691+0000 7f30ae99c700 1 --2- 172.21.15.92:0/693624637 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560831039fa0 0x56083109f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:53.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.691+0000 7f30ad99a700 1 -- 172.21.15.92:0/693624637 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560831039fa0 msgr2=0x56083109f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:53.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.691+0000 7f30ad99a700 1 --2- 172.21.15.92:0/693624637 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560831039fa0 0x56083109f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:53.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.691+0000 7f30ad99a700 1 -- 172.21.15.92:0/693624637 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56083101c6b0 msgr2=0x560831056120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:53.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.691+0000 7f30ad99a700 1 --2- 172.21.15.92:0/693624637 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56083101c6b0 0x560831056120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:53.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.691+0000 7f30ad99a700 1 -- 172.21.15.92:0/693624637 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560830e270f0 con 0x560831030b10 2023-11-28T11:23:53.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.691+0000 7f30ad99a700 1 --2- 172.21.15.92:0/693624637 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560831030b10 0x56083109f2b0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f3098000f10 tx=0x7f3098009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b72f54ab2c05052b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:53.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.691+0000 7f30897fa700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-11-28T11:23:53.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.692+0000 7f30897fa700 1 -- 172.21.15.92:0/693624637 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309803d070 con 0x560831030b10 2023-11-28T11:23:53.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.692+0000 7f30897fa700 1 -- 172.21.15.92:0/693624637 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(56..56 src has 1..56) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f3098004d70 con 0x560831030b10 2023-11-28T11:23:53.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.692+0000 7f30c3cfeb80 1 -- 172.21.15.92:0/693624637 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560831022850 con 0x560831030b10 2023-11-28T11:23:53.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.692+0000 7f30897fa700 1 -- 172.21.15.92:0/693624637 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3098013680 con 0x560831030b10 2023-11-28T11:23:53.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.692+0000 7f30c3cfeb80 10 client.5979 fetch_fsmap learned FSMap version 24 2023-11-28T11:23:53.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.692+0000 7f30c3cfeb80 1 -- 172.21.15.92:0/693624637 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x560830d9b220 con 0x560831030b10 2023-11-28T11:23:53.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.692+0000 7f30897fa700 1 -- 172.21.15.92:0/693624637 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560831022850 con 0x560831030b10 2023-11-28T11:23:53.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.692+0000 7f30897fa700 1 -- 172.21.15.92:0/693624637 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 24) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f3098040020 con 0x560831030b10 2023-11-28T11:23:53.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.692+0000 7f30c3cfeb80 10 client.5979 fetch_fsmap finished waiting for FSMap version 24 2023-11-28T11:23:53.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.692+0000 7f30c3cfeb80 10 client.5979 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-28T11:23:53.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.692+0000 7f30c3cfeb80 20 client.5979 populate_metadata read hostname 'smithi092' 2023-11-28T11:23:53.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.692+0000 7f30c3cfeb80 1 --2- 172.21.15.92:0/693624637 >> [v2:172.21.15.92:6836/950691230,v1:172.21.15.92:6837/950691230] conn(0x56083101bca0 0x560831031e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:53.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.692+0000 7f30c3cfeb80 4 client.5979 mds_command: new command op to 5754 tid=0[{"prefix": "scrub status"}] 2023-11-28T11:23:53.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.692+0000 7f30c3cfeb80 1 -- 172.21.15.92:0/693624637 --> [v2:172.21.15.92:6836/950691230,v1:172.21.15.92:6837/950691230] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f30980054e0 con 0x56083101bca0 2023-11-28T11:23:53.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.692+0000 7f30ae19b700 1 --2- 172.21.15.92:0/693624637 >> [v2:172.21.15.92:6836/950691230,v1:172.21.15.92:6837/950691230] conn(0x56083101bca0 0x560831031e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:53.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.693+0000 7f30897fa700 10 client.5979 ms_handle_connect on v2:172.21.15.92:6836/950691230 2023-11-28T11:23:53.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.693+0000 7f30ae19b700 1 --2- 172.21.15.92:0/693624637 >> [v2:172.21.15.92:6836/950691230,v1:172.21.15.92:6837/950691230] conn(0x56083101bca0 0x560831031e40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4afea771fd8681c server_cookie=accaa935a72e0865 in_seq=0 out_seq=0 2023-11-28T11:23:53.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.693+0000 7f30897fa700 1 -- 172.21.15.92:0/693624637 <== mds.0 v2:172.21.15.92:6836/950691230 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x7f30980054e0 con 0x56083101bca0 2023-11-28T11:23:53.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.693+0000 7f30897fa700 10 client.5979 handle_command_reply: tid=0 2023-11-28T11:23:53.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.693+0000 7f30c3cfeb80 1 client.5979 shutdown 2023-11-28T11:23:53.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.693+0000 7f30c3cfeb80 1 -- 172.21.15.92:0/693624637 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560831030b10 msgr2=0x56083109f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:53.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.693+0000 7f30c3cfeb80 1 --2- 172.21.15.92:0/693624637 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560831030b10 0x56083109f2b0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f3098000f10 tx=0x7f3098009040 comp rx=0 tx=0).stop 2023-11-28T11:23:53.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.693+0000 7f30c3cfeb80 1 -- 172.21.15.92:0/693624637 shutdown_connections 2023-11-28T11:23:53.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.693+0000 7f30c3cfeb80 1 --2- 172.21.15.92:0/693624637 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x560831039fa0 0x56083109f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:53.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.693+0000 7f30c3cfeb80 1 --2- 172.21.15.92:0/693624637 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56083101c6b0 0x560831056120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:53.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.693+0000 7f30c3cfeb80 1 --2- 172.21.15.92:0/693624637 >> [v2:172.21.15.92:6836/950691230,v1:172.21.15.92:6837/950691230] conn(0x56083101bca0 0x560831031e40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:53.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.693+0000 7f30c3cfeb80 1 --2- 172.21.15.92:0/693624637 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560831030b10 0x56083109f2b0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:53.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.693+0000 7f30897fa700 0 client.5979 ms_handle_reset on v2:172.21.15.92:6836/950691230 2023-11-28T11:23:53.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.693+0000 7f30c3cfeb80 1 -- 172.21.15.92:0/693624637 >> 172.21.15.92:0/693624637 conn(0x560830fe9d70 msgr2=0x56083101ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:53.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.694+0000 7f30c3cfeb80 1 -- 172.21.15.92:0/693624637 shutdown_connections 2023-11-28T11:23:53.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.694+0000 7f30c3cfeb80 1 -- 172.21.15.92:0/693624637 wait complete. 2023-11-28T11:23:53.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.694+0000 7f30c3cfeb80 20 client.5979 trim_cache size 0 max 16384 2023-11-28T11:23:53.700 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-28T11:23:53.700 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "no active scrubs running", 2023-11-28T11:23:53.700 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": {} 2023-11-28T11:23:53.701 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-28T11:23:53.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.694+0000 7f30b03ff700 1 -- 172.21.15.92:0/3599219950 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f308c0682f0 msgr2=0x7f308c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:53.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.694+0000 7f30b03ff700 1 --2- 172.21.15.92:0/3599219950 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f308c0682f0 0x7f308c06a7a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f30a4000c00 tx=0x7f30a401c040 comp rx=0 tx=0).stop 2023-11-28T11:23:53.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.694+0000 7f30b03ff700 1 -- 172.21.15.92:0/3599219950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30a80d0d60 msgr2=0x7f30a81e6840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:53.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.694+0000 7f30b03ff700 1 --2- 172.21.15.92:0/3599219950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30a80d0d60 0x7f30a81e6840 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f30a000cba0 tx=0x7f30a000cf60 comp rx=0 tx=0).stop 2023-11-28T11:23:53.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.694+0000 7f30b03ff700 1 -- 172.21.15.92:0/3599219950 shutdown_connections 2023-11-28T11:23:53.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.694+0000 7f30b03ff700 1 --2- 172.21.15.92:0/3599219950 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f308c0682f0 0x7f308c06a7a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:53.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.694+0000 7f30b03ff700 1 --2- 172.21.15.92:0/3599219950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30a80d0d60 0x7f30a81e6840 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:53.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.694+0000 7f30b03ff700 1 --2- 172.21.15.92:0/3599219950 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f30a80b8510 0x7f30a815a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:53.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.694+0000 7f30b03ff700 1 --2- 172.21.15.92:0/3599219950 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f30a80cd460 0x7f30a81e5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:53.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.694+0000 7f30b03ff700 1 -- 172.21.15.92:0/3599219950 >> 172.21.15.92:0/3599219950 conn(0x7f30a80719a0 msgr2=0x7f30a80c9680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:53.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.694+0000 7f30b03ff700 1 -- 172.21.15.92:0/3599219950 shutdown_connections 2023-11-28T11:23:53.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:53.695+0000 7f30b03ff700 1 -- 172.21.15.92:0/3599219950 wait complete. 2023-11-28T11:23:53.712 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-28T11:23:53.814 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:23:54.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.010+0000 7fd12b1ff700 1 Processor -- start 2023-11-28T11:23:54.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.010+0000 7fd12b1ff700 1 -- start start 2023-11-28T11:23:54.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.011+0000 7fd12b1ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1240d1d70 0x7fd124167270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:54.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.011+0000 7fd12b1ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd12406c790 con 0x7fd1240b8510 2023-11-28T11:23:54.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.011+0000 7fd12b1ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1240b88e0 con 0x7fd1240cd460 2023-11-28T11:23:54.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.011+0000 7fd12b1ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1240cdbd0 con 0x7fd1240d1d70 2023-11-28T11:23:54.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.011+0000 7fd128f9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd1240b8510 0x7fd12406b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36410/0 (socket says 172.21.15.92:36410) 2023-11-28T11:23:54.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.011+0000 7fd128f9b700 1 -- 172.21.15.92:0/199535935 learned_addr learned my addr 172.21.15.92:0/199535935 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:23:54.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.012+0000 7fd12979c700 1 --2- 172.21.15.92:0/199535935 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1240d1d70 0x7fd124167270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:54.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.012+0000 7fd1237fe700 1 -- 172.21.15.92:0/199535935 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2377853274 0 0) 0x7fd12406c790 con 0x7fd1240b8510 2023-11-28T11:23:54.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.012+0000 7fd1237fe700 1 -- 172.21.15.92:0/199535935 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1140035b0 con 0x7fd1240b8510 2023-11-28T11:23:54.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.012+0000 7fd1237fe700 1 -- 172.21.15.92:0/199535935 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 171995444 0 0) 0x7fd1240b88e0 con 0x7fd1240cd460 2023-11-28T11:23:54.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.012+0000 7fd1237fe700 1 -- 172.21.15.92:0/199535935 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd12406c790 con 0x7fd1240cd460 2023-11-28T11:23:54.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.012+0000 7fd1237fe700 1 -- 172.21.15.92:0/199535935 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3259713934 0 0) 0x7fd1140035b0 con 0x7fd1240b8510 2023-11-28T11:23:54.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.012+0000 7fd1237fe700 1 -- 172.21.15.92:0/199535935 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1240d1d70 msgr2=0x7fd124167270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:54.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.012+0000 7fd1237fe700 1 --2- 172.21.15.92:0/199535935 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1240d1d70 0x7fd124167270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:54.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.012+0000 7fd1237fe700 1 -- 172.21.15.92:0/199535935 >> v1:172.21.15.169:6789/0 conn(0x7fd1240cd460 legacy=0x7fd12406c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:54.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.012+0000 7fd1237fe700 1 -- 172.21.15.92:0/199535935 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1240cd8f0 con 0x7fd1240b8510 2023-11-28T11:23:54.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.012+0000 7fd1237fe700 1 -- 172.21.15.92:0/199535935 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532161587 0 0) 0x7fd118001670 con 0x7fd1240b8510 2023-11-28T11:23:54.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.012+0000 7fd1237fe700 1 -- 172.21.15.92:0/199535935 >> v1:172.21.15.92:6789/0 conn(0x7fd1240b8510 legacy=0x7fd12406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:54.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.012+0000 7fd1237fe700 1 --2- 172.21.15.92:0/199535935 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1240d1d70 0x7fd114003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:54.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.012+0000 7fd1237fe700 1 --2- 172.21.15.92:0/199535935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd114004170 0x7fd114004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:54.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.012+0000 7fd1237fe700 1 --2- 172.21.15.92:0/199535935 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd114004d90 0x7fd114005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:54.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.012+0000 7fd1237fe700 1 -- 172.21.15.92:0/199535935 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1240cdbd0 con 0x7fd114004170 2023-11-28T11:23:54.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.012+0000 7fd1237fe700 1 -- 172.21.15.92:0/199535935 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd11400bb20 con 0x7fd114004d90 2023-11-28T11:23:54.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.012+0000 7fd1237fe700 1 -- 172.21.15.92:0/199535935 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd11400bce0 con 0x7fd1240d1d70 2023-11-28T11:23:54.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.012+0000 7fd123fff700 1 --2- 172.21.15.92:0/199535935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd114004170 0x7fd114004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:54.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.012+0000 7fd128f9b700 1 --2- 172.21.15.92:0/199535935 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1240d1d70 0x7fd114003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:54.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.013+0000 7fd12979c700 1 --2- 172.21.15.92:0/199535935 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd114004d90 0x7fd114005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:54.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.013+0000 7fd123fff700 1 -- 172.21.15.92:0/199535935 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1240d1d70 msgr2=0x7fd114003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:54.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.013+0000 7fd123fff700 1 --2- 172.21.15.92:0/199535935 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1240d1d70 0x7fd114003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:54.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.013+0000 7fd123fff700 1 -- 172.21.15.92:0/199535935 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd114004d90 msgr2=0x7fd114005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:54.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.013+0000 7fd123fff700 1 --2- 172.21.15.92:0/199535935 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd114004d90 0x7fd114005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:54.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.013+0000 7fd123fff700 1 -- 172.21.15.92:0/199535935 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1140035b0 con 0x7fd114004170 2023-11-28T11:23:54.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.013+0000 7fd123fff700 1 --2- 172.21.15.92:0/199535935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd114004170 0x7fd114004850 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fd12408af10 tx=0x7fd114005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7df88952981ef474 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:54.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.013+0000 7fd1237fe700 1 -- 172.21.15.92:0/199535935 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd110001430 con 0x7fd114004170 2023-11-28T11:23:54.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.013+0000 7fd1237fe700 1 -- 172.21.15.92:0/199535935 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1100052d0 con 0x7fd114004170 2023-11-28T11:23:54.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.013+0000 7fd1237fe700 1 -- 172.21.15.92:0/199535935 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd110001d40 con 0x7fd114004170 2023-11-28T11:23:54.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.014+0000 7fd12b1ff700 1 -- 172.21.15.92:0/199535935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd114004170 msgr2=0x7fd114004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:54.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.014+0000 7fd12b1ff700 1 --2- 172.21.15.92:0/199535935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd114004170 0x7fd114004850 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fd12408af10 tx=0x7fd114005880 comp rx=0 tx=0).stop 2023-11-28T11:23:54.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.014+0000 7fd12979c700 1 -- 172.21.15.92:0/199535935 reap_dead start 2023-11-28T11:23:54.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.014+0000 7fd12b1ff700 1 -- 172.21.15.92:0/199535935 shutdown_connections 2023-11-28T11:23:54.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.014+0000 7fd12b1ff700 1 -- 172.21.15.92:0/199535935 >> 172.21.15.92:0/199535935 conn(0x7fd1240719a0 msgr2=0x7fd124071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:54.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.014+0000 7fd12b1ff700 1 -- 172.21.15.92:0/199535935 shutdown_connections 2023-11-28T11:23:54.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.014+0000 7fd12b1ff700 1 -- 172.21.15.92:0/199535935 wait complete. 2023-11-28T11:23:54.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.016+0000 7fd12b1ff700 1 Processor -- start 2023-11-28T11:23:54.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.016+0000 7fd12b1ff700 1 -- start start 2023-11-28T11:23:54.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.017+0000 7fd12b1ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd114004170 0x7fd1240c9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:54.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.017+0000 7fd12b1ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1240c7e80 0x7fd1241f4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:54.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.017+0000 7fd12b1ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1240c9ee0 0x7fd124157a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:54.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.017+0000 7fd12b1ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1240b7b10 con 0x7fd1240c9ee0 2023-11-28T11:23:54.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.017+0000 7fd12b1ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1240b75e0 con 0x7fd114004170 2023-11-28T11:23:54.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.017+0000 7fd12b1ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1240b7830 con 0x7fd1240c7e80 2023-11-28T11:23:54.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.017+0000 7fd12979c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1240c9ee0 0x7fd124157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:54.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.017+0000 7fd12979c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1240c9ee0 0x7fd124157a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52116/0 (socket says 172.21.15.92:52116) 2023-11-28T11:23:54.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.017+0000 7fd12979c700 1 -- 172.21.15.92:0/1502286981 learned_addr learned my addr 172.21.15.92:0/1502286981 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:54.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.017+0000 7fd128f9b700 1 --2- 172.21.15.92:0/1502286981 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd114004170 0x7fd1240c9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:54.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.017+0000 7fd123fff700 1 --2- 172.21.15.92:0/1502286981 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1240c7e80 0x7fd1241f4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:54.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.018+0000 7fd12979c700 1 -- 172.21.15.92:0/1502286981 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1240c7e80 msgr2=0x7fd1241f4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:54.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.018+0000 7fd12979c700 1 --2- 172.21.15.92:0/1502286981 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1240c7e80 0x7fd1241f4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:54.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.018+0000 7fd12979c700 1 -- 172.21.15.92:0/1502286981 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd114004170 msgr2=0x7fd1240c9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:54.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.018+0000 7fd12979c700 1 --2- 172.21.15.92:0/1502286981 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd114004170 0x7fd1240c9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:54.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.018+0000 7fd12979c700 1 -- 172.21.15.92:0/1502286981 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1240cfff0 con 0x7fd1240c9ee0 2023-11-28T11:23:54.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.018+0000 7fd128f9b700 1 --2- 172.21.15.92:0/1502286981 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd114004170 0x7fd1240c9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:23:54.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.018+0000 7fd12979c700 1 --2- 172.21.15.92:0/1502286981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1240c9ee0 0x7fd124157a50 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fd124072610 tx=0x7fd11c011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:54.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.018+0000 7fd1217fa700 1 -- 172.21.15.92:0/1502286981 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11c00e040 con 0x7fd1240c9ee0 2023-11-28T11:23:54.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.018+0000 7fd1217fa700 1 -- 172.21.15.92:0/1502286981 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd11c00f040 con 0x7fd1240c9ee0 2023-11-28T11:23:54.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.018+0000 7fd1217fa700 1 -- 172.21.15.92:0/1502286981 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11c00fd50 con 0x7fd1240c9ee0 2023-11-28T11:23:54.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.018+0000 7fd12b1ff700 1 -- 172.21.15.92:0/1502286981 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1240cfd10 con 0x7fd1240c9ee0 2023-11-28T11:23:54.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.018+0000 7fd12b1ff700 1 -- 172.21.15.92:0/1502286981 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1240d29c0 con 0x7fd1240c9ee0 2023-11-28T11:23:54.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.019+0000 7fd12b1ff700 1 -- 172.21.15.92:0/1502286981 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd124067880 con 0x7fd1240c9ee0 2023-11-28T11:23:54.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.020+0000 7fd1217fa700 1 -- 172.21.15.92:0/1502286981 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd11c015430 con 0x7fd1240c9ee0 2023-11-28T11:23:54.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.021+0000 7fd1217fa700 1 --2- 172.21.15.92:0/1502286981 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd1080682f0 0x7fd10806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:54.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.021+0000 7fd1217fa700 1 -- 172.21.15.92:0/1502286981 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fd11c086870 con 0x7fd1240c9ee0 2023-11-28T11:23:54.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.021+0000 7fd128f9b700 1 --2- 172.21.15.92:0/1502286981 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd1080682f0 0x7fd10806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:54.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.022+0000 7fd128f9b700 1 --2- 172.21.15.92:0/1502286981 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd1080682f0 0x7fd10806a7a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd118001eb0 tx=0x7fd118009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:54.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.026+0000 7fd1217fa700 1 -- 172.21.15.92:0/1502286981 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd11c052e40 con 0x7fd1240c9ee0 2023-11-28T11:23:54.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.261+0000 7fd12b1ff700 1 -- 172.21.15.92:0/1502286981 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd1240b2340 con 0x7fd1240c9ee0 2023-11-28T11:23:54.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.261+0000 7fd1217fa700 1 -- 172.21.15.92:0/1502286981 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4651 (secure 0 0 0) 0x7fd11c056460 con 0x7fd1240c9ee0 2023-11-28T11:23:54.262 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:23:54.262 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5593,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/3937189339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3937189339},{"type":"v1","addr":"172.21.15.169:6834","nonce":3937189339}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5596,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2020201016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2020201016},{"type":"v1","addr":"172.21.15.169:6835","nonce":2020201016}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5751,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2423750324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2423750324},{"type":"v1","addr":"172.21.15.92:6835","nonce":2423750324}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:23:13.247233+0000","modified":"2023-11-28T11:23:16.254256+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5754},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5754":{"gid":5754,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/950691230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":950691230},{"type":"v1","addr":"172.21.15.92:6837","nonce":950691230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T11:23:54.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.263+0000 7fd12b1ff700 1 -- 172.21.15.92:0/1502286981 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd1080682f0 msgr2=0x7fd10806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:54.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.263+0000 7fd12b1ff700 1 --2- 172.21.15.92:0/1502286981 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd1080682f0 0x7fd10806a7a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd118001eb0 tx=0x7fd118009040 comp rx=0 tx=0).stop 2023-11-28T11:23:54.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.263+0000 7fd12b1ff700 1 -- 172.21.15.92:0/1502286981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1240c9ee0 msgr2=0x7fd124157a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:54.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.263+0000 7fd12b1ff700 1 --2- 172.21.15.92:0/1502286981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1240c9ee0 0x7fd124157a50 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fd124072610 tx=0x7fd11c011f80 comp rx=0 tx=0).stop 2023-11-28T11:23:54.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.263+0000 7fd12b1ff700 1 -- 172.21.15.92:0/1502286981 shutdown_connections 2023-11-28T11:23:54.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.263+0000 7fd12b1ff700 1 --2- 172.21.15.92:0/1502286981 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd1080682f0 0x7fd10806a7a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:54.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.263+0000 7fd12b1ff700 1 --2- 172.21.15.92:0/1502286981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd1240c9ee0 0x7fd124157a50 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:54.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.263+0000 7fd12b1ff700 1 --2- 172.21.15.92:0/1502286981 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd114004170 0x7fd1240c9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:54.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.263+0000 7fd12b1ff700 1 --2- 172.21.15.92:0/1502286981 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd1240c7e80 0x7fd1241f4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:54.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.263+0000 7fd12b1ff700 1 -- 172.21.15.92:0/1502286981 >> 172.21.15.92:0/1502286981 conn(0x7fd1240719a0 msgr2=0x7fd12406b970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:54.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.264+0000 7fd12b1ff700 1 -- 172.21.15.92:0/1502286981 shutdown_connections 2023-11-28T11:23:54.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.264+0000 7fd12b1ff700 1 -- 172.21.15.92:0/1502286981 wait complete. 2023-11-28T11:23:54.266 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 24 2023-11-28T11:23:54.282 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2023-11-28T11:23:54.509 INFO:teuthology.orchestra.run.smithi092.stdout:{"message":"","return_code":0} 2023-11-28T11:23:54.509 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-28T11:23:54.510 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2023-11-28T11:23:54.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.596+0000 7fb75b8960c0 1 Processor -- start 2023-11-28T11:23:54.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.597+0000 7fb75b8960c0 1 -- start start 2023-11-28T11:23:54.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.597+0000 7fb75b8960c0 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x555a92c91db0 0x555a92ca7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:54.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.597+0000 7fb75b8960c0 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555a92cb04b0 0x555a92cadc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:54.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.597+0000 7fb75b8960c0 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a92d3d060 0x555a92d3d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:54.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.597+0000 7fb75b8960c0 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a92ba0fe0 con 0x555a92d3d060 2023-11-28T11:23:54.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.597+0000 7fb75b8960c0 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a92caf990 con 0x555a92cb04b0 2023-11-28T11:23:54.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.597+0000 7fb75b8960c0 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x555a92cae980 con 0x555a92c91db0 2023-11-28T11:23:54.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.598+0000 7fb74d79c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a92d3d060 0x555a92d3d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:54.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.598+0000 7fb74d79c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a92d3d060 0x555a92d3d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52126/0 (socket says 172.21.15.92:52126) 2023-11-28T11:23:54.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.598+0000 7fb74d79c700 1 -- 172.21.15.92:0/4235796933 learned_addr learned my addr 172.21.15.92:0/4235796933 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:54.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.598+0000 7fb74cf9b700 1 --2- 172.21.15.92:0/4235796933 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x555a92c91db0 0x555a92ca7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:54.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.598+0000 7fb747fff700 1 --2- 172.21.15.92:0/4235796933 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555a92cb04b0 0x555a92cadc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:54.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.598+0000 7fb74d79c700 1 -- 172.21.15.92:0/4235796933 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x555a92c91db0 msgr2=0x555a92ca7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:54.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.598+0000 7fb74d79c700 1 --2- 172.21.15.92:0/4235796933 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x555a92c91db0 0x555a92ca7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:54.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.598+0000 7fb74d79c700 1 -- 172.21.15.92:0/4235796933 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555a92cb04b0 msgr2=0x555a92cadc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:54.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.598+0000 7fb74d79c700 1 --2- 172.21.15.92:0/4235796933 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555a92cb04b0 0x555a92cadc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:54.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.598+0000 7fb74d79c700 1 -- 172.21.15.92:0/4235796933 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb738005d20 con 0x555a92d3d060 2023-11-28T11:23:54.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.598+0000 7fb74d79c700 1 --2- 172.21.15.92:0/4235796933 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a92d3d060 0x555a92d3d4f0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fb748001560 tx=0x7fb748006760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e24b60377c09f0cd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:54.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.599+0000 7fb746ffd700 1 -- 172.21.15.92:0/4235796933 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7480062f0 con 0x555a92d3d060 2023-11-28T11:23:54.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.599+0000 7fb746ffd700 1 -- 172.21.15.92:0/4235796933 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb748003450 con 0x555a92d3d060 2023-11-28T11:23:54.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.599+0000 7fb746ffd700 1 -- 172.21.15.92:0/4235796933 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb748003820 con 0x555a92d3d060 2023-11-28T11:23:54.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.599+0000 7fb75b8960c0 1 -- 172.21.15.92:0/4235796933 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a92d3d060 msgr2=0x555a92d3d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:54.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.599+0000 7fb75b8960c0 1 --2- 172.21.15.92:0/4235796933 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a92d3d060 0x555a92d3d4f0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fb748001560 tx=0x7fb748006760 comp rx=0 tx=0).stop 2023-11-28T11:23:54.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.599+0000 7fb75b8960c0 1 -- 172.21.15.92:0/4235796933 shutdown_connections 2023-11-28T11:23:54.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.599+0000 7fb75b8960c0 1 --2- 172.21.15.92:0/4235796933 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a92d3d060 0x555a92d3d4f0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:54.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.599+0000 7fb75b8960c0 1 --2- 172.21.15.92:0/4235796933 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x555a92c91db0 0x555a92ca7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:54.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.599+0000 7fb75b8960c0 1 --2- 172.21.15.92:0/4235796933 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555a92cb04b0 0x555a92cadc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:54.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.599+0000 7fb75b8960c0 1 -- 172.21.15.92:0/4235796933 >> 172.21.15.92:0/4235796933 conn(0x555a92c695b0 msgr2=0x555a92ca75f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:54.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.599+0000 7fb75b8960c0 1 -- 172.21.15.92:0/4235796933 shutdown_connections 2023-11-28T11:23:54.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.599+0000 7fb75b8960c0 1 -- 172.21.15.92:0/4235796933 wait complete. 2023-11-28T11:23:54.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.600+0000 7fb75b8960c0 1 Processor -- start 2023-11-28T11:23:54.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.600+0000 7fb75b8960c0 1 -- start start 2023-11-28T11:23:54.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.601+0000 7fb75b8960c0 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a92c91db0 0x555a92d1d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:54.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.601+0000 7fb75b8960c0 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x555a92cb04b0 0x555a92caa1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:54.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.601+0000 7fb75b8960c0 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555a92d3d060 0x555a92cab7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:54.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.601+0000 7fb75b8960c0 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a92c93270 con 0x555a92c91db0 2023-11-28T11:23:54.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.601+0000 7fb75b8960c0 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a92c93b10 con 0x555a92d3d060 2023-11-28T11:23:54.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.601+0000 7fb75b8960c0 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x555a92c964e0 con 0x555a92cb04b0 2023-11-28T11:23:54.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.601+0000 7fb74cf9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a92c91db0 0x555a92d1d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:54.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.601+0000 7fb74cf9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a92c91db0 0x555a92d1d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52138/0 (socket says 172.21.15.92:52138) 2023-11-28T11:23:54.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.601+0000 7fb74cf9b700 1 -- 172.21.15.92:0/2673476935 learned_addr learned my addr 172.21.15.92:0/2673476935 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:54.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.602+0000 7fb74cf9b700 1 -- 172.21.15.92:0/2673476935 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x555a92cb04b0 msgr2=0x555a92caa1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:54.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.602+0000 7fb74d79c700 1 --2- 172.21.15.92:0/2673476935 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555a92d3d060 0x555a92cab7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:54.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.602+0000 7fb747fff700 1 --2- 172.21.15.92:0/2673476935 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x555a92cb04b0 0x555a92caa1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:54.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.602+0000 7fb74cf9b700 1 --2- 172.21.15.92:0/2673476935 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x555a92cb04b0 0x555a92caa1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:54.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.602+0000 7fb74cf9b700 1 -- 172.21.15.92:0/2673476935 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555a92d3d060 msgr2=0x555a92cab7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:54.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.602+0000 7fb74cf9b700 1 --2- 172.21.15.92:0/2673476935 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555a92d3d060 0x555a92cab7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:54.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.602+0000 7fb74cf9b700 1 -- 172.21.15.92:0/2673476935 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7480017b0 con 0x555a92c91db0 2023-11-28T11:23:54.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.602+0000 7fb74d79c700 1 --2- 172.21.15.92:0/2673476935 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x555a92d3d060 0x555a92cab7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:23:54.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.602+0000 7fb747fff700 1 --2- 172.21.15.92:0/2673476935 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x555a92cb04b0 0x555a92caa1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:23:54.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.602+0000 7fb74cf9b700 1 --2- 172.21.15.92:0/2673476935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a92c91db0 0x555a92d1d920 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fb7400086e0 tx=0x7fb740008aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:54.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.602+0000 7fb745ffb700 1 -- 172.21.15.92:0/2673476935 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb74000f040 con 0x555a92c91db0 2023-11-28T11:23:54.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.602+0000 7fb745ffb700 1 -- 172.21.15.92:0/2673476935 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb740015860 con 0x555a92c91db0 2023-11-28T11:23:54.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.602+0000 7fb75b8960c0 1 -- 172.21.15.92:0/2673476935 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555a92bb9120 con 0x555a92c91db0 2023-11-28T11:23:54.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.602+0000 7fb745ffb700 1 -- 172.21.15.92:0/2673476935 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb740015bf0 con 0x555a92c91db0 2023-11-28T11:23:54.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.602+0000 7fb75b8960c0 1 -- 172.21.15.92:0/2673476935 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555a92bac830 con 0x555a92c91db0 2023-11-28T11:23:54.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.604+0000 7fb745ffb700 1 -- 172.21.15.92:0/2673476935 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb740025070 con 0x555a92c91db0 2023-11-28T11:23:54.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.605+0000 7fb745ffb700 1 --2- 172.21.15.92:0/2673476935 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb730066c40 0x7fb7300690f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:54.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.605+0000 7fb745ffb700 1 -- 172.21.15.92:0/2673476935 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fb7400852f0 con 0x555a92c91db0 2023-11-28T11:23:54.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.605+0000 7fb747fff700 1 --2- 172.21.15.92:0/2673476935 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb730066c40 0x7fb7300690f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:54.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.605+0000 7fb75b8960c0 1 --2- 172.21.15.92:0/2673476935 >> [v2:172.21.15.169:6800/116033,v1:172.21.15.169:6801/116033] conn(0x555a92caa730 0x555a92c944b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:54.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.605+0000 7fb75b8960c0 1 -- 172.21.15.92:0/2673476935 --> [v2:172.21.15.169:6800/116033,v1:172.21.15.169:6801/116033] -- osd_op(unknown.0.0:1 9.7 9:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56) v8 -- 0x555a92c949f0 con 0x555a92caa730 2023-11-28T11:23:54.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.606+0000 7fb74d79c700 1 --2- 172.21.15.92:0/2673476935 >> [v2:172.21.15.169:6800/116033,v1:172.21.15.169:6801/116033] conn(0x555a92caa730 0x555a92c944b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:54.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.606+0000 7fb747fff700 1 --2- 172.21.15.92:0/2673476935 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb730066c40 0x7fb7300690f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x555a92cb08c0 tx=0x7fb73800e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:54.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.606+0000 7fb74d79c700 1 --2- 172.21.15.92:0/2673476935 >> [v2:172.21.15.169:6800/116033,v1:172.21.15.169:6801/116033] conn(0x555a92caa730 0x555a92c944b0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:54.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.607+0000 7fb74d79c700 1 -- 172.21.15.92:0/2673476935 <== osd.5 v2:172.21.15.169:6800/116033 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x7fb7480035c0 con 0x555a92caa730 2023-11-28T11:23:54.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.607+0000 7fb75b8960c0 1 -- 172.21.15.92:0/2673476935 >> [v2:172.21.15.169:6800/116033,v1:172.21.15.169:6801/116033] conn(0x555a92caa730 msgr2=0x555a92c944b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:54.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.607+0000 7fb75b8960c0 1 --2- 172.21.15.92:0/2673476935 >> [v2:172.21.15.169:6800/116033,v1:172.21.15.169:6801/116033] conn(0x555a92caa730 0x555a92c944b0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:54.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.607+0000 7fb75b8960c0 1 -- 172.21.15.92:0/2673476935 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb730066c40 msgr2=0x7fb7300690f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:54.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.607+0000 7fb75b8960c0 1 --2- 172.21.15.92:0/2673476935 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb730066c40 0x7fb7300690f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x555a92cb08c0 tx=0x7fb73800e000 comp rx=0 tx=0).stop 2023-11-28T11:23:54.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.607+0000 7fb75b8960c0 1 -- 172.21.15.92:0/2673476935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a92c91db0 msgr2=0x555a92d1d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:54.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.607+0000 7fb75b8960c0 1 --2- 172.21.15.92:0/2673476935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a92c91db0 0x555a92d1d920 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fb7400086e0 tx=0x7fb740008aa0 comp rx=0 tx=0).stop 2023-11-28T11:23:54.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.607+0000 7fb74d79c700 1 -- 172.21.15.92:0/2673476935 reap_dead start 2023-11-28T11:23:54.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.607+0000 7fb75b8960c0 1 -- 172.21.15.92:0/2673476935 shutdown_connections 2023-11-28T11:23:54.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.607+0000 7fb75b8960c0 1 -- 172.21.15.92:0/2673476935 >> 172.21.15.92:0/2673476935 conn(0x555a92c695b0 msgr2=0x555a92caef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:54.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.607+0000 7fb75b8960c0 1 -- 172.21.15.92:0/2673476935 shutdown_connections 2023-11-28T11:23:54.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:54.607+0000 7fb75b8960c0 1 -- 172.21.15.92:0/2673476935 wait complete. 2023-11-28T11:23:54.612 INFO:teuthology.orchestra.run.smithi092.stdout:Yalphaparent_a 2023-11-28T11:23:54.612 DEBUG:teuthology.orchestra.run.smithi092:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2023-11-28T11:23:54.710 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-28T11:23:54.711 INFO:teuthology.orchestra.run.smithi092.stdout: "ino": 1099511627777, 2023-11-28T11:23:54.711 INFO:teuthology.orchestra.run.smithi092.stdout: "ancestors": [ 2023-11-28T11:23:54.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:23:54.711 INFO:teuthology.orchestra.run.smithi092.stdout: "dirino": 1099511627776, 2023-11-28T11:23:54.711 INFO:teuthology.orchestra.run.smithi092.stdout: "dname": "alpha", 2023-11-28T11:23:54.711 INFO:teuthology.orchestra.run.smithi092.stdout: "version": 4 2023-11-28T11:23:54.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:23:54.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:23:54.711 INFO:teuthology.orchestra.run.smithi092.stdout: "dirino": 1, 2023-11-28T11:23:54.712 INFO:teuthology.orchestra.run.smithi092.stdout: "dname": "parent_a", 2023-11-28T11:23:54.712 INFO:teuthology.orchestra.run.smithi092.stdout: "version": 7 2023-11-28T11:23:54.712 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:23:54.712 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2023-11-28T11:23:54.712 INFO:teuthology.orchestra.run.smithi092.stdout: "pool": 9, 2023-11-28T11:23:54.712 INFO:teuthology.orchestra.run.smithi092.stdout: "old_pools": [] 2023-11-28T11:23:54.712 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-28T11:23:54.712 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:23:54.717 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T11:23:54.717 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:23:54.808 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:23:54.911 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T11:23:54.911 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-28T11:23:54.911 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:23:54.958 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-28T11:23:54.958 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:23:55.045 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:23:55.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.280+0000 7f96bc3ff700 1 Processor -- start 2023-11-28T11:23:55.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.280+0000 7f96bc3ff700 1 -- start start 2023-11-28T11:23:55.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.282+0000 7f96bc3ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f96b40b8510 0x7f96b406b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:55.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.282+0000 7f96bc3ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96b40d0d60 0x7f96b40d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:55.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.282+0000 7f96bc3ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96b40b88e0 con 0x7f96b40cd460 2023-11-28T11:23:55.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.282+0000 7f96bc3ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96b40cdbd0 con 0x7f96b40b8510 2023-11-28T11:23:55.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.282+0000 7f96bc3ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96b40d02d0 con 0x7f96b40d0d60 2023-11-28T11:23:55.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.282+0000 7f96ba19b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f96b40b8510 0x7f96b406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:55.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.282+0000 7f96b999a700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f96b40cd460 0x7f96b406beb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36424/0 (socket says 172.21.15.92:36424) 2023-11-28T11:23:55.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.282+0000 7f96ba19b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f96b40b8510 0x7f96b406b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52150/0 (socket says 172.21.15.92:52150) 2023-11-28T11:23:55.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.282+0000 7f96b999a700 1 -- 172.21.15.92:0/3310151742 learned_addr learned my addr 172.21.15.92:0/3310151742 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:23:55.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.282+0000 7f96ba99c700 1 --2- 172.21.15.92:0/3310151742 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96b40d0d60 0x7f96b40d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:55.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.282+0000 7f96ba19b700 1 -- 172.21.15.92:0/3310151742 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96b40d0d60 msgr2=0x7f96b40d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:55.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.282+0000 7f96ba19b700 1 --2- 172.21.15.92:0/3310151742 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96b40d0d60 0x7f96b40d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:55.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.282+0000 7f96ba19b700 1 -- 172.21.15.92:0/3310151742 >> v1:172.21.15.92:6789/0 conn(0x7f96b40cd460 legacy=0x7f96b406beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:55.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.282+0000 7f96ba19b700 1 -- 172.21.15.92:0/3310151742 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b40cd8f0 con 0x7f96b40b8510 2023-11-28T11:23:55.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.282+0000 7f96b9199700 1 -- 172.21.15.92:0/3310151742 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3725470188 0 0) 0x7f96b40b88e0 con 0x7f96b40cd460 2023-11-28T11:23:55.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.283+0000 7f96ba19b700 1 --2- 172.21.15.92:0/3310151742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96b40b8510 0x7f96b406b970 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f96b0009870 tx=0x7f96b0009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2705101adee86de server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:55.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.283+0000 7f96b9199700 1 -- 172.21.15.92:0/3310151742 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b0011830 con 0x7f96b40b8510 2023-11-28T11:23:55.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.283+0000 7f96b9199700 1 -- 172.21.15.92:0/3310151742 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96b0011990 con 0x7f96b40b8510 2023-11-28T11:23:55.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.283+0000 7f96b9199700 1 -- 172.21.15.92:0/3310151742 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b0011d20 con 0x7f96b40b8510 2023-11-28T11:23:55.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.283+0000 7f96bc3ff700 1 -- 172.21.15.92:0/3310151742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96b40b8510 msgr2=0x7f96b406b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:55.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.283+0000 7f96bc3ff700 1 --2- 172.21.15.92:0/3310151742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96b40b8510 0x7f96b406b970 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f96b0009870 tx=0x7f96b0009b80 comp rx=0 tx=0).stop 2023-11-28T11:23:55.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.283+0000 7f96bc3ff700 1 -- 172.21.15.92:0/3310151742 shutdown_connections 2023-11-28T11:23:55.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.283+0000 7f96bc3ff700 1 --2- 172.21.15.92:0/3310151742 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96b40d0d60 0x7f96b40d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:55.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.283+0000 7f96bc3ff700 1 --2- 172.21.15.92:0/3310151742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96b40b8510 0x7f96b406b970 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:55.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.283+0000 7f96bc3ff700 1 -- 172.21.15.92:0/3310151742 >> 172.21.15.92:0/3310151742 conn(0x7f96b40719a0 msgr2=0x7f96b4071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:55.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.284+0000 7f96bc3ff700 1 -- 172.21.15.92:0/3310151742 shutdown_connections 2023-11-28T11:23:55.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.284+0000 7f96bc3ff700 1 -- 172.21.15.92:0/3310151742 wait complete. 2023-11-28T11:23:55.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.285+0000 7f96bc3ff700 1 Processor -- start 2023-11-28T11:23:55.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.285+0000 7f96bc3ff700 1 -- start start 2023-11-28T11:23:55.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.286+0000 7f96bc3ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96b40b8510 0x7f96b40cbf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:55.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.286+0000 7f96bc3ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96b40cd460 0x7f96b40cc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:55.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.286+0000 7f96bc3ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96b40d0d60 0x7f96b40ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:55.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.286+0000 7f96bc3ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b40ca5a0 con 0x7f96b40cd460 2023-11-28T11:23:55.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.286+0000 7f96bc3ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b40ccc30 con 0x7f96b40d0d60 2023-11-28T11:23:55.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.286+0000 7f96bc3ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96b40cce70 con 0x7f96b40b8510 2023-11-28T11:23:55.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.287+0000 7f96b999a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96b40cd460 0x7f96b40cc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:55.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.287+0000 7f96ba19b700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96b40b8510 0x7f96b40cbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:55.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.287+0000 7f96b999a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96b40cd460 0x7f96b40cc6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52164/0 (socket says 172.21.15.92:52164) 2023-11-28T11:23:55.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.287+0000 7f96b999a700 1 -- 172.21.15.92:0/2039864641 learned_addr learned my addr 172.21.15.92:0/2039864641 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:55.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.287+0000 7f96ba99c700 1 --2- 172.21.15.92:0/2039864641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96b40d0d60 0x7f96b40ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:55.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.287+0000 7f96b999a700 1 -- 172.21.15.92:0/2039864641 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96b40b8510 msgr2=0x7f96b40cbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:55.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.287+0000 7f96b999a700 1 --2- 172.21.15.92:0/2039864641 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96b40b8510 0x7f96b40cbf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:55.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.287+0000 7f96b999a700 1 -- 172.21.15.92:0/2039864641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96b40d0d60 msgr2=0x7f96b40ca9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:55.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.287+0000 7f96b999a700 1 --2- 172.21.15.92:0/2039864641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96b40d0d60 0x7f96b40ca9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:55.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.287+0000 7f96b999a700 1 -- 172.21.15.92:0/2039864641 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b00094a0 con 0x7f96b40cd460 2023-11-28T11:23:55.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.287+0000 7f96ba99c700 1 --2- 172.21.15.92:0/2039864641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96b40d0d60 0x7f96b40ca9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:23:55.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.287+0000 7f96b999a700 1 --2- 172.21.15.92:0/2039864641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96b40cd460 0x7f96b40cc6f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f96a400c520 tx=0x7f96a400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:55.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.287+0000 7f96aaffd700 1 -- 172.21.15.92:0/2039864641 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a4004c80 con 0x7f96b40cd460 2023-11-28T11:23:55.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.287+0000 7f96aaffd700 1 -- 172.21.15.92:0/2039864641 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96a4004de0 con 0x7f96b40cd460 2023-11-28T11:23:55.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.287+0000 7f96bc3ff700 1 -- 172.21.15.92:0/2039864641 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96b40cfff0 con 0x7f96b40cd460 2023-11-28T11:23:55.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.288+0000 7f96aaffd700 1 -- 172.21.15.92:0/2039864641 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a4007310 con 0x7f96b40cd460 2023-11-28T11:23:55.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.288+0000 7f96bc3ff700 1 -- 172.21.15.92:0/2039864641 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96b40cfd10 con 0x7f96b40cd460 2023-11-28T11:23:55.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.289+0000 7f96bc3ff700 1 -- 172.21.15.92:0/2039864641 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96b4067880 con 0x7f96b40cd460 2023-11-28T11:23:55.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.289+0000 7f96aaffd700 1 -- 172.21.15.92:0/2039864641 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f96a401a020 con 0x7f96b40cd460 2023-11-28T11:23:55.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.291+0000 7f96aaffd700 1 --2- 172.21.15.92:0/2039864641 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f96980682f0 0x7f969806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:55.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.291+0000 7f96aaffd700 1 -- 172.21.15.92:0/2039864641 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f96a4085a10 con 0x7f96b40cd460 2023-11-28T11:23:55.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.291+0000 7f96ba19b700 1 --2- 172.21.15.92:0/2039864641 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f96980682f0 0x7f969806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:55.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.291+0000 7f96ba19b700 1 --2- 172.21.15.92:0/2039864641 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f96980682f0 0x7f969806a7a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f96b0000c00 tx=0x7f96b0008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:55.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.295+0000 7f96aaffd700 1 -- 172.21.15.92:0/2039864641 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f96a40520f0 con 0x7f96b40cd460 2023-11-28T11:23:55.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.484+0000 7f96bc3ff700 1 -- 172.21.15.92:0/2039864641 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f96b40d2940 con 0x7f96b40cd460 2023-11-28T11:23:55.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.486+0000 7f96aaffd700 1 -- 172.21.15.92:0/2039864641 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+18180 (secure 0 0 0) 0x7f96a4055710 con 0x7f96b40cd460 2023-11-28T11:23:55.487 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:23:55.487 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":56,"fsid":"68956e0c-c16f-47cd-a220-f98364b0831d","created":"2023-11-28T11:20:28.697573+0000","modified":"2023-11-28T11:23:14.249429+0000","last_up_change":"2023-11-28T11:20:41.257011+0000","last_in_change":"2023-11-28T11:20:32.249767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:20:41.925237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:20:48.682497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:23:10.939884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-28T11:23:12.227433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6abe483f-0e03-4a26-b9cf-62041291747e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":115823},{"type":"v1","addr":"172.21.15.92:6809","nonce":115823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":115823},{"type":"v1","addr":"172.21.15.92:6813","nonce":115823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":115823},{"type":"v1","addr":"172.21.15.92:6821","nonce":115823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":115823},{"type":"v1","addr":"172.21.15.92:6817","nonce":115823}]},"public_addr":"172.21.15.92:6809/115823","cluster_addr":"172.21.15.92:6813/115823","heartbeat_back_addr":"172.21.15.92:6821/115823","heartbeat_front_addr":"172.21.15.92:6817/115823","state":["exists","up"]},{"osd":1,"uuid":"13c1de73-1a8c-4d7e-8a28-9dae52dfe089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":115824},{"type":"v1","addr":"172.21.15.92:6812","nonce":115824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":115824},{"type":"v1","addr":"172.21.15.92:6816","nonce":115824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":115824},{"type":"v1","addr":"172.21.15.92:6823","nonce":115824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":115824},{"type":"v1","addr":"172.21.15.92:6820","nonce":115824}]},"public_addr":"172.21.15.92:6812/115824","cluster_addr":"172.21.15.92:6816/115824","heartbeat_back_addr":"172.21.15.92:6823/115824","heartbeat_front_addr":"172.21.15.92:6820/115824","state":["exists","up"]},{"osd":2,"uuid":"25788a55-112f-4f00-810d-2bc37cf76778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":115821},{"type":"v1","addr":"172.21.15.92:6801","nonce":115821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":115821},{"type":"v1","addr":"172.21.15.92:6803","nonce":115821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":115821},{"type":"v1","addr":"172.21.15.92:6807","nonce":115821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":115821},{"type":"v1","addr":"172.21.15.92:6805","nonce":115821}]},"public_addr":"172.21.15.92:6801/115821","cluster_addr":"172.21.15.92:6803/115821","heartbeat_back_addr":"172.21.15.92:6807/115821","heartbeat_front_addr":"172.21.15.92:6805/115821","state":["exists","up"]},{"osd":3,"uuid":"adf54e24-33a9-41fe-9ce3-98a5ab6c158e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":115825},{"type":"v1","addr":"172.21.15.92:6825","nonce":115825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":115825},{"type":"v1","addr":"172.21.15.92:6827","nonce":115825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":115825},{"type":"v1","addr":"172.21.15.92:6831","nonce":115825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":115825},{"type":"v1","addr":"172.21.15.92:6829","nonce":115825}]},"public_addr":"172.21.15.92:6825/115825","cluster_addr":"172.21.15.92:6827/115825","heartbeat_back_addr":"172.21.15.92:6831/115825","heartbeat_front_addr":"172.21.15.92:6829/115825","state":["exists","up"]},{"osd":4,"uuid":"ccb9b747-f399-44ea-9a18-9db3152227e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":116035},{"type":"v1","addr":"172.21.15.169:6817","nonce":116035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":116035},{"type":"v1","addr":"172.21.15.169:6819","nonce":116035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":116035},{"type":"v1","addr":"172.21.15.169:6826","nonce":116035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":116035},{"type":"v1","addr":"172.21.15.169:6822","nonce":116035}]},"public_addr":"172.21.15.169:6817/116035","cluster_addr":"172.21.15.169:6819/116035","heartbeat_back_addr":"172.21.15.169:6826/116035","heartbeat_front_addr":"172.21.15.169:6822/116035","state":["exists","up"]},{"osd":5,"uuid":"308a1e63-75aa-4d7d-8a7c-cecc24714fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":116033},{"type":"v1","addr":"172.21.15.169:6801","nonce":116033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":116033},{"type":"v1","addr":"172.21.15.169:6803","nonce":116033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":116033},{"type":"v1","addr":"172.21.15.169:6807","nonce":116033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":116033},{"type":"v1","addr":"172.21.15.169:6805","nonce":116033}]},"public_addr":"172.21.15.169:6801/116033","cluster_addr":"172.21.15.169:6803/116033","heartbeat_back_addr":"172.21.15.169:6807/116033","heartbeat_front_addr":"172.21.15.169:6805/116033","state":["exists","up"]},{"osd":6,"uuid":"9c459b7b-b821-4b4b-b91e-6aaae271457c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":116034},{"type":"v1","addr":"172.21.15.169:6809","nonce":116034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":116034},{"type":"v1","addr":"172.21.15.169:6811","nonce":116034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":116034},{"type":"v1","addr":"172.21.15.169:6815","nonce":116034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":116034},{"type":"v1","addr":"172.21.15.169:6813","nonce":116034}]},"public_addr":"172.21.15.169:6809/116034","cluster_addr":"172.21.15.169:6811/116034","heartbeat_back_addr":"172.21.15.169:6815/116034","heartbeat_front_addr":"172.21.15.169:6813/116034","state":["exists","up"]},{"osd":7,"uuid":"1ca9aa4a-755d-4e28-a937-4d8dccab4df3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":116044},{"type":"v1","addr":"172.21.15.169:6823","nonce":116044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6825","nonce":116044},{"type":"v1","addr":"172.21.15.169:6827","nonce":116044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":116044},{"type":"v1","addr":"172.21.15.169:6831","nonce":116044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":116044},{"type":"v1","addr":"172.21.15.169:6829","nonce":116044}]},"public_addr":"172.21.15.169:6823/116044","cluster_addr":"172.21.15.169:6827/116044","heartbeat_back_addr":"172.21.15.169:6831/116044","heartbeat_front_addr":"172.21.15.169:6829/116044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.586781+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.869778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.508594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.860493+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.872267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.746933+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.611520+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.871446+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:23:55.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.488+0000 7f96bc3ff700 1 -- 172.21.15.92:0/2039864641 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f96980682f0 msgr2=0x7f969806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:55.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.488+0000 7f96bc3ff700 1 --2- 172.21.15.92:0/2039864641 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f96980682f0 0x7f969806a7a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f96b0000c00 tx=0x7f96b0008030 comp rx=0 tx=0).stop 2023-11-28T11:23:55.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.488+0000 7f96bc3ff700 1 -- 172.21.15.92:0/2039864641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96b40cd460 msgr2=0x7f96b40cc6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:55.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.488+0000 7f96bc3ff700 1 --2- 172.21.15.92:0/2039864641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96b40cd460 0x7f96b40cc6f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f96a400c520 tx=0x7f96a400c830 comp rx=0 tx=0).stop 2023-11-28T11:23:55.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.488+0000 7f96bc3ff700 1 -- 172.21.15.92:0/2039864641 shutdown_connections 2023-11-28T11:23:55.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.488+0000 7f96bc3ff700 1 --2- 172.21.15.92:0/2039864641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f96b40d0d60 0x7f96b40ca9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:55.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.488+0000 7f96bc3ff700 1 --2- 172.21.15.92:0/2039864641 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f96b40b8510 0x7f96b40cbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:55.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.488+0000 7f96bc3ff700 1 --2- 172.21.15.92:0/2039864641 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f96980682f0 0x7f969806a7a0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:55.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.488+0000 7f96bc3ff700 1 --2- 172.21.15.92:0/2039864641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96b40cd460 0x7f96b40cc6f0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:55.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.488+0000 7f96bc3ff700 1 -- 172.21.15.92:0/2039864641 >> 172.21.15.92:0/2039864641 conn(0x7f96b40719a0 msgr2=0x7f96b40b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:55.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.488+0000 7f96bc3ff700 1 -- 172.21.15.92:0/2039864641 shutdown_connections 2023-11-28T11:23:55.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.488+0000 7f96bc3ff700 1 -- 172.21.15.92:0/2039864641 wait complete. 2023-11-28T11:23:55.507 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:20:41.925237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:20:48.682497+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:23:10.939884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:23:12.227433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:23:55.507 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:23:55.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.707+0000 7faeeb1ff700 1 Processor -- start 2023-11-28T11:23:55.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.707+0000 7faeeb1ff700 1 -- start start 2023-11-28T11:23:55.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.708+0000 7faeeb1ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7faee40cd460 0x7faee406c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:55.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.708+0000 7faeeb1ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faee40d0d60 0x7faee40d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:55.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.708+0000 7faeeb1ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faee40b88e0 con 0x7faee40b8510 2023-11-28T11:23:55.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.708+0000 7faeeb1ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faee40cdbd0 con 0x7faee40cd460 2023-11-28T11:23:55.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.708+0000 7faeeb1ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faee40d02d0 con 0x7faee40d0d60 2023-11-28T11:23:55.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.708+0000 7faee8f9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7faee40b8510 0x7faee406b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36438/0 (socket says 172.21.15.92:36438) 2023-11-28T11:23:55.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.708+0000 7faee8f9b700 1 -- 172.21.15.92:0/2523358683 learned_addr learned my addr 172.21.15.92:0/2523358683 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:23:55.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.709+0000 7faee979c700 1 --2- 172.21.15.92:0/2523358683 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faee40d0d60 0x7faee40d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:55.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.709+0000 7faedbfff700 1 --2- 172.21.15.92:0/2523358683 >> v2:172.21.15.169:3300/0 conn(0x7faee40cd460 0x7faee406c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:55.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.709+0000 7faedb7fe700 1 -- 172.21.15.92:0/2523358683 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 869083653 0 0) 0x7faee40b88e0 con 0x7faee40b8510 2023-11-28T11:23:55.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.709+0000 7faedb7fe700 1 -- 172.21.15.92:0/2523358683 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faecc0035b0 con 0x7faee40b8510 2023-11-28T11:23:55.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.709+0000 7faedb7fe700 1 -- 172.21.15.92:0/2523358683 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 915630717 0 0) 0x7faecc0035b0 con 0x7faee40b8510 2023-11-28T11:23:55.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.709+0000 7faedb7fe700 1 -- 172.21.15.92:0/2523358683 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faee40d0d60 msgr2=0x7faee40d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:55.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.709+0000 7faedb7fe700 1 --2- 172.21.15.92:0/2523358683 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faee40d0d60 0x7faee40d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:55.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.709+0000 7faedb7fe700 1 -- 172.21.15.92:0/2523358683 >> v2:172.21.15.169:3300/0 conn(0x7faee40cd460 msgr2=0x7faee406c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:55.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.709+0000 7faedb7fe700 1 --2- 172.21.15.92:0/2523358683 >> v2:172.21.15.169:3300/0 conn(0x7faee40cd460 0x7faee406c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:55.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.709+0000 7faedb7fe700 1 -- 172.21.15.92:0/2523358683 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee40cd8f0 con 0x7faee40b8510 2023-11-28T11:23:55.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.709+0000 7faedb7fe700 1 -- 172.21.15.92:0/2523358683 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532161587 0 0) 0x7faee0001670 con 0x7faee40b8510 2023-11-28T11:23:55.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.709+0000 7faedb7fe700 1 -- 172.21.15.92:0/2523358683 >> v1:172.21.15.92:6789/0 conn(0x7faee40b8510 legacy=0x7faee406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:55.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.709+0000 7faedb7fe700 1 --2- 172.21.15.92:0/2523358683 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faecc003a30 0x7faecc003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:55.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.709+0000 7faedb7fe700 1 --2- 172.21.15.92:0/2523358683 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faecc004850 0x7faecc004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:55.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.709+0000 7faedb7fe700 1 --2- 172.21.15.92:0/2523358683 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faee40d0d60 0x7faecc005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:55.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.709+0000 7faedb7fe700 1 -- 172.21.15.92:0/2523358683 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee40cdbd0 con 0x7faecc004850 2023-11-28T11:23:55.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.709+0000 7faedb7fe700 1 -- 172.21.15.92:0/2523358683 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee40d02d0 con 0x7faecc003a30 2023-11-28T11:23:55.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.709+0000 7faedb7fe700 1 -- 172.21.15.92:0/2523358683 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faecc00ba10 con 0x7faee40d0d60 2023-11-28T11:23:55.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.710+0000 7faedbfff700 1 --2- 172.21.15.92:0/2523358683 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faecc004850 0x7faecc004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:55.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.710+0000 7faee8f9b700 1 --2- 172.21.15.92:0/2523358683 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faecc003a30 0x7faecc003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:55.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.710+0000 7faee979c700 1 --2- 172.21.15.92:0/2523358683 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faee40d0d60 0x7faecc005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:55.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.710+0000 7faedbfff700 1 -- 172.21.15.92:0/2523358683 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faee40d0d60 msgr2=0x7faecc005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:55.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.710+0000 7faedbfff700 1 --2- 172.21.15.92:0/2523358683 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faee40d0d60 0x7faecc005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:55.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.710+0000 7faedbfff700 1 -- 172.21.15.92:0/2523358683 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faecc003a30 msgr2=0x7faecc003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:55.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.710+0000 7faedbfff700 1 --2- 172.21.15.92:0/2523358683 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faecc003a30 0x7faecc003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:55.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.710+0000 7faedbfff700 1 -- 172.21.15.92:0/2523358683 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faecc0035b0 con 0x7faecc004850 2023-11-28T11:23:55.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.710+0000 7faee979c700 1 --2- 172.21.15.92:0/2523358683 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faee40d0d60 0x7faecc005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:23:55.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.710+0000 7faee8f9b700 1 --2- 172.21.15.92:0/2523358683 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faecc003a30 0x7faecc003e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:23:55.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.710+0000 7faedbfff700 1 --2- 172.21.15.92:0/2523358683 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faecc004850 0x7faecc004cc0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7faee40b1de0 tx=0x7faecc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63f2ce3da3bac50b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:55.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.710+0000 7faedb7fe700 1 -- 172.21.15.92:0/2523358683 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed400e6c0 con 0x7faecc004850 2023-11-28T11:23:55.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.711+0000 7faedb7fe700 1 -- 172.21.15.92:0/2523358683 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faed400e820 con 0x7faecc004850 2023-11-28T11:23:55.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.711+0000 7faedb7fe700 1 -- 172.21.15.92:0/2523358683 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed400ebb0 con 0x7faecc004850 2023-11-28T11:23:55.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.711+0000 7faeeb1ff700 1 -- 172.21.15.92:0/2523358683 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faecc004850 msgr2=0x7faecc004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:55.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.711+0000 7faeeb1ff700 1 --2- 172.21.15.92:0/2523358683 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faecc004850 0x7faecc004cc0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7faee40b1de0 tx=0x7faecc005880 comp rx=0 tx=0).stop 2023-11-28T11:23:55.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.711+0000 7faee979c700 1 -- 172.21.15.92:0/2523358683 reap_dead start 2023-11-28T11:23:55.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.711+0000 7faeeb1ff700 1 -- 172.21.15.92:0/2523358683 shutdown_connections 2023-11-28T11:23:55.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.711+0000 7faeeb1ff700 1 -- 172.21.15.92:0/2523358683 >> 172.21.15.92:0/2523358683 conn(0x7faee40719a0 msgr2=0x7faee4071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:55.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.711+0000 7faeeb1ff700 1 -- 172.21.15.92:0/2523358683 shutdown_connections 2023-11-28T11:23:55.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.711+0000 7faeeb1ff700 1 -- 172.21.15.92:0/2523358683 wait complete. 2023-11-28T11:23:55.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.713+0000 7faeeb1ff700 1 Processor -- start 2023-11-28T11:23:55.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.713+0000 7faeeb1ff700 1 -- start start 2023-11-28T11:23:55.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.714+0000 7faeeb1ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faecc004850 0x7faee40cc190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:55.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.714+0000 7faeeb1ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faee41f3420 0x7faee40c8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:55.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.714+0000 7faeeb1ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faee41544e0 0x7faee40c9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:55.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.714+0000 7faeeb1ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee41f2750 con 0x7faee41f3420 2023-11-28T11:23:55.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.714+0000 7faeeb1ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee40cc6d0 con 0x7faecc004850 2023-11-28T11:23:55.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.714+0000 7faeeb1ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faee415b6b0 con 0x7faee41544e0 2023-11-28T11:23:55.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.714+0000 7faedbfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faee41f3420 0x7faee40c8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:55.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.714+0000 7faedbfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faee41f3420 0x7faee40c8e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52180/0 (socket says 172.21.15.92:52180) 2023-11-28T11:23:55.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.714+0000 7faedbfff700 1 -- 172.21.15.92:0/3038761837 learned_addr learned my addr 172.21.15.92:0/3038761837 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:55.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.715+0000 7faee979c700 1 --2- 172.21.15.92:0/3038761837 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faee41544e0 0x7faee40c9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:55.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.715+0000 7faee8f9b700 1 --2- 172.21.15.92:0/3038761837 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faecc004850 0x7faee40cc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:55.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.715+0000 7faedbfff700 1 -- 172.21.15.92:0/3038761837 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faee41544e0 msgr2=0x7faee40c9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:55.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.715+0000 7faedbfff700 1 --2- 172.21.15.92:0/3038761837 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faee41544e0 0x7faee40c9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:55.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.715+0000 7faedbfff700 1 -- 172.21.15.92:0/3038761837 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faecc004850 msgr2=0x7faee40cc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:55.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.715+0000 7faedbfff700 1 --2- 172.21.15.92:0/3038761837 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faecc004850 0x7faee40cc190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:55.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.715+0000 7faedbfff700 1 -- 172.21.15.92:0/3038761837 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee40cfff0 con 0x7faee41f3420 2023-11-28T11:23:55.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.715+0000 7faee8f9b700 1 --2- 172.21.15.92:0/3038761837 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faecc004850 0x7faee40cc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:23:55.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.715+0000 7faedbfff700 1 --2- 172.21.15.92:0/3038761837 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faee41f3420 0x7faee40c8e10 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7faecc0046a0 tx=0x7faed40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:55.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.715+0000 7faed97fa700 1 -- 172.21.15.92:0/3038761837 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed400e6c0 con 0x7faee41f3420 2023-11-28T11:23:55.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.715+0000 7faed97fa700 1 -- 172.21.15.92:0/3038761837 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faed4003ab0 con 0x7faee41f3420 2023-11-28T11:23:55.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.715+0000 7faeeb1ff700 1 -- 172.21.15.92:0/3038761837 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faee40cfd10 con 0x7faee41f3420 2023-11-28T11:23:55.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.715+0000 7faed97fa700 1 -- 172.21.15.92:0/3038761837 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed4006e60 con 0x7faee41f3420 2023-11-28T11:23:55.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.715+0000 7faeeb1ff700 1 -- 172.21.15.92:0/3038761837 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faee40d29c0 con 0x7faee41f3420 2023-11-28T11:23:55.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.716+0000 7faeeb1ff700 1 -- 172.21.15.92:0/3038761837 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faee4067880 con 0x7faee41f3420 2023-11-28T11:23:55.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.717+0000 7faed97fa700 1 -- 172.21.15.92:0/3038761837 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7faed4025070 con 0x7faee41f3420 2023-11-28T11:23:55.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.718+0000 7faed97fa700 1 --2- 172.21.15.92:0/3038761837 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7faed00682f0 0x7faed006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:55.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.718+0000 7faed97fa700 1 -- 172.21.15.92:0/3038761837 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6382+0+0 (secure 0 0 0) 0x7faed4087850 con 0x7faee41f3420 2023-11-28T11:23:55.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.718+0000 7faee8f9b700 1 --2- 172.21.15.92:0/3038761837 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7faed00682f0 0x7faed006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:55.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.719+0000 7faee8f9b700 1 --2- 172.21.15.92:0/3038761837 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7faed00682f0 0x7faed006a7a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7faee0001eb0 tx=0x7faee0001430 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:55.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.722+0000 7faed97fa700 1 -- 172.21.15.92:0/3038761837 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7faed4053e80 con 0x7faee41f3420 2023-11-28T11:23:55.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.912+0000 7faeeb1ff700 1 -- 172.21.15.92:0/3038761837 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faee40c66e0 con 0x7faee41f3420 2023-11-28T11:23:55.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.912+0000 7faed97fa700 1 -- 172.21.15.92:0/3038761837 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v56) v1 ==== 93+0+10 (secure 0 0 0) 0x7faed4085e60 con 0x7faee41f3420 2023-11-28T11:23:55.913 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-11-28T11:23:55.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.914+0000 7faeeb1ff700 1 -- 172.21.15.92:0/3038761837 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7faed00682f0 msgr2=0x7faed006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:55.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.914+0000 7faeeb1ff700 1 --2- 172.21.15.92:0/3038761837 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7faed00682f0 0x7faed006a7a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7faee0001eb0 tx=0x7faee0001430 comp rx=0 tx=0).stop 2023-11-28T11:23:55.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.914+0000 7faeeb1ff700 1 -- 172.21.15.92:0/3038761837 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faee41f3420 msgr2=0x7faee40c8e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:55.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.914+0000 7faeeb1ff700 1 --2- 172.21.15.92:0/3038761837 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faee41f3420 0x7faee40c8e10 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7faecc0046a0 tx=0x7faed40047c0 comp rx=0 tx=0).stop 2023-11-28T11:23:55.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.915+0000 7faeeb1ff700 1 -- 172.21.15.92:0/3038761837 shutdown_connections 2023-11-28T11:23:55.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.915+0000 7faeeb1ff700 1 --2- 172.21.15.92:0/3038761837 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faee41544e0 0x7faee40c9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:55.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.915+0000 7faeeb1ff700 1 --2- 172.21.15.92:0/3038761837 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faecc004850 0x7faee40cc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:55.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.915+0000 7faeeb1ff700 1 --2- 172.21.15.92:0/3038761837 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7faed00682f0 0x7faed006a7a0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:55.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.915+0000 7faeeb1ff700 1 --2- 172.21.15.92:0/3038761837 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faee41f3420 0x7faee40c8e10 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:55.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.915+0000 7faeeb1ff700 1 -- 172.21.15.92:0/3038761837 >> 172.21.15.92:0/3038761837 conn(0x7faee40719a0 msgr2=0x7faee40d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:55.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.915+0000 7faeeb1ff700 1 -- 172.21.15.92:0/3038761837 shutdown_connections 2023-11-28T11:23:55.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:55.915+0000 7faeeb1ff700 1 -- 172.21.15.92:0/3038761837 wait complete. 2023-11-28T11:23:55.934 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:23:56.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.133+0000 7fac997ff700 1 Processor -- start 2023-11-28T11:23:56.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.134+0000 7fac997ff700 1 -- start start 2023-11-28T11:23:56.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.135+0000 7fac997ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fac9415a110 0x7fac94160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:56.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.135+0000 7fac997ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fac9415d820 0x7fac940cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:56.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.135+0000 7fac997ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac94161070 con 0x7fac940cfcf0 2023-11-28T11:23:56.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.135+0000 7fac997ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac940cdae0 con 0x7fac9415d820 2023-11-28T11:23:56.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.135+0000 7fac997ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac940ce4f0 con 0x7fac9415a110 2023-11-28T11:23:56.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.135+0000 7fac8bfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fac9415d820 0x7fac940cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:56.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.135+0000 7fac8bfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fac9415d820 0x7fac940cf7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52184/0 (socket says 172.21.15.92:52184) 2023-11-28T11:23:56.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.135+0000 7fac8bfff700 1 -- 172.21.15.92:0/1669714833 learned_addr learned my addr 172.21.15.92:0/1669714833 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:56.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.135+0000 7fac92ffd700 1 --2- 172.21.15.92:0/1669714833 >> v2:172.21.15.169:3300/0 conn(0x7fac9415a110 0x7fac94160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:56.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.136+0000 7fac8bfff700 1 -- 172.21.15.92:0/1669714833 >> v2:172.21.15.169:3300/0 conn(0x7fac9415a110 msgr2=0x7fac94160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:56.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.136+0000 7fac8bfff700 1 --2- 172.21.15.92:0/1669714833 >> v2:172.21.15.169:3300/0 conn(0x7fac9415a110 0x7fac94160b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:56.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.136+0000 7fac8bfff700 1 -- 172.21.15.92:0/1669714833 >> v1:172.21.15.92:6789/0 conn(0x7fac940cfcf0 legacy=0x7fac940b19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:56.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.136+0000 7fac927fc700 1 -- 172.21.15.92:0/1669714833 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3806912411 0 0) 0x7fac94161070 con 0x7fac940cfcf0 2023-11-28T11:23:56.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.136+0000 7fac8bfff700 1 -- 172.21.15.92:0/1669714833 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac940d0170 con 0x7fac9415d820 2023-11-28T11:23:56.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.136+0000 7fac8bfff700 1 --2- 172.21.15.92:0/1669714833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac9415d820 0x7fac940cf7b0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fac9408bf90 tx=0x7fac7c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=631e7257bb5181b5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:56.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.136+0000 7fac927fc700 1 -- 172.21.15.92:0/1669714833 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac7c00b870 con 0x7fac9415d820 2023-11-28T11:23:56.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.136+0000 7fac927fc700 1 -- 172.21.15.92:0/1669714833 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac7c00b9d0 con 0x7fac9415d820 2023-11-28T11:23:56.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.136+0000 7fac927fc700 1 -- 172.21.15.92:0/1669714833 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac7c00bd60 con 0x7fac9415d820 2023-11-28T11:23:56.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.137+0000 7fac997ff700 1 -- 172.21.15.92:0/1669714833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac9415d820 msgr2=0x7fac940cf7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:56.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.137+0000 7fac997ff700 1 --2- 172.21.15.92:0/1669714833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac9415d820 0x7fac940cf7b0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fac9408bf90 tx=0x7fac7c009bd0 comp rx=0 tx=0).stop 2023-11-28T11:23:56.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.137+0000 7fac997ff700 1 -- 172.21.15.92:0/1669714833 shutdown_connections 2023-11-28T11:23:56.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.137+0000 7fac997ff700 1 --2- 172.21.15.92:0/1669714833 >> v2:172.21.15.169:3300/0 conn(0x7fac9415a110 0x7fac94160b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:56.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.137+0000 7fac997ff700 1 --2- 172.21.15.92:0/1669714833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac9415d820 0x7fac940cf7b0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:56.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.137+0000 7fac997ff700 1 -- 172.21.15.92:0/1669714833 >> 172.21.15.92:0/1669714833 conn(0x7fac940719a0 msgr2=0x7fac94071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:56.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.137+0000 7fac997ff700 1 -- 172.21.15.92:0/1669714833 shutdown_connections 2023-11-28T11:23:56.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.137+0000 7fac997ff700 1 -- 172.21.15.92:0/1669714833 wait complete. 2023-11-28T11:23:56.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.139+0000 7fac997ff700 1 Processor -- start 2023-11-28T11:23:56.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.139+0000 7fac997ff700 1 -- start start 2023-11-28T11:23:56.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.140+0000 7fac997ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac940cfcf0 0x7fac941561e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:56.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.140+0000 7fac997ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac9415a110 0x7fac940b6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:56.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.140+0000 7fac997ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac9415d820 0x7fac9414a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:56.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.140+0000 7fac997ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac940b6890 con 0x7fac9415d820 2023-11-28T11:23:56.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.140+0000 7fac997ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac940caa90 con 0x7fac9415a110 2023-11-28T11:23:56.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.140+0000 7fac997ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac940cad70 con 0x7fac940cfcf0 2023-11-28T11:23:56.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.140+0000 7fac937fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac9415d820 0x7fac9414a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:56.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.140+0000 7fac937fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac9415d820 0x7fac9414a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52198/0 (socket says 172.21.15.92:52198) 2023-11-28T11:23:56.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.140+0000 7fac937fe700 1 -- 172.21.15.92:0/3928089403 learned_addr learned my addr 172.21.15.92:0/3928089403 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:56.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.140+0000 7fac92ffd700 1 --2- 172.21.15.92:0/3928089403 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac940cfcf0 0x7fac941561e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:56.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.140+0000 7fac8bfff700 1 --2- 172.21.15.92:0/3928089403 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac9415a110 0x7fac940b6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:56.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.140+0000 7fac937fe700 1 -- 172.21.15.92:0/3928089403 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac940cfcf0 msgr2=0x7fac941561e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:56.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.140+0000 7fac937fe700 1 --2- 172.21.15.92:0/3928089403 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac940cfcf0 0x7fac941561e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:56.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.140+0000 7fac937fe700 1 -- 172.21.15.92:0/3928089403 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac9415a110 msgr2=0x7fac940b6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:56.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.140+0000 7fac937fe700 1 --2- 172.21.15.92:0/3928089403 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac9415a110 0x7fac940b6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:56.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.140+0000 7fac937fe700 1 -- 172.21.15.92:0/3928089403 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac7c009550 con 0x7fac9415d820 2023-11-28T11:23:56.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.141+0000 7fac937fe700 1 --2- 172.21.15.92:0/3928089403 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac9415d820 0x7fac9414a110 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fac8c001560 tx=0x7fac8c00cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:56.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.141+0000 7fac8b7fe700 1 -- 172.21.15.92:0/3928089403 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8c003e30 con 0x7fac9415d820 2023-11-28T11:23:56.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.141+0000 7fac8b7fe700 1 -- 172.21.15.92:0/3928089403 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac8c005070 con 0x7fac9415d820 2023-11-28T11:23:56.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.141+0000 7fac8b7fe700 1 -- 172.21.15.92:0/3928089403 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8c005420 con 0x7fac9415d820 2023-11-28T11:23:56.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.141+0000 7fac997ff700 1 -- 172.21.15.92:0/3928089403 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac940cd800 con 0x7fac9415d820 2023-11-28T11:23:56.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.141+0000 7fac997ff700 1 -- 172.21.15.92:0/3928089403 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac940cf2d0 con 0x7fac9415d820 2023-11-28T11:23:56.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.142+0000 7fac997ff700 1 -- 172.21.15.92:0/3928089403 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac9415cd00 con 0x7fac9415d820 2023-11-28T11:23:56.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.143+0000 7fac8b7fe700 1 -- 172.21.15.92:0/3928089403 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fac8c019030 con 0x7fac9415d820 2023-11-28T11:23:56.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.144+0000 7fac8b7fe700 1 --2- 172.21.15.92:0/3928089403 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fac740682f0 0x7fac7406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:56.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.144+0000 7fac8b7fe700 1 -- 172.21.15.92:0/3928089403 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fac8c085420 con 0x7fac9415d820 2023-11-28T11:23:56.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.144+0000 7fac92ffd700 1 --2- 172.21.15.92:0/3928089403 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fac740682f0 0x7fac7406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:56.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.145+0000 7fac92ffd700 1 --2- 172.21.15.92:0/3928089403 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fac740682f0 0x7fac7406a7a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fac940d2a60 tx=0x7fac84008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:56.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.148+0000 7fac8b7fe700 1 -- 172.21.15.92:0/3928089403 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fac8c051a50 con 0x7fac9415d820 2023-11-28T11:23:56.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.340+0000 7fac997ff700 1 -- 172.21.15.92:0/3928089403 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fac94067880 con 0x7fac9415d820 2023-11-28T11:23:56.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.340+0000 7fac8b7fe700 1 -- 172.21.15.92:0/3928089403 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v56) v1 ==== 92+0+10 (secure 0 0 0) 0x7fac8c055070 con 0x7fac9415d820 2023-11-28T11:23:56.342 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-11-28T11:23:56.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.343+0000 7fac997ff700 1 -- 172.21.15.92:0/3928089403 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fac740682f0 msgr2=0x7fac7406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:56.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.343+0000 7fac997ff700 1 --2- 172.21.15.92:0/3928089403 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fac740682f0 0x7fac7406a7a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fac940d2a60 tx=0x7fac84008040 comp rx=0 tx=0).stop 2023-11-28T11:23:56.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.343+0000 7fac997ff700 1 -- 172.21.15.92:0/3928089403 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac9415d820 msgr2=0x7fac9414a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:56.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.343+0000 7fac997ff700 1 --2- 172.21.15.92:0/3928089403 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac9415d820 0x7fac9414a110 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fac8c001560 tx=0x7fac8c00cb50 comp rx=0 tx=0).stop 2023-11-28T11:23:56.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.343+0000 7fac997ff700 1 -- 172.21.15.92:0/3928089403 shutdown_connections 2023-11-28T11:23:56.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.343+0000 7fac997ff700 1 --2- 172.21.15.92:0/3928089403 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fac740682f0 0x7fac7406a7a0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:56.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.343+0000 7fac997ff700 1 --2- 172.21.15.92:0/3928089403 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac9415d820 0x7fac9414a110 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:56.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.343+0000 7fac997ff700 1 --2- 172.21.15.92:0/3928089403 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fac940cfcf0 0x7fac941561e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:56.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.343+0000 7fac997ff700 1 --2- 172.21.15.92:0/3928089403 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fac9415a110 0x7fac940b6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:56.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.343+0000 7fac997ff700 1 -- 172.21.15.92:0/3928089403 >> 172.21.15.92:0/3928089403 conn(0x7fac940719a0 msgr2=0x7fac940d1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:56.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.343+0000 7fac997ff700 1 -- 172.21.15.92:0/3928089403 shutdown_connections 2023-11-28T11:23:56.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.343+0000 7fac997ff700 1 -- 172.21.15.92:0/3928089403 wait complete. 2023-11-28T11:23:56.363 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:23:56.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.561+0000 7ff0d73ff700 1 Processor -- start 2023-11-28T11:23:56.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.561+0000 7ff0d73ff700 1 -- start start 2023-11-28T11:23:56.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.562+0000 7ff0d73ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff0d00b8510 0x7ff0d006b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:56.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.562+0000 7ff0d73ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0d00b88e0 con 0x7ff0d00cd460 2023-11-28T11:23:56.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.562+0000 7ff0d73ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0d00cd8f0 con 0x7ff0d00d0d60 2023-11-28T11:23:56.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.562+0000 7ff0d73ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d00cdbd0 con 0x7ff0d00b8510 2023-11-28T11:23:56.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.563+0000 7ff0d499a700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff0d00cd460 0x7ff0d006beb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36450/0 (socket says 172.21.15.92:36450) 2023-11-28T11:23:56.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.563+0000 7ff0d499a700 1 -- 172.21.15.92:0/1779442203 learned_addr learned my addr 172.21.15.92:0/1779442203 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:23:56.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.563+0000 7ff0d519b700 1 --2- 172.21.15.92:0/1779442203 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff0d00b8510 0x7ff0d006b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:56.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.563+0000 7ff0c7fff700 1 -- 172.21.15.92:0/1779442203 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3344737442 0 0) 0x7ff0d00b88e0 con 0x7ff0d00cd460 2023-11-28T11:23:56.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.563+0000 7ff0c7fff700 1 -- 172.21.15.92:0/1779442203 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0bc0035b0 con 0x7ff0d00cd460 2023-11-28T11:23:56.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.563+0000 7ff0c7fff700 1 -- 172.21.15.92:0/1779442203 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 266376289 0 0) 0x7ff0d00cd8f0 con 0x7ff0d00d0d60 2023-11-28T11:23:56.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.563+0000 7ff0c7fff700 1 -- 172.21.15.92:0/1779442203 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0d00b88e0 con 0x7ff0d00d0d60 2023-11-28T11:23:56.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.563+0000 7ff0d519b700 1 -- 172.21.15.92:0/1779442203 >> v1:172.21.15.169:6789/0 conn(0x7ff0d00d0d60 legacy=0x7ff0d00d11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:56.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.563+0000 7ff0d519b700 1 -- 172.21.15.92:0/1779442203 >> v1:172.21.15.92:6789/0 conn(0x7ff0d00cd460 legacy=0x7ff0d006beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:56.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.563+0000 7ff0d519b700 1 -- 172.21.15.92:0/1779442203 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0d00cfd10 con 0x7ff0d00b8510 2023-11-28T11:23:56.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.564+0000 7ff0d519b700 1 --2- 172.21.15.92:0/1779442203 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff0d00b8510 0x7ff0d006b970 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc009870 tx=0x7ff0cc009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a110d4202b759f0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:56.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.564+0000 7ff0c7fff700 1 -- 172.21.15.92:0/1779442203 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc011830 con 0x7ff0d00b8510 2023-11-28T11:23:56.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.564+0000 7ff0c7fff700 1 -- 172.21.15.92:0/1779442203 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0cc011990 con 0x7ff0d00b8510 2023-11-28T11:23:56.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.564+0000 7ff0c7fff700 1 -- 172.21.15.92:0/1779442203 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc011d20 con 0x7ff0d00b8510 2023-11-28T11:23:56.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.564+0000 7ff0d73ff700 1 -- 172.21.15.92:0/1779442203 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff0d00b8510 msgr2=0x7ff0d006b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:56.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.564+0000 7ff0d73ff700 1 --2- 172.21.15.92:0/1779442203 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff0d00b8510 0x7ff0d006b970 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc009870 tx=0x7ff0cc009b80 comp rx=0 tx=0).stop 2023-11-28T11:23:56.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.564+0000 7ff0d73ff700 1 -- 172.21.15.92:0/1779442203 shutdown_connections 2023-11-28T11:23:56.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.564+0000 7ff0d73ff700 1 --2- 172.21.15.92:0/1779442203 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff0d00b8510 0x7ff0d006b970 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:56.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.564+0000 7ff0d73ff700 1 -- 172.21.15.92:0/1779442203 >> 172.21.15.92:0/1779442203 conn(0x7ff0d00719a0 msgr2=0x7ff0d0071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:56.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.564+0000 7ff0d73ff700 1 -- 172.21.15.92:0/1779442203 shutdown_connections 2023-11-28T11:23:56.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.565+0000 7ff0d73ff700 1 -- 172.21.15.92:0/1779442203 wait complete. 2023-11-28T11:23:56.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.566+0000 7ff0d73ff700 1 Processor -- start 2023-11-28T11:23:56.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.566+0000 7ff0d73ff700 1 -- start start 2023-11-28T11:23:56.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.567+0000 7ff0d73ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff0d00b8510 0x7ff0d00ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:56.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.567+0000 7ff0d73ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff0d00cd460 0x7ff0d0153d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:56.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.567+0000 7ff0d73ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0d00d0d60 0x7ff0d01572a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:56.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.567+0000 7ff0d73ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d01585d0 con 0x7ff0d00d0d60 2023-11-28T11:23:56.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.567+0000 7ff0d73ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d0159430 con 0x7ff0d00cd460 2023-11-28T11:23:56.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.567+0000 7ff0d73ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d00cd270 con 0x7ff0d00b8510 2023-11-28T11:23:56.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.568+0000 7ff0d599c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0d00d0d60 0x7ff0d01572a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:56.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.568+0000 7ff0d599c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0d00d0d60 0x7ff0d01572a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52204/0 (socket says 172.21.15.92:52204) 2023-11-28T11:23:56.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.568+0000 7ff0d599c700 1 -- 172.21.15.92:0/2893150343 learned_addr learned my addr 172.21.15.92:0/2893150343 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:56.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.568+0000 7ff0d519b700 1 --2- 172.21.15.92:0/2893150343 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff0d00b8510 0x7ff0d00ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:56.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.568+0000 7ff0d499a700 1 --2- 172.21.15.92:0/2893150343 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff0d00cd460 0x7ff0d0153d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:56.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.568+0000 7ff0d599c700 1 -- 172.21.15.92:0/2893150343 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff0d00b8510 msgr2=0x7ff0d00ccd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:56.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.568+0000 7ff0d599c700 1 --2- 172.21.15.92:0/2893150343 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff0d00b8510 0x7ff0d00ccd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:56.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.568+0000 7ff0d599c700 1 -- 172.21.15.92:0/2893150343 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff0d00cd460 msgr2=0x7ff0d0153d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:56.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.568+0000 7ff0d599c700 1 --2- 172.21.15.92:0/2893150343 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff0d00cd460 0x7ff0d0153d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:56.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.568+0000 7ff0d599c700 1 -- 172.21.15.92:0/2893150343 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0cc0094a0 con 0x7ff0d00d0d60 2023-11-28T11:23:56.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.568+0000 7ff0d599c700 1 --2- 172.21.15.92:0/2893150343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0d00d0d60 0x7ff0d01572a0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7ff0c800cbc0 tx=0x7ff0c800cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:56.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.568+0000 7ff0c5ffb700 1 -- 172.21.15.92:0/2893150343 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c8003c00 con 0x7ff0d00d0d60 2023-11-28T11:23:56.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.568+0000 7ff0c5ffb700 1 -- 172.21.15.92:0/2893150343 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c8003d60 con 0x7ff0d00d0d60 2023-11-28T11:23:56.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.568+0000 7ff0d73ff700 1 -- 172.21.15.92:0/2893150343 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0d00d29c0 con 0x7ff0d00d0d60 2023-11-28T11:23:56.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.568+0000 7ff0c5ffb700 1 -- 172.21.15.92:0/2893150343 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c800ab50 con 0x7ff0d00d0d60 2023-11-28T11:23:56.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.569+0000 7ff0d73ff700 1 -- 172.21.15.92:0/2893150343 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0d00cfff0 con 0x7ff0d00d0d60 2023-11-28T11:23:56.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.570+0000 7ff0d73ff700 1 -- 172.21.15.92:0/2893150343 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0d00d2b50 con 0x7ff0d00d0d60 2023-11-28T11:23:56.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.570+0000 7ff0c5ffb700 1 -- 172.21.15.92:0/2893150343 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff0c8010050 con 0x7ff0d00d0d60 2023-11-28T11:23:56.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.571+0000 7ff0c5ffb700 1 --2- 172.21.15.92:0/2893150343 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff0b40682f0 0x7ff0b406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:56.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.571+0000 7ff0c5ffb700 1 -- 172.21.15.92:0/2893150343 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6382+0+0 (secure 0 0 0) 0x7ff0c8085e90 con 0x7ff0d00d0d60 2023-11-28T11:23:56.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.572+0000 7ff0d519b700 1 --2- 172.21.15.92:0/2893150343 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff0b40682f0 0x7ff0b406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:56.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.572+0000 7ff0d519b700 1 --2- 172.21.15.92:0/2893150343 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff0b40682f0 0x7ff0b406a7a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc0097c0 tx=0x7ff0cc01c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:56.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.576+0000 7ff0c5ffb700 1 -- 172.21.15.92:0/2893150343 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff0c8052430 con 0x7ff0d00d0d60 2023-11-28T11:23:56.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.767+0000 7ff0d73ff700 1 -- 172.21.15.92:0/2893150343 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff0d0067880 con 0x7ff0d00d0d60 2023-11-28T11:23:56.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.768+0000 7ff0c5ffb700 1 -- 172.21.15.92:0/2893150343 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v56) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff0c800acb0 con 0x7ff0d00d0d60 2023-11-28T11:23:56.769 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-11-28T11:23:56.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.770+0000 7ff0d73ff700 1 -- 172.21.15.92:0/2893150343 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff0b40682f0 msgr2=0x7ff0b406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:56.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.770+0000 7ff0d73ff700 1 --2- 172.21.15.92:0/2893150343 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff0b40682f0 0x7ff0b406a7a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc0097c0 tx=0x7ff0cc01c040 comp rx=0 tx=0).stop 2023-11-28T11:23:56.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.770+0000 7ff0d73ff700 1 -- 172.21.15.92:0/2893150343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0d00d0d60 msgr2=0x7ff0d01572a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:56.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.770+0000 7ff0d73ff700 1 --2- 172.21.15.92:0/2893150343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0d00d0d60 0x7ff0d01572a0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7ff0c800cbc0 tx=0x7ff0c800cf80 comp rx=0 tx=0).stop 2023-11-28T11:23:56.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.770+0000 7ff0d73ff700 1 -- 172.21.15.92:0/2893150343 shutdown_connections 2023-11-28T11:23:56.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.770+0000 7ff0d73ff700 1 --2- 172.21.15.92:0/2893150343 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff0b40682f0 0x7ff0b406a7a0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:56.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.770+0000 7ff0d73ff700 1 --2- 172.21.15.92:0/2893150343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0d00d0d60 0x7ff0d01572a0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:56.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.770+0000 7ff0d73ff700 1 --2- 172.21.15.92:0/2893150343 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff0d00b8510 0x7ff0d00ccd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:56.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.770+0000 7ff0d73ff700 1 --2- 172.21.15.92:0/2893150343 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff0d00cd460 0x7ff0d0153d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:56.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.770+0000 7ff0d73ff700 1 -- 172.21.15.92:0/2893150343 >> 172.21.15.92:0/2893150343 conn(0x7ff0d00719a0 msgr2=0x7ff0d00b1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:56.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.770+0000 7ff0d73ff700 1 -- 172.21.15.92:0/2893150343 shutdown_connections 2023-11-28T11:23:56.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.770+0000 7ff0d73ff700 1 -- 172.21.15.92:0/2893150343 wait complete. 2023-11-28T11:23:56.789 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:23:56.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.988+0000 7ff2a5dff700 1 Processor -- start 2023-11-28T11:23:56.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.989+0000 7ff2a5dff700 1 -- start start 2023-11-28T11:23:56.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.990+0000 7ff2a5dff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2a00cd460 0x7ff2a006c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:56.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.990+0000 7ff2a5dff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff2a00d0d60 0x7ff2a00d11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:56.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.990+0000 7ff2a5dff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2a00b88e0 con 0x7ff2a00b8510 2023-11-28T11:23:56.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.990+0000 7ff2a5dff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2a00cdbd0 con 0x7ff2a00d0d60 2023-11-28T11:23:56.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.990+0000 7ff2a5dff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a00d02d0 con 0x7ff2a00cd460 2023-11-28T11:23:56.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.990+0000 7ff29ffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff2a00d0d60 0x7ff2a00d11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:56.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.990+0000 7ff29ffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff2a00d0d60 0x7ff2a00d11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52214/0 (socket says 172.21.15.92:52214) 2023-11-28T11:23:56.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.990+0000 7ff29ffff700 1 -- 172.21.15.92:0/2543269558 learned_addr learned my addr 172.21.15.92:0/2543269558 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:56.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.990+0000 7ff29effd700 1 --2- 172.21.15.92:0/2543269558 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2a00cd460 0x7ff2a006c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:56.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.991+0000 7ff29ffff700 1 -- 172.21.15.92:0/2543269558 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2a00cd460 msgr2=0x7ff2a006c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:56.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.991+0000 7ff29ffff700 1 --2- 172.21.15.92:0/2543269558 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2a00cd460 0x7ff2a006c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:56.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.991+0000 7ff29ffff700 1 -- 172.21.15.92:0/2543269558 >> v1:172.21.15.169:6789/0 conn(0x7ff2a00b8510 legacy=0x7ff2a006b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:56.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.991+0000 7ff29ffff700 1 -- 172.21.15.92:0/2543269558 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a00cd8f0 con 0x7ff2a00d0d60 2023-11-28T11:23:56.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.991+0000 7ff29ffff700 1 --2- 172.21.15.92:0/2543269558 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2a00d0d60 0x7ff2a00d11e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff298009e80 tx=0x7ff29800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=849901059fc26fa5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:56.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.991+0000 7ff29e7fc700 1 -- 172.21.15.92:0/2543269558 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29800e070 con 0x7ff2a00d0d60 2023-11-28T11:23:56.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.991+0000 7ff29e7fc700 1 -- 172.21.15.92:0/2543269558 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff29800b8d0 con 0x7ff2a00d0d60 2023-11-28T11:23:56.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.991+0000 7ff29e7fc700 1 -- 172.21.15.92:0/2543269558 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29800bc80 con 0x7ff2a00d0d60 2023-11-28T11:23:56.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.992+0000 7ff2a5dff700 1 -- 172.21.15.92:0/2543269558 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2a00d0d60 msgr2=0x7ff2a00d11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:56.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.992+0000 7ff2a5dff700 1 --2- 172.21.15.92:0/2543269558 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2a00d0d60 0x7ff2a00d11e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff298009e80 tx=0x7ff29800b330 comp rx=0 tx=0).stop 2023-11-28T11:23:56.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.992+0000 7ff2a5dff700 1 -- 172.21.15.92:0/2543269558 shutdown_connections 2023-11-28T11:23:56.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.992+0000 7ff2a5dff700 1 --2- 172.21.15.92:0/2543269558 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2a00d0d60 0x7ff2a00d11e0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:56.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.992+0000 7ff2a5dff700 1 --2- 172.21.15.92:0/2543269558 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2a00cd460 0x7ff2a006c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:56.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.992+0000 7ff2a5dff700 1 -- 172.21.15.92:0/2543269558 >> 172.21.15.92:0/2543269558 conn(0x7ff2a00719a0 msgr2=0x7ff2a0071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:56.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.992+0000 7ff2a5dff700 1 -- 172.21.15.92:0/2543269558 shutdown_connections 2023-11-28T11:23:56.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.992+0000 7ff2a5dff700 1 -- 172.21.15.92:0/2543269558 wait complete. 2023-11-28T11:23:56.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.993+0000 7ff2a5dff700 1 Processor -- start 2023-11-28T11:23:56.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.994+0000 7ff2a5dff700 1 -- start start 2023-11-28T11:23:56.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.994+0000 7ff2a5dff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2a00b8510 0x7ff2a01f2b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:56.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.995+0000 7ff2a5dff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2a00cd460 0x7ff2a015a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:56.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.995+0000 7ff2a5dff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2a00d0d60 0x7ff2a01f55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:56.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.995+0000 7ff2a5dff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a00cbbc0 con 0x7ff2a00b8510 2023-11-28T11:23:56.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.995+0000 7ff2a5dff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a01f6150 con 0x7ff2a00d0d60 2023-11-28T11:23:56.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.995+0000 7ff2a5dff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a01f6430 con 0x7ff2a00cd460 2023-11-28T11:23:56.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.995+0000 7ff29f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2a00b8510 0x7ff2a01f2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:56.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.995+0000 7ff29f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2a00b8510 0x7ff2a01f2b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52222/0 (socket says 172.21.15.92:52222) 2023-11-28T11:23:56.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.995+0000 7ff29effd700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2a00cd460 0x7ff2a015a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:56.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.995+0000 7ff29f7fe700 1 -- 172.21.15.92:0/1148031409 learned_addr learned my addr 172.21.15.92:0/1148031409 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:56.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.995+0000 7ff29ffff700 1 --2- 172.21.15.92:0/1148031409 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2a00d0d60 0x7ff2a01f55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:56.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.995+0000 7ff29f7fe700 1 -- 172.21.15.92:0/1148031409 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2a00cd460 msgr2=0x7ff2a015a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:56.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.995+0000 7ff29f7fe700 1 --2- 172.21.15.92:0/1148031409 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2a00cd460 0x7ff2a015a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:56.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.995+0000 7ff29f7fe700 1 -- 172.21.15.92:0/1148031409 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2a00d0d60 msgr2=0x7ff2a01f55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:56.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.995+0000 7ff29f7fe700 1 --2- 172.21.15.92:0/1148031409 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2a00d0d60 0x7ff2a01f55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:56.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.995+0000 7ff29f7fe700 1 -- 172.21.15.92:0/1148031409 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff298009b30 con 0x7ff2a00b8510 2023-11-28T11:23:56.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.996+0000 7ff29f7fe700 1 --2- 172.21.15.92:0/1148031409 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2a00b8510 0x7ff2a01f2b40 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7ff29000c4e0 tx=0x7ff29000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:56.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.996+0000 7ff287fff700 1 -- 172.21.15.92:0/1148031409 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2900052d0 con 0x7ff2a00b8510 2023-11-28T11:23:56.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.996+0000 7ff287fff700 1 -- 172.21.15.92:0/1148031409 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff290005430 con 0x7ff2a00b8510 2023-11-28T11:23:56.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.996+0000 7ff287fff700 1 -- 172.21.15.92:0/1148031409 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2900038c0 con 0x7ff2a00b8510 2023-11-28T11:23:56.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.996+0000 7ff2a5dff700 1 -- 172.21.15.92:0/1148031409 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2a00cfff0 con 0x7ff2a00b8510 2023-11-28T11:23:56.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.996+0000 7ff2a5dff700 1 -- 172.21.15.92:0/1148031409 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2a00cfd10 con 0x7ff2a00b8510 2023-11-28T11:23:56.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.997+0000 7ff2a5dff700 1 -- 172.21.15.92:0/1148031409 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2a00b73f0 con 0x7ff2a00b8510 2023-11-28T11:23:57.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.998+0000 7ff287fff700 1 -- 172.21.15.92:0/1148031409 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff290019030 con 0x7ff2a00b8510 2023-11-28T11:23:57.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.999+0000 7ff287fff700 1 --2- 172.21.15.92:0/1148031409 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff28c0682f0 0x7ff28c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:57.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.999+0000 7ff287fff700 1 -- 172.21.15.92:0/1148031409 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6382+0+0 (secure 0 0 0) 0x7ff2900851c0 con 0x7ff2a00b8510 2023-11-28T11:23:57.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:56.999+0000 7ff29effd700 1 --2- 172.21.15.92:0/1148031409 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff28c0682f0 0x7ff28c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:57.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.000+0000 7ff29effd700 1 --2- 172.21.15.92:0/1148031409 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff28c0682f0 0x7ff28c06a7a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff2a01581e0 tx=0x7ff294009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:57.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.003+0000 7ff287fff700 1 -- 172.21.15.92:0/1148031409 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff290051770 con 0x7ff2a00b8510 2023-11-28T11:23:57.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.192+0000 7ff2a5dff700 1 -- 172.21.15.92:0/1148031409 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff2a0067880 con 0x7ff2a00b8510 2023-11-28T11:23:57.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.193+0000 7ff287fff700 1 -- 172.21.15.92:0/1148031409 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v56) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff290054d90 con 0x7ff2a00b8510 2023-11-28T11:23:57.194 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-11-28T11:23:57.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.195+0000 7ff2a5dff700 1 -- 172.21.15.92:0/1148031409 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff28c0682f0 msgr2=0x7ff28c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:57.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.195+0000 7ff2a5dff700 1 --2- 172.21.15.92:0/1148031409 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff28c0682f0 0x7ff28c06a7a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff2a01581e0 tx=0x7ff294009210 comp rx=0 tx=0).stop 2023-11-28T11:23:57.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.195+0000 7ff2a5dff700 1 -- 172.21.15.92:0/1148031409 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2a00b8510 msgr2=0x7ff2a01f2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:57.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.195+0000 7ff2a5dff700 1 --2- 172.21.15.92:0/1148031409 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2a00b8510 0x7ff2a01f2b40 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7ff29000c4e0 tx=0x7ff29000c7f0 comp rx=0 tx=0).stop 2023-11-28T11:23:57.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.195+0000 7ff2a5dff700 1 -- 172.21.15.92:0/1148031409 shutdown_connections 2023-11-28T11:23:57.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.195+0000 7ff2a5dff700 1 --2- 172.21.15.92:0/1148031409 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff2a00d0d60 0x7ff2a01f55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:57.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.195+0000 7ff2a5dff700 1 --2- 172.21.15.92:0/1148031409 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff28c0682f0 0x7ff28c06a7a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:57.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.195+0000 7ff2a5dff700 1 --2- 172.21.15.92:0/1148031409 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2a00b8510 0x7ff2a01f2b40 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:57.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.195+0000 7ff2a5dff700 1 --2- 172.21.15.92:0/1148031409 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff2a00cd460 0x7ff2a015a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:57.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.195+0000 7ff2a5dff700 1 -- 172.21.15.92:0/1148031409 >> 172.21.15.92:0/1148031409 conn(0x7ff2a00719a0 msgr2=0x7ff2a00b7780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:57.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.196+0000 7ff2a5dff700 1 -- 172.21.15.92:0/1148031409 shutdown_connections 2023-11-28T11:23:57.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.196+0000 7ff2a5dff700 1 -- 172.21.15.92:0/1148031409 wait complete. 2023-11-28T11:23:57.215 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T11:23:57.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.411+0000 7f4efadff700 1 Processor -- start 2023-11-28T11:23:57.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.411+0000 7f4efadff700 1 -- start start 2023-11-28T11:23:57.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.412+0000 7f4efadff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4ef40b8510 0x7f4ef406b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:57.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.413+0000 7f4efadff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ef40b88e0 con 0x7f4ef40d0d60 2023-11-28T11:23:57.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.413+0000 7f4efadff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ef40cd8f0 con 0x7f4ef40cd460 2023-11-28T11:23:57.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.413+0000 7f4efadff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ef40cdbd0 con 0x7f4ef40b8510 2023-11-28T11:23:57.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.413+0000 7f4ef8b9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4ef40b8510 0x7f4ef406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:57.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.413+0000 7f4ef939c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f4ef40d0d60 0x7f4ef40d1200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36464/0 (socket says 172.21.15.92:36464) 2023-11-28T11:23:57.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.413+0000 7f4ef939c700 1 -- 172.21.15.92:0/1149449692 learned_addr learned my addr 172.21.15.92:0/1149449692 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:23:57.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.413+0000 7f4ef8b9b700 1 -- 172.21.15.92:0/1149449692 >> v1:172.21.15.169:6789/0 conn(0x7f4ef40cd460 legacy=0x7f4ef406beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:57.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.413+0000 7f4ef8b9b700 1 -- 172.21.15.92:0/1149449692 >> v1:172.21.15.92:6789/0 conn(0x7f4ef40d0d60 legacy=0x7f4ef40d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:57.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.413+0000 7f4ef37fe700 1 -- 172.21.15.92:0/1149449692 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 739970279 0 0) 0x7f4ef40b88e0 con 0x7f4ef40d0d60 2023-11-28T11:23:57.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.413+0000 7f4ef8b9b700 1 -- 172.21.15.92:0/1149449692 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef40cfd10 con 0x7f4ef40b8510 2023-11-28T11:23:57.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.414+0000 7f4ef8b9b700 1 --2- 172.21.15.92:0/1149449692 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ef40b8510 0x7f4ef406b970 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8009840 tx=0x7f4ee8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=600435a9e8aeea3a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:57.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.414+0000 7f4ef37fe700 1 -- 172.21.15.92:0/1149449692 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee8004230 con 0x7f4ef40b8510 2023-11-28T11:23:57.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.414+0000 7f4ef37fe700 1 -- 172.21.15.92:0/1149449692 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ee8004390 con 0x7f4ef40b8510 2023-11-28T11:23:57.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.414+0000 7f4ef37fe700 1 -- 172.21.15.92:0/1149449692 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee80036a0 con 0x7f4ef40b8510 2023-11-28T11:23:57.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.414+0000 7f4efadff700 1 -- 172.21.15.92:0/1149449692 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ef40b8510 msgr2=0x7f4ef406b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:57.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.414+0000 7f4efadff700 1 --2- 172.21.15.92:0/1149449692 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ef40b8510 0x7f4ef406b970 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8009840 tx=0x7f4ee8009b50 comp rx=0 tx=0).stop 2023-11-28T11:23:57.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.414+0000 7f4efadff700 1 -- 172.21.15.92:0/1149449692 shutdown_connections 2023-11-28T11:23:57.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.414+0000 7f4efadff700 1 --2- 172.21.15.92:0/1149449692 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ef40b8510 0x7f4ef406b970 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:57.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.414+0000 7f4efadff700 1 -- 172.21.15.92:0/1149449692 >> 172.21.15.92:0/1149449692 conn(0x7f4ef40719a0 msgr2=0x7f4ef4071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:57.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.415+0000 7f4efadff700 1 -- 172.21.15.92:0/1149449692 shutdown_connections 2023-11-28T11:23:57.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.415+0000 7f4efadff700 1 -- 172.21.15.92:0/1149449692 wait complete. 2023-11-28T11:23:57.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.416+0000 7f4efadff700 1 Processor -- start 2023-11-28T11:23:57.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.416+0000 7f4efadff700 1 -- start start 2023-11-28T11:23:57.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.417+0000 7f4efadff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ef40b8510 0x7f4ef4155c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:57.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.417+0000 7f4efadff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ef40cd460 0x7f4ef40cb810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:57.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.417+0000 7f4efadff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ef40d0d60 0x7f4ef4159360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:57.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.417+0000 7f4efadff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef40ccea0 con 0x7f4ef40d0d60 2023-11-28T11:23:57.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.417+0000 7f4efadff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef40cc2d0 con 0x7f4ef40cd460 2023-11-28T11:23:57.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.417+0000 7f4efadff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef40cc570 con 0x7f4ef40b8510 2023-11-28T11:23:57.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.418+0000 7f4ef939c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ef40d0d60 0x7f4ef4159360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:57.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.418+0000 7f4ef939c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ef40d0d60 0x7f4ef4159360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52244/0 (socket says 172.21.15.92:52244) 2023-11-28T11:23:57.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.418+0000 7f4ef939c700 1 -- 172.21.15.92:0/2394522413 learned_addr learned my addr 172.21.15.92:0/2394522413 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:57.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.418+0000 7f4ef8b9b700 1 --2- 172.21.15.92:0/2394522413 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ef40b8510 0x7f4ef4155c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:57.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.418+0000 7f4ef3fff700 1 --2- 172.21.15.92:0/2394522413 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ef40cd460 0x7f4ef40cb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:57.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.418+0000 7f4ef939c700 1 -- 172.21.15.92:0/2394522413 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ef40b8510 msgr2=0x7f4ef4155c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:57.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.418+0000 7f4ef939c700 1 --2- 172.21.15.92:0/2394522413 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ef40b8510 0x7f4ef4155c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:57.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.418+0000 7f4ef939c700 1 -- 172.21.15.92:0/2394522413 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ef40cd460 msgr2=0x7f4ef40cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:57.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.418+0000 7f4ef939c700 1 --2- 172.21.15.92:0/2394522413 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ef40cd460 0x7f4ef40cb810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:57.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.418+0000 7f4ef939c700 1 -- 172.21.15.92:0/2394522413 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee80094a0 con 0x7f4ef40d0d60 2023-11-28T11:23:57.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.418+0000 7f4ef3fff700 1 --2- 172.21.15.92:0/2394522413 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ef40cd460 0x7f4ef40cb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:23:57.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.418+0000 7f4ef939c700 1 --2- 172.21.15.92:0/2394522413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ef40d0d60 0x7f4ef4159360 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f4eec00cb60 tx=0x7f4eec00ce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:57.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.418+0000 7f4ef17fa700 1 -- 172.21.15.92:0/2394522413 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eec00aaa0 con 0x7f4ef40d0d60 2023-11-28T11:23:57.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.418+0000 7f4ef17fa700 1 -- 172.21.15.92:0/2394522413 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eec00ac00 con 0x7f4ef40d0d60 2023-11-28T11:23:57.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.418+0000 7f4ef17fa700 1 -- 172.21.15.92:0/2394522413 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eec0070d0 con 0x7f4ef40d0d60 2023-11-28T11:23:57.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.419+0000 7f4efadff700 1 -- 172.21.15.92:0/2394522413 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ef40d29c0 con 0x7f4ef40d0d60 2023-11-28T11:23:57.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.419+0000 7f4efadff700 1 -- 172.21.15.92:0/2394522413 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ef40cfff0 con 0x7f4ef40d0d60 2023-11-28T11:23:57.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.420+0000 7f4efadff700 1 -- 172.21.15.92:0/2394522413 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ef40d2b50 con 0x7f4ef40d0d60 2023-11-28T11:23:57.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.420+0000 7f4ef17fa700 1 -- 172.21.15.92:0/2394522413 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4eec010070 con 0x7f4ef40d0d60 2023-11-28T11:23:57.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.422+0000 7f4ef17fa700 1 --2- 172.21.15.92:0/2394522413 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f4ed80682f0 0x7f4ed806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:57.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.422+0000 7f4ef17fa700 1 -- 172.21.15.92:0/2394522413 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f4eec085840 con 0x7f4ef40d0d60 2023-11-28T11:23:57.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.422+0000 7f4ef8b9b700 1 --2- 172.21.15.92:0/2394522413 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f4ed80682f0 0x7f4ed806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:57.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.422+0000 7f4ef8b9b700 1 --2- 172.21.15.92:0/2394522413 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f4ed80682f0 0x7f4ed806a7a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8009b90 tx=0x7f4ee801c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:57.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.425+0000 7f4ef17fa700 1 -- 172.21.15.92:0/2394522413 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4eec051e70 con 0x7f4ef40d0d60 2023-11-28T11:23:57.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.636+0000 7f4efadff700 1 -- 172.21.15.92:0/2394522413 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4ef4067880 con 0x7f4ef40d0d60 2023-11-28T11:23:57.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.636+0000 7f4ef17fa700 1 -- 172.21.15.92:0/2394522413 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v56) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4eec018080 con 0x7f4ef40d0d60 2023-11-28T11:23:57.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.638+0000 7f4efadff700 1 -- 172.21.15.92:0/2394522413 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f4ed80682f0 msgr2=0x7f4ed806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:57.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.638+0000 7f4efadff700 1 --2- 172.21.15.92:0/2394522413 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f4ed80682f0 0x7f4ed806a7a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8009b90 tx=0x7f4ee801c040 comp rx=0 tx=0).stop 2023-11-28T11:23:57.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.638+0000 7f4efadff700 1 -- 172.21.15.92:0/2394522413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ef40d0d60 msgr2=0x7f4ef4159360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:57.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.638+0000 7f4efadff700 1 --2- 172.21.15.92:0/2394522413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ef40d0d60 0x7f4ef4159360 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f4eec00cb60 tx=0x7f4eec00ce70 comp rx=0 tx=0).stop 2023-11-28T11:23:57.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.639+0000 7f4efadff700 1 -- 172.21.15.92:0/2394522413 shutdown_connections 2023-11-28T11:23:57.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.639+0000 7f4efadff700 1 --2- 172.21.15.92:0/2394522413 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f4ed80682f0 0x7f4ed806a7a0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:57.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.639+0000 7f4efadff700 1 --2- 172.21.15.92:0/2394522413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ef40d0d60 0x7f4ef4159360 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:57.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.639+0000 7f4efadff700 1 --2- 172.21.15.92:0/2394522413 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4ef40b8510 0x7f4ef4155c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:57.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.639+0000 7f4efadff700 1 --2- 172.21.15.92:0/2394522413 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4ef40cd460 0x7f4ef40cb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:57.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.639+0000 7f4efadff700 1 -- 172.21.15.92:0/2394522413 >> 172.21.15.92:0/2394522413 conn(0x7f4ef40719a0 msgr2=0x7f4ef40b5d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:57.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.639+0000 7f4efadff700 1 -- 172.21.15.92:0/2394522413 shutdown_connections 2023-11-28T11:23:57.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:57.639+0000 7f4efadff700 1 -- 172.21.15.92:0/2394522413 wait complete. 2023-11-28T11:23:57.641 INFO:teuthology.orchestra.run.smithi092.stderr:listed 0 entries 2023-11-28T11:23:57.657 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-28T11:23:57.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:23:57.658 DEBUG:teuthology.orchestra.run.smithi092:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-28T11:23:57.759 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2023-11-28T11:23:57.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:23:57.760 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T11:23:57.820 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-28T11:23:57.821 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:23:57.870 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348296k,nr_inodes=4087074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275620k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:23:57.870 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-28T11:23:57.870 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:23:57.917 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348288k,nr_inodes=4087072,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275616k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:23:57.918 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-28T11:23:57.918 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:23:57.972 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348296k,nr_inodes=4087074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275620k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:23:57.972 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:23:58.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.167+0000 7f8d4cbff700 1 Processor -- start 2023-11-28T11:23:58.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.167+0000 7f8d4cbff700 1 -- start start 2023-11-28T11:23:58.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.168+0000 7f8d4cbff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8d4815d820 0x7f8d480cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:58.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.168+0000 7f8d4cbff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d480cfcf0 0x7f8d480b19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:58.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.168+0000 7f8d4cbff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d480d0170 con 0x7f8d4815a110 2023-11-28T11:23:58.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.168+0000 7f8d4cbff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d480cdae0 con 0x7f8d4815d820 2023-11-28T11:23:58.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.168+0000 7f8d4cbff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d480ce4f0 con 0x7f8d480cfcf0 2023-11-28T11:23:58.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.168+0000 7f8d46d9d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d480cfcf0 0x7f8d480b19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:58.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.169+0000 7f8d45d9b700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8d4815d820 0x7f8d480cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:58.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.169+0000 7f8d46d9d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d480cfcf0 0x7f8d480b19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.92:48070/0 (socket says 172.21.15.92:48070) 2023-11-28T11:23:58.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.169+0000 7f8d46d9d700 1 -- 172.21.15.92:0/3953192291 learned_addr learned my addr 172.21.15.92:0/3953192291 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:58.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.169+0000 7f8d4659c700 1 --1- 172.21.15.92:0/3953192291 >> v1:172.21.15.169:6789/0 conn(0x7f8d4815a110 0x7f8d48160b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.92:60752/0 (socket says 172.21.15.92:60752) 2023-11-28T11:23:58.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.169+0000 7f8d46d9d700 1 -- 172.21.15.92:0/3953192291 >> v2:172.21.15.169:3300/0 conn(0x7f8d4815d820 msgr2=0x7f8d480cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:58.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.169+0000 7f8d46d9d700 1 --2- 172.21.15.92:0/3953192291 >> v2:172.21.15.169:3300/0 conn(0x7f8d4815d820 0x7f8d480cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:58.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.169+0000 7f8d46d9d700 1 -- 172.21.15.92:0/3953192291 >> v1:172.21.15.169:6789/0 conn(0x7f8d4815a110 legacy=0x7f8d48160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:58.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.169+0000 7f8d46d9d700 1 -- 172.21.15.92:0/3953192291 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d480cf2d0 con 0x7f8d480cfcf0 2023-11-28T11:23:58.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.169+0000 7f8d4559a700 1 -- 172.21.15.92:0/3953192291 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 317424387 0 0) 0x7f8d480d0170 con 0x7f8d4815a110 2023-11-28T11:23:58.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.169+0000 7f8d45d9b700 1 --2- 172.21.15.92:0/3953192291 >> v2:172.21.15.169:3300/0 conn(0x7f8d4815d820 0x7f8d480cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:23:58.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.169+0000 7f8d46d9d700 1 --2- 172.21.15.92:0/3953192291 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d480cfcf0 0x7f8d480b19b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8d40009e50 tx=0x7f8d4000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f6645718919db3a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:58.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.170+0000 7f8d4559a700 1 -- 172.21.15.92:0/3953192291 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4000e070 con 0x7f8d480cfcf0 2023-11-28T11:23:58.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.170+0000 7f8d4559a700 1 -- 172.21.15.92:0/3953192291 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d4000b910 con 0x7f8d480cfcf0 2023-11-28T11:23:58.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.170+0000 7f8d4559a700 1 -- 172.21.15.92:0/3953192291 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4000bcc0 con 0x7f8d480cfcf0 2023-11-28T11:23:58.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.170+0000 7f8d4cbff700 1 -- 172.21.15.92:0/3953192291 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d480cfcf0 msgr2=0x7f8d480b19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:58.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.170+0000 7f8d4cbff700 1 --2- 172.21.15.92:0/3953192291 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d480cfcf0 0x7f8d480b19b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8d40009e50 tx=0x7f8d4000b330 comp rx=0 tx=0).stop 2023-11-28T11:23:58.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.170+0000 7f8d4cbff700 1 -- 172.21.15.92:0/3953192291 shutdown_connections 2023-11-28T11:23:58.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.170+0000 7f8d4cbff700 1 --2- 172.21.15.92:0/3953192291 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d480cfcf0 0x7f8d480b19b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:58.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.170+0000 7f8d4cbff700 1 --2- 172.21.15.92:0/3953192291 >> v2:172.21.15.169:3300/0 conn(0x7f8d4815d820 0x7f8d480cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:58.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.170+0000 7f8d4cbff700 1 -- 172.21.15.92:0/3953192291 >> 172.21.15.92:0/3953192291 conn(0x7f8d480719a0 msgr2=0x7f8d48071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:58.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.170+0000 7f8d4cbff700 1 -- 172.21.15.92:0/3953192291 shutdown_connections 2023-11-28T11:23:58.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.170+0000 7f8d4cbff700 1 -- 172.21.15.92:0/3953192291 wait complete. 2023-11-28T11:23:58.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.172+0000 7f8d4cbff700 1 Processor -- start 2023-11-28T11:23:58.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.172+0000 7f8d4cbff700 1 -- start start 2023-11-28T11:23:58.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.173+0000 7f8d4cbff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d480cfcf0 0x7f8d480b6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:58.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.173+0000 7f8d4cbff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d4815a110 0x7f8d480b71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:58.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.173+0000 7f8d4cbff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d4815d820 0x7f8d481f1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:58.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.173+0000 7f8d4cbff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d480b8430 con 0x7f8d4815d820 2023-11-28T11:23:58.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.173+0000 7f8d4cbff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d480b7820 con 0x7f8d4815a110 2023-11-28T11:23:58.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.173+0000 7f8d4cbff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d480b7b00 con 0x7f8d480cfcf0 2023-11-28T11:23:58.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.173+0000 7f8d46d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d4815d820 0x7f8d481f1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:58.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.173+0000 7f8d45d9b700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d4815a110 0x7f8d480b71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:58.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.173+0000 7f8d4659c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d480cfcf0 0x7f8d480b6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:58.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.173+0000 7f8d46d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d4815d820 0x7f8d481f1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52248/0 (socket says 172.21.15.92:52248) 2023-11-28T11:23:58.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.173+0000 7f8d46d9d700 1 -- 172.21.15.92:0/3019140229 learned_addr learned my addr 172.21.15.92:0/3019140229 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:58.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.173+0000 7f8d45d9b700 1 --2- 172.21.15.92:0/3019140229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d4815a110 0x7f8d480b71f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.92:40084/0 (socket says 172.21.15.92:40084) 2023-11-28T11:23:58.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.174+0000 7f8d45d9b700 1 -- 172.21.15.92:0/3019140229 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d480cfcf0 msgr2=0x7f8d480b6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:58.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.174+0000 7f8d45d9b700 1 --2- 172.21.15.92:0/3019140229 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d480cfcf0 0x7f8d480b6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:58.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.174+0000 7f8d45d9b700 1 -- 172.21.15.92:0/3019140229 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d4815d820 msgr2=0x7f8d481f1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:58.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.174+0000 7f8d45d9b700 1 --2- 172.21.15.92:0/3019140229 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d4815d820 0x7f8d481f1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:58.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.174+0000 7f8d45d9b700 1 -- 172.21.15.92:0/3019140229 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d3c0094a0 con 0x7f8d4815a110 2023-11-28T11:23:58.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.174+0000 7f8d46d9d700 1 --2- 172.21.15.92:0/3019140229 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d4815d820 0x7f8d481f1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:23:58.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.174+0000 7f8d45d9b700 1 --2- 172.21.15.92:0/3019140229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d4815a110 0x7f8d480b71f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c00e9b0 tx=0x7f8d3c00ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:58.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.174+0000 7f8d36ffd700 1 -- 172.21.15.92:0/3019140229 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c00ee50 con 0x7f8d4815a110 2023-11-28T11:23:58.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.174+0000 7f8d36ffd700 1 -- 172.21.15.92:0/3019140229 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d3c00c850 con 0x7f8d4815a110 2023-11-28T11:23:58.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.174+0000 7f8d36ffd700 1 -- 172.21.15.92:0/3019140229 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c00cc00 con 0x7f8d4815a110 2023-11-28T11:23:58.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.174+0000 7f8d4cbff700 1 -- 172.21.15.92:0/3019140229 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d40009b30 con 0x7f8d4815a110 2023-11-28T11:23:58.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.175+0000 7f8d4cbff700 1 -- 172.21.15.92:0/3019140229 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d4815cd40 con 0x7f8d4815a110 2023-11-28T11:23:58.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.176+0000 7f8d4cbff700 1 -- 172.21.15.92:0/3019140229 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d480cd800 con 0x7f8d4815a110 2023-11-28T11:23:58.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.176+0000 7f8d36ffd700 1 -- 172.21.15.92:0/3019140229 <== mon.1 v2:172.21.15.169:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8d3c019030 con 0x7f8d4815a110 2023-11-28T11:23:58.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.177+0000 7f8d36ffd700 1 --2- 172.21.15.92:0/3019140229 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8d280682f0 0x7f8d2806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:58.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.177+0000 7f8d36ffd700 1 -- 172.21.15.92:0/3019140229 <== mon.1 v2:172.21.15.169:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f8d3c085d10 con 0x7f8d4815a110 2023-11-28T11:23:58.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.177+0000 7f8d4659c700 1 --2- 172.21.15.92:0/3019140229 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8d280682f0 0x7f8d2806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:58.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.178+0000 7f8d4659c700 1 --2- 172.21.15.92:0/3019140229 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8d280682f0 0x7f8d2806a7a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8d38007fd0 tx=0x7f8d38007de0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:58.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.182+0000 7f8d36ffd700 1 -- 172.21.15.92:0/3019140229 <== mon.1 v2:172.21.15.169:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8d3c08b020 con 0x7f8d4815a110 2023-11-28T11:23:58.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.411+0000 7f8d4cbff700 1 -- 172.21.15.92:0/3019140229 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8d4815d020 con 0x7f8d4815a110 2023-11-28T11:23:58.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.412+0000 7f8d36ffd700 1 -- 172.21.15.92:0/3019140229 <== mon.1 v2:172.21.15.169:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4651 (secure 0 0 0) 0x7f8d3c052340 con 0x7f8d4815a110 2023-11-28T11:23:58.413 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:23:58.413 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5593,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/3937189339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3937189339},{"type":"v1","addr":"172.21.15.169:6834","nonce":3937189339}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5596,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2020201016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2020201016},{"type":"v1","addr":"172.21.15.169:6835","nonce":2020201016}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5751,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2423750324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2423750324},{"type":"v1","addr":"172.21.15.92:6835","nonce":2423750324}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:23:13.247233+0000","modified":"2023-11-28T11:23:16.254256+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5754},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5754":{"gid":5754,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/950691230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":950691230},{"type":"v1","addr":"172.21.15.92:6837","nonce":950691230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T11:23:58.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.414+0000 7f8d4cbff700 1 -- 172.21.15.92:0/3019140229 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8d280682f0 msgr2=0x7f8d2806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:58.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.414+0000 7f8d4cbff700 1 --2- 172.21.15.92:0/3019140229 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8d280682f0 0x7f8d2806a7a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8d38007fd0 tx=0x7f8d38007de0 comp rx=0 tx=0).stop 2023-11-28T11:23:58.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.414+0000 7f8d4cbff700 1 -- 172.21.15.92:0/3019140229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d4815a110 msgr2=0x7f8d480b71f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:58.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.414+0000 7f8d4cbff700 1 --2- 172.21.15.92:0/3019140229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d4815a110 0x7f8d480b71f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c00e9b0 tx=0x7f8d3c00ecc0 comp rx=0 tx=0).stop 2023-11-28T11:23:58.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.414+0000 7f8d4cbff700 1 -- 172.21.15.92:0/3019140229 shutdown_connections 2023-11-28T11:23:58.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.414+0000 7f8d4cbff700 1 --2- 172.21.15.92:0/3019140229 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8d280682f0 0x7f8d2806a7a0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:58.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.414+0000 7f8d4cbff700 1 --2- 172.21.15.92:0/3019140229 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d4815d820 0x7f8d481f1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:58.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.414+0000 7f8d4cbff700 1 --2- 172.21.15.92:0/3019140229 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d480cfcf0 0x7f8d480b6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:58.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.414+0000 7f8d4cbff700 1 --2- 172.21.15.92:0/3019140229 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d4815a110 0x7f8d480b71f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:58.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.414+0000 7f8d4cbff700 1 -- 172.21.15.92:0/3019140229 >> 172.21.15.92:0/3019140229 conn(0x7f8d480719a0 msgr2=0x7f8d481592b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:58.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.415+0000 7f8d4cbff700 1 -- 172.21.15.92:0/3019140229 shutdown_connections 2023-11-28T11:23:58.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.415+0000 7f8d4cbff700 1 -- 172.21.15.92:0/3019140229 wait complete. 2023-11-28T11:23:58.417 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 24 2023-11-28T11:23:58.434 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:23:58.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.629+0000 7f431c7ff700 1 Processor -- start 2023-11-28T11:23:58.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.629+0000 7f431c7ff700 1 -- start start 2023-11-28T11:23:58.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.630+0000 7f431c7ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f43140b8510 0x7f431406b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:58.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.630+0000 7f431c7ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43140d0d60 0x7f43140d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:58.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.630+0000 7f431c7ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43140b88e0 con 0x7f43140cd460 2023-11-28T11:23:58.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.630+0000 7f431c7ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43140cdbd0 con 0x7f43140b8510 2023-11-28T11:23:58.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.630+0000 7f431c7ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43140d02d0 con 0x7f43140d0d60 2023-11-28T11:23:58.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.630+0000 7f431a59b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f43140b8510 0x7f431406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:58.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.630+0000 7f431a59b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f43140b8510 0x7f431406b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52262/0 (socket says 172.21.15.92:52262) 2023-11-28T11:23:58.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.630+0000 7f431a59b700 1 -- 172.21.15.92:0/2489600889 learned_addr learned my addr 172.21.15.92:0/2489600889 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:58.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.631+0000 7f431ad9c700 1 --2- 172.21.15.92:0/2489600889 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43140d0d60 0x7f43140d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:58.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.631+0000 7f431a59b700 1 -- 172.21.15.92:0/2489600889 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43140d0d60 msgr2=0x7f43140d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:58.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.631+0000 7f431a59b700 1 --2- 172.21.15.92:0/2489600889 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43140d0d60 0x7f43140d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:58.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.631+0000 7f4319599700 1 -- 172.21.15.92:0/2489600889 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1675944076 0 0) 0x7f43140b88e0 con 0x7f43140cd460 2023-11-28T11:23:58.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.631+0000 7f431a59b700 1 -- 172.21.15.92:0/2489600889 >> v1:172.21.15.169:6789/0 conn(0x7f43140cd460 legacy=0x7f431406beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:58.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.631+0000 7f431a59b700 1 -- 172.21.15.92:0/2489600889 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43140cd8f0 con 0x7f43140b8510 2023-11-28T11:23:58.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.631+0000 7f431a59b700 1 --2- 172.21.15.92:0/2489600889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43140b8510 0x7f431406b970 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f4310009920 tx=0x7f4310009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdea2160b4c61fa1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:58.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.631+0000 7f4319599700 1 -- 172.21.15.92:0/2489600889 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4310011830 con 0x7f43140b8510 2023-11-28T11:23:58.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.631+0000 7f4319599700 1 -- 172.21.15.92:0/2489600889 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4310011990 con 0x7f43140b8510 2023-11-28T11:23:58.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.631+0000 7f4319599700 1 -- 172.21.15.92:0/2489600889 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4310011d20 con 0x7f43140b8510 2023-11-28T11:23:58.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.632+0000 7f431c7ff700 1 -- 172.21.15.92:0/2489600889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43140b8510 msgr2=0x7f431406b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:58.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.632+0000 7f431c7ff700 1 --2- 172.21.15.92:0/2489600889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43140b8510 0x7f431406b970 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f4310009920 tx=0x7f4310009c30 comp rx=0 tx=0).stop 2023-11-28T11:23:58.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.632+0000 7f431c7ff700 1 -- 172.21.15.92:0/2489600889 shutdown_connections 2023-11-28T11:23:58.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.632+0000 7f431c7ff700 1 --2- 172.21.15.92:0/2489600889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43140b8510 0x7f431406b970 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:58.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.632+0000 7f431c7ff700 1 --2- 172.21.15.92:0/2489600889 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43140d0d60 0x7f43140d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:58.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.632+0000 7f431c7ff700 1 -- 172.21.15.92:0/2489600889 >> 172.21.15.92:0/2489600889 conn(0x7f43140719a0 msgr2=0x7f4314071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:58.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.632+0000 7f431c7ff700 1 -- 172.21.15.92:0/2489600889 shutdown_connections 2023-11-28T11:23:58.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.632+0000 7f431c7ff700 1 -- 172.21.15.92:0/2489600889 wait complete. 2023-11-28T11:23:58.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.634+0000 7f431c7ff700 1 Processor -- start 2023-11-28T11:23:58.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.634+0000 7f431c7ff700 1 -- start start 2023-11-28T11:23:58.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.635+0000 7f431c7ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43140b8510 0x7f4314156c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:58.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.635+0000 7f431c7ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43140cd460 0x7f43141599e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:58.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.635+0000 7f431c7ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43140d0d60 0x7f43141f2a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:58.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.635+0000 7f431c7ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43141f4150 con 0x7f43140d0d60 2023-11-28T11:23:58.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.635+0000 7f431c7ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43141f3540 con 0x7f43140cd460 2023-11-28T11:23:58.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.635+0000 7f431c7ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43141f3820 con 0x7f43140b8510 2023-11-28T11:23:58.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.635+0000 7f431ad9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43140d0d60 0x7f43141f2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:58.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.635+0000 7f431ad9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43140d0d60 0x7f43141f2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52268/0 (socket says 172.21.15.92:52268) 2023-11-28T11:23:58.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.635+0000 7f431ad9c700 1 -- 172.21.15.92:0/1150635327 learned_addr learned my addr 172.21.15.92:0/1150635327 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:58.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.635+0000 7f431a59b700 1 --2- 172.21.15.92:0/1150635327 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43140b8510 0x7f4314156c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:58.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.635+0000 7f4319d9a700 1 --2- 172.21.15.92:0/1150635327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43140cd460 0x7f43141599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:58.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.635+0000 7f431ad9c700 1 -- 172.21.15.92:0/1150635327 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43140b8510 msgr2=0x7f4314156c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:58.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.635+0000 7f431ad9c700 1 --2- 172.21.15.92:0/1150635327 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43140b8510 0x7f4314156c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:58.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.635+0000 7f431ad9c700 1 -- 172.21.15.92:0/1150635327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43140cd460 msgr2=0x7f43141599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:58.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.635+0000 7f431ad9c700 1 --2- 172.21.15.92:0/1150635327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43140cd460 0x7f43141599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:58.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.635+0000 7f431ad9c700 1 -- 172.21.15.92:0/1150635327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4310009550 con 0x7f43140d0d60 2023-11-28T11:23:58.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.636+0000 7f431ad9c700 1 --2- 172.21.15.92:0/1150635327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43140d0d60 0x7f43141f2a80 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f430c00efc0 tx=0x7f430c00c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:58.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.636+0000 7f430affd700 1 -- 172.21.15.92:0/1150635327 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430c00cad0 con 0x7f43140d0d60 2023-11-28T11:23:58.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.636+0000 7f430affd700 1 -- 172.21.15.92:0/1150635327 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f430c00f040 con 0x7f43140d0d60 2023-11-28T11:23:58.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.636+0000 7f431c7ff700 1 -- 172.21.15.92:0/1150635327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43140cfff0 con 0x7f43140d0d60 2023-11-28T11:23:58.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.636+0000 7f430affd700 1 -- 172.21.15.92:0/1150635327 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430c00ce70 con 0x7f43140d0d60 2023-11-28T11:23:58.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.636+0000 7f431c7ff700 1 -- 172.21.15.92:0/1150635327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43140cfd10 con 0x7f43140d0d60 2023-11-28T11:23:58.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.637+0000 7f431c7ff700 1 -- 172.21.15.92:0/1150635327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43141f42c0 con 0x7f43140d0d60 2023-11-28T11:23:58.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.638+0000 7f430affd700 1 -- 172.21.15.92:0/1150635327 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f430c014480 con 0x7f43140d0d60 2023-11-28T11:23:58.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.639+0000 7f430affd700 1 --2- 172.21.15.92:0/1150635327 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f42f80682f0 0x7f42f806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:58.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.639+0000 7f430affd700 1 -- 172.21.15.92:0/1150635327 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f430c086800 con 0x7f43140d0d60 2023-11-28T11:23:58.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.640+0000 7f431a59b700 1 --2- 172.21.15.92:0/1150635327 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f42f80682f0 0x7f42f806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:58.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.640+0000 7f431a59b700 1 --2- 172.21.15.92:0/1150635327 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f42f80682f0 0x7f42f806a7a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4310000c00 tx=0x7f4310008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:58.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.644+0000 7f430affd700 1 -- 172.21.15.92:0/1150635327 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f430c052dd0 con 0x7f43140d0d60 2023-11-28T11:23:58.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.835+0000 7f431c7ff700 1 -- 172.21.15.92:0/1150635327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4314067880 con 0x7f43140d0d60 2023-11-28T11:23:58.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.836+0000 7f430affd700 1 -- 172.21.15.92:0/1150635327 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+18180 (secure 0 0 0) 0x7f430c0563f0 con 0x7f43140d0d60 2023-11-28T11:23:58.838 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:23:58.838 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":56,"fsid":"68956e0c-c16f-47cd-a220-f98364b0831d","created":"2023-11-28T11:20:28.697573+0000","modified":"2023-11-28T11:23:14.249429+0000","last_up_change":"2023-11-28T11:20:41.257011+0000","last_in_change":"2023-11-28T11:20:32.249767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:20:41.925237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:20:48.682497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:23:10.939884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-28T11:23:12.227433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6abe483f-0e03-4a26-b9cf-62041291747e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":115823},{"type":"v1","addr":"172.21.15.92:6809","nonce":115823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":115823},{"type":"v1","addr":"172.21.15.92:6813","nonce":115823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":115823},{"type":"v1","addr":"172.21.15.92:6821","nonce":115823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":115823},{"type":"v1","addr":"172.21.15.92:6817","nonce":115823}]},"public_addr":"172.21.15.92:6809/115823","cluster_addr":"172.21.15.92:6813/115823","heartbeat_back_addr":"172.21.15.92:6821/115823","heartbeat_front_addr":"172.21.15.92:6817/115823","state":["exists","up"]},{"osd":1,"uuid":"13c1de73-1a8c-4d7e-8a28-9dae52dfe089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":115824},{"type":"v1","addr":"172.21.15.92:6812","nonce":115824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":115824},{"type":"v1","addr":"172.21.15.92:6816","nonce":115824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":115824},{"type":"v1","addr":"172.21.15.92:6823","nonce":115824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":115824},{"type":"v1","addr":"172.21.15.92:6820","nonce":115824}]},"public_addr":"172.21.15.92:6812/115824","cluster_addr":"172.21.15.92:6816/115824","heartbeat_back_addr":"172.21.15.92:6823/115824","heartbeat_front_addr":"172.21.15.92:6820/115824","state":["exists","up"]},{"osd":2,"uuid":"25788a55-112f-4f00-810d-2bc37cf76778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":115821},{"type":"v1","addr":"172.21.15.92:6801","nonce":115821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":115821},{"type":"v1","addr":"172.21.15.92:6803","nonce":115821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":115821},{"type":"v1","addr":"172.21.15.92:6807","nonce":115821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":115821},{"type":"v1","addr":"172.21.15.92:6805","nonce":115821}]},"public_addr":"172.21.15.92:6801/115821","cluster_addr":"172.21.15.92:6803/115821","heartbeat_back_addr":"172.21.15.92:6807/115821","heartbeat_front_addr":"172.21.15.92:6805/115821","state":["exists","up"]},{"osd":3,"uuid":"adf54e24-33a9-41fe-9ce3-98a5ab6c158e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":115825},{"type":"v1","addr":"172.21.15.92:6825","nonce":115825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":115825},{"type":"v1","addr":"172.21.15.92:6827","nonce":115825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":115825},{"type":"v1","addr":"172.21.15.92:6831","nonce":115825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":115825},{"type":"v1","addr":"172.21.15.92:6829","nonce":115825}]},"public_addr":"172.21.15.92:6825/115825","cluster_addr":"172.21.15.92:6827/115825","heartbeat_back_addr":"172.21.15.92:6831/115825","heartbeat_front_addr":"172.21.15.92:6829/115825","state":["exists","up"]},{"osd":4,"uuid":"ccb9b747-f399-44ea-9a18-9db3152227e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":116035},{"type":"v1","addr":"172.21.15.169:6817","nonce":116035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":116035},{"type":"v1","addr":"172.21.15.169:6819","nonce":116035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":116035},{"type":"v1","addr":"172.21.15.169:6826","nonce":116035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":116035},{"type":"v1","addr":"172.21.15.169:6822","nonce":116035}]},"public_addr":"172.21.15.169:6817/116035","cluster_addr":"172.21.15.169:6819/116035","heartbeat_back_addr":"172.21.15.169:6826/116035","heartbeat_front_addr":"172.21.15.169:6822/116035","state":["exists","up"]},{"osd":5,"uuid":"308a1e63-75aa-4d7d-8a7c-cecc24714fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":116033},{"type":"v1","addr":"172.21.15.169:6801","nonce":116033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":116033},{"type":"v1","addr":"172.21.15.169:6803","nonce":116033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":116033},{"type":"v1","addr":"172.21.15.169:6807","nonce":116033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":116033},{"type":"v1","addr":"172.21.15.169:6805","nonce":116033}]},"public_addr":"172.21.15.169:6801/116033","cluster_addr":"172.21.15.169:6803/116033","heartbeat_back_addr":"172.21.15.169:6807/116033","heartbeat_front_addr":"172.21.15.169:6805/116033","state":["exists","up"]},{"osd":6,"uuid":"9c459b7b-b821-4b4b-b91e-6aaae271457c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":116034},{"type":"v1","addr":"172.21.15.169:6809","nonce":116034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":116034},{"type":"v1","addr":"172.21.15.169:6811","nonce":116034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":116034},{"type":"v1","addr":"172.21.15.169:6815","nonce":116034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":116034},{"type":"v1","addr":"172.21.15.169:6813","nonce":116034}]},"public_addr":"172.21.15.169:6809/116034","cluster_addr":"172.21.15.169:6811/116034","heartbeat_back_addr":"172.21.15.169:6815/116034","heartbeat_front_addr":"172.21.15.169:6813/116034","state":["exists","up"]},{"osd":7,"uuid":"1ca9aa4a-755d-4e28-a937-4d8dccab4df3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":116044},{"type":"v1","addr":"172.21.15.169:6823","nonce":116044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6825","nonce":116044},{"type":"v1","addr":"172.21.15.169:6827","nonce":116044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":116044},{"type":"v1","addr":"172.21.15.169:6831","nonce":116044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":116044},{"type":"v1","addr":"172.21.15.169:6829","nonce":116044}]},"public_addr":"172.21.15.169:6823/116044","cluster_addr":"172.21.15.169:6827/116044","heartbeat_back_addr":"172.21.15.169:6831/116044","heartbeat_front_addr":"172.21.15.169:6829/116044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.586781+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.869778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.508594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.860493+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.872267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.746933+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.611520+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.871446+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:23:58.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.839+0000 7f431c7ff700 1 -- 172.21.15.92:0/1150635327 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f42f80682f0 msgr2=0x7f42f806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:58.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.839+0000 7f431c7ff700 1 --2- 172.21.15.92:0/1150635327 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f42f80682f0 0x7f42f806a7a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4310000c00 tx=0x7f4310008030 comp rx=0 tx=0).stop 2023-11-28T11:23:58.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.839+0000 7f431c7ff700 1 -- 172.21.15.92:0/1150635327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43140d0d60 msgr2=0x7f43141f2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:58.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.839+0000 7f431c7ff700 1 --2- 172.21.15.92:0/1150635327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43140d0d60 0x7f43141f2a80 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f430c00efc0 tx=0x7f430c00c0c0 comp rx=0 tx=0).stop 2023-11-28T11:23:58.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.839+0000 7f431c7ff700 1 -- 172.21.15.92:0/1150635327 shutdown_connections 2023-11-28T11:23:58.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.839+0000 7f431c7ff700 1 --2- 172.21.15.92:0/1150635327 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f42f80682f0 0x7f42f806a7a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:58.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.839+0000 7f431c7ff700 1 --2- 172.21.15.92:0/1150635327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43140d0d60 0x7f43141f2a80 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:58.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.839+0000 7f431c7ff700 1 --2- 172.21.15.92:0/1150635327 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43140b8510 0x7f4314156c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:58.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.839+0000 7f431c7ff700 1 --2- 172.21.15.92:0/1150635327 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43140cd460 0x7f43141599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:58.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.839+0000 7f431c7ff700 1 -- 172.21.15.92:0/1150635327 >> 172.21.15.92:0/1150635327 conn(0x7f43140719a0 msgr2=0x7f43140b3e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:58.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.839+0000 7f431c7ff700 1 -- 172.21.15.92:0/1150635327 shutdown_connections 2023-11-28T11:23:58.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:58.839+0000 7f431c7ff700 1 -- 172.21.15.92:0/1150635327 wait complete. 2023-11-28T11:23:58.858 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:20:41.925237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:20:48.682497+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:23:10.939884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:23:12.227433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:23:58.859 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:23:59.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.055+0000 7feffd7ff700 1 Processor -- start 2023-11-28T11:23:59.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.055+0000 7feffd7ff700 1 -- start start 2023-11-28T11:23:59.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.057+0000 7feffd7ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7feff80cd460 0x7feff806c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:59.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.057+0000 7feffd7ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feff80b88e0 con 0x7feff80b8510 2023-11-28T11:23:59.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.057+0000 7feffd7ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feff80cd8f0 con 0x7feff80d0d60 2023-11-28T11:23:59.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.057+0000 7feffd7ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feff80cdbd0 con 0x7feff80cd460 2023-11-28T11:23:59.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.057+0000 7feff6ffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7feff80b8510 0x7feff806b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55134/0 (socket says 172.21.15.92:55134) 2023-11-28T11:23:59.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.057+0000 7feff6ffd700 1 -- 172.21.15.92:0/4086386974 learned_addr learned my addr 172.21.15.92:0/4086386974 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:23:59.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.058+0000 7fefeffff700 1 --2- 172.21.15.92:0/4086386974 >> v2:172.21.15.169:3300/0 conn(0x7feff80cd460 0x7feff806c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:59.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.058+0000 7feff67fc700 1 -- 172.21.15.92:0/4086386974 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1102091060 0 0) 0x7feff80b88e0 con 0x7feff80b8510 2023-11-28T11:23:59.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.058+0000 7feff67fc700 1 -- 172.21.15.92:0/4086386974 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fefe40035b0 con 0x7feff80b8510 2023-11-28T11:23:59.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.058+0000 7feff67fc700 1 -- 172.21.15.92:0/4086386974 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4135286027 0 0) 0x7feff80cd8f0 con 0x7feff80d0d60 2023-11-28T11:23:59.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.058+0000 7feff67fc700 1 -- 172.21.15.92:0/4086386974 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feff80b88e0 con 0x7feff80d0d60 2023-11-28T11:23:59.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.058+0000 7fefeffff700 1 -- 172.21.15.92:0/4086386974 >> v1:172.21.15.169:6789/0 conn(0x7feff80d0d60 legacy=0x7feff80d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:59.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.058+0000 7fefeffff700 1 -- 172.21.15.92:0/4086386974 >> v1:172.21.15.92:6789/0 conn(0x7feff80b8510 legacy=0x7feff806b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:59.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.058+0000 7fefeffff700 1 -- 172.21.15.92:0/4086386974 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff80cfd10 con 0x7feff80cd460 2023-11-28T11:23:59.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.058+0000 7fefeffff700 1 --2- 172.21.15.92:0/4086386974 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feff80cd460 0x7feff806c080 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fefe0009840 tx=0x7fefe0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=195777369549b4e6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:59.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.059+0000 7feff67fc700 1 -- 172.21.15.92:0/4086386974 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe0011830 con 0x7feff80cd460 2023-11-28T11:23:59.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.059+0000 7feff67fc700 1 -- 172.21.15.92:0/4086386974 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefe0011990 con 0x7feff80cd460 2023-11-28T11:23:59.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.059+0000 7feff67fc700 1 -- 172.21.15.92:0/4086386974 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe0011d20 con 0x7feff80cd460 2023-11-28T11:23:59.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.059+0000 7feffd7ff700 1 -- 172.21.15.92:0/4086386974 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feff80cd460 msgr2=0x7feff806c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:59.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.059+0000 7feffd7ff700 1 --2- 172.21.15.92:0/4086386974 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feff80cd460 0x7feff806c080 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fefe0009840 tx=0x7fefe0009b50 comp rx=0 tx=0).stop 2023-11-28T11:23:59.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.059+0000 7feffd7ff700 1 -- 172.21.15.92:0/4086386974 shutdown_connections 2023-11-28T11:23:59.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.059+0000 7feffd7ff700 1 --2- 172.21.15.92:0/4086386974 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feff80cd460 0x7feff806c080 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:59.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.059+0000 7feffd7ff700 1 -- 172.21.15.92:0/4086386974 >> 172.21.15.92:0/4086386974 conn(0x7feff80719a0 msgr2=0x7feff8071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:59.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.059+0000 7feffd7ff700 1 -- 172.21.15.92:0/4086386974 shutdown_connections 2023-11-28T11:23:59.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.059+0000 7feffd7ff700 1 -- 172.21.15.92:0/4086386974 wait complete. 2023-11-28T11:23:59.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.061+0000 7feffd7ff700 1 Processor -- start 2023-11-28T11:23:59.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.061+0000 7feffd7ff700 1 -- start start 2023-11-28T11:23:59.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.062+0000 7feffd7ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feff80b8510 0x7feff80c97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:59.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.062+0000 7feffd7ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feff80cd460 0x7feff81e74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:59.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.062+0000 7feffd7ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feff80d0d60 0x7feff81e7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:59.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.062+0000 7feffd7ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff80c9d30 con 0x7feff80cd460 2023-11-28T11:23:59.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.062+0000 7feffd7ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff81e4490 con 0x7feff80d0d60 2023-11-28T11:23:59.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.062+0000 7feffd7ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feff81e41b0 con 0x7feff80b8510 2023-11-28T11:23:59.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.063+0000 7fefeffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feff80cd460 0x7feff81e74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:59.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.063+0000 7fefeffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feff80cd460 0x7feff81e74e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50108/0 (socket says 172.21.15.92:50108) 2023-11-28T11:23:59.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.063+0000 7fefeffff700 1 -- 172.21.15.92:0/3798419763 learned_addr learned my addr 172.21.15.92:0/3798419763 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:59.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.063+0000 7feff77fe700 1 --2- 172.21.15.92:0/3798419763 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feff80d0d60 0x7feff81e7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:59.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.063+0000 7feff6ffd700 1 --2- 172.21.15.92:0/3798419763 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feff80b8510 0x7feff80c97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:59.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.063+0000 7fefeffff700 1 -- 172.21.15.92:0/3798419763 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feff80b8510 msgr2=0x7feff80c97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:59.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.063+0000 7fefeffff700 1 --2- 172.21.15.92:0/3798419763 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feff80b8510 0x7feff80c97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:59.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.063+0000 7fefeffff700 1 -- 172.21.15.92:0/3798419763 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feff80d0d60 msgr2=0x7feff81e7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:59.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.063+0000 7fefeffff700 1 --2- 172.21.15.92:0/3798419763 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feff80d0d60 0x7feff81e7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:59.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.063+0000 7fefeffff700 1 -- 172.21.15.92:0/3798419763 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefe00094a0 con 0x7feff80cd460 2023-11-28T11:23:59.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.063+0000 7feff77fe700 1 --2- 172.21.15.92:0/3798419763 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feff80d0d60 0x7feff81e7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:23:59.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.063+0000 7fefeffff700 1 --2- 172.21.15.92:0/3798419763 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feff80cd460 0x7feff81e74e0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fefe00090b0 tx=0x7fefe0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:59.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.063+0000 7fefef7fe700 1 -- 172.21.15.92:0/3798419763 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe001a860 con 0x7feff80cd460 2023-11-28T11:23:59.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.064+0000 7fefef7fe700 1 -- 172.21.15.92:0/3798419763 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefe001a9c0 con 0x7feff80cd460 2023-11-28T11:23:59.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.064+0000 7feffd7ff700 1 -- 172.21.15.92:0/3798419763 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feff80d29c0 con 0x7feff80cd460 2023-11-28T11:23:59.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.064+0000 7fefef7fe700 1 -- 172.21.15.92:0/3798419763 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe001ad30 con 0x7feff80cd460 2023-11-28T11:23:59.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.064+0000 7feffd7ff700 1 -- 172.21.15.92:0/3798419763 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feff80cfff0 con 0x7feff80cd460 2023-11-28T11:23:59.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.065+0000 7feffd7ff700 1 -- 172.21.15.92:0/3798419763 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feff80d2b50 con 0x7feff80cd460 2023-11-28T11:23:59.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.066+0000 7fefef7fe700 1 -- 172.21.15.92:0/3798419763 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fefe000f050 con 0x7feff80cd460 2023-11-28T11:23:59.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.066+0000 7fefef7fe700 1 --2- 172.21.15.92:0/3798419763 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fefd80682f0 0x7fefd806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:59.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.067+0000 7fefef7fe700 1 -- 172.21.15.92:0/3798419763 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fefe0086b60 con 0x7feff80cd460 2023-11-28T11:23:59.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.067+0000 7feff6ffd700 1 --2- 172.21.15.92:0/3798419763 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fefd80682f0 0x7fefd806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:59.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.067+0000 7feff6ffd700 1 --2- 172.21.15.92:0/3798419763 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fefd80682f0 0x7fefd806a7a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fefe8000f60 tx=0x7fefe8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:59.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.071+0000 7fefef7fe700 1 -- 172.21.15.92:0/3798419763 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fefe0053330 con 0x7feff80cd460 2023-11-28T11:23:59.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.260+0000 7feffd7ff700 1 -- 172.21.15.92:0/3798419763 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7feff8067880 con 0x7feff80cd460 2023-11-28T11:23:59.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.260+0000 7fefef7fe700 1 -- 172.21.15.92:0/3798419763 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v56) v1 ==== 93+0+10 (secure 0 0 0) 0x7fefe00290a0 con 0x7feff80cd460 2023-11-28T11:23:59.261 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-11-28T11:23:59.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.262+0000 7feffd7ff700 1 -- 172.21.15.92:0/3798419763 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fefd80682f0 msgr2=0x7fefd806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:59.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.262+0000 7feffd7ff700 1 --2- 172.21.15.92:0/3798419763 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fefd80682f0 0x7fefd806a7a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fefe8000f60 tx=0x7fefe8009040 comp rx=0 tx=0).stop 2023-11-28T11:23:59.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.262+0000 7feffd7ff700 1 -- 172.21.15.92:0/3798419763 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feff80cd460 msgr2=0x7feff81e74e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:59.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.262+0000 7feffd7ff700 1 --2- 172.21.15.92:0/3798419763 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feff80cd460 0x7feff81e74e0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fefe00090b0 tx=0x7fefe0004750 comp rx=0 tx=0).stop 2023-11-28T11:23:59.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.262+0000 7feffd7ff700 1 -- 172.21.15.92:0/3798419763 shutdown_connections 2023-11-28T11:23:59.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.262+0000 7feffd7ff700 1 --2- 172.21.15.92:0/3798419763 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7feff80d0d60 0x7feff81e7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:59.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.262+0000 7feffd7ff700 1 --2- 172.21.15.92:0/3798419763 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7feff80b8510 0x7feff80c97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:59.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.262+0000 7feffd7ff700 1 --2- 172.21.15.92:0/3798419763 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fefd80682f0 0x7fefd806a7a0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:59.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.262+0000 7feffd7ff700 1 --2- 172.21.15.92:0/3798419763 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feff80cd460 0x7feff81e74e0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:59.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.262+0000 7feffd7ff700 1 -- 172.21.15.92:0/3798419763 >> 172.21.15.92:0/3798419763 conn(0x7feff80719a0 msgr2=0x7feff80b3280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:59.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.263+0000 7feffd7ff700 1 -- 172.21.15.92:0/3798419763 shutdown_connections 2023-11-28T11:23:59.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.263+0000 7feffd7ff700 1 -- 172.21.15.92:0/3798419763 wait complete. 2023-11-28T11:23:59.281 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:23:59.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.479+0000 7fd609dff700 1 Processor -- start 2023-11-28T11:23:59.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.479+0000 7fd609dff700 1 -- start start 2023-11-28T11:23:59.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.480+0000 7fd609dff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6040b8510 0x7fd60406b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:59.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.480+0000 7fd609dff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd6040d0d60 0x7fd6040d11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:59.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.481+0000 7fd609dff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6040b88e0 con 0x7fd6040cd460 2023-11-28T11:23:59.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.481+0000 7fd609dff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6040cdbd0 con 0x7fd6040d0d60 2023-11-28T11:23:59.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.481+0000 7fd609dff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6040d02d0 con 0x7fd6040b8510 2023-11-28T11:23:59.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.481+0000 7fd603fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd6040d0d60 0x7fd6040d11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:59.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.481+0000 7fd5fbfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd6040cd460 0x7fd60406beb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55136/0 (socket says 172.21.15.92:55136) 2023-11-28T11:23:59.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.481+0000 7fd5fbfff700 1 -- 172.21.15.92:0/471428713 learned_addr learned my addr 172.21.15.92:0/471428713 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:23:59.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.481+0000 7fd6037fe700 1 --2- 172.21.15.92:0/471428713 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6040b8510 0x7fd60406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:59.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.481+0000 7fd603fff700 1 -- 172.21.15.92:0/471428713 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6040b8510 msgr2=0x7fd60406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:23:59.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.481+0000 7fd603fff700 1 --2- 172.21.15.92:0/471428713 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6040b8510 0x7fd60406b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:59.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.481+0000 7fd603fff700 1 -- 172.21.15.92:0/471428713 >> v1:172.21.15.92:6789/0 conn(0x7fd6040cd460 legacy=0x7fd60406beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:59.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.481+0000 7fd603fff700 1 -- 172.21.15.92:0/471428713 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6040cd8f0 con 0x7fd6040d0d60 2023-11-28T11:23:59.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.481+0000 7fd602ffd700 1 -- 172.21.15.92:0/471428713 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1099666400 0 0) 0x7fd6040b88e0 con 0x7fd6040cd460 2023-11-28T11:23:59.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.481+0000 7fd6037fe700 1 --2- 172.21.15.92:0/471428713 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6040b8510 0x7fd60406b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:23:59.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.482+0000 7fd603fff700 1 --2- 172.21.15.92:0/471428713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6040d0d60 0x7fd6040d11a0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc009dc0 tx=0x7fd5fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec8cd53971d5d3e6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:59.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.482+0000 7fd602ffd700 1 -- 172.21.15.92:0/471428713 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc00e070 con 0x7fd6040d0d60 2023-11-28T11:23:59.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.482+0000 7fd602ffd700 1 -- 172.21.15.92:0/471428713 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5fc00bb90 con 0x7fd6040d0d60 2023-11-28T11:23:59.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.482+0000 7fd602ffd700 1 -- 172.21.15.92:0/471428713 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc012430 con 0x7fd6040d0d60 2023-11-28T11:23:59.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.482+0000 7fd609dff700 1 -- 172.21.15.92:0/471428713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6040d0d60 msgr2=0x7fd6040d11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:59.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.482+0000 7fd609dff700 1 --2- 172.21.15.92:0/471428713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6040d0d60 0x7fd6040d11a0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc009dc0 tx=0x7fd5fc00b330 comp rx=0 tx=0).stop 2023-11-28T11:23:59.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.482+0000 7fd609dff700 1 -- 172.21.15.92:0/471428713 shutdown_connections 2023-11-28T11:23:59.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.482+0000 7fd609dff700 1 --2- 172.21.15.92:0/471428713 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6040b8510 0x7fd60406b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:59.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.482+0000 7fd609dff700 1 --2- 172.21.15.92:0/471428713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6040d0d60 0x7fd6040d11a0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:59.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.482+0000 7fd609dff700 1 -- 172.21.15.92:0/471428713 >> 172.21.15.92:0/471428713 conn(0x7fd6040719a0 msgr2=0x7fd604071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:59.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.483+0000 7fd609dff700 1 -- 172.21.15.92:0/471428713 shutdown_connections 2023-11-28T11:23:59.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.483+0000 7fd609dff700 1 -- 172.21.15.92:0/471428713 wait complete. 2023-11-28T11:23:59.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.484+0000 7fd609dff700 1 Processor -- start 2023-11-28T11:23:59.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.484+0000 7fd609dff700 1 -- start start 2023-11-28T11:23:59.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.485+0000 7fd609dff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6040b8510 0x7fd6041f59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:59.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.485+0000 7fd609dff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6040cd460 0x7fd6040b1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:59.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.485+0000 7fd609dff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6040d0d60 0x7fd60415af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:59.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.485+0000 7fd609dff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6040a9aa0 con 0x7fd6040b8510 2023-11-28T11:23:59.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.485+0000 7fd609dff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6040cbff0 con 0x7fd6040d0d60 2023-11-28T11:23:59.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.485+0000 7fd609dff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6040b1970 con 0x7fd6040cd460 2023-11-28T11:23:59.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.485+0000 7fd6037fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6040b8510 0x7fd6041f59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:59.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.485+0000 7fd6037fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6040b8510 0x7fd6041f59c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50126/0 (socket says 172.21.15.92:50126) 2023-11-28T11:23:59.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.486+0000 7fd6037fe700 1 -- 172.21.15.92:0/96041740 learned_addr learned my addr 172.21.15.92:0/96041740 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:59.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.486+0000 7fd603fff700 1 --2- 172.21.15.92:0/96041740 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6040d0d60 0x7fd60415af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:59.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.486+0000 7fd5fbfff700 1 --2- 172.21.15.92:0/96041740 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6040cd460 0x7fd6040b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:59.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.486+0000 7fd6037fe700 1 -- 172.21.15.92:0/96041740 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6040cd460 msgr2=0x7fd6040b1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:59.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.486+0000 7fd6037fe700 1 --2- 172.21.15.92:0/96041740 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6040cd460 0x7fd6040b1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:59.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.486+0000 7fd6037fe700 1 -- 172.21.15.92:0/96041740 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6040d0d60 msgr2=0x7fd60415af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:59.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.486+0000 7fd6037fe700 1 --2- 172.21.15.92:0/96041740 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6040d0d60 0x7fd60415af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:59.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.486+0000 7fd6037fe700 1 -- 172.21.15.92:0/96041740 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5fc009a70 con 0x7fd6040b8510 2023-11-28T11:23:59.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.486+0000 7fd5fbfff700 1 --2- 172.21.15.92:0/96041740 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6040cd460 0x7fd6040b1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:23:59.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.486+0000 7fd6037fe700 1 --2- 172.21.15.92:0/96041740 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6040b8510 0x7fd6041f59c0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fd5f400e810 tx=0x7fd5f400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:59.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.486+0000 7fd600ff9700 1 -- 172.21.15.92:0/96041740 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f4004230 con 0x7fd6040b8510 2023-11-28T11:23:59.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.486+0000 7fd600ff9700 1 -- 172.21.15.92:0/96041740 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5f4004390 con 0x7fd6040b8510 2023-11-28T11:23:59.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.486+0000 7fd609dff700 1 -- 172.21.15.92:0/96041740 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6040cfff0 con 0x7fd6040b8510 2023-11-28T11:23:59.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.486+0000 7fd600ff9700 1 -- 172.21.15.92:0/96041740 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f40036a0 con 0x7fd6040b8510 2023-11-28T11:23:59.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.487+0000 7fd609dff700 1 -- 172.21.15.92:0/96041740 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6040cfd10 con 0x7fd6040b8510 2023-11-28T11:23:59.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.488+0000 7fd609dff700 1 -- 172.21.15.92:0/96041740 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd604067880 con 0x7fd6040b8510 2023-11-28T11:23:59.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.488+0000 7fd600ff9700 1 -- 172.21.15.92:0/96041740 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd5f4022070 con 0x7fd6040b8510 2023-11-28T11:23:59.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.489+0000 7fd600ff9700 1 --2- 172.21.15.92:0/96041740 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd5e40682f0 0x7fd5e406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:59.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.489+0000 7fd600ff9700 1 -- 172.21.15.92:0/96041740 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fd5f4085970 con 0x7fd6040b8510 2023-11-28T11:23:59.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.489+0000 7fd5fbfff700 1 --2- 172.21.15.92:0/96041740 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd5e40682f0 0x7fd5e406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:59.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.490+0000 7fd5fbfff700 1 --2- 172.21.15.92:0/96041740 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd5e40682f0 0x7fd5e406a7a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd6040b3970 tx=0x7fd5ec009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:59.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.493+0000 7fd600ff9700 1 -- 172.21.15.92:0/96041740 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd5f4051f40 con 0x7fd6040b8510 2023-11-28T11:23:59.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.682+0000 7fd609dff700 1 -- 172.21.15.92:0/96041740 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd6040d2940 con 0x7fd6040b8510 2023-11-28T11:23:59.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.683+0000 7fd600ff9700 1 -- 172.21.15.92:0/96041740 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v56) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd5f4055560 con 0x7fd6040b8510 2023-11-28T11:23:59.684 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-11-28T11:23:59.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.685+0000 7fd609dff700 1 -- 172.21.15.92:0/96041740 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd5e40682f0 msgr2=0x7fd5e406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:59.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.685+0000 7fd609dff700 1 --2- 172.21.15.92:0/96041740 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd5e40682f0 0x7fd5e406a7a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd6040b3970 tx=0x7fd5ec009040 comp rx=0 tx=0).stop 2023-11-28T11:23:59.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.685+0000 7fd609dff700 1 -- 172.21.15.92:0/96041740 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6040b8510 msgr2=0x7fd6041f59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:59.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.685+0000 7fd609dff700 1 --2- 172.21.15.92:0/96041740 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6040b8510 0x7fd6041f59c0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fd5f400e810 tx=0x7fd5f400ebd0 comp rx=0 tx=0).stop 2023-11-28T11:23:59.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.685+0000 7fd609dff700 1 -- 172.21.15.92:0/96041740 shutdown_connections 2023-11-28T11:23:59.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.685+0000 7fd609dff700 1 --2- 172.21.15.92:0/96041740 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd6040d0d60 0x7fd60415af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:59.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.685+0000 7fd609dff700 1 --2- 172.21.15.92:0/96041740 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd5e40682f0 0x7fd5e406a7a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:59.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.685+0000 7fd609dff700 1 --2- 172.21.15.92:0/96041740 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6040b8510 0x7fd6041f59c0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:59.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.685+0000 7fd609dff700 1 --2- 172.21.15.92:0/96041740 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd6040cd460 0x7fd6040b1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:59.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.686+0000 7fd609dff700 1 -- 172.21.15.92:0/96041740 >> 172.21.15.92:0/96041740 conn(0x7fd6040719a0 msgr2=0x7fd6040d36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:59.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.686+0000 7fd609dff700 1 -- 172.21.15.92:0/96041740 shutdown_connections 2023-11-28T11:23:59.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.686+0000 7fd609dff700 1 -- 172.21.15.92:0/96041740 wait complete. 2023-11-28T11:23:59.704 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:23:59.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.902+0000 7f20af5ff700 1 Processor -- start 2023-11-28T11:23:59.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.902+0000 7f20af5ff700 1 -- start start 2023-11-28T11:23:59.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.903+0000 7f20af5ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20a80b8510 0x7f20a806b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:59.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.904+0000 7f20af5ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20a80b88e0 con 0x7f20a80d0d60 2023-11-28T11:23:59.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.904+0000 7f20af5ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20a80cd8f0 con 0x7f20a80cd460 2023-11-28T11:23:59.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.904+0000 7f20af5ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20a80cdbd0 con 0x7f20a80b8510 2023-11-28T11:23:59.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.904+0000 7f20adb9c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f20a80d0d60 0x7f20a80d11a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55144/0 (socket says 172.21.15.92:55144) 2023-11-28T11:23:59.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.904+0000 7f20adb9c700 1 -- 172.21.15.92:0/2764818874 learned_addr learned my addr 172.21.15.92:0/2764818874 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:23:59.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.904+0000 7f20ad39b700 1 --2- 172.21.15.92:0/2764818874 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20a80b8510 0x7f20a806b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:59.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.904+0000 7f209ffff700 1 -- 172.21.15.92:0/2764818874 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1946040173 0 0) 0x7f20a80b88e0 con 0x7f20a80d0d60 2023-11-28T11:23:59.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.904+0000 7f209ffff700 1 -- 172.21.15.92:0/2764818874 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20940035b0 con 0x7f20a80d0d60 2023-11-28T11:23:59.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.905+0000 7f209ffff700 1 -- 172.21.15.92:0/2764818874 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3188088614 0 0) 0x7f20a80cd8f0 con 0x7f20a80cd460 2023-11-28T11:23:59.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.905+0000 7f209ffff700 1 -- 172.21.15.92:0/2764818874 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20a80b88e0 con 0x7f20a80cd460 2023-11-28T11:23:59.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.905+0000 7f209ffff700 1 -- 172.21.15.92:0/2764818874 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3596133269 0 0) 0x7f20940035b0 con 0x7f20a80d0d60 2023-11-28T11:23:59.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.905+0000 7f20ad39b700 1 -- 172.21.15.92:0/2764818874 >> v1:172.21.15.169:6789/0 conn(0x7f20a80cd460 legacy=0x7f20a806beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:59.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.905+0000 7f20ad39b700 1 -- 172.21.15.92:0/2764818874 >> v1:172.21.15.92:6789/0 conn(0x7f20a80d0d60 legacy=0x7f20a80d11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:59.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.905+0000 7f20ad39b700 1 -- 172.21.15.92:0/2764818874 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20a80cfd10 con 0x7f20a80b8510 2023-11-28T11:23:59.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.905+0000 7f20ad39b700 1 --2- 172.21.15.92:0/2764818874 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20a80b8510 0x7f20a806b970 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f20a4009870 tx=0x7f20a4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f37f553e8b7aa6d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:59.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.905+0000 7f209ffff700 1 -- 172.21.15.92:0/2764818874 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a0001970 con 0x7f20a80b8510 2023-11-28T11:23:59.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.905+0000 7f209ffff700 1 -- 172.21.15.92:0/2764818874 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20a4004030 con 0x7f20a80b8510 2023-11-28T11:23:59.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.905+0000 7f209ffff700 1 -- 172.21.15.92:0/2764818874 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a40043c0 con 0x7f20a80b8510 2023-11-28T11:23:59.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.906+0000 7f20af5ff700 1 -- 172.21.15.92:0/2764818874 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20a80b8510 msgr2=0x7f20a806b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:59.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.906+0000 7f20af5ff700 1 --2- 172.21.15.92:0/2764818874 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20a80b8510 0x7f20a806b970 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f20a4009870 tx=0x7f20a4009b80 comp rx=0 tx=0).stop 2023-11-28T11:23:59.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.906+0000 7f20af5ff700 1 -- 172.21.15.92:0/2764818874 shutdown_connections 2023-11-28T11:23:59.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.906+0000 7f20af5ff700 1 --2- 172.21.15.92:0/2764818874 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20a80b8510 0x7f20a806b970 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:59.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.906+0000 7f20af5ff700 1 -- 172.21.15.92:0/2764818874 >> 172.21.15.92:0/2764818874 conn(0x7f20a80719a0 msgr2=0x7f20a8071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:23:59.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.906+0000 7f20af5ff700 1 -- 172.21.15.92:0/2764818874 shutdown_connections 2023-11-28T11:23:59.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.906+0000 7f20af5ff700 1 -- 172.21.15.92:0/2764818874 wait complete. 2023-11-28T11:23:59.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.908+0000 7f20af5ff700 1 Processor -- start 2023-11-28T11:23:59.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.908+0000 7f20af5ff700 1 -- start start 2023-11-28T11:23:59.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.910+0000 7f20af5ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20a80b8510 0x7f20a80c7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:59.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.910+0000 7f20af5ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20a80cd460 0x7f20a81e4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:59.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.910+0000 7f20af5ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20a80d0d60 0x7f20a814dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:59.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.910+0000 7f20af5ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a81e5a20 con 0x7f20a80b8510 2023-11-28T11:23:59.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.910+0000 7f20af5ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a80c8290 con 0x7f20a80cd460 2023-11-28T11:23:59.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.910+0000 7f20af5ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20a80c8570 con 0x7f20a80d0d60 2023-11-28T11:23:59.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.910+0000 7f20ad39b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20a80b8510 0x7f20a80c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:59.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.910+0000 7f20ad39b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20a80b8510 0x7f20a80c7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50138/0 (socket says 172.21.15.92:50138) 2023-11-28T11:23:59.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.910+0000 7f20ad39b700 1 -- 172.21.15.92:0/1692261001 learned_addr learned my addr 172.21.15.92:0/1692261001 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:23:59.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.910+0000 7f20ad39b700 1 -- 172.21.15.92:0/1692261001 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20a80d0d60 msgr2=0x7f20a814dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:59.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.910+0000 7f20ad39b700 1 --2- 172.21.15.92:0/1692261001 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20a80d0d60 0x7f20a814dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:59.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.910+0000 7f20ad39b700 1 -- 172.21.15.92:0/1692261001 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20a80cd460 msgr2=0x7f20a81e4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:23:59.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.910+0000 7f20acb9a700 1 --2- 172.21.15.92:0/1692261001 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20a80cd460 0x7f20a81e4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:59.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.910+0000 7f20ad39b700 1 --2- 172.21.15.92:0/1692261001 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20a80cd460 0x7f20a81e4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:23:59.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.911+0000 7f20ad39b700 1 -- 172.21.15.92:0/1692261001 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20a40094a0 con 0x7f20a80b8510 2023-11-28T11:23:59.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.911+0000 7f20acb9a700 1 --2- 172.21.15.92:0/1692261001 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20a80cd460 0x7f20a81e4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:23:59.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.911+0000 7f20ad39b700 1 --2- 172.21.15.92:0/1692261001 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20a80b8510 0x7f20a80c7990 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f20a4000c00 tx=0x7f20a4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:59.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.911+0000 7f209dffb700 1 -- 172.21.15.92:0/1692261001 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a4011630 con 0x7f20a80b8510 2023-11-28T11:23:59.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.911+0000 7f209dffb700 1 -- 172.21.15.92:0/1692261001 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20a401f070 con 0x7f20a80b8510 2023-11-28T11:23:59.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.911+0000 7f20af5ff700 1 -- 172.21.15.92:0/1692261001 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20a80d29c0 con 0x7f20a80b8510 2023-11-28T11:23:59.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.911+0000 7f209dffb700 1 -- 172.21.15.92:0/1692261001 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a40119f0 con 0x7f20a80b8510 2023-11-28T11:23:59.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.911+0000 7f20af5ff700 1 -- 172.21.15.92:0/1692261001 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20a80cfff0 con 0x7f20a80b8510 2023-11-28T11:23:59.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.912+0000 7f20af5ff700 1 -- 172.21.15.92:0/1692261001 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20a80d2b50 con 0x7f20a80b8510 2023-11-28T11:23:59.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.913+0000 7f209dffb700 1 -- 172.21.15.92:0/1692261001 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f20a4017020 con 0x7f20a80b8510 2023-11-28T11:23:59.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.914+0000 7f209dffb700 1 --2- 172.21.15.92:0/1692261001 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f208c0682f0 0x7f208c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:23:59.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.914+0000 7f209dffb700 1 -- 172.21.15.92:0/1692261001 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f20a4021070 con 0x7f20a80b8510 2023-11-28T11:23:59.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.914+0000 7f20acb9a700 1 --2- 172.21.15.92:0/1692261001 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f208c0682f0 0x7f208c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:23:59.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.915+0000 7f20acb9a700 1 --2- 172.21.15.92:0/1692261001 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f208c0682f0 0x7f208c06a7a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f20a806c5e0 tx=0x7f2098009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:23:59.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:23:59.918+0000 7f209dffb700 1 -- 172.21.15.92:0/1692261001 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f20a4053e40 con 0x7f20a80b8510 2023-11-28T11:24:00.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.108+0000 7f20af5ff700 1 -- 172.21.15.92:0/1692261001 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f20a8067880 con 0x7f20a80b8510 2023-11-28T11:24:00.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.108+0000 7f209dffb700 1 -- 172.21.15.92:0/1692261001 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v56) v1 ==== 104+0+11 (secure 0 0 0) 0x7f20a4057460 con 0x7f20a80b8510 2023-11-28T11:24:00.109 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-11-28T11:24:00.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.111+0000 7f20af5ff700 1 -- 172.21.15.92:0/1692261001 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f208c0682f0 msgr2=0x7f208c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:00.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.111+0000 7f20af5ff700 1 --2- 172.21.15.92:0/1692261001 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f208c0682f0 0x7f208c06a7a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f20a806c5e0 tx=0x7f2098009040 comp rx=0 tx=0).stop 2023-11-28T11:24:00.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.111+0000 7f20af5ff700 1 -- 172.21.15.92:0/1692261001 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20a80b8510 msgr2=0x7f20a80c7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:00.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.111+0000 7f20af5ff700 1 --2- 172.21.15.92:0/1692261001 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20a80b8510 0x7f20a80c7990 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f20a4000c00 tx=0x7f20a4004750 comp rx=0 tx=0).stop 2023-11-28T11:24:00.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.111+0000 7f20af5ff700 1 -- 172.21.15.92:0/1692261001 shutdown_connections 2023-11-28T11:24:00.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.111+0000 7f20af5ff700 1 --2- 172.21.15.92:0/1692261001 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f20a80d0d60 0x7f20a814dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:00.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.111+0000 7f20af5ff700 1 --2- 172.21.15.92:0/1692261001 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f208c0682f0 0x7f208c06a7a0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:00.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.111+0000 7f20af5ff700 1 --2- 172.21.15.92:0/1692261001 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20a80b8510 0x7f20a80c7990 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:00.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.111+0000 7f20af5ff700 1 --2- 172.21.15.92:0/1692261001 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f20a80cd460 0x7f20a81e4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:00.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.111+0000 7f20af5ff700 1 -- 172.21.15.92:0/1692261001 >> 172.21.15.92:0/1692261001 conn(0x7f20a80719a0 msgr2=0x7f20a80d1160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:00.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.111+0000 7f20af5ff700 1 -- 172.21.15.92:0/1692261001 shutdown_connections 2023-11-28T11:24:00.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.111+0000 7f20af5ff700 1 -- 172.21.15.92:0/1692261001 wait complete. 2023-11-28T11:24:00.129 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:24:00.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.328+0000 7f44db9ff700 1 Processor -- start 2023-11-28T11:24:00.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.329+0000 7f44db9ff700 1 -- start start 2023-11-28T11:24:00.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.330+0000 7f44db9ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44d40b8510 0x7f44d406b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:00.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.330+0000 7f44db9ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f44d40d0d60 0x7f44d40d11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:00.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.330+0000 7f44db9ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44d40b88e0 con 0x7f44d40cd460 2023-11-28T11:24:00.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.330+0000 7f44db9ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44d40cdbd0 con 0x7f44d40d0d60 2023-11-28T11:24:00.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.330+0000 7f44db9ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44d40d02d0 con 0x7f44d40b8510 2023-11-28T11:24:00.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.330+0000 7f44d8f9a700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f44d40cd460 0x7f44d406beb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55152/0 (socket says 172.21.15.92:55152) 2023-11-28T11:24:00.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.330+0000 7f44d8f9a700 1 -- 172.21.15.92:0/1710943846 learned_addr learned my addr 172.21.15.92:0/1710943846 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:24:00.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.330+0000 7f44d979b700 1 --2- 172.21.15.92:0/1710943846 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44d40b8510 0x7f44d406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:00.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.330+0000 7f44d9f9c700 1 --2- 172.21.15.92:0/1710943846 >> v2:172.21.15.169:3300/0 conn(0x7f44d40d0d60 0x7f44d40d11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:00.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.331+0000 7f44cbfff700 1 -- 172.21.15.92:0/1710943846 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3108817294 0 0) 0x7f44d40b88e0 con 0x7f44d40cd460 2023-11-28T11:24:00.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.331+0000 7f44cbfff700 1 -- 172.21.15.92:0/1710943846 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44c00035b0 con 0x7f44d40cd460 2023-11-28T11:24:00.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.331+0000 7f44d979b700 1 -- 172.21.15.92:0/1710943846 >> v2:172.21.15.169:3300/0 conn(0x7f44d40d0d60 msgr2=0x7f44d40d11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:00.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.331+0000 7f44d979b700 1 --2- 172.21.15.92:0/1710943846 >> v2:172.21.15.169:3300/0 conn(0x7f44d40d0d60 0x7f44d40d11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:00.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.331+0000 7f44d979b700 1 -- 172.21.15.92:0/1710943846 >> v1:172.21.15.92:6789/0 conn(0x7f44d40cd460 legacy=0x7f44d406beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:00.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.331+0000 7f44d979b700 1 -- 172.21.15.92:0/1710943846 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d40cd8f0 con 0x7f44d40b8510 2023-11-28T11:24:00.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.331+0000 7f44d979b700 1 --2- 172.21.15.92:0/1710943846 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44d40b8510 0x7f44d406b970 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f44d0009840 tx=0x7f44d0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=804eb90e5f75f733 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:00.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.331+0000 7f44cbfff700 1 -- 172.21.15.92:0/1710943846 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d0011830 con 0x7f44d40b8510 2023-11-28T11:24:00.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.331+0000 7f44cbfff700 1 -- 172.21.15.92:0/1710943846 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44d0011990 con 0x7f44d40b8510 2023-11-28T11:24:00.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.331+0000 7f44cbfff700 1 -- 172.21.15.92:0/1710943846 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d0011d00 con 0x7f44d40b8510 2023-11-28T11:24:00.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.332+0000 7f44db9ff700 1 -- 172.21.15.92:0/1710943846 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44d40b8510 msgr2=0x7f44d406b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:00.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.332+0000 7f44db9ff700 1 --2- 172.21.15.92:0/1710943846 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44d40b8510 0x7f44d406b970 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f44d0009840 tx=0x7f44d0009b50 comp rx=0 tx=0).stop 2023-11-28T11:24:00.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.332+0000 7f44db9ff700 1 -- 172.21.15.92:0/1710943846 shutdown_connections 2023-11-28T11:24:00.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.332+0000 7f44db9ff700 1 --2- 172.21.15.92:0/1710943846 >> v2:172.21.15.169:3300/0 conn(0x7f44d40d0d60 0x7f44d40d11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:00.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.332+0000 7f44db9ff700 1 --2- 172.21.15.92:0/1710943846 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44d40b8510 0x7f44d406b970 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:00.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.332+0000 7f44db9ff700 1 -- 172.21.15.92:0/1710943846 >> 172.21.15.92:0/1710943846 conn(0x7f44d40719a0 msgr2=0x7f44d4071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:00.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.332+0000 7f44db9ff700 1 -- 172.21.15.92:0/1710943846 shutdown_connections 2023-11-28T11:24:00.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.332+0000 7f44db9ff700 1 -- 172.21.15.92:0/1710943846 wait complete. 2023-11-28T11:24:00.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.334+0000 7f44db9ff700 1 Processor -- start 2023-11-28T11:24:00.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.334+0000 7f44db9ff700 1 -- start start 2023-11-28T11:24:00.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.335+0000 7f44db9ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44d40b8510 0x7f44d41e6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:00.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.335+0000 7f44db9ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44d40cd460 0x7f44d415a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:00.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.335+0000 7f44db9ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44d40d0d60 0x7f44d41e5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:00.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.335+0000 7f44db9ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d40ca510 con 0x7f44d40d0d60 2023-11-28T11:24:00.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.335+0000 7f44db9ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d41e6c40 con 0x7f44d40cd460 2023-11-28T11:24:00.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.335+0000 7f44db9ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44d40ca230 con 0x7f44d40b8510 2023-11-28T11:24:00.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.335+0000 7f44d9f9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44d40d0d60 0x7f44d41e5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:00.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.335+0000 7f44d9f9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44d40d0d60 0x7f44d41e5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50142/0 (socket says 172.21.15.92:50142) 2023-11-28T11:24:00.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.335+0000 7f44d9f9c700 1 -- 172.21.15.92:0/2437903795 learned_addr learned my addr 172.21.15.92:0/2437903795 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:00.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.335+0000 7f44d8f9a700 1 --2- 172.21.15.92:0/2437903795 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44d40cd460 0x7f44d415a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:00.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.335+0000 7f44d979b700 1 --2- 172.21.15.92:0/2437903795 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44d40b8510 0x7f44d41e6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:00.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.335+0000 7f44d9f9c700 1 -- 172.21.15.92:0/2437903795 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44d40b8510 msgr2=0x7f44d41e6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:00.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.335+0000 7f44d9f9c700 1 --2- 172.21.15.92:0/2437903795 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44d40b8510 0x7f44d41e6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:00.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.335+0000 7f44d9f9c700 1 -- 172.21.15.92:0/2437903795 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44d40cd460 msgr2=0x7f44d415a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:00.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.335+0000 7f44d9f9c700 1 --2- 172.21.15.92:0/2437903795 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44d40cd460 0x7f44d415a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:00.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.335+0000 7f44d9f9c700 1 -- 172.21.15.92:0/2437903795 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d00094a0 con 0x7f44d40d0d60 2023-11-28T11:24:00.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.335+0000 7f44d979b700 1 --2- 172.21.15.92:0/2437903795 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44d40b8510 0x7f44d41e6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:24:00.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.336+0000 7f44d9f9c700 1 --2- 172.21.15.92:0/2437903795 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44d40d0d60 0x7f44d41e5e40 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f44cc00efc0 tx=0x7f44cc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:00.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.336+0000 7f44c9ffb700 1 -- 172.21.15.92:0/2437903795 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44cc009ba0 con 0x7f44d40d0d60 2023-11-28T11:24:00.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.336+0000 7f44c9ffb700 1 -- 172.21.15.92:0/2437903795 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44cc00f040 con 0x7f44d40d0d60 2023-11-28T11:24:00.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.336+0000 7f44c9ffb700 1 -- 172.21.15.92:0/2437903795 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44cc014480 con 0x7f44d40d0d60 2023-11-28T11:24:00.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.336+0000 7f44db9ff700 1 -- 172.21.15.92:0/2437903795 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44d40cfff0 con 0x7f44d40d0d60 2023-11-28T11:24:00.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.336+0000 7f44db9ff700 1 -- 172.21.15.92:0/2437903795 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44d40cfd10 con 0x7f44d40d0d60 2023-11-28T11:24:00.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.337+0000 7f44db9ff700 1 -- 172.21.15.92:0/2437903795 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44d4067880 con 0x7f44d40d0d60 2023-11-28T11:24:00.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.338+0000 7f44c9ffb700 1 -- 172.21.15.92:0/2437903795 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f44cc0145e0 con 0x7f44d40d0d60 2023-11-28T11:24:00.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.339+0000 7f44c9ffb700 1 --2- 172.21.15.92:0/2437903795 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f44b80682f0 0x7f44b806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:00.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.339+0000 7f44c9ffb700 1 -- 172.21.15.92:0/2437903795 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f44cc086bc0 con 0x7f44d40d0d60 2023-11-28T11:24:00.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.339+0000 7f44d979b700 1 --2- 172.21.15.92:0/2437903795 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f44b80682f0 0x7f44b806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:00.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.340+0000 7f44d979b700 1 --2- 172.21.15.92:0/2437903795 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f44b80682f0 0x7f44b806a7a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f44d0000c00 tx=0x7f44d0008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:00.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.343+0000 7f44c9ffb700 1 -- 172.21.15.92:0/2437903795 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f44cc0531f0 con 0x7f44d40d0d60 2023-11-28T11:24:00.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.531+0000 7f44db9ff700 1 -- 172.21.15.92:0/2437903795 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f44d40d2940 con 0x7f44d40d0d60 2023-11-28T11:24:00.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.532+0000 7f44c9ffb700 1 -- 172.21.15.92:0/2437903795 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v56) v1 ==== 100+0+11 (secure 0 0 0) 0x7f44cc0180a0 con 0x7f44d40d0d60 2023-11-28T11:24:00.533 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-11-28T11:24:00.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.534+0000 7f44db9ff700 1 -- 172.21.15.92:0/2437903795 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f44b80682f0 msgr2=0x7f44b806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:00.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.534+0000 7f44db9ff700 1 --2- 172.21.15.92:0/2437903795 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f44b80682f0 0x7f44b806a7a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f44d0000c00 tx=0x7f44d0008030 comp rx=0 tx=0).stop 2023-11-28T11:24:00.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.534+0000 7f44db9ff700 1 -- 172.21.15.92:0/2437903795 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44d40d0d60 msgr2=0x7f44d41e5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:00.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.534+0000 7f44db9ff700 1 --2- 172.21.15.92:0/2437903795 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44d40d0d60 0x7f44d41e5e40 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f44cc00efc0 tx=0x7f44cc00c3a0 comp rx=0 tx=0).stop 2023-11-28T11:24:00.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.534+0000 7f44db9ff700 1 -- 172.21.15.92:0/2437903795 shutdown_connections 2023-11-28T11:24:00.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.534+0000 7f44db9ff700 1 --2- 172.21.15.92:0/2437903795 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f44b80682f0 0x7f44b806a7a0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:00.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.534+0000 7f44db9ff700 1 --2- 172.21.15.92:0/2437903795 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44d40d0d60 0x7f44d41e5e40 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:00.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.534+0000 7f44db9ff700 1 --2- 172.21.15.92:0/2437903795 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f44d40b8510 0x7f44d41e6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:00.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.534+0000 7f44db9ff700 1 --2- 172.21.15.92:0/2437903795 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f44d40cd460 0x7f44d415a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:00.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.534+0000 7f44db9ff700 1 -- 172.21.15.92:0/2437903795 >> 172.21.15.92:0/2437903795 conn(0x7f44d40719a0 msgr2=0x7f44d40b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:00.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.534+0000 7f44db9ff700 1 -- 172.21.15.92:0/2437903795 shutdown_connections 2023-11-28T11:24:00.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.534+0000 7f44db9ff700 1 -- 172.21.15.92:0/2437903795 wait complete. 2023-11-28T11:24:00.553 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:24:00.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.749+0000 7f869c1ff700 1 Processor -- start 2023-11-28T11:24:00.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.750+0000 7f869c1ff700 1 -- start start 2023-11-28T11:24:00.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.751+0000 7f869c1ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f86940b8510 0x7f869406b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:00.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.751+0000 7f869c1ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f86940cd460 0x7f869406beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:00.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.751+0000 7f869c1ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86940b88e0 con 0x7f869406c3f0 2023-11-28T11:24:00.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.751+0000 7f869c1ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86940cdbd0 con 0x7f86940cd460 2023-11-28T11:24:00.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.751+0000 7f869c1ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86940d02d0 con 0x7f86940b8510 2023-11-28T11:24:00.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.751+0000 7f869979a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f86940cd460 0x7f869406beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:00.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.751+0000 7f869a79c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f869406c3f0 0x7f869415cc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55168/0 (socket says 172.21.15.92:55168) 2023-11-28T11:24:00.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.751+0000 7f869979a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f86940cd460 0x7f869406beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50154/0 (socket says 172.21.15.92:50154) 2023-11-28T11:24:00.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.751+0000 7f869a79c700 1 -- 172.21.15.92:0/262401117 learned_addr learned my addr 172.21.15.92:0/262401117 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:24:00.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.751+0000 7f8699f9b700 1 --2- 172.21.15.92:0/262401117 >> v2:172.21.15.169:3300/0 conn(0x7f86940b8510 0x7f869406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:00.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.751+0000 7f869979a700 1 -- 172.21.15.92:0/262401117 >> v2:172.21.15.169:3300/0 conn(0x7f86940b8510 msgr2=0x7f869406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:00.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.751+0000 7f869979a700 1 --2- 172.21.15.92:0/262401117 >> v2:172.21.15.169:3300/0 conn(0x7f86940b8510 0x7f869406b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:00.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.751+0000 7f869979a700 1 -- 172.21.15.92:0/262401117 >> v1:172.21.15.92:6789/0 conn(0x7f869406c3f0 legacy=0x7f869415cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:00.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.751+0000 7f869979a700 1 -- 172.21.15.92:0/262401117 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86940cd8f0 con 0x7f86940cd460 2023-11-28T11:24:00.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.752+0000 7f869979a700 1 --2- 172.21.15.92:0/262401117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86940cd460 0x7f869406beb0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f8684009870 tx=0x7f8684009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=651b2df7a2629fff server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:00.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.752+0000 7f8698f99700 1 -- 172.21.15.92:0/262401117 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8684011830 con 0x7f86940cd460 2023-11-28T11:24:00.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.752+0000 7f8698f99700 1 -- 172.21.15.92:0/262401117 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8684011990 con 0x7f86940cd460 2023-11-28T11:24:00.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.752+0000 7f8698f99700 1 -- 172.21.15.92:0/262401117 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8684011d20 con 0x7f86940cd460 2023-11-28T11:24:00.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.752+0000 7f869c1ff700 1 -- 172.21.15.92:0/262401117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86940cd460 msgr2=0x7f869406beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:00.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.752+0000 7f869c1ff700 1 --2- 172.21.15.92:0/262401117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86940cd460 0x7f869406beb0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f8684009870 tx=0x7f8684009b80 comp rx=0 tx=0).stop 2023-11-28T11:24:00.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.753+0000 7f869c1ff700 1 -- 172.21.15.92:0/262401117 shutdown_connections 2023-11-28T11:24:00.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.753+0000 7f869c1ff700 1 --2- 172.21.15.92:0/262401117 >> v2:172.21.15.169:3300/0 conn(0x7f86940b8510 0x7f869406b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:00.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.753+0000 7f869c1ff700 1 --2- 172.21.15.92:0/262401117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86940cd460 0x7f869406beb0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:00.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.753+0000 7f869c1ff700 1 -- 172.21.15.92:0/262401117 >> 172.21.15.92:0/262401117 conn(0x7f86940719a0 msgr2=0x7f8694071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:00.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.753+0000 7f869c1ff700 1 -- 172.21.15.92:0/262401117 shutdown_connections 2023-11-28T11:24:00.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.753+0000 7f869c1ff700 1 -- 172.21.15.92:0/262401117 wait complete. 2023-11-28T11:24:00.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.754+0000 7f869c1ff700 1 Processor -- start 2023-11-28T11:24:00.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.754+0000 7f869c1ff700 1 -- start start 2023-11-28T11:24:00.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.755+0000 7f869c1ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f869406c3f0 0x7f86940b3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:00.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.755+0000 7f869c1ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86940b8510 0x7f8694157a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:00.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.755+0000 7f869c1ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86940cd460 0x7f8694143ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:00.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.755+0000 7f869c1ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8694142f50 con 0x7f869406c3f0 2023-11-28T11:24:00.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.755+0000 7f869c1ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8694143230 con 0x7f86940b8510 2023-11-28T11:24:00.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.755+0000 7f869c1ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8694143510 con 0x7f86940cd460 2023-11-28T11:24:00.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.756+0000 7f8699f9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f869406c3f0 0x7f86940b3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:00.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.756+0000 7f8699f9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f869406c3f0 0x7f86940b3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50168/0 (socket says 172.21.15.92:50168) 2023-11-28T11:24:00.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.756+0000 7f8699f9b700 1 -- 172.21.15.92:0/284376057 learned_addr learned my addr 172.21.15.92:0/284376057 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:00.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.756+0000 7f869a79c700 1 --2- 172.21.15.92:0/284376057 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86940cd460 0x7f8694143ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:00.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.756+0000 7f869979a700 1 --2- 172.21.15.92:0/284376057 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86940b8510 0x7f8694157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:00.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.756+0000 7f8699f9b700 1 -- 172.21.15.92:0/284376057 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86940cd460 msgr2=0x7f8694143ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:00.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.756+0000 7f8699f9b700 1 --2- 172.21.15.92:0/284376057 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86940cd460 0x7f8694143ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:00.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.756+0000 7f8699f9b700 1 -- 172.21.15.92:0/284376057 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86940b8510 msgr2=0x7f8694157a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:00.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.756+0000 7f8699f9b700 1 --2- 172.21.15.92:0/284376057 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86940b8510 0x7f8694157a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:00.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.756+0000 7f8699f9b700 1 -- 172.21.15.92:0/284376057 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86840094a0 con 0x7f869406c3f0 2023-11-28T11:24:00.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.756+0000 7f8699f9b700 1 --2- 172.21.15.92:0/284376057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f869406c3f0 0x7f86940b3bb0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f869000e810 tx=0x7f869000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:00.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.756+0000 7f868a7fc700 1 -- 172.21.15.92:0/284376057 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869000c070 con 0x7f869406c3f0 2023-11-28T11:24:00.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.757+0000 7f868a7fc700 1 -- 172.21.15.92:0/284376057 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8690013610 con 0x7f869406c3f0 2023-11-28T11:24:00.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.757+0000 7f869c1ff700 1 -- 172.21.15.92:0/284376057 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86940cfff0 con 0x7f869406c3f0 2023-11-28T11:24:00.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.757+0000 7f868a7fc700 1 -- 172.21.15.92:0/284376057 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86900139c0 con 0x7f869406c3f0 2023-11-28T11:24:00.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.757+0000 7f869c1ff700 1 -- 172.21.15.92:0/284376057 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86940cfd10 con 0x7f869406c3f0 2023-11-28T11:24:00.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.758+0000 7f869c1ff700 1 -- 172.21.15.92:0/284376057 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8694067880 con 0x7f869406c3f0 2023-11-28T11:24:00.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.759+0000 7f868a7fc700 1 -- 172.21.15.92:0/284376057 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8690023070 con 0x7f869406c3f0 2023-11-28T11:24:00.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.759+0000 7f868a7fc700 1 --2- 172.21.15.92:0/284376057 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f86780682f0 0x7f867806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:00.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.760+0000 7f868a7fc700 1 -- 172.21.15.92:0/284376057 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f8690085b30 con 0x7f869406c3f0 2023-11-28T11:24:00.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.760+0000 7f869979a700 1 --2- 172.21.15.92:0/284376057 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f86780682f0 0x7f867806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:00.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.760+0000 7f869979a700 1 --2- 172.21.15.92:0/284376057 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f86780682f0 0x7f867806a7a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f86940b1de0 tx=0x7f868401c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:00.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:00.764+0000 7f868a7fc700 1 -- 172.21.15.92:0/284376057 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f86900521d0 con 0x7f869406c3f0 2023-11-28T11:24:01.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.000+0000 7f869c1ff700 1 -- 172.21.15.92:0/284376057 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f869415c9d0 con 0x7f869406c3f0 2023-11-28T11:24:01.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.000+0000 7f868a7fc700 1 -- 172.21.15.92:0/284376057 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4651 (secure 0 0 0) 0x7f869000f040 con 0x7f869406c3f0 2023-11-28T11:24:01.001 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:24:01.001 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5593,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/3937189339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3937189339},{"type":"v1","addr":"172.21.15.169:6834","nonce":3937189339}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5596,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2020201016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2020201016},{"type":"v1","addr":"172.21.15.169:6835","nonce":2020201016}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5751,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2423750324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2423750324},{"type":"v1","addr":"172.21.15.92:6835","nonce":2423750324}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:23:13.247233+0000","modified":"2023-11-28T11:23:16.254256+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5754},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5754":{"gid":5754,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/950691230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":950691230},{"type":"v1","addr":"172.21.15.92:6837","nonce":950691230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T11:24:01.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.003+0000 7f869c1ff700 1 -- 172.21.15.92:0/284376057 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f86780682f0 msgr2=0x7f867806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:01.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.003+0000 7f869c1ff700 1 --2- 172.21.15.92:0/284376057 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f86780682f0 0x7f867806a7a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f86940b1de0 tx=0x7f868401c040 comp rx=0 tx=0).stop 2023-11-28T11:24:01.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.003+0000 7f869c1ff700 1 -- 172.21.15.92:0/284376057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f869406c3f0 msgr2=0x7f86940b3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:01.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.003+0000 7f869c1ff700 1 --2- 172.21.15.92:0/284376057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f869406c3f0 0x7f86940b3bb0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f869000e810 tx=0x7f869000eb20 comp rx=0 tx=0).stop 2023-11-28T11:24:01.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.003+0000 7f869c1ff700 1 -- 172.21.15.92:0/284376057 shutdown_connections 2023-11-28T11:24:01.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.003+0000 7f869c1ff700 1 --2- 172.21.15.92:0/284376057 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86940cd460 0x7f8694143ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:01.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.003+0000 7f869c1ff700 1 --2- 172.21.15.92:0/284376057 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f86780682f0 0x7f867806a7a0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:01.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.003+0000 7f869c1ff700 1 --2- 172.21.15.92:0/284376057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f869406c3f0 0x7f86940b3bb0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:01.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.003+0000 7f869c1ff700 1 --2- 172.21.15.92:0/284376057 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86940b8510 0x7f8694157a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:01.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.003+0000 7f869c1ff700 1 -- 172.21.15.92:0/284376057 >> 172.21.15.92:0/284376057 conn(0x7f86940719a0 msgr2=0x7f86940b6930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:01.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.003+0000 7f869c1ff700 1 -- 172.21.15.92:0/284376057 shutdown_connections 2023-11-28T11:24:01.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.003+0000 7f869c1ff700 1 -- 172.21.15.92:0/284376057 wait complete. 2023-11-28T11:24:01.005 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 24 2023-11-28T11:24:01.022 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:24:01.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.217+0000 7fc5a93ff700 1 Processor -- start 2023-11-28T11:24:01.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.217+0000 7fc5a93ff700 1 -- start start 2023-11-28T11:24:01.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.218+0000 7fc5a93ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc5a40b8510 0x7fc5a406b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:01.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.218+0000 7fc5a93ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5a40b88e0 con 0x7fc5a40d0d60 2023-11-28T11:24:01.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.218+0000 7fc5a93ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5a40cd8f0 con 0x7fc5a40cd460 2023-11-28T11:24:01.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.218+0000 7fc5a93ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5a40cdbd0 con 0x7fc5a40b8510 2023-11-28T11:24:01.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.218+0000 7fc5a37fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc5a40d0d60 0x7fc5a40d1200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55176/0 (socket says 172.21.15.92:55176) 2023-11-28T11:24:01.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.218+0000 7fc5a37fe700 1 -- 172.21.15.92:0/1284757653 learned_addr learned my addr 172.21.15.92:0/1284757653 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:24:01.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.219+0000 7fc5a1ffb700 1 -- 172.21.15.92:0/1284757653 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2789361385 0 0) 0x7fc5a40b88e0 con 0x7fc5a40d0d60 2023-11-28T11:24:01.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.219+0000 7fc5a1ffb700 1 -- 172.21.15.92:0/1284757653 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc58c0035b0 con 0x7fc5a40d0d60 2023-11-28T11:24:01.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.219+0000 7fc5a2ffd700 1 --2- 172.21.15.92:0/1284757653 >> v2:172.21.15.169:3300/0 conn(0x7fc5a40b8510 0x7fc5a406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:01.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.219+0000 7fc5a1ffb700 1 -- 172.21.15.92:0/1284757653 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2771648560 0 0) 0x7fc58c0035b0 con 0x7fc5a40d0d60 2023-11-28T11:24:01.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.219+0000 7fc5a1ffb700 1 -- 172.21.15.92:0/1284757653 >> v2:172.21.15.169:3300/0 conn(0x7fc5a40b8510 msgr2=0x7fc5a406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:01.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.219+0000 7fc5a1ffb700 1 --2- 172.21.15.92:0/1284757653 >> v2:172.21.15.169:3300/0 conn(0x7fc5a40b8510 0x7fc5a406b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:01.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.219+0000 7fc5a1ffb700 1 -- 172.21.15.92:0/1284757653 >> v1:172.21.15.169:6789/0 conn(0x7fc5a40cd460 legacy=0x7fc5a406beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:01.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.219+0000 7fc5a1ffb700 1 -- 172.21.15.92:0/1284757653 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a40cfd10 con 0x7fc5a40d0d60 2023-11-28T11:24:01.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.219+0000 7fc5a1ffb700 1 -- 172.21.15.92:0/1284757653 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532161587 0 0) 0x7fc59c0019f0 con 0x7fc5a40d0d60 2023-11-28T11:24:01.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.219+0000 7fc5a1ffb700 1 -- 172.21.15.92:0/1284757653 >> v1:172.21.15.92:6789/0 conn(0x7fc5a40d0d60 legacy=0x7fc5a40d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:01.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.219+0000 7fc5a1ffb700 1 --2- 172.21.15.92:0/1284757653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc58c003a00 0x7fc58c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:01.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.219+0000 7fc5a1ffb700 1 --2- 172.21.15.92:0/1284757653 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc58c004850 0x7fc58c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:01.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.219+0000 7fc5a1ffb700 1 --2- 172.21.15.92:0/1284757653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc58c005200 0x7fc58c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:01.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.219+0000 7fc5a1ffb700 1 -- 172.21.15.92:0/1284757653 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a40cdbd0 con 0x7fc58c005200 2023-11-28T11:24:01.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.219+0000 7fc5a1ffb700 1 -- 172.21.15.92:0/1284757653 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc58c00c190 con 0x7fc58c003a00 2023-11-28T11:24:01.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.219+0000 7fc5a1ffb700 1 -- 172.21.15.92:0/1284757653 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc58c00c350 con 0x7fc58c004850 2023-11-28T11:24:01.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.220+0000 7fc5a37fe700 1 --2- 172.21.15.92:0/1284757653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc58c005200 0x7fc58c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:01.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.220+0000 7fc5a2ffd700 1 --2- 172.21.15.92:0/1284757653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc58c003a00 0x7fc58c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:01.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.220+0000 7fc5a27fc700 1 --2- 172.21.15.92:0/1284757653 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc58c004850 0x7fc58c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:01.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.220+0000 7fc5a37fe700 1 -- 172.21.15.92:0/1284757653 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc58c004850 msgr2=0x7fc58c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:01.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.220+0000 7fc5a37fe700 1 --2- 172.21.15.92:0/1284757653 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc58c004850 0x7fc58c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:01.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.220+0000 7fc5a37fe700 1 -- 172.21.15.92:0/1284757653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc58c003a00 msgr2=0x7fc58c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:01.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.220+0000 7fc5a37fe700 1 --2- 172.21.15.92:0/1284757653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc58c003a00 0x7fc58c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:01.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.220+0000 7fc5a37fe700 1 -- 172.21.15.92:0/1284757653 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc58c0035b0 con 0x7fc58c005200 2023-11-28T11:24:01.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.220+0000 7fc5a2ffd700 1 --2- 172.21.15.92:0/1284757653 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc58c003a00 0x7fc58c003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:24:01.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.220+0000 7fc5a37fe700 1 -- 172.21.15.92:0/1284757653 reap_dead start 2023-11-28T11:24:01.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.220+0000 7fc5a37fe700 1 --2- 172.21.15.92:0/1284757653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc58c005200 0x7fc58c00ba10 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fc59c00a800 tx=0x7fc58c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2753ac22af46151c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:01.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.220+0000 7fc5a1ffb700 1 -- 172.21.15.92:0/1284757653 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59c00a2f0 con 0x7fc58c005200 2023-11-28T11:24:01.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.221+0000 7fc5a1ffb700 1 -- 172.21.15.92:0/1284757653 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc59c012620 con 0x7fc58c005200 2023-11-28T11:24:01.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.221+0000 7fc5a1ffb700 1 -- 172.21.15.92:0/1284757653 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59c0129d0 con 0x7fc58c005200 2023-11-28T11:24:01.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.221+0000 7fc5a93ff700 1 -- 172.21.15.92:0/1284757653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc58c005200 msgr2=0x7fc58c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:01.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.221+0000 7fc5a93ff700 1 --2- 172.21.15.92:0/1284757653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc58c005200 0x7fc58c00ba10 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fc59c00a800 tx=0x7fc58c005880 comp rx=0 tx=0).stop 2023-11-28T11:24:01.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.221+0000 7fc5a93ff700 1 -- 172.21.15.92:0/1284757653 shutdown_connections 2023-11-28T11:24:01.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.221+0000 7fc5a93ff700 1 --2- 172.21.15.92:0/1284757653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc58c005200 0x7fc58c00ba10 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:01.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.221+0000 7fc5a93ff700 1 -- 172.21.15.92:0/1284757653 >> 172.21.15.92:0/1284757653 conn(0x7fc5a40719a0 msgr2=0x7fc5a4071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:01.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.221+0000 7fc5a93ff700 1 -- 172.21.15.92:0/1284757653 shutdown_connections 2023-11-28T11:24:01.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.221+0000 7fc5a93ff700 1 -- 172.21.15.92:0/1284757653 wait complete. 2023-11-28T11:24:01.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.223+0000 7fc5a93ff700 1 Processor -- start 2023-11-28T11:24:01.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.223+0000 7fc5a93ff700 1 -- start start 2023-11-28T11:24:01.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.224+0000 7fc5a93ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc58c005200 0x7fc5a41f5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:01.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.224+0000 7fc5a93ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5a40ca180 0x7fc5a40ca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:01.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.224+0000 7fc5a93ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5a40b3c20 0x7fc5a40b4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:01.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.224+0000 7fc5a93ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a40cab30 con 0x7fc5a40b3c20 2023-11-28T11:24:01.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.224+0000 7fc5a93ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a40b45d0 con 0x7fc58c005200 2023-11-28T11:24:01.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.224+0000 7fc5a93ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a4158860 con 0x7fc5a40ca180 2023-11-28T11:24:01.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.224+0000 7fc5a37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5a40b3c20 0x7fc5a40b4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:01.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.224+0000 7fc5a37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5a40b3c20 0x7fc5a40b4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50178/0 (socket says 172.21.15.92:50178) 2023-11-28T11:24:01.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.224+0000 7fc5a37fe700 1 -- 172.21.15.92:0/2306850562 learned_addr learned my addr 172.21.15.92:0/2306850562 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:01.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.224+0000 7fc5a37fe700 1 -- 172.21.15.92:0/2306850562 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5a40ca180 msgr2=0x7fc5a40ca5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:01.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.224+0000 7fc5a37fe700 1 --2- 172.21.15.92:0/2306850562 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5a40ca180 0x7fc5a40ca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:01.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.224+0000 7fc5a37fe700 1 -- 172.21.15.92:0/2306850562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc58c005200 msgr2=0x7fc5a41f5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:01.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.224+0000 7fc5a37fe700 1 --2- 172.21.15.92:0/2306850562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc58c005200 0x7fc5a41f5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:01.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.224+0000 7fc5a37fe700 1 -- 172.21.15.92:0/2306850562 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a40d29c0 con 0x7fc5a40b3c20 2023-11-28T11:24:01.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.225+0000 7fc5a37fe700 1 --2- 172.21.15.92:0/2306850562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5a40b3c20 0x7fc5a40b4090 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fc5a40d3f10 tx=0x7fc59c001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:01.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.225+0000 7fc58b7fe700 1 -- 172.21.15.92:0/2306850562 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59c0017b0 con 0x7fc5a40b3c20 2023-11-28T11:24:01.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.225+0000 7fc58b7fe700 1 -- 172.21.15.92:0/2306850562 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc59c001910 con 0x7fc5a40b3c20 2023-11-28T11:24:01.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.225+0000 7fc58b7fe700 1 -- 172.21.15.92:0/2306850562 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59c01d640 con 0x7fc5a40b3c20 2023-11-28T11:24:01.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.225+0000 7fc5a93ff700 1 -- 172.21.15.92:0/2306850562 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5a40cfff0 con 0x7fc5a40b3c20 2023-11-28T11:24:01.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.225+0000 7fc5a93ff700 1 -- 172.21.15.92:0/2306850562 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5a40d2b50 con 0x7fc5a40b3c20 2023-11-28T11:24:01.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.226+0000 7fc5a93ff700 1 -- 172.21.15.92:0/2306850562 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5a4066d10 con 0x7fc5a40b3c20 2023-11-28T11:24:01.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.227+0000 7fc58b7fe700 1 -- 172.21.15.92:0/2306850562 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc59c010070 con 0x7fc5a40b3c20 2023-11-28T11:24:01.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.228+0000 7fc58b7fe700 1 --2- 172.21.15.92:0/2306850562 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc5900682f0 0x7fc59006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:01.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.228+0000 7fc58b7fe700 1 -- 172.21.15.92:0/2306850562 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fc59c087e20 con 0x7fc5a40b3c20 2023-11-28T11:24:01.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.228+0000 7fc5a2ffd700 1 --2- 172.21.15.92:0/2306850562 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc5900682f0 0x7fc59006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:01.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.229+0000 7fc5a2ffd700 1 --2- 172.21.15.92:0/2306850562 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc5900682f0 0x7fc59006a7a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc594009870 tx=0x7fc594010040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:01.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.232+0000 7fc58b7fe700 1 -- 172.21.15.92:0/2306850562 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc59c0543f0 con 0x7fc5a40b3c20 2023-11-28T11:24:01.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.431+0000 7fc5a93ff700 1 -- 172.21.15.92:0/2306850562 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc5a415a2d0 con 0x7fc5a40b3c20 2023-11-28T11:24:01.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.432+0000 7fc58b7fe700 1 -- 172.21.15.92:0/2306850562 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+18180 (secure 0 0 0) 0x7fc59c057a10 con 0x7fc5a40b3c20 2023-11-28T11:24:01.433 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:24:01.433 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":56,"fsid":"68956e0c-c16f-47cd-a220-f98364b0831d","created":"2023-11-28T11:20:28.697573+0000","modified":"2023-11-28T11:23:14.249429+0000","last_up_change":"2023-11-28T11:20:41.257011+0000","last_in_change":"2023-11-28T11:20:32.249767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:20:41.925237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:20:48.682497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:23:10.939884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-28T11:23:12.227433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6abe483f-0e03-4a26-b9cf-62041291747e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":115823},{"type":"v1","addr":"172.21.15.92:6809","nonce":115823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":115823},{"type":"v1","addr":"172.21.15.92:6813","nonce":115823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":115823},{"type":"v1","addr":"172.21.15.92:6821","nonce":115823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":115823},{"type":"v1","addr":"172.21.15.92:6817","nonce":115823}]},"public_addr":"172.21.15.92:6809/115823","cluster_addr":"172.21.15.92:6813/115823","heartbeat_back_addr":"172.21.15.92:6821/115823","heartbeat_front_addr":"172.21.15.92:6817/115823","state":["exists","up"]},{"osd":1,"uuid":"13c1de73-1a8c-4d7e-8a28-9dae52dfe089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":115824},{"type":"v1","addr":"172.21.15.92:6812","nonce":115824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":115824},{"type":"v1","addr":"172.21.15.92:6816","nonce":115824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":115824},{"type":"v1","addr":"172.21.15.92:6823","nonce":115824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":115824},{"type":"v1","addr":"172.21.15.92:6820","nonce":115824}]},"public_addr":"172.21.15.92:6812/115824","cluster_addr":"172.21.15.92:6816/115824","heartbeat_back_addr":"172.21.15.92:6823/115824","heartbeat_front_addr":"172.21.15.92:6820/115824","state":["exists","up"]},{"osd":2,"uuid":"25788a55-112f-4f00-810d-2bc37cf76778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":115821},{"type":"v1","addr":"172.21.15.92:6801","nonce":115821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":115821},{"type":"v1","addr":"172.21.15.92:6803","nonce":115821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":115821},{"type":"v1","addr":"172.21.15.92:6807","nonce":115821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":115821},{"type":"v1","addr":"172.21.15.92:6805","nonce":115821}]},"public_addr":"172.21.15.92:6801/115821","cluster_addr":"172.21.15.92:6803/115821","heartbeat_back_addr":"172.21.15.92:6807/115821","heartbeat_front_addr":"172.21.15.92:6805/115821","state":["exists","up"]},{"osd":3,"uuid":"adf54e24-33a9-41fe-9ce3-98a5ab6c158e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":115825},{"type":"v1","addr":"172.21.15.92:6825","nonce":115825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":115825},{"type":"v1","addr":"172.21.15.92:6827","nonce":115825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":115825},{"type":"v1","addr":"172.21.15.92:6831","nonce":115825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":115825},{"type":"v1","addr":"172.21.15.92:6829","nonce":115825}]},"public_addr":"172.21.15.92:6825/115825","cluster_addr":"172.21.15.92:6827/115825","heartbeat_back_addr":"172.21.15.92:6831/115825","heartbeat_front_addr":"172.21.15.92:6829/115825","state":["exists","up"]},{"osd":4,"uuid":"ccb9b747-f399-44ea-9a18-9db3152227e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":116035},{"type":"v1","addr":"172.21.15.169:6817","nonce":116035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":116035},{"type":"v1","addr":"172.21.15.169:6819","nonce":116035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":116035},{"type":"v1","addr":"172.21.15.169:6826","nonce":116035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":116035},{"type":"v1","addr":"172.21.15.169:6822","nonce":116035}]},"public_addr":"172.21.15.169:6817/116035","cluster_addr":"172.21.15.169:6819/116035","heartbeat_back_addr":"172.21.15.169:6826/116035","heartbeat_front_addr":"172.21.15.169:6822/116035","state":["exists","up"]},{"osd":5,"uuid":"308a1e63-75aa-4d7d-8a7c-cecc24714fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":116033},{"type":"v1","addr":"172.21.15.169:6801","nonce":116033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":116033},{"type":"v1","addr":"172.21.15.169:6803","nonce":116033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":116033},{"type":"v1","addr":"172.21.15.169:6807","nonce":116033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":116033},{"type":"v1","addr":"172.21.15.169:6805","nonce":116033}]},"public_addr":"172.21.15.169:6801/116033","cluster_addr":"172.21.15.169:6803/116033","heartbeat_back_addr":"172.21.15.169:6807/116033","heartbeat_front_addr":"172.21.15.169:6805/116033","state":["exists","up"]},{"osd":6,"uuid":"9c459b7b-b821-4b4b-b91e-6aaae271457c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":116034},{"type":"v1","addr":"172.21.15.169:6809","nonce":116034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":116034},{"type":"v1","addr":"172.21.15.169:6811","nonce":116034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":116034},{"type":"v1","addr":"172.21.15.169:6815","nonce":116034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":116034},{"type":"v1","addr":"172.21.15.169:6813","nonce":116034}]},"public_addr":"172.21.15.169:6809/116034","cluster_addr":"172.21.15.169:6811/116034","heartbeat_back_addr":"172.21.15.169:6815/116034","heartbeat_front_addr":"172.21.15.169:6813/116034","state":["exists","up"]},{"osd":7,"uuid":"1ca9aa4a-755d-4e28-a937-4d8dccab4df3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":116044},{"type":"v1","addr":"172.21.15.169:6823","nonce":116044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6825","nonce":116044},{"type":"v1","addr":"172.21.15.169:6827","nonce":116044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":116044},{"type":"v1","addr":"172.21.15.169:6831","nonce":116044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":116044},{"type":"v1","addr":"172.21.15.169:6829","nonce":116044}]},"public_addr":"172.21.15.169:6823/116044","cluster_addr":"172.21.15.169:6827/116044","heartbeat_back_addr":"172.21.15.169:6831/116044","heartbeat_front_addr":"172.21.15.169:6829/116044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.586781+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.869778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.508594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.860493+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.872267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.746933+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.611520+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.871446+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:24:01.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.434+0000 7fc5a93ff700 1 -- 172.21.15.92:0/2306850562 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc5900682f0 msgr2=0x7fc59006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:01.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.434+0000 7fc5a93ff700 1 --2- 172.21.15.92:0/2306850562 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc5900682f0 0x7fc59006a7a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc594009870 tx=0x7fc594010040 comp rx=0 tx=0).stop 2023-11-28T11:24:01.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.434+0000 7fc5a93ff700 1 -- 172.21.15.92:0/2306850562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5a40b3c20 msgr2=0x7fc5a40b4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:01.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.434+0000 7fc5a93ff700 1 --2- 172.21.15.92:0/2306850562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5a40b3c20 0x7fc5a40b4090 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fc5a40d3f10 tx=0x7fc59c001d80 comp rx=0 tx=0).stop 2023-11-28T11:24:01.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.435+0000 7fc5a93ff700 1 -- 172.21.15.92:0/2306850562 shutdown_connections 2023-11-28T11:24:01.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.435+0000 7fc5a93ff700 1 --2- 172.21.15.92:0/2306850562 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc5900682f0 0x7fc59006a7a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:01.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.435+0000 7fc5a93ff700 1 --2- 172.21.15.92:0/2306850562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5a40b3c20 0x7fc5a40b4090 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:01.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.435+0000 7fc5a93ff700 1 --2- 172.21.15.92:0/2306850562 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc58c005200 0x7fc5a41f5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:01.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.435+0000 7fc5a93ff700 1 --2- 172.21.15.92:0/2306850562 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc5a40ca180 0x7fc5a40ca5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:01.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.435+0000 7fc5a93ff700 1 -- 172.21.15.92:0/2306850562 >> 172.21.15.92:0/2306850562 conn(0x7fc5a40719a0 msgr2=0x7fc5a40d18a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:01.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.435+0000 7fc5a93ff700 1 -- 172.21.15.92:0/2306850562 shutdown_connections 2023-11-28T11:24:01.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.435+0000 7fc5a93ff700 1 -- 172.21.15.92:0/2306850562 wait complete. 2023-11-28T11:24:01.454 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T11:24:01.454 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:24:01.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.648+0000 7f575c3ff700 1 Processor -- start 2023-11-28T11:24:01.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.649+0000 7f575c3ff700 1 -- start start 2023-11-28T11:24:01.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.650+0000 7f575c3ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f57540b8510 0x7f575406b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:01.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.650+0000 7f575c3ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f57540d0d60 0x7f57540d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:01.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.650+0000 7f575c3ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57540b88e0 con 0x7f57540cd460 2023-11-28T11:24:01.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.650+0000 7f575c3ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57540cdbd0 con 0x7f57540b8510 2023-11-28T11:24:01.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.650+0000 7f575c3ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57540d02d0 con 0x7f57540d0d60 2023-11-28T11:24:01.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.650+0000 7f575a19b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f57540b8510 0x7f575406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:01.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.650+0000 7f575999a700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f57540cd460 0x7f575406beb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55192/0 (socket says 172.21.15.92:55192) 2023-11-28T11:24:01.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.650+0000 7f575999a700 1 -- 172.21.15.92:0/1324969491 learned_addr learned my addr 172.21.15.92:0/1324969491 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:24:01.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.650+0000 7f575a99c700 1 --2- 172.21.15.92:0/1324969491 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f57540d0d60 0x7f57540d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:01.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.650+0000 7f575a19b700 1 -- 172.21.15.92:0/1324969491 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f57540d0d60 msgr2=0x7f57540d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:01.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.650+0000 7f575a19b700 1 --2- 172.21.15.92:0/1324969491 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f57540d0d60 0x7f57540d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:01.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.650+0000 7f575a19b700 1 -- 172.21.15.92:0/1324969491 >> v1:172.21.15.92:6789/0 conn(0x7f57540cd460 legacy=0x7f575406beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:01.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.651+0000 7f575a19b700 1 -- 172.21.15.92:0/1324969491 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57540cd8f0 con 0x7f57540b8510 2023-11-28T11:24:01.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.651+0000 7f575a19b700 1 --2- 172.21.15.92:0/1324969491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57540b8510 0x7f575406b970 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f5750009870 tx=0x7f5750009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9c42d8df60896f8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:01.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.651+0000 7f5759199700 1 -- 172.21.15.92:0/1324969491 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5750004230 con 0x7f57540b8510 2023-11-28T11:24:01.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.651+0000 7f5759199700 1 -- 172.21.15.92:0/1324969491 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5750004390 con 0x7f57540b8510 2023-11-28T11:24:01.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.651+0000 7f5759199700 1 -- 172.21.15.92:0/1324969491 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57500036a0 con 0x7f57540b8510 2023-11-28T11:24:01.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.651+0000 7f575c3ff700 1 -- 172.21.15.92:0/1324969491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57540b8510 msgr2=0x7f575406b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:01.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.651+0000 7f575c3ff700 1 --2- 172.21.15.92:0/1324969491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57540b8510 0x7f575406b970 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f5750009870 tx=0x7f5750009b80 comp rx=0 tx=0).stop 2023-11-28T11:24:01.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.652+0000 7f575c3ff700 1 -- 172.21.15.92:0/1324969491 shutdown_connections 2023-11-28T11:24:01.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.652+0000 7f575c3ff700 1 --2- 172.21.15.92:0/1324969491 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f57540d0d60 0x7f57540d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:01.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.652+0000 7f575c3ff700 1 --2- 172.21.15.92:0/1324969491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57540b8510 0x7f575406b970 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:01.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.652+0000 7f575c3ff700 1 -- 172.21.15.92:0/1324969491 >> 172.21.15.92:0/1324969491 conn(0x7f57540719a0 msgr2=0x7f5754071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:01.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.652+0000 7f575c3ff700 1 -- 172.21.15.92:0/1324969491 shutdown_connections 2023-11-28T11:24:01.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.652+0000 7f575c3ff700 1 -- 172.21.15.92:0/1324969491 wait complete. 2023-11-28T11:24:01.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.653+0000 7f575c3ff700 1 Processor -- start 2023-11-28T11:24:01.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.654+0000 7f575c3ff700 1 -- start start 2023-11-28T11:24:01.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.654+0000 7f575c3ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57540b8510 0x7f57540a9b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:01.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.654+0000 7f575c3ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f57540cd460 0x7f57540c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:01.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.654+0000 7f575c3ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57540d0d60 0x7f5754158e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:01.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.654+0000 7f575c3ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57540caf20 con 0x7f57540d0d60 2023-11-28T11:24:01.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.654+0000 7f575c3ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57540ca420 con 0x7f57540b8510 2023-11-28T11:24:01.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.654+0000 7f575c3ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f575414de50 con 0x7f57540cd460 2023-11-28T11:24:01.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.655+0000 7f575a99c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57540d0d60 0x7f5754158e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:01.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.655+0000 7f575a99c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57540d0d60 0x7f5754158e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50200/0 (socket says 172.21.15.92:50200) 2023-11-28T11:24:01.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.655+0000 7f575a99c700 1 -- 172.21.15.92:0/1266136942 learned_addr learned my addr 172.21.15.92:0/1266136942 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:01.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.655+0000 7f575a19b700 1 --2- 172.21.15.92:0/1266136942 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57540b8510 0x7f57540a9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:01.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.655+0000 7f575999a700 1 --2- 172.21.15.92:0/1266136942 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f57540cd460 0x7f57540c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:01.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.655+0000 7f575a99c700 1 -- 172.21.15.92:0/1266136942 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f57540cd460 msgr2=0x7f57540c9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:01.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.655+0000 7f575a99c700 1 --2- 172.21.15.92:0/1266136942 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f57540cd460 0x7f57540c9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:01.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.655+0000 7f575a99c700 1 -- 172.21.15.92:0/1266136942 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57540b8510 msgr2=0x7f57540a9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:01.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.655+0000 7f575a99c700 1 --2- 172.21.15.92:0/1266136942 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57540b8510 0x7f57540a9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:01.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.655+0000 7f575a99c700 1 -- 172.21.15.92:0/1266136942 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57500094a0 con 0x7f57540d0d60 2023-11-28T11:24:01.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.655+0000 7f575a99c700 1 --2- 172.21.15.92:0/1266136942 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57540d0d60 0x7f5754158e30 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f574c007e60 tx=0x7f574c00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:01.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.656+0000 7f574affd700 1 -- 172.21.15.92:0/1266136942 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f574c00ed30 con 0x7f57540d0d60 2023-11-28T11:24:01.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.656+0000 7f574affd700 1 -- 172.21.15.92:0/1266136942 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f574c014420 con 0x7f57540d0d60 2023-11-28T11:24:01.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.656+0000 7f575c3ff700 1 -- 172.21.15.92:0/1266136942 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57540cfff0 con 0x7f57540d0d60 2023-11-28T11:24:01.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.656+0000 7f574affd700 1 -- 172.21.15.92:0/1266136942 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f574c0147d0 con 0x7f57540d0d60 2023-11-28T11:24:01.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.656+0000 7f575c3ff700 1 -- 172.21.15.92:0/1266136942 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57540cfd10 con 0x7f57540d0d60 2023-11-28T11:24:01.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.657+0000 7f575c3ff700 1 -- 172.21.15.92:0/1266136942 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5754067880 con 0x7f57540d0d60 2023-11-28T11:24:01.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.658+0000 7f574affd700 1 -- 172.21.15.92:0/1266136942 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f574c012050 con 0x7f57540d0d60 2023-11-28T11:24:01.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.658+0000 7f574affd700 1 --2- 172.21.15.92:0/1266136942 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f57380682f0 0x7f573806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:01.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.659+0000 7f574affd700 1 -- 172.21.15.92:0/1266136942 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f574c0862f0 con 0x7f57540d0d60 2023-11-28T11:24:01.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.659+0000 7f575a19b700 1 --2- 172.21.15.92:0/1266136942 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f57380682f0 0x7f573806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:01.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.659+0000 7f575a19b700 1 --2- 172.21.15.92:0/1266136942 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f57380682f0 0x7f573806a7a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f57500090b0 tx=0x7f575001c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:01.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.663+0000 7f574affd700 1 -- 172.21.15.92:0/1266136942 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f574c052a10 con 0x7f57540d0d60 2023-11-28T11:24:01.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.893+0000 7f575c3ff700 1 -- 172.21.15.92:0/1266136942 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f57540d2940 con 0x7f57540d0d60 2023-11-28T11:24:01.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.894+0000 7f574affd700 1 -- 172.21.15.92:0/1266136942 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4651 (secure 0 0 0) 0x7f574c056030 con 0x7f57540d0d60 2023-11-28T11:24:01.898 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:24:01.898 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5593,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/3937189339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3937189339},{"type":"v1","addr":"172.21.15.169:6834","nonce":3937189339}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5596,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2020201016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2020201016},{"type":"v1","addr":"172.21.15.169:6835","nonce":2020201016}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5751,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2423750324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2423750324},{"type":"v1","addr":"172.21.15.92:6835","nonce":2423750324}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:23:13.247233+0000","modified":"2023-11-28T11:23:16.254256+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5754},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5754":{"gid":5754,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/950691230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":950691230},{"type":"v1","addr":"172.21.15.92:6837","nonce":950691230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T11:24:01.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.896+0000 7f575c3ff700 1 -- 172.21.15.92:0/1266136942 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f57380682f0 msgr2=0x7f573806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:01.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.896+0000 7f575c3ff700 1 --2- 172.21.15.92:0/1266136942 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f57380682f0 0x7f573806a7a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f57500090b0 tx=0x7f575001c040 comp rx=0 tx=0).stop 2023-11-28T11:24:01.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.896+0000 7f575c3ff700 1 -- 172.21.15.92:0/1266136942 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57540d0d60 msgr2=0x7f5754158e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:01.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.896+0000 7f575c3ff700 1 --2- 172.21.15.92:0/1266136942 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57540d0d60 0x7f5754158e30 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f574c007e60 tx=0x7f574c00e3c0 comp rx=0 tx=0).stop 2023-11-28T11:24:01.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.896+0000 7f575c3ff700 1 -- 172.21.15.92:0/1266136942 shutdown_connections 2023-11-28T11:24:01.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.896+0000 7f575c3ff700 1 --2- 172.21.15.92:0/1266136942 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f57380682f0 0x7f573806a7a0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:01.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.896+0000 7f575c3ff700 1 --2- 172.21.15.92:0/1266136942 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57540d0d60 0x7f5754158e30 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:01.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.896+0000 7f575c3ff700 1 --2- 172.21.15.92:0/1266136942 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57540b8510 0x7f57540a9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:01.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.896+0000 7f575c3ff700 1 --2- 172.21.15.92:0/1266136942 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f57540cd460 0x7f57540c9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:01.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.896+0000 7f575c3ff700 1 -- 172.21.15.92:0/1266136942 >> 172.21.15.92:0/1266136942 conn(0x7f57540719a0 msgr2=0x7f57540b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:01.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.896+0000 7f575c3ff700 1 -- 172.21.15.92:0/1266136942 shutdown_connections 2023-11-28T11:24:01.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:01.896+0000 7f575c3ff700 1 -- 172.21.15.92:0/1266136942 wait complete. 2023-11-28T11:24:01.901 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 24 2023-11-28T11:24:01.916 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:24:02.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.111+0000 7fef969ff700 1 Processor -- start 2023-11-28T11:24:02.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.111+0000 7fef969ff700 1 -- start start 2023-11-28T11:24:02.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.112+0000 7fef969ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fef900cd460 0x7fef9006c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:02.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.112+0000 7fef969ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fef900d0d60 0x7fef900d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:02.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.112+0000 7fef969ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef900b88e0 con 0x7fef900b8510 2023-11-28T11:24:02.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.112+0000 7fef969ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef900cdbd0 con 0x7fef900cd460 2023-11-28T11:24:02.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.112+0000 7fef969ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef900d02d0 con 0x7fef900d0d60 2023-11-28T11:24:02.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.112+0000 7fef8f7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fef900cd460 0x7fef9006c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:02.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.112+0000 7fef8f7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fef900cd460 0x7fef9006c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50204/0 (socket says 172.21.15.92:50204) 2023-11-28T11:24:02.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.112+0000 7fef8f7fe700 1 -- 172.21.15.92:0/2870601403 learned_addr learned my addr 172.21.15.92:0/2870601403 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:02.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.113+0000 7fef94f9c700 1 --2- 172.21.15.92:0/2870601403 >> v2:172.21.15.169:3300/0 conn(0x7fef900d0d60 0x7fef900d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:02.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.113+0000 7fef8f7fe700 1 -- 172.21.15.92:0/2870601403 >> v2:172.21.15.169:3300/0 conn(0x7fef900d0d60 msgr2=0x7fef900d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:02.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.113+0000 7fef8f7fe700 1 --2- 172.21.15.92:0/2870601403 >> v2:172.21.15.169:3300/0 conn(0x7fef900d0d60 0x7fef900d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:02.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.113+0000 7fef8f7fe700 1 -- 172.21.15.92:0/2870601403 >> v1:172.21.15.169:6789/0 conn(0x7fef900b8510 legacy=0x7fef9006b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:02.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.113+0000 7fef8f7fe700 1 -- 172.21.15.92:0/2870601403 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef900cd8f0 con 0x7fef900cd460 2023-11-28T11:24:02.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.113+0000 7fef94f9c700 1 --2- 172.21.15.92:0/2870601403 >> v2:172.21.15.169:3300/0 conn(0x7fef900d0d60 0x7fef900d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:24:02.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.113+0000 7fef8f7fe700 1 --2- 172.21.15.92:0/2870601403 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef900cd460 0x7fef9006c080 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fef900ce8c0 tx=0x7fef84009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=528e9a63d63b2830 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:02.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.113+0000 7fef8effd700 1 -- 172.21.15.92:0/2870601403 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8400b860 con 0x7fef900cd460 2023-11-28T11:24:02.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.113+0000 7fef8effd700 1 -- 172.21.15.92:0/2870601403 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef8400b9c0 con 0x7fef900cd460 2023-11-28T11:24:02.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.113+0000 7fef8effd700 1 -- 172.21.15.92:0/2870601403 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8400bd50 con 0x7fef900cd460 2023-11-28T11:24:02.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.114+0000 7fef969ff700 1 -- 172.21.15.92:0/2870601403 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef900cd460 msgr2=0x7fef9006c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:02.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.114+0000 7fef969ff700 1 --2- 172.21.15.92:0/2870601403 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef900cd460 0x7fef9006c080 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fef900ce8c0 tx=0x7fef84009c00 comp rx=0 tx=0).stop 2023-11-28T11:24:02.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.114+0000 7fef969ff700 1 -- 172.21.15.92:0/2870601403 shutdown_connections 2023-11-28T11:24:02.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.114+0000 7fef969ff700 1 --2- 172.21.15.92:0/2870601403 >> v2:172.21.15.169:3300/0 conn(0x7fef900d0d60 0x7fef900d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:02.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.114+0000 7fef969ff700 1 --2- 172.21.15.92:0/2870601403 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef900cd460 0x7fef9006c080 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:02.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.114+0000 7fef969ff700 1 -- 172.21.15.92:0/2870601403 >> 172.21.15.92:0/2870601403 conn(0x7fef900719a0 msgr2=0x7fef90071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:02.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.114+0000 7fef969ff700 1 -- 172.21.15.92:0/2870601403 shutdown_connections 2023-11-28T11:24:02.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.114+0000 7fef969ff700 1 -- 172.21.15.92:0/2870601403 wait complete. 2023-11-28T11:24:02.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.116+0000 7fef969ff700 1 Processor -- start 2023-11-28T11:24:02.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.116+0000 7fef969ff700 1 -- start start 2023-11-28T11:24:02.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.117+0000 7fef969ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef900b8510 0x7fef901e3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:02.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.117+0000 7fef969ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef900cd460 0x7fef9006bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:02.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.117+0000 7fef969ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef900d0d60 0x7fef901e1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:02.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.117+0000 7fef969ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef901e78b0 con 0x7fef900d0d60 2023-11-28T11:24:02.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.117+0000 7fef969ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef901e6a00 con 0x7fef900cd460 2023-11-28T11:24:02.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.117+0000 7fef969ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef901e6ca0 con 0x7fef900b8510 2023-11-28T11:24:02.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.117+0000 7fef94f9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef900d0d60 0x7fef901e1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:02.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.117+0000 7fef94f9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef900d0d60 0x7fef901e1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50218/0 (socket says 172.21.15.92:50218) 2023-11-28T11:24:02.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.117+0000 7fef94f9c700 1 -- 172.21.15.92:0/3593276139 learned_addr learned my addr 172.21.15.92:0/3593276139 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:02.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.118+0000 7fef8f7fe700 1 --2- 172.21.15.92:0/3593276139 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef900cd460 0x7fef9006bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:02.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.118+0000 7fef8ffff700 1 --2- 172.21.15.92:0/3593276139 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef900b8510 0x7fef901e3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:02.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.118+0000 7fef94f9c700 1 -- 172.21.15.92:0/3593276139 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef900b8510 msgr2=0x7fef901e3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:02.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.118+0000 7fef94f9c700 1 --2- 172.21.15.92:0/3593276139 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef900b8510 0x7fef901e3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:02.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.118+0000 7fef94f9c700 1 -- 172.21.15.92:0/3593276139 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef900cd460 msgr2=0x7fef9006bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:02.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.118+0000 7fef94f9c700 1 --2- 172.21.15.92:0/3593276139 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef900cd460 0x7fef9006bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:02.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.118+0000 7fef94f9c700 1 -- 172.21.15.92:0/3593276139 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef84009550 con 0x7fef900d0d60 2023-11-28T11:24:02.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.118+0000 7fef8f7fe700 1 --2- 172.21.15.92:0/3593276139 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef900cd460 0x7fef9006bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:24:02.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.118+0000 7fef8ffff700 1 --2- 172.21.15.92:0/3593276139 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef900b8510 0x7fef901e3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:24:02.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.118+0000 7fef94f9c700 1 --2- 172.21.15.92:0/3593276139 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef900d0d60 0x7fef901e1900 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fef88007df0 tx=0x7fef8800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:02.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.118+0000 7fef8cff9700 1 -- 172.21.15.92:0/3593276139 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8800ddd0 con 0x7fef900d0d60 2023-11-28T11:24:02.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.118+0000 7fef8cff9700 1 -- 172.21.15.92:0/3593276139 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef8800f040 con 0x7fef900d0d60 2023-11-28T11:24:02.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.118+0000 7fef8cff9700 1 -- 172.21.15.92:0/3593276139 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef88003c60 con 0x7fef900d0d60 2023-11-28T11:24:02.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.118+0000 7fef969ff700 1 -- 172.21.15.92:0/3593276139 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef900cfff0 con 0x7fef900d0d60 2023-11-28T11:24:02.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.118+0000 7fef969ff700 1 -- 172.21.15.92:0/3593276139 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef900cfd10 con 0x7fef900d0d60 2023-11-28T11:24:02.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.119+0000 7fef969ff700 1 -- 172.21.15.92:0/3593276139 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef90067880 con 0x7fef900d0d60 2023-11-28T11:24:02.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.120+0000 7fef8cff9700 1 -- 172.21.15.92:0/3593276139 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fef8801c030 con 0x7fef900d0d60 2023-11-28T11:24:02.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.121+0000 7fef8cff9700 1 --2- 172.21.15.92:0/3593276139 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fef7c0682f0 0x7fef7c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:02.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.121+0000 7fef8cff9700 1 -- 172.21.15.92:0/3593276139 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fef88085ec0 con 0x7fef900d0d60 2023-11-28T11:24:02.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.121+0000 7fef8ffff700 1 --2- 172.21.15.92:0/3593276139 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fef7c0682f0 0x7fef7c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:02.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.122+0000 7fef8ffff700 1 --2- 172.21.15.92:0/3593276139 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fef7c0682f0 0x7fef7c06a7a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fef80000f60 tx=0x7fef80009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:02.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.125+0000 7fef8cff9700 1 -- 172.21.15.92:0/3593276139 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fef880525e0 con 0x7fef900d0d60 2023-11-28T11:24:02.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.358+0000 7fef969ff700 1 -- 172.21.15.92:0/3593276139 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef900d2940 con 0x7fef900d0d60 2023-11-28T11:24:02.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.359+0000 7fef8cff9700 1 -- 172.21.15.92:0/3593276139 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4651 (secure 0 0 0) 0x7fef8801a090 con 0x7fef900d0d60 2023-11-28T11:24:02.360 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:24:02.360 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5593,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/3937189339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3937189339},{"type":"v1","addr":"172.21.15.169:6834","nonce":3937189339}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5596,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2020201016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2020201016},{"type":"v1","addr":"172.21.15.169:6835","nonce":2020201016}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5751,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2423750324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2423750324},{"type":"v1","addr":"172.21.15.92:6835","nonce":2423750324}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:23:13.247233+0000","modified":"2023-11-28T11:23:16.254256+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5754},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5754":{"gid":5754,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/950691230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":950691230},{"type":"v1","addr":"172.21.15.92:6837","nonce":950691230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T11:24:02.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.361+0000 7fef969ff700 1 -- 172.21.15.92:0/3593276139 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fef7c0682f0 msgr2=0x7fef7c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:02.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.361+0000 7fef969ff700 1 --2- 172.21.15.92:0/3593276139 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fef7c0682f0 0x7fef7c06a7a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fef80000f60 tx=0x7fef80009040 comp rx=0 tx=0).stop 2023-11-28T11:24:02.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.361+0000 7fef969ff700 1 -- 172.21.15.92:0/3593276139 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef900d0d60 msgr2=0x7fef901e1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:02.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.361+0000 7fef969ff700 1 --2- 172.21.15.92:0/3593276139 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef900d0d60 0x7fef901e1900 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fef88007df0 tx=0x7fef8800d3b0 comp rx=0 tx=0).stop 2023-11-28T11:24:02.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.361+0000 7fef969ff700 1 -- 172.21.15.92:0/3593276139 shutdown_connections 2023-11-28T11:24:02.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.361+0000 7fef969ff700 1 --2- 172.21.15.92:0/3593276139 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fef7c0682f0 0x7fef7c06a7a0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:02.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.361+0000 7fef969ff700 1 --2- 172.21.15.92:0/3593276139 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef900d0d60 0x7fef901e1900 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:02.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.361+0000 7fef969ff700 1 --2- 172.21.15.92:0/3593276139 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fef900b8510 0x7fef901e3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:02.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.361+0000 7fef969ff700 1 --2- 172.21.15.92:0/3593276139 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fef900cd460 0x7fef9006bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:02.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.361+0000 7fef969ff700 1 -- 172.21.15.92:0/3593276139 >> 172.21.15.92:0/3593276139 conn(0x7fef900719a0 msgr2=0x7fef900c9710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:02.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.362+0000 7fef969ff700 1 -- 172.21.15.92:0/3593276139 shutdown_connections 2023-11-28T11:24:02.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.362+0000 7fef969ff700 1 -- 172.21.15.92:0/3593276139 wait complete. 2023-11-28T11:24:02.364 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 24 2023-11-28T11:24:02.381 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:24:02.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.576+0000 7f4b721ff700 1 Processor -- start 2023-11-28T11:24:02.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.576+0000 7f4b721ff700 1 -- start start 2023-11-28T11:24:02.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.577+0000 7f4b721ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4b6c0b8510 0x7f4b6c06b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:02.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.577+0000 7f4b721ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4b6c0cd460 0x7f4b6c06beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:02.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.577+0000 7f4b721ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b6c0b88e0 con 0x7f4b6c06c3f0 2023-11-28T11:24:02.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.577+0000 7f4b721ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b6c0cdbd0 con 0x7f4b6c0cd460 2023-11-28T11:24:02.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.577+0000 7f4b721ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b6c0d02d0 con 0x7f4b6c0b8510 2023-11-28T11:24:02.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.577+0000 7f4b6affd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4b6c0cd460 0x7f4b6c06beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:02.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.577+0000 7f4b6bfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f4b6c06c3f0 0x7f4b6c15cc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55204/0 (socket says 172.21.15.92:55204) 2023-11-28T11:24:02.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.577+0000 7f4b6bfff700 1 -- 172.21.15.92:0/3868737926 learned_addr learned my addr 172.21.15.92:0/3868737926 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:24:02.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.578+0000 7f4b6affd700 1 -- 172.21.15.92:0/3868737926 >> v2:172.21.15.169:3300/0 conn(0x7f4b6c0b8510 msgr2=0x7f4b6c06b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:02.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.578+0000 7f4b6b7fe700 1 --2- 172.21.15.92:0/3868737926 >> v2:172.21.15.169:3300/0 conn(0x7f4b6c0b8510 0x7f4b6c06b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:02.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.578+0000 7f4b6affd700 1 --2- 172.21.15.92:0/3868737926 >> v2:172.21.15.169:3300/0 conn(0x7f4b6c0b8510 0x7f4b6c06b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:02.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.578+0000 7f4b6affd700 1 -- 172.21.15.92:0/3868737926 >> v1:172.21.15.92:6789/0 conn(0x7f4b6c06c3f0 legacy=0x7f4b6c15cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:02.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.578+0000 7f4b6affd700 1 -- 172.21.15.92:0/3868737926 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b6c0cd8f0 con 0x7f4b6c0cd460 2023-11-28T11:24:02.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.578+0000 7f4b6a7fc700 1 -- 172.21.15.92:0/3868737926 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2161226908 0 0) 0x7f4b6c0b88e0 con 0x7f4b6c06c3f0 2023-11-28T11:24:02.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.578+0000 7f4b6b7fe700 1 --2- 172.21.15.92:0/3868737926 >> v2:172.21.15.169:3300/0 conn(0x7f4b6c0b8510 0x7f4b6c06b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:24:02.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.578+0000 7f4b6affd700 1 --2- 172.21.15.92:0/3868737926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b6c0cd460 0x7f4b6c06beb0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f4b60009870 tx=0x7f4b60009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebdbe364b2efec9e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:02.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.578+0000 7f4b6a7fc700 1 -- 172.21.15.92:0/3868737926 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b60011830 con 0x7f4b6c0cd460 2023-11-28T11:24:02.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.578+0000 7f4b6a7fc700 1 -- 172.21.15.92:0/3868737926 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b60011990 con 0x7f4b6c0cd460 2023-11-28T11:24:02.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.578+0000 7f4b6a7fc700 1 -- 172.21.15.92:0/3868737926 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b60011d20 con 0x7f4b6c0cd460 2023-11-28T11:24:02.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.579+0000 7f4b721ff700 1 -- 172.21.15.92:0/3868737926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b6c0cd460 msgr2=0x7f4b6c06beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:02.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.579+0000 7f4b721ff700 1 --2- 172.21.15.92:0/3868737926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b6c0cd460 0x7f4b6c06beb0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f4b60009870 tx=0x7f4b60009b80 comp rx=0 tx=0).stop 2023-11-28T11:24:02.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.579+0000 7f4b721ff700 1 -- 172.21.15.92:0/3868737926 shutdown_connections 2023-11-28T11:24:02.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.579+0000 7f4b721ff700 1 --2- 172.21.15.92:0/3868737926 >> v2:172.21.15.169:3300/0 conn(0x7f4b6c0b8510 0x7f4b6c06b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:02.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.579+0000 7f4b721ff700 1 --2- 172.21.15.92:0/3868737926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b6c0cd460 0x7f4b6c06beb0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:02.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.579+0000 7f4b721ff700 1 -- 172.21.15.92:0/3868737926 >> 172.21.15.92:0/3868737926 conn(0x7f4b6c0719a0 msgr2=0x7f4b6c071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:02.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.579+0000 7f4b721ff700 1 -- 172.21.15.92:0/3868737926 shutdown_connections 2023-11-28T11:24:02.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.579+0000 7f4b721ff700 1 -- 172.21.15.92:0/3868737926 wait complete. 2023-11-28T11:24:02.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.581+0000 7f4b721ff700 1 Processor -- start 2023-11-28T11:24:02.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.581+0000 7f4b721ff700 1 -- start start 2023-11-28T11:24:02.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.582+0000 7f4b721ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b6c06c3f0 0x7f4b6c155840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:02.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.582+0000 7f4b721ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b6c0b8510 0x7f4b6c0cc130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:02.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.582+0000 7f4b721ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b6c0cd460 0x7f4b6c0d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:02.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.582+0000 7f4b721ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6c15b610 con 0x7f4b6c0cd460 2023-11-28T11:24:02.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.582+0000 7f4b721ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6c155d80 con 0x7f4b6c06c3f0 2023-11-28T11:24:02.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.582+0000 7f4b721ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6c0cc670 con 0x7f4b6c0b8510 2023-11-28T11:24:02.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.582+0000 7f4b6bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b6c0cd460 0x7f4b6c0d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:02.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.582+0000 7f4b6bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b6c0cd460 0x7f4b6c0d0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50234/0 (socket says 172.21.15.92:50234) 2023-11-28T11:24:02.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.582+0000 7f4b6bfff700 1 -- 172.21.15.92:0/39797249 learned_addr learned my addr 172.21.15.92:0/39797249 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:02.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.582+0000 7f4b6b7fe700 1 --2- 172.21.15.92:0/39797249 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b6c06c3f0 0x7f4b6c155840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:02.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.582+0000 7f4b6affd700 1 --2- 172.21.15.92:0/39797249 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b6c0b8510 0x7f4b6c0cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:02.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.582+0000 7f4b6bfff700 1 -- 172.21.15.92:0/39797249 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b6c0b8510 msgr2=0x7f4b6c0cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:02.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.582+0000 7f4b6bfff700 1 --2- 172.21.15.92:0/39797249 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b6c0b8510 0x7f4b6c0cc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:02.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.582+0000 7f4b6bfff700 1 -- 172.21.15.92:0/39797249 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b6c06c3f0 msgr2=0x7f4b6c155840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:02.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.582+0000 7f4b6bfff700 1 --2- 172.21.15.92:0/39797249 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b6c06c3f0 0x7f4b6c155840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:02.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.582+0000 7f4b6bfff700 1 -- 172.21.15.92:0/39797249 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b600094a0 con 0x7f4b6c0cd460 2023-11-28T11:24:02.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.583+0000 7f4b6bfff700 1 --2- 172.21.15.92:0/39797249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b6c0cd460 0x7f4b6c0d0c10 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f4b64002230 tx=0x7f4b6400cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:02.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.583+0000 7f4b53fff700 1 -- 172.21.15.92:0/39797249 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6400aac0 con 0x7f4b6c0cd460 2023-11-28T11:24:02.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.583+0000 7f4b53fff700 1 -- 172.21.15.92:0/39797249 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b6400ac40 con 0x7f4b6c0cd460 2023-11-28T11:24:02.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.583+0000 7f4b721ff700 1 -- 172.21.15.92:0/39797249 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b6c0cfff0 con 0x7f4b6c0cd460 2023-11-28T11:24:02.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.583+0000 7f4b53fff700 1 -- 172.21.15.92:0/39797249 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b640070d0 con 0x7f4b6c0cd460 2023-11-28T11:24:02.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.583+0000 7f4b721ff700 1 -- 172.21.15.92:0/39797249 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b6c0cfd10 con 0x7f4b6c0cd460 2023-11-28T11:24:02.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.584+0000 7f4b721ff700 1 -- 172.21.15.92:0/39797249 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b6c067880 con 0x7f4b6c0cd460 2023-11-28T11:24:02.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.585+0000 7f4b53fff700 1 -- 172.21.15.92:0/39797249 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4b6401a030 con 0x7f4b6c0cd460 2023-11-28T11:24:02.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.586+0000 7f4b53fff700 1 --2- 172.21.15.92:0/39797249 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f4b580682f0 0x7f4b5806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:02.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.586+0000 7f4b53fff700 1 -- 172.21.15.92:0/39797249 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f4b64085ad0 con 0x7f4b6c0cd460 2023-11-28T11:24:02.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.586+0000 7f4b6b7fe700 1 --2- 172.21.15.92:0/39797249 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f4b580682f0 0x7f4b5806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:02.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.587+0000 7f4b6b7fe700 1 --2- 172.21.15.92:0/39797249 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f4b580682f0 0x7f4b5806a7a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c006fd0 tx=0x7f4b5c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:02.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.590+0000 7f4b53fff700 1 -- 172.21.15.92:0/39797249 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4b64052100 con 0x7f4b6c0cd460 2023-11-28T11:24:02.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.785+0000 7f4b721ff700 1 -- 172.21.15.92:0/39797249 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4b6c0d2940 con 0x7f4b6c0cd460 2023-11-28T11:24:02.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.786+0000 7f4b53fff700 1 -- 172.21.15.92:0/39797249 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+18180 (secure 0 0 0) 0x7f4b64018090 con 0x7f4b6c0cd460 2023-11-28T11:24:02.787 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:24:02.787 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":56,"fsid":"68956e0c-c16f-47cd-a220-f98364b0831d","created":"2023-11-28T11:20:28.697573+0000","modified":"2023-11-28T11:23:14.249429+0000","last_up_change":"2023-11-28T11:20:41.257011+0000","last_in_change":"2023-11-28T11:20:32.249767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:20:41.925237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:20:48.682497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:23:10.939884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-28T11:23:12.227433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6abe483f-0e03-4a26-b9cf-62041291747e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":115823},{"type":"v1","addr":"172.21.15.92:6809","nonce":115823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":115823},{"type":"v1","addr":"172.21.15.92:6813","nonce":115823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":115823},{"type":"v1","addr":"172.21.15.92:6821","nonce":115823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":115823},{"type":"v1","addr":"172.21.15.92:6817","nonce":115823}]},"public_addr":"172.21.15.92:6809/115823","cluster_addr":"172.21.15.92:6813/115823","heartbeat_back_addr":"172.21.15.92:6821/115823","heartbeat_front_addr":"172.21.15.92:6817/115823","state":["exists","up"]},{"osd":1,"uuid":"13c1de73-1a8c-4d7e-8a28-9dae52dfe089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":115824},{"type":"v1","addr":"172.21.15.92:6812","nonce":115824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":115824},{"type":"v1","addr":"172.21.15.92:6816","nonce":115824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":115824},{"type":"v1","addr":"172.21.15.92:6823","nonce":115824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":115824},{"type":"v1","addr":"172.21.15.92:6820","nonce":115824}]},"public_addr":"172.21.15.92:6812/115824","cluster_addr":"172.21.15.92:6816/115824","heartbeat_back_addr":"172.21.15.92:6823/115824","heartbeat_front_addr":"172.21.15.92:6820/115824","state":["exists","up"]},{"osd":2,"uuid":"25788a55-112f-4f00-810d-2bc37cf76778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":115821},{"type":"v1","addr":"172.21.15.92:6801","nonce":115821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":115821},{"type":"v1","addr":"172.21.15.92:6803","nonce":115821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":115821},{"type":"v1","addr":"172.21.15.92:6807","nonce":115821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":115821},{"type":"v1","addr":"172.21.15.92:6805","nonce":115821}]},"public_addr":"172.21.15.92:6801/115821","cluster_addr":"172.21.15.92:6803/115821","heartbeat_back_addr":"172.21.15.92:6807/115821","heartbeat_front_addr":"172.21.15.92:6805/115821","state":["exists","up"]},{"osd":3,"uuid":"adf54e24-33a9-41fe-9ce3-98a5ab6c158e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":115825},{"type":"v1","addr":"172.21.15.92:6825","nonce":115825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":115825},{"type":"v1","addr":"172.21.15.92:6827","nonce":115825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":115825},{"type":"v1","addr":"172.21.15.92:6831","nonce":115825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":115825},{"type":"v1","addr":"172.21.15.92:6829","nonce":115825}]},"public_addr":"172.21.15.92:6825/115825","cluster_addr":"172.21.15.92:6827/115825","heartbeat_back_addr":"172.21.15.92:6831/115825","heartbeat_front_addr":"172.21.15.92:6829/115825","state":["exists","up"]},{"osd":4,"uuid":"ccb9b747-f399-44ea-9a18-9db3152227e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":116035},{"type":"v1","addr":"172.21.15.169:6817","nonce":116035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":116035},{"type":"v1","addr":"172.21.15.169:6819","nonce":116035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":116035},{"type":"v1","addr":"172.21.15.169:6826","nonce":116035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":116035},{"type":"v1","addr":"172.21.15.169:6822","nonce":116035}]},"public_addr":"172.21.15.169:6817/116035","cluster_addr":"172.21.15.169:6819/116035","heartbeat_back_addr":"172.21.15.169:6826/116035","heartbeat_front_addr":"172.21.15.169:6822/116035","state":["exists","up"]},{"osd":5,"uuid":"308a1e63-75aa-4d7d-8a7c-cecc24714fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":116033},{"type":"v1","addr":"172.21.15.169:6801","nonce":116033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":116033},{"type":"v1","addr":"172.21.15.169:6803","nonce":116033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":116033},{"type":"v1","addr":"172.21.15.169:6807","nonce":116033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":116033},{"type":"v1","addr":"172.21.15.169:6805","nonce":116033}]},"public_addr":"172.21.15.169:6801/116033","cluster_addr":"172.21.15.169:6803/116033","heartbeat_back_addr":"172.21.15.169:6807/116033","heartbeat_front_addr":"172.21.15.169:6805/116033","state":["exists","up"]},{"osd":6,"uuid":"9c459b7b-b821-4b4b-b91e-6aaae271457c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":116034},{"type":"v1","addr":"172.21.15.169:6809","nonce":116034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":116034},{"type":"v1","addr":"172.21.15.169:6811","nonce":116034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":116034},{"type":"v1","addr":"172.21.15.169:6815","nonce":116034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":116034},{"type":"v1","addr":"172.21.15.169:6813","nonce":116034}]},"public_addr":"172.21.15.169:6809/116034","cluster_addr":"172.21.15.169:6811/116034","heartbeat_back_addr":"172.21.15.169:6815/116034","heartbeat_front_addr":"172.21.15.169:6813/116034","state":["exists","up"]},{"osd":7,"uuid":"1ca9aa4a-755d-4e28-a937-4d8dccab4df3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":116044},{"type":"v1","addr":"172.21.15.169:6823","nonce":116044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6825","nonce":116044},{"type":"v1","addr":"172.21.15.169:6827","nonce":116044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":116044},{"type":"v1","addr":"172.21.15.169:6831","nonce":116044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":116044},{"type":"v1","addr":"172.21.15.169:6829","nonce":116044}]},"public_addr":"172.21.15.169:6823/116044","cluster_addr":"172.21.15.169:6827/116044","heartbeat_back_addr":"172.21.15.169:6831/116044","heartbeat_front_addr":"172.21.15.169:6829/116044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.586781+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.869778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.508594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.860493+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.872267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.746933+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.611520+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.871446+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:24:02.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.788+0000 7f4b721ff700 1 -- 172.21.15.92:0/39797249 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f4b580682f0 msgr2=0x7f4b5806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:02.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.788+0000 7f4b721ff700 1 --2- 172.21.15.92:0/39797249 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f4b580682f0 0x7f4b5806a7a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c006fd0 tx=0x7f4b5c008040 comp rx=0 tx=0).stop 2023-11-28T11:24:02.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.788+0000 7f4b721ff700 1 -- 172.21.15.92:0/39797249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b6c0cd460 msgr2=0x7f4b6c0d0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:02.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.788+0000 7f4b721ff700 1 --2- 172.21.15.92:0/39797249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b6c0cd460 0x7f4b6c0d0c10 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f4b64002230 tx=0x7f4b6400cf50 comp rx=0 tx=0).stop 2023-11-28T11:24:02.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.789+0000 7f4b721ff700 1 -- 172.21.15.92:0/39797249 shutdown_connections 2023-11-28T11:24:02.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.789+0000 7f4b721ff700 1 --2- 172.21.15.92:0/39797249 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f4b580682f0 0x7f4b5806a7a0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:02.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.789+0000 7f4b721ff700 1 --2- 172.21.15.92:0/39797249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4b6c0cd460 0x7f4b6c0d0c10 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:02.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.789+0000 7f4b721ff700 1 --2- 172.21.15.92:0/39797249 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4b6c06c3f0 0x7f4b6c155840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:02.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.789+0000 7f4b721ff700 1 --2- 172.21.15.92:0/39797249 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4b6c0b8510 0x7f4b6c0cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:02.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.789+0000 7f4b721ff700 1 -- 172.21.15.92:0/39797249 >> 172.21.15.92:0/39797249 conn(0x7f4b6c0719a0 msgr2=0x7f4b6c0c9680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:02.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.789+0000 7f4b721ff700 1 -- 172.21.15.92:0/39797249 shutdown_connections 2023-11-28T11:24:02.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:02.789+0000 7f4b721ff700 1 -- 172.21.15.92:0/39797249 wait complete. 2023-11-28T11:24:02.809 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T11:24:03.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.006+0000 7f8d7d9ff700 1 Processor -- start 2023-11-28T11:24:03.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.006+0000 7f8d7d9ff700 1 -- start start 2023-11-28T11:24:03.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.007+0000 7f8d7d9ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d7815a110 0x7f8d78160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:03.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.007+0000 7f8d7d9ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f8d7815d820 0x7f8d780cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:03.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.007+0000 7f8d7d9ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d78161070 con 0x7f8d780cfcf0 2023-11-28T11:24:03.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.007+0000 7f8d7d9ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d780cdae0 con 0x7f8d7815d820 2023-11-28T11:24:03.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.007+0000 7f8d7d9ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d780ce4f0 con 0x7f8d7815a110 2023-11-28T11:24:03.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.007+0000 7f8d777fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f8d780cfcf0 0x7f8d780b19b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55212/0 (socket says 172.21.15.92:55212) 2023-11-28T11:24:03.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.007+0000 7f8d777fe700 1 -- 172.21.15.92:0/2180800427 learned_addr learned my addr 172.21.15.92:0/2180800427 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:24:03.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.007+0000 7f8d76ffd700 1 --2- 172.21.15.92:0/2180800427 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d7815a110 0x7f8d78160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:03.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.007+0000 7f8d6ffff700 1 --2- 172.21.15.92:0/2180800427 >> v2:172.21.15.169:3300/0 conn(0x7f8d7815d820 0x7f8d780cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:03.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.008+0000 7f8d767fc700 1 -- 172.21.15.92:0/2180800427 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 517648865 0 0) 0x7f8d78161070 con 0x7f8d780cfcf0 2023-11-28T11:24:03.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.008+0000 7f8d767fc700 1 -- 172.21.15.92:0/2180800427 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d640035b0 con 0x7f8d780cfcf0 2023-11-28T11:24:03.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.008+0000 7f8d76ffd700 1 -- 172.21.15.92:0/2180800427 >> v2:172.21.15.169:3300/0 conn(0x7f8d7815d820 msgr2=0x7f8d780cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:03.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.008+0000 7f8d76ffd700 1 --2- 172.21.15.92:0/2180800427 >> v2:172.21.15.169:3300/0 conn(0x7f8d7815d820 0x7f8d780cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:03.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.008+0000 7f8d767fc700 1 -- 172.21.15.92:0/2180800427 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1779484596 0 0) 0x7f8d640035b0 con 0x7f8d780cfcf0 2023-11-28T11:24:03.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.008+0000 7f8d76ffd700 1 -- 172.21.15.92:0/2180800427 >> v1:172.21.15.92:6789/0 conn(0x7f8d780cfcf0 legacy=0x7f8d780b19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:03.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.008+0000 7f8d76ffd700 1 -- 172.21.15.92:0/2180800427 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d780d0170 con 0x7f8d7815a110 2023-11-28T11:24:03.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.008+0000 7f8d6ffff700 1 --2- 172.21.15.92:0/2180800427 >> v2:172.21.15.169:3300/0 conn(0x7f8d7815d820 0x7f8d780cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:24:03.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.008+0000 7f8d76ffd700 1 --2- 172.21.15.92:0/2180800427 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d7815a110 0x7f8d78160b30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8d68009840 tx=0x7f8d68009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4cc127ee8745ebc5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:03.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.009+0000 7f8d767fc700 1 -- 172.21.15.92:0/2180800427 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d700019f0 con 0x7f8d7815a110 2023-11-28T11:24:03.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.009+0000 7f8d767fc700 1 -- 172.21.15.92:0/2180800427 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d68004030 con 0x7f8d7815a110 2023-11-28T11:24:03.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.009+0000 7f8d767fc700 1 -- 172.21.15.92:0/2180800427 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d680043c0 con 0x7f8d7815a110 2023-11-28T11:24:03.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.009+0000 7f8d7d9ff700 1 -- 172.21.15.92:0/2180800427 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d7815a110 msgr2=0x7f8d78160b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:03.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.009+0000 7f8d7d9ff700 1 --2- 172.21.15.92:0/2180800427 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d7815a110 0x7f8d78160b30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8d68009840 tx=0x7f8d68009b50 comp rx=0 tx=0).stop 2023-11-28T11:24:03.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.009+0000 7f8d7d9ff700 1 -- 172.21.15.92:0/2180800427 shutdown_connections 2023-11-28T11:24:03.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.009+0000 7f8d7d9ff700 1 --2- 172.21.15.92:0/2180800427 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d7815a110 0x7f8d78160b30 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:03.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.009+0000 7f8d7d9ff700 1 --2- 172.21.15.92:0/2180800427 >> v2:172.21.15.169:3300/0 conn(0x7f8d7815d820 0x7f8d780cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:03.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.009+0000 7f8d7d9ff700 1 -- 172.21.15.92:0/2180800427 >> 172.21.15.92:0/2180800427 conn(0x7f8d780719a0 msgr2=0x7f8d78071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:03.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.009+0000 7f8d7d9ff700 1 -- 172.21.15.92:0/2180800427 shutdown_connections 2023-11-28T11:24:03.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.009+0000 7f8d7d9ff700 1 -- 172.21.15.92:0/2180800427 wait complete. 2023-11-28T11:24:03.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.011+0000 7f8d7d9ff700 1 Processor -- start 2023-11-28T11:24:03.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.011+0000 7f8d7d9ff700 1 -- start start 2023-11-28T11:24:03.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.012+0000 7f8d7d9ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d780cfcf0 0x7f8d78158b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:03.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.012+0000 7f8d7d9ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d7815a110 0x7f8d781e86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:03.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.012+0000 7f8d7d9ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d7815d820 0x7f8d780b3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:03.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.012+0000 7f8d7d9ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d780b6890 con 0x7f8d780cfcf0 2023-11-28T11:24:03.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.012+0000 7f8d7d9ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d781590d0 con 0x7f8d7815d820 2023-11-28T11:24:03.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.012+0000 7f8d7d9ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d780c69e0 con 0x7f8d7815a110 2023-11-28T11:24:03.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.012+0000 7f8d76ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d780cfcf0 0x7f8d78158b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:03.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.012+0000 7f8d76ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d780cfcf0 0x7f8d78158b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50248/0 (socket says 172.21.15.92:50248) 2023-11-28T11:24:03.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.012+0000 7f8d76ffd700 1 -- 172.21.15.92:0/818894987 learned_addr learned my addr 172.21.15.92:0/818894987 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:03.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.012+0000 7f8d6ffff700 1 --2- 172.21.15.92:0/818894987 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d7815a110 0x7f8d781e86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:03.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.012+0000 7f8d777fe700 1 --2- 172.21.15.92:0/818894987 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d7815d820 0x7f8d780b3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:03.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.013+0000 7f8d76ffd700 1 -- 172.21.15.92:0/818894987 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d7815a110 msgr2=0x7f8d781e86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:03.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.013+0000 7f8d76ffd700 1 --2- 172.21.15.92:0/818894987 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d7815a110 0x7f8d781e86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:03.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.013+0000 7f8d76ffd700 1 -- 172.21.15.92:0/818894987 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d7815d820 msgr2=0x7f8d780b3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:03.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.013+0000 7f8d76ffd700 1 --2- 172.21.15.92:0/818894987 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d7815d820 0x7f8d780b3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:03.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.013+0000 7f8d76ffd700 1 -- 172.21.15.92:0/818894987 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d600094a0 con 0x7f8d780cfcf0 2023-11-28T11:24:03.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.013+0000 7f8d76ffd700 1 --2- 172.21.15.92:0/818894987 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d780cfcf0 0x7f8d78158b90 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f8d780ced60 tx=0x7f8d68004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:03.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.013+0000 7f8d74ff9700 1 -- 172.21.15.92:0/818894987 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d68003b10 con 0x7f8d780cfcf0 2023-11-28T11:24:03.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.013+0000 7f8d74ff9700 1 -- 172.21.15.92:0/818894987 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d6801f070 con 0x7f8d780cfcf0 2023-11-28T11:24:03.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.013+0000 7f8d7d9ff700 1 -- 172.21.15.92:0/818894987 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d680094a0 con 0x7f8d780cfcf0 2023-11-28T11:24:03.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.013+0000 7f8d74ff9700 1 -- 172.21.15.92:0/818894987 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d68011630 con 0x7f8d780cfcf0 2023-11-28T11:24:03.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.013+0000 7f8d7d9ff700 1 -- 172.21.15.92:0/818894987 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d780cd800 con 0x7f8d780cfcf0 2023-11-28T11:24:03.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.014+0000 7f8d7d9ff700 1 -- 172.21.15.92:0/818894987 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d780cf2d0 con 0x7f8d780cfcf0 2023-11-28T11:24:03.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.015+0000 7f8d74ff9700 1 -- 172.21.15.92:0/818894987 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8d68017020 con 0x7f8d780cfcf0 2023-11-28T11:24:03.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.016+0000 7f8d74ff9700 1 --2- 172.21.15.92:0/818894987 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8d580682f0 0x7f8d5806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:03.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.016+0000 7f8d74ff9700 1 -- 172.21.15.92:0/818894987 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f8d68087290 con 0x7f8d780cfcf0 2023-11-28T11:24:03.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.016+0000 7f8d6ffff700 1 --2- 172.21.15.92:0/818894987 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8d580682f0 0x7f8d5806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:03.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.017+0000 7f8d6ffff700 1 --2- 172.21.15.92:0/818894987 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8d580682f0 0x7f8d5806a7a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8d780b1f00 tx=0x7f8d60009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:03.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.020+0000 7f8d74ff9700 1 -- 172.21.15.92:0/818894987 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8d680539b0 con 0x7f8d780cfcf0 2023-11-28T11:24:03.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.252+0000 7f8d7d9ff700 1 -- 172.21.15.92:0/818894987 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f8d7815cd40 con 0x7f8d780cfcf0 2023-11-28T11:24:03.637 INFO:tasks.ceph.mon.a.smithi092.stderr:2023-11-28T11:24:03.636+0000 7f086abf9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T11:24:03.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.639+0000 7f8d74ff9700 1 -- 172.21.15.92:0/818894987 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v25) v1 ==== 161+0+0 (secure 0 0 0) 0x7f8d68010380 con 0x7f8d780cfcf0 2023-11-28T11:24:03.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.642+0000 7f8d7d9ff700 1 -- 172.21.15.92:0/818894987 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8d580682f0 msgr2=0x7f8d5806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:03.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.642+0000 7f8d7d9ff700 1 --2- 172.21.15.92:0/818894987 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8d580682f0 0x7f8d5806a7a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8d780b1f00 tx=0x7f8d60009210 comp rx=0 tx=0).stop 2023-11-28T11:24:03.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.642+0000 7f8d7d9ff700 1 -- 172.21.15.92:0/818894987 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d780cfcf0 msgr2=0x7f8d78158b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:03.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.642+0000 7f8d7d9ff700 1 --2- 172.21.15.92:0/818894987 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d780cfcf0 0x7f8d78158b90 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f8d780ced60 tx=0x7f8d68004750 comp rx=0 tx=0).stop 2023-11-28T11:24:03.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.642+0000 7f8d7d9ff700 1 -- 172.21.15.92:0/818894987 shutdown_connections 2023-11-28T11:24:03.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.642+0000 7f8d7d9ff700 1 --2- 172.21.15.92:0/818894987 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8d7815d820 0x7f8d780b3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:03.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.642+0000 7f8d7d9ff700 1 --2- 172.21.15.92:0/818894987 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8d580682f0 0x7f8d5806a7a0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:03.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.642+0000 7f8d7d9ff700 1 --2- 172.21.15.92:0/818894987 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d780cfcf0 0x7f8d78158b90 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:03.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.642+0000 7f8d7d9ff700 1 --2- 172.21.15.92:0/818894987 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8d7815a110 0x7f8d781e86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:03.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.642+0000 7f8d7d9ff700 1 -- 172.21.15.92:0/818894987 >> 172.21.15.92:0/818894987 conn(0x7f8d780719a0 msgr2=0x7f8d78160790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:03.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.642+0000 7f8d7d9ff700 1 -- 172.21.15.92:0/818894987 shutdown_connections 2023-11-28T11:24:03.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.642+0000 7f8d7d9ff700 1 -- 172.21.15.92:0/818894987 wait complete. 2023-11-28T11:24:03.644 INFO:teuthology.orchestra.run.smithi092.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T11:24:03.653 INFO:tasks.ceph.mds.c.smithi092.stderr: -6618> 2023-11-28T11:23:17.727+0000 7f54499f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T11:24:03.658 INFO:tasks.ceph.mds.c.smithi092.stderr: -3075> 2023-11-28T11:23:23.158+0000 7f54461ea700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={5889=0-4194304@1} caps={5889=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5889 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x562cdbfd9b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-28T11:24:03.662 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T11:24:03.705 INFO:tasks.ceph.mds.c.smithi092.stdout:starting mds.c at 2023-11-28T11:24:03.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.862+0000 7f3e57bff700 1 Processor -- start 2023-11-28T11:24:03.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.863+0000 7f3e57bff700 1 -- start start 2023-11-28T11:24:03.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.864+0000 7f3e57bff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e500b8510 0x7f3e5006b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:03.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.864+0000 7f3e57bff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3e500cd460 0x7f3e5006beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:03.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.864+0000 7f3e57bff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e500b88e0 con 0x7f3e5006c3f0 2023-11-28T11:24:03.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.864+0000 7f3e57bff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e500cdbd0 con 0x7f3e500cd460 2023-11-28T11:24:03.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.864+0000 7f3e57bff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5006c830 con 0x7f3e500b8510 2023-11-28T11:24:03.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.864+0000 7f3e5519a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3e500cd460 0x7f3e5006beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:03.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.864+0000 7f3e5519a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3e500cd460 0x7f3e5006beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50282/0 (socket says 172.21.15.92:50282) 2023-11-28T11:24:03.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.864+0000 7f3e5619c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f3e5006c3f0 0x7f3e5015cc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55222/0 (socket says 172.21.15.92:55222) 2023-11-28T11:24:03.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.864+0000 7f3e5519a700 1 -- 172.21.15.92:0/1647036526 learned_addr learned my addr 172.21.15.92:0/1647036526 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:03.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.864+0000 7f3e5599b700 1 --2- 172.21.15.92:0/1647036526 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e500b8510 0x7f3e5006b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:03.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.864+0000 7f3e5519a700 1 -- 172.21.15.92:0/1647036526 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e500b8510 msgr2=0x7f3e5006b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:03.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.864+0000 7f3e5519a700 1 --2- 172.21.15.92:0/1647036526 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e500b8510 0x7f3e5006b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:03.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.865+0000 7f3e5519a700 1 -- 172.21.15.92:0/1647036526 >> v1:172.21.15.92:6789/0 conn(0x7f3e5006c3f0 legacy=0x7f3e5015cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:03.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.865+0000 7f3e5519a700 1 -- 172.21.15.92:0/1647036526 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e500cd8f0 con 0x7f3e500cd460 2023-11-28T11:24:03.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.865+0000 7f3e5519a700 1 --2- 172.21.15.92:0/1647036526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e500cd460 0x7f3e5006beb0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f3e40009920 tx=0x7f3e40009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39480bce6cded33f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:03.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.865+0000 7f3e54999700 1 -- 172.21.15.92:0/1647036526 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4000ba20 con 0x7f3e500cd460 2023-11-28T11:24:03.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.865+0000 7f3e54999700 1 -- 172.21.15.92:0/1647036526 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e4000bb80 con 0x7f3e500cd460 2023-11-28T11:24:03.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.865+0000 7f3e54999700 1 -- 172.21.15.92:0/1647036526 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e40009e70 con 0x7f3e500cd460 2023-11-28T11:24:03.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.866+0000 7f3e57bff700 1 -- 172.21.15.92:0/1647036526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e500cd460 msgr2=0x7f3e5006beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:03.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.866+0000 7f3e57bff700 1 --2- 172.21.15.92:0/1647036526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e500cd460 0x7f3e5006beb0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f3e40009920 tx=0x7f3e40009c30 comp rx=0 tx=0).stop 2023-11-28T11:24:03.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.866+0000 7f3e57bff700 1 -- 172.21.15.92:0/1647036526 shutdown_connections 2023-11-28T11:24:03.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.866+0000 7f3e57bff700 1 --2- 172.21.15.92:0/1647036526 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e500b8510 0x7f3e5006b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:03.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.866+0000 7f3e57bff700 1 --2- 172.21.15.92:0/1647036526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e500cd460 0x7f3e5006beb0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:03.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.866+0000 7f3e57bff700 1 -- 172.21.15.92:0/1647036526 >> 172.21.15.92:0/1647036526 conn(0x7f3e500719a0 msgr2=0x7f3e50071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:03.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.866+0000 7f3e57bff700 1 -- 172.21.15.92:0/1647036526 shutdown_connections 2023-11-28T11:24:03.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.866+0000 7f3e57bff700 1 -- 172.21.15.92:0/1647036526 wait complete. 2023-11-28T11:24:03.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.867+0000 7f3e57bff700 1 Processor -- start 2023-11-28T11:24:03.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.868+0000 7f3e57bff700 1 -- start start 2023-11-28T11:24:03.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.869+0000 7f3e57bff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e5006c3f0 0x7f3e50155990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:03.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.869+0000 7f3e57bff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e500b8510 0x7f3e50159630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:03.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.869+0000 7f3e57bff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e500cd460 0x7f3e500b6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:03.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.869+0000 7f3e57bff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5015a110 con 0x7f3e500b8510 2023-11-28T11:24:03.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.869+0000 7f3e57bff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e50155ed0 con 0x7f3e500cd460 2023-11-28T11:24:03.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.869+0000 7f3e57bff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e501f3360 con 0x7f3e5006c3f0 2023-11-28T11:24:03.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.869+0000 7f3e5519a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e500b8510 0x7f3e50159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:03.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.869+0000 7f3e5519a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e500b8510 0x7f3e50159630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50298/0 (socket says 172.21.15.92:50298) 2023-11-28T11:24:03.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.869+0000 7f3e5519a700 1 -- 172.21.15.92:0/1253184874 learned_addr learned my addr 172.21.15.92:0/1253184874 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:03.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.869+0000 7f3e5599b700 1 --2- 172.21.15.92:0/1253184874 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e5006c3f0 0x7f3e50155990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:03.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.869+0000 7f3e5619c700 1 --2- 172.21.15.92:0/1253184874 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e500cd460 0x7f3e500b6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:03.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.869+0000 7f3e5519a700 1 -- 172.21.15.92:0/1253184874 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e5006c3f0 msgr2=0x7f3e50155990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:03.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.869+0000 7f3e5519a700 1 --2- 172.21.15.92:0/1253184874 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e5006c3f0 0x7f3e50155990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:03.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.869+0000 7f3e5519a700 1 -- 172.21.15.92:0/1253184874 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e500cd460 msgr2=0x7f3e500b6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:03.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.869+0000 7f3e5519a700 1 --2- 172.21.15.92:0/1253184874 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e500cd460 0x7f3e500b6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:03.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.869+0000 7f3e5519a700 1 -- 172.21.15.92:0/1253184874 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e40009550 con 0x7f3e500b8510 2023-11-28T11:24:03.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.869+0000 7f3e5519a700 1 --2- 172.21.15.92:0/1253184874 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e500b8510 0x7f3e50159630 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f3e40009130 tx=0x7f3e40004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:03.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.870+0000 7f3e467fc700 1 -- 172.21.15.92:0/1253184874 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e40004550 con 0x7f3e500b8510 2023-11-28T11:24:03.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.870+0000 7f3e467fc700 1 -- 172.21.15.92:0/1253184874 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e4001e070 con 0x7f3e500b8510 2023-11-28T11:24:03.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.870+0000 7f3e57bff700 1 -- 172.21.15.92:0/1253184874 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e500cfff0 con 0x7f3e500b8510 2023-11-28T11:24:03.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.870+0000 7f3e467fc700 1 -- 172.21.15.92:0/1253184874 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e400038e0 con 0x7f3e500b8510 2023-11-28T11:24:03.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.870+0000 7f3e57bff700 1 -- 172.21.15.92:0/1253184874 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e500cfd10 con 0x7f3e500b8510 2023-11-28T11:24:03.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.871+0000 7f3e57bff700 1 -- 172.21.15.92:0/1253184874 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e50067880 con 0x7f3e500b8510 2023-11-28T11:24:03.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.872+0000 7f3e467fc700 1 -- 172.21.15.92:0/1253184874 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3e4000f050 con 0x7f3e500b8510 2023-11-28T11:24:03.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.873+0000 7f3e467fc700 1 --2- 172.21.15.92:0/1253184874 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3e340682f0 0x7f3e3406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:03.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.873+0000 7f3e467fc700 1 -- 172.21.15.92:0/1253184874 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6468+0+0 (secure 0 0 0) 0x7f3e40020070 con 0x7f3e500b8510 2023-11-28T11:24:03.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.873+0000 7f3e5599b700 1 --2- 172.21.15.92:0/1253184874 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3e340682f0 0x7f3e3406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:03.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.874+0000 7f3e5599b700 1 --2- 172.21.15.92:0/1253184874 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3e340682f0 0x7f3e3406a7a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3e50156d30 tx=0x7f3e4c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:03.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:03.877+0000 7f3e467fc700 1 -- 172.21.15.92:0/1253184874 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3e40053a30 con 0x7f3e500b8510 2023-11-28T11:24:04.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.109+0000 7f3e57bff700 1 -- 172.21.15.92:0/1253184874 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f3e500d2940 con 0x7f3e500b8510 2023-11-28T11:24:04.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.646+0000 7f3e467fc700 1 -- 172.21.15.92:0/1253184874 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v26) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3e40057050 con 0x7f3e500b8510 2023-11-28T11:24:04.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.649+0000 7f3e57bff700 1 -- 172.21.15.92:0/1253184874 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3e340682f0 msgr2=0x7f3e3406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:04.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.649+0000 7f3e57bff700 1 --2- 172.21.15.92:0/1253184874 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3e340682f0 0x7f3e3406a7a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3e50156d30 tx=0x7f3e4c008040 comp rx=0 tx=0).stop 2023-11-28T11:24:04.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.649+0000 7f3e57bff700 1 -- 172.21.15.92:0/1253184874 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e500b8510 msgr2=0x7f3e50159630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:04.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.649+0000 7f3e57bff700 1 --2- 172.21.15.92:0/1253184874 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e500b8510 0x7f3e50159630 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f3e40009130 tx=0x7f3e40004750 comp rx=0 tx=0).stop 2023-11-28T11:24:04.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.649+0000 7f3e57bff700 1 -- 172.21.15.92:0/1253184874 shutdown_connections 2023-11-28T11:24:04.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.649+0000 7f3e57bff700 1 --2- 172.21.15.92:0/1253184874 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3e500cd460 0x7f3e500b6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:04.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.649+0000 7f3e57bff700 1 --2- 172.21.15.92:0/1253184874 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3e5006c3f0 0x7f3e50155990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:04.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.649+0000 7f3e57bff700 1 --2- 172.21.15.92:0/1253184874 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3e340682f0 0x7f3e3406a7a0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:04.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.649+0000 7f3e57bff700 1 --2- 172.21.15.92:0/1253184874 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e500b8510 0x7f3e50159630 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:04.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.650+0000 7f3e57bff700 1 -- 172.21.15.92:0/1253184874 >> 172.21.15.92:0/1253184874 conn(0x7f3e500719a0 msgr2=0x7f3e500b3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:04.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.650+0000 7f3e57bff700 1 -- 172.21.15.92:0/1253184874 shutdown_connections 2023-11-28T11:24:04.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.650+0000 7f3e57bff700 1 -- 172.21.15.92:0/1253184874 wait complete. 2023-11-28T11:24:04.671 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T11:24:04.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.913+0000 7faaf9fff700 1 Processor -- start 2023-11-28T11:24:04.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.913+0000 7faaf9fff700 1 -- start start 2023-11-28T11:24:04.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.915+0000 7faaf9fff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7faaf415d580 0x7faaf40b5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:04.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.915+0000 7faaf9fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7faaf40d2db0 0x7faaf40d3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:04.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.915+0000 7faaf9fff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaf40d3c60 con 0x7faaf4156cb0 2023-11-28T11:24:04.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.915+0000 7faaf9fff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faaf4157080 con 0x7faaf40d2db0 2023-11-28T11:24:04.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.915+0000 7faaf9fff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faaf40cdb00 con 0x7faaf415d580 2023-11-28T11:24:04.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.915+0000 7faaf3fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7faaf40d2db0 0x7faaf40d3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:04.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.915+0000 7faaf3fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7faaf40d2db0 0x7faaf40d3210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50312/0 (socket says 172.21.15.92:50312) 2023-11-28T11:24:04.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.915+0000 7faaf3fff700 1 -- 172.21.15.92:0/516405875 learned_addr learned my addr 172.21.15.92:0/516405875 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:04.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.915+0000 7faaebfff700 1 --2- 172.21.15.92:0/516405875 >> v2:172.21.15.169:3300/0 conn(0x7faaf415d580 0x7faaf40b5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:04.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.915+0000 7faaf3fff700 1 -- 172.21.15.92:0/516405875 >> v2:172.21.15.169:3300/0 conn(0x7faaf415d580 msgr2=0x7faaf40b5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:04.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.915+0000 7faaf3fff700 1 --2- 172.21.15.92:0/516405875 >> v2:172.21.15.169:3300/0 conn(0x7faaf415d580 0x7faaf40b5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:04.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.915+0000 7faaf3fff700 1 -- 172.21.15.92:0/516405875 >> v1:172.21.15.169:6789/0 conn(0x7faaf4156cb0 legacy=0x7faaf40d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:04.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.915+0000 7faaf3fff700 1 -- 172.21.15.92:0/516405875 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaf40cf000 con 0x7faaf40d2db0 2023-11-28T11:24:04.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.916+0000 7faaf3fff700 1 --2- 172.21.15.92:0/516405875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faaf40d2db0 0x7faaf40d3210 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7faaf40b4aa0 tx=0x7faaec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b961c61b1ea621cc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:04.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.916+0000 7faaf2ffd700 1 -- 172.21.15.92:0/516405875 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaec00e070 con 0x7faaf40d2db0 2023-11-28T11:24:04.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.916+0000 7faaf2ffd700 1 -- 172.21.15.92:0/516405875 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaec00b9c0 con 0x7faaf40d2db0 2023-11-28T11:24:04.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.916+0000 7faaf2ffd700 1 -- 172.21.15.92:0/516405875 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaec00bd70 con 0x7faaf40d2db0 2023-11-28T11:24:04.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.916+0000 7faaf9fff700 1 -- 172.21.15.92:0/516405875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faaf40d2db0 msgr2=0x7faaf40d3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:04.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.916+0000 7faaf9fff700 1 --2- 172.21.15.92:0/516405875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faaf40d2db0 0x7faaf40d3210 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7faaf40b4aa0 tx=0x7faaec00b330 comp rx=0 tx=0).stop 2023-11-28T11:24:04.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.917+0000 7faaf9fff700 1 -- 172.21.15.92:0/516405875 shutdown_connections 2023-11-28T11:24:04.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.917+0000 7faaf9fff700 1 --2- 172.21.15.92:0/516405875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faaf40d2db0 0x7faaf40d3210 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:04.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.917+0000 7faaf9fff700 1 --2- 172.21.15.92:0/516405875 >> v2:172.21.15.169:3300/0 conn(0x7faaf415d580 0x7faaf40b5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:04.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.917+0000 7faaf9fff700 1 -- 172.21.15.92:0/516405875 >> 172.21.15.92:0/516405875 conn(0x7faaf40719a0 msgr2=0x7faaf4071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:04.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.917+0000 7faaf9fff700 1 -- 172.21.15.92:0/516405875 shutdown_connections 2023-11-28T11:24:04.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.917+0000 7faaf9fff700 1 -- 172.21.15.92:0/516405875 wait complete. 2023-11-28T11:24:04.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.919+0000 7faaf9fff700 1 Processor -- start 2023-11-28T11:24:04.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.919+0000 7faaf9fff700 1 -- start start 2023-11-28T11:24:04.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.920+0000 7faaf9fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faaf40d2db0 0x7faaf41f9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:04.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.920+0000 7faaf9fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faaf4156cb0 0x7faaf406bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:04.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.920+0000 7faaf9fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faaf415d580 0x7faaf41f6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:04.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.920+0000 7faaf9fff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf41f9f40 con 0x7faaf40d2db0 2023-11-28T11:24:04.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.921+0000 7faaf9fff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf41f63c0 con 0x7faaf4156cb0 2023-11-28T11:24:04.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.921+0000 7faaf9fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaf41fa1e0 con 0x7faaf415d580 2023-11-28T11:24:04.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.921+0000 7faaf37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faaf40d2db0 0x7faaf41f9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:04.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.921+0000 7faaf37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faaf40d2db0 0x7faaf41f9a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50326/0 (socket says 172.21.15.92:50326) 2023-11-28T11:24:04.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.921+0000 7faaf37fe700 1 -- 172.21.15.92:0/1588952938 learned_addr learned my addr 172.21.15.92:0/1588952938 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:04.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.921+0000 7faaebfff700 1 --2- 172.21.15.92:0/1588952938 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faaf4156cb0 0x7faaf406bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:04.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.921+0000 7faaf3fff700 1 --2- 172.21.15.92:0/1588952938 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faaf415d580 0x7faaf41f6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:04.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.921+0000 7faaf37fe700 1 -- 172.21.15.92:0/1588952938 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faaf415d580 msgr2=0x7faaf41f6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:04.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.921+0000 7faaf37fe700 1 --2- 172.21.15.92:0/1588952938 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faaf415d580 0x7faaf41f6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:04.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.921+0000 7faaf37fe700 1 -- 172.21.15.92:0/1588952938 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faaf4156cb0 msgr2=0x7faaf406bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:04.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.921+0000 7faaf37fe700 1 --2- 172.21.15.92:0/1588952938 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faaf4156cb0 0x7faaf406bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:04.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.921+0000 7faaf37fe700 1 -- 172.21.15.92:0/1588952938 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaec009b30 con 0x7faaf40d2db0 2023-11-28T11:24:04.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.921+0000 7faaf37fe700 1 --2- 172.21.15.92:0/1588952938 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faaf40d2db0 0x7faaf41f9a00 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7faae400c4e0 tx=0x7faae400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:04.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.922+0000 7faaf0ff9700 1 -- 172.21.15.92:0/1588952938 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae40052d0 con 0x7faaf40d2db0 2023-11-28T11:24:04.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.922+0000 7faaf0ff9700 1 -- 172.21.15.92:0/1588952938 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faae4005430 con 0x7faaf40d2db0 2023-11-28T11:24:04.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.922+0000 7faaf0ff9700 1 -- 172.21.15.92:0/1588952938 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae40038c0 con 0x7faaf40d2db0 2023-11-28T11:24:04.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.922+0000 7faaf9fff700 1 -- 172.21.15.92:0/1588952938 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaf40b58b0 con 0x7faaf40d2db0 2023-11-28T11:24:04.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.922+0000 7faaf9fff700 1 -- 172.21.15.92:0/1588952938 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaf40cf2c0 con 0x7faaf40d2db0 2023-11-28T11:24:04.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.923+0000 7faaf9fff700 1 -- 172.21.15.92:0/1588952938 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaf40d3750 con 0x7faaf40d2db0 2023-11-28T11:24:04.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.924+0000 7faaf0ff9700 1 -- 172.21.15.92:0/1588952938 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7faae4019030 con 0x7faaf40d2db0 2023-11-28T11:24:04.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.925+0000 7faaf0ff9700 1 --2- 172.21.15.92:0/1588952938 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7faad40682f0 0x7faad406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:04.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.925+0000 7faaf0ff9700 1 -- 172.21.15.92:0/1588952938 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6468+0+0 (secure 0 0 0) 0x7faae4085180 con 0x7faaf40d2db0 2023-11-28T11:24:04.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.925+0000 7faaebfff700 1 --2- 172.21.15.92:0/1588952938 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7faad40682f0 0x7faad406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:04.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.926+0000 7faaebfff700 1 --2- 172.21.15.92:0/1588952938 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7faad40682f0 0x7faad406a7a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7faaf415e5a0 tx=0x7faadc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:04.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:04.929+0000 7faaf0ff9700 1 -- 172.21.15.92:0/1588952938 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7faae4051750 con 0x7faaf40d2db0 2023-11-28T11:24:05.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.119+0000 7faaf9fff700 1 -- 172.21.15.92:0/1588952938 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7faaf4067170 con 0x7faaf40d2db0 2023-11-28T11:24:05.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.718+0000 7faaf0ff9700 1 -- 172.21.15.92:0/1588952938 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v58) v1 ==== 181+0+0 (secure 0 0 0) 0x7faae4054d70 con 0x7faaf40d2db0 2023-11-28T11:24:05.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.721+0000 7faaf9fff700 1 -- 172.21.15.92:0/1588952938 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7faad40682f0 msgr2=0x7faad406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:05.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.721+0000 7faaf9fff700 1 --2- 172.21.15.92:0/1588952938 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7faad40682f0 0x7faad406a7a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7faaf415e5a0 tx=0x7faadc008040 comp rx=0 tx=0).stop 2023-11-28T11:24:05.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.721+0000 7faaf9fff700 1 -- 172.21.15.92:0/1588952938 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faaf40d2db0 msgr2=0x7faaf41f9a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:05.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.721+0000 7faaf9fff700 1 --2- 172.21.15.92:0/1588952938 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faaf40d2db0 0x7faaf41f9a00 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7faae400c4e0 tx=0x7faae400c7f0 comp rx=0 tx=0).stop 2023-11-28T11:24:05.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.722+0000 7faaf9fff700 1 -- 172.21.15.92:0/1588952938 shutdown_connections 2023-11-28T11:24:05.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.722+0000 7faaf9fff700 1 --2- 172.21.15.92:0/1588952938 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faaf415d580 0x7faaf41f6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:05.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.722+0000 7faaf9fff700 1 --2- 172.21.15.92:0/1588952938 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7faad40682f0 0x7faad406a7a0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:05.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.722+0000 7faaf9fff700 1 --2- 172.21.15.92:0/1588952938 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faaf40d2db0 0x7faaf41f9a00 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:05.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.722+0000 7faaf9fff700 1 --2- 172.21.15.92:0/1588952938 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faaf4156cb0 0x7faaf406bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:05.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.722+0000 7faaf9fff700 1 -- 172.21.15.92:0/1588952938 >> 172.21.15.92:0/1588952938 conn(0x7faaf40719a0 msgr2=0x7faaf40d1830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:05.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.722+0000 7faaf9fff700 1 -- 172.21.15.92:0/1588952938 shutdown_connections 2023-11-28T11:24:05.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.722+0000 7faaf9fff700 1 -- 172.21.15.92:0/1588952938 wait complete. 2023-11-28T11:24:05.724 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' removed 2023-11-28T11:24:05.748 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T11:24:05.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.948+0000 7fb03d1ff700 1 Processor -- start 2023-11-28T11:24:05.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.949+0000 7fb03d1ff700 1 -- start start 2023-11-28T11:24:05.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.950+0000 7fb03d1ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb03806c050 0x7fb038167410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:05.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.950+0000 7fb03d1ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0380b5240 con 0x7fb0380cd460 2023-11-28T11:24:05.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.950+0000 7fb03d1ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0380b5520 con 0x7fb0380b8790 2023-11-28T11:24:05.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.950+0000 7fb03d1ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0380d0a80 con 0x7fb03806c050 2023-11-28T11:24:05.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.950+0000 7fb02ffff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb0380cd460 0x7fb03806b940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55228/0 (socket says 172.21.15.92:55228) 2023-11-28T11:24:05.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.950+0000 7fb02ffff700 1 -- 172.21.15.92:0/1119749391 learned_addr learned my addr 172.21.15.92:0/1119749391 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:24:05.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.950+0000 7fb03759e700 1 --2- 172.21.15.92:0/1119749391 >> v2:172.21.15.92:3300/0 conn(0x7fb03806c050 0x7fb038167410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:05.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.950+0000 7fb03659c700 1 -- 172.21.15.92:0/1119749391 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3724752026 0 0) 0x7fb0380b5240 con 0x7fb0380cd460 2023-11-28T11:24:05.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.951+0000 7fb03659c700 1 -- 172.21.15.92:0/1119749391 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0240035b0 con 0x7fb0380cd460 2023-11-28T11:24:05.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.951+0000 7fb03759e700 1 -- 172.21.15.92:0/1119749391 >> v1:172.21.15.169:6789/0 conn(0x7fb0380b8790 legacy=0x7fb0380cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:05.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.951+0000 7fb03759e700 1 -- 172.21.15.92:0/1119749391 >> v1:172.21.15.92:6789/0 conn(0x7fb0380cd460 legacy=0x7fb03806b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:05.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.951+0000 7fb03759e700 1 -- 172.21.15.92:0/1119749391 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0380b5800 con 0x7fb03806c050 2023-11-28T11:24:05.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.951+0000 7fb03759e700 1 --2- 172.21.15.92:0/1119749391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb03806c050 0x7fb038167410 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fb030009dc0 tx=0x7fb03000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db3d6f70cfcc7b0a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:05.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.951+0000 7fb03659c700 1 -- 172.21.15.92:0/1119749391 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03000e070 con 0x7fb03806c050 2023-11-28T11:24:05.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.951+0000 7fb03659c700 1 -- 172.21.15.92:0/1119749391 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb03000bae0 con 0x7fb03806c050 2023-11-28T11:24:05.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.951+0000 7fb03659c700 1 -- 172.21.15.92:0/1119749391 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03000be90 con 0x7fb03806c050 2023-11-28T11:24:05.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.952+0000 7fb03d1ff700 1 -- 172.21.15.92:0/1119749391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb03806c050 msgr2=0x7fb038167410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:05.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.952+0000 7fb03d1ff700 1 --2- 172.21.15.92:0/1119749391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb03806c050 0x7fb038167410 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fb030009dc0 tx=0x7fb03000b330 comp rx=0 tx=0).stop 2023-11-28T11:24:05.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.952+0000 7fb03d1ff700 1 -- 172.21.15.92:0/1119749391 shutdown_connections 2023-11-28T11:24:05.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.952+0000 7fb03d1ff700 1 --2- 172.21.15.92:0/1119749391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb03806c050 0x7fb038167410 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:05.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.952+0000 7fb03d1ff700 1 -- 172.21.15.92:0/1119749391 >> 172.21.15.92:0/1119749391 conn(0x7fb0380719a0 msgr2=0x7fb038071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:05.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.952+0000 7fb03d1ff700 1 -- 172.21.15.92:0/1119749391 shutdown_connections 2023-11-28T11:24:05.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.952+0000 7fb03d1ff700 1 -- 172.21.15.92:0/1119749391 wait complete. 2023-11-28T11:24:05.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.953+0000 7fb03d1ff700 1 Processor -- start 2023-11-28T11:24:05.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.954+0000 7fb03d1ff700 1 -- start start 2023-11-28T11:24:05.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.954+0000 7fb03d1ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb03806c050 0x7fb038155b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:05.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.954+0000 7fb03d1ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0380b8790 0x7fb038156040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:05.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.955+0000 7fb03d1ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0380cd460 0x7fb038156580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:05.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.955+0000 7fb03d1ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0381556b0 con 0x7fb0380cd460 2023-11-28T11:24:05.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.955+0000 7fb03d1ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0380cadd0 con 0x7fb0380b8790 2023-11-28T11:24:05.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.955+0000 7fb03d1ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0380cb0b0 con 0x7fb03806c050 2023-11-28T11:24:05.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.955+0000 7fb03759e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0380cd460 0x7fb038156580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:05.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.955+0000 7fb03759e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0380cd460 0x7fb038156580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50344/0 (socket says 172.21.15.92:50344) 2023-11-28T11:24:05.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.955+0000 7fb03759e700 1 -- 172.21.15.92:0/1783732792 learned_addr learned my addr 172.21.15.92:0/1783732792 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:05.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.955+0000 7fb02ffff700 1 --2- 172.21.15.92:0/1783732792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0380b8790 0x7fb038156040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:05.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.955+0000 7fb036d9d700 1 --2- 172.21.15.92:0/1783732792 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb03806c050 0x7fb038155b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:05.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.955+0000 7fb03759e700 1 -- 172.21.15.92:0/1783732792 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb03806c050 msgr2=0x7fb038155b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:05.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.955+0000 7fb03759e700 1 --2- 172.21.15.92:0/1783732792 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb03806c050 0x7fb038155b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:05.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.955+0000 7fb03759e700 1 -- 172.21.15.92:0/1783732792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0380b8790 msgr2=0x7fb038156040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:05.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.955+0000 7fb03759e700 1 --2- 172.21.15.92:0/1783732792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0380b8790 0x7fb038156040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:05.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.955+0000 7fb03759e700 1 -- 172.21.15.92:0/1783732792 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb030009a70 con 0x7fb0380cd460 2023-11-28T11:24:05.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.956+0000 7fb03759e700 1 --2- 172.21.15.92:0/1783732792 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0380cd460 0x7fb038156580 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fb030014040 tx=0x7fb030004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:05.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.956+0000 7fb02f7fe700 1 -- 172.21.15.92:0/1783732792 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb030004550 con 0x7fb0380cd460 2023-11-28T11:24:05.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.956+0000 7fb02f7fe700 1 -- 172.21.15.92:0/1783732792 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb030020070 con 0x7fb0380cd460 2023-11-28T11:24:05.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.956+0000 7fb02f7fe700 1 -- 172.21.15.92:0/1783732792 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb030004910 con 0x7fb0380cd460 2023-11-28T11:24:05.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.956+0000 7fb03d1ff700 1 -- 172.21.15.92:0/1783732792 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0380d2140 con 0x7fb0380cd460 2023-11-28T11:24:05.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.956+0000 7fb03d1ff700 1 -- 172.21.15.92:0/1783732792 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0380d1fb0 con 0x7fb0380cd460 2023-11-28T11:24:05.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.957+0000 7fb03d1ff700 1 -- 172.21.15.92:0/1783732792 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0380d22d0 con 0x7fb0380cd460 2023-11-28T11:24:05.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.958+0000 7fb02f7fe700 1 -- 172.21.15.92:0/1783732792 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb030012430 con 0x7fb0380cd460 2023-11-28T11:24:05.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.959+0000 7fb02f7fe700 1 --2- 172.21.15.92:0/1783732792 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb0180682f0 0x7fb01806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:05.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.959+0000 7fb02f7fe700 1 -- 172.21.15.92:0/1783732792 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6030+0+0 (secure 0 0 0) 0x7fb030022070 con 0x7fb0380cd460 2023-11-28T11:24:05.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.959+0000 7fb036d9d700 1 --2- 172.21.15.92:0/1783732792 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb0180682f0 0x7fb01806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:05.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.959+0000 7fb036d9d700 1 --2- 172.21.15.92:0/1783732792 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb0180682f0 0x7fb01806a7a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb028007fd0 tx=0x7fb028009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:05.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:05.963+0000 7fb02f7fe700 1 -- 172.21.15.92:0/1783732792 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb0380d22d0 con 0x7fb0380cd460 2023-11-28T11:24:06.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.152+0000 7fb03d1ff700 1 -- 172.21.15.92:0/1783732792 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb0380d22d0 con 0x7fb0380cd460 2023-11-28T11:24:06.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.725+0000 7fb02f7fe700 1 -- 172.21.15.92:0/1783732792 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v59) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb0380d22d0 con 0x7fb0380cd460 2023-11-28T11:24:06.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.729+0000 7fb03d1ff700 1 -- 172.21.15.92:0/1783732792 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb0180682f0 msgr2=0x7fb01806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:06.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.729+0000 7fb03d1ff700 1 --2- 172.21.15.92:0/1783732792 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb0180682f0 0x7fb01806a7a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb028007fd0 tx=0x7fb028009040 comp rx=0 tx=0).stop 2023-11-28T11:24:06.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.729+0000 7fb03d1ff700 1 -- 172.21.15.92:0/1783732792 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0380cd460 msgr2=0x7fb038156580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:06.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.729+0000 7fb03d1ff700 1 --2- 172.21.15.92:0/1783732792 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0380cd460 0x7fb038156580 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fb030014040 tx=0x7fb030004ad0 comp rx=0 tx=0).stop 2023-11-28T11:24:06.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.729+0000 7fb03d1ff700 1 -- 172.21.15.92:0/1783732792 shutdown_connections 2023-11-28T11:24:06.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.729+0000 7fb03d1ff700 1 --2- 172.21.15.92:0/1783732792 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb0180682f0 0x7fb01806a7a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:06.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.729+0000 7fb03d1ff700 1 --2- 172.21.15.92:0/1783732792 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0380cd460 0x7fb038156580 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:06.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.729+0000 7fb03d1ff700 1 --2- 172.21.15.92:0/1783732792 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb03806c050 0x7fb038155b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:06.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.729+0000 7fb03d1ff700 1 --2- 172.21.15.92:0/1783732792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb0380b8790 0x7fb038156040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:06.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.729+0000 7fb03d1ff700 1 -- 172.21.15.92:0/1783732792 >> 172.21.15.92:0/1783732792 conn(0x7fb0380719a0 msgr2=0x7fb03806b940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:06.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.729+0000 7fb03d1ff700 1 -- 172.21.15.92:0/1783732792 shutdown_connections 2023-11-28T11:24:06.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.729+0000 7fb03d1ff700 1 -- 172.21.15.92:0/1783732792 wait complete. 2023-11-28T11:24:06.731 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' removed 2023-11-28T11:24:06.751 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-28T11:24:06.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.952+0000 7f6ee9fff700 1 Processor -- start 2023-11-28T11:24:06.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.952+0000 7f6ee9fff700 1 -- start start 2023-11-28T11:24:06.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.953+0000 7f6ee9fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6ee40b8510 0x7f6ee406b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:06.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.953+0000 7f6ee9fff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6ee40cd460 0x7f6ee406beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:06.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.953+0000 7f6ee9fff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ee40b88e0 con 0x7f6ee406c3f0 2023-11-28T11:24:06.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.953+0000 7f6ee9fff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ee40cdbd0 con 0x7f6ee40b8510 2023-11-28T11:24:06.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.953+0000 7f6ee9fff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ee40d02d0 con 0x7f6ee40cd460 2023-11-28T11:24:06.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.954+0000 7f6ee3fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6ee406c3f0 0x7f6ee415cc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55230/0 (socket says 172.21.15.92:55230) 2023-11-28T11:24:06.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.954+0000 7f6ee3fff700 1 -- 172.21.15.92:0/1182575095 learned_addr learned my addr 172.21.15.92:0/1182575095 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:24:06.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.954+0000 7f6ee37fe700 1 --2- 172.21.15.92:0/1182575095 >> v2:172.21.15.92:3300/0 conn(0x7f6ee40b8510 0x7f6ee406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:06.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.954+0000 7f6ee2ffd700 1 --2- 172.21.15.92:0/1182575095 >> v2:172.21.15.169:3300/0 conn(0x7f6ee40cd460 0x7f6ee406beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:06.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.954+0000 7f6ee27fc700 1 -- 172.21.15.92:0/1182575095 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1519816039 0 0) 0x7f6ee40b88e0 con 0x7f6ee406c3f0 2023-11-28T11:24:06.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.954+0000 7f6ee27fc700 1 -- 172.21.15.92:0/1182575095 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ecc0035b0 con 0x7f6ee406c3f0 2023-11-28T11:24:06.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.954+0000 7f6ee37fe700 1 -- 172.21.15.92:0/1182575095 >> v2:172.21.15.169:3300/0 conn(0x7f6ee40cd460 msgr2=0x7f6ee406beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:06.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.954+0000 7f6ee37fe700 1 --2- 172.21.15.92:0/1182575095 >> v2:172.21.15.169:3300/0 conn(0x7f6ee40cd460 0x7f6ee406beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:06.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.954+0000 7f6ee37fe700 1 -- 172.21.15.92:0/1182575095 >> v1:172.21.15.92:6789/0 conn(0x7f6ee406c3f0 legacy=0x7f6ee415cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:06.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.954+0000 7f6ee37fe700 1 -- 172.21.15.92:0/1182575095 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ee40cd8f0 con 0x7f6ee40b8510 2023-11-28T11:24:06.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.955+0000 7f6ee37fe700 1 --2- 172.21.15.92:0/1182575095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ee40b8510 0x7f6ee406b970 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4009870 tx=0x7f6ed4009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68bc62062c9f1b69 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:06.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.955+0000 7f6ee27fc700 1 -- 172.21.15.92:0/1182575095 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed4011830 con 0x7f6ee40b8510 2023-11-28T11:24:06.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.955+0000 7f6ee27fc700 1 -- 172.21.15.92:0/1182575095 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ed4011990 con 0x7f6ee40b8510 2023-11-28T11:24:06.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.955+0000 7f6ee27fc700 1 -- 172.21.15.92:0/1182575095 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed4011d20 con 0x7f6ee40b8510 2023-11-28T11:24:06.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.955+0000 7f6ee9fff700 1 -- 172.21.15.92:0/1182575095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ee40b8510 msgr2=0x7f6ee406b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:06.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.955+0000 7f6ee9fff700 1 --2- 172.21.15.92:0/1182575095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ee40b8510 0x7f6ee406b970 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4009870 tx=0x7f6ed4009b80 comp rx=0 tx=0).stop 2023-11-28T11:24:06.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.955+0000 7f6ee9fff700 1 -- 172.21.15.92:0/1182575095 shutdown_connections 2023-11-28T11:24:06.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.955+0000 7f6ee9fff700 1 --2- 172.21.15.92:0/1182575095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ee40b8510 0x7f6ee406b970 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:06.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.955+0000 7f6ee9fff700 1 --2- 172.21.15.92:0/1182575095 >> v2:172.21.15.169:3300/0 conn(0x7f6ee40cd460 0x7f6ee406beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:06.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.955+0000 7f6ee9fff700 1 -- 172.21.15.92:0/1182575095 >> 172.21.15.92:0/1182575095 conn(0x7f6ee40719a0 msgr2=0x7f6ee4071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:06.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.956+0000 7f6ee9fff700 1 -- 172.21.15.92:0/1182575095 shutdown_connections 2023-11-28T11:24:06.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.956+0000 7f6ee9fff700 1 -- 172.21.15.92:0/1182575095 wait complete. 2023-11-28T11:24:06.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.957+0000 7f6ee9fff700 1 Processor -- start 2023-11-28T11:24:06.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.957+0000 7f6ee9fff700 1 -- start start 2023-11-28T11:24:06.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.958+0000 7f6ee9fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ee406c3f0 0x7f6ee4159430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:06.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.958+0000 7f6ee9fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ee40b8510 0x7f6ee415a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:06.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.958+0000 7f6ee9fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ee40cd460 0x7f6ee41e5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:06.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.958+0000 7f6ee9fff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee41e4d00 con 0x7f6ee40cd460 2023-11-28T11:24:06.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.958+0000 7f6ee9fff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee4159970 con 0x7f6ee40b8510 2023-11-28T11:24:06.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.958+0000 7f6ee9fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee41e4080 con 0x7f6ee406c3f0 2023-11-28T11:24:06.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.958+0000 7f6ee3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ee40cd460 0x7f6ee41e5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:06.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.959+0000 7f6ee3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ee40cd460 0x7f6ee41e5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50370/0 (socket says 172.21.15.92:50370) 2023-11-28T11:24:06.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.959+0000 7f6ee3fff700 1 -- 172.21.15.92:0/212219752 learned_addr learned my addr 172.21.15.92:0/212219752 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:06.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.959+0000 7f6ee37fe700 1 --2- 172.21.15.92:0/212219752 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ee406c3f0 0x7f6ee4159430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:06.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.959+0000 7f6ee2ffd700 1 --2- 172.21.15.92:0/212219752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ee40b8510 0x7f6ee415a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:06.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.959+0000 7f6ee3fff700 1 -- 172.21.15.92:0/212219752 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ee406c3f0 msgr2=0x7f6ee4159430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:06.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.959+0000 7f6ee3fff700 1 --2- 172.21.15.92:0/212219752 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ee406c3f0 0x7f6ee4159430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:06.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.959+0000 7f6ee3fff700 1 -- 172.21.15.92:0/212219752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ee40b8510 msgr2=0x7f6ee415a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:06.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.959+0000 7f6ee3fff700 1 --2- 172.21.15.92:0/212219752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ee40b8510 0x7f6ee415a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:06.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.959+0000 7f6ee3fff700 1 -- 172.21.15.92:0/212219752 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ed40094a0 con 0x7f6ee40cd460 2023-11-28T11:24:06.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.959+0000 7f6ee37fe700 1 --2- 172.21.15.92:0/212219752 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ee406c3f0 0x7f6ee4159430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:24:06.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.959+0000 7f6ee2ffd700 1 --2- 172.21.15.92:0/212219752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ee40b8510 0x7f6ee415a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:24:06.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.959+0000 7f6ee3fff700 1 --2- 172.21.15.92:0/212219752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ee40cd460 0x7f6ee41e5e50 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f6edc00cb70 tx=0x7f6edc00cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:06.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.959+0000 7f6ecbfff700 1 -- 172.21.15.92:0/212219752 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6edc00ab10 con 0x7f6ee40cd460 2023-11-28T11:24:06.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.959+0000 7f6ecbfff700 1 -- 172.21.15.92:0/212219752 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6edc00ac70 con 0x7f6ee40cd460 2023-11-28T11:24:06.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.959+0000 7f6ecbfff700 1 -- 172.21.15.92:0/212219752 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6edc012400 con 0x7f6ee40cd460 2023-11-28T11:24:06.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.959+0000 7f6ee9fff700 1 -- 172.21.15.92:0/212219752 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ee40cfff0 con 0x7f6ee40cd460 2023-11-28T11:24:06.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.959+0000 7f6ee9fff700 1 -- 172.21.15.92:0/212219752 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ee40cfd10 con 0x7f6ee40cd460 2023-11-28T11:24:06.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.960+0000 7f6ee9fff700 1 -- 172.21.15.92:0/212219752 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ee4067880 con 0x7f6ee40cd460 2023-11-28T11:24:06.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.961+0000 7f6ecbfff700 1 -- 172.21.15.92:0/212219752 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6edc019030 con 0x7f6ee40cd460 2023-11-28T11:24:06.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.962+0000 7f6ecbfff700 1 --2- 172.21.15.92:0/212219752 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6ed00682f0 0x7f6ed006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:06.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.962+0000 7f6ecbfff700 1 -- 172.21.15.92:0/212219752 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f6edc085860 con 0x7f6ee40cd460 2023-11-28T11:24:06.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.962+0000 7f6ee37fe700 1 --2- 172.21.15.92:0/212219752 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6ed00682f0 0x7f6ed006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:06.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.963+0000 7f6ee37fe700 1 --2- 172.21.15.92:0/212219752 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6ed00682f0 0x7f6ed006a7a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6ed40090b0 tx=0x7f6ed4008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:06.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:06.967+0000 7f6ecbfff700 1 -- 172.21.15.92:0/212219752 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6edc0521b0 con 0x7f6ee40cd460 2023-11-28T11:24:07.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.157+0000 7f6ee9fff700 1 -- 172.21.15.92:0/212219752 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f6ee40d2940 con 0x7f6ee40cd460 2023-11-28T11:24:07.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.161+0000 7f6ecbfff700 1 -- 172.21.15.92:0/212219752 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6edc0557d0 con 0x7f6ee40cd460 2023-11-28T11:24:07.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.164+0000 7f6ee9fff700 1 -- 172.21.15.92:0/212219752 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6ed00682f0 msgr2=0x7f6ed006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:07.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.164+0000 7f6ee9fff700 1 --2- 172.21.15.92:0/212219752 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6ed00682f0 0x7f6ed006a7a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6ed40090b0 tx=0x7f6ed4008030 comp rx=0 tx=0).stop 2023-11-28T11:24:07.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.164+0000 7f6ee9fff700 1 -- 172.21.15.92:0/212219752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ee40cd460 msgr2=0x7f6ee41e5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:07.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.164+0000 7f6ee9fff700 1 --2- 172.21.15.92:0/212219752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ee40cd460 0x7f6ee41e5e50 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f6edc00cb70 tx=0x7f6edc00cf30 comp rx=0 tx=0).stop 2023-11-28T11:24:07.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.164+0000 7f6ee9fff700 1 -- 172.21.15.92:0/212219752 shutdown_connections 2023-11-28T11:24:07.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.164+0000 7f6ee9fff700 1 --2- 172.21.15.92:0/212219752 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6ed00682f0 0x7f6ed006a7a0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:07.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.164+0000 7f6ee9fff700 1 --2- 172.21.15.92:0/212219752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ee40cd460 0x7f6ee41e5e50 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:07.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.164+0000 7f6ee9fff700 1 --2- 172.21.15.92:0/212219752 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6ee406c3f0 0x7f6ee4159430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:07.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.164+0000 7f6ee9fff700 1 --2- 172.21.15.92:0/212219752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6ee40b8510 0x7f6ee415a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:07.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.164+0000 7f6ee9fff700 1 -- 172.21.15.92:0/212219752 >> 172.21.15.92:0/212219752 conn(0x7f6ee40719a0 msgr2=0x7f6ee40d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:07.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.164+0000 7f6ee9fff700 1 -- 172.21.15.92:0/212219752 shutdown_connections 2023-11-28T11:24:07.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.164+0000 7f6ee9fff700 1 -- 172.21.15.92:0/212219752 wait complete. 2023-11-28T11:24:07.183 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2023-11-28T11:24:07.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.382+0000 7fa8081ff700 1 Processor -- start 2023-11-28T11:24:07.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.382+0000 7fa8081ff700 1 -- start start 2023-11-28T11:24:07.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.383+0000 7fa8081ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa8000b1c60 0x7fa8000cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:07.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.383+0000 7fa8081ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8000d1950 con 0x7fa8000d2df0 2023-11-28T11:24:07.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.383+0000 7fa8081ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8000b2500 con 0x7fa8000b4fc0 2023-11-28T11:24:07.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.383+0000 7fa8081ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8000d3d30 con 0x7fa8000b1c60 2023-11-28T11:24:07.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.383+0000 7fa805f9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fa8000d2df0 0x7fa8000d31c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55244/0 (socket says 172.21.15.92:55244) 2023-11-28T11:24:07.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.383+0000 7fa805f9b700 1 -- 172.21.15.92:0/4197284558 learned_addr learned my addr 172.21.15.92:0/4197284558 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:24:07.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.384+0000 7fa80679c700 1 --2- 172.21.15.92:0/4197284558 >> v2:172.21.15.169:3300/0 conn(0x7fa8000b1c60 0x7fa8000cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:07.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.384+0000 7fa804f99700 1 -- 172.21.15.92:0/4197284558 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1031382313 0 0) 0x7fa8000d1950 con 0x7fa8000d2df0 2023-11-28T11:24:07.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.384+0000 7fa804f99700 1 -- 172.21.15.92:0/4197284558 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7ec0035b0 con 0x7fa8000d2df0 2023-11-28T11:24:07.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.384+0000 7fa804f99700 1 -- 172.21.15.92:0/4197284558 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1275258243 0 0) 0x7fa8000b2500 con 0x7fa8000b4fc0 2023-11-28T11:24:07.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.384+0000 7fa804f99700 1 -- 172.21.15.92:0/4197284558 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8000d1950 con 0x7fa8000b4fc0 2023-11-28T11:24:07.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.384+0000 7fa804f99700 1 -- 172.21.15.92:0/4197284558 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3777697041 0 0) 0x7fa7ec0035b0 con 0x7fa8000d2df0 2023-11-28T11:24:07.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.384+0000 7fa804f99700 1 -- 172.21.15.92:0/4197284558 >> v2:172.21.15.169:3300/0 conn(0x7fa8000b1c60 msgr2=0x7fa8000cd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:07.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.384+0000 7fa804f99700 1 --2- 172.21.15.92:0/4197284558 >> v2:172.21.15.169:3300/0 conn(0x7fa8000b1c60 0x7fa8000cd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:07.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.384+0000 7fa804f99700 1 -- 172.21.15.92:0/4197284558 >> v1:172.21.15.169:6789/0 conn(0x7fa8000b4fc0 legacy=0x7fa8000cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:07.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.384+0000 7fa804f99700 1 -- 172.21.15.92:0/4197284558 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8000d3a50 con 0x7fa8000d2df0 2023-11-28T11:24:07.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.384+0000 7fa804f99700 1 -- 172.21.15.92:0/4197284558 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532161587 0 0) 0x7fa7fc001670 con 0x7fa8000d2df0 2023-11-28T11:24:07.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.384+0000 7fa804f99700 1 -- 172.21.15.92:0/4197284558 >> v1:172.21.15.92:6789/0 conn(0x7fa8000d2df0 legacy=0x7fa8000d31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:07.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.384+0000 7fa804f99700 1 --2- 172.21.15.92:0/4197284558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7ec003b90 0x7fa7ec003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:07.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.384+0000 7fa804f99700 1 --2- 172.21.15.92:0/4197284558 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7ec004850 0x7fa7ec004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:07.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.384+0000 7fa804f99700 1 --2- 172.21.15.92:0/4197284558 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7ec005200 0x7fa7ec00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:07.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.384+0000 7fa804f99700 1 -- 172.21.15.92:0/4197284558 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8000d3d30 con 0x7fa7ec005200 2023-11-28T11:24:07.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.384+0000 7fa804f99700 1 -- 172.21.15.92:0/4197284558 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ec00c060 con 0x7fa7ec003b90 2023-11-28T11:24:07.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.384+0000 7fa804f99700 1 -- 172.21.15.92:0/4197284558 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ec00c220 con 0x7fa7ec004850 2023-11-28T11:24:07.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.385+0000 7fa80679c700 1 --2- 172.21.15.92:0/4197284558 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7ec005200 0x7fa7ec00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:07.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.385+0000 7fa805f9b700 1 --2- 172.21.15.92:0/4197284558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7ec003b90 0x7fa7ec003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:07.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.385+0000 7fa80579a700 1 --2- 172.21.15.92:0/4197284558 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7ec004850 0x7fa7ec004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:07.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.385+0000 7fa80679c700 1 -- 172.21.15.92:0/4197284558 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7ec004850 msgr2=0x7fa7ec004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:07.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.385+0000 7fa80679c700 1 --2- 172.21.15.92:0/4197284558 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7ec004850 0x7fa7ec004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:07.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.385+0000 7fa80679c700 1 -- 172.21.15.92:0/4197284558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7ec003b90 msgr2=0x7fa7ec003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:07.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.385+0000 7fa80679c700 1 --2- 172.21.15.92:0/4197284558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa7ec003b90 0x7fa7ec003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:07.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.385+0000 7fa80679c700 1 -- 172.21.15.92:0/4197284558 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7ec0035b0 con 0x7fa7ec005200 2023-11-28T11:24:07.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.385+0000 7fa80679c700 1 -- 172.21.15.92:0/4197284558 reap_dead start 2023-11-28T11:24:07.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.385+0000 7fa80679c700 1 --2- 172.21.15.92:0/4197284558 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7ec005200 0x7fa7ec00ba10 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8009fc0 tx=0x7fa7ec005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20abc9c859ea509f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:07.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.385+0000 7fa804f99700 1 -- 172.21.15.92:0/4197284558 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f8003a20 con 0x7fa7ec005200 2023-11-28T11:24:07.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.385+0000 7fa804f99700 1 -- 172.21.15.92:0/4197284558 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7f800f040 con 0x7fa7ec005200 2023-11-28T11:24:07.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.385+0000 7fa804f99700 1 -- 172.21.15.92:0/4197284558 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f8003de0 con 0x7fa7ec005200 2023-11-28T11:24:07.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.386+0000 7fa8081ff700 1 -- 172.21.15.92:0/4197284558 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7ec005200 msgr2=0x7fa7ec00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:07.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.386+0000 7fa8081ff700 1 --2- 172.21.15.92:0/4197284558 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7ec005200 0x7fa7ec00ba10 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8009fc0 tx=0x7fa7ec005880 comp rx=0 tx=0).stop 2023-11-28T11:24:07.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.386+0000 7fa8081ff700 1 -- 172.21.15.92:0/4197284558 shutdown_connections 2023-11-28T11:24:07.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.386+0000 7fa8081ff700 1 --2- 172.21.15.92:0/4197284558 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7ec005200 0x7fa7ec00ba10 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:07.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.386+0000 7fa8081ff700 1 -- 172.21.15.92:0/4197284558 >> 172.21.15.92:0/4197284558 conn(0x7fa8000719a0 msgr2=0x7fa800071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:07.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.386+0000 7fa8081ff700 1 -- 172.21.15.92:0/4197284558 shutdown_connections 2023-11-28T11:24:07.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.386+0000 7fa8081ff700 1 -- 172.21.15.92:0/4197284558 wait complete. 2023-11-28T11:24:07.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.388+0000 7fa8081ff700 1 Processor -- start 2023-11-28T11:24:07.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.389+0000 7fa8081ff700 1 -- start start 2023-11-28T11:24:07.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.390+0000 7fa8081ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7ec005200 0x7fa8000cb460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:07.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.390+0000 7fa8081ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8000c9f30 0x7fa8000c8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:07.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.390+0000 7fa8081ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8000cc880 0x7fa8000ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:07.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.390+0000 7fa8081ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8000cd260 con 0x7fa8000c9f30 2023-11-28T11:24:07.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.390+0000 7fa8081ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa80015a980 con 0x7fa8000cc880 2023-11-28T11:24:07.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.390+0000 7fa8081ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8000ca3c0 con 0x7fa7ec005200 2023-11-28T11:24:07.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.390+0000 7fa80579a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8000c9f30 0x7fa8000c8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:07.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.390+0000 7fa80579a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8000c9f30 0x7fa8000c8f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50390/0 (socket says 172.21.15.92:50390) 2023-11-28T11:24:07.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.390+0000 7fa80579a700 1 -- 172.21.15.92:0/1303327567 learned_addr learned my addr 172.21.15.92:0/1303327567 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:07.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.391+0000 7fa80579a700 1 -- 172.21.15.92:0/1303327567 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7ec005200 msgr2=0x7fa8000cb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:07.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.391+0000 7fa80579a700 1 --2- 172.21.15.92:0/1303327567 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7ec005200 0x7fa8000cb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:07.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.391+0000 7fa80579a700 1 -- 172.21.15.92:0/1303327567 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8000cc880 msgr2=0x7fa8000ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:07.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.391+0000 7fa80579a700 1 --2- 172.21.15.92:0/1303327567 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8000cc880 0x7fa8000ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:07.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.391+0000 7fa80579a700 1 -- 172.21.15.92:0/1303327567 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8000b88f0 con 0x7fa8000c9f30 2023-11-28T11:24:07.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.391+0000 7fa80579a700 1 --2- 172.21.15.92:0/1303327567 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8000c9f30 0x7fa8000c8f80 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fa7f000f770 tx=0x7fa7f000fb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:07.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.391+0000 7fa7f67fc700 1 -- 172.21.15.92:0/1303327567 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f0005320 con 0x7fa8000c9f30 2023-11-28T11:24:07.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.391+0000 7fa7f67fc700 1 -- 172.21.15.92:0/1303327567 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7f00054a0 con 0x7fa8000c9f30 2023-11-28T11:24:07.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.391+0000 7fa7f67fc700 1 -- 172.21.15.92:0/1303327567 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f0006aa0 con 0x7fa8000c9f30 2023-11-28T11:24:07.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.391+0000 7fa8081ff700 1 -- 172.21.15.92:0/1303327567 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8000d15e0 con 0x7fa8000c9f30 2023-11-28T11:24:07.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.391+0000 7fa8081ff700 1 -- 172.21.15.92:0/1303327567 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8000b21b0 con 0x7fa8000c9f30 2023-11-28T11:24:07.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.392+0000 7fa8081ff700 1 -- 172.21.15.92:0/1303327567 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa800067170 con 0x7fa8000c9f30 2023-11-28T11:24:07.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.393+0000 7fa7f67fc700 1 -- 172.21.15.92:0/1303327567 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa7f0006c00 con 0x7fa8000c9f30 2023-11-28T11:24:07.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.394+0000 7fa7f67fc700 1 --2- 172.21.15.92:0/1303327567 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa7e40682f0 0x7fa7e406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:07.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.394+0000 7fa7f67fc700 1 -- 172.21.15.92:0/1303327567 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fa7f0085280 con 0x7fa8000c9f30 2023-11-28T11:24:07.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.394+0000 7fa805f9b700 1 --2- 172.21.15.92:0/1303327567 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa7e40682f0 0x7fa7e406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:07.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.395+0000 7fa805f9b700 1 --2- 172.21.15.92:0/1303327567 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa7e40682f0 0x7fa7e406a7a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc001eb0 tx=0x7fa7fc000ed0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:07.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.398+0000 7fa7f67fc700 1 -- 172.21.15.92:0/1303327567 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa7f0051bd0 con 0x7fa8000c9f30 2023-11-28T11:24:07.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.663+0000 7fa8081ff700 1 -- 172.21.15.92:0/1303327567 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) v1 -- 0x7fa8000b6f40 con 0x7fa8000c9f30 2023-11-28T11:24:07.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.726+0000 7fa7f67fc700 1 -- 172.21.15.92:0/1303327567 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v188) v1 ==== 149+0+0 (secure 0 0 0) 0x7fa7f00551f0 con 0x7fa8000c9f30 2023-11-28T11:24:07.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.728+0000 7fa8081ff700 1 -- 172.21.15.92:0/1303327567 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa7e40682f0 msgr2=0x7fa7e406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:07.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.728+0000 7fa8081ff700 1 --2- 172.21.15.92:0/1303327567 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa7e40682f0 0x7fa7e406a7a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc001eb0 tx=0x7fa7fc000ed0 comp rx=0 tx=0).stop 2023-11-28T11:24:07.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.728+0000 7fa8081ff700 1 -- 172.21.15.92:0/1303327567 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8000c9f30 msgr2=0x7fa8000c8f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:07.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.728+0000 7fa8081ff700 1 --2- 172.21.15.92:0/1303327567 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8000c9f30 0x7fa8000c8f80 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fa7f000f770 tx=0x7fa7f000fb30 comp rx=0 tx=0).stop 2023-11-28T11:24:07.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.729+0000 7fa8081ff700 1 -- 172.21.15.92:0/1303327567 shutdown_connections 2023-11-28T11:24:07.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.729+0000 7fa8081ff700 1 --2- 172.21.15.92:0/1303327567 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8000cc880 0x7fa8000ccd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:07.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.729+0000 7fa8081ff700 1 --2- 172.21.15.92:0/1303327567 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa7ec005200 0x7fa8000cb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:07.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.729+0000 7fa8081ff700 1 --2- 172.21.15.92:0/1303327567 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa7e40682f0 0x7fa7e406a7a0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:07.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.729+0000 7fa8081ff700 1 --2- 172.21.15.92:0/1303327567 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8000c9f30 0x7fa8000c8f80 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:07.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.729+0000 7fa8081ff700 1 -- 172.21.15.92:0/1303327567 >> 172.21.15.92:0/1303327567 conn(0x7fa8000719a0 msgr2=0x7fa8000b5c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:07.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.729+0000 7fa8081ff700 1 -- 172.21.15.92:0/1303327567 shutdown_connections 2023-11-28T11:24:07.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.729+0000 7fa8081ff700 1 -- 172.21.15.92:0/1303327567 wait complete. 2023-11-28T11:24:07.748 INFO:tasks.cephfs_test_runner:That the MDS can repair an inodes backtrace in the data pool ... ok 2023-11-28T11:24:07.749 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2023-11-28T11:24:07.749 INFO:tasks.cephfs_test_runner:test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T11:24:07.749 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2023-11-28T11:24:07.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.948+0000 7f7891dff700 1 Processor -- start 2023-11-28T11:24:07.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.948+0000 7f7891dff700 1 -- start start 2023-11-28T11:24:07.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.950+0000 7f7891dff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f788c0b8510 0x7f788c06b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:07.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.950+0000 7f7891dff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f788c0cd460 0x7f788c06beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:07.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.950+0000 7f7891dff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f788c0b88e0 con 0x7f788c06c3f0 2023-11-28T11:24:07.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.950+0000 7f7891dff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f788c0cdbd0 con 0x7f788c0b8510 2023-11-28T11:24:07.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.950+0000 7f7891dff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f788c0d02d0 con 0x7f788c0cd460 2023-11-28T11:24:07.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.950+0000 7f788b7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f788c0b8510 0x7f788c06b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:07.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.950+0000 7f788bfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f788c06c3f0 0x7f788c15cc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:55254/0 (socket says 172.21.15.92:55254) 2023-11-28T11:24:07.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.950+0000 7f788bfff700 1 -- 172.21.15.92:0/1657281098 learned_addr learned my addr 172.21.15.92:0/1657281098 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:24:07.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.951+0000 7f788affd700 1 --2- 172.21.15.92:0/1657281098 >> v2:172.21.15.169:3300/0 conn(0x7f788c0cd460 0x7f788c06beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:07.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.951+0000 7f788b7fe700 1 -- 172.21.15.92:0/1657281098 >> v2:172.21.15.169:3300/0 conn(0x7f788c0cd460 msgr2=0x7f788c06beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:07.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.951+0000 7f788b7fe700 1 --2- 172.21.15.92:0/1657281098 >> v2:172.21.15.169:3300/0 conn(0x7f788c0cd460 0x7f788c06beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:07.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.951+0000 7f788b7fe700 1 -- 172.21.15.92:0/1657281098 >> v1:172.21.15.92:6789/0 conn(0x7f788c06c3f0 legacy=0x7f788c15cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:07.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.951+0000 7f788b7fe700 1 -- 172.21.15.92:0/1657281098 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f788c0cd8f0 con 0x7f788c0b8510 2023-11-28T11:24:07.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.951+0000 7f788b7fe700 1 --2- 172.21.15.92:0/1657281098 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f788c0b8510 0x7f788c06b970 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f787c009870 tx=0x7f787c009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77fa35173780229b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:07.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.951+0000 7f788a7fc700 1 -- 172.21.15.92:0/1657281098 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787c011830 con 0x7f788c0b8510 2023-11-28T11:24:07.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.951+0000 7f788a7fc700 1 -- 172.21.15.92:0/1657281098 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f787c011990 con 0x7f788c0b8510 2023-11-28T11:24:07.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.951+0000 7f788a7fc700 1 -- 172.21.15.92:0/1657281098 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787c011d20 con 0x7f788c0b8510 2023-11-28T11:24:07.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.952+0000 7f7891dff700 1 -- 172.21.15.92:0/1657281098 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f788c0b8510 msgr2=0x7f788c06b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:07.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.952+0000 7f7891dff700 1 --2- 172.21.15.92:0/1657281098 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f788c0b8510 0x7f788c06b970 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f787c009870 tx=0x7f787c009b80 comp rx=0 tx=0).stop 2023-11-28T11:24:07.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.952+0000 7f7891dff700 1 -- 172.21.15.92:0/1657281098 shutdown_connections 2023-11-28T11:24:07.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.952+0000 7f7891dff700 1 --2- 172.21.15.92:0/1657281098 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f788c0b8510 0x7f788c06b970 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:07.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.952+0000 7f7891dff700 1 --2- 172.21.15.92:0/1657281098 >> v2:172.21.15.169:3300/0 conn(0x7f788c0cd460 0x7f788c06beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:07.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.952+0000 7f7891dff700 1 -- 172.21.15.92:0/1657281098 >> 172.21.15.92:0/1657281098 conn(0x7f788c0719a0 msgr2=0x7f788c071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:07.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.952+0000 7f7891dff700 1 -- 172.21.15.92:0/1657281098 shutdown_connections 2023-11-28T11:24:07.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.952+0000 7f7891dff700 1 -- 172.21.15.92:0/1657281098 wait complete. 2023-11-28T11:24:07.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.954+0000 7f7891dff700 1 Processor -- start 2023-11-28T11:24:07.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.954+0000 7f7891dff700 1 -- start start 2023-11-28T11:24:07.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.955+0000 7f7891dff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f788c06c3f0 0x7f788c156770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:07.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.955+0000 7f7891dff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f788c0b8510 0x7f788c0cac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:07.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.955+0000 7f7891dff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f788c0cd460 0x7f788c1582c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:07.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.955+0000 7f7891dff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788c158b60 con 0x7f788c0cd460 2023-11-28T11:24:07.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.955+0000 7f7891dff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788c158e40 con 0x7f788c0b8510 2023-11-28T11:24:07.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.955+0000 7f7891dff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f788c159120 con 0x7f788c06c3f0 2023-11-28T11:24:07.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.955+0000 7f788bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f788c0cd460 0x7f788c1582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:07.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.955+0000 7f788bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f788c0cd460 0x7f788c1582c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50412/0 (socket says 172.21.15.92:50412) 2023-11-28T11:24:07.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.955+0000 7f788bfff700 1 -- 172.21.15.92:0/523989936 learned_addr learned my addr 172.21.15.92:0/523989936 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:07.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.955+0000 7f788b7fe700 1 --2- 172.21.15.92:0/523989936 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f788c06c3f0 0x7f788c156770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:07.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.955+0000 7f788affd700 1 --2- 172.21.15.92:0/523989936 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f788c0b8510 0x7f788c0cac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:07.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.955+0000 7f788bfff700 1 -- 172.21.15.92:0/523989936 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f788c06c3f0 msgr2=0x7f788c156770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:07.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.955+0000 7f788bfff700 1 --2- 172.21.15.92:0/523989936 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f788c06c3f0 0x7f788c156770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:07.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.955+0000 7f788bfff700 1 -- 172.21.15.92:0/523989936 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f788c0b8510 msgr2=0x7f788c0cac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:07.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.955+0000 7f788bfff700 1 --2- 172.21.15.92:0/523989936 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f788c0b8510 0x7f788c0cac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:07.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.955+0000 7f788bfff700 1 -- 172.21.15.92:0/523989936 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f787c0094a0 con 0x7f788c0cd460 2023-11-28T11:24:07.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.956+0000 7f788bfff700 1 --2- 172.21.15.92:0/523989936 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f788c0cd460 0x7f788c1582c0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f788400cbc0 tx=0x7f788400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:07.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.956+0000 7f7873fff700 1 -- 172.21.15.92:0/523989936 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f788400a9e0 con 0x7f788c0cd460 2023-11-28T11:24:07.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.956+0000 7f7873fff700 1 -- 172.21.15.92:0/523989936 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f788400ab40 con 0x7f788c0cd460 2023-11-28T11:24:07.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.956+0000 7f7891dff700 1 -- 172.21.15.92:0/523989936 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f788c0cfff0 con 0x7f788c0cd460 2023-11-28T11:24:07.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.956+0000 7f7873fff700 1 -- 172.21.15.92:0/523989936 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7884012400 con 0x7f788c0cd460 2023-11-28T11:24:07.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.956+0000 7f7891dff700 1 -- 172.21.15.92:0/523989936 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f788c0cfd10 con 0x7f788c0cd460 2023-11-28T11:24:07.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.957+0000 7f7891dff700 1 -- 172.21.15.92:0/523989936 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f788c156cb0 con 0x7f788c0cd460 2023-11-28T11:24:07.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.958+0000 7f7873fff700 1 -- 172.21.15.92:0/523989936 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7884019030 con 0x7f788c0cd460 2023-11-28T11:24:07.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.959+0000 7f7873fff700 1 --2- 172.21.15.92:0/523989936 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f78780682f0 0x7f787806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:07.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.959+0000 7f7873fff700 1 -- 172.21.15.92:0/523989936 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f7884085460 con 0x7f788c0cd460 2023-11-28T11:24:07.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.959+0000 7f788b7fe700 1 --2- 172.21.15.92:0/523989936 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f78780682f0 0x7f787806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:07.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.960+0000 7f788b7fe700 1 --2- 172.21.15.92:0/523989936 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f78780682f0 0x7f787806a7a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f787c0090b0 tx=0x7f787c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:07.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:07.963+0000 7f7873fff700 1 -- 172.21.15.92:0/523989936 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f788404b5e0 con 0x7f788c0cd460 2023-11-28T11:24:08.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.224+0000 7f7891dff700 1 -- 172.21.15.92:0/523989936 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) v1 -- 0x7f788c067880 con 0x7f788c0cd460 2023-11-28T11:24:08.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.732+0000 7f7873fff700 1 -- 172.21.15.92:0/523989936 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v189) v1 ==== 172+0+0 (secure 0 0 0) 0x7f7884051f30 con 0x7f788c0cd460 2023-11-28T11:24:08.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.735+0000 7f7891dff700 1 -- 172.21.15.92:0/523989936 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f78780682f0 msgr2=0x7f787806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:08.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.735+0000 7f7891dff700 1 --2- 172.21.15.92:0/523989936 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f78780682f0 0x7f787806a7a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f787c0090b0 tx=0x7f787c008030 comp rx=0 tx=0).stop 2023-11-28T11:24:08.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.735+0000 7f7891dff700 1 -- 172.21.15.92:0/523989936 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f788c0cd460 msgr2=0x7f788c1582c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:08.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.735+0000 7f7891dff700 1 --2- 172.21.15.92:0/523989936 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f788c0cd460 0x7f788c1582c0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f788400cbc0 tx=0x7f788400ced0 comp rx=0 tx=0).stop 2023-11-28T11:24:08.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.735+0000 7f7891dff700 1 -- 172.21.15.92:0/523989936 shutdown_connections 2023-11-28T11:24:08.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.735+0000 7f7891dff700 1 --2- 172.21.15.92:0/523989936 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f78780682f0 0x7f787806a7a0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:08.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.735+0000 7f7891dff700 1 --2- 172.21.15.92:0/523989936 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f788c0cd460 0x7f788c1582c0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:08.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.735+0000 7f7891dff700 1 --2- 172.21.15.92:0/523989936 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f788c06c3f0 0x7f788c156770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:08.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.735+0000 7f7891dff700 1 --2- 172.21.15.92:0/523989936 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f788c0b8510 0x7f788c0cac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:08.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.735+0000 7f7891dff700 1 -- 172.21.15.92:0/523989936 >> 172.21.15.92:0/523989936 conn(0x7f788c0719a0 msgr2=0x7f788c0b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:08.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.735+0000 7f7891dff700 1 -- 172.21.15.92:0/523989936 shutdown_connections 2023-11-28T11:24:08.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.736+0000 7f7891dff700 1 -- 172.21.15.92:0/523989936 wait complete. 2023-11-28T11:24:08.758 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-28T11:24:08.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.957+0000 7f37ea1ff700 1 Processor -- start 2023-11-28T11:24:08.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.957+0000 7f37ea1ff700 1 -- start start 2023-11-28T11:24:08.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.959+0000 7f37ea1ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37e40b8510 0x7f37e406b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:08.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.959+0000 7f37ea1ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f37e40cd460 0x7f37e406beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:08.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.959+0000 7f37ea1ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37e40b88e0 con 0x7f37e406c3f0 2023-11-28T11:24:08.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.959+0000 7f37ea1ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37e40cdbd0 con 0x7f37e40cd460 2023-11-28T11:24:08.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.959+0000 7f37ea1ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37e406c830 con 0x7f37e40b8510 2023-11-28T11:24:08.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.959+0000 7f37e2ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f37e40cd460 0x7f37e406beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:08.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.959+0000 7f37e2ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f37e40cd460 0x7f37e406beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50418/0 (socket says 172.21.15.92:50418) 2023-11-28T11:24:08.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.959+0000 7f37e2ffd700 1 -- 172.21.15.92:0/2349769648 learned_addr learned my addr 172.21.15.92:0/2349769648 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:08.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.959+0000 7f37e2ffd700 1 -- 172.21.15.92:0/2349769648 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37e40b8510 msgr2=0x7f37e406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:08.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.959+0000 7f37e37fe700 1 --2- 172.21.15.92:0/2349769648 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37e40b8510 0x7f37e406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:08.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.959+0000 7f37e2ffd700 1 --2- 172.21.15.92:0/2349769648 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37e40b8510 0x7f37e406b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:08.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.959+0000 7f37e2ffd700 1 -- 172.21.15.92:0/2349769648 >> v1:172.21.15.169:6789/0 conn(0x7f37e406c3f0 legacy=0x7f37e415cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:08.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.959+0000 7f37e2ffd700 1 -- 172.21.15.92:0/2349769648 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37e40cd8f0 con 0x7f37e40cd460 2023-11-28T11:24:08.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.959+0000 7f37e37fe700 1 --2- 172.21.15.92:0/2349769648 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37e40b8510 0x7f37e406b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:24:08.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.960+0000 7f37e2ffd700 1 --2- 172.21.15.92:0/2349769648 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37e40cd460 0x7f37e406beb0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f37d8009920 tx=0x7f37d8009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26d84e9ceac1bb87 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:08.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.960+0000 7f37e27fc700 1 -- 172.21.15.92:0/2349769648 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d8004230 con 0x7f37e40cd460 2023-11-28T11:24:08.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.960+0000 7f37e27fc700 1 -- 172.21.15.92:0/2349769648 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37d8004390 con 0x7f37e40cd460 2023-11-28T11:24:08.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.960+0000 7f37e27fc700 1 -- 172.21.15.92:0/2349769648 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d8009e70 con 0x7f37e40cd460 2023-11-28T11:24:08.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.960+0000 7f37ea1ff700 1 -- 172.21.15.92:0/2349769648 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37e40cd460 msgr2=0x7f37e406beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:08.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.960+0000 7f37ea1ff700 1 --2- 172.21.15.92:0/2349769648 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37e40cd460 0x7f37e406beb0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f37d8009920 tx=0x7f37d8009c30 comp rx=0 tx=0).stop 2023-11-28T11:24:08.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.960+0000 7f37ea1ff700 1 -- 172.21.15.92:0/2349769648 shutdown_connections 2023-11-28T11:24:08.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.960+0000 7f37ea1ff700 1 --2- 172.21.15.92:0/2349769648 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37e40b8510 0x7f37e406b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:08.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.960+0000 7f37ea1ff700 1 --2- 172.21.15.92:0/2349769648 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37e40cd460 0x7f37e406beb0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:08.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.960+0000 7f37ea1ff700 1 -- 172.21.15.92:0/2349769648 >> 172.21.15.92:0/2349769648 conn(0x7f37e40719a0 msgr2=0x7f37e4071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:08.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.961+0000 7f37ea1ff700 1 -- 172.21.15.92:0/2349769648 shutdown_connections 2023-11-28T11:24:08.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.961+0000 7f37ea1ff700 1 -- 172.21.15.92:0/2349769648 wait complete. 2023-11-28T11:24:08.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.963+0000 7f37ea1ff700 1 Processor -- start 2023-11-28T11:24:08.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.963+0000 7f37ea1ff700 1 -- start start 2023-11-28T11:24:08.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.964+0000 7f37ea1ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37e406c3f0 0x7f37e41569f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:08.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.964+0000 7f37ea1ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37e40b8510 0x7f37e40ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:08.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.964+0000 7f37ea1ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37e40cd460 0x7f37e41490c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:08.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.964+0000 7f37ea1ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e40cc800 con 0x7f37e40cd460 2023-11-28T11:24:08.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.964+0000 7f37ea1ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e40cbbf0 con 0x7f37e40b8510 2023-11-28T11:24:08.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.964+0000 7f37ea1ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37e40cbed0 con 0x7f37e406c3f0 2023-11-28T11:24:08.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.965+0000 7f37e3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37e40cd460 0x7f37e41490c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:08.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.965+0000 7f37e3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37e40cd460 0x7f37e41490c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50432/0 (socket says 172.21.15.92:50432) 2023-11-28T11:24:08.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.965+0000 7f37e3fff700 1 -- 172.21.15.92:0/3542255542 learned_addr learned my addr 172.21.15.92:0/3542255542 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:08.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.965+0000 7f37e37fe700 1 --2- 172.21.15.92:0/3542255542 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37e406c3f0 0x7f37e41569f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:08.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.965+0000 7f37e2ffd700 1 --2- 172.21.15.92:0/3542255542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37e40b8510 0x7f37e40ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:08.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.965+0000 7f37e3fff700 1 -- 172.21.15.92:0/3542255542 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37e406c3f0 msgr2=0x7f37e41569f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:08.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.965+0000 7f37e3fff700 1 --2- 172.21.15.92:0/3542255542 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37e406c3f0 0x7f37e41569f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:08.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.965+0000 7f37e3fff700 1 -- 172.21.15.92:0/3542255542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37e40b8510 msgr2=0x7f37e40ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:08.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.965+0000 7f37e3fff700 1 --2- 172.21.15.92:0/3542255542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37e40b8510 0x7f37e40ccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:08.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.965+0000 7f37e3fff700 1 -- 172.21.15.92:0/3542255542 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37d8009550 con 0x7f37e40cd460 2023-11-28T11:24:08.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.965+0000 7f37e2ffd700 1 --2- 172.21.15.92:0/3542255542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37e40b8510 0x7f37e40ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:24:08.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.965+0000 7f37e3fff700 1 --2- 172.21.15.92:0/3542255542 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37e40cd460 0x7f37e41490c0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f37dc00cc10 tx=0x7f37dc00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:08.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.966+0000 7f37cbfff700 1 -- 172.21.15.92:0/3542255542 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37dc00abb0 con 0x7f37e40cd460 2023-11-28T11:24:08.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.966+0000 7f37cbfff700 1 -- 172.21.15.92:0/3542255542 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37dc00ad10 con 0x7f37e40cd460 2023-11-28T11:24:08.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.966+0000 7f37ea1ff700 1 -- 172.21.15.92:0/3542255542 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37e40cfff0 con 0x7f37e40cd460 2023-11-28T11:24:08.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.966+0000 7f37cbfff700 1 -- 172.21.15.92:0/3542255542 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37dc012660 con 0x7f37e40cd460 2023-11-28T11:24:08.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.966+0000 7f37ea1ff700 1 -- 172.21.15.92:0/3542255542 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37e40cfd10 con 0x7f37e40cd460 2023-11-28T11:24:08.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.967+0000 7f37ea1ff700 1 -- 172.21.15.92:0/3542255542 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37e40cc970 con 0x7f37e40cd460 2023-11-28T11:24:08.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.968+0000 7f37cbfff700 1 -- 172.21.15.92:0/3542255542 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f37dc022070 con 0x7f37e40cd460 2023-11-28T11:24:08.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.969+0000 7f37cbfff700 1 --2- 172.21.15.92:0/3542255542 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f37d00682f0 0x7f37d006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:08.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.969+0000 7f37cbfff700 1 -- 172.21.15.92:0/3542255542 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f37dc0861e0 con 0x7f37e40cd460 2023-11-28T11:24:08.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.969+0000 7f37e37fe700 1 --2- 172.21.15.92:0/3542255542 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f37d00682f0 0x7f37d006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:08.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.970+0000 7f37e37fe700 1 --2- 172.21.15.92:0/3542255542 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f37d00682f0 0x7f37d006a7a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f37d4006fd0 tx=0x7f37d4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:08.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:08.974+0000 7f37cbfff700 1 -- 172.21.15.92:0/3542255542 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f37dc052b30 con 0x7f37e40cd460 2023-11-28T11:24:09.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.163+0000 7f37ea1ff700 1 -- 172.21.15.92:0/3542255542 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f37e4067880 con 0x7f37e40cd460 2023-11-28T11:24:09.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.167+0000 7f37cbfff700 1 -- 172.21.15.92:0/3542255542 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f37dc0160c0 con 0x7f37e40cd460 2023-11-28T11:24:09.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.169+0000 7f37ea1ff700 1 -- 172.21.15.92:0/3542255542 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f37d00682f0 msgr2=0x7f37d006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:09.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.169+0000 7f37ea1ff700 1 --2- 172.21.15.92:0/3542255542 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f37d00682f0 0x7f37d006a7a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f37d4006fd0 tx=0x7f37d4008040 comp rx=0 tx=0).stop 2023-11-28T11:24:09.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.169+0000 7f37ea1ff700 1 -- 172.21.15.92:0/3542255542 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37e40cd460 msgr2=0x7f37e41490c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:09.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.169+0000 7f37ea1ff700 1 --2- 172.21.15.92:0/3542255542 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37e40cd460 0x7f37e41490c0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f37dc00cc10 tx=0x7f37dc00a060 comp rx=0 tx=0).stop 2023-11-28T11:24:09.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.169+0000 7f37ea1ff700 1 -- 172.21.15.92:0/3542255542 shutdown_connections 2023-11-28T11:24:09.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.169+0000 7f37ea1ff700 1 --2- 172.21.15.92:0/3542255542 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f37d00682f0 0x7f37d006a7a0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:09.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.169+0000 7f37ea1ff700 1 --2- 172.21.15.92:0/3542255542 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37e40cd460 0x7f37e41490c0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:09.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.169+0000 7f37ea1ff700 1 --2- 172.21.15.92:0/3542255542 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37e406c3f0 0x7f37e41569f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:09.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.169+0000 7f37ea1ff700 1 --2- 172.21.15.92:0/3542255542 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37e40b8510 0x7f37e40ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:09.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.169+0000 7f37ea1ff700 1 -- 172.21.15.92:0/3542255542 >> 172.21.15.92:0/3542255542 conn(0x7f37e40719a0 msgr2=0x7f37e40b7b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:09.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.169+0000 7f37ea1ff700 1 -- 172.21.15.92:0/3542255542 shutdown_connections 2023-11-28T11:24:09.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.169+0000 7f37ea1ff700 1 -- 172.21.15.92:0/3542255542 wait complete. 2023-11-28T11:24:09.189 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T11:24:09.189 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:24:09.276 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:24:09.378 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T11:24:09.378 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-28T11:24:09.378 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:24:09.424 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348288k,nr_inodes=4087072,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275616k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:24:09.425 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-28T11:24:09.425 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:24:09.475 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348296k,nr_inodes=4087074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275620k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:24:09.475 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-28T11:24:09.475 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:24:09.522 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348288k,nr_inodes=4087072,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275616k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:24:09.523 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-28T11:24:09.523 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:24:09.570 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348296k,nr_inodes=4087074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275620k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:24:09.570 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-28T11:24:09.570 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:24:09.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.765+0000 7ffb83fff700 1 Processor -- start 2023-11-28T11:24:09.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.765+0000 7ffb83fff700 1 -- start start 2023-11-28T11:24:09.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.766+0000 7ffb83fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ffb7c0cd460 0x7ffb7c06c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:09.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.766+0000 7ffb83fff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ffb7c0d0d60 0x7ffb7c0d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:09.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.766+0000 7ffb83fff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb7c0b88e0 con 0x7ffb7c0b8510 2023-11-28T11:24:09.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.766+0000 7ffb83fff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb7c0cdbd0 con 0x7ffb7c0cd460 2023-11-28T11:24:09.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.766+0000 7ffb83fff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb7c0d02d0 con 0x7ffb7c0d0d60 2023-11-28T11:24:09.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.766+0000 7ffb8159a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ffb7c0cd460 0x7ffb7c06c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:09.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.766+0000 7ffb8159a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ffb7c0cd460 0x7ffb7c06c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57460/0 (socket says 172.21.15.92:57460) 2023-11-28T11:24:09.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.766+0000 7ffb8159a700 1 -- 172.21.15.92:0/2790754670 learned_addr learned my addr 172.21.15.92:0/2790754670 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:09.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.767+0000 7ffb8259c700 1 --2- 172.21.15.92:0/2790754670 >> v2:172.21.15.169:3300/0 conn(0x7ffb7c0d0d60 0x7ffb7c0d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:09.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.767+0000 7ffb8159a700 1 -- 172.21.15.92:0/2790754670 >> v2:172.21.15.169:3300/0 conn(0x7ffb7c0d0d60 msgr2=0x7ffb7c0d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:09.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.767+0000 7ffb8159a700 1 --2- 172.21.15.92:0/2790754670 >> v2:172.21.15.169:3300/0 conn(0x7ffb7c0d0d60 0x7ffb7c0d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:09.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.767+0000 7ffb8159a700 1 -- 172.21.15.92:0/2790754670 >> v1:172.21.15.169:6789/0 conn(0x7ffb7c0b8510 legacy=0x7ffb7c06b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:09.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.767+0000 7ffb8159a700 1 -- 172.21.15.92:0/2790754670 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb7c0cd8f0 con 0x7ffb7c0cd460 2023-11-28T11:24:09.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.767+0000 7ffb8159a700 1 --2- 172.21.15.92:0/2790754670 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb7c0cd460 0x7ffb7c06c080 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c0ce8c0 tx=0x7ffb6c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa860108fd47bab6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:09.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.767+0000 7ffb80d99700 1 -- 172.21.15.92:0/2790754670 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c011830 con 0x7ffb7c0cd460 2023-11-28T11:24:09.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.767+0000 7ffb80d99700 1 -- 172.21.15.92:0/2790754670 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb6c011990 con 0x7ffb7c0cd460 2023-11-28T11:24:09.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.767+0000 7ffb80d99700 1 -- 172.21.15.92:0/2790754670 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c011d20 con 0x7ffb7c0cd460 2023-11-28T11:24:09.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.768+0000 7ffb83fff700 1 -- 172.21.15.92:0/2790754670 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb7c0cd460 msgr2=0x7ffb7c06c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:09.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.768+0000 7ffb83fff700 1 --2- 172.21.15.92:0/2790754670 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb7c0cd460 0x7ffb7c06c080 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c0ce8c0 tx=0x7ffb6c009c00 comp rx=0 tx=0).stop 2023-11-28T11:24:09.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.768+0000 7ffb83fff700 1 -- 172.21.15.92:0/2790754670 shutdown_connections 2023-11-28T11:24:09.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.768+0000 7ffb83fff700 1 --2- 172.21.15.92:0/2790754670 >> v2:172.21.15.169:3300/0 conn(0x7ffb7c0d0d60 0x7ffb7c0d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:09.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.768+0000 7ffb83fff700 1 --2- 172.21.15.92:0/2790754670 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb7c0cd460 0x7ffb7c06c080 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:09.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.768+0000 7ffb83fff700 1 -- 172.21.15.92:0/2790754670 >> 172.21.15.92:0/2790754670 conn(0x7ffb7c0719a0 msgr2=0x7ffb7c071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:09.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.768+0000 7ffb83fff700 1 -- 172.21.15.92:0/2790754670 shutdown_connections 2023-11-28T11:24:09.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.768+0000 7ffb83fff700 1 -- 172.21.15.92:0/2790754670 wait complete. 2023-11-28T11:24:09.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.770+0000 7ffb83fff700 1 Processor -- start 2023-11-28T11:24:09.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.770+0000 7ffb83fff700 1 -- start start 2023-11-28T11:24:09.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.771+0000 7ffb83fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb7c0b8510 0x7ffb7c1e1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:09.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.771+0000 7ffb83fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb7c0cd460 0x7ffb7c1e4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:09.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.771+0000 7ffb83fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb7c0d0d60 0x7ffb7c14de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:09.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.771+0000 7ffb83fff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7c1e5120 con 0x7ffb7c0cd460 2023-11-28T11:24:09.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.771+0000 7ffb83fff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7c1e2320 con 0x7ffb7c0b8510 2023-11-28T11:24:09.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.771+0000 7ffb83fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7c1e2600 con 0x7ffb7c0d0d60 2023-11-28T11:24:09.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.771+0000 7ffb8159a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb7c0cd460 0x7ffb7c1e4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:09.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.771+0000 7ffb8159a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb7c0cd460 0x7ffb7c1e4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57462/0 (socket says 172.21.15.92:57462) 2023-11-28T11:24:09.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.771+0000 7ffb8159a700 1 -- 172.21.15.92:0/3323877464 learned_addr learned my addr 172.21.15.92:0/3323877464 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:09.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.771+0000 7ffb81d9b700 1 --2- 172.21.15.92:0/3323877464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb7c0b8510 0x7ffb7c1e1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:09.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.771+0000 7ffb8259c700 1 --2- 172.21.15.92:0/3323877464 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb7c0d0d60 0x7ffb7c14de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:09.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.771+0000 7ffb8159a700 1 -- 172.21.15.92:0/3323877464 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb7c0d0d60 msgr2=0x7ffb7c14de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:09.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.771+0000 7ffb8159a700 1 --2- 172.21.15.92:0/3323877464 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb7c0d0d60 0x7ffb7c14de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:09.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.771+0000 7ffb8159a700 1 -- 172.21.15.92:0/3323877464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb7c0b8510 msgr2=0x7ffb7c1e1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:09.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.771+0000 7ffb8159a700 1 --2- 172.21.15.92:0/3323877464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb7c0b8510 0x7ffb7c1e1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:09.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.771+0000 7ffb8159a700 1 -- 172.21.15.92:0/3323877464 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb6c009550 con 0x7ffb7c0cd460 2023-11-28T11:24:09.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.772+0000 7ffb8159a700 1 --2- 172.21.15.92:0/3323877464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb7c0cd460 0x7ffb7c1e4760 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c009c40 tx=0x7ffb6c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:09.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.772+0000 7ffb727fc700 1 -- 172.21.15.92:0/3323877464 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c004300 con 0x7ffb7c0cd460 2023-11-28T11:24:09.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.772+0000 7ffb727fc700 1 -- 172.21.15.92:0/3323877464 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb6c01f070 con 0x7ffb7c0cd460 2023-11-28T11:24:09.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.772+0000 7ffb727fc700 1 -- 172.21.15.92:0/3323877464 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c0036a0 con 0x7ffb7c0cd460 2023-11-28T11:24:09.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.772+0000 7ffb83fff700 1 -- 172.21.15.92:0/3323877464 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb7c0cfff0 con 0x7ffb7c0cd460 2023-11-28T11:24:09.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.772+0000 7ffb83fff700 1 -- 172.21.15.92:0/3323877464 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb7c0cfd10 con 0x7ffb7c0cd460 2023-11-28T11:24:09.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.773+0000 7ffb83fff700 1 -- 172.21.15.92:0/3323877464 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb7c067880 con 0x7ffb7c0cd460 2023-11-28T11:24:09.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.774+0000 7ffb727fc700 1 -- 172.21.15.92:0/3323877464 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ffb6c02a020 con 0x7ffb7c0cd460 2023-11-28T11:24:09.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.775+0000 7ffb727fc700 1 --2- 172.21.15.92:0/3323877464 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ffb600682f0 0x7ffb6006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:09.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.775+0000 7ffb727fc700 1 -- 172.21.15.92:0/3323877464 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5616+0+0 (secure 0 0 0) 0x7ffb6c0870b0 con 0x7ffb7c0cd460 2023-11-28T11:24:09.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.775+0000 7ffb81d9b700 1 --2- 172.21.15.92:0/3323877464 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ffb600682f0 0x7ffb6006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:09.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.776+0000 7ffb81d9b700 1 --2- 172.21.15.92:0/3323877464 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ffb600682f0 0x7ffb6006a7a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ffb78007fd0 tx=0x7ffb78009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:09.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:09.779+0000 7ffb727fc700 1 -- 172.21.15.92:0/3323877464 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ffb6c053980 con 0x7ffb7c0cd460 2023-11-28T11:24:10.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.013+0000 7ffb83fff700 1 -- 172.21.15.92:0/3323877464 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb7c0d2940 con 0x7ffb7c0cd460 2023-11-28T11:24:10.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.014+0000 7ffb727fc700 1 -- 172.21.15.92:0/3323877464 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+3532 (secure 0 0 0) 0x7ffb6c015de0 con 0x7ffb7c0cd460 2023-11-28T11:24:10.016 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:24:10.016 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":26,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5593,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/3937189339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":3937189339},{"type":"v1","addr":"172.21.15.169:6834","nonce":3937189339}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5596,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2020201016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2020201016},{"type":"v1","addr":"172.21.15.169:6835","nonce":2020201016}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5751,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/2423750324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2423750324},{"type":"v1","addr":"172.21.15.92:6835","nonce":2423750324}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":6129,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/634034773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":634034773},{"type":"v1","addr":"172.21.15.92:6837","nonce":634034773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[]} 2023-11-28T11:24:10.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.016+0000 7ffb83fff700 1 -- 172.21.15.92:0/3323877464 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ffb600682f0 msgr2=0x7ffb6006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:10.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.016+0000 7ffb83fff700 1 --2- 172.21.15.92:0/3323877464 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ffb600682f0 0x7ffb6006a7a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ffb78007fd0 tx=0x7ffb78009040 comp rx=0 tx=0).stop 2023-11-28T11:24:10.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.016+0000 7ffb83fff700 1 -- 172.21.15.92:0/3323877464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb7c0cd460 msgr2=0x7ffb7c1e4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:10.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.016+0000 7ffb83fff700 1 --2- 172.21.15.92:0/3323877464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb7c0cd460 0x7ffb7c1e4760 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c009c40 tx=0x7ffb6c004750 comp rx=0 tx=0).stop 2023-11-28T11:24:10.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.016+0000 7ffb83fff700 1 -- 172.21.15.92:0/3323877464 shutdown_connections 2023-11-28T11:24:10.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.016+0000 7ffb83fff700 1 --2- 172.21.15.92:0/3323877464 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ffb7c0d0d60 0x7ffb7c14de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:10.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.016+0000 7ffb83fff700 1 --2- 172.21.15.92:0/3323877464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ffb7c0b8510 0x7ffb7c1e1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:10.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.016+0000 7ffb83fff700 1 --2- 172.21.15.92:0/3323877464 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ffb600682f0 0x7ffb6006a7a0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:10.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.016+0000 7ffb83fff700 1 --2- 172.21.15.92:0/3323877464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb7c0cd460 0x7ffb7c1e4760 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:10.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.016+0000 7ffb83fff700 1 -- 172.21.15.92:0/3323877464 >> 172.21.15.92:0/3323877464 conn(0x7ffb7c0719a0 msgr2=0x7ffb7c0b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:10.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.017+0000 7ffb83fff700 1 -- 172.21.15.92:0/3323877464 shutdown_connections 2023-11-28T11:24:10.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.017+0000 7ffb83fff700 1 -- 172.21.15.92:0/3323877464 wait complete. 2023-11-28T11:24:10.020 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 26 2023-11-28T11:24:10.037 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T11:24:10.037 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-28T11:24:10.037 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T11:24:10.038 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:24:10.038 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T11:24:10.038 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-28T11:24:10.038 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T11:24:10.038 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:24:10.038 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T11:24:10.038 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-28T11:24:10.039 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T11:24:10.039 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:24:10.039 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T11:24:10.039 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-28T11:24:10.039 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T11:24:10.039 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:24:10.051 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T11:24:10.052 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T11:24:10.052 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T11:24:10.053 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T11:24:10.054 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T11:24:10.054 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T11:24:10.055 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T11:24:10.055 DEBUG:teuthology.orchestra.run.smithi169:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T11:24:10.097 INFO:tasks.ceph.mds.a:Started 2023-11-28T11:24:10.097 DEBUG:teuthology.parallel:result is None 2023-11-28T11:24:10.099 INFO:tasks.ceph.mds.b:Started 2023-11-28T11:24:10.099 DEBUG:teuthology.parallel:result is None 2023-11-28T11:24:10.100 INFO:tasks.ceph.mds.c:Started 2023-11-28T11:24:10.100 DEBUG:teuthology.parallel:result is None 2023-11-28T11:24:10.102 INFO:tasks.ceph.mds.d:Started 2023-11-28T11:24:10.102 DEBUG:teuthology.parallel:result is None 2023-11-28T11:24:10.102 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-28T11:24:10.248 INFO:tasks.ceph.mds.b.smithi169.stdout:starting mds.b at 2023-11-28T11:24:10.249 INFO:tasks.ceph.mds.d.smithi169.stdout:starting mds.d at 2023-11-28T11:24:10.249 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-11-28T11:24:10.249 INFO:tasks.ceph.mds.c.smithi092.stdout:starting mds.c at 2023-11-28T11:24:10.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.314+0000 7fbb4a3ff700 1 Processor -- start 2023-11-28T11:24:10.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.315+0000 7fbb4a3ff700 1 -- start start 2023-11-28T11:24:10.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.316+0000 7fbb4a3ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fbb440cd460 0x7fbb4406c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:10.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.316+0000 7fbb4a3ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb440d0d60 0x7fbb440d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:10.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.316+0000 7fbb4a3ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb440b88e0 con 0x7fbb440b8510 2023-11-28T11:24:10.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.316+0000 7fbb4a3ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb440cdbd0 con 0x7fbb440cd460 2023-11-28T11:24:10.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.316+0000 7fbb4a3ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb440d02d0 con 0x7fbb440d0d60 2023-11-28T11:24:10.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.316+0000 7fbb43fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fbb440b8510 0x7fbb4406b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47440/0 (socket says 172.21.15.92:47440) 2023-11-28T11:24:10.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.316+0000 7fbb43fff700 1 -- 172.21.15.92:0/1248031234 learned_addr learned my addr 172.21.15.92:0/1248031234 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:24:10.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.316+0000 7fbb437fe700 1 --2- 172.21.15.92:0/1248031234 >> v2:172.21.15.169:3300/0 conn(0x7fbb440cd460 0x7fbb4406c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:10.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.316+0000 7fbb4899c700 1 --2- 172.21.15.92:0/1248031234 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb440d0d60 0x7fbb440d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:10.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.316+0000 7fbb42ffd700 1 -- 172.21.15.92:0/1248031234 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2020629493 0 0) 0x7fbb440b88e0 con 0x7fbb440b8510 2023-11-28T11:24:10.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.316+0000 7fbb42ffd700 1 -- 172.21.15.92:0/1248031234 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb2c0035b0 con 0x7fbb440b8510 2023-11-28T11:24:10.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.317+0000 7fbb437fe700 1 -- 172.21.15.92:0/1248031234 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb440d0d60 msgr2=0x7fbb440d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:10.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.317+0000 7fbb42ffd700 1 -- 172.21.15.92:0/1248031234 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2505093504 0 0) 0x7fbb2c0035b0 con 0x7fbb440b8510 2023-11-28T11:24:10.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.317+0000 7fbb437fe700 1 --2- 172.21.15.92:0/1248031234 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb440d0d60 0x7fbb440d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:10.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.317+0000 7fbb437fe700 1 -- 172.21.15.92:0/1248031234 >> v1:172.21.15.92:6789/0 conn(0x7fbb440b8510 legacy=0x7fbb4406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:10.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.317+0000 7fbb437fe700 1 -- 172.21.15.92:0/1248031234 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb440cd8f0 con 0x7fbb440cd460 2023-11-28T11:24:10.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.317+0000 7fbb4899c700 1 --2- 172.21.15.92:0/1248031234 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb440d0d60 0x7fbb440d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:24:10.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.317+0000 7fbb437fe700 1 --2- 172.21.15.92:0/1248031234 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb440cd460 0x7fbb4406c080 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fbb38009870 tx=0x7fbb38009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c3fcec99144ec16 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:10.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.317+0000 7fbb42ffd700 1 -- 172.21.15.92:0/1248031234 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb34001670 con 0x7fbb440cd460 2023-11-28T11:24:10.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.317+0000 7fbb42ffd700 1 -- 172.21.15.92:0/1248031234 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb38004030 con 0x7fbb440cd460 2023-11-28T11:24:10.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.317+0000 7fbb42ffd700 1 -- 172.21.15.92:0/1248031234 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb380043c0 con 0x7fbb440cd460 2023-11-28T11:24:10.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.318+0000 7fbb4a3ff700 1 -- 172.21.15.92:0/1248031234 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb440cd460 msgr2=0x7fbb4406c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:10.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.318+0000 7fbb4a3ff700 1 --2- 172.21.15.92:0/1248031234 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb440cd460 0x7fbb4406c080 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fbb38009870 tx=0x7fbb38009b80 comp rx=0 tx=0).stop 2023-11-28T11:24:10.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.318+0000 7fbb4a3ff700 1 -- 172.21.15.92:0/1248031234 shutdown_connections 2023-11-28T11:24:10.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.318+0000 7fbb4a3ff700 1 --2- 172.21.15.92:0/1248031234 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb440d0d60 0x7fbb440d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:10.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.318+0000 7fbb4a3ff700 1 --2- 172.21.15.92:0/1248031234 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb440cd460 0x7fbb4406c080 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:10.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.318+0000 7fbb4a3ff700 1 -- 172.21.15.92:0/1248031234 >> 172.21.15.92:0/1248031234 conn(0x7fbb440719a0 msgr2=0x7fbb44071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:10.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.318+0000 7fbb4a3ff700 1 -- 172.21.15.92:0/1248031234 shutdown_connections 2023-11-28T11:24:10.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.318+0000 7fbb4a3ff700 1 -- 172.21.15.92:0/1248031234 wait complete. 2023-11-28T11:24:10.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.320+0000 7fbb4a3ff700 1 Processor -- start 2023-11-28T11:24:10.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.320+0000 7fbb4a3ff700 1 -- start start 2023-11-28T11:24:10.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.321+0000 7fbb4a3ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb440b8510 0x7fbb440b0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:10.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.321+0000 7fbb4a3ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb440cd460 0x7fbb441f46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:10.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.321+0000 7fbb4a3ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb440d0d60 0x7fbb441f4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:10.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.321+0000 7fbb4a3ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb441e2440 con 0x7fbb440cd460 2023-11-28T11:24:10.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.321+0000 7fbb4a3ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb441e1aa0 con 0x7fbb440b8510 2023-11-28T11:24:10.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.321+0000 7fbb4a3ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb441e1d80 con 0x7fbb440d0d60 2023-11-28T11:24:10.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.321+0000 7fbb437fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb440cd460 0x7fbb441f46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:10.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.321+0000 7fbb43fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb440b8510 0x7fbb440b0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:10.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.321+0000 7fbb437fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb440cd460 0x7fbb441f46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57508/0 (socket says 172.21.15.92:57508) 2023-11-28T11:24:10.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.321+0000 7fbb437fe700 1 -- 172.21.15.92:0/878790998 learned_addr learned my addr 172.21.15.92:0/878790998 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:10.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.321+0000 7fbb4899c700 1 --2- 172.21.15.92:0/878790998 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb440d0d60 0x7fbb441f4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:10.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.321+0000 7fbb437fe700 1 -- 172.21.15.92:0/878790998 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb440d0d60 msgr2=0x7fbb441f4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:10.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.321+0000 7fbb437fe700 1 --2- 172.21.15.92:0/878790998 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb440d0d60 0x7fbb441f4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:10.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.321+0000 7fbb437fe700 1 -- 172.21.15.92:0/878790998 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb440b8510 msgr2=0x7fbb440b0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:10.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.321+0000 7fbb437fe700 1 --2- 172.21.15.92:0/878790998 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb440b8510 0x7fbb440b0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:10.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.321+0000 7fbb437fe700 1 -- 172.21.15.92:0/878790998 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb3c009a70 con 0x7fbb440cd460 2023-11-28T11:24:10.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.322+0000 7fbb437fe700 1 --2- 172.21.15.92:0/878790998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb440cd460 0x7fbb441f46b0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fbb380090b0 tx=0x7fbb38004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:10.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.322+0000 7fbb40ff9700 1 -- 172.21.15.92:0/878790998 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb38003cb0 con 0x7fbb440cd460 2023-11-28T11:24:10.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.322+0000 7fbb40ff9700 1 -- 172.21.15.92:0/878790998 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb3801e070 con 0x7fbb440cd460 2023-11-28T11:24:10.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.322+0000 7fbb4a3ff700 1 -- 172.21.15.92:0/878790998 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb380094a0 con 0x7fbb440cd460 2023-11-28T11:24:10.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.322+0000 7fbb40ff9700 1 -- 172.21.15.92:0/878790998 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb38011830 con 0x7fbb440cd460 2023-11-28T11:24:10.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.322+0000 7fbb4a3ff700 1 -- 172.21.15.92:0/878790998 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb440cfff0 con 0x7fbb440cd460 2023-11-28T11:24:10.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.323+0000 7fbb4a3ff700 1 -- 172.21.15.92:0/878790998 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb440cfd10 con 0x7fbb440cd460 2023-11-28T11:24:10.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.324+0000 7fbb40ff9700 1 -- 172.21.15.92:0/878790998 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbb38017020 con 0x7fbb440cd460 2023-11-28T11:24:10.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.325+0000 7fbb40ff9700 1 --2- 172.21.15.92:0/878790998 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fbb300682f0 0x7fbb3006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:10.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.325+0000 7fbb40ff9700 1 -- 172.21.15.92:0/878790998 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fbb38086a70 con 0x7fbb440cd460 2023-11-28T11:24:10.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.325+0000 7fbb43fff700 1 --2- 172.21.15.92:0/878790998 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fbb300682f0 0x7fbb3006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:10.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.326+0000 7fbb43fff700 1 --2- 172.21.15.92:0/878790998 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fbb300682f0 0x7fbb3006a7a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fbb340017a0 tx=0x7fbb34001230 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:10.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.329+0000 7fbb40ff9700 1 -- 172.21.15.92:0/878790998 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbb380533c0 con 0x7fbb440cd460 2023-11-28T11:24:10.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:10.539+0000 7fbb4a3ff700 1 -- 172.21.15.92:0/878790998 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fbb441e2580 con 0x7fbb440cd460 2023-11-28T11:24:11.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.291+0000 7fbb40ff9700 1 -- 172.21.15.92:0/878790998 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v64) v1 ==== 97+0+0 (secure 0 0 0) 0x7fbb441e2580 con 0x7fbb440cd460 2023-11-28T11:24:11.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.294+0000 7fbb4a3ff700 1 -- 172.21.15.92:0/878790998 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fbb300682f0 msgr2=0x7fbb3006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:11.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.294+0000 7fbb4a3ff700 1 --2- 172.21.15.92:0/878790998 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fbb300682f0 0x7fbb3006a7a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fbb340017a0 tx=0x7fbb34001230 comp rx=0 tx=0).stop 2023-11-28T11:24:11.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.294+0000 7fbb4a3ff700 1 -- 172.21.15.92:0/878790998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb440cd460 msgr2=0x7fbb441f46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:11.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.294+0000 7fbb4a3ff700 1 --2- 172.21.15.92:0/878790998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb440cd460 0x7fbb441f46b0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fbb380090b0 tx=0x7fbb38004750 comp rx=0 tx=0).stop 2023-11-28T11:24:11.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.294+0000 7fbb4a3ff700 1 -- 172.21.15.92:0/878790998 shutdown_connections 2023-11-28T11:24:11.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.294+0000 7fbb4a3ff700 1 --2- 172.21.15.92:0/878790998 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbb440d0d60 0x7fbb441f4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:11.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.294+0000 7fbb4a3ff700 1 --2- 172.21.15.92:0/878790998 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbb440b8510 0x7fbb440b0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:11.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.294+0000 7fbb4a3ff700 1 --2- 172.21.15.92:0/878790998 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fbb300682f0 0x7fbb3006a7a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:11.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.294+0000 7fbb4a3ff700 1 --2- 172.21.15.92:0/878790998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb440cd460 0x7fbb441f46b0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:11.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.294+0000 7fbb4a3ff700 1 -- 172.21.15.92:0/878790998 >> 172.21.15.92:0/878790998 conn(0x7fbb440719a0 msgr2=0x7fbb440b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:11.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.294+0000 7fbb4a3ff700 1 -- 172.21.15.92:0/878790998 shutdown_connections 2023-11-28T11:24:11.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.294+0000 7fbb4a3ff700 1 -- 172.21.15.92:0/878790998 wait complete. 2023-11-28T11:24:11.297 INFO:teuthology.orchestra.run.smithi092.stderr: removed all blocklist entries 2023-11-28T11:24:11.317 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-28T11:24:11.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.514+0000 7fd2531ff700 1 Processor -- start 2023-11-28T11:24:11.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.514+0000 7fd2531ff700 1 -- start start 2023-11-28T11:24:11.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.516+0000 7fd2531ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd24c0cd460 0x7fd24c06c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:11.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.516+0000 7fd2531ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd24c0b88e0 con 0x7fd24c0d0d60 2023-11-28T11:24:11.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.516+0000 7fd2531ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd24c0cd8f0 con 0x7fd24c0b8510 2023-11-28T11:24:11.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.516+0000 7fd2531ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd24c0cdbd0 con 0x7fd24c0cd460 2023-11-28T11:24:11.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.516+0000 7fd24bfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd24c0cd460 0x7fd24c06c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:11.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.516+0000 7fd25179c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd24c0d0d60 0x7fd24c0d1200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47450/0 (socket says 172.21.15.92:47450) 2023-11-28T11:24:11.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.516+0000 7fd25179c700 1 -- 172.21.15.92:0/3537827229 learned_addr learned my addr 172.21.15.92:0/3537827229 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:24:11.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.516+0000 7fd24bfff700 1 -- 172.21.15.92:0/3537827229 >> v1:172.21.15.169:6789/0 conn(0x7fd24c0b8510 legacy=0x7fd24c06b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:11.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.516+0000 7fd24bfff700 1 -- 172.21.15.92:0/3537827229 >> v1:172.21.15.92:6789/0 conn(0x7fd24c0d0d60 legacy=0x7fd24c0d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:11.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.516+0000 7fd24bfff700 1 -- 172.21.15.92:0/3537827229 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd24c0cfd10 con 0x7fd24c0cd460 2023-11-28T11:24:11.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.517+0000 7fd24bfff700 1 --2- 172.21.15.92:0/3537827229 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd24c0cd460 0x7fd24c06c080 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fd238009840 tx=0x7fd238009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf0914a0de32b4a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:11.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.517+0000 7fd24b7fe700 1 -- 172.21.15.92:0/3537827229 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd238004230 con 0x7fd24c0cd460 2023-11-28T11:24:11.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.517+0000 7fd24b7fe700 1 -- 172.21.15.92:0/3537827229 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd238004390 con 0x7fd24c0cd460 2023-11-28T11:24:11.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.517+0000 7fd24b7fe700 1 -- 172.21.15.92:0/3537827229 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2380036a0 con 0x7fd24c0cd460 2023-11-28T11:24:11.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.517+0000 7fd2531ff700 1 -- 172.21.15.92:0/3537827229 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd24c0cd460 msgr2=0x7fd24c06c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:11.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.517+0000 7fd2531ff700 1 --2- 172.21.15.92:0/3537827229 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd24c0cd460 0x7fd24c06c080 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fd238009840 tx=0x7fd238009b50 comp rx=0 tx=0).stop 2023-11-28T11:24:11.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.517+0000 7fd2531ff700 1 -- 172.21.15.92:0/3537827229 shutdown_connections 2023-11-28T11:24:11.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.517+0000 7fd2531ff700 1 --2- 172.21.15.92:0/3537827229 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd24c0cd460 0x7fd24c06c080 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:11.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.517+0000 7fd2531ff700 1 -- 172.21.15.92:0/3537827229 >> 172.21.15.92:0/3537827229 conn(0x7fd24c0719a0 msgr2=0x7fd24c071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:11.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.518+0000 7fd2531ff700 1 -- 172.21.15.92:0/3537827229 shutdown_connections 2023-11-28T11:24:11.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.518+0000 7fd2531ff700 1 -- 172.21.15.92:0/3537827229 wait complete. 2023-11-28T11:24:11.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.519+0000 7fd2531ff700 1 Processor -- start 2023-11-28T11:24:11.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.519+0000 7fd2531ff700 1 -- start start 2023-11-28T11:24:11.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.520+0000 7fd2531ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd24c0b8510 0x7fd24c0cbad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:11.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.520+0000 7fd2531ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd24c0cd460 0x7fd24c0c9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:11.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.520+0000 7fd2531ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd24c0d0d60 0x7fd24c15aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:11.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.520+0000 7fd2531ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd24c0cd050 con 0x7fd24c0d0d60 2023-11-28T11:24:11.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.520+0000 7fd2531ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd24c0cc3d0 con 0x7fd24c0cd460 2023-11-28T11:24:11.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.520+0000 7fd2531ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd24c0cc6b0 con 0x7fd24c0b8510 2023-11-28T11:24:11.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.520+0000 7fd25179c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd24c0d0d60 0x7fd24c15aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:11.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.521+0000 7fd25179c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd24c0d0d60 0x7fd24c15aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57530/0 (socket says 172.21.15.92:57530) 2023-11-28T11:24:11.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.521+0000 7fd25179c700 1 -- 172.21.15.92:0/3662049464 learned_addr learned my addr 172.21.15.92:0/3662049464 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:11.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.521+0000 7fd24bfff700 1 --2- 172.21.15.92:0/3662049464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd24c0cd460 0x7fd24c0c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:11.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.521+0000 7fd250f9b700 1 --2- 172.21.15.92:0/3662049464 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd24c0b8510 0x7fd24c0cbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:11.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.521+0000 7fd25179c700 1 -- 172.21.15.92:0/3662049464 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd24c0b8510 msgr2=0x7fd24c0cbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:11.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.521+0000 7fd25179c700 1 --2- 172.21.15.92:0/3662049464 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd24c0b8510 0x7fd24c0cbad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:11.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.521+0000 7fd25179c700 1 -- 172.21.15.92:0/3662049464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd24c0cd460 msgr2=0x7fd24c0c9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:11.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.521+0000 7fd25179c700 1 --2- 172.21.15.92:0/3662049464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd24c0cd460 0x7fd24c0c9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:11.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.521+0000 7fd25179c700 1 -- 172.21.15.92:0/3662049464 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2380094a0 con 0x7fd24c0d0d60 2023-11-28T11:24:11.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.521+0000 7fd24bfff700 1 --2- 172.21.15.92:0/3662049464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd24c0cd460 0x7fd24c0c9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:24:11.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.521+0000 7fd250f9b700 1 --2- 172.21.15.92:0/3662049464 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd24c0b8510 0x7fd24c0cbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:24:11.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.521+0000 7fd25179c700 1 --2- 172.21.15.92:0/3662049464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd24c0d0d60 0x7fd24c15aa70 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fd24400ca80 tx=0x7fd24400ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:11.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.521+0000 7fd2497fa700 1 -- 172.21.15.92:0/3662049464 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24400aaa0 con 0x7fd24c0d0d60 2023-11-28T11:24:11.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.521+0000 7fd2497fa700 1 -- 172.21.15.92:0/3662049464 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd24400ac00 con 0x7fd24c0d0d60 2023-11-28T11:24:11.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.521+0000 7fd2531ff700 1 -- 172.21.15.92:0/3662049464 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd24c0d29c0 con 0x7fd24c0d0d60 2023-11-28T11:24:11.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.521+0000 7fd2497fa700 1 -- 172.21.15.92:0/3662049464 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd244012400 con 0x7fd24c0d0d60 2023-11-28T11:24:11.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.522+0000 7fd2531ff700 1 -- 172.21.15.92:0/3662049464 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd24c0cfff0 con 0x7fd24c0d0d60 2023-11-28T11:24:11.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.523+0000 7fd2531ff700 1 -- 172.21.15.92:0/3662049464 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd24c0d2b50 con 0x7fd24c0d0d60 2023-11-28T11:24:11.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.523+0000 7fd2497fa700 1 -- 172.21.15.92:0/3662049464 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd244019030 con 0x7fd24c0d0d60 2023-11-28T11:24:11.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.525+0000 7fd2497fa700 1 --2- 172.21.15.92:0/3662049464 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd2300682f0 0x7fd23006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:11.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.525+0000 7fd2497fa700 1 -- 172.21.15.92:0/3662049464 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fd244085550 con 0x7fd24c0d0d60 2023-11-28T11:24:11.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.525+0000 7fd250f9b700 1 --2- 172.21.15.92:0/3662049464 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd2300682f0 0x7fd23006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:11.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.525+0000 7fd250f9b700 1 --2- 172.21.15.92:0/3662049464 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd2300682f0 0x7fd23006a7a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd240003fc0 tx=0x7fd240009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:11.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.528+0000 7fd2497fa700 1 -- 172.21.15.92:0/3662049464 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd24404b5e0 con 0x7fd24c0d0d60 2023-11-28T11:24:11.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.791+0000 7fd2531ff700 1 -- 172.21.15.92:0/3662049464 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fd24c0cd190 con 0x7fd24c0d0d60 2023-11-28T11:24:11.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.792+0000 7fd2497fa700 1 -- 172.21.15.92:0/3662049464 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5298 (secure 0 0 0) 0x7fd24c0cd190 con 0x7fd24c0d0d60 2023-11-28T11:24:11.794 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:24:11.794 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-28T11:24:11.794 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_dump": [ 2023-11-28T11:24:11.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:24:11.794 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.a", 2023-11-28T11:24:11.794 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDXzGVlH/PGCBAAor2D9Zz+LcAPQsi1BhIuQA==", 2023-11-28T11:24:11.795 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:24:11.795 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-11-28T11:24:11.795 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-11-28T11:24:11.795 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-11-28T11:24:11.795 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-28T11:24:11.795 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:24:11.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:24:11.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:24:11.795 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.b", 2023-11-28T11:24:11.795 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDXzGVlamQ0IBAAbJVb+gWu1RqSykFqIM7Hhg==", 2023-11-28T11:24:11.795 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:24:11.796 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-11-28T11:24:11.796 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-11-28T11:24:11.796 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-11-28T11:24:11.796 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-28T11:24:11.796 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:24:11.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:24:11.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:24:11.796 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.c", 2023-11-28T11:24:11.796 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDXzGVlpfloFBAAVSQxpfstP6SpWbWUMhB/hg==", 2023-11-28T11:24:11.796 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:24:11.797 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-11-28T11:24:11.797 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-11-28T11:24:11.797 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-11-28T11:24:11.797 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-28T11:24:11.798 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:24:11.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:24:11.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:24:11.798 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.d", 2023-11-28T11:24:11.798 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDXzGVlqQ8ALBAAESaHrVIhfnF5ocn44xdS5A==", 2023-11-28T11:24:11.798 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:24:11.798 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-11-28T11:24:11.798 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-11-28T11:24:11.799 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-11-28T11:24:11.799 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-28T11:24:11.799 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:24:11.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:24:11.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:24:11.799 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.0", 2023-11-28T11:24:11.799 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDbzGVl9MVvKxAA1Pd/AN+XdW47auUMbazqdw==", 2023-11-28T11:24:11.799 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:24:11.799 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-11-28T11:24:11.799 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-11-28T11:24:11.799 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-28T11:24:11.800 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:24:11.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:24:11.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:24:11.800 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.1", 2023-11-28T11:24:11.800 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDezGVldH9fLxAASVpF1+s0RIj4ObUNZoLDKg==", 2023-11-28T11:24:11.800 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:24:11.800 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-11-28T11:24:11.800 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-11-28T11:24:11.800 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-28T11:24:11.800 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:24:11.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:24:11.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:24:11.801 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.2", 2023-11-28T11:24:11.801 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDhzGVl+hnoMxAAXG0K3wm2UYF3BUeGjOkHhw==", 2023-11-28T11:24:11.801 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:24:11.801 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-11-28T11:24:11.801 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-11-28T11:24:11.801 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-28T11:24:11.801 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:24:11.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:24:11.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:24:11.802 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.3", 2023-11-28T11:24:11.802 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDkzGVlBB0EORAAQ3HJyXBI2AFfb/I5REmALA==", 2023-11-28T11:24:11.802 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:24:11.802 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-11-28T11:24:11.802 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-11-28T11:24:11.802 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-28T11:24:11.802 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:24:11.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:24:11.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:24:11.802 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.4", 2023-11-28T11:24:11.802 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDrzGVlgOjDGBAAqwSISduMc7UF8VWdiSNJXQ==", 2023-11-28T11:24:11.803 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:24:11.803 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-11-28T11:24:11.803 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-11-28T11:24:11.803 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-28T11:24:11.803 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:24:11.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:24:11.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:24:11.803 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.5", 2023-11-28T11:24:11.803 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDuzGVl9FMMHhAAohYzehTD1xpWI+/zxipY4Q==", 2023-11-28T11:24:11.803 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:24:11.803 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-11-28T11:24:11.804 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-11-28T11:24:11.804 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-28T11:24:11.804 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:24:11.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:24:11.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:24:11.804 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.6", 2023-11-28T11:24:11.804 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDxzGVl+gOMIhAA0FwY/JqNT12aDRet8sSTYw==", 2023-11-28T11:24:11.804 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:24:11.804 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-11-28T11:24:11.804 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-11-28T11:24:11.805 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-28T11:24:11.805 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:24:11.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:24:11.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:24:11.805 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.7", 2023-11-28T11:24:11.805 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQD0zGVlFPIiJhAATxpSqw+VG3u46b/IIb+yqA==", 2023-11-28T11:24:11.805 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:24:11.805 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-11-28T11:24:11.805 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-11-28T11:24:11.805 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-28T11:24:11.806 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:24:11.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:24:11.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:24:11.806 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.0", 2023-11-28T11:24:11.806 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDXzGVl8nA7NhAAGB5c8Mp6PgErGowSH4BBdw==", 2023-11-28T11:24:11.806 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:24:11.806 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-11-28T11:24:11.807 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow r", 2023-11-28T11:24:11.807 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rw pool=cephfs_data" 2023-11-28T11:24:11.807 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:24:11.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:24:11.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:24:11.807 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.1", 2023-11-28T11:24:11.807 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDYzGVlbl/KCRAA23f+Q5tU6l1e3gNiomDzqQ==", 2023-11-28T11:24:11.807 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:24:11.807 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-11-28T11:24:11.807 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-11-28T11:24:11.808 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-11-28T11:24:11.808 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-11-28T11:24:11.808 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:24:11.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:24:11.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:24:11.808 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.2", 2023-11-28T11:24:11.808 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDYzGVl1oM3AhAAdrqZFbX0CRVpZ1q531/+Dg==", 2023-11-28T11:24:11.808 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:24:11.808 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-11-28T11:24:11.808 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-11-28T11:24:11.808 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-11-28T11:24:11.808 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-11-28T11:24:11.809 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:24:11.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:24:11.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:24:11.809 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.3", 2023-11-28T11:24:11.809 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDYzGVlGZJjERAA6iq24nyDkkyLjaWb9NkrtA==", 2023-11-28T11:24:11.809 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:24:11.809 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-11-28T11:24:11.809 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-11-28T11:24:11.809 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-11-28T11:24:11.809 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-11-28T11:24:11.809 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:24:11.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:24:11.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:24:11.810 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.admin", 2023-11-28T11:24:11.810 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDWzGVlmiaaDBAAcA6vmjgBJaurjIyTEGXdLg==", 2023-11-28T11:24:11.810 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:24:11.810 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-11-28T11:24:11.810 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-11-28T11:24:11.810 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-11-28T11:24:11.810 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-28T11:24:11.810 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:24:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:24:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:24:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.x", 2023-11-28T11:24:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDXzGVlb4AkARAAxs1XCTncmt28cyu+vL2AYA==", 2023-11-28T11:24:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:24:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-11-28T11:24:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-11-28T11:24:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-28T11:24:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:24:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:24:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:24:11.811 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.y", 2023-11-28T11:24:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDWzGVldLQpNBAAhdEG2ocL6T0ByZkLnwic2g==", 2023-11-28T11:24:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-11-28T11:24:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-11-28T11:24:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-11-28T11:24:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-11-28T11:24:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:24:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:24:11.812 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-28T11:24:11.812 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-28T11:24:11.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.794+0000 7fd2531ff700 1 -- 172.21.15.92:0/3662049464 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd2300682f0 msgr2=0x7fd23006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:11.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.794+0000 7fd2531ff700 1 --2- 172.21.15.92:0/3662049464 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd2300682f0 0x7fd23006a7a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd240003fc0 tx=0x7fd240009040 comp rx=0 tx=0).stop 2023-11-28T11:24:11.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.794+0000 7fd2531ff700 1 -- 172.21.15.92:0/3662049464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd24c0d0d60 msgr2=0x7fd24c15aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:11.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.794+0000 7fd2531ff700 1 --2- 172.21.15.92:0/3662049464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd24c0d0d60 0x7fd24c15aa70 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fd24400ca80 tx=0x7fd24400ce40 comp rx=0 tx=0).stop 2023-11-28T11:24:11.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.794+0000 7fd2531ff700 1 -- 172.21.15.92:0/3662049464 shutdown_connections 2023-11-28T11:24:11.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.794+0000 7fd2531ff700 1 --2- 172.21.15.92:0/3662049464 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd2300682f0 0x7fd23006a7a0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:11.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.794+0000 7fd2531ff700 1 --2- 172.21.15.92:0/3662049464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd24c0d0d60 0x7fd24c15aa70 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:11.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.794+0000 7fd2531ff700 1 --2- 172.21.15.92:0/3662049464 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd24c0b8510 0x7fd24c0cbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:11.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.794+0000 7fd2531ff700 1 --2- 172.21.15.92:0/3662049464 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd24c0cd460 0x7fd24c0c9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:11.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.794+0000 7fd2531ff700 1 -- 172.21.15.92:0/3662049464 >> 172.21.15.92:0/3662049464 conn(0x7fd24c0719a0 msgr2=0x7fd24c06b970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:11.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.794+0000 7fd2531ff700 1 -- 172.21.15.92:0/3662049464 shutdown_connections 2023-11-28T11:24:11.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:11.794+0000 7fd2531ff700 1 -- 172.21.15.92:0/3662049464 wait complete. 2023-11-28T11:24:11.816 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:24:12.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.012+0000 7fe0345ff700 1 Processor -- start 2023-11-28T11:24:12.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.012+0000 7fe0345ff700 1 -- start start 2023-11-28T11:24:12.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.013+0000 7fe0345ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe02c0d1d70 0x7fe02c167270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:12.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.013+0000 7fe0345ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe02c06c790 con 0x7fe02c0b8510 2023-11-28T11:24:12.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.014+0000 7fe0345ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe02c0b88e0 con 0x7fe02c0cd460 2023-11-28T11:24:12.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.014+0000 7fe0345ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe02c0cdbd0 con 0x7fe02c0d1d70 2023-11-28T11:24:12.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.014+0000 7fe03239b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe02c0b8510 0x7fe02c06b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47462/0 (socket says 172.21.15.92:47462) 2023-11-28T11:24:12.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.014+0000 7fe03239b700 1 -- 172.21.15.92:0/2782201733 learned_addr learned my addr 172.21.15.92:0/2782201733 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:24:12.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.014+0000 7fe032b9c700 1 --2- 172.21.15.92:0/2782201733 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe02c0d1d70 0x7fe02c167270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:12.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.014+0000 7fe031399700 1 -- 172.21.15.92:0/2782201733 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2598110321 0 0) 0x7fe02c06c790 con 0x7fe02c0b8510 2023-11-28T11:24:12.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.014+0000 7fe031399700 1 -- 172.21.15.92:0/2782201733 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0180035b0 con 0x7fe02c0b8510 2023-11-28T11:24:12.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.014+0000 7fe031399700 1 -- 172.21.15.92:0/2782201733 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 795404797 0 0) 0x7fe0180035b0 con 0x7fe02c0b8510 2023-11-28T11:24:12.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.014+0000 7fe031399700 1 -- 172.21.15.92:0/2782201733 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe02c0d1d70 msgr2=0x7fe02c167270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:12.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.014+0000 7fe031399700 1 --2- 172.21.15.92:0/2782201733 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe02c0d1d70 0x7fe02c167270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:12.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.014+0000 7fe031399700 1 -- 172.21.15.92:0/2782201733 >> v1:172.21.15.169:6789/0 conn(0x7fe02c0cd460 legacy=0x7fe02c06c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:12.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.015+0000 7fe031399700 1 -- 172.21.15.92:0/2782201733 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe02c0cd8f0 con 0x7fe02c0b8510 2023-11-28T11:24:12.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.015+0000 7fe031399700 1 -- 172.21.15.92:0/2782201733 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532161587 0 0) 0x7fe028001670 con 0x7fe02c0b8510 2023-11-28T11:24:12.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.015+0000 7fe031399700 1 -- 172.21.15.92:0/2782201733 >> v1:172.21.15.92:6789/0 conn(0x7fe02c0b8510 legacy=0x7fe02c06b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:12.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.015+0000 7fe031399700 1 --2- 172.21.15.92:0/2782201733 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe018003a30 0x7fe018003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:12.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.015+0000 7fe031399700 1 --2- 172.21.15.92:0/2782201733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe018004850 0x7fe018004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:12.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.015+0000 7fe031399700 1 --2- 172.21.15.92:0/2782201733 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe02c0d1d70 0x7fe018005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:12.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.015+0000 7fe031399700 1 -- 172.21.15.92:0/2782201733 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02c0cdbd0 con 0x7fe018004850 2023-11-28T11:24:12.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.015+0000 7fe031399700 1 -- 172.21.15.92:0/2782201733 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01800bb20 con 0x7fe018003a30 2023-11-28T11:24:12.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.015+0000 7fe031399700 1 -- 172.21.15.92:0/2782201733 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe01800bce0 con 0x7fe02c0d1d70 2023-11-28T11:24:12.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.015+0000 7fe031b9a700 1 --2- 172.21.15.92:0/2782201733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe018004850 0x7fe018004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:12.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.015+0000 7fe03239b700 1 --2- 172.21.15.92:0/2782201733 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe018003a30 0x7fe018003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:12.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.015+0000 7fe032b9c700 1 --2- 172.21.15.92:0/2782201733 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe02c0d1d70 0x7fe018005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:12.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.015+0000 7fe031b9a700 1 -- 172.21.15.92:0/2782201733 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe02c0d1d70 msgr2=0x7fe018005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:12.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.015+0000 7fe031b9a700 1 --2- 172.21.15.92:0/2782201733 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe02c0d1d70 0x7fe018005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:12.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.015+0000 7fe031b9a700 1 -- 172.21.15.92:0/2782201733 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe018003a30 msgr2=0x7fe018003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:12.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.015+0000 7fe031b9a700 1 --2- 172.21.15.92:0/2782201733 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe018003a30 0x7fe018003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:12.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.015+0000 7fe031b9a700 1 -- 172.21.15.92:0/2782201733 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0180035b0 con 0x7fe018004850 2023-11-28T11:24:12.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.016+0000 7fe031b9a700 1 --2- 172.21.15.92:0/2782201733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe018004850 0x7fe018004cc0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fe02c08af10 tx=0x7fe018005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c104f422545f4e4e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:12.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.016+0000 7fe031399700 1 -- 172.21.15.92:0/2782201733 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01c0052d0 con 0x7fe018004850 2023-11-28T11:24:12.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.016+0000 7fe031399700 1 -- 172.21.15.92:0/2782201733 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe01c005430 con 0x7fe018004850 2023-11-28T11:24:12.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.016+0000 7fe031399700 1 -- 172.21.15.92:0/2782201733 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01c0038c0 con 0x7fe018004850 2023-11-28T11:24:12.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.016+0000 7fe0345ff700 1 -- 172.21.15.92:0/2782201733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe018004850 msgr2=0x7fe018004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:12.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.016+0000 7fe0345ff700 1 --2- 172.21.15.92:0/2782201733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe018004850 0x7fe018004cc0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fe02c08af10 tx=0x7fe018005880 comp rx=0 tx=0).stop 2023-11-28T11:24:12.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.017+0000 7fe032b9c700 1 -- 172.21.15.92:0/2782201733 reap_dead start 2023-11-28T11:24:12.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.017+0000 7fe0345ff700 1 -- 172.21.15.92:0/2782201733 shutdown_connections 2023-11-28T11:24:12.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.017+0000 7fe0345ff700 1 -- 172.21.15.92:0/2782201733 >> 172.21.15.92:0/2782201733 conn(0x7fe02c0719a0 msgr2=0x7fe02c071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:12.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.017+0000 7fe0345ff700 1 -- 172.21.15.92:0/2782201733 shutdown_connections 2023-11-28T11:24:12.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.017+0000 7fe0345ff700 1 -- 172.21.15.92:0/2782201733 wait complete. 2023-11-28T11:24:12.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.018+0000 7fe0345ff700 1 Processor -- start 2023-11-28T11:24:12.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.019+0000 7fe0345ff700 1 -- start start 2023-11-28T11:24:12.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.020+0000 7fe0345ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe018004850 0x7fe02c157ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:12.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.020+0000 7fe0345ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe02c0c99c0 0x7fe02c0c8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:12.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.020+0000 7fe0345ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe02c1470a0 0x7fe02c0b6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:12.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.020+0000 7fe0345ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02c1548c0 con 0x7fe018004850 2023-11-28T11:24:12.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.020+0000 7fe0345ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02c0c8920 con 0x7fe02c1470a0 2023-11-28T11:24:12.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.020+0000 7fe0345ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe02c0c8c00 con 0x7fe02c0c99c0 2023-11-28T11:24:12.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.020+0000 7fe03239b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe018004850 0x7fe02c157ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:12.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.020+0000 7fe03239b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe018004850 0x7fe02c157ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57558/0 (socket says 172.21.15.92:57558) 2023-11-28T11:24:12.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.020+0000 7fe03239b700 1 -- 172.21.15.92:0/2832512869 learned_addr learned my addr 172.21.15.92:0/2832512869 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:12.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.020+0000 7fe031b9a700 1 --2- 172.21.15.92:0/2832512869 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe02c0c99c0 0x7fe02c0c8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:12.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.020+0000 7fe032b9c700 1 --2- 172.21.15.92:0/2832512869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe02c1470a0 0x7fe02c0b6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:12.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.020+0000 7fe03239b700 1 -- 172.21.15.92:0/2832512869 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe02c0c99c0 msgr2=0x7fe02c0c8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:12.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.020+0000 7fe03239b700 1 --2- 172.21.15.92:0/2832512869 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe02c0c99c0 0x7fe02c0c8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:12.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.020+0000 7fe03239b700 1 -- 172.21.15.92:0/2832512869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe02c1470a0 msgr2=0x7fe02c0b6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:12.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.020+0000 7fe03239b700 1 --2- 172.21.15.92:0/2832512869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe02c1470a0 0x7fe02c0b6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:12.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.020+0000 7fe03239b700 1 -- 172.21.15.92:0/2832512869 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe02c0cfff0 con 0x7fe018004850 2023-11-28T11:24:12.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.020+0000 7fe03239b700 1 --2- 172.21.15.92:0/2832512869 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe018004850 0x7fe02c157ec0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fe028007fd0 tx=0x7fe028007570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:12.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.021+0000 7fe022ffd700 1 -- 172.21.15.92:0/2832512869 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe028011580 con 0x7fe018004850 2023-11-28T11:24:12.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.021+0000 7fe022ffd700 1 -- 172.21.15.92:0/2832512869 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe028011720 con 0x7fe018004850 2023-11-28T11:24:12.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.021+0000 7fe0345ff700 1 -- 172.21.15.92:0/2832512869 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe02c0cfd10 con 0x7fe018004850 2023-11-28T11:24:12.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.021+0000 7fe022ffd700 1 -- 172.21.15.92:0/2832512869 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe028011ad0 con 0x7fe018004850 2023-11-28T11:24:12.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.021+0000 7fe0345ff700 1 -- 172.21.15.92:0/2832512869 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe02c0d29c0 con 0x7fe018004850 2023-11-28T11:24:12.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.022+0000 7fe0345ff700 1 -- 172.21.15.92:0/2832512869 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe02c067880 con 0x7fe018004850 2023-11-28T11:24:12.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.023+0000 7fe022ffd700 1 -- 172.21.15.92:0/2832512869 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe028011c30 con 0x7fe018004850 2023-11-28T11:24:12.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.024+0000 7fe022ffd700 1 --2- 172.21.15.92:0/2832512869 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe0100682f0 0x7fe01006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:12.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.024+0000 7fe022ffd700 1 -- 172.21.15.92:0/2832512869 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fe028085c00 con 0x7fe018004850 2023-11-28T11:24:12.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.024+0000 7fe031b9a700 1 --2- 172.21.15.92:0/2832512869 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe0100682f0 0x7fe01006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:12.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.025+0000 7fe031b9a700 1 --2- 172.21.15.92:0/2832512869 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe0100682f0 0x7fe01006a7a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe01c000c00 tx=0x7fe01c009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:12.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.028+0000 7fe022ffd700 1 -- 172.21.15.92:0/2832512869 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe028052630 con 0x7fe018004850 2023-11-28T11:24:12.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.220+0000 7fe0345ff700 1 -- 172.21.15.92:0/2832512869 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe02c0b2b60 con 0x7fe018004850 2023-11-28T11:24:12.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.220+0000 7fe022ffd700 1 -- 172.21.15.92:0/2832512869 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+14472 (secure 0 0 0) 0x7fe028055c50 con 0x7fe018004850 2023-11-28T11:24:12.222 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:24:12.223 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":64,"fsid":"68956e0c-c16f-47cd-a220-f98364b0831d","created":"2023-11-28T11:20:28.697573+0000","modified":"2023-11-28T11:24:11.288040+0000","last_up_change":"2023-11-28T11:20:41.257011+0000","last_in_change":"2023-11-28T11:20:32.249767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:20:41.925237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:20:48.682497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6abe483f-0e03-4a26-b9cf-62041291747e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":115823},{"type":"v1","addr":"172.21.15.92:6809","nonce":115823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":115823},{"type":"v1","addr":"172.21.15.92:6813","nonce":115823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":115823},{"type":"v1","addr":"172.21.15.92:6821","nonce":115823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":115823},{"type":"v1","addr":"172.21.15.92:6817","nonce":115823}]},"public_addr":"172.21.15.92:6809/115823","cluster_addr":"172.21.15.92:6813/115823","heartbeat_back_addr":"172.21.15.92:6821/115823","heartbeat_front_addr":"172.21.15.92:6817/115823","state":["exists","up"]},{"osd":1,"uuid":"13c1de73-1a8c-4d7e-8a28-9dae52dfe089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":115824},{"type":"v1","addr":"172.21.15.92:6812","nonce":115824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":115824},{"type":"v1","addr":"172.21.15.92:6816","nonce":115824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":115824},{"type":"v1","addr":"172.21.15.92:6823","nonce":115824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":115824},{"type":"v1","addr":"172.21.15.92:6820","nonce":115824}]},"public_addr":"172.21.15.92:6812/115824","cluster_addr":"172.21.15.92:6816/115824","heartbeat_back_addr":"172.21.15.92:6823/115824","heartbeat_front_addr":"172.21.15.92:6820/115824","state":["exists","up"]},{"osd":2,"uuid":"25788a55-112f-4f00-810d-2bc37cf76778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":115821},{"type":"v1","addr":"172.21.15.92:6801","nonce":115821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":115821},{"type":"v1","addr":"172.21.15.92:6803","nonce":115821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":115821},{"type":"v1","addr":"172.21.15.92:6807","nonce":115821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":115821},{"type":"v1","addr":"172.21.15.92:6805","nonce":115821}]},"public_addr":"172.21.15.92:6801/115821","cluster_addr":"172.21.15.92:6803/115821","heartbeat_back_addr":"172.21.15.92:6807/115821","heartbeat_front_addr":"172.21.15.92:6805/115821","state":["exists","up"]},{"osd":3,"uuid":"adf54e24-33a9-41fe-9ce3-98a5ab6c158e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":115825},{"type":"v1","addr":"172.21.15.92:6825","nonce":115825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":115825},{"type":"v1","addr":"172.21.15.92:6827","nonce":115825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":115825},{"type":"v1","addr":"172.21.15.92:6831","nonce":115825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":115825},{"type":"v1","addr":"172.21.15.92:6829","nonce":115825}]},"public_addr":"172.21.15.92:6825/115825","cluster_addr":"172.21.15.92:6827/115825","heartbeat_back_addr":"172.21.15.92:6831/115825","heartbeat_front_addr":"172.21.15.92:6829/115825","state":["exists","up"]},{"osd":4,"uuid":"ccb9b747-f399-44ea-9a18-9db3152227e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":116035},{"type":"v1","addr":"172.21.15.169:6817","nonce":116035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":116035},{"type":"v1","addr":"172.21.15.169:6819","nonce":116035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":116035},{"type":"v1","addr":"172.21.15.169:6826","nonce":116035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":116035},{"type":"v1","addr":"172.21.15.169:6822","nonce":116035}]},"public_addr":"172.21.15.169:6817/116035","cluster_addr":"172.21.15.169:6819/116035","heartbeat_back_addr":"172.21.15.169:6826/116035","heartbeat_front_addr":"172.21.15.169:6822/116035","state":["exists","up"]},{"osd":5,"uuid":"308a1e63-75aa-4d7d-8a7c-cecc24714fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":116033},{"type":"v1","addr":"172.21.15.169:6801","nonce":116033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":116033},{"type":"v1","addr":"172.21.15.169:6803","nonce":116033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":116033},{"type":"v1","addr":"172.21.15.169:6807","nonce":116033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":116033},{"type":"v1","addr":"172.21.15.169:6805","nonce":116033}]},"public_addr":"172.21.15.169:6801/116033","cluster_addr":"172.21.15.169:6803/116033","heartbeat_back_addr":"172.21.15.169:6807/116033","heartbeat_front_addr":"172.21.15.169:6805/116033","state":["exists","up"]},{"osd":6,"uuid":"9c459b7b-b821-4b4b-b91e-6aaae271457c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":116034},{"type":"v1","addr":"172.21.15.169:6809","nonce":116034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":116034},{"type":"v1","addr":"172.21.15.169:6811","nonce":116034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":116034},{"type":"v1","addr":"172.21.15.169:6815","nonce":116034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":116034},{"type":"v1","addr":"172.21.15.169:6813","nonce":116034}]},"public_addr":"172.21.15.169:6809/116034","cluster_addr":"172.21.15.169:6811/116034","heartbeat_back_addr":"172.21.15.169:6815/116034","heartbeat_front_addr":"172.21.15.169:6813/116034","state":["exists","up"]},{"osd":7,"uuid":"1ca9aa4a-755d-4e28-a937-4d8dccab4df3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":116044},{"type":"v1","addr":"172.21.15.169:6823","nonce":116044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6825","nonce":116044},{"type":"v1","addr":"172.21.15.169:6827","nonce":116044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":116044},{"type":"v1","addr":"172.21.15.169:6831","nonce":116044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":116044},{"type":"v1","addr":"172.21.15.169:6829","nonce":116044}]},"public_addr":"172.21.15.169:6823/116044","cluster_addr":"172.21.15.169:6827/116044","heartbeat_back_addr":"172.21.15.169:6831/116044","heartbeat_front_addr":"172.21.15.169:6829/116044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.586781+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.869778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.508594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.860493+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.872267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.746933+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.611520+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.871446+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:24:12.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.223+0000 7fe0345ff700 1 -- 172.21.15.92:0/2832512869 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe0100682f0 msgr2=0x7fe01006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:12.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.223+0000 7fe0345ff700 1 --2- 172.21.15.92:0/2832512869 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe0100682f0 0x7fe01006a7a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe01c000c00 tx=0x7fe01c009030 comp rx=0 tx=0).stop 2023-11-28T11:24:12.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.223+0000 7fe0345ff700 1 -- 172.21.15.92:0/2832512869 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe018004850 msgr2=0x7fe02c157ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:12.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.223+0000 7fe0345ff700 1 --2- 172.21.15.92:0/2832512869 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe018004850 0x7fe02c157ec0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fe028007fd0 tx=0x7fe028007570 comp rx=0 tx=0).stop 2023-11-28T11:24:12.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.223+0000 7fe0345ff700 1 -- 172.21.15.92:0/2832512869 shutdown_connections 2023-11-28T11:24:12.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.223+0000 7fe0345ff700 1 --2- 172.21.15.92:0/2832512869 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe02c1470a0 0x7fe02c0b6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:12.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.223+0000 7fe0345ff700 1 --2- 172.21.15.92:0/2832512869 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe0100682f0 0x7fe01006a7a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:12.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.223+0000 7fe0345ff700 1 --2- 172.21.15.92:0/2832512869 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe018004850 0x7fe02c157ec0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:12.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.223+0000 7fe0345ff700 1 --2- 172.21.15.92:0/2832512869 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe02c0c99c0 0x7fe02c0c8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:12.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.223+0000 7fe0345ff700 1 -- 172.21.15.92:0/2832512869 >> 172.21.15.92:0/2832512869 conn(0x7fe02c0719a0 msgr2=0x7fe02c15d400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:12.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.223+0000 7fe0345ff700 1 -- 172.21.15.92:0/2832512869 shutdown_connections 2023-11-28T11:24:12.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.223+0000 7fe0345ff700 1 -- 172.21.15.92:0/2832512869 wait complete. 2023-11-28T11:24:12.243 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:20:41.925237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:20:48.682497+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:24:12.244 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:24:12.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.441+0000 7f67027ff700 1 Processor -- start 2023-11-28T11:24:12.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.441+0000 7f67027ff700 1 -- start start 2023-11-28T11:24:12.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.442+0000 7f67027ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f66fc15a110 0x7f66fc160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:12.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.442+0000 7f67027ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f66fc0cfec0 0x7f66fc0b19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:12.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.443+0000 7f67027ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66fc161070 con 0x7f66fc15d820 2023-11-28T11:24:12.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.443+0000 7f67027ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66fc0cdae0 con 0x7f66fc0cfec0 2023-11-28T11:24:12.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.443+0000 7f67027ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66fc0ce4f0 con 0x7f66fc15a110 2023-11-28T11:24:12.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.443+0000 7f6700d9c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f66fc0cfec0 0x7f66fc0b19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:12.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.443+0000 7f6700d9c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f66fc0cfec0 0x7f66fc0b19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57574/0 (socket says 172.21.15.92:57574) 2023-11-28T11:24:12.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.443+0000 7f6700d9c700 1 -- 172.21.15.92:0/1524316483 learned_addr learned my addr 172.21.15.92:0/1524316483 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:12.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.443+0000 7f66fbfff700 1 --2- 172.21.15.92:0/1524316483 >> v2:172.21.15.169:3300/0 conn(0x7f66fc15a110 0x7f66fc160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:12.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.443+0000 7f6700d9c700 1 -- 172.21.15.92:0/1524316483 >> v2:172.21.15.169:3300/0 conn(0x7f66fc15a110 msgr2=0x7f66fc160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:12.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.443+0000 7f6700d9c700 1 --2- 172.21.15.92:0/1524316483 >> v2:172.21.15.169:3300/0 conn(0x7f66fc15a110 0x7f66fc160b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:12.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.443+0000 7f6700d9c700 1 -- 172.21.15.92:0/1524316483 >> v1:172.21.15.169:6789/0 conn(0x7f66fc15d820 legacy=0x7f66fc0cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:12.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.443+0000 7f6700d9c700 1 -- 172.21.15.92:0/1524316483 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66fc0cf2d0 con 0x7f66fc0cfec0 2023-11-28T11:24:12.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.444+0000 7f6700d9c700 1 --2- 172.21.15.92:0/1524316483 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66fc0cfec0 0x7f66fc0b19b0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f66fc08bf90 tx=0x7f66f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f758b2e4e332b50c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:12.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.444+0000 7f66faffd700 1 -- 172.21.15.92:0/1524316483 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f400e070 con 0x7f66fc0cfec0 2023-11-28T11:24:12.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.444+0000 7f66faffd700 1 -- 172.21.15.92:0/1524316483 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66f400b910 con 0x7f66fc0cfec0 2023-11-28T11:24:12.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.444+0000 7f66faffd700 1 -- 172.21.15.92:0/1524316483 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f400bcc0 con 0x7f66fc0cfec0 2023-11-28T11:24:12.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.444+0000 7f67027ff700 1 -- 172.21.15.92:0/1524316483 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66fc0cfec0 msgr2=0x7f66fc0b19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:12.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.444+0000 7f67027ff700 1 --2- 172.21.15.92:0/1524316483 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66fc0cfec0 0x7f66fc0b19b0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f66fc08bf90 tx=0x7f66f400b330 comp rx=0 tx=0).stop 2023-11-28T11:24:12.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.444+0000 7f67027ff700 1 -- 172.21.15.92:0/1524316483 shutdown_connections 2023-11-28T11:24:12.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.444+0000 7f67027ff700 1 --2- 172.21.15.92:0/1524316483 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66fc0cfec0 0x7f66fc0b19b0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:12.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.444+0000 7f67027ff700 1 --2- 172.21.15.92:0/1524316483 >> v2:172.21.15.169:3300/0 conn(0x7f66fc15a110 0x7f66fc160b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:12.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.444+0000 7f67027ff700 1 -- 172.21.15.92:0/1524316483 >> 172.21.15.92:0/1524316483 conn(0x7f66fc0719a0 msgr2=0x7f66fc071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:12.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.444+0000 7f67027ff700 1 -- 172.21.15.92:0/1524316483 shutdown_connections 2023-11-28T11:24:12.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.445+0000 7f67027ff700 1 -- 172.21.15.92:0/1524316483 wait complete. 2023-11-28T11:24:12.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.446+0000 7f67027ff700 1 Processor -- start 2023-11-28T11:24:12.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.446+0000 7f67027ff700 1 -- start start 2023-11-28T11:24:12.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.447+0000 7f67027ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66fc0cfec0 0x7f66fc150b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:12.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.447+0000 7f67027ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66fc15a110 0x7f66fc15a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:12.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.447+0000 7f67027ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66fc15d820 0x7f66fc15aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:12.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.447+0000 7f67027ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66fc0c7710 con 0x7f66fc15a110 2023-11-28T11:24:12.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.447+0000 7f67027ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66fc0c6b00 con 0x7f66fc15d820 2023-11-28T11:24:12.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.447+0000 7f67027ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66fc0c6de0 con 0x7f66fc0cfec0 2023-11-28T11:24:12.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.447+0000 7f66fb7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66fc15a110 0x7f66fc15a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:12.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.448+0000 7f66fb7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66fc15a110 0x7f66fc15a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57580/0 (socket says 172.21.15.92:57580) 2023-11-28T11:24:12.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.448+0000 7f66fb7fe700 1 -- 172.21.15.92:0/1125730984 learned_addr learned my addr 172.21.15.92:0/1125730984 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:12.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.448+0000 7f6700d9c700 1 --2- 172.21.15.92:0/1125730984 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66fc15d820 0x7f66fc15aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:12.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.448+0000 7f66fbfff700 1 --2- 172.21.15.92:0/1125730984 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66fc0cfec0 0x7f66fc150b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:12.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.448+0000 7f66fb7fe700 1 -- 172.21.15.92:0/1125730984 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66fc0cfec0 msgr2=0x7f66fc150b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:12.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.448+0000 7f66fb7fe700 1 --2- 172.21.15.92:0/1125730984 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66fc0cfec0 0x7f66fc150b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:12.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.448+0000 7f66fb7fe700 1 -- 172.21.15.92:0/1125730984 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66fc15d820 msgr2=0x7f66fc15aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:12.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.448+0000 7f66fb7fe700 1 --2- 172.21.15.92:0/1125730984 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66fc15d820 0x7f66fc15aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:12.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.448+0000 7f66fb7fe700 1 -- 172.21.15.92:0/1125730984 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f4009b30 con 0x7f66fc15a110 2023-11-28T11:24:12.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.448+0000 7f66fbfff700 1 --2- 172.21.15.92:0/1125730984 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66fc0cfec0 0x7f66fc150b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:24:12.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.448+0000 7f6700d9c700 1 --2- 172.21.15.92:0/1125730984 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66fc15d820 0x7f66fc15aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:24:12.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.448+0000 7f66fb7fe700 1 --2- 172.21.15.92:0/1125730984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66fc15a110 0x7f66fc15a960 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f66f000c450 tx=0x7f66f000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:12.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.448+0000 7f66f8ff9700 1 -- 172.21.15.92:0/1125730984 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f00052d0 con 0x7f66fc15a110 2023-11-28T11:24:12.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.448+0000 7f66f8ff9700 1 -- 172.21.15.92:0/1125730984 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66f0005430 con 0x7f66fc15a110 2023-11-28T11:24:12.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.448+0000 7f67027ff700 1 -- 172.21.15.92:0/1125730984 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66fc15cd40 con 0x7f66fc15a110 2023-11-28T11:24:12.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.448+0000 7f66f8ff9700 1 -- 172.21.15.92:0/1125730984 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f00038c0 con 0x7f66fc15a110 2023-11-28T11:24:12.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.449+0000 7f67027ff700 1 -- 172.21.15.92:0/1125730984 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66fc0cd800 con 0x7f66fc15a110 2023-11-28T11:24:12.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.449+0000 7f67027ff700 1 -- 172.21.15.92:0/1125730984 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66fc15d020 con 0x7f66fc15a110 2023-11-28T11:24:12.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.450+0000 7f66f8ff9700 1 -- 172.21.15.92:0/1125730984 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f66f0019030 con 0x7f66fc15a110 2023-11-28T11:24:12.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.451+0000 7f66f8ff9700 1 --2- 172.21.15.92:0/1125730984 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f66e80682f0 0x7f66e806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:12.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.451+0000 7f66f8ff9700 1 -- 172.21.15.92:0/1125730984 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f66f0084d10 con 0x7f66fc15a110 2023-11-28T11:24:12.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.451+0000 7f66fbfff700 1 --2- 172.21.15.92:0/1125730984 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f66e80682f0 0x7f66e806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:12.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.452+0000 7f66fbfff700 1 --2- 172.21.15.92:0/1125730984 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f66e80682f0 0x7f66e806a7a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f66fc155410 tx=0x7f66ec008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:12.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.455+0000 7f66f8ff9700 1 -- 172.21.15.92:0/1125730984 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f66f008a020 con 0x7f66fc15a110 2023-11-28T11:24:12.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.646+0000 7f67027ff700 1 -- 172.21.15.92:0/1125730984 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f66fc067880 con 0x7f66fc15a110 2023-11-28T11:24:12.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.647+0000 7f66f8ff9700 1 -- 172.21.15.92:0/1125730984 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v64) v1 ==== 93+0+10 (secure 0 0 0) 0x7f66f0051690 con 0x7f66fc15a110 2023-11-28T11:24:12.649 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-11-28T11:24:12.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.649+0000 7f67027ff700 1 -- 172.21.15.92:0/1125730984 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f66e80682f0 msgr2=0x7f66e806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:12.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.649+0000 7f67027ff700 1 --2- 172.21.15.92:0/1125730984 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f66e80682f0 0x7f66e806a7a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f66fc155410 tx=0x7f66ec008040 comp rx=0 tx=0).stop 2023-11-28T11:24:12.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.649+0000 7f67027ff700 1 -- 172.21.15.92:0/1125730984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66fc15a110 msgr2=0x7f66fc15a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:12.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.649+0000 7f67027ff700 1 --2- 172.21.15.92:0/1125730984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66fc15a110 0x7f66fc15a960 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f66f000c450 tx=0x7f66f000c810 comp rx=0 tx=0).stop 2023-11-28T11:24:12.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.649+0000 7f67027ff700 1 -- 172.21.15.92:0/1125730984 shutdown_connections 2023-11-28T11:24:12.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.649+0000 7f67027ff700 1 --2- 172.21.15.92:0/1125730984 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f66fc15d820 0x7f66fc15aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:12.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.649+0000 7f67027ff700 1 --2- 172.21.15.92:0/1125730984 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f66fc0cfec0 0x7f66fc150b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:12.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.649+0000 7f67027ff700 1 --2- 172.21.15.92:0/1125730984 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f66e80682f0 0x7f66e806a7a0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:12.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.649+0000 7f67027ff700 1 --2- 172.21.15.92:0/1125730984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66fc15a110 0x7f66fc15a960 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:12.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.649+0000 7f67027ff700 1 -- 172.21.15.92:0/1125730984 >> 172.21.15.92:0/1125730984 conn(0x7f66fc0719a0 msgr2=0x7f66fc159140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:12.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.649+0000 7f67027ff700 1 -- 172.21.15.92:0/1125730984 shutdown_connections 2023-11-28T11:24:12.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.649+0000 7f67027ff700 1 -- 172.21.15.92:0/1125730984 wait complete. 2023-11-28T11:24:12.669 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:24:12.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.866+0000 7f9c5b9ff700 1 Processor -- start 2023-11-28T11:24:12.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.866+0000 7f9c5b9ff700 1 -- start start 2023-11-28T11:24:12.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.867+0000 7f9c5b9ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9c540cd460 0x7f9c5406c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:12.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.868+0000 7f9c5b9ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c540d0d60 0x7f9c540d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:12.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.868+0000 7f9c5b9ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c540b88e0 con 0x7f9c540b8510 2023-11-28T11:24:12.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.868+0000 7f9c5b9ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c540cdbd0 con 0x7f9c540cd460 2023-11-28T11:24:12.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.868+0000 7f9c5b9ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c540d02d0 con 0x7f9c540d0d60 2023-11-28T11:24:12.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.868+0000 7f9c58f9a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9c540cd460 0x7f9c5406c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:12.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.868+0000 7f9c58f9a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9c540cd460 0x7f9c5406c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57592/0 (socket says 172.21.15.92:57592) 2023-11-28T11:24:12.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.868+0000 7f9c58f9a700 1 -- 172.21.15.92:0/445720681 learned_addr learned my addr 172.21.15.92:0/445720681 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:12.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.868+0000 7f9c59f9c700 1 --2- 172.21.15.92:0/445720681 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c540d0d60 0x7f9c540d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:12.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.868+0000 7f9c58f9a700 1 -- 172.21.15.92:0/445720681 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c540d0d60 msgr2=0x7f9c540d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:12.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.868+0000 7f9c58f9a700 1 --2- 172.21.15.92:0/445720681 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c540d0d60 0x7f9c540d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:12.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.868+0000 7f9c58f9a700 1 -- 172.21.15.92:0/445720681 >> v1:172.21.15.169:6789/0 conn(0x7f9c540b8510 legacy=0x7f9c5406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:12.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.868+0000 7f9c58f9a700 1 -- 172.21.15.92:0/445720681 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c540cd8f0 con 0x7f9c540cd460 2023-11-28T11:24:12.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.869+0000 7f9c58f9a700 1 --2- 172.21.15.92:0/445720681 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c540cd460 0x7f9c5406c080 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f9c44009920 tx=0x7f9c44009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17f6ab87271445e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:12.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.869+0000 7f9c4bfff700 1 -- 172.21.15.92:0/445720681 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c44011830 con 0x7f9c540cd460 2023-11-28T11:24:12.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.869+0000 7f9c4bfff700 1 -- 172.21.15.92:0/445720681 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c44011990 con 0x7f9c540cd460 2023-11-28T11:24:12.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.869+0000 7f9c4bfff700 1 -- 172.21.15.92:0/445720681 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c44011d20 con 0x7f9c540cd460 2023-11-28T11:24:12.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.869+0000 7f9c5b9ff700 1 -- 172.21.15.92:0/445720681 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c540cd460 msgr2=0x7f9c5406c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:12.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.869+0000 7f9c5b9ff700 1 --2- 172.21.15.92:0/445720681 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c540cd460 0x7f9c5406c080 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f9c44009920 tx=0x7f9c44009c30 comp rx=0 tx=0).stop 2023-11-28T11:24:12.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.870+0000 7f9c5b9ff700 1 -- 172.21.15.92:0/445720681 shutdown_connections 2023-11-28T11:24:12.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.870+0000 7f9c5b9ff700 1 --2- 172.21.15.92:0/445720681 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c540d0d60 0x7f9c540d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:12.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.870+0000 7f9c5b9ff700 1 --2- 172.21.15.92:0/445720681 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c540cd460 0x7f9c5406c080 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:12.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.870+0000 7f9c5b9ff700 1 -- 172.21.15.92:0/445720681 >> 172.21.15.92:0/445720681 conn(0x7f9c540719a0 msgr2=0x7f9c54071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:12.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.870+0000 7f9c5b9ff700 1 -- 172.21.15.92:0/445720681 shutdown_connections 2023-11-28T11:24:12.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.870+0000 7f9c5b9ff700 1 -- 172.21.15.92:0/445720681 wait complete. 2023-11-28T11:24:12.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.871+0000 7f9c5b9ff700 1 Processor -- start 2023-11-28T11:24:12.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.871+0000 7f9c5b9ff700 1 -- start start 2023-11-28T11:24:12.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.872+0000 7f9c5b9ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c540b8510 0x7f9c5414a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:12.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.872+0000 7f9c5b9ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c540cd460 0x7f9c540c9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:12.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.872+0000 7f9c5b9ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c540d0d60 0x7f9c540c9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:12.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.872+0000 7f9c5b9ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5415b8a0 con 0x7f9c540cd460 2023-11-28T11:24:12.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.872+0000 7f9c5b9ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c54149770 con 0x7f9c540d0d60 2023-11-28T11:24:12.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.872+0000 7f9c5b9ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c54148b60 con 0x7f9c540b8510 2023-11-28T11:24:12.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.873+0000 7f9c58f9a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c540cd460 0x7f9c540c9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:12.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.873+0000 7f9c58f9a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c540cd460 0x7f9c540c9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57594/0 (socket says 172.21.15.92:57594) 2023-11-28T11:24:12.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.873+0000 7f9c58f9a700 1 -- 172.21.15.92:0/568619278 learned_addr learned my addr 172.21.15.92:0/568619278 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:12.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.873+0000 7f9c5979b700 1 --2- 172.21.15.92:0/568619278 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c540b8510 0x7f9c5414a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:12.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.873+0000 7f9c59f9c700 1 --2- 172.21.15.92:0/568619278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c540d0d60 0x7f9c540c9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:12.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.873+0000 7f9c58f9a700 1 -- 172.21.15.92:0/568619278 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c540b8510 msgr2=0x7f9c5414a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:12.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.873+0000 7f9c58f9a700 1 --2- 172.21.15.92:0/568619278 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c540b8510 0x7f9c5414a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:12.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.873+0000 7f9c58f9a700 1 -- 172.21.15.92:0/568619278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c540d0d60 msgr2=0x7f9c540c9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:12.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.873+0000 7f9c58f9a700 1 --2- 172.21.15.92:0/568619278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c540d0d60 0x7f9c540c9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:12.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.873+0000 7f9c58f9a700 1 -- 172.21.15.92:0/568619278 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c44009550 con 0x7f9c540cd460 2023-11-28T11:24:12.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.873+0000 7f9c58f9a700 1 --2- 172.21.15.92:0/568619278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c540cd460 0x7f9c540c9350 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f9c44000c00 tx=0x7f9c44004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:12.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.874+0000 7f9c49ffb700 1 -- 172.21.15.92:0/568619278 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c44004300 con 0x7f9c540cd460 2023-11-28T11:24:12.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.874+0000 7f9c49ffb700 1 -- 172.21.15.92:0/568619278 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c4401f070 con 0x7f9c540cd460 2023-11-28T11:24:12.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.874+0000 7f9c5b9ff700 1 -- 172.21.15.92:0/568619278 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c540cfff0 con 0x7f9c540cd460 2023-11-28T11:24:12.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.874+0000 7f9c49ffb700 1 -- 172.21.15.92:0/568619278 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c440036a0 con 0x7f9c540cd460 2023-11-28T11:24:12.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.874+0000 7f9c5b9ff700 1 -- 172.21.15.92:0/568619278 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c540cfd10 con 0x7f9c540cd460 2023-11-28T11:24:12.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.875+0000 7f9c5b9ff700 1 -- 172.21.15.92:0/568619278 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c54149900 con 0x7f9c540cd460 2023-11-28T11:24:12.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.876+0000 7f9c49ffb700 1 -- 172.21.15.92:0/568619278 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9c4402a020 con 0x7f9c540cd460 2023-11-28T11:24:12.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.876+0000 7f9c49ffb700 1 --2- 172.21.15.92:0/568619278 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9c380682f0 0x7f9c3806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:12.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.877+0000 7f9c49ffb700 1 -- 172.21.15.92:0/568619278 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f9c44086ee0 con 0x7f9c540cd460 2023-11-28T11:24:12.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.877+0000 7f9c5979b700 1 --2- 172.21.15.92:0/568619278 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9c380682f0 0x7f9c3806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:12.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.877+0000 7f9c5979b700 1 --2- 172.21.15.92:0/568619278 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9c380682f0 0x7f9c3806a7a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9c50007fd0 tx=0x7f9c50009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:12.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:12.881+0000 7f9c49ffb700 1 -- 172.21.15.92:0/568619278 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9c44053860 con 0x7f9c540cd460 2023-11-28T11:24:13.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.072+0000 7f9c5b9ff700 1 -- 172.21.15.92:0/568619278 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9c54067880 con 0x7f9c540cd460 2023-11-28T11:24:13.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.073+0000 7f9c49ffb700 1 -- 172.21.15.92:0/568619278 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v64) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9c440150a0 con 0x7f9c540cd460 2023-11-28T11:24:13.075 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-11-28T11:24:13.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.075+0000 7f9c5b9ff700 1 -- 172.21.15.92:0/568619278 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9c380682f0 msgr2=0x7f9c3806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:13.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.075+0000 7f9c5b9ff700 1 --2- 172.21.15.92:0/568619278 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9c380682f0 0x7f9c3806a7a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9c50007fd0 tx=0x7f9c50009040 comp rx=0 tx=0).stop 2023-11-28T11:24:13.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.075+0000 7f9c5b9ff700 1 -- 172.21.15.92:0/568619278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c540cd460 msgr2=0x7f9c540c9350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:13.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.075+0000 7f9c5b9ff700 1 --2- 172.21.15.92:0/568619278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c540cd460 0x7f9c540c9350 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f9c44000c00 tx=0x7f9c44004750 comp rx=0 tx=0).stop 2023-11-28T11:24:13.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.075+0000 7f9c5b9ff700 1 -- 172.21.15.92:0/568619278 shutdown_connections 2023-11-28T11:24:13.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.075+0000 7f9c5b9ff700 1 --2- 172.21.15.92:0/568619278 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9c540d0d60 0x7f9c540c9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:13.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.075+0000 7f9c5b9ff700 1 --2- 172.21.15.92:0/568619278 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9c540b8510 0x7f9c5414a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:13.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.075+0000 7f9c5b9ff700 1 --2- 172.21.15.92:0/568619278 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9c380682f0 0x7f9c3806a7a0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:13.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.075+0000 7f9c5b9ff700 1 --2- 172.21.15.92:0/568619278 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c540cd460 0x7f9c540c9350 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:13.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.075+0000 7f9c5b9ff700 1 -- 172.21.15.92:0/568619278 >> 172.21.15.92:0/568619278 conn(0x7f9c540719a0 msgr2=0x7f9c540b5dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:13.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.075+0000 7f9c5b9ff700 1 -- 172.21.15.92:0/568619278 shutdown_connections 2023-11-28T11:24:13.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.075+0000 7f9c5b9ff700 1 -- 172.21.15.92:0/568619278 wait complete. 2023-11-28T11:24:13.096 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T11:24:13.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.290+0000 7f1488dff700 1 Processor -- start 2023-11-28T11:24:13.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.290+0000 7f1488dff700 1 -- start start 2023-11-28T11:24:13.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.291+0000 7f1488dff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f14840cd460 0x7f148406c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:13.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.291+0000 7f1488dff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14840d0d60 0x7f14840d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:13.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.291+0000 7f1488dff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14840b88e0 con 0x7f14840b8510 2023-11-28T11:24:13.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.291+0000 7f1488dff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14840cdbd0 con 0x7f14840cd460 2023-11-28T11:24:13.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.291+0000 7f1488dff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14840d02d0 con 0x7f14840d0d60 2023-11-28T11:24:13.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.291+0000 7f148259c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f14840b8510 0x7f148406b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47474/0 (socket says 172.21.15.92:47474) 2023-11-28T11:24:13.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.291+0000 7f148259c700 1 -- 172.21.15.92:0/1732612131 learned_addr learned my addr 172.21.15.92:0/1732612131 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:24:13.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.292+0000 7f1481d9b700 1 --2- 172.21.15.92:0/1732612131 >> v2:172.21.15.169:3300/0 conn(0x7f14840cd460 0x7f148406c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:13.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.292+0000 7f1482d9d700 1 --2- 172.21.15.92:0/1732612131 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14840d0d60 0x7f14840d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:13.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.292+0000 7f148159a700 1 -- 172.21.15.92:0/1732612131 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2163746323 0 0) 0x7f14840b88e0 con 0x7f14840b8510 2023-11-28T11:24:13.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.292+0000 7f148159a700 1 -- 172.21.15.92:0/1732612131 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f146c0035b0 con 0x7f14840b8510 2023-11-28T11:24:13.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.292+0000 7f1481d9b700 1 -- 172.21.15.92:0/1732612131 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14840d0d60 msgr2=0x7f14840d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:13.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.292+0000 7f1481d9b700 1 --2- 172.21.15.92:0/1732612131 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14840d0d60 0x7f14840d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:13.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.292+0000 7f1481d9b700 1 -- 172.21.15.92:0/1732612131 >> v1:172.21.15.92:6789/0 conn(0x7f14840b8510 legacy=0x7f148406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:13.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.292+0000 7f1481d9b700 1 -- 172.21.15.92:0/1732612131 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14840cd8f0 con 0x7f14840cd460 2023-11-28T11:24:13.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.292+0000 7f1481d9b700 1 --2- 172.21.15.92:0/1732612131 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14840cd460 0x7f148406c080 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1478009870 tx=0x7f1478009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7af144c70027157 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:13.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.293+0000 7f148159a700 1 -- 172.21.15.92:0/1732612131 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1478011830 con 0x7f14840cd460 2023-11-28T11:24:13.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.293+0000 7f148159a700 1 -- 172.21.15.92:0/1732612131 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1478011990 con 0x7f14840cd460 2023-11-28T11:24:13.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.293+0000 7f148159a700 1 -- 172.21.15.92:0/1732612131 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1478011d00 con 0x7f14840cd460 2023-11-28T11:24:13.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.293+0000 7f1488dff700 1 -- 172.21.15.92:0/1732612131 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14840cd460 msgr2=0x7f148406c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:13.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.293+0000 7f1488dff700 1 --2- 172.21.15.92:0/1732612131 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14840cd460 0x7f148406c080 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1478009870 tx=0x7f1478009b80 comp rx=0 tx=0).stop 2023-11-28T11:24:13.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.293+0000 7f1488dff700 1 -- 172.21.15.92:0/1732612131 shutdown_connections 2023-11-28T11:24:13.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.293+0000 7f1488dff700 1 --2- 172.21.15.92:0/1732612131 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14840d0d60 0x7f14840d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:13.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.293+0000 7f1488dff700 1 --2- 172.21.15.92:0/1732612131 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14840cd460 0x7f148406c080 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:13.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.293+0000 7f1488dff700 1 -- 172.21.15.92:0/1732612131 >> 172.21.15.92:0/1732612131 conn(0x7f14840719a0 msgr2=0x7f1484071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:13.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.293+0000 7f1488dff700 1 -- 172.21.15.92:0/1732612131 shutdown_connections 2023-11-28T11:24:13.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.293+0000 7f1488dff700 1 -- 172.21.15.92:0/1732612131 wait complete. 2023-11-28T11:24:13.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.295+0000 7f1488dff700 1 Processor -- start 2023-11-28T11:24:13.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.295+0000 7f1488dff700 1 -- start start 2023-11-28T11:24:13.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.296+0000 7f1488dff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14840b8510 0x7f14841e22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:13.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.296+0000 7f1488dff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14840cd460 0x7f14841f1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:13.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.296+0000 7f1488dff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14840d0d60 0x7f14840cc710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:13.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.296+0000 7f1488dff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14840aa3d0 con 0x7f14840cd460 2023-11-28T11:24:13.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.296+0000 7f1488dff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14840a9ab0 con 0x7f14840d0d60 2023-11-28T11:24:13.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.296+0000 7f1488dff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14841e2800 con 0x7f14840b8510 2023-11-28T11:24:13.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.297+0000 7f1481d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14840cd460 0x7f14841f1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:13.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.297+0000 7f1481d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14840cd460 0x7f14841f1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57596/0 (socket says 172.21.15.92:57596) 2023-11-28T11:24:13.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.297+0000 7f1481d9b700 1 -- 172.21.15.92:0/2087499017 learned_addr learned my addr 172.21.15.92:0/2087499017 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:13.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.297+0000 7f148259c700 1 --2- 172.21.15.92:0/2087499017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14840b8510 0x7f14841e22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:13.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.297+0000 7f1482d9d700 1 --2- 172.21.15.92:0/2087499017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14840d0d60 0x7f14840cc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:13.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.297+0000 7f1481d9b700 1 -- 172.21.15.92:0/2087499017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14840b8510 msgr2=0x7f14841e22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:13.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.297+0000 7f1481d9b700 1 --2- 172.21.15.92:0/2087499017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14840b8510 0x7f14841e22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:13.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.297+0000 7f1481d9b700 1 -- 172.21.15.92:0/2087499017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14840d0d60 msgr2=0x7f14840cc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:13.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.297+0000 7f1481d9b700 1 --2- 172.21.15.92:0/2087499017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14840d0d60 0x7f14840cc710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:13.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.297+0000 7f1481d9b700 1 -- 172.21.15.92:0/2087499017 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14780094a0 con 0x7f14840cd460 2023-11-28T11:24:13.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.297+0000 7f1481d9b700 1 --2- 172.21.15.92:0/2087499017 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14840cd460 0x7f14841f1750 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f14780090b0 tx=0x7f1478004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:13.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.297+0000 7f1472ffd700 1 -- 172.21.15.92:0/2087499017 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1478004450 con 0x7f14840cd460 2023-11-28T11:24:13.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.298+0000 7f1472ffd700 1 -- 172.21.15.92:0/2087499017 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f147801f070 con 0x7f14840cd460 2023-11-28T11:24:13.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.298+0000 7f1488dff700 1 -- 172.21.15.92:0/2087499017 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14840cfff0 con 0x7f14840cd460 2023-11-28T11:24:13.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.298+0000 7f1472ffd700 1 -- 172.21.15.92:0/2087499017 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14780038c0 con 0x7f14840cd460 2023-11-28T11:24:13.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.298+0000 7f1488dff700 1 -- 172.21.15.92:0/2087499017 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14840cfd10 con 0x7f14840cd460 2023-11-28T11:24:13.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.299+0000 7f1488dff700 1 -- 172.21.15.92:0/2087499017 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1484067880 con 0x7f14840cd460 2023-11-28T11:24:13.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.300+0000 7f1472ffd700 1 -- 172.21.15.92:0/2087499017 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f147800f050 con 0x7f14840cd460 2023-11-28T11:24:13.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.300+0000 7f1472ffd700 1 --2- 172.21.15.92:0/2087499017 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f14640682f0 0x7f146406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:13.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.301+0000 7f1472ffd700 1 -- 172.21.15.92:0/2087499017 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f1478021070 con 0x7f14840cd460 2023-11-28T11:24:13.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.301+0000 7f148259c700 1 --2- 172.21.15.92:0/2087499017 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f14640682f0 0x7f146406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:13.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.301+0000 7f148259c700 1 --2- 172.21.15.92:0/2087499017 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f14640682f0 0x7f146406a7a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1474007fd0 tx=0x7f1474009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:13.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.305+0000 7f1472ffd700 1 -- 172.21.15.92:0/2087499017 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1478053c30 con 0x7f14840cd460 2023-11-28T11:24:13.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.498+0000 7f1488dff700 1 -- 172.21.15.92:0/2087499017 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f14840b72d0 con 0x7f14840cd460 2023-11-28T11:24:13.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.498+0000 7f1472ffd700 1 -- 172.21.15.92:0/2087499017 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v64) v1 ==== 84+0+126 (secure 0 0 0) 0x7f1478015090 con 0x7f14840cd460 2023-11-28T11:24:13.501 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:24:13.501 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-11-28T11:24:13.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:24:13.501 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 1, 2023-11-28T11:24:13.501 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": ".mgr" 2023-11-28T11:24:13.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:24:13.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:24:13.502 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 2, 2023-11-28T11:24:13.502 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": "rbd" 2023-11-28T11:24:13.502 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:24:13.502 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-11-28T11:24:13.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.500+0000 7f1488dff700 1 -- 172.21.15.92:0/2087499017 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f14640682f0 msgr2=0x7f146406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:13.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.500+0000 7f1488dff700 1 --2- 172.21.15.92:0/2087499017 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f14640682f0 0x7f146406a7a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1474007fd0 tx=0x7f1474009040 comp rx=0 tx=0).stop 2023-11-28T11:24:13.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.500+0000 7f1488dff700 1 -- 172.21.15.92:0/2087499017 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14840cd460 msgr2=0x7f14841f1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:13.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.500+0000 7f1488dff700 1 --2- 172.21.15.92:0/2087499017 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14840cd460 0x7f14841f1750 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f14780090b0 tx=0x7f1478004750 comp rx=0 tx=0).stop 2023-11-28T11:24:13.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.501+0000 7f1488dff700 1 -- 172.21.15.92:0/2087499017 shutdown_connections 2023-11-28T11:24:13.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.501+0000 7f1488dff700 1 --2- 172.21.15.92:0/2087499017 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f14840d0d60 0x7f14840cc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:13.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.501+0000 7f1488dff700 1 --2- 172.21.15.92:0/2087499017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f14840b8510 0x7f14841e22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:13.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.501+0000 7f1488dff700 1 --2- 172.21.15.92:0/2087499017 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f14640682f0 0x7f146406a7a0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:13.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.501+0000 7f1488dff700 1 --2- 172.21.15.92:0/2087499017 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f14840cd460 0x7f14841f1750 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:13.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.501+0000 7f1488dff700 1 -- 172.21.15.92:0/2087499017 >> 172.21.15.92:0/2087499017 conn(0x7f14840719a0 msgr2=0x7f14840b5d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:13.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.501+0000 7f1488dff700 1 -- 172.21.15.92:0/2087499017 shutdown_connections 2023-11-28T11:24:13.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.501+0000 7f1488dff700 1 -- 172.21.15.92:0/2087499017 wait complete. 2023-11-28T11:24:13.520 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T11:24:13.521 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-28T11:24:13.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.719+0000 7f5f611ff700 1 Processor -- start 2023-11-28T11:24:13.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.719+0000 7f5f611ff700 1 -- start start 2023-11-28T11:24:13.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.721+0000 7f5f611ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5f5c0b8790 0x7f5f5c0cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:13.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.721+0000 7f5f611ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f5c06c050 0x7f5f5c0d0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:13.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.721+0000 7f5f611ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f5c0d00b0 con 0x7f5f5c0cd460 2023-11-28T11:24:13.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.721+0000 7f5f611ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f5c0b5240 con 0x7f5f5c0b8790 2023-11-28T11:24:13.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.721+0000 7f5f611ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5c0b5520 con 0x7f5f5c06c050 2023-11-28T11:24:13.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.721+0000 7f5f53fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f5f5c0cd460 0x7f5f5c06b940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47482/0 (socket says 172.21.15.92:47482) 2023-11-28T11:24:13.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.721+0000 7f5f53fff700 1 -- 172.21.15.92:0/2759160582 learned_addr learned my addr 172.21.15.92:0/2759160582 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:24:13.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.721+0000 7f5f5ad9d700 1 --2- 172.21.15.92:0/2759160582 >> v2:172.21.15.92:3300/0 conn(0x7f5f5c0b8790 0x7f5f5c0cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:13.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.721+0000 7f5f5b59e700 1 --2- 172.21.15.92:0/2759160582 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f5c06c050 0x7f5f5c0d0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:13.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.721+0000 7f5f5a59c700 1 -- 172.21.15.92:0/2759160582 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1385511420 0 0) 0x7f5f5c0d00b0 con 0x7f5f5c0cd460 2023-11-28T11:24:13.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.721+0000 7f5f5ad9d700 1 -- 172.21.15.92:0/2759160582 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f5c06c050 msgr2=0x7f5f5c0d0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:13.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.721+0000 7f5f5ad9d700 1 --2- 172.21.15.92:0/2759160582 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f5c06c050 0x7f5f5c0d0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:13.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.721+0000 7f5f5ad9d700 1 -- 172.21.15.92:0/2759160582 >> v1:172.21.15.92:6789/0 conn(0x7f5f5c0cd460 legacy=0x7f5f5c06b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:13.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.721+0000 7f5f5ad9d700 1 -- 172.21.15.92:0/2759160582 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f5c0d0a80 con 0x7f5f5c0b8790 2023-11-28T11:24:13.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.722+0000 7f5f5ad9d700 1 --2- 172.21.15.92:0/2759160582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f5c0b8790 0x7f5f5c0cfb70 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c009870 tx=0x7f5f4c009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e63042733c133ce server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:13.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.722+0000 7f5f5a59c700 1 -- 172.21.15.92:0/2759160582 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c011830 con 0x7f5f5c0b8790 2023-11-28T11:24:13.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.722+0000 7f5f5a59c700 1 -- 172.21.15.92:0/2759160582 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f4c011990 con 0x7f5f5c0b8790 2023-11-28T11:24:13.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.722+0000 7f5f5a59c700 1 -- 172.21.15.92:0/2759160582 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c011d20 con 0x7f5f5c0b8790 2023-11-28T11:24:13.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.722+0000 7f5f611ff700 1 -- 172.21.15.92:0/2759160582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f5c0b8790 msgr2=0x7f5f5c0cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:13.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.722+0000 7f5f611ff700 1 --2- 172.21.15.92:0/2759160582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f5c0b8790 0x7f5f5c0cfb70 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c009870 tx=0x7f5f4c009b80 comp rx=0 tx=0).stop 2023-11-28T11:24:13.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.722+0000 7f5f611ff700 1 -- 172.21.15.92:0/2759160582 shutdown_connections 2023-11-28T11:24:13.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.722+0000 7f5f611ff700 1 --2- 172.21.15.92:0/2759160582 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f5c06c050 0x7f5f5c0d0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:13.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.722+0000 7f5f611ff700 1 --2- 172.21.15.92:0/2759160582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f5c0b8790 0x7f5f5c0cfb70 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:13.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.723+0000 7f5f611ff700 1 -- 172.21.15.92:0/2759160582 >> 172.21.15.92:0/2759160582 conn(0x7f5f5c0719a0 msgr2=0x7f5f5c071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:13.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.723+0000 7f5f611ff700 1 -- 172.21.15.92:0/2759160582 shutdown_connections 2023-11-28T11:24:13.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.723+0000 7f5f611ff700 1 -- 172.21.15.92:0/2759160582 wait complete. 2023-11-28T11:24:13.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.725+0000 7f5f611ff700 1 Processor -- start 2023-11-28T11:24:13.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.725+0000 7f5f611ff700 1 -- start start 2023-11-28T11:24:13.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.726+0000 7f5f611ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f5c06c050 0x7f5f5c1f2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:13.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.726+0000 7f5f611ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f5c0b8790 0x7f5f5c1f31f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:13.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.726+0000 7f5f611ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f5c0cd460 0x7f5f5c1f17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:13.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.726+0000 7f5f611ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5c1f2ff0 con 0x7f5f5c0cd460 2023-11-28T11:24:13.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.726+0000 7f5f611ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5c155410 con 0x7f5f5c0b8790 2023-11-28T11:24:13.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.726+0000 7f5f611ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5c1556f0 con 0x7f5f5c06c050 2023-11-28T11:24:13.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.727+0000 7f5f5b59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f5c0cd460 0x7f5f5c1f17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:13.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.727+0000 7f5f5b59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f5c0cd460 0x7f5f5c1f17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57614/0 (socket says 172.21.15.92:57614) 2023-11-28T11:24:13.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.727+0000 7f5f5b59e700 1 -- 172.21.15.92:0/2755499482 learned_addr learned my addr 172.21.15.92:0/2755499482 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:13.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.727+0000 7f5f5ad9d700 1 --2- 172.21.15.92:0/2755499482 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f5c06c050 0x7f5f5c1f2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:13.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.727+0000 7f5f53fff700 1 --2- 172.21.15.92:0/2755499482 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f5c0b8790 0x7f5f5c1f31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:13.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.727+0000 7f5f5b59e700 1 -- 172.21.15.92:0/2755499482 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f5c06c050 msgr2=0x7f5f5c1f2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:13.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.727+0000 7f5f5b59e700 1 --2- 172.21.15.92:0/2755499482 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f5c06c050 0x7f5f5c1f2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:13.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.727+0000 7f5f5b59e700 1 -- 172.21.15.92:0/2755499482 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f5c0b8790 msgr2=0x7f5f5c1f31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:13.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.727+0000 7f5f5b59e700 1 --2- 172.21.15.92:0/2755499482 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f5c0b8790 0x7f5f5c1f31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:13.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.727+0000 7f5f5b59e700 1 -- 172.21.15.92:0/2755499482 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f4c0094a0 con 0x7f5f5c0cd460 2023-11-28T11:24:13.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.727+0000 7f5f53fff700 1 --2- 172.21.15.92:0/2755499482 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f5c0b8790 0x7f5f5c1f31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:24:13.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.727+0000 7f5f5b59e700 1 --2- 172.21.15.92:0/2755499482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f5c0cd460 0x7f5f5c1f17f0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f5f54007de0 tx=0x7f5f5400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:13.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.728+0000 7f5f537fe700 1 -- 172.21.15.92:0/2755499482 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f5400ede0 con 0x7f5f5c0cd460 2023-11-28T11:24:13.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.728+0000 7f5f537fe700 1 -- 172.21.15.92:0/2755499482 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f54014420 con 0x7f5f5c0cd460 2023-11-28T11:24:13.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.728+0000 7f5f537fe700 1 -- 172.21.15.92:0/2755499482 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f540147d0 con 0x7f5f5c0cd460 2023-11-28T11:24:13.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.728+0000 7f5f611ff700 1 -- 172.21.15.92:0/2755499482 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f5c0d14e0 con 0x7f5f5c0cd460 2023-11-28T11:24:13.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.728+0000 7f5f611ff700 1 -- 172.21.15.92:0/2755499482 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f5c0b5800 con 0x7f5f5c0cd460 2023-11-28T11:24:13.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.729+0000 7f5f611ff700 1 -- 172.21.15.92:0/2755499482 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f5c0d1670 con 0x7f5f5c0cd460 2023-11-28T11:24:13.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.730+0000 7f5f537fe700 1 -- 172.21.15.92:0/2755499482 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5f54012050 con 0x7f5f5c0cd460 2023-11-28T11:24:13.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.731+0000 7f5f537fe700 1 --2- 172.21.15.92:0/2755499482 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5f3c0682f0 0x7f5f3c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:13.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.731+0000 7f5f537fe700 1 -- 172.21.15.92:0/2755499482 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f5f54086170 con 0x7f5f5c0cd460 2023-11-28T11:24:13.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.731+0000 7f5f5ad9d700 1 --2- 172.21.15.92:0/2755499482 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5f3c0682f0 0x7f5f3c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:13.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.732+0000 7f5f5ad9d700 1 --2- 172.21.15.92:0/2755499482 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5f3c0682f0 0x7f5f3c06a7a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c000c00 tx=0x7f5f4c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:13.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.735+0000 7f5f537fe700 1 -- 172.21.15.92:0/2755499482 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5f54052b80 con 0x7f5f5c0cd460 2023-11-28T11:24:13.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:13.931+0000 7f5f611ff700 1 -- 172.21.15.92:0/2755499482 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f5f5c067880 con 0x7f5f5c0cd460 2023-11-28T11:24:14.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.759+0000 7f5f537fe700 1 -- 172.21.15.92:0/2755499482 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v65) v1 ==== 138+0+0 (secure 0 0 0) 0x7f5f540561a0 con 0x7f5f5c0cd460 2023-11-28T11:24:14.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.763+0000 7f5f611ff700 1 -- 172.21.15.92:0/2755499482 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5f3c0682f0 msgr2=0x7f5f3c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:14.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.763+0000 7f5f611ff700 1 --2- 172.21.15.92:0/2755499482 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5f3c0682f0 0x7f5f3c06a7a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c000c00 tx=0x7f5f4c008030 comp rx=0 tx=0).stop 2023-11-28T11:24:14.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.763+0000 7f5f611ff700 1 -- 172.21.15.92:0/2755499482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f5c0cd460 msgr2=0x7f5f5c1f17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:14.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.763+0000 7f5f611ff700 1 --2- 172.21.15.92:0/2755499482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f5c0cd460 0x7f5f5c1f17f0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f5f54007de0 tx=0x7f5f5400e3c0 comp rx=0 tx=0).stop 2023-11-28T11:24:14.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.764+0000 7f5f611ff700 1 -- 172.21.15.92:0/2755499482 shutdown_connections 2023-11-28T11:24:14.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.764+0000 7f5f611ff700 1 --2- 172.21.15.92:0/2755499482 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5f3c0682f0 0x7f5f3c06a7a0 secure :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c000c00 tx=0x7f5f4c008030 comp rx=0 tx=0).stop 2023-11-28T11:24:14.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.764+0000 7f5f611ff700 1 --2- 172.21.15.92:0/2755499482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f5c0cd460 0x7f5f5c1f17f0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:14.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.764+0000 7f5f611ff700 1 --2- 172.21.15.92:0/2755499482 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5f5c06c050 0x7f5f5c1f2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:14.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.764+0000 7f5f611ff700 1 --2- 172.21.15.92:0/2755499482 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f5c0b8790 0x7f5f5c1f31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:14.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.764+0000 7f5f611ff700 1 -- 172.21.15.92:0/2755499482 >> 172.21.15.92:0/2755499482 conn(0x7f5f5c0719a0 msgr2=0x7f5f5c06b940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:14.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.764+0000 7f5f611ff700 1 -- 172.21.15.92:0/2755499482 shutdown_connections 2023-11-28T11:24:14.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.765+0000 7f5f611ff700 1 -- 172.21.15.92:0/2755499482 wait complete. 2023-11-28T11:24:14.768 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' created 2023-11-28T11:24:14.787 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T11:24:14.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.990+0000 7f9d44dff700 1 Processor -- start 2023-11-28T11:24:14.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.990+0000 7f9d44dff700 1 -- start start 2023-11-28T11:24:14.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.991+0000 7f9d44dff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f9d400b2280 0x7f9d400cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:14.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.991+0000 7f9d44dff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d400cd4d0 con 0x7f9d400d2e10 2023-11-28T11:24:14.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.992+0000 7f9d44dff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d400cf470 con 0x7f9d4015d590 2023-11-28T11:24:14.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.992+0000 7f9d44dff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d400cdce0 con 0x7f9d400b2280 2023-11-28T11:24:14.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.992+0000 7f9d3e59c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f9d400d2e10 0x7f9d400d31e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47488/0 (socket says 172.21.15.92:47488) 2023-11-28T11:24:14.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.992+0000 7f9d3e59c700 1 -- 172.21.15.92:0/2504352918 learned_addr learned my addr 172.21.15.92:0/2504352918 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:24:14.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.992+0000 7f9d3ed9d700 1 --2- 172.21.15.92:0/2504352918 >> v2:172.21.15.169:3300/0 conn(0x7f9d400b2280 0x7f9d400cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:14.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.992+0000 7f9d3dd9b700 1 -- 172.21.15.92:0/2504352918 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3666650768 0 0) 0x7f9d400cd4d0 con 0x7f9d400d2e10 2023-11-28T11:24:14.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.992+0000 7f9d3dd9b700 1 -- 172.21.15.92:0/2504352918 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d2c0035b0 con 0x7f9d400d2e10 2023-11-28T11:24:14.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.993+0000 7f9d3dd9b700 1 -- 172.21.15.92:0/2504352918 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 400225236 0 0) 0x7f9d2c0035b0 con 0x7f9d400d2e10 2023-11-28T11:24:14.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.993+0000 7f9d3dd9b700 1 -- 172.21.15.92:0/2504352918 >> v2:172.21.15.169:3300/0 conn(0x7f9d400b2280 msgr2=0x7f9d400cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:14.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.993+0000 7f9d3dd9b700 1 --2- 172.21.15.92:0/2504352918 >> v2:172.21.15.169:3300/0 conn(0x7f9d400b2280 0x7f9d400cf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:14.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.993+0000 7f9d3dd9b700 1 -- 172.21.15.92:0/2504352918 >> v1:172.21.15.169:6789/0 conn(0x7f9d4015d590 legacy=0x7f9d4015dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:14.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.993+0000 7f9d3dd9b700 1 -- 172.21.15.92:0/2504352918 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d40156fa0 con 0x7f9d400d2e10 2023-11-28T11:24:14.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.993+0000 7f9d3dd9b700 1 -- 172.21.15.92:0/2504352918 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532161587 0 0) 0x7f9d30001670 con 0x7f9d400d2e10 2023-11-28T11:24:14.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.993+0000 7f9d3dd9b700 1 -- 172.21.15.92:0/2504352918 >> v1:172.21.15.92:6789/0 conn(0x7f9d400d2e10 legacy=0x7f9d400d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:14.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.993+0000 7f9d3dd9b700 1 --2- 172.21.15.92:0/2504352918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d2c003a00 0x7f9d2c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:14.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.993+0000 7f9d3dd9b700 1 --2- 172.21.15.92:0/2504352918 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d2c004850 0x7f9d2c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:14.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.993+0000 7f9d3dd9b700 1 --2- 172.21.15.92:0/2504352918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d2c005200 0x7f9d2c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:14.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.993+0000 7f9d3dd9b700 1 -- 172.21.15.92:0/2504352918 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d400cdce0 con 0x7f9d2c003a00 2023-11-28T11:24:14.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.993+0000 7f9d3dd9b700 1 -- 172.21.15.92:0/2504352918 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2c00c110 con 0x7f9d2c005200 2023-11-28T11:24:14.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.993+0000 7f9d3dd9b700 1 -- 172.21.15.92:0/2504352918 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2c00c2d0 con 0x7f9d2c004850 2023-11-28T11:24:14.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.993+0000 7f9d3e59c700 1 --2- 172.21.15.92:0/2504352918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d2c003a00 0x7f9d2c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:14.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.993+0000 7f9d3ed9d700 1 --2- 172.21.15.92:0/2504352918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d2c005200 0x7f9d2c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:14.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.993+0000 7f9d37fff700 1 --2- 172.21.15.92:0/2504352918 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d2c004850 0x7f9d2c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:14.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.993+0000 7f9d3e59c700 1 -- 172.21.15.92:0/2504352918 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d2c004850 msgr2=0x7f9d2c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:14.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.993+0000 7f9d3e59c700 1 --2- 172.21.15.92:0/2504352918 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d2c004850 0x7f9d2c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:14.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.993+0000 7f9d3e59c700 1 -- 172.21.15.92:0/2504352918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d2c005200 msgr2=0x7f9d2c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:14.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.993+0000 7f9d3e59c700 1 --2- 172.21.15.92:0/2504352918 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d2c005200 0x7f9d2c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:14.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.993+0000 7f9d3e59c700 1 -- 172.21.15.92:0/2504352918 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d2c0035b0 con 0x7f9d2c003a00 2023-11-28T11:24:14.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.993+0000 7f9d3ed9d700 1 -- 172.21.15.92:0/2504352918 reap_dead start 2023-11-28T11:24:14.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.993+0000 7f9d37fff700 1 --2- 172.21.15.92:0/2504352918 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d2c004850 0x7f9d2c004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:24:14.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.994+0000 7f9d3e59c700 1 --2- 172.21.15.92:0/2504352918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d2c003a00 0x7f9d2c003e10 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f9d3000c240 tx=0x7f9d2c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=766d865cc76c2f9b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:14.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.994+0000 7f9d3dd9b700 1 -- 172.21.15.92:0/2504352918 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d30007c10 con 0x7f9d2c003a00 2023-11-28T11:24:14.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.994+0000 7f9d3dd9b700 1 -- 172.21.15.92:0/2504352918 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d3000ce00 con 0x7f9d2c003a00 2023-11-28T11:24:14.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.994+0000 7f9d3dd9b700 1 -- 172.21.15.92:0/2504352918 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d30005ba0 con 0x7f9d2c003a00 2023-11-28T11:24:14.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.994+0000 7f9d44dff700 1 -- 172.21.15.92:0/2504352918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d2c003a00 msgr2=0x7f9d2c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:14.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.994+0000 7f9d44dff700 1 --2- 172.21.15.92:0/2504352918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d2c003a00 0x7f9d2c003e10 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f9d3000c240 tx=0x7f9d2c005880 comp rx=0 tx=0).stop 2023-11-28T11:24:15.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.995+0000 7f9d44dff700 1 -- 172.21.15.92:0/2504352918 shutdown_connections 2023-11-28T11:24:15.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.995+0000 7f9d44dff700 1 --2- 172.21.15.92:0/2504352918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d2c003a00 0x7f9d2c003e10 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:15.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.995+0000 7f9d44dff700 1 -- 172.21.15.92:0/2504352918 >> 172.21.15.92:0/2504352918 conn(0x7f9d400719a0 msgr2=0x7f9d40071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:15.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.995+0000 7f9d44dff700 1 -- 172.21.15.92:0/2504352918 shutdown_connections 2023-11-28T11:24:15.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.995+0000 7f9d44dff700 1 -- 172.21.15.92:0/2504352918 wait complete. 2023-11-28T11:24:15.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.996+0000 7f9d44dff700 1 Processor -- start 2023-11-28T11:24:15.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.996+0000 7f9d44dff700 1 -- start start 2023-11-28T11:24:15.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.997+0000 7f9d44dff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d2c003a00 0x7f9d400c7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:15.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.997+0000 7f9d44dff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d400b76e0 0x7f9d400d0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:15.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.997+0000 7f9d44dff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d40151c50 0x7f9d401520c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:15.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.997+0000 7f9d44dff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d400c8700 con 0x7f9d2c003a00 2023-11-28T11:24:15.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.997+0000 7f9d44dff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d400c78c0 con 0x7f9d40151c50 2023-11-28T11:24:15.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.997+0000 7f9d44dff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d400c7af0 con 0x7f9d400b76e0 2023-11-28T11:24:15.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.998+0000 7f9d3e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d2c003a00 0x7f9d400c7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:15.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.998+0000 7f9d3e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d2c003a00 0x7f9d400c7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57630/0 (socket says 172.21.15.92:57630) 2023-11-28T11:24:15.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.998+0000 7f9d3e59c700 1 -- 172.21.15.92:0/3761971731 learned_addr learned my addr 172.21.15.92:0/3761971731 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:15.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.998+0000 7f9d37fff700 1 --2- 172.21.15.92:0/3761971731 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d400b76e0 0x7f9d400d0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:15.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.998+0000 7f9d3ed9d700 1 --2- 172.21.15.92:0/3761971731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d40151c50 0x7f9d401520c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:15.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.998+0000 7f9d3e59c700 1 -- 172.21.15.92:0/3761971731 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d400b76e0 msgr2=0x7f9d400d0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:15.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.998+0000 7f9d3e59c700 1 --2- 172.21.15.92:0/3761971731 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d400b76e0 0x7f9d400d0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:15.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.998+0000 7f9d3e59c700 1 -- 172.21.15.92:0/3761971731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d40151c50 msgr2=0x7f9d401520c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:15.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.998+0000 7f9d3e59c700 1 --2- 172.21.15.92:0/3761971731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d40151c50 0x7f9d401520c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:15.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.998+0000 7f9d3e59c700 1 -- 172.21.15.92:0/3761971731 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d400cd720 con 0x7f9d2c003a00 2023-11-28T11:24:15.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.998+0000 7f9d3e59c700 1 --2- 172.21.15.92:0/3761971731 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d2c003a00 0x7f9d400c7380 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c00c0e0 tx=0x7f9d30001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:15.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.999+0000 7f9d36ffd700 1 -- 172.21.15.92:0/3761971731 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d30004c80 con 0x7f9d2c003a00 2023-11-28T11:24:15.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.999+0000 7f9d36ffd700 1 -- 172.21.15.92:0/3761971731 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d30004e00 con 0x7f9d2c003a00 2023-11-28T11:24:15.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.999+0000 7f9d36ffd700 1 -- 172.21.15.92:0/3761971731 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d30007310 con 0x7f9d2c003a00 2023-11-28T11:24:15.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.999+0000 7f9d44dff700 1 -- 172.21.15.92:0/3761971731 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d400b2680 con 0x7f9d2c003a00 2023-11-28T11:24:15.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:14.999+0000 7f9d44dff700 1 -- 172.21.15.92:0/3761971731 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d400cda00 con 0x7f9d2c003a00 2023-11-28T11:24:15.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:15.000+0000 7f9d44dff700 1 -- 172.21.15.92:0/3761971731 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d40157370 con 0x7f9d2c003a00 2023-11-28T11:24:15.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:15.001+0000 7f9d36ffd700 1 -- 172.21.15.92:0/3761971731 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9d30010070 con 0x7f9d2c003a00 2023-11-28T11:24:15.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:15.001+0000 7f9d36ffd700 1 --2- 172.21.15.92:0/3761971731 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9d200682f0 0x7f9d2006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:15.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:15.002+0000 7f9d36ffd700 1 -- 172.21.15.92:0/3761971731 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5900+0+0 (secure 0 0 0) 0x7f9d30086770 con 0x7f9d2c003a00 2023-11-28T11:24:15.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:15.002+0000 7f9d37fff700 1 --2- 172.21.15.92:0/3761971731 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9d200682f0 0x7f9d2006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:15.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:15.002+0000 7f9d37fff700 1 --2- 172.21.15.92:0/3761971731 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9d200682f0 0x7f9d2006a7a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9d28000c00 tx=0x7f9d28009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:15.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:15.006+0000 7f9d36ffd700 1 -- 172.21.15.92:0/3761971731 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9d40157370 con 0x7f9d2c003a00 2023-11-28T11:24:15.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:15.196+0000 7f9d44dff700 1 -- 172.21.15.92:0/3761971731 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f9d40157530 con 0x7f9d2c003a00 2023-11-28T11:24:15.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:15.766+0000 7f9d36ffd700 1 -- 172.21.15.92:0/3761971731 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v66) v1 ==== 170+0+0 (secure 0 0 0) 0x7f9d30052f80 con 0x7f9d2c003a00 2023-11-28T11:24:15.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:15.771+0000 7f9d44dff700 1 -- 172.21.15.92:0/3761971731 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9d200682f0 msgr2=0x7f9d2006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:15.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:15.771+0000 7f9d44dff700 1 --2- 172.21.15.92:0/3761971731 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9d200682f0 0x7f9d2006a7a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9d28000c00 tx=0x7f9d28009040 comp rx=0 tx=0).stop 2023-11-28T11:24:15.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:15.771+0000 7f9d44dff700 1 -- 172.21.15.92:0/3761971731 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d2c003a00 msgr2=0x7f9d400c7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:15.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:15.771+0000 7f9d44dff700 1 --2- 172.21.15.92:0/3761971731 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d2c003a00 0x7f9d400c7380 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c00c0e0 tx=0x7f9d30001a00 comp rx=0 tx=0).stop 2023-11-28T11:24:15.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:15.772+0000 7f9d44dff700 1 -- 172.21.15.92:0/3761971731 shutdown_connections 2023-11-28T11:24:15.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:15.773+0000 7f9d44dff700 1 --2- 172.21.15.92:0/3761971731 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9d40151c50 0x7f9d401520c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:15.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:15.773+0000 7f9d44dff700 1 --2- 172.21.15.92:0/3761971731 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9d200682f0 0x7f9d2006a7a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:15.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:15.773+0000 7f9d44dff700 1 --2- 172.21.15.92:0/3761971731 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9d2c003a00 0x7f9d400c7380 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:15.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:15.773+0000 7f9d44dff700 1 --2- 172.21.15.92:0/3761971731 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f9d400b76e0 0x7f9d400d0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:15.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:15.773+0000 7f9d44dff700 1 -- 172.21.15.92:0/3761971731 >> 172.21.15.92:0/3761971731 conn(0x7f9d400719a0 msgr2=0x7f9d400d31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:15.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:15.774+0000 7f9d44dff700 1 -- 172.21.15.92:0/3761971731 shutdown_connections 2023-11-28T11:24:15.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:15.774+0000 7f9d44dff700 1 -- 172.21.15.92:0/3761971731 wait complete. 2023-11-28T11:24:15.780 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' created 2023-11-28T11:24:15.805 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T11:24:16.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.003+0000 7fe4ab1ff700 1 Processor -- start 2023-11-28T11:24:16.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.003+0000 7fe4ab1ff700 1 -- start start 2023-11-28T11:24:16.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.004+0000 7fe4ab1ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe4a40b8510 0x7fe4a406b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:16.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.005+0000 7fe4ab1ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4a40b88e0 con 0x7fe4a40cd460 2023-11-28T11:24:16.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.005+0000 7fe4ab1ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4a40cd8f0 con 0x7fe4a40d0d60 2023-11-28T11:24:16.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.005+0000 7fe4ab1ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4a40cdbd0 con 0x7fe4a40b8510 2023-11-28T11:24:16.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.005+0000 7fe4a8f9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe4a40b8510 0x7fe4a406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:16.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.005+0000 7fe4a3fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe4a40cd460 0x7fe4a406beb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47494/0 (socket says 172.21.15.92:47494) 2023-11-28T11:24:16.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.005+0000 7fe4a3fff700 1 -- 172.21.15.92:0/1234107114 learned_addr learned my addr 172.21.15.92:0/1234107114 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:24:16.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.005+0000 7fe4a8f9b700 1 -- 172.21.15.92:0/1234107114 >> v1:172.21.15.169:6789/0 conn(0x7fe4a40d0d60 legacy=0x7fe4a40d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:16.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.005+0000 7fe4a8f9b700 1 -- 172.21.15.92:0/1234107114 >> v1:172.21.15.92:6789/0 conn(0x7fe4a40cd460 legacy=0x7fe4a406beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:16.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.005+0000 7fe4a37fe700 1 -- 172.21.15.92:0/1234107114 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3392017697 0 0) 0x7fe4a40b88e0 con 0x7fe4a40cd460 2023-11-28T11:24:16.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.005+0000 7fe4a8f9b700 1 -- 172.21.15.92:0/1234107114 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4a40cfd10 con 0x7fe4a40b8510 2023-11-28T11:24:16.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.006+0000 7fe4a8f9b700 1 --2- 172.21.15.92:0/1234107114 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4a40b8510 0x7fe4a406b970 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fe498009840 tx=0x7fe498009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d4d0539e1f29da9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:16.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.006+0000 7fe4a37fe700 1 -- 172.21.15.92:0/1234107114 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe498011830 con 0x7fe4a40b8510 2023-11-28T11:24:16.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.006+0000 7fe4a37fe700 1 -- 172.21.15.92:0/1234107114 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe498011990 con 0x7fe4a40b8510 2023-11-28T11:24:16.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.006+0000 7fe4a37fe700 1 -- 172.21.15.92:0/1234107114 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe498011d20 con 0x7fe4a40b8510 2023-11-28T11:24:16.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.006+0000 7fe4ab1ff700 1 -- 172.21.15.92:0/1234107114 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4a40b8510 msgr2=0x7fe4a406b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:16.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.006+0000 7fe4ab1ff700 1 --2- 172.21.15.92:0/1234107114 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4a40b8510 0x7fe4a406b970 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fe498009840 tx=0x7fe498009b50 comp rx=0 tx=0).stop 2023-11-28T11:24:16.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.006+0000 7fe4ab1ff700 1 -- 172.21.15.92:0/1234107114 shutdown_connections 2023-11-28T11:24:16.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.006+0000 7fe4ab1ff700 1 --2- 172.21.15.92:0/1234107114 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4a40b8510 0x7fe4a406b970 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:16.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.006+0000 7fe4ab1ff700 1 -- 172.21.15.92:0/1234107114 >> 172.21.15.92:0/1234107114 conn(0x7fe4a40719a0 msgr2=0x7fe4a4071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:16.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.007+0000 7fe4ab1ff700 1 -- 172.21.15.92:0/1234107114 shutdown_connections 2023-11-28T11:24:16.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.007+0000 7fe4ab1ff700 1 -- 172.21.15.92:0/1234107114 wait complete. 2023-11-28T11:24:16.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.009+0000 7fe4ab1ff700 1 Processor -- start 2023-11-28T11:24:16.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.009+0000 7fe4ab1ff700 1 -- start start 2023-11-28T11:24:16.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.010+0000 7fe4ab1ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4a40b8510 0x7fe4a415abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:16.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.010+0000 7fe4ab1ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4a40cd460 0x7fe4a41e1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:16.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.010+0000 7fe4ab1ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe4a40d0d60 0x7fe4a40b7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:16.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.010+0000 7fe4ab1ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a41e30b0 con 0x7fe4a40cd460 2023-11-28T11:24:16.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.010+0000 7fe4ab1ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a41e24a0 con 0x7fe4a40b8510 2023-11-28T11:24:16.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.010+0000 7fe4ab1ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a41e2780 con 0x7fe4a40d0d60 2023-11-28T11:24:16.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.010+0000 7fe4a3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4a40cd460 0x7fe4a41e1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:16.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.011+0000 7fe4a3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4a40cd460 0x7fe4a41e1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57646/0 (socket says 172.21.15.92:57646) 2023-11-28T11:24:16.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.011+0000 7fe4a3fff700 1 -- 172.21.15.92:0/2642127814 learned_addr learned my addr 172.21.15.92:0/2642127814 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:16.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.011+0000 7fe4a8f9b700 1 --2- 172.21.15.92:0/2642127814 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4a40b8510 0x7fe4a415abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:16.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.011+0000 7fe4a979c700 1 --2- 172.21.15.92:0/2642127814 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe4a40d0d60 0x7fe4a40b7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:16.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.011+0000 7fe4a3fff700 1 -- 172.21.15.92:0/2642127814 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe4a40d0d60 msgr2=0x7fe4a40b7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:16.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.011+0000 7fe4a3fff700 1 --2- 172.21.15.92:0/2642127814 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe4a40d0d60 0x7fe4a40b7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:16.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.011+0000 7fe4a3fff700 1 -- 172.21.15.92:0/2642127814 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4a40b8510 msgr2=0x7fe4a415abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:16.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.011+0000 7fe4a3fff700 1 --2- 172.21.15.92:0/2642127814 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4a40b8510 0x7fe4a415abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:16.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.011+0000 7fe4a3fff700 1 -- 172.21.15.92:0/2642127814 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4980094a0 con 0x7fe4a40cd460 2023-11-28T11:24:16.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.011+0000 7fe4a979c700 1 --2- 172.21.15.92:0/2642127814 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe4a40d0d60 0x7fe4a40b7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:24:16.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.011+0000 7fe4a3fff700 1 --2- 172.21.15.92:0/2642127814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4a40cd460 0x7fe4a41e1a20 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fe49000c550 tx=0x7fe49000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:16.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.012+0000 7fe4a17fa700 1 -- 172.21.15.92:0/2642127814 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe490004c80 con 0x7fe4a40cd460 2023-11-28T11:24:16.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.012+0000 7fe4a17fa700 1 -- 172.21.15.92:0/2642127814 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe490004e20 con 0x7fe4a40cd460 2023-11-28T11:24:16.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.012+0000 7fe4ab1ff700 1 -- 172.21.15.92:0/2642127814 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4a40d29c0 con 0x7fe4a40cd460 2023-11-28T11:24:16.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.012+0000 7fe4a17fa700 1 -- 172.21.15.92:0/2642127814 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe490007310 con 0x7fe4a40cd460 2023-11-28T11:24:16.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.012+0000 7fe4ab1ff700 1 -- 172.21.15.92:0/2642127814 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4a40cfff0 con 0x7fe4a40cd460 2023-11-28T11:24:16.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.013+0000 7fe4ab1ff700 1 -- 172.21.15.92:0/2642127814 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4a40d2b50 con 0x7fe4a40cd460 2023-11-28T11:24:16.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.014+0000 7fe4a17fa700 1 -- 172.21.15.92:0/2642127814 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe49001a020 con 0x7fe4a40cd460 2023-11-28T11:24:16.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.015+0000 7fe4a17fa700 1 --2- 172.21.15.92:0/2642127814 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe4880682f0 0x7fe48806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:16.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.015+0000 7fe4a17fa700 1 -- 172.21.15.92:0/2642127814 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6282+0+0 (secure 0 0 0) 0x7fe4900859f0 con 0x7fe4a40cd460 2023-11-28T11:24:16.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.015+0000 7fe4a8f9b700 1 --2- 172.21.15.92:0/2642127814 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe4880682f0 0x7fe48806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:16.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.016+0000 7fe4a8f9b700 1 --2- 172.21.15.92:0/2642127814 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe4880682f0 0x7fe48806a7a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe4980090b0 tx=0x7fe498008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:16.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.020+0000 7fe4a17fa700 1 -- 172.21.15.92:0/2642127814 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe490052080 con 0x7fe4a40cd460 2023-11-28T11:24:16.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.255+0000 7fe4ab1ff700 1 -- 172.21.15.92:0/2642127814 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe4a4067880 con 0x7fe4a40cd460 2023-11-28T11:24:16.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.263+0000 7fe4a17fa700 1 -- 172.21.15.92:0/2642127814 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v28) v1 ==== 174+0+0 (secure 0 0 0) 0x7fe4900556a0 con 0x7fe4a40cd460 2023-11-28T11:24:16.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.270+0000 7fe4ab1ff700 1 -- 172.21.15.92:0/2642127814 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe4880682f0 msgr2=0x7fe48806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:16.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.270+0000 7fe4ab1ff700 1 --2- 172.21.15.92:0/2642127814 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe4880682f0 0x7fe48806a7a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe4980090b0 tx=0x7fe498008030 comp rx=0 tx=0).stop 2023-11-28T11:24:16.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.270+0000 7fe4ab1ff700 1 -- 172.21.15.92:0/2642127814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4a40cd460 msgr2=0x7fe4a41e1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:16.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.270+0000 7fe4ab1ff700 1 --2- 172.21.15.92:0/2642127814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4a40cd460 0x7fe4a41e1a20 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fe49000c550 tx=0x7fe49000c860 comp rx=0 tx=0).stop 2023-11-28T11:24:16.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.270+0000 7fe4ab1ff700 1 -- 172.21.15.92:0/2642127814 shutdown_connections 2023-11-28T11:24:16.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.270+0000 7fe4ab1ff700 1 --2- 172.21.15.92:0/2642127814 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe4a40d0d60 0x7fe4a40b7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:16.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.270+0000 7fe4ab1ff700 1 --2- 172.21.15.92:0/2642127814 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4a40b8510 0x7fe4a415abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:16.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.270+0000 7fe4ab1ff700 1 --2- 172.21.15.92:0/2642127814 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe4880682f0 0x7fe48806a7a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:16.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.270+0000 7fe4ab1ff700 1 --2- 172.21.15.92:0/2642127814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4a40cd460 0x7fe4a41e1a20 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:16.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.270+0000 7fe4ab1ff700 1 -- 172.21.15.92:0/2642127814 >> 172.21.15.92:0/2642127814 conn(0x7fe4a40719a0 msgr2=0x7fe4a40cbef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:16.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.272+0000 7fe4ab1ff700 1 -- 172.21.15.92:0/2642127814 shutdown_connections 2023-11-28T11:24:16.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.273+0000 7fe4ab1ff700 1 -- 172.21.15.92:0/2642127814 wait complete. 2023-11-28T11:24:16.281 INFO:teuthology.orchestra.run.smithi092.stderr:new fs with metadata pool 10 and data pool 11 2023-11-28T11:24:16.307 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:24:16.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.501+0000 7f8fc81ff700 1 Processor -- start 2023-11-28T11:24:16.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.501+0000 7f8fc81ff700 1 -- start start 2023-11-28T11:24:16.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.502+0000 7f8fc81ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8fc00d1d70 0x7f8fc0167270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:16.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.502+0000 7f8fc81ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fc006c790 con 0x7f8fc00b8510 2023-11-28T11:24:16.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.502+0000 7f8fc81ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fc00b88e0 con 0x7f8fc00cd460 2023-11-28T11:24:16.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.502+0000 7f8fc81ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fc00cdbd0 con 0x7f8fc00d1d70 2023-11-28T11:24:16.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.503+0000 7f8fc679c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8fc00d1d70 0x7f8fc0167270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:16.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.503+0000 7f8fc5f9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f8fc00b8510 0x7f8fc006b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47500/0 (socket says 172.21.15.92:47500) 2023-11-28T11:24:16.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.503+0000 7f8fc5f9b700 1 -- 172.21.15.92:0/1786225857 learned_addr learned my addr 172.21.15.92:0/1786225857 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:24:16.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.503+0000 7f8fc679c700 1 -- 172.21.15.92:0/1786225857 >> v1:172.21.15.169:6789/0 conn(0x7f8fc00cd460 legacy=0x7f8fc006c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:16.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.503+0000 7f8fc679c700 1 -- 172.21.15.92:0/1786225857 >> v1:172.21.15.92:6789/0 conn(0x7f8fc00b8510 legacy=0x7f8fc006b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:16.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.503+0000 7f8fc679c700 1 -- 172.21.15.92:0/1786225857 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc00cd8f0 con 0x7f8fc00d1d70 2023-11-28T11:24:16.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.503+0000 7f8fc4f99700 1 -- 172.21.15.92:0/1786225857 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 207165933 0 0) 0x7f8fc006c790 con 0x7f8fc00b8510 2023-11-28T11:24:16.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.503+0000 7f8fc679c700 1 --2- 172.21.15.92:0/1786225857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8fc00d1d70 0x7f8fc0167270 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8009d90 tx=0x7f8fb800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bce27b0c7e71c60d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:16.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.504+0000 7f8fc4f99700 1 -- 172.21.15.92:0/1786225857 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb800e070 con 0x7f8fc00d1d70 2023-11-28T11:24:16.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.504+0000 7f8fc4f99700 1 -- 172.21.15.92:0/1786225857 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb800bbd0 con 0x7f8fc00d1d70 2023-11-28T11:24:16.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.504+0000 7f8fc4f99700 1 -- 172.21.15.92:0/1786225857 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb8012430 con 0x7f8fc00d1d70 2023-11-28T11:24:16.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.504+0000 7f8fc81ff700 1 -- 172.21.15.92:0/1786225857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8fc00d1d70 msgr2=0x7f8fc0167270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:16.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.504+0000 7f8fc81ff700 1 --2- 172.21.15.92:0/1786225857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8fc00d1d70 0x7f8fc0167270 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8009d90 tx=0x7f8fb800b330 comp rx=0 tx=0).stop 2023-11-28T11:24:16.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.504+0000 7f8fc81ff700 1 -- 172.21.15.92:0/1786225857 shutdown_connections 2023-11-28T11:24:16.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.504+0000 7f8fc81ff700 1 --2- 172.21.15.92:0/1786225857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8fc00d1d70 0x7f8fc0167270 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:16.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.504+0000 7f8fc81ff700 1 -- 172.21.15.92:0/1786225857 >> 172.21.15.92:0/1786225857 conn(0x7f8fc00719a0 msgr2=0x7f8fc0071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:16.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.504+0000 7f8fc81ff700 1 -- 172.21.15.92:0/1786225857 shutdown_connections 2023-11-28T11:24:16.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.504+0000 7f8fc81ff700 1 -- 172.21.15.92:0/1786225857 wait complete. 2023-11-28T11:24:16.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.506+0000 7f8fc81ff700 1 Processor -- start 2023-11-28T11:24:16.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.506+0000 7f8fc81ff700 1 -- start start 2023-11-28T11:24:16.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.507+0000 7f8fc81ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8fc00b8510 0x7f8fc01e7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:16.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.507+0000 7f8fc81ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8fc00cd460 0x7f8fc01e4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:16.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.507+0000 7f8fc81ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8fc00d1d70 0x7f8fc015a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:16.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.507+0000 7f8fc81ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc01e6680 con 0x7f8fc00d1d70 2023-11-28T11:24:16.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.507+0000 7f8fc81ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc01e5a00 con 0x7f8fc00cd460 2023-11-28T11:24:16.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.507+0000 7f8fc81ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc01e5ce0 con 0x7f8fc00b8510 2023-11-28T11:24:16.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.507+0000 7f8fc679c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8fc00d1d70 0x7f8fc015a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:16.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.507+0000 7f8fc679c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8fc00d1d70 0x7f8fc015a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57660/0 (socket says 172.21.15.92:57660) 2023-11-28T11:24:16.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.507+0000 7f8fc679c700 1 -- 172.21.15.92:0/4133408637 learned_addr learned my addr 172.21.15.92:0/4133408637 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:16.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.507+0000 7f8fc579a700 1 --2- 172.21.15.92:0/4133408637 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8fc00cd460 0x7f8fc01e4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:16.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.507+0000 7f8fc5f9b700 1 --2- 172.21.15.92:0/4133408637 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8fc00b8510 0x7f8fc01e7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:16.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.508+0000 7f8fc679c700 1 -- 172.21.15.92:0/4133408637 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8fc00b8510 msgr2=0x7f8fc01e7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:16.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.508+0000 7f8fc679c700 1 --2- 172.21.15.92:0/4133408637 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8fc00b8510 0x7f8fc01e7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:16.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.508+0000 7f8fc679c700 1 -- 172.21.15.92:0/4133408637 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8fc00cd460 msgr2=0x7f8fc01e4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:16.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.508+0000 7f8fc679c700 1 --2- 172.21.15.92:0/4133408637 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8fc00cd460 0x7f8fc01e4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:16.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.508+0000 7f8fc679c700 1 -- 172.21.15.92:0/4133408637 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb8009a70 con 0x7f8fc00d1d70 2023-11-28T11:24:16.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.508+0000 7f8fc5f9b700 1 --2- 172.21.15.92:0/4133408637 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8fc00b8510 0x7f8fc01e7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:24:16.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.508+0000 7f8fc679c700 1 --2- 172.21.15.92:0/4133408637 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8fc00d1d70 0x7f8fc015a600 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8014040 tx=0x7f8fb8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:16.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.508+0000 7f8fb67fc700 1 -- 172.21.15.92:0/4133408637 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb8012430 con 0x7f8fc00d1d70 2023-11-28T11:24:16.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.508+0000 7f8fb67fc700 1 -- 172.21.15.92:0/4133408637 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb8020070 con 0x7f8fc00d1d70 2023-11-28T11:24:16.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.508+0000 7f8fc81ff700 1 -- 172.21.15.92:0/4133408637 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fc00cfff0 con 0x7f8fc00d1d70 2023-11-28T11:24:16.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.508+0000 7f8fb67fc700 1 -- 172.21.15.92:0/4133408637 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb8007000 con 0x7f8fc00d1d70 2023-11-28T11:24:16.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.508+0000 7f8fc81ff700 1 -- 172.21.15.92:0/4133408637 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fc00cfd10 con 0x7f8fc00d1d70 2023-11-28T11:24:16.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.509+0000 7f8fc81ff700 1 -- 172.21.15.92:0/4133408637 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fc01e67c0 con 0x7f8fc00d1d70 2023-11-28T11:24:16.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.510+0000 7f8fb67fc700 1 -- 172.21.15.92:0/4133408637 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8fb802e070 con 0x7f8fc00d1d70 2023-11-28T11:24:16.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.511+0000 7f8fb67fc700 1 --2- 172.21.15.92:0/4133408637 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8fa40682f0 0x7f8fa406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:16.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.511+0000 7f8fb67fc700 1 -- 172.21.15.92:0/4133408637 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f8fb8022070 con 0x7f8fc00d1d70 2023-11-28T11:24:16.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.511+0000 7f8fc5f9b700 1 --2- 172.21.15.92:0/4133408637 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8fa40682f0 0x7f8fa406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:16.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.512+0000 7f8fc5f9b700 1 --2- 172.21.15.92:0/4133408637 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8fa40682f0 0x7f8fa406a7a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc000f60 tx=0x7f8fbc007ce0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:16.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.515+0000 7f8fb67fc700 1 -- 172.21.15.92:0/4133408637 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8fc01e67c0 con 0x7f8fc00d1d70 2023-11-28T11:24:16.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.704+0000 7f8fc81ff700 1 -- 172.21.15.92:0/4133408637 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8fc01e67c0 con 0x7f8fc00d1d70 2023-11-28T11:24:16.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.706+0000 7f8fb67fc700 1 -- 172.21.15.92:0/4133408637 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+18196 (secure 0 0 0) 0x7f8fc01e67c0 con 0x7f8fc00d1d70 2023-11-28T11:24:16.708 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:24:16.708 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":67,"fsid":"68956e0c-c16f-47cd-a220-f98364b0831d","created":"2023-11-28T11:20:28.697573+0000","modified":"2023-11-28T11:24:16.257439+0000","last_up_change":"2023-11-28T11:20:41.257011+0000","last_in_change":"2023-11-28T11:20:32.249767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:20:41.925237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:20:48.682497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:24:13.972356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-28T11:24:15.237693+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6abe483f-0e03-4a26-b9cf-62041291747e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":115823},{"type":"v1","addr":"172.21.15.92:6809","nonce":115823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":115823},{"type":"v1","addr":"172.21.15.92:6813","nonce":115823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":115823},{"type":"v1","addr":"172.21.15.92:6821","nonce":115823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":115823},{"type":"v1","addr":"172.21.15.92:6817","nonce":115823}]},"public_addr":"172.21.15.92:6809/115823","cluster_addr":"172.21.15.92:6813/115823","heartbeat_back_addr":"172.21.15.92:6821/115823","heartbeat_front_addr":"172.21.15.92:6817/115823","state":["exists","up"]},{"osd":1,"uuid":"13c1de73-1a8c-4d7e-8a28-9dae52dfe089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":115824},{"type":"v1","addr":"172.21.15.92:6812","nonce":115824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":115824},{"type":"v1","addr":"172.21.15.92:6816","nonce":115824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":115824},{"type":"v1","addr":"172.21.15.92:6823","nonce":115824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":115824},{"type":"v1","addr":"172.21.15.92:6820","nonce":115824}]},"public_addr":"172.21.15.92:6812/115824","cluster_addr":"172.21.15.92:6816/115824","heartbeat_back_addr":"172.21.15.92:6823/115824","heartbeat_front_addr":"172.21.15.92:6820/115824","state":["exists","up"]},{"osd":2,"uuid":"25788a55-112f-4f00-810d-2bc37cf76778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":115821},{"type":"v1","addr":"172.21.15.92:6801","nonce":115821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":115821},{"type":"v1","addr":"172.21.15.92:6803","nonce":115821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":115821},{"type":"v1","addr":"172.21.15.92:6807","nonce":115821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":115821},{"type":"v1","addr":"172.21.15.92:6805","nonce":115821}]},"public_addr":"172.21.15.92:6801/115821","cluster_addr":"172.21.15.92:6803/115821","heartbeat_back_addr":"172.21.15.92:6807/115821","heartbeat_front_addr":"172.21.15.92:6805/115821","state":["exists","up"]},{"osd":3,"uuid":"adf54e24-33a9-41fe-9ce3-98a5ab6c158e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":115825},{"type":"v1","addr":"172.21.15.92:6825","nonce":115825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":115825},{"type":"v1","addr":"172.21.15.92:6827","nonce":115825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":115825},{"type":"v1","addr":"172.21.15.92:6831","nonce":115825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":115825},{"type":"v1","addr":"172.21.15.92:6829","nonce":115825}]},"public_addr":"172.21.15.92:6825/115825","cluster_addr":"172.21.15.92:6827/115825","heartbeat_back_addr":"172.21.15.92:6831/115825","heartbeat_front_addr":"172.21.15.92:6829/115825","state":["exists","up"]},{"osd":4,"uuid":"ccb9b747-f399-44ea-9a18-9db3152227e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":116035},{"type":"v1","addr":"172.21.15.169:6817","nonce":116035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":116035},{"type":"v1","addr":"172.21.15.169:6819","nonce":116035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":116035},{"type":"v1","addr":"172.21.15.169:6826","nonce":116035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":116035},{"type":"v1","addr":"172.21.15.169:6822","nonce":116035}]},"public_addr":"172.21.15.169:6817/116035","cluster_addr":"172.21.15.169:6819/116035","heartbeat_back_addr":"172.21.15.169:6826/116035","heartbeat_front_addr":"172.21.15.169:6822/116035","state":["exists","up"]},{"osd":5,"uuid":"308a1e63-75aa-4d7d-8a7c-cecc24714fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":116033},{"type":"v1","addr":"172.21.15.169:6801","nonce":116033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":116033},{"type":"v1","addr":"172.21.15.169:6803","nonce":116033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":116033},{"type":"v1","addr":"172.21.15.169:6807","nonce":116033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":116033},{"type":"v1","addr":"172.21.15.169:6805","nonce":116033}]},"public_addr":"172.21.15.169:6801/116033","cluster_addr":"172.21.15.169:6803/116033","heartbeat_back_addr":"172.21.15.169:6807/116033","heartbeat_front_addr":"172.21.15.169:6805/116033","state":["exists","up"]},{"osd":6,"uuid":"9c459b7b-b821-4b4b-b91e-6aaae271457c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":116034},{"type":"v1","addr":"172.21.15.169:6809","nonce":116034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":116034},{"type":"v1","addr":"172.21.15.169:6811","nonce":116034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":116034},{"type":"v1","addr":"172.21.15.169:6815","nonce":116034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":116034},{"type":"v1","addr":"172.21.15.169:6813","nonce":116034}]},"public_addr":"172.21.15.169:6809/116034","cluster_addr":"172.21.15.169:6811/116034","heartbeat_back_addr":"172.21.15.169:6815/116034","heartbeat_front_addr":"172.21.15.169:6813/116034","state":["exists","up"]},{"osd":7,"uuid":"1ca9aa4a-755d-4e28-a937-4d8dccab4df3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":116044},{"type":"v1","addr":"172.21.15.169:6823","nonce":116044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6825","nonce":116044},{"type":"v1","addr":"172.21.15.169:6827","nonce":116044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":116044},{"type":"v1","addr":"172.21.15.169:6831","nonce":116044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":116044},{"type":"v1","addr":"172.21.15.169:6829","nonce":116044}]},"public_addr":"172.21.15.169:6823/116044","cluster_addr":"172.21.15.169:6827/116044","heartbeat_back_addr":"172.21.15.169:6831/116044","heartbeat_front_addr":"172.21.15.169:6829/116044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.586781+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.869778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.508594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.860493+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.872267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.746933+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.611520+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.871446+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:24:16.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.708+0000 7f8fc81ff700 1 -- 172.21.15.92:0/4133408637 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8fa40682f0 msgr2=0x7f8fa406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:16.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.708+0000 7f8fc81ff700 1 --2- 172.21.15.92:0/4133408637 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8fa40682f0 0x7f8fa406a7a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc000f60 tx=0x7f8fbc007ce0 comp rx=0 tx=0).stop 2023-11-28T11:24:16.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.708+0000 7f8fc81ff700 1 -- 172.21.15.92:0/4133408637 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8fc00d1d70 msgr2=0x7f8fc015a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:16.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.708+0000 7f8fc81ff700 1 --2- 172.21.15.92:0/4133408637 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8fc00d1d70 0x7f8fc015a600 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8014040 tx=0x7f8fb8004ad0 comp rx=0 tx=0).stop 2023-11-28T11:24:16.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.708+0000 7f8fc81ff700 1 -- 172.21.15.92:0/4133408637 shutdown_connections 2023-11-28T11:24:16.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.708+0000 7f8fc81ff700 1 --2- 172.21.15.92:0/4133408637 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f8fa40682f0 0x7f8fa406a7a0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:16.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.708+0000 7f8fc81ff700 1 --2- 172.21.15.92:0/4133408637 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8fc00d1d70 0x7f8fc015a600 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:16.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.708+0000 7f8fc81ff700 1 --2- 172.21.15.92:0/4133408637 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f8fc00b8510 0x7f8fc01e7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:16.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.708+0000 7f8fc81ff700 1 --2- 172.21.15.92:0/4133408637 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8fc00cd460 0x7f8fc01e4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:16.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.708+0000 7f8fc81ff700 1 -- 172.21.15.92:0/4133408637 >> 172.21.15.92:0/4133408637 conn(0x7f8fc00719a0 msgr2=0x7f8fc00b6a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:16.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.708+0000 7f8fc81ff700 1 -- 172.21.15.92:0/4133408637 shutdown_connections 2023-11-28T11:24:16.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.708+0000 7f8fc81ff700 1 -- 172.21.15.92:0/4133408637 wait complete. 2023-11-28T11:24:16.728 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:24:16.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.923+0000 7f92127ff700 1 Processor -- start 2023-11-28T11:24:16.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.924+0000 7f92127ff700 1 -- start start 2023-11-28T11:24:16.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.925+0000 7f92127ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f920c0cd460 0x7f920c06c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:16.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.925+0000 7f92127ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f920c0d0d60 0x7f920c0d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:16.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.925+0000 7f92127ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f920c0b88e0 con 0x7f920c0b8510 2023-11-28T11:24:16.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.925+0000 7f92127ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f920c0cdbd0 con 0x7f920c0cd460 2023-11-28T11:24:16.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.925+0000 7f92127ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f920c0d02d0 con 0x7f920c0d0d60 2023-11-28T11:24:16.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.925+0000 7f920bfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f920c0b8510 0x7f920c06b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47514/0 (socket says 172.21.15.92:47514) 2023-11-28T11:24:16.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.925+0000 7f920bfff700 1 -- 172.21.15.92:0/1706183976 learned_addr learned my addr 172.21.15.92:0/1706183976 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:24:16.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.925+0000 7f920b7fe700 1 --2- 172.21.15.92:0/1706183976 >> v2:172.21.15.169:3300/0 conn(0x7f920c0cd460 0x7f920c06c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:16.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.925+0000 7f9210d9c700 1 --2- 172.21.15.92:0/1706183976 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f920c0d0d60 0x7f920c0d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:16.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.925+0000 7f920affd700 1 -- 172.21.15.92:0/1706183976 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1902505900 0 0) 0x7f920c0b88e0 con 0x7f920c0b8510 2023-11-28T11:24:16.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.925+0000 7f920affd700 1 -- 172.21.15.92:0/1706183976 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91f40035b0 con 0x7f920c0b8510 2023-11-28T11:24:16.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.926+0000 7f920affd700 1 -- 172.21.15.92:0/1706183976 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 900578261 0 0) 0x7f91f40035b0 con 0x7f920c0b8510 2023-11-28T11:24:16.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.926+0000 7f920affd700 1 -- 172.21.15.92:0/1706183976 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f920c0d0d60 msgr2=0x7f920c0d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:16.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.926+0000 7f920affd700 1 --2- 172.21.15.92:0/1706183976 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f920c0d0d60 0x7f920c0d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:16.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.926+0000 7f920affd700 1 -- 172.21.15.92:0/1706183976 >> v2:172.21.15.169:3300/0 conn(0x7f920c0cd460 msgr2=0x7f920c06c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:16.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.926+0000 7f920affd700 1 --2- 172.21.15.92:0/1706183976 >> v2:172.21.15.169:3300/0 conn(0x7f920c0cd460 0x7f920c06c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:16.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.926+0000 7f920affd700 1 -- 172.21.15.92:0/1706183976 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f920c0cd8f0 con 0x7f920c0b8510 2023-11-28T11:24:16.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.926+0000 7f920affd700 1 -- 172.21.15.92:0/1706183976 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532161587 0 0) 0x7f91fc001670 con 0x7f920c0b8510 2023-11-28T11:24:16.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.926+0000 7f920b7fe700 1 --2- 172.21.15.92:0/1706183976 >> v2:172.21.15.169:3300/0 conn(0x7f920c0cd460 0x7f920c06c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:24:16.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.926+0000 7f920affd700 1 -- 172.21.15.92:0/1706183976 >> v1:172.21.15.92:6789/0 conn(0x7f920c0b8510 legacy=0x7f920c06b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:16.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.926+0000 7f920affd700 1 --2- 172.21.15.92:0/1706183976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91f4003a30 0x7f91f4003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:16.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.926+0000 7f920affd700 1 --2- 172.21.15.92:0/1706183976 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91f4004850 0x7f91f4004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:16.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.926+0000 7f920affd700 1 --2- 172.21.15.92:0/1706183976 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91f4005200 0x7f91f400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:16.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.926+0000 7f920affd700 1 -- 172.21.15.92:0/1706183976 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920c0cdbd0 con 0x7f91f4003a30 2023-11-28T11:24:16.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.926+0000 7f920affd700 1 -- 172.21.15.92:0/1706183976 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920c0d02d0 con 0x7f91f4005200 2023-11-28T11:24:16.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.926+0000 7f920affd700 1 -- 172.21.15.92:0/1706183976 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91f400bf50 con 0x7f91f4004850 2023-11-28T11:24:16.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.926+0000 7f9210d9c700 1 --2- 172.21.15.92:0/1706183976 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f920c0d0d60 0x7f920c0d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:24:16.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.926+0000 7f920bfff700 1 --2- 172.21.15.92:0/1706183976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91f4003a30 0x7f91f4003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:16.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.926+0000 7f920b7fe700 1 --2- 172.21.15.92:0/1706183976 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91f4004850 0x7f91f4004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:16.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.927+0000 7f9210d9c700 1 --2- 172.21.15.92:0/1706183976 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91f4005200 0x7f91f400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:16.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.927+0000 7f920bfff700 1 -- 172.21.15.92:0/1706183976 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91f4004850 msgr2=0x7f91f4004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:16.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.927+0000 7f920bfff700 1 --2- 172.21.15.92:0/1706183976 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91f4004850 0x7f91f4004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:16.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.927+0000 7f920bfff700 1 -- 172.21.15.92:0/1706183976 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91f4005200 msgr2=0x7f91f400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:16.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.927+0000 7f920bfff700 1 --2- 172.21.15.92:0/1706183976 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91f4005200 0x7f91f400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:16.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.927+0000 7f920bfff700 1 -- 172.21.15.92:0/1706183976 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f40035b0 con 0x7f91f4003a30 2023-11-28T11:24:16.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.927+0000 7f9210d9c700 1 --2- 172.21.15.92:0/1706183976 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91f4005200 0x7f91f400ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:24:16.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.927+0000 7f920bfff700 1 --2- 172.21.15.92:0/1706183976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91f4003a30 0x7f91f4003e40 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f91fc00c240 tx=0x7f91f4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f68b27a57986f55e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:16.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.927+0000 7f920affd700 1 -- 172.21.15.92:0/1706183976 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91fc007c10 con 0x7f91f4003a30 2023-11-28T11:24:16.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.927+0000 7f920affd700 1 -- 172.21.15.92:0/1706183976 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91fc00cda0 con 0x7f91f4003a30 2023-11-28T11:24:16.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.927+0000 7f920affd700 1 -- 172.21.15.92:0/1706183976 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91fc005ba0 con 0x7f91f4003a30 2023-11-28T11:24:16.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.928+0000 7f92127ff700 1 -- 172.21.15.92:0/1706183976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91f4003a30 msgr2=0x7f91f4003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:16.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.928+0000 7f92127ff700 1 --2- 172.21.15.92:0/1706183976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91f4003a30 0x7f91f4003e40 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f91fc00c240 tx=0x7f91f4005880 comp rx=0 tx=0).stop 2023-11-28T11:24:16.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.928+0000 7f9210d9c700 1 -- 172.21.15.92:0/1706183976 reap_dead start 2023-11-28T11:24:16.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.928+0000 7f92127ff700 1 -- 172.21.15.92:0/1706183976 shutdown_connections 2023-11-28T11:24:16.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.928+0000 7f92127ff700 1 -- 172.21.15.92:0/1706183976 >> 172.21.15.92:0/1706183976 conn(0x7f920c0719a0 msgr2=0x7f920c071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:16.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.928+0000 7f92127ff700 1 -- 172.21.15.92:0/1706183976 shutdown_connections 2023-11-28T11:24:16.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.928+0000 7f92127ff700 1 -- 172.21.15.92:0/1706183976 wait complete. 2023-11-28T11:24:16.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.930+0000 7f92127ff700 1 Processor -- start 2023-11-28T11:24:16.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.930+0000 7f92127ff700 1 -- start start 2023-11-28T11:24:16.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.931+0000 7f92127ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91f4003a30 0x7f920c155050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:16.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.931+0000 7f92127ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f920c1548d0 0x7f920c0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:16.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.931+0000 7f92127ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f920c0cc5d0 0x7f920c0cca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:16.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.931+0000 7f92127ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920c0cb530 con 0x7f920c1548d0 2023-11-28T11:24:16.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.931+0000 7f92127ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920c0cb810 con 0x7f91f4003a30 2023-11-28T11:24:16.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.931+0000 7f92127ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f920c0cbaf0 con 0x7f920c0cc5d0 2023-11-28T11:24:16.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.931+0000 7f920b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f920c1548d0 0x7f920c0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:16.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.931+0000 7f920b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f920c1548d0 0x7f920c0b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57676/0 (socket says 172.21.15.92:57676) 2023-11-28T11:24:16.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.931+0000 7f920b7fe700 1 -- 172.21.15.92:0/177722435 learned_addr learned my addr 172.21.15.92:0/177722435 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:16.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.931+0000 7f920bfff700 1 --2- 172.21.15.92:0/177722435 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91f4003a30 0x7f920c155050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:16.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.931+0000 7f920b7fe700 1 -- 172.21.15.92:0/177722435 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f920c0cc5d0 msgr2=0x7f920c0cca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:16.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.931+0000 7f9210d9c700 1 --2- 172.21.15.92:0/177722435 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f920c0cc5d0 0x7f920c0cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:16.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.931+0000 7f920b7fe700 1 --2- 172.21.15.92:0/177722435 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f920c0cc5d0 0x7f920c0cca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:16.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.931+0000 7f920b7fe700 1 -- 172.21.15.92:0/177722435 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91f4003a30 msgr2=0x7f920c155050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:16.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.931+0000 7f920b7fe700 1 --2- 172.21.15.92:0/177722435 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91f4003a30 0x7f920c155050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:16.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.931+0000 7f920b7fe700 1 -- 172.21.15.92:0/177722435 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f920c0cfff0 con 0x7f920c1548d0 2023-11-28T11:24:16.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.931+0000 7f920bfff700 1 --2- 172.21.15.92:0/177722435 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91f4003a30 0x7f920c155050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:24:16.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.932+0000 7f920b7fe700 1 --2- 172.21.15.92:0/177722435 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f920c1548d0 0x7f920c0b5f40 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f9200011ca0 tx=0x7f920000f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:16.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.932+0000 7f9208ff9700 1 -- 172.21.15.92:0/177722435 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920000fb30 con 0x7f920c1548d0 2023-11-28T11:24:16.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.932+0000 7f9208ff9700 1 -- 172.21.15.92:0/177722435 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f920000fc90 con 0x7f920c1548d0 2023-11-28T11:24:16.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.932+0000 7f9208ff9700 1 -- 172.21.15.92:0/177722435 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9200004020 con 0x7f920c1548d0 2023-11-28T11:24:16.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.932+0000 7f92127ff700 1 -- 172.21.15.92:0/177722435 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f920c0cfd10 con 0x7f920c1548d0 2023-11-28T11:24:16.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.932+0000 7f92127ff700 1 -- 172.21.15.92:0/177722435 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f920c0d29c0 con 0x7f920c1548d0 2023-11-28T11:24:16.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.933+0000 7f92127ff700 1 -- 172.21.15.92:0/177722435 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f920c067880 con 0x7f920c1548d0 2023-11-28T11:24:16.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.934+0000 7f9208ff9700 1 -- 172.21.15.92:0/177722435 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9200024070 con 0x7f920c1548d0 2023-11-28T11:24:16.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.935+0000 7f9208ff9700 1 --2- 172.21.15.92:0/177722435 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f91f8068320 0x7f91f806a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:16.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.935+0000 7f9208ff9700 1 -- 172.21.15.92:0/177722435 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f9200086270 con 0x7f920c1548d0 2023-11-28T11:24:16.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.935+0000 7f920bfff700 1 --2- 172.21.15.92:0/177722435 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f91f8068320 0x7f91f806a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:16.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.936+0000 7f920bfff700 1 --2- 172.21.15.92:0/177722435 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f91f8068320 0x7f91f806a7d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f91fc015040 tx=0x7f91fc007000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:16.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:16.939+0000 7f9208ff9700 1 -- 172.21.15.92:0/177722435 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f92000528a0 con 0x7f920c1548d0 2023-11-28T11:24:17.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.130+0000 7f92127ff700 1 -- 172.21.15.92:0/177722435 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f920c0cece0 con 0x7f920c1548d0 2023-11-28T11:24:17.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.131+0000 7f9208ff9700 1 -- 172.21.15.92:0/177722435 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+18196 (secure 0 0 0) 0x7f9200022090 con 0x7f920c1548d0 2023-11-28T11:24:17.133 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:24:17.133 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":67,"fsid":"68956e0c-c16f-47cd-a220-f98364b0831d","created":"2023-11-28T11:20:28.697573+0000","modified":"2023-11-28T11:24:16.257439+0000","last_up_change":"2023-11-28T11:20:41.257011+0000","last_in_change":"2023-11-28T11:20:32.249767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:20:41.925237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:20:48.682497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:24:13.972356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-28T11:24:15.237693+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6abe483f-0e03-4a26-b9cf-62041291747e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":115823},{"type":"v1","addr":"172.21.15.92:6809","nonce":115823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":115823},{"type":"v1","addr":"172.21.15.92:6813","nonce":115823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":115823},{"type":"v1","addr":"172.21.15.92:6821","nonce":115823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":115823},{"type":"v1","addr":"172.21.15.92:6817","nonce":115823}]},"public_addr":"172.21.15.92:6809/115823","cluster_addr":"172.21.15.92:6813/115823","heartbeat_back_addr":"172.21.15.92:6821/115823","heartbeat_front_addr":"172.21.15.92:6817/115823","state":["exists","up"]},{"osd":1,"uuid":"13c1de73-1a8c-4d7e-8a28-9dae52dfe089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":115824},{"type":"v1","addr":"172.21.15.92:6812","nonce":115824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":115824},{"type":"v1","addr":"172.21.15.92:6816","nonce":115824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":115824},{"type":"v1","addr":"172.21.15.92:6823","nonce":115824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":115824},{"type":"v1","addr":"172.21.15.92:6820","nonce":115824}]},"public_addr":"172.21.15.92:6812/115824","cluster_addr":"172.21.15.92:6816/115824","heartbeat_back_addr":"172.21.15.92:6823/115824","heartbeat_front_addr":"172.21.15.92:6820/115824","state":["exists","up"]},{"osd":2,"uuid":"25788a55-112f-4f00-810d-2bc37cf76778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":115821},{"type":"v1","addr":"172.21.15.92:6801","nonce":115821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":115821},{"type":"v1","addr":"172.21.15.92:6803","nonce":115821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":115821},{"type":"v1","addr":"172.21.15.92:6807","nonce":115821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":115821},{"type":"v1","addr":"172.21.15.92:6805","nonce":115821}]},"public_addr":"172.21.15.92:6801/115821","cluster_addr":"172.21.15.92:6803/115821","heartbeat_back_addr":"172.21.15.92:6807/115821","heartbeat_front_addr":"172.21.15.92:6805/115821","state":["exists","up"]},{"osd":3,"uuid":"adf54e24-33a9-41fe-9ce3-98a5ab6c158e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":115825},{"type":"v1","addr":"172.21.15.92:6825","nonce":115825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":115825},{"type":"v1","addr":"172.21.15.92:6827","nonce":115825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":115825},{"type":"v1","addr":"172.21.15.92:6831","nonce":115825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":115825},{"type":"v1","addr":"172.21.15.92:6829","nonce":115825}]},"public_addr":"172.21.15.92:6825/115825","cluster_addr":"172.21.15.92:6827/115825","heartbeat_back_addr":"172.21.15.92:6831/115825","heartbeat_front_addr":"172.21.15.92:6829/115825","state":["exists","up"]},{"osd":4,"uuid":"ccb9b747-f399-44ea-9a18-9db3152227e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":116035},{"type":"v1","addr":"172.21.15.169:6817","nonce":116035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":116035},{"type":"v1","addr":"172.21.15.169:6819","nonce":116035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":116035},{"type":"v1","addr":"172.21.15.169:6826","nonce":116035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":116035},{"type":"v1","addr":"172.21.15.169:6822","nonce":116035}]},"public_addr":"172.21.15.169:6817/116035","cluster_addr":"172.21.15.169:6819/116035","heartbeat_back_addr":"172.21.15.169:6826/116035","heartbeat_front_addr":"172.21.15.169:6822/116035","state":["exists","up"]},{"osd":5,"uuid":"308a1e63-75aa-4d7d-8a7c-cecc24714fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":116033},{"type":"v1","addr":"172.21.15.169:6801","nonce":116033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":116033},{"type":"v1","addr":"172.21.15.169:6803","nonce":116033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":116033},{"type":"v1","addr":"172.21.15.169:6807","nonce":116033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":116033},{"type":"v1","addr":"172.21.15.169:6805","nonce":116033}]},"public_addr":"172.21.15.169:6801/116033","cluster_addr":"172.21.15.169:6803/116033","heartbeat_back_addr":"172.21.15.169:6807/116033","heartbeat_front_addr":"172.21.15.169:6805/116033","state":["exists","up"]},{"osd":6,"uuid":"9c459b7b-b821-4b4b-b91e-6aaae271457c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":116034},{"type":"v1","addr":"172.21.15.169:6809","nonce":116034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":116034},{"type":"v1","addr":"172.21.15.169:6811","nonce":116034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":116034},{"type":"v1","addr":"172.21.15.169:6815","nonce":116034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":116034},{"type":"v1","addr":"172.21.15.169:6813","nonce":116034}]},"public_addr":"172.21.15.169:6809/116034","cluster_addr":"172.21.15.169:6811/116034","heartbeat_back_addr":"172.21.15.169:6815/116034","heartbeat_front_addr":"172.21.15.169:6813/116034","state":["exists","up"]},{"osd":7,"uuid":"1ca9aa4a-755d-4e28-a937-4d8dccab4df3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":116044},{"type":"v1","addr":"172.21.15.169:6823","nonce":116044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6825","nonce":116044},{"type":"v1","addr":"172.21.15.169:6827","nonce":116044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":116044},{"type":"v1","addr":"172.21.15.169:6831","nonce":116044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":116044},{"type":"v1","addr":"172.21.15.169:6829","nonce":116044}]},"public_addr":"172.21.15.169:6823/116044","cluster_addr":"172.21.15.169:6827/116044","heartbeat_back_addr":"172.21.15.169:6831/116044","heartbeat_front_addr":"172.21.15.169:6829/116044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.586781+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.869778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.508594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.860493+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.872267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.746933+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.611520+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.871446+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:24:17.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.133+0000 7f92127ff700 1 -- 172.21.15.92:0/177722435 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f91f8068320 msgr2=0x7f91f806a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:17.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.133+0000 7f92127ff700 1 --2- 172.21.15.92:0/177722435 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f91f8068320 0x7f91f806a7d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f91fc015040 tx=0x7f91fc007000 comp rx=0 tx=0).stop 2023-11-28T11:24:17.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.133+0000 7f92127ff700 1 -- 172.21.15.92:0/177722435 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f920c1548d0 msgr2=0x7f920c0b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:17.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.133+0000 7f92127ff700 1 --2- 172.21.15.92:0/177722435 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f920c1548d0 0x7f920c0b5f40 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f9200011ca0 tx=0x7f920000f520 comp rx=0 tx=0).stop 2023-11-28T11:24:17.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.133+0000 7f92127ff700 1 -- 172.21.15.92:0/177722435 shutdown_connections 2023-11-28T11:24:17.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.133+0000 7f92127ff700 1 --2- 172.21.15.92:0/177722435 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f920c0cc5d0 0x7f920c0cca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:17.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.133+0000 7f92127ff700 1 --2- 172.21.15.92:0/177722435 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91f4003a30 0x7f920c155050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:17.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.133+0000 7f92127ff700 1 --2- 172.21.15.92:0/177722435 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f91f8068320 0x7f91f806a7d0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:17.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.133+0000 7f92127ff700 1 --2- 172.21.15.92:0/177722435 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f920c1548d0 0x7f920c0b5f40 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:17.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.133+0000 7f92127ff700 1 -- 172.21.15.92:0/177722435 >> 172.21.15.92:0/177722435 conn(0x7f920c0719a0 msgr2=0x7f920c0d1160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:17.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.133+0000 7f92127ff700 1 -- 172.21.15.92:0/177722435 shutdown_connections 2023-11-28T11:24:17.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.133+0000 7f92127ff700 1 -- 172.21.15.92:0/177722435 wait complete. 2023-11-28T11:24:17.154 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T11:24:17.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.357+0000 7f93749ff700 1 Processor -- start 2023-11-28T11:24:17.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.357+0000 7f93749ff700 1 -- start start 2023-11-28T11:24:17.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.358+0000 7f93749ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f93700b8510 0x7f937006b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:17.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.358+0000 7f93749ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f93700d0d60 0x7f93700d11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:17.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.358+0000 7f93749ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93700b88e0 con 0x7f93700cd460 2023-11-28T11:24:17.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.358+0000 7f93749ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93700cdbd0 con 0x7f93700d0d60 2023-11-28T11:24:17.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.358+0000 7f93749ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93700d02d0 con 0x7f93700b8510 2023-11-28T11:24:17.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.359+0000 7f936ed9d700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f93700d0d60 0x7f93700d11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:17.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.359+0000 7f936e59c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f93700b8510 0x7f937006b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:17.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.359+0000 7f936dd9b700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f93700cd460 0x7f937006beb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.92:48170/0 (socket says 172.21.15.92:48170) 2023-11-28T11:24:17.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.359+0000 7f936dd9b700 1 -- 172.21.15.92:0/3730866348 learned_addr learned my addr 172.21.15.92:0/3730866348 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:24:17.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.359+0000 7f936d59a700 1 -- 172.21.15.92:0/3730866348 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3205036076 0 0) 0x7f93700b88e0 con 0x7f93700cd460 2023-11-28T11:24:17.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.359+0000 7f936d59a700 1 -- 172.21.15.92:0/3730866348 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93580035b0 con 0x7f93700cd460 2023-11-28T11:24:17.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.359+0000 7f936ed9d700 1 -- 172.21.15.92:0/3730866348 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f93700b8510 msgr2=0x7f937006b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:17.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.359+0000 7f936ed9d700 1 --2- 172.21.15.92:0/3730866348 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f93700b8510 0x7f937006b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:17.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.359+0000 7f936ed9d700 1 -- 172.21.15.92:0/3730866348 >> v1:172.21.15.169:6789/0 conn(0x7f93700cd460 legacy=0x7f937006beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:17.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.359+0000 7f936ed9d700 1 -- 172.21.15.92:0/3730866348 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93700cd8f0 con 0x7f93700d0d60 2023-11-28T11:24:17.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.359+0000 7f936e59c700 1 --2- 172.21.15.92:0/3730866348 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f93700b8510 0x7f937006b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:24:17.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.360+0000 7f936ed9d700 1 --2- 172.21.15.92:0/3730866348 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93700d0d60 0x7f93700d11a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9368009dc0 tx=0x7f936800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0ff1faab6f7301f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:17.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.360+0000 7f936d59a700 1 -- 172.21.15.92:0/3730866348 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936800e070 con 0x7f93700d0d60 2023-11-28T11:24:17.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.360+0000 7f936d59a700 1 -- 172.21.15.92:0/3730866348 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f936800bae0 con 0x7f93700d0d60 2023-11-28T11:24:17.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.360+0000 7f936d59a700 1 -- 172.21.15.92:0/3730866348 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936800be90 con 0x7f93700d0d60 2023-11-28T11:24:17.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.360+0000 7f93749ff700 1 -- 172.21.15.92:0/3730866348 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93700d0d60 msgr2=0x7f93700d11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:17.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.360+0000 7f93749ff700 1 --2- 172.21.15.92:0/3730866348 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93700d0d60 0x7f93700d11a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9368009dc0 tx=0x7f936800b330 comp rx=0 tx=0).stop 2023-11-28T11:24:17.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.361+0000 7f93749ff700 1 -- 172.21.15.92:0/3730866348 shutdown_connections 2023-11-28T11:24:17.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.361+0000 7f93749ff700 1 --2- 172.21.15.92:0/3730866348 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93700d0d60 0x7f93700d11a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:17.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.361+0000 7f93749ff700 1 --2- 172.21.15.92:0/3730866348 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f93700b8510 0x7f937006b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:17.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.361+0000 7f93749ff700 1 -- 172.21.15.92:0/3730866348 >> 172.21.15.92:0/3730866348 conn(0x7f93700719a0 msgr2=0x7f9370071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:17.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.361+0000 7f93749ff700 1 -- 172.21.15.92:0/3730866348 shutdown_connections 2023-11-28T11:24:17.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.361+0000 7f93749ff700 1 -- 172.21.15.92:0/3730866348 wait complete. 2023-11-28T11:24:17.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.362+0000 7f93749ff700 1 Processor -- start 2023-11-28T11:24:17.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.363+0000 7f93749ff700 1 -- start start 2023-11-28T11:24:17.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.363+0000 7f93749ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93700b8510 0x7f93700c7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:17.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.363+0000 7f93749ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93700cd460 0x7f93701f1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:17.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.363+0000 7f93749ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f93700d0d60 0x7f93701f1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:17.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.363+0000 7f93749ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93700acc90 con 0x7f93700cd460 2023-11-28T11:24:17.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.363+0000 7f93749ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93700c83b0 con 0x7f93700b8510 2023-11-28T11:24:17.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.363+0000 7f93749ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93700c80d0 con 0x7f93700d0d60 2023-11-28T11:24:17.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.364+0000 7f936dd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93700cd460 0x7f93701f1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:17.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.364+0000 7f936dd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93700cd460 0x7f93701f1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57682/0 (socket says 172.21.15.92:57682) 2023-11-28T11:24:17.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.364+0000 7f936dd9b700 1 -- 172.21.15.92:0/1889534558 learned_addr learned my addr 172.21.15.92:0/1889534558 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:17.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.364+0000 7f936ed9d700 1 --2- 172.21.15.92:0/1889534558 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f93700d0d60 0x7f93701f1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:17.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.364+0000 7f936e59c700 1 --2- 172.21.15.92:0/1889534558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93700b8510 0x7f93700c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:17.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.364+0000 7f936dd9b700 1 -- 172.21.15.92:0/1889534558 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f93700d0d60 msgr2=0x7f93701f1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:17.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.364+0000 7f936dd9b700 1 --2- 172.21.15.92:0/1889534558 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f93700d0d60 0x7f93701f1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:17.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.364+0000 7f936dd9b700 1 -- 172.21.15.92:0/1889534558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93700b8510 msgr2=0x7f93700c7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:17.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.364+0000 7f936dd9b700 1 --2- 172.21.15.92:0/1889534558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93700b8510 0x7f93700c7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:17.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.364+0000 7f936dd9b700 1 -- 172.21.15.92:0/1889534558 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93600094a0 con 0x7f93700cd460 2023-11-28T11:24:17.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.364+0000 7f936dd9b700 1 --2- 172.21.15.92:0/1889534558 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93700cd460 0x7f93701f1750 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f936400c700 tx=0x7f936400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:17.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.365+0000 7f935effd700 1 -- 172.21.15.92:0/1889534558 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9364005d70 con 0x7f93700cd460 2023-11-28T11:24:17.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.365+0000 7f935effd700 1 -- 172.21.15.92:0/1889534558 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93640070d0 con 0x7f93700cd460 2023-11-28T11:24:17.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.365+0000 7f935effd700 1 -- 172.21.15.92:0/1889534558 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9364007460 con 0x7f93700cd460 2023-11-28T11:24:17.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.365+0000 7f93749ff700 1 -- 172.21.15.92:0/1889534558 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9368009a70 con 0x7f93700cd460 2023-11-28T11:24:17.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.365+0000 7f93749ff700 1 -- 172.21.15.92:0/1889534558 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93700cfff0 con 0x7f93700cd460 2023-11-28T11:24:17.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.366+0000 7f93749ff700 1 -- 172.21.15.92:0/1889534558 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93700cfd10 con 0x7f93700cd460 2023-11-28T11:24:17.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.367+0000 7f935effd700 1 -- 172.21.15.92:0/1889534558 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9364019030 con 0x7f93700cd460 2023-11-28T11:24:17.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.368+0000 7f935effd700 1 --2- 172.21.15.92:0/1889534558 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f93500682f0 0x7f935006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:17.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.368+0000 7f935effd700 1 -- 172.21.15.92:0/1889534558 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f93640851d0 con 0x7f93700cd460 2023-11-28T11:24:17.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.368+0000 7f936e59c700 1 --2- 172.21.15.92:0/1889534558 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f93500682f0 0x7f935006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:17.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.369+0000 7f936e59c700 1 --2- 172.21.15.92:0/1889534558 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f93500682f0 0x7f935006a7a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f937015c850 tx=0x7f9360009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:17.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.372+0000 7f935effd700 1 -- 172.21.15.92:0/1889534558 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f93640517a0 con 0x7f93700cd460 2023-11-28T11:24:17.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:17.603+0000 7f93749ff700 1 -- 172.21.15.92:0/1889534558 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f9370067880 con 0x7f93700cd460 2023-11-28T11:24:18.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.270+0000 7f935effd700 1 -- 172.21.15.92:0/1889534558 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v31) v1 ==== 118+0+0 (secure 0 0 0) 0x7f9364054dc0 con 0x7f93700cd460 2023-11-28T11:24:18.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.273+0000 7f93749ff700 1 -- 172.21.15.92:0/1889534558 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f93500682f0 msgr2=0x7f935006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:18.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.273+0000 7f93749ff700 1 --2- 172.21.15.92:0/1889534558 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f93500682f0 0x7f935006a7a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f937015c850 tx=0x7f9360009210 comp rx=0 tx=0).stop 2023-11-28T11:24:18.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.273+0000 7f93749ff700 1 -- 172.21.15.92:0/1889534558 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93700cd460 msgr2=0x7f93701f1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:18.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.273+0000 7f93749ff700 1 --2- 172.21.15.92:0/1889534558 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93700cd460 0x7f93701f1750 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f936400c700 tx=0x7f936400ca10 comp rx=0 tx=0).stop 2023-11-28T11:24:18.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.273+0000 7f93749ff700 1 -- 172.21.15.92:0/1889534558 shutdown_connections 2023-11-28T11:24:18.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.273+0000 7f93749ff700 1 --2- 172.21.15.92:0/1889534558 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f93700d0d60 0x7f93701f1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:18.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.273+0000 7f93749ff700 1 --2- 172.21.15.92:0/1889534558 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f93700b8510 0x7f93700c7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:18.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.273+0000 7f93749ff700 1 --2- 172.21.15.92:0/1889534558 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f93500682f0 0x7f935006a7a0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:18.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.273+0000 7f93749ff700 1 --2- 172.21.15.92:0/1889534558 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93700cd460 0x7f93701f1750 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:18.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.273+0000 7f93749ff700 1 -- 172.21.15.92:0/1889534558 >> 172.21.15.92:0/1889534558 conn(0x7f93700719a0 msgr2=0x7f93700b71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:18.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.273+0000 7f93749ff700 1 -- 172.21.15.92:0/1889534558 shutdown_connections 2023-11-28T11:24:18.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.273+0000 7f93749ff700 1 -- 172.21.15.92:0/1889534558 wait complete. 2023-11-28T11:24:18.295 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-28T11:24:18.296 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T11:24:18.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.496+0000 7fa4733ff700 1 Processor -- start 2023-11-28T11:24:18.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.496+0000 7fa4733ff700 1 -- start start 2023-11-28T11:24:18.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.497+0000 7fa4733ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa46c0cd460 0x7fa46c06c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:18.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.497+0000 7fa4733ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa46c0d0d60 0x7fa46c0d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:18.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.497+0000 7fa4733ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa46c0b88e0 con 0x7fa46c0b8510 2023-11-28T11:24:18.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.497+0000 7fa4733ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa46c0cdbd0 con 0x7fa46c0cd460 2023-11-28T11:24:18.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.497+0000 7fa4733ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa46c0d02d0 con 0x7fa46c0d0d60 2023-11-28T11:24:18.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.497+0000 7fa47119b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fa46c0b8510 0x7fa46c06b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47524/0 (socket says 172.21.15.92:47524) 2023-11-28T11:24:18.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.497+0000 7fa47099a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa46c0cd460 0x7fa46c06c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:18.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.497+0000 7fa47119b700 1 -- 172.21.15.92:0/995191990 learned_addr learned my addr 172.21.15.92:0/995191990 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:24:18.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.498+0000 7fa47199c700 1 --2- 172.21.15.92:0/995191990 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa46c0d0d60 0x7fa46c0d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:18.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.498+0000 7fa463fff700 1 -- 172.21.15.92:0/995191990 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 859030092 0 0) 0x7fa46c0b88e0 con 0x7fa46c0b8510 2023-11-28T11:24:18.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.498+0000 7fa463fff700 1 -- 172.21.15.92:0/995191990 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4580035b0 con 0x7fa46c0b8510 2023-11-28T11:24:18.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.498+0000 7fa47099a700 1 -- 172.21.15.92:0/995191990 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa46c0d0d60 msgr2=0x7fa46c0d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:18.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.498+0000 7fa47099a700 1 --2- 172.21.15.92:0/995191990 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa46c0d0d60 0x7fa46c0d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:18.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.498+0000 7fa47099a700 1 -- 172.21.15.92:0/995191990 >> v1:172.21.15.92:6789/0 conn(0x7fa46c0b8510 legacy=0x7fa46c06b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:18.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.498+0000 7fa47099a700 1 -- 172.21.15.92:0/995191990 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa46c0cd8f0 con 0x7fa46c0cd460 2023-11-28T11:24:18.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.498+0000 7fa47199c700 1 --2- 172.21.15.92:0/995191990 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa46c0d0d60 0x7fa46c0d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:24:18.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.498+0000 7fa47099a700 1 --2- 172.21.15.92:0/995191990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa46c0cd460 0x7fa46c06c080 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fa45c009870 tx=0x7fa45c009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9d51267000783f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:18.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.498+0000 7fa463fff700 1 -- 172.21.15.92:0/995191990 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45c011830 con 0x7fa46c0cd460 2023-11-28T11:24:18.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.498+0000 7fa463fff700 1 -- 172.21.15.92:0/995191990 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa45c011990 con 0x7fa46c0cd460 2023-11-28T11:24:18.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.498+0000 7fa463fff700 1 -- 172.21.15.92:0/995191990 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45c011d20 con 0x7fa46c0cd460 2023-11-28T11:24:18.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.499+0000 7fa4733ff700 1 -- 172.21.15.92:0/995191990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa46c0cd460 msgr2=0x7fa46c06c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:18.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.499+0000 7fa4733ff700 1 --2- 172.21.15.92:0/995191990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa46c0cd460 0x7fa46c06c080 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fa45c009870 tx=0x7fa45c009b80 comp rx=0 tx=0).stop 2023-11-28T11:24:18.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.499+0000 7fa4733ff700 1 -- 172.21.15.92:0/995191990 shutdown_connections 2023-11-28T11:24:18.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.499+0000 7fa4733ff700 1 --2- 172.21.15.92:0/995191990 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa46c0d0d60 0x7fa46c0d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:18.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.499+0000 7fa4733ff700 1 --2- 172.21.15.92:0/995191990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa46c0cd460 0x7fa46c06c080 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:18.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.499+0000 7fa4733ff700 1 -- 172.21.15.92:0/995191990 >> 172.21.15.92:0/995191990 conn(0x7fa46c0719a0 msgr2=0x7fa46c071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:18.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.499+0000 7fa4733ff700 1 -- 172.21.15.92:0/995191990 shutdown_connections 2023-11-28T11:24:18.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.499+0000 7fa4733ff700 1 -- 172.21.15.92:0/995191990 wait complete. 2023-11-28T11:24:18.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.501+0000 7fa4733ff700 1 Processor -- start 2023-11-28T11:24:18.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.501+0000 7fa4733ff700 1 -- start start 2023-11-28T11:24:18.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.502+0000 7fa4733ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa46c0b8510 0x7fa46c0b1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:18.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.502+0000 7fa4733ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa46c0cd460 0x7fa46c0aa880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:18.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.502+0000 7fa4733ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa46c0d0d60 0x7fa46c0aadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:18.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.502+0000 7fa4733ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46c1e6fa0 con 0x7fa46c0b8510 2023-11-28T11:24:18.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.502+0000 7fa4733ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46c1e2bf0 con 0x7fa46c0cd460 2023-11-28T11:24:18.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.502+0000 7fa4733ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa46c1e6700 con 0x7fa46c0d0d60 2023-11-28T11:24:18.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.502+0000 7fa47119b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa46c0b8510 0x7fa46c0b1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:18.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.502+0000 7fa47119b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa46c0b8510 0x7fa46c0b1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57712/0 (socket says 172.21.15.92:57712) 2023-11-28T11:24:18.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.502+0000 7fa47119b700 1 -- 172.21.15.92:0/3589286514 learned_addr learned my addr 172.21.15.92:0/3589286514 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:18.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.502+0000 7fa47119b700 1 -- 172.21.15.92:0/3589286514 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa46c0d0d60 msgr2=0x7fa46c0aadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:18.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.502+0000 7fa47119b700 1 --2- 172.21.15.92:0/3589286514 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa46c0d0d60 0x7fa46c0aadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:18.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.502+0000 7fa47119b700 1 -- 172.21.15.92:0/3589286514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa46c0cd460 msgr2=0x7fa46c0aa880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:18.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.502+0000 7fa47119b700 1 --2- 172.21.15.92:0/3589286514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa46c0cd460 0x7fa46c0aa880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:18.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.502+0000 7fa47119b700 1 -- 172.21.15.92:0/3589286514 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa45c0094a0 con 0x7fa46c0b8510 2023-11-28T11:24:18.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.503+0000 7fa47119b700 1 --2- 172.21.15.92:0/3589286514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa46c0b8510 0x7fa46c0b1090 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fa46800c700 tx=0x7fa46800cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:18.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.503+0000 7fa461ffb700 1 -- 172.21.15.92:0/3589286514 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa468005d70 con 0x7fa46c0b8510 2023-11-28T11:24:18.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.503+0000 7fa461ffb700 1 -- 172.21.15.92:0/3589286514 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4680070d0 con 0x7fa46c0b8510 2023-11-28T11:24:18.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.503+0000 7fa4733ff700 1 -- 172.21.15.92:0/3589286514 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa46c0cfff0 con 0x7fa46c0b8510 2023-11-28T11:24:18.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.503+0000 7fa461ffb700 1 -- 172.21.15.92:0/3589286514 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa468007460 con 0x7fa46c0b8510 2023-11-28T11:24:18.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.503+0000 7fa4733ff700 1 -- 172.21.15.92:0/3589286514 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa46c0cfd10 con 0x7fa46c0b8510 2023-11-28T11:24:18.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.504+0000 7fa4733ff700 1 -- 172.21.15.92:0/3589286514 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa46c1e2950 con 0x7fa46c0b8510 2023-11-28T11:24:18.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.505+0000 7fa461ffb700 1 -- 172.21.15.92:0/3589286514 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa468019030 con 0x7fa46c0b8510 2023-11-28T11:24:18.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.506+0000 7fa461ffb700 1 --2- 172.21.15.92:0/3589286514 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa4500682f0 0x7fa45006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:18.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.506+0000 7fa461ffb700 1 -- 172.21.15.92:0/3589286514 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fa468085380 con 0x7fa46c0b8510 2023-11-28T11:24:18.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.506+0000 7fa47099a700 1 --2- 172.21.15.92:0/3589286514 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa4500682f0 0x7fa45006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:18.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.507+0000 7fa47099a700 1 --2- 172.21.15.92:0/3589286514 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa4500682f0 0x7fa45006a7a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa46c0b1de0 tx=0x7fa45c010000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:18.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.510+0000 7fa461ffb700 1 -- 172.21.15.92:0/3589286514 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa46c1e2950 con 0x7fa46c0b8510 2023-11-28T11:24:18.692 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:24:18.696 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:24:18.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:18.752+0000 7fa4733ff700 1 -- 172.21.15.92:0/3589286514 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fa46c1e2950 con 0x7fa46c0b8510 2023-11-28T11:24:19.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.543+0000 7fa461ffb700 1 -- 172.21.15.92:0/3589286514 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v32) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa46c1e2950 con 0x7fa46c0b8510 2023-11-28T11:24:19.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.546+0000 7fa4733ff700 1 -- 172.21.15.92:0/3589286514 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa4500682f0 msgr2=0x7fa45006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:19.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.546+0000 7fa4733ff700 1 --2- 172.21.15.92:0/3589286514 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa4500682f0 0x7fa45006a7a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa46c0b1de0 tx=0x7fa45c010000 comp rx=0 tx=0).stop 2023-11-28T11:24:19.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.546+0000 7fa4733ff700 1 -- 172.21.15.92:0/3589286514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa46c0b8510 msgr2=0x7fa46c0b1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:19.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.546+0000 7fa4733ff700 1 --2- 172.21.15.92:0/3589286514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa46c0b8510 0x7fa46c0b1090 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fa46800c700 tx=0x7fa46800cac0 comp rx=0 tx=0).stop 2023-11-28T11:24:19.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.546+0000 7fa4733ff700 1 -- 172.21.15.92:0/3589286514 shutdown_connections 2023-11-28T11:24:19.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.546+0000 7fa4733ff700 1 --2- 172.21.15.92:0/3589286514 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa46c0d0d60 0x7fa46c0aadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:19.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.546+0000 7fa4733ff700 1 --2- 172.21.15.92:0/3589286514 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa4500682f0 0x7fa45006a7a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:19.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.546+0000 7fa4733ff700 1 --2- 172.21.15.92:0/3589286514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa46c0b8510 0x7fa46c0b1090 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:19.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.546+0000 7fa4733ff700 1 --2- 172.21.15.92:0/3589286514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa46c0cd460 0x7fa46c0aa880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:19.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.546+0000 7fa4733ff700 1 -- 172.21.15.92:0/3589286514 >> 172.21.15.92:0/3589286514 conn(0x7fa46c0719a0 msgr2=0x7fa46c0cbef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:19.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.546+0000 7fa4733ff700 1 -- 172.21.15.92:0/3589286514 shutdown_connections 2023-11-28T11:24:19.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.546+0000 7fa4733ff700 1 -- 172.21.15.92:0/3589286514 wait complete. 2023-11-28T11:24:19.569 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T11:24:19.569 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:24:19.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.767+0000 7fb4d21ff700 1 Processor -- start 2023-11-28T11:24:19.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.768+0000 7fb4d21ff700 1 -- start start 2023-11-28T11:24:19.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.769+0000 7fb4d21ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4cc0b8510 0x7fb4cc06b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:19.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.769+0000 7fb4d21ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb4cc0cd460 0x7fb4cc06beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:19.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.769+0000 7fb4d21ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4cc0b88e0 con 0x7fb4cc06c3f0 2023-11-28T11:24:19.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.769+0000 7fb4d21ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4cc0cdbd0 con 0x7fb4cc0cd460 2023-11-28T11:24:19.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.769+0000 7fb4d21ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc06c830 con 0x7fb4cc0b8510 2023-11-28T11:24:19.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.769+0000 7fb4cb7fe700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4cc0b8510 0x7fb4cc06b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:19.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.769+0000 7fb4c3fff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb4cc0cd460 0x7fb4cc06beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:19.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.769+0000 7fb4cb7fe700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4cc0b8510 0x7fb4cc06b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.92:44572/0 (socket says 172.21.15.92:44572) 2023-11-28T11:24:19.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.769+0000 7fb4cb7fe700 1 -- 172.21.15.92:0/2485240569 learned_addr learned my addr 172.21.15.92:0/2485240569 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:19.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.770+0000 7fb4cb7fe700 1 -- 172.21.15.92:0/2485240569 >> v2:172.21.15.169:3300/0 conn(0x7fb4cc0cd460 msgr2=0x7fb4cc06beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:19.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.770+0000 7fb4cb7fe700 1 --2- 172.21.15.92:0/2485240569 >> v2:172.21.15.169:3300/0 conn(0x7fb4cc0cd460 0x7fb4cc06beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:19.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.770+0000 7fb4cb7fe700 1 -- 172.21.15.92:0/2485240569 >> v1:172.21.15.169:6789/0 conn(0x7fb4cc06c3f0 legacy=0x7fb4cc15cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:19.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.770+0000 7fb4cb7fe700 1 -- 172.21.15.92:0/2485240569 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4cc0cd8f0 con 0x7fb4cc0b8510 2023-11-28T11:24:19.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.770+0000 7fb4cb7fe700 1 --2- 172.21.15.92:0/2485240569 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4cc0b8510 0x7fb4cc06b970 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc0098f0 tx=0x7fb4bc009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc957d880276b689 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:19.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.770+0000 7fb4caffd700 1 -- 172.21.15.92:0/2485240569 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc011830 con 0x7fb4cc0b8510 2023-11-28T11:24:19.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.770+0000 7fb4caffd700 1 -- 172.21.15.92:0/2485240569 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4bc011990 con 0x7fb4cc0b8510 2023-11-28T11:24:19.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.770+0000 7fb4caffd700 1 -- 172.21.15.92:0/2485240569 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc011d20 con 0x7fb4cc0b8510 2023-11-28T11:24:19.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.771+0000 7fb4d21ff700 1 -- 172.21.15.92:0/2485240569 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4cc0b8510 msgr2=0x7fb4cc06b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:19.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.771+0000 7fb4d21ff700 1 --2- 172.21.15.92:0/2485240569 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4cc0b8510 0x7fb4cc06b970 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc0098f0 tx=0x7fb4bc009c00 comp rx=0 tx=0).stop 2023-11-28T11:24:19.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.771+0000 7fb4d21ff700 1 -- 172.21.15.92:0/2485240569 shutdown_connections 2023-11-28T11:24:19.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.771+0000 7fb4d21ff700 1 --2- 172.21.15.92:0/2485240569 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4cc0b8510 0x7fb4cc06b970 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:19.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.771+0000 7fb4d21ff700 1 --2- 172.21.15.92:0/2485240569 >> v2:172.21.15.169:3300/0 conn(0x7fb4cc0cd460 0x7fb4cc06beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:19.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.771+0000 7fb4d21ff700 1 -- 172.21.15.92:0/2485240569 >> 172.21.15.92:0/2485240569 conn(0x7fb4cc0719a0 msgr2=0x7fb4cc071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:19.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.771+0000 7fb4d21ff700 1 -- 172.21.15.92:0/2485240569 shutdown_connections 2023-11-28T11:24:19.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.771+0000 7fb4d21ff700 1 -- 172.21.15.92:0/2485240569 wait complete. 2023-11-28T11:24:19.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.773+0000 7fb4d21ff700 1 Processor -- start 2023-11-28T11:24:19.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.773+0000 7fb4d21ff700 1 -- start start 2023-11-28T11:24:19.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.774+0000 7fb4d21ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4cc06c3f0 0x7fb4cc0c9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:19.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.774+0000 7fb4d21ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4cc0b8510 0x7fb4cc0cb740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:19.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.774+0000 7fb4d21ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4cc0cd460 0x7fb4cc0c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:19.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.774+0000 7fb4d21ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc0cb2b0 con 0x7fb4cc0b8510 2023-11-28T11:24:19.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.774+0000 7fb4d21ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc0cbc80 con 0x7fb4cc06c3f0 2023-11-28T11:24:19.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.774+0000 7fb4d21ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc0ca420 con 0x7fb4cc0cd460 2023-11-28T11:24:19.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.774+0000 7fb4c3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4cc0b8510 0x7fb4cc0cb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:19.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.774+0000 7fb4c3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4cc0b8510 0x7fb4cc0cb740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55158/0 (socket says 172.21.15.92:55158) 2023-11-28T11:24:19.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.774+0000 7fb4c3fff700 1 -- 172.21.15.92:0/1298361619 learned_addr learned my addr 172.21.15.92:0/1298361619 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:19.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.774+0000 7fb4cb7fe700 1 --2- 172.21.15.92:0/1298361619 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4cc06c3f0 0x7fb4cc0c9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:19.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.774+0000 7fb4cbfff700 1 --2- 172.21.15.92:0/1298361619 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4cc0cd460 0x7fb4cc0c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:19.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.774+0000 7fb4c3fff700 1 -- 172.21.15.92:0/1298361619 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4cc0cd460 msgr2=0x7fb4cc0c9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:19.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.774+0000 7fb4c3fff700 1 --2- 172.21.15.92:0/1298361619 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4cc0cd460 0x7fb4cc0c9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:19.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.774+0000 7fb4c3fff700 1 -- 172.21.15.92:0/1298361619 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4cc06c3f0 msgr2=0x7fb4cc0c9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:19.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.774+0000 7fb4c3fff700 1 --2- 172.21.15.92:0/1298361619 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4cc06c3f0 0x7fb4cc0c9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:19.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.774+0000 7fb4c3fff700 1 -- 172.21.15.92:0/1298361619 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4bc009550 con 0x7fb4cc0b8510 2023-11-28T11:24:19.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.775+0000 7fb4c3fff700 1 --2- 172.21.15.92:0/1298361619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4cc0b8510 0x7fb4cc0cb740 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fb4b400e8a0 tx=0x7fb4b400ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:19.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.775+0000 7fb4c8ff9700 1 -- 172.21.15.92:0/1298361619 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b4009590 con 0x7fb4cc0b8510 2023-11-28T11:24:19.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.775+0000 7fb4c8ff9700 1 -- 172.21.15.92:0/1298361619 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4b4013680 con 0x7fb4cc0b8510 2023-11-28T11:24:19.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.775+0000 7fb4d21ff700 1 -- 172.21.15.92:0/1298361619 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4cc0cfff0 con 0x7fb4cc0b8510 2023-11-28T11:24:19.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.775+0000 7fb4c8ff9700 1 -- 172.21.15.92:0/1298361619 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b40139f0 con 0x7fb4cc0b8510 2023-11-28T11:24:19.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.775+0000 7fb4d21ff700 1 -- 172.21.15.92:0/1298361619 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4cc0cfd10 con 0x7fb4cc0b8510 2023-11-28T11:24:19.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.776+0000 7fb4d21ff700 1 -- 172.21.15.92:0/1298361619 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4cc0cb460 con 0x7fb4cc0b8510 2023-11-28T11:24:19.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.777+0000 7fb4c8ff9700 1 -- 172.21.15.92:0/1298361619 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb4b4019030 con 0x7fb4cc0b8510 2023-11-28T11:24:19.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.778+0000 7fb4c8ff9700 1 --2- 172.21.15.92:0/1298361619 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb4ac0682f0 0x7fb4ac06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:19.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.778+0000 7fb4c8ff9700 1 -- 172.21.15.92:0/1298361619 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fb4b4085bc0 con 0x7fb4cc0b8510 2023-11-28T11:24:19.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.778+0000 7fb4cb7fe700 1 --2- 172.21.15.92:0/1298361619 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb4ac0682f0 0x7fb4ac06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:19.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.779+0000 7fb4cb7fe700 1 --2- 172.21.15.92:0/1298361619 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb4ac0682f0 0x7fb4ac06a7a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc000c00 tx=0x7fb4bc008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:19.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:19.782+0000 7fb4c8ff9700 1 -- 172.21.15.92:0/1298361619 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb4cc0cb460 con 0x7fb4cc0b8510 2023-11-28T11:24:20.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.015+0000 7fb4d21ff700 1 -- 172.21.15.92:0/1298361619 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4cc0cb460 con 0x7fb4cc0b8510 2023-11-28T11:24:20.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.016+0000 7fb4c8ff9700 1 -- 172.21.15.92:0/1298361619 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4656 (secure 0 0 0) 0x7fb4cc0cb460 con 0x7fb4cc0b8510 2023-11-28T11:24:20.113 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:24:20.113 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5525,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2067956596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2067956596},{"type":"v1","addr":"172.21.15.169:6835","nonce":2067956596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5971,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2367049160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2367049160},{"type":"v1","addr":"172.21.15.169:6834","nonce":2367049160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5977,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1424858585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":1424858585},{"type":"v1","addr":"172.21.15.92:6837","nonce":1424858585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:24:16.257826+0000","modified":"2023-11-28T11:24:19.539818+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6213},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6213":{"gid":6213,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6836/3634521248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3634521248},{"type":"v1","addr":"172.21.15.92:6836","nonce":3634521248}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T11:24:20.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.018+0000 7fb4d21ff700 1 -- 172.21.15.92:0/1298361619 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb4ac0682f0 msgr2=0x7fb4ac06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:20.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.018+0000 7fb4d21ff700 1 --2- 172.21.15.92:0/1298361619 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb4ac0682f0 0x7fb4ac06a7a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc000c00 tx=0x7fb4bc008030 comp rx=0 tx=0).stop 2023-11-28T11:24:20.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.019+0000 7fb4d21ff700 1 -- 172.21.15.92:0/1298361619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4cc0b8510 msgr2=0x7fb4cc0cb740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:20.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.019+0000 7fb4d21ff700 1 --2- 172.21.15.92:0/1298361619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4cc0b8510 0x7fb4cc0cb740 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fb4b400e8a0 tx=0x7fb4b400ebb0 comp rx=0 tx=0).stop 2023-11-28T11:24:20.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.019+0000 7fb4d21ff700 1 -- 172.21.15.92:0/1298361619 shutdown_connections 2023-11-28T11:24:20.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.019+0000 7fb4d21ff700 1 --2- 172.21.15.92:0/1298361619 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb4cc0cd460 0x7fb4cc0c9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:20.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.019+0000 7fb4d21ff700 1 --2- 172.21.15.92:0/1298361619 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb4cc06c3f0 0x7fb4cc0c9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:20.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.019+0000 7fb4d21ff700 1 --2- 172.21.15.92:0/1298361619 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb4ac0682f0 0x7fb4ac06a7a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:20.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.019+0000 7fb4d21ff700 1 --2- 172.21.15.92:0/1298361619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4cc0b8510 0x7fb4cc0cb740 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:20.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.019+0000 7fb4d21ff700 1 -- 172.21.15.92:0/1298361619 >> 172.21.15.92:0/1298361619 conn(0x7fb4cc0719a0 msgr2=0x7fb4cc0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:20.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.019+0000 7fb4d21ff700 1 -- 172.21.15.92:0/1298361619 shutdown_connections 2023-11-28T11:24:20.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.019+0000 7fb4d21ff700 1 -- 172.21.15.92:0/1298361619 wait complete. 2023-11-28T11:24:20.117 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 32 2023-11-28T11:24:20.118 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:24:20.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.316+0000 7f94173ff700 1 Processor -- start 2023-11-28T11:24:20.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.316+0000 7f94173ff700 1 -- start start 2023-11-28T11:24:20.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.317+0000 7f94173ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94100cd460 0x7f941006c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:20.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.317+0000 7f94173ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f94100d0d60 0x7f94100d11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:20.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.317+0000 7f94173ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94100b88e0 con 0x7f94100b8510 2023-11-28T11:24:20.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.317+0000 7f94173ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94100cdbd0 con 0x7f94100d0d60 2023-11-28T11:24:20.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.317+0000 7f94173ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94100d02d0 con 0x7f94100cd460 2023-11-28T11:24:20.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.317+0000 7f941519b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f94100b8510 0x7f941006b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35714/0 (socket says 172.21.15.92:35714) 2023-11-28T11:24:20.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.317+0000 7f941519b700 1 -- 172.21.15.92:0/737412756 learned_addr learned my addr 172.21.15.92:0/737412756 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:24:20.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.318+0000 7f941499a700 1 --2- 172.21.15.92:0/737412756 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94100cd460 0x7f941006c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:20.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.318+0000 7f941599c700 1 --2- 172.21.15.92:0/737412756 >> v2:172.21.15.169:3300/0 conn(0x7f94100d0d60 0x7f94100d11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:20.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.318+0000 7f9407fff700 1 -- 172.21.15.92:0/737412756 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3608041337 0 0) 0x7f94100b88e0 con 0x7f94100b8510 2023-11-28T11:24:20.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.318+0000 7f9407fff700 1 -- 172.21.15.92:0/737412756 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93fc0035b0 con 0x7f94100b8510 2023-11-28T11:24:20.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.318+0000 7f941499a700 1 -- 172.21.15.92:0/737412756 >> v2:172.21.15.169:3300/0 conn(0x7f94100d0d60 msgr2=0x7f94100d11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:20.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.318+0000 7f941499a700 1 --2- 172.21.15.92:0/737412756 >> v2:172.21.15.169:3300/0 conn(0x7f94100d0d60 0x7f94100d11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:20.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.318+0000 7f941499a700 1 -- 172.21.15.92:0/737412756 >> v1:172.21.15.92:6789/0 conn(0x7f94100b8510 legacy=0x7f941006b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:20.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.318+0000 7f941499a700 1 -- 172.21.15.92:0/737412756 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94100cd8f0 con 0x7f94100cd460 2023-11-28T11:24:20.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.318+0000 7f941499a700 1 --2- 172.21.15.92:0/737412756 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94100cd460 0x7f941006c080 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9400009840 tx=0x7f9400009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e304458fced844fa server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:20.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.319+0000 7f9407fff700 1 -- 172.21.15.92:0/737412756 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9400011830 con 0x7f94100cd460 2023-11-28T11:24:20.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.319+0000 7f9407fff700 1 -- 172.21.15.92:0/737412756 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9400011990 con 0x7f94100cd460 2023-11-28T11:24:20.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.319+0000 7f9407fff700 1 -- 172.21.15.92:0/737412756 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9400011d20 con 0x7f94100cd460 2023-11-28T11:24:20.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.319+0000 7f94173ff700 1 -- 172.21.15.92:0/737412756 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94100cd460 msgr2=0x7f941006c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:20.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.319+0000 7f94173ff700 1 --2- 172.21.15.92:0/737412756 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94100cd460 0x7f941006c080 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9400009840 tx=0x7f9400009b50 comp rx=0 tx=0).stop 2023-11-28T11:24:20.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.319+0000 7f94173ff700 1 -- 172.21.15.92:0/737412756 shutdown_connections 2023-11-28T11:24:20.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.319+0000 7f94173ff700 1 --2- 172.21.15.92:0/737412756 >> v2:172.21.15.169:3300/0 conn(0x7f94100d0d60 0x7f94100d11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:20.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.319+0000 7f94173ff700 1 --2- 172.21.15.92:0/737412756 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94100cd460 0x7f941006c080 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:20.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.319+0000 7f94173ff700 1 -- 172.21.15.92:0/737412756 >> 172.21.15.92:0/737412756 conn(0x7f94100719a0 msgr2=0x7f9410071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:20.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.319+0000 7f94173ff700 1 -- 172.21.15.92:0/737412756 shutdown_connections 2023-11-28T11:24:20.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.319+0000 7f94173ff700 1 -- 172.21.15.92:0/737412756 wait complete. 2023-11-28T11:24:20.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.321+0000 7f94173ff700 1 Processor -- start 2023-11-28T11:24:20.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.321+0000 7f94173ff700 1 -- start start 2023-11-28T11:24:20.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.322+0000 7f94173ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94100b8510 0x7f9410157460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:20.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.322+0000 7f94173ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94100cd460 0x7f94101595c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:20.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.322+0000 7f94173ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94100d0d60 0x7f94100b36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:20.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.322+0000 7f94173ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94101e3ca0 con 0x7f94100d0d60 2023-11-28T11:24:20.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.322+0000 7f94173ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94101e3f80 con 0x7f94100cd460 2023-11-28T11:24:20.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.322+0000 7f94173ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94101e4260 con 0x7f94100b8510 2023-11-28T11:24:20.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.322+0000 7f941599c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94100d0d60 0x7f94100b36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:20.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.322+0000 7f941599c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94100d0d60 0x7f94100b36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55160/0 (socket says 172.21.15.92:55160) 2023-11-28T11:24:20.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.322+0000 7f941599c700 1 -- 172.21.15.92:0/1094428343 learned_addr learned my addr 172.21.15.92:0/1094428343 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:20.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.323+0000 7f941499a700 1 --2- 172.21.15.92:0/1094428343 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94100cd460 0x7f94101595c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:20.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.323+0000 7f941519b700 1 --2- 172.21.15.92:0/1094428343 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94100b8510 0x7f9410157460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:20.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.323+0000 7f941599c700 1 -- 172.21.15.92:0/1094428343 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94100b8510 msgr2=0x7f9410157460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:20.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.323+0000 7f941599c700 1 --2- 172.21.15.92:0/1094428343 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94100b8510 0x7f9410157460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:20.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.323+0000 7f941599c700 1 -- 172.21.15.92:0/1094428343 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94100cd460 msgr2=0x7f94101595c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:20.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.323+0000 7f941599c700 1 --2- 172.21.15.92:0/1094428343 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94100cd460 0x7f94101595c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:20.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.323+0000 7f941599c700 1 -- 172.21.15.92:0/1094428343 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94000094a0 con 0x7f94100d0d60 2023-11-28T11:24:20.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.323+0000 7f941599c700 1 --2- 172.21.15.92:0/1094428343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94100d0d60 0x7f94100b36b0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f940800efc0 tx=0x7f940800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:20.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.323+0000 7f9405ffb700 1 -- 172.21.15.92:0/1094428343 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9408009ba0 con 0x7f94100d0d60 2023-11-28T11:24:20.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.323+0000 7f9405ffb700 1 -- 172.21.15.92:0/1094428343 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f940800f040 con 0x7f94100d0d60 2023-11-28T11:24:20.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.323+0000 7f94173ff700 1 -- 172.21.15.92:0/1094428343 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94100cfff0 con 0x7f94100d0d60 2023-11-28T11:24:20.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.323+0000 7f9405ffb700 1 -- 172.21.15.92:0/1094428343 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9408014480 con 0x7f94100d0d60 2023-11-28T11:24:20.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.324+0000 7f94173ff700 1 -- 172.21.15.92:0/1094428343 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94100cfd10 con 0x7f94100d0d60 2023-11-28T11:24:20.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.324+0000 7f94173ff700 1 -- 172.21.15.92:0/1094428343 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94100b47b0 con 0x7f94100d0d60 2023-11-28T11:24:20.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.326+0000 7f9405ffb700 1 -- 172.21.15.92:0/1094428343 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f94080145e0 con 0x7f94100d0d60 2023-11-28T11:24:20.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.326+0000 7f9405ffb700 1 --2- 172.21.15.92:0/1094428343 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f93f40682f0 0x7f93f406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:20.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.326+0000 7f9405ffb700 1 -- 172.21.15.92:0/1094428343 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f94080869a0 con 0x7f94100d0d60 2023-11-28T11:24:20.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.327+0000 7f941519b700 1 --2- 172.21.15.92:0/1094428343 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f93f40682f0 0x7f93f406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:20.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.327+0000 7f941519b700 1 --2- 172.21.15.92:0/1094428343 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f93f40682f0 0x7f93f406a7a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f940c0011e0 tx=0x7f940c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:20.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.331+0000 7f9405ffb700 1 -- 172.21.15.92:0/1094428343 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9408052f40 con 0x7f94100d0d60 2023-11-28T11:24:20.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.521+0000 7f94173ff700 1 -- 172.21.15.92:0/1094428343 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9410067880 con 0x7f94100d0d60 2023-11-28T11:24:20.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.523+0000 7f9405ffb700 1 -- 172.21.15.92:0/1094428343 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+18183 (secure 0 0 0) 0x7f9408018020 con 0x7f94100d0d60 2023-11-28T11:24:20.524 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:24:20.524 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":68,"fsid":"68956e0c-c16f-47cd-a220-f98364b0831d","created":"2023-11-28T11:20:28.697573+0000","modified":"2023-11-28T11:24:17.260290+0000","last_up_change":"2023-11-28T11:20:41.257011+0000","last_in_change":"2023-11-28T11:20:32.249767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:20:41.925237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:20:48.682497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:24:13.972356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-28T11:24:15.237693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6abe483f-0e03-4a26-b9cf-62041291747e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":115823},{"type":"v1","addr":"172.21.15.92:6809","nonce":115823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":115823},{"type":"v1","addr":"172.21.15.92:6813","nonce":115823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":115823},{"type":"v1","addr":"172.21.15.92:6821","nonce":115823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":115823},{"type":"v1","addr":"172.21.15.92:6817","nonce":115823}]},"public_addr":"172.21.15.92:6809/115823","cluster_addr":"172.21.15.92:6813/115823","heartbeat_back_addr":"172.21.15.92:6821/115823","heartbeat_front_addr":"172.21.15.92:6817/115823","state":["exists","up"]},{"osd":1,"uuid":"13c1de73-1a8c-4d7e-8a28-9dae52dfe089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":115824},{"type":"v1","addr":"172.21.15.92:6812","nonce":115824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":115824},{"type":"v1","addr":"172.21.15.92:6816","nonce":115824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":115824},{"type":"v1","addr":"172.21.15.92:6823","nonce":115824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":115824},{"type":"v1","addr":"172.21.15.92:6820","nonce":115824}]},"public_addr":"172.21.15.92:6812/115824","cluster_addr":"172.21.15.92:6816/115824","heartbeat_back_addr":"172.21.15.92:6823/115824","heartbeat_front_addr":"172.21.15.92:6820/115824","state":["exists","up"]},{"osd":2,"uuid":"25788a55-112f-4f00-810d-2bc37cf76778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":115821},{"type":"v1","addr":"172.21.15.92:6801","nonce":115821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":115821},{"type":"v1","addr":"172.21.15.92:6803","nonce":115821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":115821},{"type":"v1","addr":"172.21.15.92:6807","nonce":115821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":115821},{"type":"v1","addr":"172.21.15.92:6805","nonce":115821}]},"public_addr":"172.21.15.92:6801/115821","cluster_addr":"172.21.15.92:6803/115821","heartbeat_back_addr":"172.21.15.92:6807/115821","heartbeat_front_addr":"172.21.15.92:6805/115821","state":["exists","up"]},{"osd":3,"uuid":"adf54e24-33a9-41fe-9ce3-98a5ab6c158e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":115825},{"type":"v1","addr":"172.21.15.92:6825","nonce":115825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":115825},{"type":"v1","addr":"172.21.15.92:6827","nonce":115825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":115825},{"type":"v1","addr":"172.21.15.92:6831","nonce":115825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":115825},{"type":"v1","addr":"172.21.15.92:6829","nonce":115825}]},"public_addr":"172.21.15.92:6825/115825","cluster_addr":"172.21.15.92:6827/115825","heartbeat_back_addr":"172.21.15.92:6831/115825","heartbeat_front_addr":"172.21.15.92:6829/115825","state":["exists","up"]},{"osd":4,"uuid":"ccb9b747-f399-44ea-9a18-9db3152227e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":116035},{"type":"v1","addr":"172.21.15.169:6817","nonce":116035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":116035},{"type":"v1","addr":"172.21.15.169:6819","nonce":116035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":116035},{"type":"v1","addr":"172.21.15.169:6826","nonce":116035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":116035},{"type":"v1","addr":"172.21.15.169:6822","nonce":116035}]},"public_addr":"172.21.15.169:6817/116035","cluster_addr":"172.21.15.169:6819/116035","heartbeat_back_addr":"172.21.15.169:6826/116035","heartbeat_front_addr":"172.21.15.169:6822/116035","state":["exists","up"]},{"osd":5,"uuid":"308a1e63-75aa-4d7d-8a7c-cecc24714fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":116033},{"type":"v1","addr":"172.21.15.169:6801","nonce":116033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":116033},{"type":"v1","addr":"172.21.15.169:6803","nonce":116033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":116033},{"type":"v1","addr":"172.21.15.169:6807","nonce":116033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":116033},{"type":"v1","addr":"172.21.15.169:6805","nonce":116033}]},"public_addr":"172.21.15.169:6801/116033","cluster_addr":"172.21.15.169:6803/116033","heartbeat_back_addr":"172.21.15.169:6807/116033","heartbeat_front_addr":"172.21.15.169:6805/116033","state":["exists","up"]},{"osd":6,"uuid":"9c459b7b-b821-4b4b-b91e-6aaae271457c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":116034},{"type":"v1","addr":"172.21.15.169:6809","nonce":116034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":116034},{"type":"v1","addr":"172.21.15.169:6811","nonce":116034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":116034},{"type":"v1","addr":"172.21.15.169:6815","nonce":116034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":116034},{"type":"v1","addr":"172.21.15.169:6813","nonce":116034}]},"public_addr":"172.21.15.169:6809/116034","cluster_addr":"172.21.15.169:6811/116034","heartbeat_back_addr":"172.21.15.169:6815/116034","heartbeat_front_addr":"172.21.15.169:6813/116034","state":["exists","up"]},{"osd":7,"uuid":"1ca9aa4a-755d-4e28-a937-4d8dccab4df3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":116044},{"type":"v1","addr":"172.21.15.169:6823","nonce":116044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6825","nonce":116044},{"type":"v1","addr":"172.21.15.169:6827","nonce":116044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":116044},{"type":"v1","addr":"172.21.15.169:6831","nonce":116044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":116044},{"type":"v1","addr":"172.21.15.169:6829","nonce":116044}]},"public_addr":"172.21.15.169:6823/116044","cluster_addr":"172.21.15.169:6827/116044","heartbeat_back_addr":"172.21.15.169:6831/116044","heartbeat_front_addr":"172.21.15.169:6829/116044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.586781+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.869778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.508594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.860493+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.872267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.746933+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.611520+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.871446+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:24:20.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.525+0000 7f94173ff700 1 -- 172.21.15.92:0/1094428343 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f93f40682f0 msgr2=0x7f93f406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:20.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.525+0000 7f94173ff700 1 --2- 172.21.15.92:0/1094428343 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f93f40682f0 0x7f93f406a7a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f940c0011e0 tx=0x7f940c009040 comp rx=0 tx=0).stop 2023-11-28T11:24:20.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.525+0000 7f94173ff700 1 -- 172.21.15.92:0/1094428343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94100d0d60 msgr2=0x7f94100b36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:20.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.525+0000 7f94173ff700 1 --2- 172.21.15.92:0/1094428343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94100d0d60 0x7f94100b36b0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f940800efc0 tx=0x7f940800c3a0 comp rx=0 tx=0).stop 2023-11-28T11:24:20.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.525+0000 7f94173ff700 1 -- 172.21.15.92:0/1094428343 shutdown_connections 2023-11-28T11:24:20.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.525+0000 7f94173ff700 1 --2- 172.21.15.92:0/1094428343 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f93f40682f0 0x7f93f406a7a0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:20.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.525+0000 7f94173ff700 1 --2- 172.21.15.92:0/1094428343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94100d0d60 0x7f94100b36b0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:20.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.525+0000 7f94173ff700 1 --2- 172.21.15.92:0/1094428343 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94100b8510 0x7f9410157460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:20.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.525+0000 7f94173ff700 1 --2- 172.21.15.92:0/1094428343 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f94100cd460 0x7f94101595c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:20.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.525+0000 7f94173ff700 1 -- 172.21.15.92:0/1094428343 >> 172.21.15.92:0/1094428343 conn(0x7f94100719a0 msgr2=0x7f941006b970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:20.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.525+0000 7f94173ff700 1 -- 172.21.15.92:0/1094428343 shutdown_connections 2023-11-28T11:24:20.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.526+0000 7f94173ff700 1 -- 172.21.15.92:0/1094428343 wait complete. 2023-11-28T11:24:20.545 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T11:24:20.545 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:24:20.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.741+0000 7f5d1abff700 1 Processor -- start 2023-11-28T11:24:20.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.741+0000 7f5d1abff700 1 -- start start 2023-11-28T11:24:20.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.742+0000 7f5d1abff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5d140b8510 0x7f5d1406b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:20.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.742+0000 7f5d1abff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5d140cd460 0x7f5d1406beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:20.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.742+0000 7f5d1abff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d140b88e0 con 0x7f5d1406c3f0 2023-11-28T11:24:20.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.742+0000 7f5d1abff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d140cdbd0 con 0x7f5d140cd460 2023-11-28T11:24:20.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.742+0000 7f5d1abff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d140d02d0 con 0x7f5d140b8510 2023-11-28T11:24:20.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.743+0000 7f5d13fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5d140cd460 0x7f5d1406beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:20.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.743+0000 7f5d13fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5d140cd460 0x7f5d1406beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55174/0 (socket says 172.21.15.92:55174) 2023-11-28T11:24:20.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.743+0000 7f5d13fff700 1 -- 172.21.15.92:0/1155410111 learned_addr learned my addr 172.21.15.92:0/1155410111 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:20.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.743+0000 7f5d1899b700 1 --2- 172.21.15.92:0/1155410111 >> v2:172.21.15.169:3300/0 conn(0x7f5d140b8510 0x7f5d1406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:20.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.743+0000 7f5d13fff700 1 -- 172.21.15.92:0/1155410111 >> v2:172.21.15.169:3300/0 conn(0x7f5d140b8510 msgr2=0x7f5d1406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:20.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.743+0000 7f5d13fff700 1 --2- 172.21.15.92:0/1155410111 >> v2:172.21.15.169:3300/0 conn(0x7f5d140b8510 0x7f5d1406b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:20.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.743+0000 7f5d13fff700 1 -- 172.21.15.92:0/1155410111 >> v1:172.21.15.169:6789/0 conn(0x7f5d1406c3f0 legacy=0x7f5d1415cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:20.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.743+0000 7f5d13fff700 1 -- 172.21.15.92:0/1155410111 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d140cd8f0 con 0x7f5d140cd460 2023-11-28T11:24:20.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.744+0000 7f5d13fff700 1 --2- 172.21.15.92:0/1155410111 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d140cd460 0x7f5d1406beb0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f5d00009920 tx=0x7f5d00009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71296f23e9db48ba server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:20.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.744+0000 7f5d137fe700 1 -- 172.21.15.92:0/1155410111 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d00011830 con 0x7f5d140cd460 2023-11-28T11:24:20.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.744+0000 7f5d137fe700 1 -- 172.21.15.92:0/1155410111 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d00011990 con 0x7f5d140cd460 2023-11-28T11:24:20.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.744+0000 7f5d137fe700 1 -- 172.21.15.92:0/1155410111 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d00011d20 con 0x7f5d140cd460 2023-11-28T11:24:20.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.744+0000 7f5d1abff700 1 -- 172.21.15.92:0/1155410111 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d140cd460 msgr2=0x7f5d1406beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:20.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.744+0000 7f5d1abff700 1 --2- 172.21.15.92:0/1155410111 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d140cd460 0x7f5d1406beb0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f5d00009920 tx=0x7f5d00009c30 comp rx=0 tx=0).stop 2023-11-28T11:24:20.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.744+0000 7f5d1abff700 1 -- 172.21.15.92:0/1155410111 shutdown_connections 2023-11-28T11:24:20.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.744+0000 7f5d1abff700 1 --2- 172.21.15.92:0/1155410111 >> v2:172.21.15.169:3300/0 conn(0x7f5d140b8510 0x7f5d1406b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:20.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.744+0000 7f5d1abff700 1 --2- 172.21.15.92:0/1155410111 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d140cd460 0x7f5d1406beb0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:20.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.744+0000 7f5d1abff700 1 -- 172.21.15.92:0/1155410111 >> 172.21.15.92:0/1155410111 conn(0x7f5d140719a0 msgr2=0x7f5d14071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:20.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.744+0000 7f5d1abff700 1 -- 172.21.15.92:0/1155410111 shutdown_connections 2023-11-28T11:24:20.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.744+0000 7f5d1abff700 1 -- 172.21.15.92:0/1155410111 wait complete. 2023-11-28T11:24:20.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.746+0000 7f5d1abff700 1 Processor -- start 2023-11-28T11:24:20.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.746+0000 7f5d1abff700 1 -- start start 2023-11-28T11:24:20.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.747+0000 7f5d1abff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d1406c3f0 0x7f5d140cb470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:20.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.747+0000 7f5d1abff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d140b8510 0x7f5d141e2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:20.748 INFO:tasks.ceph.mds.a.smithi092.stderr:2023-11-28T11:24:20.747+0000 7f4f55bf1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T11:24:20.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.747+0000 7f5d1abff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d140cd460 0x7f5d140c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:20.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.747+0000 7f5d1abff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d140c9490 con 0x7f5d140b8510 2023-11-28T11:24:20.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.747+0000 7f5d1abff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d140cbb50 con 0x7f5d1406c3f0 2023-11-28T11:24:20.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.747+0000 7f5d1abff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d140cbe30 con 0x7f5d140cd460 2023-11-28T11:24:20.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.748+0000 7f5d13fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d140b8510 0x7f5d141e2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:20.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.748+0000 7f5d13fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d140b8510 0x7f5d141e2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55188/0 (socket says 172.21.15.92:55188) 2023-11-28T11:24:20.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.748+0000 7f5d13fff700 1 -- 172.21.15.92:0/1885266370 learned_addr learned my addr 172.21.15.92:0/1885266370 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:20.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.748+0000 7f5d1899b700 1 --2- 172.21.15.92:0/1885266370 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d1406c3f0 0x7f5d140cb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:20.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.748+0000 7f5d1919c700 1 --2- 172.21.15.92:0/1885266370 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d140cd460 0x7f5d140c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:20.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.748+0000 7f5d13fff700 1 -- 172.21.15.92:0/1885266370 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d140cd460 msgr2=0x7f5d140c8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:20.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.748+0000 7f5d13fff700 1 --2- 172.21.15.92:0/1885266370 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d140cd460 0x7f5d140c8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:20.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.748+0000 7f5d13fff700 1 -- 172.21.15.92:0/1885266370 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d1406c3f0 msgr2=0x7f5d140cb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:20.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.748+0000 7f5d13fff700 1 --2- 172.21.15.92:0/1885266370 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d1406c3f0 0x7f5d140cb470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:20.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.748+0000 7f5d13fff700 1 -- 172.21.15.92:0/1885266370 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d00009550 con 0x7f5d140b8510 2023-11-28T11:24:20.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.748+0000 7f5d13fff700 1 --2- 172.21.15.92:0/1885266370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d140b8510 0x7f5d141e2a00 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f5d00000c00 tx=0x7f5d00004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:20.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.749+0000 7f5d117fa700 1 -- 172.21.15.92:0/1885266370 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d00004510 con 0x7f5d140b8510 2023-11-28T11:24:20.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.749+0000 7f5d117fa700 1 -- 172.21.15.92:0/1885266370 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d0001e070 con 0x7f5d140b8510 2023-11-28T11:24:20.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.749+0000 7f5d117fa700 1 -- 172.21.15.92:0/1885266370 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d000038e0 con 0x7f5d140b8510 2023-11-28T11:24:20.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.749+0000 7f5d1abff700 1 -- 172.21.15.92:0/1885266370 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d140cfff0 con 0x7f5d140b8510 2023-11-28T11:24:20.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.749+0000 7f5d1abff700 1 -- 172.21.15.92:0/1885266370 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d140cfd10 con 0x7f5d140b8510 2023-11-28T11:24:20.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.750+0000 7f5d1abff700 1 -- 172.21.15.92:0/1885266370 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d14067880 con 0x7f5d140b8510 2023-11-28T11:24:20.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.751+0000 7f5d117fa700 1 -- 172.21.15.92:0/1885266370 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5d0000f050 con 0x7f5d140b8510 2023-11-28T11:24:20.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.752+0000 7f5d117fa700 1 --2- 172.21.15.92:0/1885266370 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5cf80682f0 0x7f5cf806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:20.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.752+0000 7f5d117fa700 1 -- 172.21.15.92:0/1885266370 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f5d00086f40 con 0x7f5d140b8510 2023-11-28T11:24:20.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.752+0000 7f5d1899b700 1 --2- 172.21.15.92:0/1885266370 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5cf80682f0 0x7f5cf806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:20.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.753+0000 7f5d1899b700 1 --2- 172.21.15.92:0/1885266370 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5cf80682f0 0x7f5cf806a7a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f5d08009be0 tx=0x7f5d08009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:20.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.756+0000 7f5d117fa700 1 -- 172.21.15.92:0/1885266370 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5d000534f0 con 0x7f5d140b8510 2023-11-28T11:24:20.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.941+0000 7f5d1abff700 1 -- 172.21.15.92:0/1885266370 --> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5d140cbfa0 con 0x7f5cf80682f0 2023-11-28T11:24:20.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.951+0000 7f5d117fa700 1 -- 172.21.15.92:0/1885266370 <== mgr.4103 v2:172.21.15.92:6832/115482 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311210 (secure 0 0 0) 0x7f5d140cbfa0 con 0x7f5cf80682f0 2023-11-28T11:24:20.956 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:24:20.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.955+0000 7f5d1abff700 1 -- 172.21.15.92:0/1885266370 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5cf80682f0 msgr2=0x7f5cf806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:20.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.955+0000 7f5d1abff700 1 --2- 172.21.15.92:0/1885266370 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5cf80682f0 0x7f5cf806a7a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f5d08009be0 tx=0x7f5d08009210 comp rx=0 tx=0).stop 2023-11-28T11:24:20.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.955+0000 7f5d1abff700 1 -- 172.21.15.92:0/1885266370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d140b8510 msgr2=0x7f5d141e2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:20.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.955+0000 7f5d1abff700 1 --2- 172.21.15.92:0/1885266370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d140b8510 0x7f5d141e2a00 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f5d00000c00 tx=0x7f5d00004750 comp rx=0 tx=0).stop 2023-11-28T11:24:20.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.955+0000 7f5d1abff700 1 -- 172.21.15.92:0/1885266370 shutdown_connections 2023-11-28T11:24:20.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.955+0000 7f5d1abff700 1 --2- 172.21.15.92:0/1885266370 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5d140cd460 0x7f5d140c8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:20.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.955+0000 7f5d1abff700 1 --2- 172.21.15.92:0/1885266370 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5d1406c3f0 0x7f5d140cb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:20.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.955+0000 7f5d1abff700 1 --2- 172.21.15.92:0/1885266370 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5cf80682f0 0x7f5cf806a7a0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:20.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.955+0000 7f5d1abff700 1 --2- 172.21.15.92:0/1885266370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d140b8510 0x7f5d141e2a00 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:20.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.955+0000 7f5d1abff700 1 -- 172.21.15.92:0/1885266370 >> 172.21.15.92:0/1885266370 conn(0x7f5d140719a0 msgr2=0x7f5d140b5d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:20.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.955+0000 7f5d1abff700 1 -- 172.21.15.92:0/1885266370 shutdown_connections 2023-11-28T11:24:20.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:20.955+0000 7f5d1abff700 1 -- 172.21.15.92:0/1885266370 wait complete. 2023-11-28T11:24:20.960 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-11-28T11:24:20.975 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":170,"stamp":"2023-11-28T11:24:19.903602+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":44,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":84136,"kb_used_data":7464,"kb_used_omap":0,"kb_used_meta":76608,"kb_avail":754890584,"statfs":{"total":773094113280,"available":773007958016,"internally_reserved":0,"allocated":7643136,"data_stored":4180344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":78446592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.142274"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.294401+0000","last_change":"2023-11-28T11:24:16.289334+0000","last_active":"2023-11-28T11:24:16.294401+0000","last_peered":"2023-11-28T11:24:16.294401+0000","last_clean":"2023-11-28T11:24:16.294401+0000","last_became_active":"2023-11-28T11:24:16.289053+0000","last_became_peered":"2023-11-28T11:24:16.289053+0000","last_unstale":"2023-11-28T11:24:16.294401+0000","last_undegraded":"2023-11-28T11:24:16.294401+0000","last_fullsized":"2023-11-28T11:24:16.294401+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:52:10.685572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.301703+0000","last_change":"2023-11-28T11:24:16.293787+0000","last_active":"2023-11-28T11:24:16.301703+0000","last_peered":"2023-11-28T11:24:16.301703+0000","last_clean":"2023-11-28T11:24:16.301703+0000","last_became_active":"2023-11-28T11:24:16.292360+0000","last_became_peered":"2023-11-28T11:24:16.292360+0000","last_unstale":"2023-11-28T11:24:16.301703+0000","last_undegraded":"2023-11-28T11:24:16.301703+0000","last_fullsized":"2023-11-28T11:24:16.301703+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:38.876001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.293954+0000","last_change":"2023-11-28T11:24:16.284598+0000","last_active":"2023-11-28T11:24:16.293954+0000","last_peered":"2023-11-28T11:24:16.293954+0000","last_clean":"2023-11-28T11:24:16.293954+0000","last_became_active":"2023-11-28T11:24:16.284338+0000","last_became_peered":"2023-11-28T11:24:16.284338+0000","last_unstale":"2023-11-28T11:24:16.293954+0000","last_undegraded":"2023-11-28T11:24:16.293954+0000","last_fullsized":"2023-11-28T11:24:16.293954+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:22:37.183147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.301671+0000","last_change":"2023-11-28T11:24:16.297775+0000","last_active":"2023-11-28T11:24:16.301671+0000","last_peered":"2023-11-28T11:24:16.301671+0000","last_clean":"2023-11-28T11:24:16.301671+0000","last_became_active":"2023-11-28T11:24:16.297488+0000","last_became_peered":"2023-11-28T11:24:16.297488+0000","last_unstale":"2023-11-28T11:24:16.301671+0000","last_undegraded":"2023-11-28T11:24:16.301671+0000","last_fullsized":"2023-11-28T11:24:16.301671+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:38.876001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.293181+0000","last_change":"2023-11-28T11:24:16.287597+0000","last_active":"2023-11-28T11:24:16.293181+0000","last_peered":"2023-11-28T11:24:16.293181+0000","last_clean":"2023-11-28T11:24:16.293181+0000","last_became_active":"2023-11-28T11:24:16.287344+0000","last_became_peered":"2023-11-28T11:24:16.287344+0000","last_unstale":"2023-11-28T11:24:16.293181+0000","last_undegraded":"2023-11-28T11:24:16.293181+0000","last_fullsized":"2023-11-28T11:24:16.293181+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:03:57.216059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.301975+0000","last_change":"2023-11-28T11:24:16.297616+0000","last_active":"2023-11-28T11:24:16.301975+0000","last_peered":"2023-11-28T11:24:16.301975+0000","last_clean":"2023-11-28T11:24:16.301975+0000","last_became_active":"2023-11-28T11:24:16.296978+0000","last_became_peered":"2023-11-28T11:24:16.296978+0000","last_unstale":"2023-11-28T11:24:16.301975+0000","last_undegraded":"2023-11-28T11:24:16.301975+0000","last_fullsized":"2023-11-28T11:24:16.301975+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:26:02.103744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.284542+0000","last_change":"2023-11-28T11:24:16.276770+0000","last_active":"2023-11-28T11:24:16.284542+0000","last_peered":"2023-11-28T11:24:16.284542+0000","last_clean":"2023-11-28T11:24:16.284542+0000","last_became_active":"2023-11-28T11:24:16.275880+0000","last_became_peered":"2023-11-28T11:24:16.275880+0000","last_unstale":"2023-11-28T11:24:16.284542+0000","last_undegraded":"2023-11-28T11:24:16.284542+0000","last_fullsized":"2023-11-28T11:24:16.284542+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:54:23.138456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.302006+0000","last_change":"2023-11-28T11:24:16.296181+0000","last_active":"2023-11-28T11:24:16.302006+0000","last_peered":"2023-11-28T11:24:16.302006+0000","last_clean":"2023-11-28T11:24:16.302006+0000","last_became_active":"2023-11-28T11:24:16.295911+0000","last_became_peered":"2023-11-28T11:24:16.295911+0000","last_unstale":"2023-11-28T11:24:16.302006+0000","last_undegraded":"2023-11-28T11:24:16.302006+0000","last_fullsized":"2023-11-28T11:24:16.302006+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:30:53.256213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.294435+0000","last_change":"2023-11-28T11:24:16.290462+0000","last_active":"2023-11-28T11:24:16.294435+0000","last_peered":"2023-11-28T11:24:16.294435+0000","last_clean":"2023-11-28T11:24:16.294435+0000","last_became_active":"2023-11-28T11:24:16.289602+0000","last_became_peered":"2023-11-28T11:24:16.289602+0000","last_unstale":"2023-11-28T11:24:16.294435+0000","last_undegraded":"2023-11-28T11:24:16.294435+0000","last_fullsized":"2023-11-28T11:24:16.294435+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:38:12.535382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.299735+0000","last_change":"2023-11-28T11:24:16.291626+0000","last_active":"2023-11-28T11:24:16.299735+0000","last_peered":"2023-11-28T11:24:16.299735+0000","last_clean":"2023-11-28T11:24:16.299735+0000","last_became_active":"2023-11-28T11:24:16.291295+0000","last_became_peered":"2023-11-28T11:24:16.291295+0000","last_unstale":"2023-11-28T11:24:16.299735+0000","last_undegraded":"2023-11-28T11:24:16.299735+0000","last_fullsized":"2023-11-28T11:24:16.299735+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:42:33.731752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.290787+0000","last_change":"2023-11-28T11:24:16.286111+0000","last_active":"2023-11-28T11:24:16.290787+0000","last_peered":"2023-11-28T11:24:16.290787+0000","last_clean":"2023-11-28T11:24:16.290787+0000","last_became_active":"2023-11-28T11:24:16.284625+0000","last_became_peered":"2023-11-28T11:24:16.284625+0000","last_unstale":"2023-11-28T11:24:16.290787+0000","last_undegraded":"2023-11-28T11:24:16.290787+0000","last_fullsized":"2023-11-28T11:24:16.290787+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:07:18.046315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.284841+0000","last_change":"2023-11-28T11:24:16.279931+0000","last_active":"2023-11-28T11:24:16.284841+0000","last_peered":"2023-11-28T11:24:16.284841+0000","last_clean":"2023-11-28T11:24:16.284841+0000","last_became_active":"2023-11-28T11:24:16.279559+0000","last_became_peered":"2023-11-28T11:24:16.279559+0000","last_unstale":"2023-11-28T11:24:16.284841+0000","last_undegraded":"2023-11-28T11:24:16.284841+0000","last_fullsized":"2023-11-28T11:24:16.284841+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:00:11.407649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.304103+0000","last_change":"2023-11-28T11:24:16.300273+0000","last_active":"2023-11-28T11:24:16.304103+0000","last_peered":"2023-11-28T11:24:16.304103+0000","last_clean":"2023-11-28T11:24:16.304103+0000","last_became_active":"2023-11-28T11:24:16.300014+0000","last_became_peered":"2023-11-28T11:24:16.300014+0000","last_unstale":"2023-11-28T11:24:16.304103+0000","last_undegraded":"2023-11-28T11:24:16.304103+0000","last_fullsized":"2023-11-28T11:24:16.304103+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:01:22.073242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.287299+0000","last_change":"2023-11-28T11:24:16.282970+0000","last_active":"2023-11-28T11:24:16.287299+0000","last_peered":"2023-11-28T11:24:16.287299+0000","last_clean":"2023-11-28T11:24:16.287299+0000","last_became_active":"2023-11-28T11:24:16.282388+0000","last_became_peered":"2023-11-28T11:24:16.282388+0000","last_unstale":"2023-11-28T11:24:16.287299+0000","last_undegraded":"2023-11-28T11:24:16.287299+0000","last_fullsized":"2023-11-28T11:24:16.287299+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:14:11.685643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.293714+0000","last_change":"2023-11-28T11:24:16.288932+0000","last_active":"2023-11-28T11:24:16.293714+0000","last_peered":"2023-11-28T11:24:16.293714+0000","last_clean":"2023-11-28T11:24:16.293714+0000","last_became_active":"2023-11-28T11:24:16.288281+0000","last_became_peered":"2023-11-28T11:24:16.288281+0000","last_unstale":"2023-11-28T11:24:16.293714+0000","last_undegraded":"2023-11-28T11:24:16.293714+0000","last_fullsized":"2023-11-28T11:24:16.293714+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:44:11.394584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.300392+0000","last_change":"2023-11-28T11:24:16.297105+0000","last_active":"2023-11-28T11:24:16.300392+0000","last_peered":"2023-11-28T11:24:16.300392+0000","last_clean":"2023-11-28T11:24:16.300392+0000","last_became_active":"2023-11-28T11:24:16.296505+0000","last_became_peered":"2023-11-28T11:24:16.296505+0000","last_unstale":"2023-11-28T11:24:16.300392+0000","last_undegraded":"2023-11-28T11:24:16.300392+0000","last_fullsized":"2023-11-28T11:24:16.300392+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:44:11.394584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.290617+0000","last_change":"2023-11-28T11:24:16.285467+0000","last_active":"2023-11-28T11:24:16.290617+0000","last_peered":"2023-11-28T11:24:16.290617+0000","last_clean":"2023-11-28T11:24:16.290617+0000","last_became_active":"2023-11-28T11:24:16.284615+0000","last_became_peered":"2023-11-28T11:24:16.284615+0000","last_unstale":"2023-11-28T11:24:16.290617+0000","last_undegraded":"2023-11-28T11:24:16.290617+0000","last_fullsized":"2023-11-28T11:24:16.290617+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:18:15.187665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.289046+0000","last_change":"2023-11-28T11:24:16.283340+0000","last_active":"2023-11-28T11:24:16.289046+0000","last_peered":"2023-11-28T11:24:16.289046+0000","last_clean":"2023-11-28T11:24:16.289046+0000","last_became_active":"2023-11-28T11:24:16.283065+0000","last_became_peered":"2023-11-28T11:24:16.283065+0000","last_unstale":"2023-11-28T11:24:16.289046+0000","last_undegraded":"2023-11-28T11:24:16.289046+0000","last_fullsized":"2023-11-28T11:24:16.289046+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:44:50.050157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.302012+0000","last_change":"2023-11-28T11:24:16.295998+0000","last_active":"2023-11-28T11:24:16.302012+0000","last_peered":"2023-11-28T11:24:16.302012+0000","last_clean":"2023-11-28T11:24:16.302012+0000","last_became_active":"2023-11-28T11:24:16.295306+0000","last_became_peered":"2023-11-28T11:24:16.295306+0000","last_unstale":"2023-11-28T11:24:16.302012+0000","last_undegraded":"2023-11-28T11:24:16.302012+0000","last_fullsized":"2023-11-28T11:24:16.302012+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:15:09.698174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.294639+0000","last_change":"2023-11-28T11:24:16.291575+0000","last_active":"2023-11-28T11:24:16.294639+0000","last_peered":"2023-11-28T11:24:16.294639+0000","last_clean":"2023-11-28T11:24:16.294639+0000","last_became_active":"2023-11-28T11:24:16.291306+0000","last_became_peered":"2023-11-28T11:24:16.291306+0000","last_unstale":"2023-11-28T11:24:16.294639+0000","last_undegraded":"2023-11-28T11:24:16.294639+0000","last_fullsized":"2023-11-28T11:24:16.294639+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:42:33.731752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.295535+0000","last_change":"2023-11-28T11:24:16.289872+0000","last_active":"2023-11-28T11:24:16.295535+0000","last_peered":"2023-11-28T11:24:16.295535+0000","last_clean":"2023-11-28T11:24:16.295535+0000","last_became_active":"2023-11-28T11:24:16.285288+0000","last_became_peered":"2023-11-28T11:24:16.285288+0000","last_unstale":"2023-11-28T11:24:16.295535+0000","last_undegraded":"2023-11-28T11:24:16.295535+0000","last_fullsized":"2023-11-28T11:24:16.295535+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:24:52.490566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.300098+0000","last_change":"2023-11-28T11:24:16.290820+0000","last_active":"2023-11-28T11:24:16.300098+0000","last_peered":"2023-11-28T11:24:16.300098+0000","last_clean":"2023-11-28T11:24:16.300098+0000","last_became_active":"2023-11-28T11:24:16.290412+0000","last_became_peered":"2023-11-28T11:24:16.290412+0000","last_unstale":"2023-11-28T11:24:16.300098+0000","last_undegraded":"2023-11-28T11:24:16.300098+0000","last_fullsized":"2023-11-28T11:24:16.300098+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:53:17.454560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.286848+0000","last_change":"2023-11-28T11:24:16.282351+0000","last_active":"2023-11-28T11:24:16.286848+0000","last_peered":"2023-11-28T11:24:16.286848+0000","last_clean":"2023-11-28T11:24:16.286848+0000","last_became_active":"2023-11-28T11:24:16.282091+0000","last_became_peered":"2023-11-28T11:24:16.282091+0000","last_unstale":"2023-11-28T11:24:16.286848+0000","last_undegraded":"2023-11-28T11:24:16.286848+0000","last_fullsized":"2023-11-28T11:24:16.286848+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:04:41.624812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.302246+0000","last_change":"2023-11-28T11:24:16.299736+0000","last_active":"2023-11-28T11:24:16.302246+0000","last_peered":"2023-11-28T11:24:16.302246+0000","last_clean":"2023-11-28T11:24:16.302246+0000","last_became_active":"2023-11-28T11:24:16.299400+0000","last_became_peered":"2023-11-28T11:24:16.299400+0000","last_unstale":"2023-11-28T11:24:16.302246+0000","last_undegraded":"2023-11-28T11:24:16.302246+0000","last_fullsized":"2023-11-28T11:24:16.302246+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:57:13.973827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.299839+0000","last_change":"2023-11-28T11:24:16.293562+0000","last_active":"2023-11-28T11:24:16.299839+0000","last_peered":"2023-11-28T11:24:16.299839+0000","last_clean":"2023-11-28T11:24:16.299839+0000","last_became_active":"2023-11-28T11:24:16.291284+0000","last_became_peered":"2023-11-28T11:24:16.291284+0000","last_unstale":"2023-11-28T11:24:16.299839+0000","last_undegraded":"2023-11-28T11:24:16.299839+0000","last_fullsized":"2023-11-28T11:24:16.299839+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:49:11.229814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.290228+0000","last_change":"2023-11-28T11:24:16.287559+0000","last_active":"2023-11-28T11:24:16.290228+0000","last_peered":"2023-11-28T11:24:16.290228+0000","last_clean":"2023-11-28T11:24:16.290228+0000","last_became_active":"2023-11-28T11:24:16.285911+0000","last_became_peered":"2023-11-28T11:24:16.285911+0000","last_unstale":"2023-11-28T11:24:16.290228+0000","last_undegraded":"2023-11-28T11:24:16.290228+0000","last_fullsized":"2023-11-28T11:24:16.290228+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:12:10.613903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.291884+0000","last_change":"2023-11-28T11:24:16.282634+0000","last_active":"2023-11-28T11:24:16.291884+0000","last_peered":"2023-11-28T11:24:16.291884+0000","last_clean":"2023-11-28T11:24:16.291884+0000","last_became_active":"2023-11-28T11:24:16.282323+0000","last_became_peered":"2023-11-28T11:24:16.282323+0000","last_unstale":"2023-11-28T11:24:16.291884+0000","last_undegraded":"2023-11-28T11:24:16.291884+0000","last_fullsized":"2023-11-28T11:24:16.291884+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:42:58.518773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.300123+0000","last_change":"2023-11-28T11:24:16.290970+0000","last_active":"2023-11-28T11:24:16.300123+0000","last_peered":"2023-11-28T11:24:16.300123+0000","last_clean":"2023-11-28T11:24:16.300123+0000","last_became_active":"2023-11-28T11:24:16.290667+0000","last_became_peered":"2023-11-28T11:24:16.290667+0000","last_unstale":"2023-11-28T11:24:16.300123+0000","last_undegraded":"2023-11-28T11:24:16.300123+0000","last_fullsized":"2023-11-28T11:24:16.300123+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:32:43.162554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.292148+0000","last_change":"2023-11-28T11:24:16.285904+0000","last_active":"2023-11-28T11:24:16.292148+0000","last_peered":"2023-11-28T11:24:16.292148+0000","last_clean":"2023-11-28T11:24:16.292148+0000","last_became_active":"2023-11-28T11:24:16.285627+0000","last_became_peered":"2023-11-28T11:24:16.285627+0000","last_unstale":"2023-11-28T11:24:16.292148+0000","last_undegraded":"2023-11-28T11:24:16.292148+0000","last_fullsized":"2023-11-28T11:24:16.292148+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:28:34.277902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.299147+0000","last_change":"2023-11-28T11:24:16.296498+0000","last_active":"2023-11-28T11:24:16.299147+0000","last_peered":"2023-11-28T11:24:16.299147+0000","last_clean":"2023-11-28T11:24:16.299147+0000","last_became_active":"2023-11-28T11:24:16.296230+0000","last_became_peered":"2023-11-28T11:24:16.296230+0000","last_unstale":"2023-11-28T11:24:16.299147+0000","last_undegraded":"2023-11-28T11:24:16.299147+0000","last_fullsized":"2023-11-28T11:24:16.299147+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:53:17.454560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.284562+0000","last_change":"2023-11-28T11:24:16.276950+0000","last_active":"2023-11-28T11:24:16.284562+0000","last_peered":"2023-11-28T11:24:16.284562+0000","last_clean":"2023-11-28T11:24:16.284562+0000","last_became_active":"2023-11-28T11:24:16.276489+0000","last_became_peered":"2023-11-28T11:24:16.276489+0000","last_unstale":"2023-11-28T11:24:16.284562+0000","last_undegraded":"2023-11-28T11:24:16.284562+0000","last_fullsized":"2023-11-28T11:24:16.284562+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:30:24.793183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.300742+0000","last_change":"2023-11-28T11:24:16.293196+0000","last_active":"2023-11-28T11:24:16.300742+0000","last_peered":"2023-11-28T11:24:16.300742+0000","last_clean":"2023-11-28T11:24:16.300742+0000","last_became_active":"2023-11-28T11:24:16.292379+0000","last_became_peered":"2023-11-28T11:24:16.292379+0000","last_unstale":"2023-11-28T11:24:16.300742+0000","last_undegraded":"2023-11-28T11:24:16.300742+0000","last_fullsized":"2023-11-28T11:24:16.300742+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:41:26.436810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.304698+0000","last_change":"2023-11-28T11:24:16.301487+0000","last_active":"2023-11-28T11:24:16.304698+0000","last_peered":"2023-11-28T11:24:16.304698+0000","last_clean":"2023-11-28T11:24:16.304698+0000","last_became_active":"2023-11-28T11:24:16.301138+0000","last_became_peered":"2023-11-28T11:24:16.301138+0000","last_unstale":"2023-11-28T11:24:16.304698+0000","last_undegraded":"2023-11-28T11:24:16.304698+0000","last_fullsized":"2023-11-28T11:24:16.304698+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:48:20.990724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"67'4","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.295460+0000","last_change":"2023-11-28T11:24:15.791407+0000","last_active":"2023-11-28T11:24:16.295460+0000","last_peered":"2023-11-28T11:24:16.295460+0000","last_clean":"2023-11-28T11:24:16.295460+0000","last_became_active":"2023-11-28T11:24:15.790447+0000","last_became_peered":"2023-11-28T11:24:15.790447+0000","last_unstale":"2023-11-28T11:24:16.295460+0000","last_undegraded":"2023-11-28T11:24:16.295460+0000","last_fullsized":"2023-11-28T11:24:16.295460+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:37.869238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.290598+0000","last_change":"2023-11-28T11:24:16.284911+0000","last_active":"2023-11-28T11:24:16.290598+0000","last_peered":"2023-11-28T11:24:16.290598+0000","last_clean":"2023-11-28T11:24:16.290598+0000","last_became_active":"2023-11-28T11:24:16.283669+0000","last_became_peered":"2023-11-28T11:24:16.283669+0000","last_unstale":"2023-11-28T11:24:16.290598+0000","last_undegraded":"2023-11-28T11:24:16.290598+0000","last_fullsized":"2023-11-28T11:24:16.290598+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:46:16.231711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"67'1","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.284555+0000","last_change":"2023-11-28T11:24:15.787045+0000","last_active":"2023-11-28T11:24:16.284555+0000","last_peered":"2023-11-28T11:24:16.284555+0000","last_clean":"2023-11-28T11:24:16.284555+0000","last_became_active":"2023-11-28T11:24:15.786714+0000","last_became_peered":"2023-11-28T11:24:15.786714+0000","last_unstale":"2023-11-28T11:24:16.284555+0000","last_undegraded":"2023-11-28T11:24:16.284555+0000","last_fullsized":"2023-11-28T11:24:16.284555+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:48:19.983961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.301988+0000","last_change":"2023-11-28T11:24:16.297404+0000","last_active":"2023-11-28T11:24:16.301988+0000","last_peered":"2023-11-28T11:24:16.301988+0000","last_clean":"2023-11-28T11:24:16.301988+0000","last_became_active":"2023-11-28T11:24:16.296864+0000","last_became_peered":"2023-11-28T11:24:16.296864+0000","last_unstale":"2023-11-28T11:24:16.301988+0000","last_undegraded":"2023-11-28T11:24:16.301988+0000","last_fullsized":"2023-11-28T11:24:16.301988+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:24:48.055382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"67'2","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.299709+0000","last_change":"2023-11-28T11:24:15.791373+0000","last_active":"2023-11-28T11:24:16.299709+0000","last_peered":"2023-11-28T11:24:16.299709+0000","last_clean":"2023-11-28T11:24:16.299709+0000","last_became_active":"2023-11-28T11:24:15.790807+0000","last_became_peered":"2023-11-28T11:24:15.790807+0000","last_unstale":"2023-11-28T11:24:16.299709+0000","last_undegraded":"2023-11-28T11:24:16.299709+0000","last_fullsized":"2023-11-28T11:24:16.299709+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:54:20.554467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.288227+0000","last_change":"2023-11-28T11:24:16.283126+0000","last_active":"2023-11-28T11:24:16.288227+0000","last_peered":"2023-11-28T11:24:16.288227+0000","last_clean":"2023-11-28T11:24:16.288227+0000","last_became_active":"2023-11-28T11:24:16.282484+0000","last_became_peered":"2023-11-28T11:24:16.282484+0000","last_unstale":"2023-11-28T11:24:16.288227+0000","last_undegraded":"2023-11-28T11:24:16.288227+0000","last_fullsized":"2023-11-28T11:24:16.288227+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:17:48.917682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"67'2","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.299463+0000","last_change":"2023-11-28T11:24:15.790330+0000","last_active":"2023-11-28T11:24:16.299463+0000","last_peered":"2023-11-28T11:24:16.299463+0000","last_clean":"2023-11-28T11:24:16.299463+0000","last_became_active":"2023-11-28T11:24:15.790012+0000","last_became_peered":"2023-11-28T11:24:15.790012+0000","last_unstale":"2023-11-28T11:24:16.299463+0000","last_undegraded":"2023-11-28T11:24:16.299463+0000","last_fullsized":"2023-11-28T11:24:16.299463+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:11:07.458538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.290328+0000","last_change":"2023-11-28T11:24:16.287777+0000","last_active":"2023-11-28T11:24:16.290328+0000","last_peered":"2023-11-28T11:24:16.290328+0000","last_clean":"2023-11-28T11:24:16.290328+0000","last_became_active":"2023-11-28T11:24:16.286193+0000","last_became_peered":"2023-11-28T11:24:16.286193+0000","last_unstale":"2023-11-28T11:24:16.290328+0000","last_undegraded":"2023-11-28T11:24:16.290328+0000","last_fullsized":"2023-11-28T11:24:16.290328+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:32:43.162554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"67'1","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.293457+0000","last_change":"2023-11-28T11:24:15.789861+0000","last_active":"2023-11-28T11:24:16.293457+0000","last_peered":"2023-11-28T11:24:16.293457+0000","last_clean":"2023-11-28T11:24:16.293457+0000","last_became_active":"2023-11-28T11:24:15.789531+0000","last_became_peered":"2023-11-28T11:24:15.789531+0000","last_unstale":"2023-11-28T11:24:16.293457+0000","last_undegraded":"2023-11-28T11:24:16.293457+0000","last_fullsized":"2023-11-28T11:24:16.293457+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:46:10.168470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.300068+0000","last_change":"2023-11-28T11:24:16.291704+0000","last_active":"2023-11-28T11:24:16.300068+0000","last_peered":"2023-11-28T11:24:16.300068+0000","last_clean":"2023-11-28T11:24:16.300068+0000","last_became_active":"2023-11-28T11:24:16.291334+0000","last_became_peered":"2023-11-28T11:24:16.291334+0000","last_unstale":"2023-11-28T11:24:16.300068+0000","last_undegraded":"2023-11-28T11:24:16.300068+0000","last_fullsized":"2023-11-28T11:24:16.300068+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:26:38.836179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"67'1","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.299076+0000","last_change":"2023-11-28T11:24:15.794958+0000","last_active":"2023-11-28T11:24:16.299076+0000","last_peered":"2023-11-28T11:24:16.299076+0000","last_clean":"2023-11-28T11:24:16.299076+0000","last_became_active":"2023-11-28T11:24:15.794623+0000","last_became_peered":"2023-11-28T11:24:15.794623+0000","last_unstale":"2023-11-28T11:24:16.299076+0000","last_undegraded":"2023-11-28T11:24:16.299076+0000","last_fullsized":"2023-11-28T11:24:16.299076+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:14:10.678880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.293986+0000","last_change":"2023-11-28T11:24:16.288044+0000","last_active":"2023-11-28T11:24:16.293986+0000","last_peered":"2023-11-28T11:24:16.293986+0000","last_clean":"2023-11-28T11:24:16.293986+0000","last_became_active":"2023-11-28T11:24:16.287795+0000","last_became_peered":"2023-11-28T11:24:16.287795+0000","last_unstale":"2023-11-28T11:24:16.293986+0000","last_undegraded":"2023-11-28T11:24:16.293986+0000","last_fullsized":"2023-11-28T11:24:16.293986+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:30:51.011752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.262625+0000","last_change":"2023-11-28T11:24:15.779840+0000","last_active":"2023-11-28T11:24:16.262625+0000","last_peered":"2023-11-28T11:24:16.262625+0000","last_clean":"2023-11-28T11:24:16.262625+0000","last_became_active":"2023-11-28T11:24:15.779558+0000","last_became_peered":"2023-11-28T11:24:15.779558+0000","last_unstale":"2023-11-28T11:24:16.262625+0000","last_undegraded":"2023-11-28T11:24:16.262625+0000","last_fullsized":"2023-11-28T11:24:16.262625+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:11:07.458538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.293595+0000","last_change":"2023-11-28T11:24:16.287946+0000","last_active":"2023-11-28T11:24:16.293595+0000","last_peered":"2023-11-28T11:24:16.293595+0000","last_clean":"2023-11-28T11:24:16.293595+0000","last_became_active":"2023-11-28T11:24:16.287610+0000","last_became_peered":"2023-11-28T11:24:16.287610+0000","last_unstale":"2023-11-28T11:24:16.293595+0000","last_undegraded":"2023-11-28T11:24:16.293595+0000","last_fullsized":"2023-11-28T11:24:16.293595+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:46:11.175232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.271397+0000","last_change":"2023-11-28T11:24:15.784188+0000","last_active":"2023-11-28T11:24:16.271397+0000","last_peered":"2023-11-28T11:24:16.271397+0000","last_clean":"2023-11-28T11:24:16.271397+0000","last_became_active":"2023-11-28T11:24:15.783912+0000","last_became_peered":"2023-11-28T11:24:15.783912+0000","last_unstale":"2023-11-28T11:24:16.271397+0000","last_undegraded":"2023-11-28T11:24:16.271397+0000","last_fullsized":"2023-11-28T11:24:16.271397+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:42:52.804541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.279171+0000","last_change":"2023-11-28T11:24:16.275083+0000","last_active":"2023-11-28T11:24:16.279171+0000","last_peered":"2023-11-28T11:24:16.279171+0000","last_clean":"2023-11-28T11:24:16.279171+0000","last_became_active":"2023-11-28T11:24:16.274819+0000","last_became_peered":"2023-11-28T11:24:16.274819+0000","last_unstale":"2023-11-28T11:24:16.279171+0000","last_undegraded":"2023-11-28T11:24:16.279171+0000","last_fullsized":"2023-11-28T11:24:16.279171+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:42:33.731752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.269932+0000","last_change":"2023-11-28T11:24:15.789937+0000","last_active":"2023-11-28T11:24:16.269932+0000","last_peered":"2023-11-28T11:24:16.269932+0000","last_clean":"2023-11-28T11:24:16.269932+0000","last_became_active":"2023-11-28T11:24:15.788196+0000","last_became_peered":"2023-11-28T11:24:15.788196+0000","last_unstale":"2023-11-28T11:24:16.269932+0000","last_undegraded":"2023-11-28T11:24:16.269932+0000","last_fullsized":"2023-11-28T11:24:16.269932+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:17:47.910920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.304691+0000","last_change":"2023-11-28T11:24:16.301321+0000","last_active":"2023-11-28T11:24:16.304691+0000","last_peered":"2023-11-28T11:24:16.304691+0000","last_clean":"2023-11-28T11:24:16.304691+0000","last_became_active":"2023-11-28T11:24:16.301057+0000","last_became_peered":"2023-11-28T11:24:16.301057+0000","last_unstale":"2023-11-28T11:24:16.304691+0000","last_undegraded":"2023-11-28T11:24:16.304691+0000","last_fullsized":"2023-11-28T11:24:16.304691+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:30:24.793183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.279441+0000","last_change":"2023-11-28T11:24:15.788090+0000","last_active":"2023-11-28T11:24:16.279441+0000","last_peered":"2023-11-28T11:24:16.279441+0000","last_clean":"2023-11-28T11:24:16.279441+0000","last_became_active":"2023-11-28T11:24:15.787812+0000","last_became_peered":"2023-11-28T11:24:15.787812+0000","last_unstale":"2023-11-28T11:24:16.279441+0000","last_undegraded":"2023-11-28T11:24:16.279441+0000","last_fullsized":"2023-11-28T11:24:16.279441+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:46:15.224948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.301967+0000","last_change":"2023-11-28T11:24:16.297238+0000","last_active":"2023-11-28T11:24:16.301967+0000","last_peered":"2023-11-28T11:24:16.301967+0000","last_clean":"2023-11-28T11:24:16.301967+0000","last_became_active":"2023-11-28T11:24:16.296511+0000","last_became_peered":"2023-11-28T11:24:16.296511+0000","last_unstale":"2023-11-28T11:24:16.301967+0000","last_undegraded":"2023-11-28T11:24:16.301967+0000","last_fullsized":"2023-11-28T11:24:16.301967+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:18:49.460782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"67'3","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.293895+0000","last_change":"2023-11-28T11:24:15.786382+0000","last_active":"2023-11-28T11:24:16.293895+0000","last_peered":"2023-11-28T11:24:16.293895+0000","last_clean":"2023-11-28T11:24:16.293895+0000","last_became_active":"2023-11-28T11:24:15.786124+0000","last_became_peered":"2023-11-28T11:24:15.786124+0000","last_unstale":"2023-11-28T11:24:16.293895+0000","last_undegraded":"2023-11-28T11:24:16.293895+0000","last_fullsized":"2023-11-28T11:24:16.293895+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:13.987166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.293738+0000","last_change":"2023-11-28T11:24:16.287632+0000","last_active":"2023-11-28T11:24:16.293738+0000","last_peered":"2023-11-28T11:24:16.293738+0000","last_clean":"2023-11-28T11:24:16.293738+0000","last_became_active":"2023-11-28T11:24:16.287371+0000","last_became_peered":"2023-11-28T11:24:16.287371+0000","last_unstale":"2023-11-28T11:24:16.293738+0000","last_undegraded":"2023-11-28T11:24:16.293738+0000","last_fullsized":"2023-11-28T11:24:16.293738+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:49:51.062127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"67'6","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.302064+0000","last_change":"2023-11-28T11:24:15.784875+0000","last_active":"2023-11-28T11:24:16.302064+0000","last_peered":"2023-11-28T11:24:16.302064+0000","last_clean":"2023-11-28T11:24:16.302064+0000","last_became_active":"2023-11-28T11:24:15.784596+0000","last_became_peered":"2023-11-28T11:24:15.784596+0000","last_unstale":"2023-11-28T11:24:16.302064+0000","last_undegraded":"2023-11-28T11:24:16.302064+0000","last_fullsized":"2023-11-28T11:24:16.302064+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:30:50.004990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.294865+0000","last_change":"2023-11-28T11:24:16.288680+0000","last_active":"2023-11-28T11:24:16.294865+0000","last_peered":"2023-11-28T11:24:16.294865+0000","last_clean":"2023-11-28T11:24:16.294865+0000","last_became_active":"2023-11-28T11:24:16.287059+0000","last_became_peered":"2023-11-28T11:24:16.287059+0000","last_unstale":"2023-11-28T11:24:16.294865+0000","last_undegraded":"2023-11-28T11:24:16.294865+0000","last_fullsized":"2023-11-28T11:24:16.294865+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:28:52.219751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"67'2","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.292139+0000","last_change":"2023-11-28T11:24:15.790658+0000","last_active":"2023-11-28T11:24:16.292139+0000","last_peered":"2023-11-28T11:24:16.292139+0000","last_clean":"2023-11-28T11:24:16.292139+0000","last_became_active":"2023-11-28T11:24:15.790263+0000","last_became_peered":"2023-11-28T11:24:15.790263+0000","last_unstale":"2023-11-28T11:24:16.292139+0000","last_undegraded":"2023-11-28T11:24:16.292139+0000","last_fullsized":"2023-11-28T11:24:16.292139+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:02:40.577973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.286721+0000","last_change":"2023-11-28T11:24:16.282554+0000","last_active":"2023-11-28T11:24:16.286721+0000","last_peered":"2023-11-28T11:24:16.286721+0000","last_clean":"2023-11-28T11:24:16.286721+0000","last_became_active":"2023-11-28T11:24:16.281211+0000","last_became_peered":"2023-11-28T11:24:16.281211+0000","last_unstale":"2023-11-28T11:24:16.286721+0000","last_undegraded":"2023-11-28T11:24:16.286721+0000","last_fullsized":"2023-11-28T11:24:16.286721+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:30:18.787935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.281948+0000","last_change":"2023-11-28T11:24:15.788508+0000","last_active":"2023-11-28T11:24:16.281948+0000","last_peered":"2023-11-28T11:24:16.281948+0000","last_clean":"2023-11-28T11:24:16.281948+0000","last_became_active":"2023-11-28T11:24:15.788174+0000","last_became_peered":"2023-11-28T11:24:15.788174+0000","last_unstale":"2023-11-28T11:24:16.281948+0000","last_undegraded":"2023-11-28T11:24:16.281948+0000","last_fullsized":"2023-11-28T11:24:16.281948+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:26:25.627047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.304101+0000","last_change":"2023-11-28T11:24:16.300155+0000","last_active":"2023-11-28T11:24:16.304101+0000","last_peered":"2023-11-28T11:24:16.304101+0000","last_clean":"2023-11-28T11:24:16.304101+0000","last_became_active":"2023-11-28T11:24:16.299896+0000","last_became_peered":"2023-11-28T11:24:16.299896+0000","last_unstale":"2023-11-28T11:24:16.304101+0000","last_undegraded":"2023-11-28T11:24:16.304101+0000","last_fullsized":"2023-11-28T11:24:16.304101+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:26:38.836179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.267858+0000","last_change":"2023-11-28T11:24:15.783430+0000","last_active":"2023-11-28T11:24:16.267858+0000","last_peered":"2023-11-28T11:24:16.267858+0000","last_clean":"2023-11-28T11:24:16.267858+0000","last_became_active":"2023-11-28T11:24:15.783059+0000","last_became_peered":"2023-11-28T11:24:15.783059+0000","last_unstale":"2023-11-28T11:24:16.267858+0000","last_undegraded":"2023-11-28T11:24:16.267858+0000","last_fullsized":"2023-11-28T11:24:16.267858+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:28:51.212988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.287272+0000","last_change":"2023-11-28T11:24:16.282622+0000","last_active":"2023-11-28T11:24:16.287272+0000","last_peered":"2023-11-28T11:24:16.287272+0000","last_clean":"2023-11-28T11:24:16.287272+0000","last_became_active":"2023-11-28T11:24:16.282076+0000","last_became_peered":"2023-11-28T11:24:16.282076+0000","last_unstale":"2023-11-28T11:24:16.287272+0000","last_undegraded":"2023-11-28T11:24:16.287272+0000","last_fullsized":"2023-11-28T11:24:16.287272+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:11:08.465301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"67'8","reported_seq":18,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.296448+0000","last_change":"2023-11-28T11:24:15.789291+0000","last_active":"2023-11-28T11:24:16.296448+0000","last_peered":"2023-11-28T11:24:16.296448+0000","last_clean":"2023-11-28T11:24:16.296448+0000","last_became_active":"2023-11-28T11:24:15.788986+0000","last_became_peered":"2023-11-28T11:24:15.788986+0000","last_unstale":"2023-11-28T11:24:16.296448+0000","last_undegraded":"2023-11-28T11:24:16.296448+0000","last_fullsized":"2023-11-28T11:24:16.296448+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:52:09.678809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.290121+0000","last_change":"2023-11-28T11:24:16.285079+0000","last_active":"2023-11-28T11:24:16.290121+0000","last_peered":"2023-11-28T11:24:16.290121+0000","last_clean":"2023-11-28T11:24:16.290121+0000","last_became_active":"2023-11-28T11:24:16.284827+0000","last_became_peered":"2023-11-28T11:24:16.284827+0000","last_unstale":"2023-11-28T11:24:16.290121+0000","last_undegraded":"2023-11-28T11:24:16.290121+0000","last_fullsized":"2023-11-28T11:24:16.290121+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:49:11.229814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"67'2","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.300490+0000","last_change":"2023-11-28T11:24:15.785815+0000","last_active":"2023-11-28T11:24:16.300490+0000","last_peered":"2023-11-28T11:24:16.300490+0000","last_clean":"2023-11-28T11:24:16.300490+0000","last_became_active":"2023-11-28T11:24:15.785277+0000","last_became_peered":"2023-11-28T11:24:15.785277+0000","last_unstale":"2023-11-28T11:24:16.300490+0000","last_undegraded":"2023-11-28T11:24:16.300490+0000","last_fullsized":"2023-11-28T11:24:16.300490+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:49:50.055365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.284435+0000","last_change":"2023-11-28T11:24:16.277984+0000","last_active":"2023-11-28T11:24:16.284435+0000","last_peered":"2023-11-28T11:24:16.284435+0000","last_clean":"2023-11-28T11:24:16.284435+0000","last_became_active":"2023-11-28T11:24:16.277706+0000","last_became_peered":"2023-11-28T11:24:16.277706+0000","last_unstale":"2023-11-28T11:24:16.284435+0000","last_undegraded":"2023-11-28T11:24:16.284435+0000","last_fullsized":"2023-11-28T11:24:16.284435+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:23:54.440671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.267911+0000","last_change":"2023-11-28T11:24:15.784761+0000","last_active":"2023-11-28T11:24:16.267911+0000","last_peered":"2023-11-28T11:24:16.267911+0000","last_clean":"2023-11-28T11:24:16.267911+0000","last_became_active":"2023-11-28T11:24:15.784469+0000","last_became_peered":"2023-11-28T11:24:15.784469+0000","last_unstale":"2023-11-28T11:24:16.267911+0000","last_undegraded":"2023-11-28T11:24:16.267911+0000","last_fullsized":"2023-11-28T11:24:16.267911+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:30:09.205229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.300386+0000","last_change":"2023-11-28T11:24:16.298424+0000","last_active":"2023-11-28T11:24:16.300386+0000","last_peered":"2023-11-28T11:24:16.300386+0000","last_clean":"2023-11-28T11:24:16.300386+0000","last_became_active":"2023-11-28T11:24:16.298170+0000","last_became_peered":"2023-11-28T11:24:16.298170+0000","last_unstale":"2023-11-28T11:24:16.300386+0000","last_undegraded":"2023-11-28T11:24:16.300386+0000","last_fullsized":"2023-11-28T11:24:16.300386+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:36:55.431715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.271895+0000","last_change":"2023-11-28T11:24:15.786191+0000","last_active":"2023-11-28T11:24:16.271895+0000","last_peered":"2023-11-28T11:24:16.271895+0000","last_clean":"2023-11-28T11:24:16.271895+0000","last_became_active":"2023-11-28T11:24:15.784407+0000","last_became_peered":"2023-11-28T11:24:15.784407+0000","last_unstale":"2023-11-28T11:24:16.271895+0000","last_undegraded":"2023-11-28T11:24:16.271895+0000","last_fullsized":"2023-11-28T11:24:16.271895+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:06:21.696007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.288247+0000","last_change":"2023-11-28T11:24:16.283266+0000","last_active":"2023-11-28T11:24:16.288247+0000","last_peered":"2023-11-28T11:24:16.288247+0000","last_clean":"2023-11-28T11:24:16.288247+0000","last_became_active":"2023-11-28T11:24:16.282830+0000","last_became_peered":"2023-11-28T11:24:16.282830+0000","last_unstale":"2023-11-28T11:24:16.288247+0000","last_undegraded":"2023-11-28T11:24:16.288247+0000","last_fullsized":"2023-11-28T11:24:16.288247+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:54:21.561230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.277978+0000","last_change":"2023-11-28T11:24:15.791886+0000","last_active":"2023-11-28T11:24:16.277978+0000","last_peered":"2023-11-28T11:24:16.277978+0000","last_clean":"2023-11-28T11:24:16.277978+0000","last_became_active":"2023-11-28T11:24:15.791568+0000","last_became_peered":"2023-11-28T11:24:15.791568+0000","last_unstale":"2023-11-28T11:24:16.277978+0000","last_undegraded":"2023-11-28T11:24:16.277978+0000","last_fullsized":"2023-11-28T11:24:16.277978+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:36:54.424952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.287317+0000","last_change":"2023-11-28T11:24:16.283124+0000","last_active":"2023-11-28T11:24:16.287317+0000","last_peered":"2023-11-28T11:24:16.287317+0000","last_clean":"2023-11-28T11:24:16.287317+0000","last_became_active":"2023-11-28T11:24:16.282730+0000","last_became_peered":"2023-11-28T11:24:16.282730+0000","last_unstale":"2023-11-28T11:24:16.287317+0000","last_undegraded":"2023-11-28T11:24:16.287317+0000","last_fullsized":"2023-11-28T11:24:16.287317+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:26:26.633809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.271250+0000","last_change":"2023-11-28T11:24:15.787882+0000","last_active":"2023-11-28T11:24:16.271250+0000","last_peered":"2023-11-28T11:24:16.271250+0000","last_clean":"2023-11-28T11:24:16.271250+0000","last_became_active":"2023-11-28T11:24:15.787534+0000","last_became_peered":"2023-11-28T11:24:15.787534+0000","last_unstale":"2023-11-28T11:24:16.271250+0000","last_undegraded":"2023-11-28T11:24:16.271250+0000","last_fullsized":"2023-11-28T11:24:16.271250+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:42:30.674044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.301699+0000","last_change":"2023-11-28T11:24:16.293811+0000","last_active":"2023-11-28T11:24:16.301699+0000","last_peered":"2023-11-28T11:24:16.301699+0000","last_clean":"2023-11-28T11:24:16.301699+0000","last_became_active":"2023-11-28T11:24:16.292397+0000","last_became_peered":"2023-11-28T11:24:16.292397+0000","last_unstale":"2023-11-28T11:24:16.301699+0000","last_undegraded":"2023-11-28T11:24:16.301699+0000","last_fullsized":"2023-11-28T11:24:16.301699+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:44:51.915900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.262623+0000","last_change":"2023-11-28T11:24:15.781417+0000","last_active":"2023-11-28T11:24:16.262623+0000","last_peered":"2023-11-28T11:24:16.262623+0000","last_clean":"2023-11-28T11:24:16.262623+0000","last_became_active":"2023-11-28T11:24:15.781179+0000","last_became_peered":"2023-11-28T11:24:15.781179+0000","last_unstale":"2023-11-28T11:24:16.262623+0000","last_undegraded":"2023-11-28T11:24:16.262623+0000","last_fullsized":"2023-11-28T11:24:16.262623+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:26:25.627047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.282990+0000","last_change":"2023-11-28T11:24:16.278841+0000","last_active":"2023-11-28T11:24:16.282990+0000","last_peered":"2023-11-28T11:24:16.282990+0000","last_clean":"2023-11-28T11:24:16.282990+0000","last_became_active":"2023-11-28T11:24:16.278550+0000","last_became_peered":"2023-11-28T11:24:16.278550+0000","last_unstale":"2023-11-28T11:24:16.282990+0000","last_undegraded":"2023-11-28T11:24:16.282990+0000","last_fullsized":"2023-11-28T11:24:16.282990+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:26:34.127522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.270000+0000","last_change":"2023-11-28T11:24:15.791998+0000","last_active":"2023-11-28T11:24:16.270000+0000","last_peered":"2023-11-28T11:24:16.270000+0000","last_clean":"2023-11-28T11:24:16.270000+0000","last_became_active":"2023-11-28T11:24:15.791163+0000","last_became_peered":"2023-11-28T11:24:15.791163+0000","last_unstale":"2023-11-28T11:24:16.270000+0000","last_undegraded":"2023-11-28T11:24:16.270000+0000","last_fullsized":"2023-11-28T11:24:16.270000+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:28:51.212988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.300800+0000","last_change":"2023-11-28T11:24:16.293082+0000","last_active":"2023-11-28T11:24:16.300800+0000","last_peered":"2023-11-28T11:24:16.300800+0000","last_clean":"2023-11-28T11:24:16.300800+0000","last_became_active":"2023-11-28T11:24:16.292418+0000","last_became_peered":"2023-11-28T11:24:16.292418+0000","last_unstale":"2023-11-28T11:24:16.300800+0000","last_undegraded":"2023-11-28T11:24:16.300800+0000","last_fullsized":"2023-11-28T11:24:16.300800+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:21:15.703589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.279377+0000","last_change":"2023-11-28T11:24:15.790249+0000","last_active":"2023-11-28T11:24:16.279377+0000","last_peered":"2023-11-28T11:24:16.279377+0000","last_clean":"2023-11-28T11:24:16.279377+0000","last_became_active":"2023-11-28T11:24:15.789966+0000","last_became_peered":"2023-11-28T11:24:15.789966+0000","last_unstale":"2023-11-28T11:24:16.279377+0000","last_undegraded":"2023-11-28T11:24:16.279377+0000","last_fullsized":"2023-11-28T11:24:16.279377+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:57:38.381675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.294023+0000","last_change":"2023-11-28T11:24:16.288388+0000","last_active":"2023-11-28T11:24:16.294023+0000","last_peered":"2023-11-28T11:24:16.294023+0000","last_clean":"2023-11-28T11:24:16.294023+0000","last_became_active":"2023-11-28T11:24:16.286204+0000","last_became_peered":"2023-11-28T11:24:16.286204+0000","last_unstale":"2023-11-28T11:24:16.294023+0000","last_undegraded":"2023-11-28T11:24:16.294023+0000","last_fullsized":"2023-11-28T11:24:16.294023+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:02:41.584735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.270520+0000","last_change":"2023-11-28T11:24:15.793185+0000","last_active":"2023-11-28T11:24:16.270520+0000","last_peered":"2023-11-28T11:24:16.270520+0000","last_clean":"2023-11-28T11:24:16.270520+0000","last_became_active":"2023-11-28T11:24:15.792887+0000","last_became_peered":"2023-11-28T11:24:15.792887+0000","last_unstale":"2023-11-28T11:24:16.270520+0000","last_undegraded":"2023-11-28T11:24:16.270520+0000","last_fullsized":"2023-11-28T11:24:16.270520+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:18:48.454020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":53,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:24:15.767172+0000","last_change":"2023-11-28T11:20:51.020016+0000","last_active":"2023-11-28T11:24:15.767172+0000","last_peered":"2023-11-28T11:24:15.767172+0000","last_clean":"2023-11-28T11:24:15.767172+0000","last_became_active":"2023-11-28T11:20:51.019492+0000","last_became_peered":"2023-11-28T11:20:51.019492+0000","last_unstale":"2023-11-28T11:24:15.767172+0000","last_undegraded":"2023-11-28T11:24:15.767172+0000","last_fullsized":"2023-11-28T11:24:15.767172+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:27.884538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.292124+0000","last_change":"2023-11-28T11:24:16.285826+0000","last_active":"2023-11-28T11:24:16.292124+0000","last_peered":"2023-11-28T11:24:16.292124+0000","last_clean":"2023-11-28T11:24:16.292124+0000","last_became_active":"2023-11-28T11:24:16.285551+0000","last_became_peered":"2023-11-28T11:24:16.285551+0000","last_unstale":"2023-11-28T11:24:16.292124+0000","last_undegraded":"2023-11-28T11:24:16.292124+0000","last_fullsized":"2023-11-28T11:24:16.292124+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:08:10.008261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.281681+0000","last_change":"2023-11-28T11:24:15.794692+0000","last_active":"2023-11-28T11:24:16.281681+0000","last_peered":"2023-11-28T11:24:16.281681+0000","last_clean":"2023-11-28T11:24:16.281681+0000","last_became_active":"2023-11-28T11:24:15.793892+0000","last_became_peered":"2023-11-28T11:24:15.793892+0000","last_unstale":"2023-11-28T11:24:16.281681+0000","last_undegraded":"2023-11-28T11:24:16.281681+0000","last_fullsized":"2023-11-28T11:24:16.281681+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:02:40.577973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":57,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:24:15.776827+0000","last_change":"2023-11-28T11:20:50.521908+0000","last_active":"2023-11-28T11:24:15.776827+0000","last_peered":"2023-11-28T11:24:15.776827+0000","last_clean":"2023-11-28T11:24:15.776827+0000","last_became_active":"2023-11-28T11:20:50.521059+0000","last_became_peered":"2023-11-28T11:20:50.521059+0000","last_unstale":"2023-11-28T11:24:15.776827+0000","last_undegraded":"2023-11-28T11:24:15.776827+0000","last_fullsized":"2023-11-28T11:24:15.776827+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:05.302498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.292505+0000","last_change":"2023-11-28T11:24:16.286009+0000","last_active":"2023-11-28T11:24:16.292505+0000","last_peered":"2023-11-28T11:24:16.292505+0000","last_clean":"2023-11-28T11:24:16.292505+0000","last_became_active":"2023-11-28T11:24:16.285080+0000","last_became_peered":"2023-11-28T11:24:16.285080+0000","last_unstale":"2023-11-28T11:24:16.292505+0000","last_undegraded":"2023-11-28T11:24:16.292505+0000","last_fullsized":"2023-11-28T11:24:16.292505+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:48:35.874902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.267968+0000","last_change":"2023-11-28T11:24:15.785470+0000","last_active":"2023-11-28T11:24:16.267968+0000","last_peered":"2023-11-28T11:24:16.267968+0000","last_clean":"2023-11-28T11:24:16.267968+0000","last_became_active":"2023-11-28T11:24:15.785175+0000","last_became_peered":"2023-11-28T11:24:15.785175+0000","last_unstale":"2023-11-28T11:24:16.267968+0000","last_undegraded":"2023-11-28T11:24:16.267968+0000","last_fullsized":"2023-11-28T11:24:16.267968+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:11:12.657318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":50,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:24:15.775444+0000","last_change":"2023-11-28T11:20:50.619004+0000","last_active":"2023-11-28T11:24:15.775444+0000","last_peered":"2023-11-28T11:24:15.775444+0000","last_clean":"2023-11-28T11:24:15.775444+0000","last_became_active":"2023-11-28T11:20:50.618315+0000","last_became_peered":"2023-11-28T11:20:50.618315+0000","last_unstale":"2023-11-28T11:24:15.775444+0000","last_undegraded":"2023-11-28T11:24:15.775444+0000","last_fullsized":"2023-11-28T11:24:15.775444+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:27.884538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.301557+0000","last_change":"2023-11-28T11:24:16.298556+0000","last_active":"2023-11-28T11:24:16.301557+0000","last_peered":"2023-11-28T11:24:16.301557+0000","last_clean":"2023-11-28T11:24:16.301557+0000","last_became_active":"2023-11-28T11:24:16.298258+0000","last_became_peered":"2023-11-28T11:24:16.298258+0000","last_unstale":"2023-11-28T11:24:16.301557+0000","last_undegraded":"2023-11-28T11:24:16.301557+0000","last_fullsized":"2023-11-28T11:24:16.301557+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:46:42.415700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"67'2","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.299605+0000","last_change":"2023-11-28T11:24:15.791634+0000","last_active":"2023-11-28T11:24:16.299605+0000","last_peered":"2023-11-28T11:24:16.299605+0000","last_clean":"2023-11-28T11:24:16.299605+0000","last_became_active":"2023-11-28T11:24:15.791381+0000","last_became_peered":"2023-11-28T11:24:15.791381+0000","last_unstale":"2023-11-28T11:24:16.299605+0000","last_undegraded":"2023-11-28T11:24:16.299605+0000","last_fullsized":"2023-11-28T11:24:16.299605+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:44:02.542531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":57,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:24:15.776890+0000","last_change":"2023-11-28T11:20:50.522582+0000","last_active":"2023-11-28T11:24:15.776890+0000","last_peered":"2023-11-28T11:24:15.776890+0000","last_clean":"2023-11-28T11:24:15.776890+0000","last_became_active":"2023-11-28T11:20:50.522157+0000","last_became_peered":"2023-11-28T11:20:50.522157+0000","last_unstale":"2023-11-28T11:24:15.776890+0000","last_undegraded":"2023-11-28T11:24:15.776890+0000","last_fullsized":"2023-11-28T11:24:15.776890+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:27.884538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":204,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:24:15.779531+0000","last_change":"2023-11-28T11:20:43.294845+0000","last_active":"2023-11-28T11:24:15.779531+0000","last_peered":"2023-11-28T11:24:15.779531+0000","last_clean":"2023-11-28T11:24:15.779531+0000","last_became_active":"2023-11-28T11:20:43.294483+0000","last_became_peered":"2023-11-28T11:20:43.294483+0000","last_unstale":"2023-11-28T11:24:15.779531+0000","last_undegraded":"2023-11-28T11:24:15.779531+0000","last_fullsized":"2023-11-28T11:24:15.779531+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:42.270512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:42.270512+0000","last_clean_scrub_stamp":"2023-11-28T11:20:42.270512+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:17:03.376680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.301630+0000","last_change":"2023-11-28T11:24:16.298648+0000","last_active":"2023-11-28T11:24:16.301630+0000","last_peered":"2023-11-28T11:24:16.301630+0000","last_clean":"2023-11-28T11:24:16.301630+0000","last_became_active":"2023-11-28T11:24:16.298264+0000","last_became_peered":"2023-11-28T11:24:16.298264+0000","last_unstale":"2023-11-28T11:24:16.301630+0000","last_undegraded":"2023-11-28T11:24:16.301630+0000","last_fullsized":"2023-11-28T11:24:16.301630+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:01:32.493557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.276939+0000","last_change":"2023-11-28T11:24:15.787841+0000","last_active":"2023-11-28T11:24:16.276939+0000","last_peered":"2023-11-28T11:24:16.276939+0000","last_clean":"2023-11-28T11:24:16.276939+0000","last_became_active":"2023-11-28T11:24:15.787081+0000","last_became_peered":"2023-11-28T11:24:15.787081+0000","last_unstale":"2023-11-28T11:24:16.276939+0000","last_undegraded":"2023-11-28T11:24:16.276939+0000","last_fullsized":"2023-11-28T11:24:16.276939+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:18:37.395260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":51,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:24:15.779618+0000","last_change":"2023-11-28T11:20:50.526865+0000","last_active":"2023-11-28T11:24:15.779618+0000","last_peered":"2023-11-28T11:24:15.779618+0000","last_clean":"2023-11-28T11:24:15.779618+0000","last_became_active":"2023-11-28T11:20:50.526576+0000","last_became_peered":"2023-11-28T11:20:50.526576+0000","last_unstale":"2023-11-28T11:24:15.779618+0000","last_undegraded":"2023-11-28T11:24:15.779618+0000","last_fullsized":"2023-11-28T11:24:15.779618+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:09:18.460112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.284862+0000","last_change":"2023-11-28T11:24:16.280368+0000","last_active":"2023-11-28T11:24:16.284862+0000","last_peered":"2023-11-28T11:24:16.284862+0000","last_clean":"2023-11-28T11:24:16.284862+0000","last_became_active":"2023-11-28T11:24:16.280067+0000","last_became_peered":"2023-11-28T11:24:16.280067+0000","last_unstale":"2023-11-28T11:24:16.284862+0000","last_undegraded":"2023-11-28T11:24:16.284862+0000","last_fullsized":"2023-11-28T11:24:16.284862+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:12:31.931403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"67'1","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.301943+0000","last_change":"2023-11-28T11:24:15.793755+0000","last_active":"2023-11-28T11:24:16.301943+0000","last_peered":"2023-11-28T11:24:16.301943+0000","last_clean":"2023-11-28T11:24:16.301943+0000","last_became_active":"2023-11-28T11:24:15.793485+0000","last_became_peered":"2023-11-28T11:24:15.793485+0000","last_unstale":"2023-11-28T11:24:16.301943+0000","last_undegraded":"2023-11-28T11:24:16.301943+0000","last_fullsized":"2023-11-28T11:24:16.301943+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:30:48.708427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":51,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:24:15.784160+0000","last_change":"2023-11-28T11:20:51.020043+0000","last_active":"2023-11-28T11:24:15.784160+0000","last_peered":"2023-11-28T11:24:15.784160+0000","last_clean":"2023-11-28T11:24:15.784160+0000","last_became_active":"2023-11-28T11:20:51.019702+0000","last_became_peered":"2023-11-28T11:20:51.019702+0000","last_unstale":"2023-11-28T11:24:15.784160+0000","last_undegraded":"2023-11-28T11:24:15.784160+0000","last_fullsized":"2023-11-28T11:24:15.784160+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:26:20.246873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.294381+0000","last_change":"2023-11-28T11:24:16.288263+0000","last_active":"2023-11-28T11:24:16.294381+0000","last_peered":"2023-11-28T11:24:16.294381+0000","last_clean":"2023-11-28T11:24:16.294381+0000","last_became_active":"2023-11-28T11:24:16.287986+0000","last_became_peered":"2023-11-28T11:24:16.287986+0000","last_unstale":"2023-11-28T11:24:16.294381+0000","last_undegraded":"2023-11-28T11:24:16.294381+0000","last_fullsized":"2023-11-28T11:24:16.294381+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:18:38.402023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"67'2","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.299641+0000","last_change":"2023-11-28T11:24:15.789630+0000","last_active":"2023-11-28T11:24:16.299641+0000","last_peered":"2023-11-28T11:24:16.299641+0000","last_clean":"2023-11-28T11:24:16.299641+0000","last_became_active":"2023-11-28T11:24:15.789316+0000","last_became_peered":"2023-11-28T11:24:15.789316+0000","last_unstale":"2023-11-28T11:24:16.299641+0000","last_undegraded":"2023-11-28T11:24:16.299641+0000","last_fullsized":"2023-11-28T11:24:16.299641+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:07:17.039552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":53,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:24:15.767262+0000","last_change":"2023-11-28T11:20:50.524251+0000","last_active":"2023-11-28T11:24:15.767262+0000","last_peered":"2023-11-28T11:24:15.767262+0000","last_clean":"2023-11-28T11:24:15.767262+0000","last_became_active":"2023-11-28T11:20:50.524012+0000","last_became_peered":"2023-11-28T11:20:50.524012+0000","last_unstale":"2023-11-28T11:24:15.767262+0000","last_undegraded":"2023-11-28T11:24:15.767262+0000","last_fullsized":"2023-11-28T11:24:15.767262+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:05.302498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.281215+0000","last_change":"2023-11-28T11:24:16.279088+0000","last_active":"2023-11-28T11:24:16.281215+0000","last_peered":"2023-11-28T11:24:16.281215+0000","last_clean":"2023-11-28T11:24:16.281215+0000","last_became_active":"2023-11-28T11:24:16.278776+0000","last_became_peered":"2023-11-28T11:24:16.278776+0000","last_unstale":"2023-11-28T11:24:16.281215+0000","last_undegraded":"2023-11-28T11:24:16.281215+0000","last_fullsized":"2023-11-28T11:24:16.281215+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:30:18.787935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"67'1","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.299205+0000","last_change":"2023-11-28T11:24:15.785442+0000","last_active":"2023-11-28T11:24:16.299205+0000","last_peered":"2023-11-28T11:24:16.299205+0000","last_clean":"2023-11-28T11:24:16.299205+0000","last_became_active":"2023-11-28T11:24:15.784801+0000","last_became_peered":"2023-11-28T11:24:15.784801+0000","last_unstale":"2023-11-28T11:24:16.299205+0000","last_undegraded":"2023-11-28T11:24:16.299205+0000","last_fullsized":"2023-11-28T11:24:16.299205+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:02:43.611821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":50,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:24:15.782093+0000","last_change":"2023-11-28T11:20:50.524838+0000","last_active":"2023-11-28T11:24:15.782093+0000","last_peered":"2023-11-28T11:24:15.782093+0000","last_clean":"2023-11-28T11:24:15.782093+0000","last_became_active":"2023-11-28T11:20:50.524402+0000","last_became_peered":"2023-11-28T11:20:50.524402+0000","last_unstale":"2023-11-28T11:24:15.782093+0000","last_undegraded":"2023-11-28T11:24:15.782093+0000","last_fullsized":"2023-11-28T11:24:15.782093+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:22.699447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574895,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10920,"kb_used_data":1376,"kb_used_omap":0,"kb_used_meta":9536,"kb_avail":94360920,"statfs":{"total":96636764160,"available":96625582080,"internally_reserved":0,"allocated":1409024,"data_stored":969001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9764864},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.621,"15min":0.623},"min":{"1min":0.41399999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.64700000000000002,"15min":0.64800000000000002},"min":{"1min":0.46500000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.56899999999999995}]},{"osd":1,"last update":"Tue Nov 28 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.64700000000000002,"15min":0.64400000000000002},"min":{"1min":0.54200000000000004,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81299999999999994,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.67200000000000004,"15min":0.67700000000000005},"min":{"1min":0.44500000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.625}]},{"osd":2,"last update":"Tue Nov 28 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.65200000000000002,"15min":0.65400000000000003},"min":{"1min":0.51000000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.71999999999999997,"15min":0.72199999999999998},"min":{"1min":0.60499999999999998,"5min":0.495,"15min":0.495},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.76800000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.65400000000000003,"15min":0.66200000000000003},"min":{"1min":0.45200000000000001,"5min":0.433,"15min":0.433},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.70199999999999996,"15min":0.70599999999999996},"min":{"1min":0.55800000000000005,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.96799999999999997,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.64600000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.63900000000000001,"15min":0.66400000000000003},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.89600000000000002,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.69499999999999995,"15min":0.70099999999999996},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91100000000000003,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.66900000000000004}]},{"osd":5,"last update":"Tue Nov 28 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67700000000000005,"15min":0.68400000000000005},"min":{"1min":0.40799999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.69999999999999996,"15min":0.71399999999999997},"min":{"1min":0.49099999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90300000000000002,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.73899999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.76400000000000001,"15min":0.78000000000000003},"min":{"1min":0.45000000000000001,"5min":0.435,"15min":0.435},"max":{"1min":1.0509999999999999,"5min":1.633,"15min":1.633},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.72599999999999998,"15min":0.752},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.85299999999999998,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.78600000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574895,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9640,"kb_used_data":800,"kb_used_omap":0,"kb_used_meta":8832,"kb_avail":94362200,"statfs":{"total":96636764160,"available":96626892800,"internally_reserved":0,"allocated":819200,"data_stored":378747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9043968},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.627,"15min":0.63700000000000001},"min":{"1min":0.45000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.93999999999999995,"5min":1.409,"15min":1.409},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.63500000000000001,"15min":0.63800000000000001},"min":{"1min":0.40999999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.98299999999999998,"5min":1.244,"15min":1.244},"last":0.76400000000000001}]},{"osd":1,"last update":"Tue Nov 28 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.629,"15min":0.627},"min":{"1min":0.44700000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.67100000000000004,"15min":0.68300000000000005},"min":{"1min":0.438,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.92600000000000005,"5min":1.038,"15min":1.038},"last":0.83499999999999996}]},{"osd":2,"last update":"Tue Nov 28 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.624,"15min":0.627},"min":{"1min":0.47499999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.746,"15min":0.77800000000000002},"min":{"1min":0.48199999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1,"5min":1.996,"15min":1.996},"last":0.86099999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.68899999999999995,"15min":0.69699999999999995},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.66600000000000004,"15min":0.68200000000000005},"min":{"1min":0.48699999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.96199999999999997,"5min":1.008,"15min":1.008},"last":0.63700000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.69699999999999995,"15min":0.72899999999999998},"min":{"1min":0.45100000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.86599999999999999,"5min":2.048,"15min":2.048},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.69799999999999995,"15min":0.72199999999999998},"min":{"1min":0.46300000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.873,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.93999999999999995}]},{"osd":5,"last update":"Tue Nov 28 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.66900000000000004,"15min":0.69699999999999995},"min":{"1min":0.42199999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.78200000000000003,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.77500000000000002,"15min":0.81499999999999995},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.89700000000000002,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":0.95899999999999996}]},{"osd":7,"last update":"Tue Nov 28 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.76000000000000001,"15min":0.77700000000000002},"min":{"1min":0.60299999999999998,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.86899999999999999,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":1.0009999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.77900000000000003,"15min":0.79800000000000004},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.018,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.88100000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574895,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9620,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":8832,"kb_avail":94362220,"statfs":{"total":96636764160,"available":96626913280,"internally_reserved":0,"allocated":798720,"data_stored":371207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9043968},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:23:46 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.44500000000000001,"15min":0.46300000000000002},"min":{"1min":0.29199999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.64200000000000002,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.45400000000000001,"15min":0.46000000000000002},"min":{"1min":0.32200000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.58899999999999997,"5min":0.999,"15min":0.999},"last":0.38400000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:23:46 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.496,"15min":0.51400000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.71199999999999997,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.49099999999999999,"15min":0.50600000000000001},"min":{"1min":0.35599999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.63,"5min":1.002,"15min":1.002},"last":0.54400000000000004}]},{"osd":3,"last update":"Tue Nov 28 11:23:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.53900000000000003,"15min":0.55400000000000005},"min":{"1min":0.40300000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.71899999999999997,"5min":1.198,"15min":1.198},"last":0.497},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51300000000000001,"15min":0.52100000000000002},"min":{"1min":0.34000000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.66400000000000003,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.51300000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:23:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.747,"15min":0.75700000000000001},"min":{"1min":0.59199999999999997,"5min":0.438,"15min":0.438},"max":{"1min":0.91600000000000004,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.74099999999999999,"15min":0.755},"min":{"1min":0.47299999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.99299999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.64900000000000002}]},{"osd":5,"last update":"Tue Nov 28 11:23:46 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.753,"15min":0.76900000000000002},"min":{"1min":0.54100000000000004,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.88800000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76000000000000001,"15min":0.77200000000000002},"min":{"1min":0.60399999999999998,"5min":0.5,"15min":0.5},"max":{"1min":0.97899999999999998,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.84499999999999997}]},{"osd":6,"last update":"Tue Nov 28 11:23:46 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.77500000000000002,"15min":0.78500000000000003},"min":{"1min":0.57699999999999996,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.98299999999999998,"5min":1.177,"15min":1.177},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.79000000000000004,"15min":0.80600000000000005},"min":{"1min":0.58899999999999997,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0329999999999999,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.73999999999999999}]},{"osd":7,"last update":"Tue Nov 28 11:23:46 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.754,"15min":0.76200000000000001},"min":{"1min":0.59799999999999998,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0680000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.752},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.79600000000000004,"15min":0.80400000000000005},"min":{"1min":0.64500000000000002,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.016,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.80100000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574895,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":15076,"kb_used_data":1372,"kb_used_omap":0,"kb_used_meta":13696,"kb_avail":94356764,"statfs":{"total":96636764160,"available":96621326336,"internally_reserved":0,"allocated":1404928,"data_stored":969023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14024704},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.434,"15min":0.44400000000000001},"min":{"1min":0.27300000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.59199999999999997,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.57199999999999995,"15min":0.63300000000000001},"min":{"1min":0.29999999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.58899999999999997,"5min":3.8420000000000001,"15min":3.8420000000000001},"last":0.44}]},{"osd":2,"last update":"Tue Nov 28 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.48799999999999999,"15min":0.504},"min":{"1min":0.36699999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.58799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.61799999999999999,"15min":0.67600000000000005},"min":{"1min":0.34300000000000003,"5min":0.311,"15min":0.311},"max":{"1min":0.71199999999999997,"5min":3.8940000000000001,"15min":3.8940000000000001},"last":0.52800000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.58399999999999996,"15min":0.63600000000000001},"min":{"1min":0.372,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.65600000000000003,"5min":3.681,"15min":3.681},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.56799999999999995,"15min":0.58999999999999997},"min":{"1min":0.42199999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.72699999999999998,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.54400000000000004}]},{"osd":4,"last update":"Tue Nov 28 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.85699999999999998,"15min":0.92100000000000004},"min":{"1min":0.54300000000000004,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.94599999999999995,"5min":3.9790000000000001,"15min":3.9790000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.88,"15min":0.94499999999999995},"min":{"1min":0.51300000000000001,"5min":0.497,"15min":0.497},"max":{"1min":0.999,"5min":3.5779999999999998,"15min":3.5779999999999998},"last":0.58099999999999996}]},{"osd":5,"last update":"Tue Nov 28 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.79600000000000004,"15min":0.82199999999999995},"min":{"1min":0.57299999999999995,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.96399999999999997,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.88800000000000001,"15min":0.96099999999999997},"min":{"1min":0.51500000000000001,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0449999999999999,"5min":3.9409999999999998,"15min":3.9409999999999998},"last":0.60999999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.878,"15min":0.94499999999999995},"min":{"1min":0.56399999999999995,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.92500000000000004,"5min":3.6160000000000001,"15min":3.6160000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.81599999999999995,"15min":0.84399999999999997},"min":{"1min":0.627,"5min":0.501,"15min":0.501},"max":{"1min":1.022,"5min":1.363,"15min":1.363},"last":0.70399999999999996}]},{"osd":7,"last update":"Tue Nov 28 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.94899999999999995,"15min":1.034},"min":{"1min":0.58399999999999996,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.94899999999999995,"5min":4.0099999999999998,"15min":4.0099999999999998},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.746,"5min":0.91500000000000004,"15min":0.99099999999999999},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0629999999999999,"5min":3.653,"15min":3.653},"last":0.68799999999999994}]}]},{"osd":2,"up_from":13,"seq":55834574893,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9492,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362348,"statfs":{"total":96636764160,"available":96627044352,"internally_reserved":0,"allocated":798720,"data_stored":371207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.50700000000000001,"15min":0.55700000000000005},"min":{"1min":0.314,"5min":0.219,"15min":0.219},"max":{"1min":0.61699999999999999,"5min":4.1109999999999998,"15min":4.1109999999999998},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.55200000000000005,"15min":0.59399999999999997},"min":{"1min":0.307,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.071,"5min":3.6230000000000002,"15min":3.6230000000000002},"last":0.47499999999999998}]},{"osd":1,"last update":"Tue Nov 28 11:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.64100000000000001,"15min":0.67600000000000005},"min":{"1min":0.38700000000000001,"5min":0.371,"15min":0.371},"max":{"1min":1.8779999999999999,"5min":3.6640000000000001,"15min":3.6640000000000001},"last":0.443},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.628,"15min":0.68999999999999995},"min":{"1min":0.40899999999999997,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.63100000000000001,"5min":4.0830000000000002,"15min":4.0830000000000002},"last":0.45900000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.59599999999999997,"15min":0.65100000000000002},"min":{"1min":0.34200000000000003,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.55100000000000005,"5min":4.1529999999999996,"15min":4.1529999999999996},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.56799999999999995,"15min":0.58499999999999996},"min":{"1min":0.40100000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.01,"5min":1.304,"15min":1.304},"last":0.39800000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.79200000000000004,"15min":0.81000000000000005},"min":{"1min":0.61499999999999999,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.107,"5min":1.649,"15min":1.649},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.747,"5min":0.85299999999999998,"15min":0.91500000000000004},"min":{"1min":0.63700000000000001,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.92100000000000004,"5min":4.0060000000000002,"15min":4.0060000000000002},"last":0.88800000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.86699999999999999,"15min":0.92700000000000005},"min":{"1min":0.61799999999999999,"5min":0.5,"15min":0.5},"max":{"1min":0.97999999999999998,"5min":4.0529999999999999,"15min":4.0529999999999999},"last":0.747},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.80000000000000004,"15min":0.81399999999999995},"min":{"1min":0.63300000000000001,"5min":0.497,"15min":0.497},"max":{"1min":0.97499999999999998,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.91700000000000004}]},{"osd":6,"last update":"Tue Nov 28 11:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.82199999999999995,"15min":0.84399999999999997},"min":{"1min":0.627,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.93000000000000005,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.84199999999999997,"15min":0.85899999999999999},"min":{"1min":0.65300000000000002,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.212,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.93100000000000005}]},{"osd":7,"last update":"Tue Nov 28 11:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.78800000000000003,"15min":0.80800000000000005},"min":{"1min":0.56699999999999995,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.996,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.81699999999999995,"15min":0.83299999999999996},"min":{"1min":0.65000000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.139,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.89800000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574894,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9748,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":8960,"kb_avail":94362092,"statfs":{"total":96636764160,"available":96626782208,"internally_reserved":0,"allocated":798720,"data_stored":371207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9175040},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:23:46 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.40400000000000003,"15min":0.40600000000000003},"min":{"1min":0.33300000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.52400000000000002,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.45900000000000002,"15min":0.46100000000000002},"min":{"1min":0.32000000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.72599999999999998,"5min":1.157,"15min":1.157},"last":0.40699999999999997}]},{"osd":1,"last update":"Tue Nov 28 11:23:46 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.54000000000000004,"15min":0.58199999999999996},"min":{"1min":0.34200000000000003,"5min":0.313,"15min":0.313},"max":{"1min":0.57999999999999996,"5min":4.8019999999999996,"15min":4.8019999999999996},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.498,"5min":0.55700000000000005,"15min":0.59499999999999997},"min":{"1min":0.38700000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.67400000000000004,"5min":4.7539999999999996,"15min":4.7539999999999996},"last":0.38200000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:23:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.61599999999999999,"15min":0.66200000000000003},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.69099999999999995,"5min":4.8419999999999996,"15min":4.8419999999999996},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.61299999999999999,"15min":0.65700000000000003},"min":{"1min":0.38700000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.64600000000000002,"5min":4.7919999999999998,"15min":4.7919999999999998},"last":0.34999999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:23:46 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.79800000000000004,"15min":0.83999999999999997},"min":{"1min":0.51000000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.90900000000000003,"5min":4.8869999999999996,"15min":4.8869999999999996},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.84699999999999998,"15min":0.90300000000000002},"min":{"1min":0.59199999999999997,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.90900000000000003,"5min":5.2460000000000004,"15min":5.2460000000000004},"last":0.64700000000000002}]},{"osd":5,"last update":"Tue Nov 28 11:23:46 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.86099999999999999,"15min":0.91500000000000004},"min":{"1min":0.54100000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.93600000000000005,"5min":5.0759999999999996,"15min":5.0759999999999996},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.746,"5min":0.82799999999999996,"15min":0.874},"min":{"1min":0.55400000000000005,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.94299999999999995,"5min":5.3010000000000002,"15min":5.3010000000000002},"last":0.51300000000000001}]},{"osd":6,"last update":"Tue Nov 28 11:23:46 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.84299999999999997,"15min":0.88700000000000001},"min":{"1min":0.55300000000000005,"5min":0.501,"15min":0.501},"max":{"1min":0.84999999999999998,"5min":5.3399999999999999,"15min":5.3399999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.83299999999999996,"15min":0.872},"min":{"1min":0.53900000000000003,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.95299999999999996,"5min":4.8330000000000002,"15min":4.8330000000000002},"last":0.60599999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:23:46 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.83799999999999997,"15min":0.88},"min":{"1min":0.59399999999999997,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.89400000000000002,"5min":4.8600000000000003,"15min":4.8600000000000003},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.754,"5min":0.88100000000000001,"15min":0.93899999999999995},"min":{"1min":0.56499999999999995,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.89500000000000002,"5min":5.1909999999999998,"15min":5.1909999999999998},"last":0.58199999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574894,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9940,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361900,"statfs":{"total":96636764160,"available":96626585600,"internally_reserved":0,"allocated":798720,"data_stored":371207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.67200000000000004,"15min":0.68300000000000005},"min":{"1min":0.45200000000000001,"5min":0.437,"15min":0.437},"max":{"1min":0.85899999999999999,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.66600000000000004,"15min":0.66200000000000003},"min":{"1min":0.48699999999999999,"5min":0.438,"15min":0.438},"max":{"1min":1.0509999999999999,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.439}]},{"osd":1,"last update":"Tue Nov 28 11:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.66400000000000003,"15min":0.64900000000000002},"min":{"1min":0.56299999999999994,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.69499999999999995,"15min":0.70499999999999996},"min":{"1min":0.50800000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.91400000000000003,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.73699999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73399999999999999,"15min":0.74299999999999999},"min":{"1min":0.56599999999999995,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.93799999999999994,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.70299999999999996,"15min":0.69899999999999995},"min":{"1min":0.54800000000000004,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.68700000000000006}]},{"osd":3,"last update":"Tue Nov 28 11:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.67800000000000005,"15min":0.68100000000000005},"min":{"1min":0.498,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88600000000000001,"5min":1.179,"15min":1.179},"last":0.498},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.76500000000000001,"15min":0.77900000000000003},"min":{"1min":0.51300000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.90600000000000003,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.66800000000000004}]},{"osd":5,"last update":"Tue Nov 28 11:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.67700000000000005,"15min":0.68700000000000006},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.81699999999999995,"5min":1.161,"15min":1.161},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.65800000000000003,"15min":0.66400000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.83899999999999997,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.56499999999999995}]},{"osd":6,"last update":"Tue Nov 28 11:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.70699999999999996,"15min":0.72899999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.79400000000000004,"5min":1.514,"15min":1.514},"last":0.622},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.71999999999999997,"15min":0.73399999999999999},"min":{"1min":0.44800000000000001,"5min":0.439,"15min":0.439},"max":{"1min":0.84599999999999997,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.80600000000000005}]},{"osd":7,"last update":"Tue Nov 28 11:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.69799999999999995,"15min":0.70199999999999996},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.92300000000000004,"5min":1.226,"15min":1.226},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.80700000000000005,"15min":0.83399999999999996},"min":{"1min":0.58099999999999996,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.94499999999999995,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":0.71899999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9700,"kb_used_data":796,"kb_used_omap":0,"kb_used_meta":8896,"kb_avail":94362140,"statfs":{"total":96636764160,"available":96626831360,"internally_reserved":0,"allocated":815104,"data_stored":378745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9109504},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:23:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.63800000000000001,"15min":0.628},"min":{"1min":0.38400000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.0109999999999999,"5min":1.032,"15min":1.032},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.69499999999999995,"15min":0.69799999999999995},"min":{"1min":0.50700000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.90100000000000002,"5min":1.02,"15min":1.02},"last":0.65200000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:23:48 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.66800000000000004,"15min":0.65900000000000003},"min":{"1min":0.51700000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.90200000000000002,"5min":1.355,"15min":1.355},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.69499999999999995,"15min":0.69699999999999995},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.111,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.68300000000000005}]},{"osd":2,"last update":"Tue Nov 28 11:23:48 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.69399999999999995,"15min":0.69799999999999995},"min":{"1min":0.45400000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.93500000000000005,"5min":1.198,"15min":1.198},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.72499999999999998,"15min":0.73299999999999998},"min":{"1min":0.51300000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.92500000000000004,"5min":1.387,"15min":1.387},"last":0.96199999999999997}]},{"osd":3,"last update":"Tue Nov 28 11:23:48 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.74299999999999999,"15min":0.754},"min":{"1min":0.503,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.71999999999999997,"15min":0.72399999999999998},"min":{"1min":0.57399999999999995,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.88400000000000001,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.85499999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:23:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.70599999999999996,"15min":0.72399999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":2.5139999999999998,"5min":2.5139999999999998,"15min":2.5139999999999998},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.72599999999999998,"15min":0.73699999999999999},"min":{"1min":0.497,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.91800000000000004,"5min":1.198,"15min":1.198},"last":0.89100000000000001}]},{"osd":6,"last update":"Tue Nov 28 11:23:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.72599999999999998,"15min":0.73399999999999999},"min":{"1min":0.51600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0589999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.73399999999999999,"15min":0.73199999999999998},"min":{"1min":0.52700000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":2.4529999999999998,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":0.879}]},{"osd":7,"last update":"Tue Nov 28 11:23:48 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.80100000000000005,"15min":0.82199999999999995},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.9059999999999999,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.80200000000000005,"15min":0.82399999999999995},"min":{"1min":0.57799999999999996,"5min":0.496,"15min":0.496},"max":{"1min":1.1699999999999999,"5min":1.298,"15min":1.298},"last":0.98599999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:24:20.981 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:24:21.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.209+0000 7fcdb41ff700 1 Processor -- start 2023-11-28T11:24:21.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.209+0000 7fcdb41ff700 1 -- start start 2023-11-28T11:24:21.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.210+0000 7fcdb41ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcdac0b8510 0x7fcdac06b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:21.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.210+0000 7fcdb41ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdac0cd460 0x7fcdac06beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:21.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.210+0000 7fcdb41ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fcdac06c3f0 0x7fcdac15cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:21.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.210+0000 7fcdb41ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdac0cdbd0 con 0x7fcdac0b8510 2023-11-28T11:24:21.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.210+0000 7fcdb41ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdac0d02d0 con 0x7fcdac06c3f0 2023-11-28T11:24:21.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.210+0000 7fcdb41ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdac0b88e0 con 0x7fcdac0cd460 2023-11-28T11:24:21.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.210+0000 7fcdb1f9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcdac0b8510 0x7fcdac06b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:21.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.211+0000 7fcdb1f9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcdac0b8510 0x7fcdac06b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55190/0 (socket says 172.21.15.92:55190) 2023-11-28T11:24:21.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.211+0000 7fcdb1f9b700 1 -- 172.21.15.92:0/198265126 learned_addr learned my addr 172.21.15.92:0/198265126 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:21.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.211+0000 7fcdb279c700 1 --2- 172.21.15.92:0/198265126 >> v2:172.21.15.169:3300/0 conn(0x7fcdac06c3f0 0x7fcdac15cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:21.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.211+0000 7fcdb179a700 1 --2- 172.21.15.92:0/198265126 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdac0cd460 0x7fcdac06beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:21.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.211+0000 7fcdb1f9b700 1 -- 172.21.15.92:0/198265126 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdac0cd460 msgr2=0x7fcdac06beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:21.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.211+0000 7fcdb1f9b700 1 --2- 172.21.15.92:0/198265126 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdac0cd460 0x7fcdac06beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:21.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.211+0000 7fcdb1f9b700 1 -- 172.21.15.92:0/198265126 >> v2:172.21.15.169:3300/0 conn(0x7fcdac06c3f0 msgr2=0x7fcdac15cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:21.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.211+0000 7fcdb1f9b700 1 --2- 172.21.15.92:0/198265126 >> v2:172.21.15.169:3300/0 conn(0x7fcdac06c3f0 0x7fcdac15cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:21.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.211+0000 7fcdb1f9b700 1 -- 172.21.15.92:0/198265126 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdac0cd8f0 con 0x7fcdac0b8510 2023-11-28T11:24:21.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.211+0000 7fcdb179a700 1 --2- 172.21.15.92:0/198265126 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdac0cd460 0x7fcdac06beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:24:21.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.211+0000 7fcdb1f9b700 1 --2- 172.21.15.92:0/198265126 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcdac0b8510 0x7fcdac06b970 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fcdac0b4c80 tx=0x7fcda8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce164274f67478b4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:21.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.211+0000 7fcdb0f99700 1 -- 172.21.15.92:0/198265126 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda8011830 con 0x7fcdac0b8510 2023-11-28T11:24:21.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.212+0000 7fcdb0f99700 1 -- 172.21.15.92:0/198265126 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcda8011990 con 0x7fcdac0b8510 2023-11-28T11:24:21.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.212+0000 7fcdb0f99700 1 -- 172.21.15.92:0/198265126 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda8011d20 con 0x7fcdac0b8510 2023-11-28T11:24:21.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.212+0000 7fcdb41ff700 1 -- 172.21.15.92:0/198265126 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcdac0b8510 msgr2=0x7fcdac06b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:21.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.212+0000 7fcdb41ff700 1 --2- 172.21.15.92:0/198265126 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcdac0b8510 0x7fcdac06b970 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fcdac0b4c80 tx=0x7fcda8009c00 comp rx=0 tx=0).stop 2023-11-28T11:24:21.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.212+0000 7fcdb41ff700 1 -- 172.21.15.92:0/198265126 shutdown_connections 2023-11-28T11:24:21.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.212+0000 7fcdb41ff700 1 --2- 172.21.15.92:0/198265126 >> v2:172.21.15.169:3300/0 conn(0x7fcdac06c3f0 0x7fcdac15cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:21.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.212+0000 7fcdb41ff700 1 --2- 172.21.15.92:0/198265126 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcdac0b8510 0x7fcdac06b970 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:21.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.212+0000 7fcdb41ff700 1 --2- 172.21.15.92:0/198265126 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdac0cd460 0x7fcdac06beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:21.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.212+0000 7fcdb41ff700 1 -- 172.21.15.92:0/198265126 >> 172.21.15.92:0/198265126 conn(0x7fcdac0719a0 msgr2=0x7fcdac071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:21.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.212+0000 7fcdb41ff700 1 -- 172.21.15.92:0/198265126 shutdown_connections 2023-11-28T11:24:21.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.212+0000 7fcdb41ff700 1 -- 172.21.15.92:0/198265126 wait complete. 2023-11-28T11:24:21.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.214+0000 7fcdb41ff700 1 Processor -- start 2023-11-28T11:24:21.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.214+0000 7fcdb41ff700 1 -- start start 2023-11-28T11:24:21.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.215+0000 7fcdb41ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcdac06c3f0 0x7fcdac1564c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:21.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.215+0000 7fcdb41ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdac0b8510 0x7fcdac0cccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:21.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.215+0000 7fcdb41ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdac0cd460 0x7fcdac155710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:21.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.215+0000 7fcdb41ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdac156cf0 con 0x7fcdac06c3f0 2023-11-28T11:24:21.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.215+0000 7fcdb41ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdac0cc840 con 0x7fcdac0b8510 2023-11-28T11:24:21.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.215+0000 7fcdb41ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdac0cb950 con 0x7fcdac0cd460 2023-11-28T11:24:21.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.215+0000 7fcdb1f9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcdac06c3f0 0x7fcdac1564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:21.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.215+0000 7fcdb1f9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcdac06c3f0 0x7fcdac1564c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55204/0 (socket says 172.21.15.92:55204) 2023-11-28T11:24:21.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.215+0000 7fcdb1f9b700 1 -- 172.21.15.92:0/3555510378 learned_addr learned my addr 172.21.15.92:0/3555510378 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:21.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.215+0000 7fcdb179a700 1 --2- 172.21.15.92:0/3555510378 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdac0b8510 0x7fcdac0cccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:21.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.215+0000 7fcdb279c700 1 --2- 172.21.15.92:0/3555510378 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdac0cd460 0x7fcdac155710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:21.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.216+0000 7fcdb1f9b700 1 -- 172.21.15.92:0/3555510378 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdac0cd460 msgr2=0x7fcdac155710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:21.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.216+0000 7fcdb1f9b700 1 --2- 172.21.15.92:0/3555510378 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdac0cd460 0x7fcdac155710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:21.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.216+0000 7fcdb1f9b700 1 -- 172.21.15.92:0/3555510378 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdac0b8510 msgr2=0x7fcdac0cccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:21.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.216+0000 7fcdb1f9b700 1 --2- 172.21.15.92:0/3555510378 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdac0b8510 0x7fcdac0cccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:21.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.216+0000 7fcdb1f9b700 1 -- 172.21.15.92:0/3555510378 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda8009550 con 0x7fcdac06c3f0 2023-11-28T11:24:21.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.216+0000 7fcdb1f9b700 1 --2- 172.21.15.92:0/3555510378 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcdac06c3f0 0x7fcdac1564c0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fcda8009c40 tx=0x7fcda8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:21.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.216+0000 7fcda27fc700 1 -- 172.21.15.92:0/3555510378 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda8004300 con 0x7fcdac06c3f0 2023-11-28T11:24:21.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.216+0000 7fcda27fc700 1 -- 172.21.15.92:0/3555510378 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcda801f070 con 0x7fcdac06c3f0 2023-11-28T11:24:21.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.216+0000 7fcda27fc700 1 -- 172.21.15.92:0/3555510378 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda80036a0 con 0x7fcdac06c3f0 2023-11-28T11:24:21.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.216+0000 7fcdb41ff700 1 -- 172.21.15.92:0/3555510378 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdac0cfff0 con 0x7fcdac06c3f0 2023-11-28T11:24:21.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.216+0000 7fcdb41ff700 1 -- 172.21.15.92:0/3555510378 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdac0cfd10 con 0x7fcdac06c3f0 2023-11-28T11:24:21.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.217+0000 7fcdb41ff700 1 -- 172.21.15.92:0/3555510378 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdac067880 con 0x7fcdac06c3f0 2023-11-28T11:24:21.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.218+0000 7fcda27fc700 1 -- 172.21.15.92:0/3555510378 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcda802a020 con 0x7fcdac06c3f0 2023-11-28T11:24:21.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.219+0000 7fcda27fc700 1 --2- 172.21.15.92:0/3555510378 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fcd900682f0 0x7fcd9006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:21.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.219+0000 7fcda27fc700 1 -- 172.21.15.92:0/3555510378 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fcda8087350 con 0x7fcdac06c3f0 2023-11-28T11:24:21.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.219+0000 7fcdb179a700 1 --2- 172.21.15.92:0/3555510378 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fcd900682f0 0x7fcd9006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:21.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.220+0000 7fcdb179a700 1 --2- 172.21.15.92:0/3555510378 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fcd900682f0 0x7fcd9006a7a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fcdac1e1de0 tx=0x7fcd9c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:21.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.223+0000 7fcda27fc700 1 -- 172.21.15.92:0/3555510378 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fcda8053980 con 0x7fcdac06c3f0 2023-11-28T11:24:21.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.408+0000 7fcdb41ff700 1 -- 172.21.15.92:0/3555510378 --> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcdac0d2940 con 0x7fcd900682f0 2023-11-28T11:24:21.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.417+0000 7fcda27fc700 1 -- 172.21.15.92:0/3555510378 <== mgr.4103 v2:172.21.15.92:6832/115482 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311210 (secure 0 0 0) 0x7fcdac0d2940 con 0x7fcd900682f0 2023-11-28T11:24:21.421 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:24:21.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.421+0000 7fcdb41ff700 1 -- 172.21.15.92:0/3555510378 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fcd900682f0 msgr2=0x7fcd9006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:21.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.421+0000 7fcdb41ff700 1 --2- 172.21.15.92:0/3555510378 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fcd900682f0 0x7fcd9006a7a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fcdac1e1de0 tx=0x7fcd9c008040 comp rx=0 tx=0).stop 2023-11-28T11:24:21.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.421+0000 7fcdb41ff700 1 -- 172.21.15.92:0/3555510378 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcdac06c3f0 msgr2=0x7fcdac1564c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:21.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.421+0000 7fcdb41ff700 1 --2- 172.21.15.92:0/3555510378 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcdac06c3f0 0x7fcdac1564c0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fcda8009c40 tx=0x7fcda8004750 comp rx=0 tx=0).stop 2023-11-28T11:24:21.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.421+0000 7fcdb41ff700 1 -- 172.21.15.92:0/3555510378 shutdown_connections 2023-11-28T11:24:21.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.421+0000 7fcdb41ff700 1 --2- 172.21.15.92:0/3555510378 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fcdac0cd460 0x7fcdac155710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:21.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.421+0000 7fcdb41ff700 1 --2- 172.21.15.92:0/3555510378 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fcd900682f0 0x7fcd9006a7a0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:21.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.421+0000 7fcdb41ff700 1 --2- 172.21.15.92:0/3555510378 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcdac06c3f0 0x7fcdac1564c0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:21.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.421+0000 7fcdb41ff700 1 --2- 172.21.15.92:0/3555510378 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fcdac0b8510 0x7fcdac0cccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:21.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.421+0000 7fcdb41ff700 1 -- 172.21.15.92:0/3555510378 >> 172.21.15.92:0/3555510378 conn(0x7fcdac0719a0 msgr2=0x7fcdac0b4260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:21.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.421+0000 7fcdb41ff700 1 -- 172.21.15.92:0/3555510378 shutdown_connections 2023-11-28T11:24:21.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.421+0000 7fcdb41ff700 1 -- 172.21.15.92:0/3555510378 wait complete. 2023-11-28T11:24:21.425 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-11-28T11:24:21.440 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":170,"stamp":"2023-11-28T11:24:19.903602+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":44,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":84136,"kb_used_data":7464,"kb_used_omap":0,"kb_used_meta":76608,"kb_avail":754890584,"statfs":{"total":773094113280,"available":773007958016,"internally_reserved":0,"allocated":7643136,"data_stored":4180344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":78446592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.142274"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.294401+0000","last_change":"2023-11-28T11:24:16.289334+0000","last_active":"2023-11-28T11:24:16.294401+0000","last_peered":"2023-11-28T11:24:16.294401+0000","last_clean":"2023-11-28T11:24:16.294401+0000","last_became_active":"2023-11-28T11:24:16.289053+0000","last_became_peered":"2023-11-28T11:24:16.289053+0000","last_unstale":"2023-11-28T11:24:16.294401+0000","last_undegraded":"2023-11-28T11:24:16.294401+0000","last_fullsized":"2023-11-28T11:24:16.294401+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:52:10.685572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.301703+0000","last_change":"2023-11-28T11:24:16.293787+0000","last_active":"2023-11-28T11:24:16.301703+0000","last_peered":"2023-11-28T11:24:16.301703+0000","last_clean":"2023-11-28T11:24:16.301703+0000","last_became_active":"2023-11-28T11:24:16.292360+0000","last_became_peered":"2023-11-28T11:24:16.292360+0000","last_unstale":"2023-11-28T11:24:16.301703+0000","last_undegraded":"2023-11-28T11:24:16.301703+0000","last_fullsized":"2023-11-28T11:24:16.301703+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:38.876001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.293954+0000","last_change":"2023-11-28T11:24:16.284598+0000","last_active":"2023-11-28T11:24:16.293954+0000","last_peered":"2023-11-28T11:24:16.293954+0000","last_clean":"2023-11-28T11:24:16.293954+0000","last_became_active":"2023-11-28T11:24:16.284338+0000","last_became_peered":"2023-11-28T11:24:16.284338+0000","last_unstale":"2023-11-28T11:24:16.293954+0000","last_undegraded":"2023-11-28T11:24:16.293954+0000","last_fullsized":"2023-11-28T11:24:16.293954+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:22:37.183147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.301671+0000","last_change":"2023-11-28T11:24:16.297775+0000","last_active":"2023-11-28T11:24:16.301671+0000","last_peered":"2023-11-28T11:24:16.301671+0000","last_clean":"2023-11-28T11:24:16.301671+0000","last_became_active":"2023-11-28T11:24:16.297488+0000","last_became_peered":"2023-11-28T11:24:16.297488+0000","last_unstale":"2023-11-28T11:24:16.301671+0000","last_undegraded":"2023-11-28T11:24:16.301671+0000","last_fullsized":"2023-11-28T11:24:16.301671+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:38.876001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.293181+0000","last_change":"2023-11-28T11:24:16.287597+0000","last_active":"2023-11-28T11:24:16.293181+0000","last_peered":"2023-11-28T11:24:16.293181+0000","last_clean":"2023-11-28T11:24:16.293181+0000","last_became_active":"2023-11-28T11:24:16.287344+0000","last_became_peered":"2023-11-28T11:24:16.287344+0000","last_unstale":"2023-11-28T11:24:16.293181+0000","last_undegraded":"2023-11-28T11:24:16.293181+0000","last_fullsized":"2023-11-28T11:24:16.293181+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:03:57.216059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.301975+0000","last_change":"2023-11-28T11:24:16.297616+0000","last_active":"2023-11-28T11:24:16.301975+0000","last_peered":"2023-11-28T11:24:16.301975+0000","last_clean":"2023-11-28T11:24:16.301975+0000","last_became_active":"2023-11-28T11:24:16.296978+0000","last_became_peered":"2023-11-28T11:24:16.296978+0000","last_unstale":"2023-11-28T11:24:16.301975+0000","last_undegraded":"2023-11-28T11:24:16.301975+0000","last_fullsized":"2023-11-28T11:24:16.301975+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:26:02.103744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.284542+0000","last_change":"2023-11-28T11:24:16.276770+0000","last_active":"2023-11-28T11:24:16.284542+0000","last_peered":"2023-11-28T11:24:16.284542+0000","last_clean":"2023-11-28T11:24:16.284542+0000","last_became_active":"2023-11-28T11:24:16.275880+0000","last_became_peered":"2023-11-28T11:24:16.275880+0000","last_unstale":"2023-11-28T11:24:16.284542+0000","last_undegraded":"2023-11-28T11:24:16.284542+0000","last_fullsized":"2023-11-28T11:24:16.284542+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:54:23.138456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.302006+0000","last_change":"2023-11-28T11:24:16.296181+0000","last_active":"2023-11-28T11:24:16.302006+0000","last_peered":"2023-11-28T11:24:16.302006+0000","last_clean":"2023-11-28T11:24:16.302006+0000","last_became_active":"2023-11-28T11:24:16.295911+0000","last_became_peered":"2023-11-28T11:24:16.295911+0000","last_unstale":"2023-11-28T11:24:16.302006+0000","last_undegraded":"2023-11-28T11:24:16.302006+0000","last_fullsized":"2023-11-28T11:24:16.302006+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:30:53.256213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.294435+0000","last_change":"2023-11-28T11:24:16.290462+0000","last_active":"2023-11-28T11:24:16.294435+0000","last_peered":"2023-11-28T11:24:16.294435+0000","last_clean":"2023-11-28T11:24:16.294435+0000","last_became_active":"2023-11-28T11:24:16.289602+0000","last_became_peered":"2023-11-28T11:24:16.289602+0000","last_unstale":"2023-11-28T11:24:16.294435+0000","last_undegraded":"2023-11-28T11:24:16.294435+0000","last_fullsized":"2023-11-28T11:24:16.294435+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:38:12.535382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.299735+0000","last_change":"2023-11-28T11:24:16.291626+0000","last_active":"2023-11-28T11:24:16.299735+0000","last_peered":"2023-11-28T11:24:16.299735+0000","last_clean":"2023-11-28T11:24:16.299735+0000","last_became_active":"2023-11-28T11:24:16.291295+0000","last_became_peered":"2023-11-28T11:24:16.291295+0000","last_unstale":"2023-11-28T11:24:16.299735+0000","last_undegraded":"2023-11-28T11:24:16.299735+0000","last_fullsized":"2023-11-28T11:24:16.299735+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:42:33.731752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.290787+0000","last_change":"2023-11-28T11:24:16.286111+0000","last_active":"2023-11-28T11:24:16.290787+0000","last_peered":"2023-11-28T11:24:16.290787+0000","last_clean":"2023-11-28T11:24:16.290787+0000","last_became_active":"2023-11-28T11:24:16.284625+0000","last_became_peered":"2023-11-28T11:24:16.284625+0000","last_unstale":"2023-11-28T11:24:16.290787+0000","last_undegraded":"2023-11-28T11:24:16.290787+0000","last_fullsized":"2023-11-28T11:24:16.290787+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:07:18.046315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.284841+0000","last_change":"2023-11-28T11:24:16.279931+0000","last_active":"2023-11-28T11:24:16.284841+0000","last_peered":"2023-11-28T11:24:16.284841+0000","last_clean":"2023-11-28T11:24:16.284841+0000","last_became_active":"2023-11-28T11:24:16.279559+0000","last_became_peered":"2023-11-28T11:24:16.279559+0000","last_unstale":"2023-11-28T11:24:16.284841+0000","last_undegraded":"2023-11-28T11:24:16.284841+0000","last_fullsized":"2023-11-28T11:24:16.284841+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:00:11.407649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.304103+0000","last_change":"2023-11-28T11:24:16.300273+0000","last_active":"2023-11-28T11:24:16.304103+0000","last_peered":"2023-11-28T11:24:16.304103+0000","last_clean":"2023-11-28T11:24:16.304103+0000","last_became_active":"2023-11-28T11:24:16.300014+0000","last_became_peered":"2023-11-28T11:24:16.300014+0000","last_unstale":"2023-11-28T11:24:16.304103+0000","last_undegraded":"2023-11-28T11:24:16.304103+0000","last_fullsized":"2023-11-28T11:24:16.304103+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:01:22.073242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.287299+0000","last_change":"2023-11-28T11:24:16.282970+0000","last_active":"2023-11-28T11:24:16.287299+0000","last_peered":"2023-11-28T11:24:16.287299+0000","last_clean":"2023-11-28T11:24:16.287299+0000","last_became_active":"2023-11-28T11:24:16.282388+0000","last_became_peered":"2023-11-28T11:24:16.282388+0000","last_unstale":"2023-11-28T11:24:16.287299+0000","last_undegraded":"2023-11-28T11:24:16.287299+0000","last_fullsized":"2023-11-28T11:24:16.287299+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:14:11.685643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.293714+0000","last_change":"2023-11-28T11:24:16.288932+0000","last_active":"2023-11-28T11:24:16.293714+0000","last_peered":"2023-11-28T11:24:16.293714+0000","last_clean":"2023-11-28T11:24:16.293714+0000","last_became_active":"2023-11-28T11:24:16.288281+0000","last_became_peered":"2023-11-28T11:24:16.288281+0000","last_unstale":"2023-11-28T11:24:16.293714+0000","last_undegraded":"2023-11-28T11:24:16.293714+0000","last_fullsized":"2023-11-28T11:24:16.293714+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:44:11.394584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.300392+0000","last_change":"2023-11-28T11:24:16.297105+0000","last_active":"2023-11-28T11:24:16.300392+0000","last_peered":"2023-11-28T11:24:16.300392+0000","last_clean":"2023-11-28T11:24:16.300392+0000","last_became_active":"2023-11-28T11:24:16.296505+0000","last_became_peered":"2023-11-28T11:24:16.296505+0000","last_unstale":"2023-11-28T11:24:16.300392+0000","last_undegraded":"2023-11-28T11:24:16.300392+0000","last_fullsized":"2023-11-28T11:24:16.300392+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:44:11.394584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.290617+0000","last_change":"2023-11-28T11:24:16.285467+0000","last_active":"2023-11-28T11:24:16.290617+0000","last_peered":"2023-11-28T11:24:16.290617+0000","last_clean":"2023-11-28T11:24:16.290617+0000","last_became_active":"2023-11-28T11:24:16.284615+0000","last_became_peered":"2023-11-28T11:24:16.284615+0000","last_unstale":"2023-11-28T11:24:16.290617+0000","last_undegraded":"2023-11-28T11:24:16.290617+0000","last_fullsized":"2023-11-28T11:24:16.290617+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:18:15.187665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.289046+0000","last_change":"2023-11-28T11:24:16.283340+0000","last_active":"2023-11-28T11:24:16.289046+0000","last_peered":"2023-11-28T11:24:16.289046+0000","last_clean":"2023-11-28T11:24:16.289046+0000","last_became_active":"2023-11-28T11:24:16.283065+0000","last_became_peered":"2023-11-28T11:24:16.283065+0000","last_unstale":"2023-11-28T11:24:16.289046+0000","last_undegraded":"2023-11-28T11:24:16.289046+0000","last_fullsized":"2023-11-28T11:24:16.289046+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:44:50.050157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.302012+0000","last_change":"2023-11-28T11:24:16.295998+0000","last_active":"2023-11-28T11:24:16.302012+0000","last_peered":"2023-11-28T11:24:16.302012+0000","last_clean":"2023-11-28T11:24:16.302012+0000","last_became_active":"2023-11-28T11:24:16.295306+0000","last_became_peered":"2023-11-28T11:24:16.295306+0000","last_unstale":"2023-11-28T11:24:16.302012+0000","last_undegraded":"2023-11-28T11:24:16.302012+0000","last_fullsized":"2023-11-28T11:24:16.302012+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:15:09.698174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.294639+0000","last_change":"2023-11-28T11:24:16.291575+0000","last_active":"2023-11-28T11:24:16.294639+0000","last_peered":"2023-11-28T11:24:16.294639+0000","last_clean":"2023-11-28T11:24:16.294639+0000","last_became_active":"2023-11-28T11:24:16.291306+0000","last_became_peered":"2023-11-28T11:24:16.291306+0000","last_unstale":"2023-11-28T11:24:16.294639+0000","last_undegraded":"2023-11-28T11:24:16.294639+0000","last_fullsized":"2023-11-28T11:24:16.294639+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:42:33.731752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.295535+0000","last_change":"2023-11-28T11:24:16.289872+0000","last_active":"2023-11-28T11:24:16.295535+0000","last_peered":"2023-11-28T11:24:16.295535+0000","last_clean":"2023-11-28T11:24:16.295535+0000","last_became_active":"2023-11-28T11:24:16.285288+0000","last_became_peered":"2023-11-28T11:24:16.285288+0000","last_unstale":"2023-11-28T11:24:16.295535+0000","last_undegraded":"2023-11-28T11:24:16.295535+0000","last_fullsized":"2023-11-28T11:24:16.295535+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:24:52.490566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.300098+0000","last_change":"2023-11-28T11:24:16.290820+0000","last_active":"2023-11-28T11:24:16.300098+0000","last_peered":"2023-11-28T11:24:16.300098+0000","last_clean":"2023-11-28T11:24:16.300098+0000","last_became_active":"2023-11-28T11:24:16.290412+0000","last_became_peered":"2023-11-28T11:24:16.290412+0000","last_unstale":"2023-11-28T11:24:16.300098+0000","last_undegraded":"2023-11-28T11:24:16.300098+0000","last_fullsized":"2023-11-28T11:24:16.300098+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:53:17.454560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.286848+0000","last_change":"2023-11-28T11:24:16.282351+0000","last_active":"2023-11-28T11:24:16.286848+0000","last_peered":"2023-11-28T11:24:16.286848+0000","last_clean":"2023-11-28T11:24:16.286848+0000","last_became_active":"2023-11-28T11:24:16.282091+0000","last_became_peered":"2023-11-28T11:24:16.282091+0000","last_unstale":"2023-11-28T11:24:16.286848+0000","last_undegraded":"2023-11-28T11:24:16.286848+0000","last_fullsized":"2023-11-28T11:24:16.286848+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:04:41.624812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.302246+0000","last_change":"2023-11-28T11:24:16.299736+0000","last_active":"2023-11-28T11:24:16.302246+0000","last_peered":"2023-11-28T11:24:16.302246+0000","last_clean":"2023-11-28T11:24:16.302246+0000","last_became_active":"2023-11-28T11:24:16.299400+0000","last_became_peered":"2023-11-28T11:24:16.299400+0000","last_unstale":"2023-11-28T11:24:16.302246+0000","last_undegraded":"2023-11-28T11:24:16.302246+0000","last_fullsized":"2023-11-28T11:24:16.302246+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:57:13.973827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.299839+0000","last_change":"2023-11-28T11:24:16.293562+0000","last_active":"2023-11-28T11:24:16.299839+0000","last_peered":"2023-11-28T11:24:16.299839+0000","last_clean":"2023-11-28T11:24:16.299839+0000","last_became_active":"2023-11-28T11:24:16.291284+0000","last_became_peered":"2023-11-28T11:24:16.291284+0000","last_unstale":"2023-11-28T11:24:16.299839+0000","last_undegraded":"2023-11-28T11:24:16.299839+0000","last_fullsized":"2023-11-28T11:24:16.299839+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:49:11.229814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.290228+0000","last_change":"2023-11-28T11:24:16.287559+0000","last_active":"2023-11-28T11:24:16.290228+0000","last_peered":"2023-11-28T11:24:16.290228+0000","last_clean":"2023-11-28T11:24:16.290228+0000","last_became_active":"2023-11-28T11:24:16.285911+0000","last_became_peered":"2023-11-28T11:24:16.285911+0000","last_unstale":"2023-11-28T11:24:16.290228+0000","last_undegraded":"2023-11-28T11:24:16.290228+0000","last_fullsized":"2023-11-28T11:24:16.290228+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:12:10.613903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.291884+0000","last_change":"2023-11-28T11:24:16.282634+0000","last_active":"2023-11-28T11:24:16.291884+0000","last_peered":"2023-11-28T11:24:16.291884+0000","last_clean":"2023-11-28T11:24:16.291884+0000","last_became_active":"2023-11-28T11:24:16.282323+0000","last_became_peered":"2023-11-28T11:24:16.282323+0000","last_unstale":"2023-11-28T11:24:16.291884+0000","last_undegraded":"2023-11-28T11:24:16.291884+0000","last_fullsized":"2023-11-28T11:24:16.291884+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:42:58.518773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.300123+0000","last_change":"2023-11-28T11:24:16.290970+0000","last_active":"2023-11-28T11:24:16.300123+0000","last_peered":"2023-11-28T11:24:16.300123+0000","last_clean":"2023-11-28T11:24:16.300123+0000","last_became_active":"2023-11-28T11:24:16.290667+0000","last_became_peered":"2023-11-28T11:24:16.290667+0000","last_unstale":"2023-11-28T11:24:16.300123+0000","last_undegraded":"2023-11-28T11:24:16.300123+0000","last_fullsized":"2023-11-28T11:24:16.300123+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:32:43.162554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.292148+0000","last_change":"2023-11-28T11:24:16.285904+0000","last_active":"2023-11-28T11:24:16.292148+0000","last_peered":"2023-11-28T11:24:16.292148+0000","last_clean":"2023-11-28T11:24:16.292148+0000","last_became_active":"2023-11-28T11:24:16.285627+0000","last_became_peered":"2023-11-28T11:24:16.285627+0000","last_unstale":"2023-11-28T11:24:16.292148+0000","last_undegraded":"2023-11-28T11:24:16.292148+0000","last_fullsized":"2023-11-28T11:24:16.292148+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:28:34.277902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.299147+0000","last_change":"2023-11-28T11:24:16.296498+0000","last_active":"2023-11-28T11:24:16.299147+0000","last_peered":"2023-11-28T11:24:16.299147+0000","last_clean":"2023-11-28T11:24:16.299147+0000","last_became_active":"2023-11-28T11:24:16.296230+0000","last_became_peered":"2023-11-28T11:24:16.296230+0000","last_unstale":"2023-11-28T11:24:16.299147+0000","last_undegraded":"2023-11-28T11:24:16.299147+0000","last_fullsized":"2023-11-28T11:24:16.299147+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:53:17.454560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.284562+0000","last_change":"2023-11-28T11:24:16.276950+0000","last_active":"2023-11-28T11:24:16.284562+0000","last_peered":"2023-11-28T11:24:16.284562+0000","last_clean":"2023-11-28T11:24:16.284562+0000","last_became_active":"2023-11-28T11:24:16.276489+0000","last_became_peered":"2023-11-28T11:24:16.276489+0000","last_unstale":"2023-11-28T11:24:16.284562+0000","last_undegraded":"2023-11-28T11:24:16.284562+0000","last_fullsized":"2023-11-28T11:24:16.284562+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:30:24.793183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.300742+0000","last_change":"2023-11-28T11:24:16.293196+0000","last_active":"2023-11-28T11:24:16.300742+0000","last_peered":"2023-11-28T11:24:16.300742+0000","last_clean":"2023-11-28T11:24:16.300742+0000","last_became_active":"2023-11-28T11:24:16.292379+0000","last_became_peered":"2023-11-28T11:24:16.292379+0000","last_unstale":"2023-11-28T11:24:16.300742+0000","last_undegraded":"2023-11-28T11:24:16.300742+0000","last_fullsized":"2023-11-28T11:24:16.300742+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:41:26.436810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.304698+0000","last_change":"2023-11-28T11:24:16.301487+0000","last_active":"2023-11-28T11:24:16.304698+0000","last_peered":"2023-11-28T11:24:16.304698+0000","last_clean":"2023-11-28T11:24:16.304698+0000","last_became_active":"2023-11-28T11:24:16.301138+0000","last_became_peered":"2023-11-28T11:24:16.301138+0000","last_unstale":"2023-11-28T11:24:16.304698+0000","last_undegraded":"2023-11-28T11:24:16.304698+0000","last_fullsized":"2023-11-28T11:24:16.304698+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:48:20.990724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"67'4","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.295460+0000","last_change":"2023-11-28T11:24:15.791407+0000","last_active":"2023-11-28T11:24:16.295460+0000","last_peered":"2023-11-28T11:24:16.295460+0000","last_clean":"2023-11-28T11:24:16.295460+0000","last_became_active":"2023-11-28T11:24:15.790447+0000","last_became_peered":"2023-11-28T11:24:15.790447+0000","last_unstale":"2023-11-28T11:24:16.295460+0000","last_undegraded":"2023-11-28T11:24:16.295460+0000","last_fullsized":"2023-11-28T11:24:16.295460+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:37.869238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.290598+0000","last_change":"2023-11-28T11:24:16.284911+0000","last_active":"2023-11-28T11:24:16.290598+0000","last_peered":"2023-11-28T11:24:16.290598+0000","last_clean":"2023-11-28T11:24:16.290598+0000","last_became_active":"2023-11-28T11:24:16.283669+0000","last_became_peered":"2023-11-28T11:24:16.283669+0000","last_unstale":"2023-11-28T11:24:16.290598+0000","last_undegraded":"2023-11-28T11:24:16.290598+0000","last_fullsized":"2023-11-28T11:24:16.290598+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:46:16.231711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"67'1","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.284555+0000","last_change":"2023-11-28T11:24:15.787045+0000","last_active":"2023-11-28T11:24:16.284555+0000","last_peered":"2023-11-28T11:24:16.284555+0000","last_clean":"2023-11-28T11:24:16.284555+0000","last_became_active":"2023-11-28T11:24:15.786714+0000","last_became_peered":"2023-11-28T11:24:15.786714+0000","last_unstale":"2023-11-28T11:24:16.284555+0000","last_undegraded":"2023-11-28T11:24:16.284555+0000","last_fullsized":"2023-11-28T11:24:16.284555+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:48:19.983961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.301988+0000","last_change":"2023-11-28T11:24:16.297404+0000","last_active":"2023-11-28T11:24:16.301988+0000","last_peered":"2023-11-28T11:24:16.301988+0000","last_clean":"2023-11-28T11:24:16.301988+0000","last_became_active":"2023-11-28T11:24:16.296864+0000","last_became_peered":"2023-11-28T11:24:16.296864+0000","last_unstale":"2023-11-28T11:24:16.301988+0000","last_undegraded":"2023-11-28T11:24:16.301988+0000","last_fullsized":"2023-11-28T11:24:16.301988+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:24:48.055382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"67'2","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.299709+0000","last_change":"2023-11-28T11:24:15.791373+0000","last_active":"2023-11-28T11:24:16.299709+0000","last_peered":"2023-11-28T11:24:16.299709+0000","last_clean":"2023-11-28T11:24:16.299709+0000","last_became_active":"2023-11-28T11:24:15.790807+0000","last_became_peered":"2023-11-28T11:24:15.790807+0000","last_unstale":"2023-11-28T11:24:16.299709+0000","last_undegraded":"2023-11-28T11:24:16.299709+0000","last_fullsized":"2023-11-28T11:24:16.299709+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:54:20.554467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.288227+0000","last_change":"2023-11-28T11:24:16.283126+0000","last_active":"2023-11-28T11:24:16.288227+0000","last_peered":"2023-11-28T11:24:16.288227+0000","last_clean":"2023-11-28T11:24:16.288227+0000","last_became_active":"2023-11-28T11:24:16.282484+0000","last_became_peered":"2023-11-28T11:24:16.282484+0000","last_unstale":"2023-11-28T11:24:16.288227+0000","last_undegraded":"2023-11-28T11:24:16.288227+0000","last_fullsized":"2023-11-28T11:24:16.288227+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:17:48.917682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"67'2","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.299463+0000","last_change":"2023-11-28T11:24:15.790330+0000","last_active":"2023-11-28T11:24:16.299463+0000","last_peered":"2023-11-28T11:24:16.299463+0000","last_clean":"2023-11-28T11:24:16.299463+0000","last_became_active":"2023-11-28T11:24:15.790012+0000","last_became_peered":"2023-11-28T11:24:15.790012+0000","last_unstale":"2023-11-28T11:24:16.299463+0000","last_undegraded":"2023-11-28T11:24:16.299463+0000","last_fullsized":"2023-11-28T11:24:16.299463+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:11:07.458538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.290328+0000","last_change":"2023-11-28T11:24:16.287777+0000","last_active":"2023-11-28T11:24:16.290328+0000","last_peered":"2023-11-28T11:24:16.290328+0000","last_clean":"2023-11-28T11:24:16.290328+0000","last_became_active":"2023-11-28T11:24:16.286193+0000","last_became_peered":"2023-11-28T11:24:16.286193+0000","last_unstale":"2023-11-28T11:24:16.290328+0000","last_undegraded":"2023-11-28T11:24:16.290328+0000","last_fullsized":"2023-11-28T11:24:16.290328+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:32:43.162554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"67'1","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.293457+0000","last_change":"2023-11-28T11:24:15.789861+0000","last_active":"2023-11-28T11:24:16.293457+0000","last_peered":"2023-11-28T11:24:16.293457+0000","last_clean":"2023-11-28T11:24:16.293457+0000","last_became_active":"2023-11-28T11:24:15.789531+0000","last_became_peered":"2023-11-28T11:24:15.789531+0000","last_unstale":"2023-11-28T11:24:16.293457+0000","last_undegraded":"2023-11-28T11:24:16.293457+0000","last_fullsized":"2023-11-28T11:24:16.293457+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:46:10.168470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.300068+0000","last_change":"2023-11-28T11:24:16.291704+0000","last_active":"2023-11-28T11:24:16.300068+0000","last_peered":"2023-11-28T11:24:16.300068+0000","last_clean":"2023-11-28T11:24:16.300068+0000","last_became_active":"2023-11-28T11:24:16.291334+0000","last_became_peered":"2023-11-28T11:24:16.291334+0000","last_unstale":"2023-11-28T11:24:16.300068+0000","last_undegraded":"2023-11-28T11:24:16.300068+0000","last_fullsized":"2023-11-28T11:24:16.300068+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:26:38.836179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"67'1","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.299076+0000","last_change":"2023-11-28T11:24:15.794958+0000","last_active":"2023-11-28T11:24:16.299076+0000","last_peered":"2023-11-28T11:24:16.299076+0000","last_clean":"2023-11-28T11:24:16.299076+0000","last_became_active":"2023-11-28T11:24:15.794623+0000","last_became_peered":"2023-11-28T11:24:15.794623+0000","last_unstale":"2023-11-28T11:24:16.299076+0000","last_undegraded":"2023-11-28T11:24:16.299076+0000","last_fullsized":"2023-11-28T11:24:16.299076+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:14:10.678880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.293986+0000","last_change":"2023-11-28T11:24:16.288044+0000","last_active":"2023-11-28T11:24:16.293986+0000","last_peered":"2023-11-28T11:24:16.293986+0000","last_clean":"2023-11-28T11:24:16.293986+0000","last_became_active":"2023-11-28T11:24:16.287795+0000","last_became_peered":"2023-11-28T11:24:16.287795+0000","last_unstale":"2023-11-28T11:24:16.293986+0000","last_undegraded":"2023-11-28T11:24:16.293986+0000","last_fullsized":"2023-11-28T11:24:16.293986+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:30:51.011752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.262625+0000","last_change":"2023-11-28T11:24:15.779840+0000","last_active":"2023-11-28T11:24:16.262625+0000","last_peered":"2023-11-28T11:24:16.262625+0000","last_clean":"2023-11-28T11:24:16.262625+0000","last_became_active":"2023-11-28T11:24:15.779558+0000","last_became_peered":"2023-11-28T11:24:15.779558+0000","last_unstale":"2023-11-28T11:24:16.262625+0000","last_undegraded":"2023-11-28T11:24:16.262625+0000","last_fullsized":"2023-11-28T11:24:16.262625+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:11:07.458538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.293595+0000","last_change":"2023-11-28T11:24:16.287946+0000","last_active":"2023-11-28T11:24:16.293595+0000","last_peered":"2023-11-28T11:24:16.293595+0000","last_clean":"2023-11-28T11:24:16.293595+0000","last_became_active":"2023-11-28T11:24:16.287610+0000","last_became_peered":"2023-11-28T11:24:16.287610+0000","last_unstale":"2023-11-28T11:24:16.293595+0000","last_undegraded":"2023-11-28T11:24:16.293595+0000","last_fullsized":"2023-11-28T11:24:16.293595+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:46:11.175232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.271397+0000","last_change":"2023-11-28T11:24:15.784188+0000","last_active":"2023-11-28T11:24:16.271397+0000","last_peered":"2023-11-28T11:24:16.271397+0000","last_clean":"2023-11-28T11:24:16.271397+0000","last_became_active":"2023-11-28T11:24:15.783912+0000","last_became_peered":"2023-11-28T11:24:15.783912+0000","last_unstale":"2023-11-28T11:24:16.271397+0000","last_undegraded":"2023-11-28T11:24:16.271397+0000","last_fullsized":"2023-11-28T11:24:16.271397+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:42:52.804541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.279171+0000","last_change":"2023-11-28T11:24:16.275083+0000","last_active":"2023-11-28T11:24:16.279171+0000","last_peered":"2023-11-28T11:24:16.279171+0000","last_clean":"2023-11-28T11:24:16.279171+0000","last_became_active":"2023-11-28T11:24:16.274819+0000","last_became_peered":"2023-11-28T11:24:16.274819+0000","last_unstale":"2023-11-28T11:24:16.279171+0000","last_undegraded":"2023-11-28T11:24:16.279171+0000","last_fullsized":"2023-11-28T11:24:16.279171+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:42:33.731752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.269932+0000","last_change":"2023-11-28T11:24:15.789937+0000","last_active":"2023-11-28T11:24:16.269932+0000","last_peered":"2023-11-28T11:24:16.269932+0000","last_clean":"2023-11-28T11:24:16.269932+0000","last_became_active":"2023-11-28T11:24:15.788196+0000","last_became_peered":"2023-11-28T11:24:15.788196+0000","last_unstale":"2023-11-28T11:24:16.269932+0000","last_undegraded":"2023-11-28T11:24:16.269932+0000","last_fullsized":"2023-11-28T11:24:16.269932+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:17:47.910920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.304691+0000","last_change":"2023-11-28T11:24:16.301321+0000","last_active":"2023-11-28T11:24:16.304691+0000","last_peered":"2023-11-28T11:24:16.304691+0000","last_clean":"2023-11-28T11:24:16.304691+0000","last_became_active":"2023-11-28T11:24:16.301057+0000","last_became_peered":"2023-11-28T11:24:16.301057+0000","last_unstale":"2023-11-28T11:24:16.304691+0000","last_undegraded":"2023-11-28T11:24:16.304691+0000","last_fullsized":"2023-11-28T11:24:16.304691+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:30:24.793183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.279441+0000","last_change":"2023-11-28T11:24:15.788090+0000","last_active":"2023-11-28T11:24:16.279441+0000","last_peered":"2023-11-28T11:24:16.279441+0000","last_clean":"2023-11-28T11:24:16.279441+0000","last_became_active":"2023-11-28T11:24:15.787812+0000","last_became_peered":"2023-11-28T11:24:15.787812+0000","last_unstale":"2023-11-28T11:24:16.279441+0000","last_undegraded":"2023-11-28T11:24:16.279441+0000","last_fullsized":"2023-11-28T11:24:16.279441+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:46:15.224948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.301967+0000","last_change":"2023-11-28T11:24:16.297238+0000","last_active":"2023-11-28T11:24:16.301967+0000","last_peered":"2023-11-28T11:24:16.301967+0000","last_clean":"2023-11-28T11:24:16.301967+0000","last_became_active":"2023-11-28T11:24:16.296511+0000","last_became_peered":"2023-11-28T11:24:16.296511+0000","last_unstale":"2023-11-28T11:24:16.301967+0000","last_undegraded":"2023-11-28T11:24:16.301967+0000","last_fullsized":"2023-11-28T11:24:16.301967+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:18:49.460782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"67'3","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.293895+0000","last_change":"2023-11-28T11:24:15.786382+0000","last_active":"2023-11-28T11:24:16.293895+0000","last_peered":"2023-11-28T11:24:16.293895+0000","last_clean":"2023-11-28T11:24:16.293895+0000","last_became_active":"2023-11-28T11:24:15.786124+0000","last_became_peered":"2023-11-28T11:24:15.786124+0000","last_unstale":"2023-11-28T11:24:16.293895+0000","last_undegraded":"2023-11-28T11:24:16.293895+0000","last_fullsized":"2023-11-28T11:24:16.293895+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:13.987166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.293738+0000","last_change":"2023-11-28T11:24:16.287632+0000","last_active":"2023-11-28T11:24:16.293738+0000","last_peered":"2023-11-28T11:24:16.293738+0000","last_clean":"2023-11-28T11:24:16.293738+0000","last_became_active":"2023-11-28T11:24:16.287371+0000","last_became_peered":"2023-11-28T11:24:16.287371+0000","last_unstale":"2023-11-28T11:24:16.293738+0000","last_undegraded":"2023-11-28T11:24:16.293738+0000","last_fullsized":"2023-11-28T11:24:16.293738+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:49:51.062127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"67'6","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.302064+0000","last_change":"2023-11-28T11:24:15.784875+0000","last_active":"2023-11-28T11:24:16.302064+0000","last_peered":"2023-11-28T11:24:16.302064+0000","last_clean":"2023-11-28T11:24:16.302064+0000","last_became_active":"2023-11-28T11:24:15.784596+0000","last_became_peered":"2023-11-28T11:24:15.784596+0000","last_unstale":"2023-11-28T11:24:16.302064+0000","last_undegraded":"2023-11-28T11:24:16.302064+0000","last_fullsized":"2023-11-28T11:24:16.302064+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:30:50.004990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.294865+0000","last_change":"2023-11-28T11:24:16.288680+0000","last_active":"2023-11-28T11:24:16.294865+0000","last_peered":"2023-11-28T11:24:16.294865+0000","last_clean":"2023-11-28T11:24:16.294865+0000","last_became_active":"2023-11-28T11:24:16.287059+0000","last_became_peered":"2023-11-28T11:24:16.287059+0000","last_unstale":"2023-11-28T11:24:16.294865+0000","last_undegraded":"2023-11-28T11:24:16.294865+0000","last_fullsized":"2023-11-28T11:24:16.294865+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:28:52.219751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"67'2","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.292139+0000","last_change":"2023-11-28T11:24:15.790658+0000","last_active":"2023-11-28T11:24:16.292139+0000","last_peered":"2023-11-28T11:24:16.292139+0000","last_clean":"2023-11-28T11:24:16.292139+0000","last_became_active":"2023-11-28T11:24:15.790263+0000","last_became_peered":"2023-11-28T11:24:15.790263+0000","last_unstale":"2023-11-28T11:24:16.292139+0000","last_undegraded":"2023-11-28T11:24:16.292139+0000","last_fullsized":"2023-11-28T11:24:16.292139+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:02:40.577973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.286721+0000","last_change":"2023-11-28T11:24:16.282554+0000","last_active":"2023-11-28T11:24:16.286721+0000","last_peered":"2023-11-28T11:24:16.286721+0000","last_clean":"2023-11-28T11:24:16.286721+0000","last_became_active":"2023-11-28T11:24:16.281211+0000","last_became_peered":"2023-11-28T11:24:16.281211+0000","last_unstale":"2023-11-28T11:24:16.286721+0000","last_undegraded":"2023-11-28T11:24:16.286721+0000","last_fullsized":"2023-11-28T11:24:16.286721+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:30:18.787935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.281948+0000","last_change":"2023-11-28T11:24:15.788508+0000","last_active":"2023-11-28T11:24:16.281948+0000","last_peered":"2023-11-28T11:24:16.281948+0000","last_clean":"2023-11-28T11:24:16.281948+0000","last_became_active":"2023-11-28T11:24:15.788174+0000","last_became_peered":"2023-11-28T11:24:15.788174+0000","last_unstale":"2023-11-28T11:24:16.281948+0000","last_undegraded":"2023-11-28T11:24:16.281948+0000","last_fullsized":"2023-11-28T11:24:16.281948+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:26:25.627047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.304101+0000","last_change":"2023-11-28T11:24:16.300155+0000","last_active":"2023-11-28T11:24:16.304101+0000","last_peered":"2023-11-28T11:24:16.304101+0000","last_clean":"2023-11-28T11:24:16.304101+0000","last_became_active":"2023-11-28T11:24:16.299896+0000","last_became_peered":"2023-11-28T11:24:16.299896+0000","last_unstale":"2023-11-28T11:24:16.304101+0000","last_undegraded":"2023-11-28T11:24:16.304101+0000","last_fullsized":"2023-11-28T11:24:16.304101+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:26:38.836179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.267858+0000","last_change":"2023-11-28T11:24:15.783430+0000","last_active":"2023-11-28T11:24:16.267858+0000","last_peered":"2023-11-28T11:24:16.267858+0000","last_clean":"2023-11-28T11:24:16.267858+0000","last_became_active":"2023-11-28T11:24:15.783059+0000","last_became_peered":"2023-11-28T11:24:15.783059+0000","last_unstale":"2023-11-28T11:24:16.267858+0000","last_undegraded":"2023-11-28T11:24:16.267858+0000","last_fullsized":"2023-11-28T11:24:16.267858+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:28:51.212988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.287272+0000","last_change":"2023-11-28T11:24:16.282622+0000","last_active":"2023-11-28T11:24:16.287272+0000","last_peered":"2023-11-28T11:24:16.287272+0000","last_clean":"2023-11-28T11:24:16.287272+0000","last_became_active":"2023-11-28T11:24:16.282076+0000","last_became_peered":"2023-11-28T11:24:16.282076+0000","last_unstale":"2023-11-28T11:24:16.287272+0000","last_undegraded":"2023-11-28T11:24:16.287272+0000","last_fullsized":"2023-11-28T11:24:16.287272+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:11:08.465301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"67'8","reported_seq":18,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.296448+0000","last_change":"2023-11-28T11:24:15.789291+0000","last_active":"2023-11-28T11:24:16.296448+0000","last_peered":"2023-11-28T11:24:16.296448+0000","last_clean":"2023-11-28T11:24:16.296448+0000","last_became_active":"2023-11-28T11:24:15.788986+0000","last_became_peered":"2023-11-28T11:24:15.788986+0000","last_unstale":"2023-11-28T11:24:16.296448+0000","last_undegraded":"2023-11-28T11:24:16.296448+0000","last_fullsized":"2023-11-28T11:24:16.296448+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:52:09.678809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.290121+0000","last_change":"2023-11-28T11:24:16.285079+0000","last_active":"2023-11-28T11:24:16.290121+0000","last_peered":"2023-11-28T11:24:16.290121+0000","last_clean":"2023-11-28T11:24:16.290121+0000","last_became_active":"2023-11-28T11:24:16.284827+0000","last_became_peered":"2023-11-28T11:24:16.284827+0000","last_unstale":"2023-11-28T11:24:16.290121+0000","last_undegraded":"2023-11-28T11:24:16.290121+0000","last_fullsized":"2023-11-28T11:24:16.290121+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:49:11.229814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"67'2","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.300490+0000","last_change":"2023-11-28T11:24:15.785815+0000","last_active":"2023-11-28T11:24:16.300490+0000","last_peered":"2023-11-28T11:24:16.300490+0000","last_clean":"2023-11-28T11:24:16.300490+0000","last_became_active":"2023-11-28T11:24:15.785277+0000","last_became_peered":"2023-11-28T11:24:15.785277+0000","last_unstale":"2023-11-28T11:24:16.300490+0000","last_undegraded":"2023-11-28T11:24:16.300490+0000","last_fullsized":"2023-11-28T11:24:16.300490+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:49:50.055365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.284435+0000","last_change":"2023-11-28T11:24:16.277984+0000","last_active":"2023-11-28T11:24:16.284435+0000","last_peered":"2023-11-28T11:24:16.284435+0000","last_clean":"2023-11-28T11:24:16.284435+0000","last_became_active":"2023-11-28T11:24:16.277706+0000","last_became_peered":"2023-11-28T11:24:16.277706+0000","last_unstale":"2023-11-28T11:24:16.284435+0000","last_undegraded":"2023-11-28T11:24:16.284435+0000","last_fullsized":"2023-11-28T11:24:16.284435+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:23:54.440671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.267911+0000","last_change":"2023-11-28T11:24:15.784761+0000","last_active":"2023-11-28T11:24:16.267911+0000","last_peered":"2023-11-28T11:24:16.267911+0000","last_clean":"2023-11-28T11:24:16.267911+0000","last_became_active":"2023-11-28T11:24:15.784469+0000","last_became_peered":"2023-11-28T11:24:15.784469+0000","last_unstale":"2023-11-28T11:24:16.267911+0000","last_undegraded":"2023-11-28T11:24:16.267911+0000","last_fullsized":"2023-11-28T11:24:16.267911+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:30:09.205229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.300386+0000","last_change":"2023-11-28T11:24:16.298424+0000","last_active":"2023-11-28T11:24:16.300386+0000","last_peered":"2023-11-28T11:24:16.300386+0000","last_clean":"2023-11-28T11:24:16.300386+0000","last_became_active":"2023-11-28T11:24:16.298170+0000","last_became_peered":"2023-11-28T11:24:16.298170+0000","last_unstale":"2023-11-28T11:24:16.300386+0000","last_undegraded":"2023-11-28T11:24:16.300386+0000","last_fullsized":"2023-11-28T11:24:16.300386+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:36:55.431715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.271895+0000","last_change":"2023-11-28T11:24:15.786191+0000","last_active":"2023-11-28T11:24:16.271895+0000","last_peered":"2023-11-28T11:24:16.271895+0000","last_clean":"2023-11-28T11:24:16.271895+0000","last_became_active":"2023-11-28T11:24:15.784407+0000","last_became_peered":"2023-11-28T11:24:15.784407+0000","last_unstale":"2023-11-28T11:24:16.271895+0000","last_undegraded":"2023-11-28T11:24:16.271895+0000","last_fullsized":"2023-11-28T11:24:16.271895+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:06:21.696007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.288247+0000","last_change":"2023-11-28T11:24:16.283266+0000","last_active":"2023-11-28T11:24:16.288247+0000","last_peered":"2023-11-28T11:24:16.288247+0000","last_clean":"2023-11-28T11:24:16.288247+0000","last_became_active":"2023-11-28T11:24:16.282830+0000","last_became_peered":"2023-11-28T11:24:16.282830+0000","last_unstale":"2023-11-28T11:24:16.288247+0000","last_undegraded":"2023-11-28T11:24:16.288247+0000","last_fullsized":"2023-11-28T11:24:16.288247+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:54:21.561230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.277978+0000","last_change":"2023-11-28T11:24:15.791886+0000","last_active":"2023-11-28T11:24:16.277978+0000","last_peered":"2023-11-28T11:24:16.277978+0000","last_clean":"2023-11-28T11:24:16.277978+0000","last_became_active":"2023-11-28T11:24:15.791568+0000","last_became_peered":"2023-11-28T11:24:15.791568+0000","last_unstale":"2023-11-28T11:24:16.277978+0000","last_undegraded":"2023-11-28T11:24:16.277978+0000","last_fullsized":"2023-11-28T11:24:16.277978+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:36:54.424952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.287317+0000","last_change":"2023-11-28T11:24:16.283124+0000","last_active":"2023-11-28T11:24:16.287317+0000","last_peered":"2023-11-28T11:24:16.287317+0000","last_clean":"2023-11-28T11:24:16.287317+0000","last_became_active":"2023-11-28T11:24:16.282730+0000","last_became_peered":"2023-11-28T11:24:16.282730+0000","last_unstale":"2023-11-28T11:24:16.287317+0000","last_undegraded":"2023-11-28T11:24:16.287317+0000","last_fullsized":"2023-11-28T11:24:16.287317+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:26:26.633809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.271250+0000","last_change":"2023-11-28T11:24:15.787882+0000","last_active":"2023-11-28T11:24:16.271250+0000","last_peered":"2023-11-28T11:24:16.271250+0000","last_clean":"2023-11-28T11:24:16.271250+0000","last_became_active":"2023-11-28T11:24:15.787534+0000","last_became_peered":"2023-11-28T11:24:15.787534+0000","last_unstale":"2023-11-28T11:24:16.271250+0000","last_undegraded":"2023-11-28T11:24:16.271250+0000","last_fullsized":"2023-11-28T11:24:16.271250+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:42:30.674044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.301699+0000","last_change":"2023-11-28T11:24:16.293811+0000","last_active":"2023-11-28T11:24:16.301699+0000","last_peered":"2023-11-28T11:24:16.301699+0000","last_clean":"2023-11-28T11:24:16.301699+0000","last_became_active":"2023-11-28T11:24:16.292397+0000","last_became_peered":"2023-11-28T11:24:16.292397+0000","last_unstale":"2023-11-28T11:24:16.301699+0000","last_undegraded":"2023-11-28T11:24:16.301699+0000","last_fullsized":"2023-11-28T11:24:16.301699+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:44:51.915900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.262623+0000","last_change":"2023-11-28T11:24:15.781417+0000","last_active":"2023-11-28T11:24:16.262623+0000","last_peered":"2023-11-28T11:24:16.262623+0000","last_clean":"2023-11-28T11:24:16.262623+0000","last_became_active":"2023-11-28T11:24:15.781179+0000","last_became_peered":"2023-11-28T11:24:15.781179+0000","last_unstale":"2023-11-28T11:24:16.262623+0000","last_undegraded":"2023-11-28T11:24:16.262623+0000","last_fullsized":"2023-11-28T11:24:16.262623+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:26:25.627047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.282990+0000","last_change":"2023-11-28T11:24:16.278841+0000","last_active":"2023-11-28T11:24:16.282990+0000","last_peered":"2023-11-28T11:24:16.282990+0000","last_clean":"2023-11-28T11:24:16.282990+0000","last_became_active":"2023-11-28T11:24:16.278550+0000","last_became_peered":"2023-11-28T11:24:16.278550+0000","last_unstale":"2023-11-28T11:24:16.282990+0000","last_undegraded":"2023-11-28T11:24:16.282990+0000","last_fullsized":"2023-11-28T11:24:16.282990+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:26:34.127522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.270000+0000","last_change":"2023-11-28T11:24:15.791998+0000","last_active":"2023-11-28T11:24:16.270000+0000","last_peered":"2023-11-28T11:24:16.270000+0000","last_clean":"2023-11-28T11:24:16.270000+0000","last_became_active":"2023-11-28T11:24:15.791163+0000","last_became_peered":"2023-11-28T11:24:15.791163+0000","last_unstale":"2023-11-28T11:24:16.270000+0000","last_undegraded":"2023-11-28T11:24:16.270000+0000","last_fullsized":"2023-11-28T11:24:16.270000+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:28:51.212988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.300800+0000","last_change":"2023-11-28T11:24:16.293082+0000","last_active":"2023-11-28T11:24:16.300800+0000","last_peered":"2023-11-28T11:24:16.300800+0000","last_clean":"2023-11-28T11:24:16.300800+0000","last_became_active":"2023-11-28T11:24:16.292418+0000","last_became_peered":"2023-11-28T11:24:16.292418+0000","last_unstale":"2023-11-28T11:24:16.300800+0000","last_undegraded":"2023-11-28T11:24:16.300800+0000","last_fullsized":"2023-11-28T11:24:16.300800+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:21:15.703589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.279377+0000","last_change":"2023-11-28T11:24:15.790249+0000","last_active":"2023-11-28T11:24:16.279377+0000","last_peered":"2023-11-28T11:24:16.279377+0000","last_clean":"2023-11-28T11:24:16.279377+0000","last_became_active":"2023-11-28T11:24:15.789966+0000","last_became_peered":"2023-11-28T11:24:15.789966+0000","last_unstale":"2023-11-28T11:24:16.279377+0000","last_undegraded":"2023-11-28T11:24:16.279377+0000","last_fullsized":"2023-11-28T11:24:16.279377+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:57:38.381675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.294023+0000","last_change":"2023-11-28T11:24:16.288388+0000","last_active":"2023-11-28T11:24:16.294023+0000","last_peered":"2023-11-28T11:24:16.294023+0000","last_clean":"2023-11-28T11:24:16.294023+0000","last_became_active":"2023-11-28T11:24:16.286204+0000","last_became_peered":"2023-11-28T11:24:16.286204+0000","last_unstale":"2023-11-28T11:24:16.294023+0000","last_undegraded":"2023-11-28T11:24:16.294023+0000","last_fullsized":"2023-11-28T11:24:16.294023+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:02:41.584735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.270520+0000","last_change":"2023-11-28T11:24:15.793185+0000","last_active":"2023-11-28T11:24:16.270520+0000","last_peered":"2023-11-28T11:24:16.270520+0000","last_clean":"2023-11-28T11:24:16.270520+0000","last_became_active":"2023-11-28T11:24:15.792887+0000","last_became_peered":"2023-11-28T11:24:15.792887+0000","last_unstale":"2023-11-28T11:24:16.270520+0000","last_undegraded":"2023-11-28T11:24:16.270520+0000","last_fullsized":"2023-11-28T11:24:16.270520+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:18:48.454020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":53,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:24:15.767172+0000","last_change":"2023-11-28T11:20:51.020016+0000","last_active":"2023-11-28T11:24:15.767172+0000","last_peered":"2023-11-28T11:24:15.767172+0000","last_clean":"2023-11-28T11:24:15.767172+0000","last_became_active":"2023-11-28T11:20:51.019492+0000","last_became_peered":"2023-11-28T11:20:51.019492+0000","last_unstale":"2023-11-28T11:24:15.767172+0000","last_undegraded":"2023-11-28T11:24:15.767172+0000","last_fullsized":"2023-11-28T11:24:15.767172+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:27.884538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.292124+0000","last_change":"2023-11-28T11:24:16.285826+0000","last_active":"2023-11-28T11:24:16.292124+0000","last_peered":"2023-11-28T11:24:16.292124+0000","last_clean":"2023-11-28T11:24:16.292124+0000","last_became_active":"2023-11-28T11:24:16.285551+0000","last_became_peered":"2023-11-28T11:24:16.285551+0000","last_unstale":"2023-11-28T11:24:16.292124+0000","last_undegraded":"2023-11-28T11:24:16.292124+0000","last_fullsized":"2023-11-28T11:24:16.292124+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:08:10.008261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.281681+0000","last_change":"2023-11-28T11:24:15.794692+0000","last_active":"2023-11-28T11:24:16.281681+0000","last_peered":"2023-11-28T11:24:16.281681+0000","last_clean":"2023-11-28T11:24:16.281681+0000","last_became_active":"2023-11-28T11:24:15.793892+0000","last_became_peered":"2023-11-28T11:24:15.793892+0000","last_unstale":"2023-11-28T11:24:16.281681+0000","last_undegraded":"2023-11-28T11:24:16.281681+0000","last_fullsized":"2023-11-28T11:24:16.281681+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:02:40.577973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":57,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:24:15.776827+0000","last_change":"2023-11-28T11:20:50.521908+0000","last_active":"2023-11-28T11:24:15.776827+0000","last_peered":"2023-11-28T11:24:15.776827+0000","last_clean":"2023-11-28T11:24:15.776827+0000","last_became_active":"2023-11-28T11:20:50.521059+0000","last_became_peered":"2023-11-28T11:20:50.521059+0000","last_unstale":"2023-11-28T11:24:15.776827+0000","last_undegraded":"2023-11-28T11:24:15.776827+0000","last_fullsized":"2023-11-28T11:24:15.776827+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:05.302498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.292505+0000","last_change":"2023-11-28T11:24:16.286009+0000","last_active":"2023-11-28T11:24:16.292505+0000","last_peered":"2023-11-28T11:24:16.292505+0000","last_clean":"2023-11-28T11:24:16.292505+0000","last_became_active":"2023-11-28T11:24:16.285080+0000","last_became_peered":"2023-11-28T11:24:16.285080+0000","last_unstale":"2023-11-28T11:24:16.292505+0000","last_undegraded":"2023-11-28T11:24:16.292505+0000","last_fullsized":"2023-11-28T11:24:16.292505+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:48:35.874902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.267968+0000","last_change":"2023-11-28T11:24:15.785470+0000","last_active":"2023-11-28T11:24:16.267968+0000","last_peered":"2023-11-28T11:24:16.267968+0000","last_clean":"2023-11-28T11:24:16.267968+0000","last_became_active":"2023-11-28T11:24:15.785175+0000","last_became_peered":"2023-11-28T11:24:15.785175+0000","last_unstale":"2023-11-28T11:24:16.267968+0000","last_undegraded":"2023-11-28T11:24:16.267968+0000","last_fullsized":"2023-11-28T11:24:16.267968+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:11:12.657318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":50,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:24:15.775444+0000","last_change":"2023-11-28T11:20:50.619004+0000","last_active":"2023-11-28T11:24:15.775444+0000","last_peered":"2023-11-28T11:24:15.775444+0000","last_clean":"2023-11-28T11:24:15.775444+0000","last_became_active":"2023-11-28T11:20:50.618315+0000","last_became_peered":"2023-11-28T11:20:50.618315+0000","last_unstale":"2023-11-28T11:24:15.775444+0000","last_undegraded":"2023-11-28T11:24:15.775444+0000","last_fullsized":"2023-11-28T11:24:15.775444+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:27.884538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.301557+0000","last_change":"2023-11-28T11:24:16.298556+0000","last_active":"2023-11-28T11:24:16.301557+0000","last_peered":"2023-11-28T11:24:16.301557+0000","last_clean":"2023-11-28T11:24:16.301557+0000","last_became_active":"2023-11-28T11:24:16.298258+0000","last_became_peered":"2023-11-28T11:24:16.298258+0000","last_unstale":"2023-11-28T11:24:16.301557+0000","last_undegraded":"2023-11-28T11:24:16.301557+0000","last_fullsized":"2023-11-28T11:24:16.301557+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:46:42.415700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"67'2","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.299605+0000","last_change":"2023-11-28T11:24:15.791634+0000","last_active":"2023-11-28T11:24:16.299605+0000","last_peered":"2023-11-28T11:24:16.299605+0000","last_clean":"2023-11-28T11:24:16.299605+0000","last_became_active":"2023-11-28T11:24:15.791381+0000","last_became_peered":"2023-11-28T11:24:15.791381+0000","last_unstale":"2023-11-28T11:24:16.299605+0000","last_undegraded":"2023-11-28T11:24:16.299605+0000","last_fullsized":"2023-11-28T11:24:16.299605+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:44:02.542531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":57,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:24:15.776890+0000","last_change":"2023-11-28T11:20:50.522582+0000","last_active":"2023-11-28T11:24:15.776890+0000","last_peered":"2023-11-28T11:24:15.776890+0000","last_clean":"2023-11-28T11:24:15.776890+0000","last_became_active":"2023-11-28T11:20:50.522157+0000","last_became_peered":"2023-11-28T11:20:50.522157+0000","last_unstale":"2023-11-28T11:24:15.776890+0000","last_undegraded":"2023-11-28T11:24:15.776890+0000","last_fullsized":"2023-11-28T11:24:15.776890+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:27.884538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":204,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:24:15.779531+0000","last_change":"2023-11-28T11:20:43.294845+0000","last_active":"2023-11-28T11:24:15.779531+0000","last_peered":"2023-11-28T11:24:15.779531+0000","last_clean":"2023-11-28T11:24:15.779531+0000","last_became_active":"2023-11-28T11:20:43.294483+0000","last_became_peered":"2023-11-28T11:20:43.294483+0000","last_unstale":"2023-11-28T11:24:15.779531+0000","last_undegraded":"2023-11-28T11:24:15.779531+0000","last_fullsized":"2023-11-28T11:24:15.779531+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:42.270512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:42.270512+0000","last_clean_scrub_stamp":"2023-11-28T11:20:42.270512+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:17:03.376680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.301630+0000","last_change":"2023-11-28T11:24:16.298648+0000","last_active":"2023-11-28T11:24:16.301630+0000","last_peered":"2023-11-28T11:24:16.301630+0000","last_clean":"2023-11-28T11:24:16.301630+0000","last_became_active":"2023-11-28T11:24:16.298264+0000","last_became_peered":"2023-11-28T11:24:16.298264+0000","last_unstale":"2023-11-28T11:24:16.301630+0000","last_undegraded":"2023-11-28T11:24:16.301630+0000","last_fullsized":"2023-11-28T11:24:16.301630+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:01:32.493557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.276939+0000","last_change":"2023-11-28T11:24:15.787841+0000","last_active":"2023-11-28T11:24:16.276939+0000","last_peered":"2023-11-28T11:24:16.276939+0000","last_clean":"2023-11-28T11:24:16.276939+0000","last_became_active":"2023-11-28T11:24:15.787081+0000","last_became_peered":"2023-11-28T11:24:15.787081+0000","last_unstale":"2023-11-28T11:24:16.276939+0000","last_undegraded":"2023-11-28T11:24:16.276939+0000","last_fullsized":"2023-11-28T11:24:16.276939+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:18:37.395260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":51,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:24:15.779618+0000","last_change":"2023-11-28T11:20:50.526865+0000","last_active":"2023-11-28T11:24:15.779618+0000","last_peered":"2023-11-28T11:24:15.779618+0000","last_clean":"2023-11-28T11:24:15.779618+0000","last_became_active":"2023-11-28T11:20:50.526576+0000","last_became_peered":"2023-11-28T11:20:50.526576+0000","last_unstale":"2023-11-28T11:24:15.779618+0000","last_undegraded":"2023-11-28T11:24:15.779618+0000","last_fullsized":"2023-11-28T11:24:15.779618+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:09:18.460112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.284862+0000","last_change":"2023-11-28T11:24:16.280368+0000","last_active":"2023-11-28T11:24:16.284862+0000","last_peered":"2023-11-28T11:24:16.284862+0000","last_clean":"2023-11-28T11:24:16.284862+0000","last_became_active":"2023-11-28T11:24:16.280067+0000","last_became_peered":"2023-11-28T11:24:16.280067+0000","last_unstale":"2023-11-28T11:24:16.284862+0000","last_undegraded":"2023-11-28T11:24:16.284862+0000","last_fullsized":"2023-11-28T11:24:16.284862+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:12:31.931403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"67'1","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.301943+0000","last_change":"2023-11-28T11:24:15.793755+0000","last_active":"2023-11-28T11:24:16.301943+0000","last_peered":"2023-11-28T11:24:16.301943+0000","last_clean":"2023-11-28T11:24:16.301943+0000","last_became_active":"2023-11-28T11:24:15.793485+0000","last_became_peered":"2023-11-28T11:24:15.793485+0000","last_unstale":"2023-11-28T11:24:16.301943+0000","last_undegraded":"2023-11-28T11:24:16.301943+0000","last_fullsized":"2023-11-28T11:24:16.301943+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:30:48.708427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":51,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:24:15.784160+0000","last_change":"2023-11-28T11:20:51.020043+0000","last_active":"2023-11-28T11:24:15.784160+0000","last_peered":"2023-11-28T11:24:15.784160+0000","last_clean":"2023-11-28T11:24:15.784160+0000","last_became_active":"2023-11-28T11:20:51.019702+0000","last_became_peered":"2023-11-28T11:20:51.019702+0000","last_unstale":"2023-11-28T11:24:15.784160+0000","last_undegraded":"2023-11-28T11:24:15.784160+0000","last_fullsized":"2023-11-28T11:24:15.784160+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:26:20.246873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.294381+0000","last_change":"2023-11-28T11:24:16.288263+0000","last_active":"2023-11-28T11:24:16.294381+0000","last_peered":"2023-11-28T11:24:16.294381+0000","last_clean":"2023-11-28T11:24:16.294381+0000","last_became_active":"2023-11-28T11:24:16.287986+0000","last_became_peered":"2023-11-28T11:24:16.287986+0000","last_unstale":"2023-11-28T11:24:16.294381+0000","last_undegraded":"2023-11-28T11:24:16.294381+0000","last_fullsized":"2023-11-28T11:24:16.294381+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:18:38.402023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"67'2","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.299641+0000","last_change":"2023-11-28T11:24:15.789630+0000","last_active":"2023-11-28T11:24:16.299641+0000","last_peered":"2023-11-28T11:24:16.299641+0000","last_clean":"2023-11-28T11:24:16.299641+0000","last_became_active":"2023-11-28T11:24:15.789316+0000","last_became_peered":"2023-11-28T11:24:15.789316+0000","last_unstale":"2023-11-28T11:24:16.299641+0000","last_undegraded":"2023-11-28T11:24:16.299641+0000","last_fullsized":"2023-11-28T11:24:16.299641+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:07:17.039552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":53,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:24:15.767262+0000","last_change":"2023-11-28T11:20:50.524251+0000","last_active":"2023-11-28T11:24:15.767262+0000","last_peered":"2023-11-28T11:24:15.767262+0000","last_clean":"2023-11-28T11:24:15.767262+0000","last_became_active":"2023-11-28T11:20:50.524012+0000","last_became_peered":"2023-11-28T11:20:50.524012+0000","last_unstale":"2023-11-28T11:24:15.767262+0000","last_undegraded":"2023-11-28T11:24:15.767262+0000","last_fullsized":"2023-11-28T11:24:15.767262+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:05.302498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.281215+0000","last_change":"2023-11-28T11:24:16.279088+0000","last_active":"2023-11-28T11:24:16.281215+0000","last_peered":"2023-11-28T11:24:16.281215+0000","last_clean":"2023-11-28T11:24:16.281215+0000","last_became_active":"2023-11-28T11:24:16.278776+0000","last_became_peered":"2023-11-28T11:24:16.278776+0000","last_unstale":"2023-11-28T11:24:16.281215+0000","last_undegraded":"2023-11-28T11:24:16.281215+0000","last_fullsized":"2023-11-28T11:24:16.281215+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:15.758151+0000","last_clean_scrub_stamp":"2023-11-28T11:24:15.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:30:18.787935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"67'1","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-28T11:24:16.299205+0000","last_change":"2023-11-28T11:24:15.785442+0000","last_active":"2023-11-28T11:24:16.299205+0000","last_peered":"2023-11-28T11:24:16.299205+0000","last_clean":"2023-11-28T11:24:16.299205+0000","last_became_active":"2023-11-28T11:24:15.784801+0000","last_became_peered":"2023-11-28T11:24:15.784801+0000","last_unstale":"2023-11-28T11:24:16.299205+0000","last_undegraded":"2023-11-28T11:24:16.299205+0000","last_fullsized":"2023-11-28T11:24:16.299205+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:24:14.751388+0000","last_clean_scrub_stamp":"2023-11-28T11:24:14.751388+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:02:43.611821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":50,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:24:15.782093+0000","last_change":"2023-11-28T11:20:50.524838+0000","last_active":"2023-11-28T11:24:15.782093+0000","last_peered":"2023-11-28T11:24:15.782093+0000","last_clean":"2023-11-28T11:24:15.782093+0000","last_became_active":"2023-11-28T11:20:50.524402+0000","last_became_peered":"2023-11-28T11:20:50.524402+0000","last_unstale":"2023-11-28T11:24:15.782093+0000","last_undegraded":"2023-11-28T11:24:15.782093+0000","last_fullsized":"2023-11-28T11:24:15.782093+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:22.699447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574895,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10920,"kb_used_data":1376,"kb_used_omap":0,"kb_used_meta":9536,"kb_avail":94360920,"statfs":{"total":96636764160,"available":96625582080,"internally_reserved":0,"allocated":1409024,"data_stored":969001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9764864},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.621,"15min":0.623},"min":{"1min":0.41399999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.64700000000000002,"15min":0.64800000000000002},"min":{"1min":0.46500000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.56899999999999995}]},{"osd":1,"last update":"Tue Nov 28 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.64700000000000002,"15min":0.64400000000000002},"min":{"1min":0.54200000000000004,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81299999999999994,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.67200000000000004,"15min":0.67700000000000005},"min":{"1min":0.44500000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.625}]},{"osd":2,"last update":"Tue Nov 28 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.65200000000000002,"15min":0.65400000000000003},"min":{"1min":0.51000000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.71999999999999997,"15min":0.72199999999999998},"min":{"1min":0.60499999999999998,"5min":0.495,"15min":0.495},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.76800000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.65400000000000003,"15min":0.66200000000000003},"min":{"1min":0.45200000000000001,"5min":0.433,"15min":0.433},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.70199999999999996,"15min":0.70599999999999996},"min":{"1min":0.55800000000000005,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.96799999999999997,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.64600000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.63900000000000001,"15min":0.66400000000000003},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.89600000000000002,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.69499999999999995,"15min":0.70099999999999996},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91100000000000003,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.66900000000000004}]},{"osd":5,"last update":"Tue Nov 28 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67700000000000005,"15min":0.68400000000000005},"min":{"1min":0.40799999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.69999999999999996,"15min":0.71399999999999997},"min":{"1min":0.49099999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90300000000000002,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.73899999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.76400000000000001,"15min":0.78000000000000003},"min":{"1min":0.45000000000000001,"5min":0.435,"15min":0.435},"max":{"1min":1.0509999999999999,"5min":1.633,"15min":1.633},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.72599999999999998,"15min":0.752},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.85299999999999998,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.78600000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574895,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9640,"kb_used_data":800,"kb_used_omap":0,"kb_used_meta":8832,"kb_avail":94362200,"statfs":{"total":96636764160,"available":96626892800,"internally_reserved":0,"allocated":819200,"data_stored":378747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9043968},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.627,"15min":0.63700000000000001},"min":{"1min":0.45000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.93999999999999995,"5min":1.409,"15min":1.409},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.63500000000000001,"15min":0.63800000000000001},"min":{"1min":0.40999999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.98299999999999998,"5min":1.244,"15min":1.244},"last":0.76400000000000001}]},{"osd":1,"last update":"Tue Nov 28 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.629,"15min":0.627},"min":{"1min":0.44700000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.67100000000000004,"15min":0.68300000000000005},"min":{"1min":0.438,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.92600000000000005,"5min":1.038,"15min":1.038},"last":0.83499999999999996}]},{"osd":2,"last update":"Tue Nov 28 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.624,"15min":0.627},"min":{"1min":0.47499999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.746,"15min":0.77800000000000002},"min":{"1min":0.48199999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1,"5min":1.996,"15min":1.996},"last":0.86099999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.68899999999999995,"15min":0.69699999999999995},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.66600000000000004,"15min":0.68200000000000005},"min":{"1min":0.48699999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.96199999999999997,"5min":1.008,"15min":1.008},"last":0.63700000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.69699999999999995,"15min":0.72899999999999998},"min":{"1min":0.45100000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.86599999999999999,"5min":2.048,"15min":2.048},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.69799999999999995,"15min":0.72199999999999998},"min":{"1min":0.46300000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.873,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.93999999999999995}]},{"osd":5,"last update":"Tue Nov 28 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.66900000000000004,"15min":0.69699999999999995},"min":{"1min":0.42199999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.78200000000000003,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.77500000000000002,"15min":0.81499999999999995},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.89700000000000002,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":0.95899999999999996}]},{"osd":7,"last update":"Tue Nov 28 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.76000000000000001,"15min":0.77700000000000002},"min":{"1min":0.60299999999999998,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.86899999999999999,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":1.0009999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.77900000000000003,"15min":0.79800000000000004},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.018,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.88100000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574895,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9620,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":8832,"kb_avail":94362220,"statfs":{"total":96636764160,"available":96626913280,"internally_reserved":0,"allocated":798720,"data_stored":371207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9043968},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:23:46 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.44500000000000001,"15min":0.46300000000000002},"min":{"1min":0.29199999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.64200000000000002,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.45400000000000001,"15min":0.46000000000000002},"min":{"1min":0.32200000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.58899999999999997,"5min":0.999,"15min":0.999},"last":0.38400000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:23:46 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.496,"15min":0.51400000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.71199999999999997,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.49099999999999999,"15min":0.50600000000000001},"min":{"1min":0.35599999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.63,"5min":1.002,"15min":1.002},"last":0.54400000000000004}]},{"osd":3,"last update":"Tue Nov 28 11:23:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.53900000000000003,"15min":0.55400000000000005},"min":{"1min":0.40300000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.71899999999999997,"5min":1.198,"15min":1.198},"last":0.497},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51300000000000001,"15min":0.52100000000000002},"min":{"1min":0.34000000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.66400000000000003,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.51300000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:23:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.747,"15min":0.75700000000000001},"min":{"1min":0.59199999999999997,"5min":0.438,"15min":0.438},"max":{"1min":0.91600000000000004,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.74099999999999999,"15min":0.755},"min":{"1min":0.47299999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.99299999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.64900000000000002}]},{"osd":5,"last update":"Tue Nov 28 11:23:46 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.753,"15min":0.76900000000000002},"min":{"1min":0.54100000000000004,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.88800000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76000000000000001,"15min":0.77200000000000002},"min":{"1min":0.60399999999999998,"5min":0.5,"15min":0.5},"max":{"1min":0.97899999999999998,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.84499999999999997}]},{"osd":6,"last update":"Tue Nov 28 11:23:46 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.77500000000000002,"15min":0.78500000000000003},"min":{"1min":0.57699999999999996,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.98299999999999998,"5min":1.177,"15min":1.177},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.79000000000000004,"15min":0.80600000000000005},"min":{"1min":0.58899999999999997,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0329999999999999,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.73999999999999999}]},{"osd":7,"last update":"Tue Nov 28 11:23:46 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.754,"15min":0.76200000000000001},"min":{"1min":0.59799999999999998,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0680000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.752},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.79600000000000004,"15min":0.80400000000000005},"min":{"1min":0.64500000000000002,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.016,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.80100000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574895,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":15076,"kb_used_data":1372,"kb_used_omap":0,"kb_used_meta":13696,"kb_avail":94356764,"statfs":{"total":96636764160,"available":96621326336,"internally_reserved":0,"allocated":1404928,"data_stored":969023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14024704},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.434,"15min":0.44400000000000001},"min":{"1min":0.27300000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.59199999999999997,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.57199999999999995,"15min":0.63300000000000001},"min":{"1min":0.29999999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.58899999999999997,"5min":3.8420000000000001,"15min":3.8420000000000001},"last":0.44}]},{"osd":2,"last update":"Tue Nov 28 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.48799999999999999,"15min":0.504},"min":{"1min":0.36699999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.58799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.61799999999999999,"15min":0.67600000000000005},"min":{"1min":0.34300000000000003,"5min":0.311,"15min":0.311},"max":{"1min":0.71199999999999997,"5min":3.8940000000000001,"15min":3.8940000000000001},"last":0.52800000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.58399999999999996,"15min":0.63600000000000001},"min":{"1min":0.372,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.65600000000000003,"5min":3.681,"15min":3.681},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.56799999999999995,"15min":0.58999999999999997},"min":{"1min":0.42199999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.72699999999999998,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.54400000000000004}]},{"osd":4,"last update":"Tue Nov 28 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.85699999999999998,"15min":0.92100000000000004},"min":{"1min":0.54300000000000004,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.94599999999999995,"5min":3.9790000000000001,"15min":3.9790000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.88,"15min":0.94499999999999995},"min":{"1min":0.51300000000000001,"5min":0.497,"15min":0.497},"max":{"1min":0.999,"5min":3.5779999999999998,"15min":3.5779999999999998},"last":0.58099999999999996}]},{"osd":5,"last update":"Tue Nov 28 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.79600000000000004,"15min":0.82199999999999995},"min":{"1min":0.57299999999999995,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.96399999999999997,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.88800000000000001,"15min":0.96099999999999997},"min":{"1min":0.51500000000000001,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0449999999999999,"5min":3.9409999999999998,"15min":3.9409999999999998},"last":0.60999999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.878,"15min":0.94499999999999995},"min":{"1min":0.56399999999999995,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.92500000000000004,"5min":3.6160000000000001,"15min":3.6160000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.81599999999999995,"15min":0.84399999999999997},"min":{"1min":0.627,"5min":0.501,"15min":0.501},"max":{"1min":1.022,"5min":1.363,"15min":1.363},"last":0.70399999999999996}]},{"osd":7,"last update":"Tue Nov 28 11:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.94899999999999995,"15min":1.034},"min":{"1min":0.58399999999999996,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.94899999999999995,"5min":4.0099999999999998,"15min":4.0099999999999998},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.746,"5min":0.91500000000000004,"15min":0.99099999999999999},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0629999999999999,"5min":3.653,"15min":3.653},"last":0.68799999999999994}]}]},{"osd":2,"up_from":13,"seq":55834574893,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9492,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362348,"statfs":{"total":96636764160,"available":96627044352,"internally_reserved":0,"allocated":798720,"data_stored":371207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.50700000000000001,"15min":0.55700000000000005},"min":{"1min":0.314,"5min":0.219,"15min":0.219},"max":{"1min":0.61699999999999999,"5min":4.1109999999999998,"15min":4.1109999999999998},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.55200000000000005,"15min":0.59399999999999997},"min":{"1min":0.307,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.071,"5min":3.6230000000000002,"15min":3.6230000000000002},"last":0.47499999999999998}]},{"osd":1,"last update":"Tue Nov 28 11:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.64100000000000001,"15min":0.67600000000000005},"min":{"1min":0.38700000000000001,"5min":0.371,"15min":0.371},"max":{"1min":1.8779999999999999,"5min":3.6640000000000001,"15min":3.6640000000000001},"last":0.443},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.628,"15min":0.68999999999999995},"min":{"1min":0.40899999999999997,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.63100000000000001,"5min":4.0830000000000002,"15min":4.0830000000000002},"last":0.45900000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.59599999999999997,"15min":0.65100000000000002},"min":{"1min":0.34200000000000003,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.55100000000000005,"5min":4.1529999999999996,"15min":4.1529999999999996},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.56799999999999995,"15min":0.58499999999999996},"min":{"1min":0.40100000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.01,"5min":1.304,"15min":1.304},"last":0.39800000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.79200000000000004,"15min":0.81000000000000005},"min":{"1min":0.61499999999999999,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.107,"5min":1.649,"15min":1.649},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.747,"5min":0.85299999999999998,"15min":0.91500000000000004},"min":{"1min":0.63700000000000001,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.92100000000000004,"5min":4.0060000000000002,"15min":4.0060000000000002},"last":0.88800000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.86699999999999999,"15min":0.92700000000000005},"min":{"1min":0.61799999999999999,"5min":0.5,"15min":0.5},"max":{"1min":0.97999999999999998,"5min":4.0529999999999999,"15min":4.0529999999999999},"last":0.747},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.80000000000000004,"15min":0.81399999999999995},"min":{"1min":0.63300000000000001,"5min":0.497,"15min":0.497},"max":{"1min":0.97499999999999998,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.91700000000000004}]},{"osd":6,"last update":"Tue Nov 28 11:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.82199999999999995,"15min":0.84399999999999997},"min":{"1min":0.627,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.93000000000000005,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.84199999999999997,"15min":0.85899999999999999},"min":{"1min":0.65300000000000002,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.212,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.93100000000000005}]},{"osd":7,"last update":"Tue Nov 28 11:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.78800000000000003,"15min":0.80800000000000005},"min":{"1min":0.56699999999999995,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.996,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.81699999999999995,"15min":0.83299999999999996},"min":{"1min":0.65000000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.139,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.89800000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574894,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9748,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":8960,"kb_avail":94362092,"statfs":{"total":96636764160,"available":96626782208,"internally_reserved":0,"allocated":798720,"data_stored":371207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9175040},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:23:46 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.40400000000000003,"15min":0.40600000000000003},"min":{"1min":0.33300000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.52400000000000002,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.45900000000000002,"15min":0.46100000000000002},"min":{"1min":0.32000000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.72599999999999998,"5min":1.157,"15min":1.157},"last":0.40699999999999997}]},{"osd":1,"last update":"Tue Nov 28 11:23:46 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.54000000000000004,"15min":0.58199999999999996},"min":{"1min":0.34200000000000003,"5min":0.313,"15min":0.313},"max":{"1min":0.57999999999999996,"5min":4.8019999999999996,"15min":4.8019999999999996},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.498,"5min":0.55700000000000005,"15min":0.59499999999999997},"min":{"1min":0.38700000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.67400000000000004,"5min":4.7539999999999996,"15min":4.7539999999999996},"last":0.38200000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:23:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.61599999999999999,"15min":0.66200000000000003},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.69099999999999995,"5min":4.8419999999999996,"15min":4.8419999999999996},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.61299999999999999,"15min":0.65700000000000003},"min":{"1min":0.38700000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.64600000000000002,"5min":4.7919999999999998,"15min":4.7919999999999998},"last":0.34999999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:23:46 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.79800000000000004,"15min":0.83999999999999997},"min":{"1min":0.51000000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.90900000000000003,"5min":4.8869999999999996,"15min":4.8869999999999996},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.84699999999999998,"15min":0.90300000000000002},"min":{"1min":0.59199999999999997,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.90900000000000003,"5min":5.2460000000000004,"15min":5.2460000000000004},"last":0.64700000000000002}]},{"osd":5,"last update":"Tue Nov 28 11:23:46 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.86099999999999999,"15min":0.91500000000000004},"min":{"1min":0.54100000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.93600000000000005,"5min":5.0759999999999996,"15min":5.0759999999999996},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.746,"5min":0.82799999999999996,"15min":0.874},"min":{"1min":0.55400000000000005,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.94299999999999995,"5min":5.3010000000000002,"15min":5.3010000000000002},"last":0.51300000000000001}]},{"osd":6,"last update":"Tue Nov 28 11:23:46 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.84299999999999997,"15min":0.88700000000000001},"min":{"1min":0.55300000000000005,"5min":0.501,"15min":0.501},"max":{"1min":0.84999999999999998,"5min":5.3399999999999999,"15min":5.3399999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.83299999999999996,"15min":0.872},"min":{"1min":0.53900000000000003,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.95299999999999996,"5min":4.8330000000000002,"15min":4.8330000000000002},"last":0.60599999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:23:46 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.83799999999999997,"15min":0.88},"min":{"1min":0.59399999999999997,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.89400000000000002,"5min":4.8600000000000003,"15min":4.8600000000000003},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.754,"5min":0.88100000000000001,"15min":0.93899999999999995},"min":{"1min":0.56499999999999995,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.89500000000000002,"5min":5.1909999999999998,"15min":5.1909999999999998},"last":0.58199999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574894,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9940,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361900,"statfs":{"total":96636764160,"available":96626585600,"internally_reserved":0,"allocated":798720,"data_stored":371207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.67200000000000004,"15min":0.68300000000000005},"min":{"1min":0.45200000000000001,"5min":0.437,"15min":0.437},"max":{"1min":0.85899999999999999,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.66600000000000004,"15min":0.66200000000000003},"min":{"1min":0.48699999999999999,"5min":0.438,"15min":0.438},"max":{"1min":1.0509999999999999,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.439}]},{"osd":1,"last update":"Tue Nov 28 11:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.66400000000000003,"15min":0.64900000000000002},"min":{"1min":0.56299999999999994,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.69499999999999995,"15min":0.70499999999999996},"min":{"1min":0.50800000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.91400000000000003,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.73699999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73399999999999999,"15min":0.74299999999999999},"min":{"1min":0.56599999999999995,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.93799999999999994,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.70299999999999996,"15min":0.69899999999999995},"min":{"1min":0.54800000000000004,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.68700000000000006}]},{"osd":3,"last update":"Tue Nov 28 11:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.67800000000000005,"15min":0.68100000000000005},"min":{"1min":0.498,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88600000000000001,"5min":1.179,"15min":1.179},"last":0.498},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.76500000000000001,"15min":0.77900000000000003},"min":{"1min":0.51300000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.90600000000000003,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.66800000000000004}]},{"osd":5,"last update":"Tue Nov 28 11:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.67700000000000005,"15min":0.68700000000000006},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.81699999999999995,"5min":1.161,"15min":1.161},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.65800000000000003,"15min":0.66400000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.83899999999999997,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.56499999999999995}]},{"osd":6,"last update":"Tue Nov 28 11:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.70699999999999996,"15min":0.72899999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.79400000000000004,"5min":1.514,"15min":1.514},"last":0.622},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.71999999999999997,"15min":0.73399999999999999},"min":{"1min":0.44800000000000001,"5min":0.439,"15min":0.439},"max":{"1min":0.84599999999999997,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.80600000000000005}]},{"osd":7,"last update":"Tue Nov 28 11:23:51 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.69799999999999995,"15min":0.70199999999999996},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.92300000000000004,"5min":1.226,"15min":1.226},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.80700000000000005,"15min":0.83399999999999996},"min":{"1min":0.58099999999999996,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.94499999999999995,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":0.71899999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9700,"kb_used_data":796,"kb_used_omap":0,"kb_used_meta":8896,"kb_avail":94362140,"statfs":{"total":96636764160,"available":96626831360,"internally_reserved":0,"allocated":815104,"data_stored":378745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9109504},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:23:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.63800000000000001,"15min":0.628},"min":{"1min":0.38400000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.0109999999999999,"5min":1.032,"15min":1.032},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.69499999999999995,"15min":0.69799999999999995},"min":{"1min":0.50700000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.90100000000000002,"5min":1.02,"15min":1.02},"last":0.65200000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:23:48 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.66800000000000004,"15min":0.65900000000000003},"min":{"1min":0.51700000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.90200000000000002,"5min":1.355,"15min":1.355},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.69499999999999995,"15min":0.69699999999999995},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.111,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.68300000000000005}]},{"osd":2,"last update":"Tue Nov 28 11:23:48 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.69399999999999995,"15min":0.69799999999999995},"min":{"1min":0.45400000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.93500000000000005,"5min":1.198,"15min":1.198},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.72499999999999998,"15min":0.73299999999999998},"min":{"1min":0.51300000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.92500000000000004,"5min":1.387,"15min":1.387},"last":0.96199999999999997}]},{"osd":3,"last update":"Tue Nov 28 11:23:48 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.74299999999999999,"15min":0.754},"min":{"1min":0.503,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.71999999999999997,"15min":0.72399999999999998},"min":{"1min":0.57399999999999995,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.88400000000000001,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.85499999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:23:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.70599999999999996,"15min":0.72399999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":2.5139999999999998,"5min":2.5139999999999998,"15min":2.5139999999999998},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.72599999999999998,"15min":0.73699999999999999},"min":{"1min":0.497,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.91800000000000004,"5min":1.198,"15min":1.198},"last":0.89100000000000001}]},{"osd":6,"last update":"Tue Nov 28 11:23:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.72599999999999998,"15min":0.73399999999999999},"min":{"1min":0.51600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0589999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.73399999999999999,"15min":0.73199999999999998},"min":{"1min":0.52700000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":2.4529999999999998,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":0.879}]},{"osd":7,"last update":"Tue Nov 28 11:23:48 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.80100000000000005,"15min":0.82199999999999995},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.9059999999999999,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.80200000000000005,"15min":0.82399999999999995},"min":{"1min":0.57799999999999996,"5min":0.496,"15min":0.496},"max":{"1min":1.1699999999999999,"5min":1.298,"15min":1.298},"last":0.98599999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:24:21.446 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T11:24:21.446 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2023-11-28T11:24:21.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.683+0000 7f84ccfff700 1 Processor -- start 2023-11-28T11:24:21.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.683+0000 7f84ccfff700 1 -- start start 2023-11-28T11:24:21.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.684+0000 7f84ccfff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f84c80d2e10 0x7f84c815d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:21.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.684+0000 7f84ccfff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84c815dee0 0x7f84c80cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:21.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.685+0000 7f84ccfff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84c80cd4d0 con 0x7f84c80b2280 2023-11-28T11:24:21.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.685+0000 7f84ccfff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84c80cdce0 con 0x7f84c80d2e10 2023-11-28T11:24:21.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.685+0000 7f84ccfff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84c80cf470 con 0x7f84c815dee0 2023-11-28T11:24:21.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.685+0000 7f84c659c700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f84c80d2e10 0x7f84c815d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:21.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.685+0000 7f84c6d9d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84c815dee0 0x7f84c80cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:21.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.685+0000 7f84c659c700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f84c80d2e10 0x7f84c815d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.92:45478/0 (socket says 172.21.15.92:45478) 2023-11-28T11:24:21.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.685+0000 7f84bffff700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f84c80b2280 0x7f84c80d31e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.92:52960/0 (socket says 172.21.15.92:52960) 2023-11-28T11:24:21.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.685+0000 7f84c6d9d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84c815dee0 0x7f84c80cf900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.92:44624/0 (socket says 172.21.15.92:44624) 2023-11-28T11:24:21.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.685+0000 7f84c659c700 1 -- 172.21.15.92:0/1321213254 learned_addr learned my addr 172.21.15.92:0/1321213254 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:21.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.686+0000 7f84c659c700 1 -- 172.21.15.92:0/1321213254 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84c815dee0 msgr2=0x7f84c80cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:21.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.686+0000 7f84c659c700 1 --2- 172.21.15.92:0/1321213254 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84c815dee0 0x7f84c80cf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:21.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.686+0000 7f84c659c700 1 -- 172.21.15.92:0/1321213254 >> v1:172.21.15.169:6789/0 conn(0x7f84c80b2280 legacy=0x7f84c80d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:21.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.686+0000 7f84c659c700 1 -- 172.21.15.92:0/1321213254 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c80b2700 con 0x7f84c80d2e10 2023-11-28T11:24:21.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.686+0000 7f84c6d9d700 1 --2- 172.21.15.92:0/1321213254 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84c815dee0 0x7f84c80cf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:24:21.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.686+0000 7f84c5d9b700 1 -- 172.21.15.92:0/1321213254 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2911545795 0 0) 0x7f84c80cd4d0 con 0x7f84c80b2280 2023-11-28T11:24:21.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.686+0000 7f84c659c700 1 --2- 172.21.15.92:0/1321213254 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84c80d2e10 0x7f84c815d590 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f84b80098f0 tx=0x7f84b8009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb091882b82ff281 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:21.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.686+0000 7f84c5d9b700 1 -- 172.21.15.92:0/1321213254 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b8011830 con 0x7f84c80d2e10 2023-11-28T11:24:21.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.686+0000 7f84c5d9b700 1 -- 172.21.15.92:0/1321213254 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84b8011990 con 0x7f84c80d2e10 2023-11-28T11:24:21.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.686+0000 7f84c5d9b700 1 -- 172.21.15.92:0/1321213254 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b8011d20 con 0x7f84c80d2e10 2023-11-28T11:24:21.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.687+0000 7f84ccfff700 1 -- 172.21.15.92:0/1321213254 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84c80d2e10 msgr2=0x7f84c815d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:21.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.687+0000 7f84ccfff700 1 --2- 172.21.15.92:0/1321213254 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84c80d2e10 0x7f84c815d590 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f84b80098f0 tx=0x7f84b8009c00 comp rx=0 tx=0).stop 2023-11-28T11:24:21.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.687+0000 7f84ccfff700 1 -- 172.21.15.92:0/1321213254 shutdown_connections 2023-11-28T11:24:21.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.687+0000 7f84ccfff700 1 --2- 172.21.15.92:0/1321213254 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84c80d2e10 0x7f84c815d590 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:21.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.687+0000 7f84ccfff700 1 --2- 172.21.15.92:0/1321213254 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84c815dee0 0x7f84c80cf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:21.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.687+0000 7f84ccfff700 1 -- 172.21.15.92:0/1321213254 >> 172.21.15.92:0/1321213254 conn(0x7f84c80719a0 msgr2=0x7f84c8071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:21.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.687+0000 7f84ccfff700 1 -- 172.21.15.92:0/1321213254 shutdown_connections 2023-11-28T11:24:21.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.687+0000 7f84ccfff700 1 -- 172.21.15.92:0/1321213254 wait complete. 2023-11-28T11:24:21.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.688+0000 7f84ccfff700 1 Processor -- start 2023-11-28T11:24:21.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.689+0000 7f84ccfff700 1 -- start start 2023-11-28T11:24:21.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.690+0000 7f84ccfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84c80b2280 0x7f84c80d2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:21.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.690+0000 7f84ccfff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84c80d2e10 0x7f84c80b76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:21.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.690+0000 7f84ccfff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84c815dee0 0x7f84c80b7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:21.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.690+0000 7f84ccfff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c80b8170 con 0x7f84c80b2280 2023-11-28T11:24:21.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.690+0000 7f84ccfff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c80b8c00 con 0x7f84c815dee0 2023-11-28T11:24:21.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.690+0000 7f84ccfff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84c80b8360 con 0x7f84c80d2e10 2023-11-28T11:24:21.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.690+0000 7f84c659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84c80b2280 0x7f84c80d2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:21.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.690+0000 7f84c659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84c80b2280 0x7f84c80d2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55210/0 (socket says 172.21.15.92:55210) 2023-11-28T11:24:21.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.690+0000 7f84c659c700 1 -- 172.21.15.92:0/1977757389 learned_addr learned my addr 172.21.15.92:0/1977757389 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:21.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.690+0000 7f84bffff700 1 --2- 172.21.15.92:0/1977757389 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84c80d2e10 0x7f84c80b76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:21.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.690+0000 7f84c6d9d700 1 --2- 172.21.15.92:0/1977757389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84c815dee0 0x7f84c80b7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:21.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.690+0000 7f84c659c700 1 -- 172.21.15.92:0/1977757389 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84c80d2e10 msgr2=0x7f84c80b76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:21.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.690+0000 7f84c659c700 1 --2- 172.21.15.92:0/1977757389 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84c80d2e10 0x7f84c80b76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:21.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.690+0000 7f84c659c700 1 -- 172.21.15.92:0/1977757389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84c815dee0 msgr2=0x7f84c80b7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:21.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.690+0000 7f84c659c700 1 --2- 172.21.15.92:0/1977757389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84c815dee0 0x7f84c80b7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:21.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.690+0000 7f84c659c700 1 -- 172.21.15.92:0/1977757389 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c0009a70 con 0x7f84c80b2280 2023-11-28T11:24:21.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.691+0000 7f84c659c700 1 --2- 172.21.15.92:0/1977757389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84c80b2280 0x7f84c80d2260 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f84b8009c40 tx=0x7f84b8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:21.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.691+0000 7f84beffd700 1 -- 172.21.15.92:0/1977757389 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b8004480 con 0x7f84c80b2280 2023-11-28T11:24:21.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.691+0000 7f84beffd700 1 -- 172.21.15.92:0/1977757389 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84b801f070 con 0x7f84c80b2280 2023-11-28T11:24:21.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.691+0000 7f84ccfff700 1 -- 172.21.15.92:0/1977757389 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84b8009550 con 0x7f84c80b2280 2023-11-28T11:24:21.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.691+0000 7f84beffd700 1 -- 172.21.15.92:0/1977757389 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b80038c0 con 0x7f84c80b2280 2023-11-28T11:24:21.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.691+0000 7f84ccfff700 1 -- 172.21.15.92:0/1977757389 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84c80cd720 con 0x7f84c80b2280 2023-11-28T11:24:21.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.692+0000 7f84ccfff700 1 -- 172.21.15.92:0/1977757389 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84c8156fa0 con 0x7f84c80b2280 2023-11-28T11:24:21.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.693+0000 7f84beffd700 1 -- 172.21.15.92:0/1977757389 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f84b802a020 con 0x7f84c80b2280 2023-11-28T11:24:21.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.694+0000 7f84beffd700 1 --2- 172.21.15.92:0/1977757389 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f84a80682f0 0x7f84a806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:21.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.694+0000 7f84beffd700 1 -- 172.21.15.92:0/1977757389 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f84b80873d0 con 0x7f84c80b2280 2023-11-28T11:24:21.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.694+0000 7f84bffff700 1 --2- 172.21.15.92:0/1977757389 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f84a80682f0 0x7f84a806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:21.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.695+0000 7f84bffff700 1 --2- 172.21.15.92:0/1977757389 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f84a80682f0 0x7f84a806a7a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f84c815d010 tx=0x7f84b0007be0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:21.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.698+0000 7f84beffd700 1 -- 172.21.15.92:0/1977757389 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f84b8053a90 con 0x7f84c80b2280 2023-11-28T11:24:21.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.964+0000 7f84ccfff700 1 -- 172.21.15.92:0/1977757389 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7f84c80cda00 con 0x7f84c80b2280 2023-11-28T11:24:21.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.969+0000 7f84beffd700 1 -- 172.21.15.92:0/1977757389 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 184+0+0 (secure 0 0 0) 0x7f84b80570b0 con 0x7f84c80b2280 2023-11-28T11:24:21.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.972+0000 7f84ccfff700 1 -- 172.21.15.92:0/1977757389 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f84a80682f0 msgr2=0x7f84a806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:21.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.972+0000 7f84ccfff700 1 --2- 172.21.15.92:0/1977757389 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f84a80682f0 0x7f84a806a7a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f84c815d010 tx=0x7f84b0007be0 comp rx=0 tx=0).stop 2023-11-28T11:24:21.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.972+0000 7f84ccfff700 1 -- 172.21.15.92:0/1977757389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84c80b2280 msgr2=0x7f84c80d2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:21.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.972+0000 7f84ccfff700 1 --2- 172.21.15.92:0/1977757389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84c80b2280 0x7f84c80d2260 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f84b8009c40 tx=0x7f84b8004750 comp rx=0 tx=0).stop 2023-11-28T11:24:21.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.972+0000 7f84ccfff700 1 -- 172.21.15.92:0/1977757389 shutdown_connections 2023-11-28T11:24:21.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.972+0000 7f84ccfff700 1 --2- 172.21.15.92:0/1977757389 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f84c815dee0 0x7f84c80b7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:21.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.972+0000 7f84ccfff700 1 --2- 172.21.15.92:0/1977757389 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f84a80682f0 0x7f84a806a7a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:21.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.972+0000 7f84ccfff700 1 --2- 172.21.15.92:0/1977757389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84c80b2280 0x7f84c80d2260 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:21.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.972+0000 7f84ccfff700 1 --2- 172.21.15.92:0/1977757389 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f84c80d2e10 0x7f84c80b76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:21.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.972+0000 7f84ccfff700 1 -- 172.21.15.92:0/1977757389 >> 172.21.15.92:0/1977757389 conn(0x7f84c80719a0 msgr2=0x7f84c80d31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:21.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.973+0000 7f84ccfff700 1 -- 172.21.15.92:0/1977757389 shutdown_connections 2023-11-28T11:24:21.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:21.973+0000 7f84ccfff700 1 -- 172.21.15.92:0/1977757389 wait complete. 2023-11-28T11:24:21.975 INFO:teuthology.orchestra.run.smithi092.stderr:updated caps for client.0 2023-11-28T11:24:21.991 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:24:22.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.186+0000 7f3fa77ff700 1 Processor -- start 2023-11-28T11:24:22.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.186+0000 7f3fa77ff700 1 -- start start 2023-11-28T11:24:22.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.187+0000 7f3fa77ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3fa00b8510 0x7f3fa006b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:22.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.187+0000 7f3fa77ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fa00b88e0 con 0x7f3fa00d0d60 2023-11-28T11:24:22.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.187+0000 7f3fa77ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fa00cd8f0 con 0x7f3fa00cd460 2023-11-28T11:24:22.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.187+0000 7f3fa77ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fa00cdbd0 con 0x7f3fa00b8510 2023-11-28T11:24:22.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.188+0000 7f3fa5d9c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f3fa00d0d60 0x7f3fa00d1200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35716/0 (socket says 172.21.15.92:35716) 2023-11-28T11:24:22.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.188+0000 7f3fa559b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3fa00b8510 0x7f3fa006b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:22.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.188+0000 7f3fa5d9c700 1 -- 172.21.15.92:0/3849217839 learned_addr learned my addr 172.21.15.92:0/3849217839 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:24:22.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.188+0000 7f3fa559b700 1 -- 172.21.15.92:0/3849217839 >> v1:172.21.15.169:6789/0 conn(0x7f3fa00cd460 legacy=0x7f3fa006beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:22.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.188+0000 7f3fa559b700 1 -- 172.21.15.92:0/3849217839 >> v1:172.21.15.92:6789/0 conn(0x7f3fa00d0d60 legacy=0x7f3fa00d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:22.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.188+0000 7f3fa559b700 1 -- 172.21.15.92:0/3849217839 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa00cfd10 con 0x7f3fa00b8510 2023-11-28T11:24:22.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.188+0000 7f3f97fff700 1 -- 172.21.15.92:0/3849217839 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3540039538 0 0) 0x7f3fa00b88e0 con 0x7f3fa00d0d60 2023-11-28T11:24:22.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.188+0000 7f3fa559b700 1 --2- 172.21.15.92:0/3849217839 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3fa00b8510 0x7f3fa006b970 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c009840 tx=0x7f3f9c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdc3e69499a58eb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:22.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.189+0000 7f3f97fff700 1 -- 172.21.15.92:0/3849217839 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9c011830 con 0x7f3fa00b8510 2023-11-28T11:24:22.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.189+0000 7f3f97fff700 1 -- 172.21.15.92:0/3849217839 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f9c011990 con 0x7f3fa00b8510 2023-11-28T11:24:22.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.189+0000 7f3f97fff700 1 -- 172.21.15.92:0/3849217839 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9c011d20 con 0x7f3fa00b8510 2023-11-28T11:24:22.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.189+0000 7f3fa77ff700 1 -- 172.21.15.92:0/3849217839 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3fa00b8510 msgr2=0x7f3fa006b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:22.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.189+0000 7f3fa77ff700 1 --2- 172.21.15.92:0/3849217839 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3fa00b8510 0x7f3fa006b970 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c009840 tx=0x7f3f9c009b50 comp rx=0 tx=0).stop 2023-11-28T11:24:22.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.189+0000 7f3fa77ff700 1 -- 172.21.15.92:0/3849217839 shutdown_connections 2023-11-28T11:24:22.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.189+0000 7f3fa77ff700 1 --2- 172.21.15.92:0/3849217839 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3fa00b8510 0x7f3fa006b970 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:22.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.189+0000 7f3fa77ff700 1 -- 172.21.15.92:0/3849217839 >> 172.21.15.92:0/3849217839 conn(0x7f3fa00719a0 msgr2=0x7f3fa0071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:22.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.189+0000 7f3fa77ff700 1 -- 172.21.15.92:0/3849217839 shutdown_connections 2023-11-28T11:24:22.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.189+0000 7f3fa77ff700 1 -- 172.21.15.92:0/3849217839 wait complete. 2023-11-28T11:24:22.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.191+0000 7f3fa77ff700 1 Processor -- start 2023-11-28T11:24:22.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.191+0000 7f3fa77ff700 1 -- start start 2023-11-28T11:24:22.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.192+0000 7f3fa77ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3fa00b8510 0x7f3fa015a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:22.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.192+0000 7f3fa77ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3fa00cd460 0x7f3fa01e5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:22.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.192+0000 7f3fa77ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3fa00d0d60 0x7f3fa01e6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:22.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.192+0000 7f3fa77ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa015acf0 con 0x7f3fa00d0d60 2023-11-28T11:24:22.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.192+0000 7f3fa77ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa01e4560 con 0x7f3fa00b8510 2023-11-28T11:24:22.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.192+0000 7f3fa77ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa01e4840 con 0x7f3fa00cd460 2023-11-28T11:24:22.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.193+0000 7f3fa5d9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3fa00d0d60 0x7f3fa01e6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:22.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.193+0000 7f3fa5d9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3fa00d0d60 0x7f3fa01e6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55224/0 (socket says 172.21.15.92:55224) 2023-11-28T11:24:22.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.193+0000 7f3fa5d9c700 1 -- 172.21.15.92:0/841001752 learned_addr learned my addr 172.21.15.92:0/841001752 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:22.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.193+0000 7f3fa559b700 1 --2- 172.21.15.92:0/841001752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3fa00b8510 0x7f3fa015a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:22.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.193+0000 7f3fa4d9a700 1 --2- 172.21.15.92:0/841001752 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3fa00cd460 0x7f3fa01e5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:22.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.193+0000 7f3fa5d9c700 1 -- 172.21.15.92:0/841001752 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3fa00cd460 msgr2=0x7f3fa01e5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:22.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.193+0000 7f3fa5d9c700 1 --2- 172.21.15.92:0/841001752 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3fa00cd460 0x7f3fa01e5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:22.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.193+0000 7f3fa5d9c700 1 -- 172.21.15.92:0/841001752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3fa00b8510 msgr2=0x7f3fa015a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:22.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.193+0000 7f3fa5d9c700 1 --2- 172.21.15.92:0/841001752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3fa00b8510 0x7f3fa015a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:22.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.193+0000 7f3fa5d9c700 1 -- 172.21.15.92:0/841001752 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f9c0094a0 con 0x7f3fa00d0d60 2023-11-28T11:24:22.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.193+0000 7f3fa5d9c700 1 --2- 172.21.15.92:0/841001752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3fa00d0d60 0x7f3fa01e6840 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f3f9800cba0 tx=0x7f3f9800ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:22.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.193+0000 7f3f95ffb700 1 -- 172.21.15.92:0/841001752 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9800aad0 con 0x7f3fa00d0d60 2023-11-28T11:24:22.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.194+0000 7f3f95ffb700 1 -- 172.21.15.92:0/841001752 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f9800ac50 con 0x7f3fa00d0d60 2023-11-28T11:24:22.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.194+0000 7f3f95ffb700 1 -- 172.21.15.92:0/841001752 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f980070d0 con 0x7f3fa00d0d60 2023-11-28T11:24:22.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.194+0000 7f3fa77ff700 1 -- 172.21.15.92:0/841001752 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fa00d29c0 con 0x7f3fa00d0d60 2023-11-28T11:24:22.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.194+0000 7f3fa77ff700 1 -- 172.21.15.92:0/841001752 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fa00cfff0 con 0x7f3fa00d0d60 2023-11-28T11:24:22.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.195+0000 7f3fa77ff700 1 -- 172.21.15.92:0/841001752 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fa00d2b50 con 0x7f3fa00d0d60 2023-11-28T11:24:22.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.196+0000 7f3f95ffb700 1 -- 172.21.15.92:0/841001752 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3f98010070 con 0x7f3fa00d0d60 2023-11-28T11:24:22.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.196+0000 7f3f95ffb700 1 --2- 172.21.15.92:0/841001752 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3f840682f0 0x7f3f8406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:22.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.197+0000 7f3f95ffb700 1 -- 172.21.15.92:0/841001752 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f3f980856e0 con 0x7f3fa00d0d60 2023-11-28T11:24:22.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.197+0000 7f3fa559b700 1 --2- 172.21.15.92:0/841001752 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3f840682f0 0x7f3f8406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:22.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.197+0000 7f3fa559b700 1 --2- 172.21.15.92:0/841001752 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3f840682f0 0x7f3f8406a7a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c0090b0 tx=0x7f3f9c01c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:22.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.201+0000 7f3f95ffb700 1 -- 172.21.15.92:0/841001752 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3f98051da0 con 0x7f3fa00d0d60 2023-11-28T11:24:22.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.434+0000 7f3fa77ff700 1 -- 172.21.15.92:0/841001752 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3fa0067880 con 0x7f3fa00d0d60 2023-11-28T11:24:22.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.434+0000 7f3f95ffb700 1 -- 172.21.15.92:0/841001752 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4656 (secure 0 0 0) 0x7f3f980553c0 con 0x7f3fa00d0d60 2023-11-28T11:24:22.435 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:24:22.435 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5525,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2067956596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2067956596},{"type":"v1","addr":"172.21.15.169:6835","nonce":2067956596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5971,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2367049160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2367049160},{"type":"v1","addr":"172.21.15.169:6834","nonce":2367049160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5977,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1424858585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":1424858585},{"type":"v1","addr":"172.21.15.92:6837","nonce":1424858585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:24:16.257826+0000","modified":"2023-11-28T11:24:19.539818+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6213},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6213":{"gid":6213,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6836/3634521248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3634521248},{"type":"v1","addr":"172.21.15.92:6836","nonce":3634521248}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T11:24:22.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.437+0000 7f3fa77ff700 1 -- 172.21.15.92:0/841001752 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3f840682f0 msgr2=0x7f3f8406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:22.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.437+0000 7f3fa77ff700 1 --2- 172.21.15.92:0/841001752 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3f840682f0 0x7f3f8406a7a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c0090b0 tx=0x7f3f9c01c040 comp rx=0 tx=0).stop 2023-11-28T11:24:22.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.437+0000 7f3fa77ff700 1 -- 172.21.15.92:0/841001752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3fa00d0d60 msgr2=0x7f3fa01e6840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:22.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.437+0000 7f3fa77ff700 1 --2- 172.21.15.92:0/841001752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3fa00d0d60 0x7f3fa01e6840 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f3f9800cba0 tx=0x7f3f9800ceb0 comp rx=0 tx=0).stop 2023-11-28T11:24:22.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.437+0000 7f3fa77ff700 1 -- 172.21.15.92:0/841001752 shutdown_connections 2023-11-28T11:24:22.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.437+0000 7f3fa77ff700 1 --2- 172.21.15.92:0/841001752 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f3f840682f0 0x7f3f8406a7a0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:22.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.437+0000 7f3fa77ff700 1 --2- 172.21.15.92:0/841001752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3fa00d0d60 0x7f3fa01e6840 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:22.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.437+0000 7f3fa77ff700 1 --2- 172.21.15.92:0/841001752 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f3fa00b8510 0x7f3fa015a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:22.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.437+0000 7f3fa77ff700 1 --2- 172.21.15.92:0/841001752 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f3fa00cd460 0x7f3fa01e5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:22.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.437+0000 7f3fa77ff700 1 -- 172.21.15.92:0/841001752 >> 172.21.15.92:0/841001752 conn(0x7f3fa00719a0 msgr2=0x7f3fa00c9680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:22.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.437+0000 7f3fa77ff700 1 -- 172.21.15.92:0/841001752 shutdown_connections 2023-11-28T11:24:22.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:22.437+0000 7f3fa77ff700 1 -- 172.21.15.92:0/841001752 wait complete. 2023-11-28T11:24:22.439 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 32 2023-11-28T11:24:22.858 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 32, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T11:24:16.257826+0000', 'modified': '2023-11-28T11:24:19.539818+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6213}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6213': {'gid': 6213, 'name': 'a', 'rank': 0, 'incarnation': 28, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.92:6836/3634521248', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6834', 'nonce': 3634521248}, {'type': 'v1', 'addr': '172.21.15.92:6836', 'nonce': 3634521248}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T11:24:22.859 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T11:24:22.859 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-28T11:24:23.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.056+0000 7f027a3ff700 1 Processor -- start 2023-11-28T11:24:23.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.056+0000 7f027a3ff700 1 -- start start 2023-11-28T11:24:23.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.057+0000 7f027a3ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02740b8510 0x7f027406b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:23.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.057+0000 7f027a3ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f02740d0d60 0x7f02740d11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:23.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.057+0000 7f027a3ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02740b88e0 con 0x7f02740cd460 2023-11-28T11:24:23.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.057+0000 7f027a3ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02740cdbd0 con 0x7f02740d0d60 2023-11-28T11:24:23.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.057+0000 7f027a3ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02740d02d0 con 0x7f02740b8510 2023-11-28T11:24:23.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.057+0000 7f027899c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f02740d0d60 0x7f02740d11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:23.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.057+0000 7f027899c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f02740d0d60 0x7f02740d11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55238/0 (socket says 172.21.15.92:55238) 2023-11-28T11:24:23.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.057+0000 7f027899c700 1 -- 172.21.15.92:0/4180643589 learned_addr learned my addr 172.21.15.92:0/4180643589 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:23.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.058+0000 7f0273fff700 1 --2- 172.21.15.92:0/4180643589 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02740b8510 0x7f027406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:23.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.058+0000 7f027899c700 1 -- 172.21.15.92:0/4180643589 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02740b8510 msgr2=0x7f027406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:23.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.058+0000 7f027899c700 1 --2- 172.21.15.92:0/4180643589 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02740b8510 0x7f027406b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:23.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.058+0000 7f027899c700 1 -- 172.21.15.92:0/4180643589 >> v1:172.21.15.169:6789/0 conn(0x7f02740cd460 legacy=0x7f027406beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:23.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.058+0000 7f027899c700 1 -- 172.21.15.92:0/4180643589 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02740cd8f0 con 0x7f02740d0d60 2023-11-28T11:24:23.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.058+0000 7f027899c700 1 --2- 172.21.15.92:0/4180643589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02740d0d60 0x7f02740d11a0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f026c009e80 tx=0x7f026c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad58fc66a4ef30 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:23.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.058+0000 7f0272ffd700 1 -- 172.21.15.92:0/4180643589 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026c00e070 con 0x7f02740d0d60 2023-11-28T11:24:23.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.058+0000 7f0272ffd700 1 -- 172.21.15.92:0/4180643589 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f026c00b980 con 0x7f02740d0d60 2023-11-28T11:24:23.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.058+0000 7f0272ffd700 1 -- 172.21.15.92:0/4180643589 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026c00bd30 con 0x7f02740d0d60 2023-11-28T11:24:23.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.059+0000 7f027a3ff700 1 -- 172.21.15.92:0/4180643589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02740d0d60 msgr2=0x7f02740d11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:23.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.059+0000 7f027a3ff700 1 --2- 172.21.15.92:0/4180643589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02740d0d60 0x7f02740d11a0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f026c009e80 tx=0x7f026c00b330 comp rx=0 tx=0).stop 2023-11-28T11:24:23.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.059+0000 7f027a3ff700 1 -- 172.21.15.92:0/4180643589 shutdown_connections 2023-11-28T11:24:23.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.059+0000 7f027a3ff700 1 --2- 172.21.15.92:0/4180643589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02740d0d60 0x7f02740d11a0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:23.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.059+0000 7f027a3ff700 1 --2- 172.21.15.92:0/4180643589 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02740b8510 0x7f027406b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:23.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.059+0000 7f027a3ff700 1 -- 172.21.15.92:0/4180643589 >> 172.21.15.92:0/4180643589 conn(0x7f02740719a0 msgr2=0x7f0274071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:23.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.059+0000 7f027a3ff700 1 -- 172.21.15.92:0/4180643589 shutdown_connections 2023-11-28T11:24:23.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.059+0000 7f027a3ff700 1 -- 172.21.15.92:0/4180643589 wait complete. 2023-11-28T11:24:23.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.061+0000 7f027a3ff700 1 Processor -- start 2023-11-28T11:24:23.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.061+0000 7f027a3ff700 1 -- start start 2023-11-28T11:24:23.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.062+0000 7f027a3ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02740b8510 0x7f02741f6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:23.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.062+0000 7f027a3ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02740cd460 0x7f02741f2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:23.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.062+0000 7f027a3ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02740d0d60 0x7f0274157c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:23.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.062+0000 7f027a3ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0274158860 con 0x7f02740d0d60 2023-11-28T11:24:23.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.062+0000 7f027a3ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02741f2740 con 0x7f02740b8510 2023-11-28T11:24:23.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.062+0000 7f027a3ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02741f29c0 con 0x7f02740cd460 2023-11-28T11:24:23.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.062+0000 7f027899c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02740d0d60 0x7f0274157c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:23.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.062+0000 7f027899c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02740d0d60 0x7f0274157c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55248/0 (socket says 172.21.15.92:55248) 2023-11-28T11:24:23.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.062+0000 7f027899c700 1 -- 172.21.15.92:0/1583310769 learned_addr learned my addr 172.21.15.92:0/1583310769 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:23.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.062+0000 7f0273fff700 1 --2- 172.21.15.92:0/1583310769 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02740b8510 0x7f02741f6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:23.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.062+0000 7f02737fe700 1 --2- 172.21.15.92:0/1583310769 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02740cd460 0x7f02741f2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:23.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.062+0000 7f027899c700 1 -- 172.21.15.92:0/1583310769 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02740cd460 msgr2=0x7f02741f2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:23.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.062+0000 7f027899c700 1 --2- 172.21.15.92:0/1583310769 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02740cd460 0x7f02741f2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:23.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.062+0000 7f027899c700 1 -- 172.21.15.92:0/1583310769 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02740b8510 msgr2=0x7f02741f6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:23.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.062+0000 7f027899c700 1 --2- 172.21.15.92:0/1583310769 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02740b8510 0x7f02741f6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:23.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.062+0000 7f027899c700 1 -- 172.21.15.92:0/1583310769 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f026c009b30 con 0x7f02740d0d60 2023-11-28T11:24:23.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.062+0000 7f0273fff700 1 --2- 172.21.15.92:0/1583310769 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02740b8510 0x7f02741f6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:24:23.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.062+0000 7f02737fe700 1 --2- 172.21.15.92:0/1583310769 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02740cd460 0x7f02741f2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:24:23.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.063+0000 7f027899c700 1 --2- 172.21.15.92:0/1583310769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02740d0d60 0x7f0274157c40 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f0274072610 tx=0x7f026c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:23.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.063+0000 7f0270ff9700 1 -- 172.21.15.92:0/1583310769 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026c004960 con 0x7f02740d0d60 2023-11-28T11:24:23.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.063+0000 7f0270ff9700 1 -- 172.21.15.92:0/1583310769 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f026c0099b0 con 0x7f02740d0d60 2023-11-28T11:24:23.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.063+0000 7f0270ff9700 1 -- 172.21.15.92:0/1583310769 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026c01b670 con 0x7f02740d0d60 2023-11-28T11:24:23.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.063+0000 7f027a3ff700 1 -- 172.21.15.92:0/1583310769 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02740cfff0 con 0x7f02740d0d60 2023-11-28T11:24:23.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.063+0000 7f027a3ff700 1 -- 172.21.15.92:0/1583310769 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02740cfd10 con 0x7f02740d0d60 2023-11-28T11:24:23.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.065+0000 7f0270ff9700 1 -- 172.21.15.92:0/1583310769 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f026c021030 con 0x7f02740d0d60 2023-11-28T11:24:23.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.066+0000 7f0270ff9700 1 --2- 172.21.15.92:0/1583310769 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f02600682f0 0x7f026006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:23.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.066+0000 7f0270ff9700 1 -- 172.21.15.92:0/1583310769 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f026c087440 con 0x7f02740d0d60 2023-11-28T11:24:23.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.066+0000 7f0273fff700 1 --2- 172.21.15.92:0/1583310769 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f02600682f0 0x7f026006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:23.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.067+0000 7f0273fff700 1 --2- 172.21.15.92:0/1583310769 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f02600682f0 0x7f026006a7a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f02640099f0 tx=0x7f0264008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:23.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.068+0000 7f028dde3b80 1 Processor -- start 2023-11-28T11:24:23.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.068+0000 7f028dde3b80 1 -- start start 2023-11-28T11:24:23.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.069+0000 7f028dde3b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5613bd58ef50 0x5613bd5489f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:23.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.069+0000 7f028dde3b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613bd585ac0 0x5613bd58f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:23.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.069+0000 7f028dde3b80 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5613bd571660 0x5613bd549b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:23.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.069+0000 7f028dde3b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613bd4c6de0 con 0x5613bd58ef50 2023-11-28T11:24:23.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.069+0000 7f028dde3b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613bd433140 con 0x5613bd585ac0 2023-11-28T11:24:23.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.069+0000 7f028dde3b80 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5613bd4a1db0 con 0x5613bd571660 2023-11-28T11:24:23.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.069+0000 7f0273fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5613bd58ef50 0x5613bd5489f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:23.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.069+0000 7f0273fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5613bd58ef50 0x5613bd5489f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55262/0 (socket says 172.21.15.92:55262) 2023-11-28T11:24:23.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.069+0000 7f0273fff700 1 -- 172.21.15.92:0/364014497 learned_addr learned my addr 172.21.15.92:0/364014497 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:23.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.069+0000 7f02737fe700 1 --2- 172.21.15.92:0/364014497 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613bd585ac0 0x5613bd58f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:23.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.069+0000 7f027899c700 1 --2- 172.21.15.92:0/364014497 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5613bd571660 0x5613bd549b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:23.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.069+0000 7f0273fff700 1 -- 172.21.15.92:0/364014497 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5613bd571660 msgr2=0x5613bd549b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:23.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.069+0000 7f0273fff700 1 --2- 172.21.15.92:0/364014497 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5613bd571660 0x5613bd549b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:23.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.069+0000 7f0273fff700 1 -- 172.21.15.92:0/364014497 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613bd585ac0 msgr2=0x5613bd58f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:23.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.069+0000 7f0273fff700 1 --2- 172.21.15.92:0/364014497 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613bd585ac0 0x5613bd58f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:23.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.069+0000 7f0273fff700 1 -- 172.21.15.92:0/364014497 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5613bd28f790 con 0x5613bd58ef50 2023-11-28T11:24:23.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.070+0000 7f0273fff700 1 --2- 172.21.15.92:0/364014497 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5613bd58ef50 0x5613bd5489f0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f0264010940 tx=0x7f0264013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e16671a893cad43 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:23.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.070+0000 7f0259ffb700 1 -- 172.21.15.92:0/364014497 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0264015a00 con 0x5613bd58ef50 2023-11-28T11:24:23.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.070+0000 7f0259ffb700 1 -- 172.21.15.92:0/364014497 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f026401a070 con 0x5613bd58ef50 2023-11-28T11:24:23.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.070+0000 7f0259ffb700 1 -- 172.21.15.92:0/364014497 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0264015d60 con 0x5613bd58ef50 2023-11-28T11:24:23.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.071+0000 7f028dde3b80 1 -- 172.21.15.92:0/364014497 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5613bd58ef50 msgr2=0x5613bd5489f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:23.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.071+0000 7f028dde3b80 1 --2- 172.21.15.92:0/364014497 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5613bd58ef50 0x5613bd5489f0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f0264010940 tx=0x7f0264013000 comp rx=0 tx=0).stop 2023-11-28T11:24:23.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.071+0000 7f028dde3b80 1 -- 172.21.15.92:0/364014497 shutdown_connections 2023-11-28T11:24:23.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.071+0000 7f028dde3b80 1 --2- 172.21.15.92:0/364014497 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5613bd571660 0x5613bd549b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:23.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.071+0000 7f028dde3b80 1 --2- 172.21.15.92:0/364014497 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5613bd58ef50 0x5613bd5489f0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:23.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.071+0000 7f028dde3b80 1 --2- 172.21.15.92:0/364014497 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613bd585ac0 0x5613bd58f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:23.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.071+0000 7f028dde3b80 1 -- 172.21.15.92:0/364014497 >> 172.21.15.92:0/364014497 conn(0x5613bd53ed20 msgr2=0x5613bd547f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:23.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.071+0000 7f028dde3b80 1 -- 172.21.15.92:0/364014497 shutdown_connections 2023-11-28T11:24:23.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.071+0000 7f028dde3b80 1 -- 172.21.15.92:0/364014497 wait complete. 2023-11-28T11:24:23.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.072+0000 7f028dde3b80 1 -- start start 2023-11-28T11:24:23.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.072+0000 7f028dde3b80 1 Processor -- start 2023-11-28T11:24:23.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.073+0000 7f028dde3b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613bd571660 0x5613bd5739f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:23.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.073+0000 7f028dde3b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5613bd585ac0 0x5613bd58dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:23.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.073+0000 7f028dde3b80 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5613bd58ef50 0x5613bd58e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:23.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.073+0000 7f028dde3b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613bd49b120 con 0x5613bd585ac0 2023-11-28T11:24:23.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.073+0000 7f028dde3b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613bd290390 con 0x5613bd571660 2023-11-28T11:24:23.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.073+0000 7f028dde3b80 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5613bd433340 con 0x5613bd58ef50 2023-11-28T11:24:23.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.073+0000 7f028dde3b80 10 client.0 _ll_register_callbacks cb 0x5613bd439400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:24:23.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.073+0000 7f02737fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5613bd585ac0 0x5613bd58dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:23.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.073+0000 7f02737fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5613bd585ac0 0x5613bd58dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55270/0 (socket says 172.21.15.92:55270) 2023-11-28T11:24:23.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.073+0000 7f02737fe700 1 -- 172.21.15.92:0/462352595 learned_addr learned my addr 172.21.15.92:0/462352595 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:23.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.073+0000 7f027899c700 1 --2- 172.21.15.92:0/462352595 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5613bd58ef50 0x5613bd58e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:23.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.073+0000 7f0273fff700 1 --2- 172.21.15.92:0/462352595 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613bd571660 0x5613bd5739f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:23.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.073+0000 7f02737fe700 1 -- 172.21.15.92:0/462352595 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5613bd58ef50 msgr2=0x5613bd58e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:23.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.073+0000 7f02737fe700 1 --2- 172.21.15.92:0/462352595 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5613bd58ef50 0x5613bd58e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:23.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.073+0000 7f02737fe700 1 -- 172.21.15.92:0/462352595 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613bd571660 msgr2=0x5613bd5739f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:23.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.073+0000 7f02737fe700 1 --2- 172.21.15.92:0/462352595 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613bd571660 0x5613bd5739f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:23.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.073+0000 7f02737fe700 1 -- 172.21.15.92:0/462352595 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f026400e050 con 0x5613bd585ac0 2023-11-28T11:24:23.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.073+0000 7f0273fff700 1 --2- 172.21.15.92:0/462352595 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613bd571660 0x5613bd5739f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:24:23.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.074+0000 7f02737fe700 1 --2- 172.21.15.92:0/462352595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5613bd585ac0 0x5613bd58dd60 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f0268003c40 tx=0x7f02680011c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=985cd1768a069795 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:23.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.074+0000 7f0252ffd700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-11-28T11:24:23.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.074+0000 7f0252ffd700 1 -- 172.21.15.92:0/462352595 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0268001b00 con 0x5613bd585ac0 2023-11-28T11:24:23.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.074+0000 7f0252ffd700 1 -- 172.21.15.92:0/462352595 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(68..68 src has 1..68) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f0268001c60 con 0x5613bd585ac0 2023-11-28T11:24:23.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.074+0000 7f028dde3b80 1 -- 172.21.15.92:0/462352595 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5613bd49f030 con 0x5613bd585ac0 2023-11-28T11:24:23.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.074+0000 7f0252ffd700 1 -- 172.21.15.92:0/462352595 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0268005250 con 0x5613bd585ac0 2023-11-28T11:24:23.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.074+0000 7f028dde3b80 10 client.6351 fetch_fsmap learned FSMap version 32 2023-11-28T11:24:23.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.074+0000 7f028dde3b80 1 -- 172.21.15.92:0/462352595 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x5613bd37c0a0 con 0x5613bd585ac0 2023-11-28T11:24:23.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.074+0000 7f0252ffd700 1 -- 172.21.15.92:0/462352595 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5613bd49f030 con 0x5613bd585ac0 2023-11-28T11:24:23.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.074+0000 7f0252ffd700 1 -- 172.21.15.92:0/462352595 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 32) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f026801c350 con 0x5613bd585ac0 2023-11-28T11:24:23.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.075+0000 7f028dde3b80 10 client.6351 fetch_fsmap finished waiting for FSMap version 32 2023-11-28T11:24:23.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.075+0000 7f028dde3b80 10 client.6351 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T11:24:23.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.075+0000 7f028dde3b80 20 client.6351 populate_metadata read hostname 'smithi092' 2023-11-28T11:24:23.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.075+0000 7f028dde3b80 1 --2- 172.21.15.92:0/462352595 >> [v2:172.21.15.92:6834/3634521248,v1:172.21.15.92:6836/3634521248] conn(0x5613bd573f30 0x5613bd5723b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:23.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.075+0000 7f028dde3b80 4 client.6351 mds_command: new command op to 6213 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:24:23.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.075+0000 7f028dde3b80 1 -- 172.21.15.92:0/462352595 --> [v2:172.21.15.92:6834/3634521248,v1:172.21.15.92:6836/3634521248] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5613bd2eaef0 con 0x5613bd573f30 2023-11-28T11:24:23.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.075+0000 7f0273fff700 1 --2- 172.21.15.92:0/462352595 >> [v2:172.21.15.92:6834/3634521248,v1:172.21.15.92:6836/3634521248] conn(0x5613bd573f30 0x5613bd5723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:23.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.075+0000 7f0252ffd700 10 client.6351 ms_handle_connect on v2:172.21.15.92:6834/3634521248 2023-11-28T11:24:23.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.075+0000 7f0273fff700 1 --2- 172.21.15.92:0/462352595 >> [v2:172.21.15.92:6834/3634521248,v1:172.21.15.92:6836/3634521248] conn(0x5613bd573f30 0x5613bd5723b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dac1083b83064493 server_cookie=58d85676e2c94521 in_seq=0 out_seq=0 2023-11-28T11:24:23.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.076+0000 7f0252ffd700 1 -- 172.21.15.92:0/462352595 <== mds.0 v2:172.21.15.92:6834/3634521248 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x5613bd2eaef0 con 0x5613bd573f30 2023-11-28T11:24:23.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.076+0000 7f0252ffd700 10 client.6351 handle_command_reply: tid=0 2023-11-28T11:24:23.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.076+0000 7f028dde3b80 1 client.6351 shutdown 2023-11-28T11:24:23.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.076+0000 7f028dde3b80 1 -- 172.21.15.92:0/462352595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5613bd585ac0 msgr2=0x5613bd58dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:23.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.076+0000 7f028dde3b80 1 --2- 172.21.15.92:0/462352595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5613bd585ac0 0x5613bd58dd60 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f0268003c40 tx=0x7f02680011c0 comp rx=0 tx=0).stop 2023-11-28T11:24:23.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.076+0000 7f028dde3b80 1 -- 172.21.15.92:0/462352595 shutdown_connections 2023-11-28T11:24:23.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.076+0000 7f028dde3b80 1 --2- 172.21.15.92:0/462352595 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5613bd58ef50 0x5613bd58e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:23.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.076+0000 7f028dde3b80 1 --2- 172.21.15.92:0/462352595 >> [v2:172.21.15.92:6834/3634521248,v1:172.21.15.92:6836/3634521248] conn(0x5613bd573f30 0x5613bd5723b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:23.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.076+0000 7f028dde3b80 1 --2- 172.21.15.92:0/462352595 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613bd571660 0x5613bd5739f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:23.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.076+0000 7f028dde3b80 1 --2- 172.21.15.92:0/462352595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5613bd585ac0 0x5613bd58dd60 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:23.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.076+0000 7f0252ffd700 0 client.6351 ms_handle_reset on v2:172.21.15.92:6834/3634521248 2023-11-28T11:24:23.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.076+0000 7f028dde3b80 1 -- 172.21.15.92:0/462352595 >> 172.21.15.92:0/462352595 conn(0x5613bd53ed20 msgr2=0x5613bd570ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:23.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.077+0000 7f028dde3b80 1 -- 172.21.15.92:0/462352595 shutdown_connections 2023-11-28T11:24:23.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.077+0000 7f028dde3b80 1 -- 172.21.15.92:0/462352595 wait complete. 2023-11-28T11:24:23.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.077+0000 7f028dde3b80 20 client.6351 trim_cache size 0 max 16384 2023-11-28T11:24:23.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.094+0000 7f028dde3b80 1 Processor -- start 2023-11-28T11:24:23.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.094+0000 7f028dde3b80 1 -- start start 2023-11-28T11:24:23.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.095+0000 7f028dde3b80 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5613bd571660 0x5613bd58e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:23.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.095+0000 7f028dde3b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5613bd585ac0 0x5613bd5776d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:23.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.095+0000 7f028dde3b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613bd58ef50 0x5613bd577c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:23.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.095+0000 7f028dde3b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613bd59d420 con 0x5613bd585ac0 2023-11-28T11:24:23.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.095+0000 7f028dde3b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613bd540da0 con 0x5613bd58ef50 2023-11-28T11:24:23.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.095+0000 7f028dde3b80 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5613bd54adb0 con 0x5613bd571660 2023-11-28T11:24:23.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.095+0000 7f02737fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5613bd585ac0 0x5613bd5776d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:23.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.095+0000 7f02737fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5613bd585ac0 0x5613bd5776d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55276/0 (socket says 172.21.15.92:55276) 2023-11-28T11:24:23.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.095+0000 7f02737fe700 1 -- 172.21.15.92:0/1481367552 learned_addr learned my addr 172.21.15.92:0/1481367552 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:23.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.096+0000 7f0273fff700 1 --2- 172.21.15.92:0/1481367552 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5613bd571660 0x5613bd58e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:23.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.096+0000 7f027899c700 1 --2- 172.21.15.92:0/1481367552 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613bd58ef50 0x5613bd577c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:23.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.096+0000 7f02737fe700 1 -- 172.21.15.92:0/1481367552 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5613bd571660 msgr2=0x5613bd58e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:23.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.096+0000 7f02737fe700 1 --2- 172.21.15.92:0/1481367552 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5613bd571660 0x5613bd58e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:23.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.096+0000 7f02737fe700 1 -- 172.21.15.92:0/1481367552 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613bd58ef50 msgr2=0x5613bd577c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:23.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.096+0000 7f02737fe700 1 --2- 172.21.15.92:0/1481367552 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613bd58ef50 0x5613bd577c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:23.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.096+0000 7f02737fe700 1 -- 172.21.15.92:0/1481367552 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5613bd4622e0 con 0x5613bd585ac0 2023-11-28T11:24:23.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.096+0000 7f027899c700 1 --2- 172.21.15.92:0/1481367552 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613bd58ef50 0x5613bd577c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:24:23.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.096+0000 7f02737fe700 1 --2- 172.21.15.92:0/1481367552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5613bd585ac0 0x5613bd5776d0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f0268007ec0 tx=0x7f026800e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=351e01e735a6bd34 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:23.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.096+0000 7f025a7fc700 1 -- 172.21.15.92:0/1481367552 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02680015c0 con 0x5613bd585ac0 2023-11-28T11:24:23.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.096+0000 7f025a7fc700 1 -- 172.21.15.92:0/1481367552 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f026801c8b0 con 0x5613bd585ac0 2023-11-28T11:24:23.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.096+0000 7f025a7fc700 1 -- 172.21.15.92:0/1481367552 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0268001b00 con 0x5613bd585ac0 2023-11-28T11:24:23.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.097+0000 7f028dde3b80 1 -- 172.21.15.92:0/1481367552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5613bd585ac0 msgr2=0x5613bd5776d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:23.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.097+0000 7f028dde3b80 1 --2- 172.21.15.92:0/1481367552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5613bd585ac0 0x5613bd5776d0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f0268007ec0 tx=0x7f026800e000 comp rx=0 tx=0).stop 2023-11-28T11:24:23.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.097+0000 7f028dde3b80 1 -- 172.21.15.92:0/1481367552 shutdown_connections 2023-11-28T11:24:23.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.097+0000 7f028dde3b80 1 --2- 172.21.15.92:0/1481367552 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613bd58ef50 0x5613bd577c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:23.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.097+0000 7f028dde3b80 1 --2- 172.21.15.92:0/1481367552 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5613bd571660 0x5613bd58e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:23.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.097+0000 7f028dde3b80 1 --2- 172.21.15.92:0/1481367552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5613bd585ac0 0x5613bd5776d0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:23.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.097+0000 7f028dde3b80 1 -- 172.21.15.92:0/1481367552 >> 172.21.15.92:0/1481367552 conn(0x5613bd53ed20 msgr2=0x5613bd570ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:23.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.097+0000 7f028dde3b80 1 -- 172.21.15.92:0/1481367552 shutdown_connections 2023-11-28T11:24:23.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.097+0000 7f028dde3b80 1 -- 172.21.15.92:0/1481367552 wait complete. 2023-11-28T11:24:23.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.098+0000 7f028dde3b80 1 -- start start 2023-11-28T11:24:23.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.098+0000 7f028dde3b80 1 Processor -- start 2023-11-28T11:24:23.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.099+0000 7f028dde3b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5613bd571660 0x5613bd5f4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:23.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.099+0000 7f028dde3b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613bd585ac0 0x5613bd542cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:23.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.099+0000 7f028dde3b80 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5613bd58ef50 0x5613bd543230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:23.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.099+0000 7f028dde3b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613bd543a80 con 0x5613bd571660 2023-11-28T11:24:23.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.099+0000 7f028dde3b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613bd584d40 con 0x5613bd585ac0 2023-11-28T11:24:23.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.099+0000 7f028dde3b80 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5613bd5f4580 con 0x5613bd58ef50 2023-11-28T11:24:23.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.099+0000 7f028dde3b80 10 client.0 _ll_register_callbacks cb 0x5613bd439400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:24:23.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.099+0000 7f0273fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5613bd571660 0x5613bd5f4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:23.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.099+0000 7f0273fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5613bd571660 0x5613bd5f4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55292/0 (socket says 172.21.15.92:55292) 2023-11-28T11:24:23.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.099+0000 7f0273fff700 1 -- 172.21.15.92:0/325154092 learned_addr learned my addr 172.21.15.92:0/325154092 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:23.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.099+0000 7f02737fe700 1 --2- 172.21.15.92:0/325154092 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613bd585ac0 0x5613bd542cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:23.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.099+0000 7f027899c700 1 --2- 172.21.15.92:0/325154092 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5613bd58ef50 0x5613bd543230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:23.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.100+0000 7f0273fff700 1 -- 172.21.15.92:0/325154092 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5613bd58ef50 msgr2=0x5613bd543230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:23.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.100+0000 7f0273fff700 1 --2- 172.21.15.92:0/325154092 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5613bd58ef50 0x5613bd543230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:23.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.100+0000 7f0273fff700 1 -- 172.21.15.92:0/325154092 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613bd585ac0 msgr2=0x5613bd542cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:23.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.100+0000 7f0273fff700 1 --2- 172.21.15.92:0/325154092 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613bd585ac0 0x5613bd542cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:23.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.100+0000 7f0273fff700 1 -- 172.21.15.92:0/325154092 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5613bd37c0a0 con 0x5613bd571660 2023-11-28T11:24:23.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.100+0000 7f0273fff700 1 --2- 172.21.15.92:0/325154092 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5613bd571660 0x5613bd5f4a10 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f0264023040 tx=0x7f02640047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b604147233a7487e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:23.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.100+0000 7f0252ffd700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-11-28T11:24:23.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.100+0000 7f0252ffd700 1 -- 172.21.15.92:0/325154092 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0264032040 con 0x5613bd571660 2023-11-28T11:24:23.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.100+0000 7f0252ffd700 1 -- 172.21.15.92:0/325154092 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(68..68 src has 1..68) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f0264005be0 con 0x5613bd571660 2023-11-28T11:24:23.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.100+0000 7f028dde3b80 1 -- 172.21.15.92:0/325154092 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f026401a070 con 0x5613bd571660 2023-11-28T11:24:23.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.100+0000 7f0252ffd700 1 -- 172.21.15.92:0/325154092 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0264035490 con 0x5613bd571660 2023-11-28T11:24:23.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.100+0000 7f028dde3b80 10 client.6357 fetch_fsmap learned FSMap version 32 2023-11-28T11:24:23.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.101+0000 7f028dde3b80 1 -- 172.21.15.92:0/325154092 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x5613bd2f01d0 con 0x5613bd571660 2023-11-28T11:24:23.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.101+0000 7f0252ffd700 1 -- 172.21.15.92:0/325154092 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f026401a070 con 0x5613bd571660 2023-11-28T11:24:23.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.101+0000 7f0252ffd700 1 -- 172.21.15.92:0/325154092 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 32) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f0264015800 con 0x5613bd571660 2023-11-28T11:24:23.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.101+0000 7f028dde3b80 10 client.6357 fetch_fsmap finished waiting for FSMap version 32 2023-11-28T11:24:23.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.101+0000 7f028dde3b80 10 client.6357 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T11:24:23.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.101+0000 7f028dde3b80 20 client.6357 populate_metadata read hostname 'smithi092' 2023-11-28T11:24:23.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.101+0000 7f028dde3b80 1 --2- 172.21.15.92:0/325154092 >> [v2:172.21.15.92:6834/3634521248,v1:172.21.15.92:6836/3634521248] conn(0x5613bd577520 0x5613bd5778f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:23.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.101+0000 7f028dde3b80 4 client.6357 mds_command: new command op to 6213 tid=0[{"prefix": "status"}] 2023-11-28T11:24:23.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.101+0000 7f028dde3b80 1 -- 172.21.15.92:0/325154092 --> [v2:172.21.15.92:6834/3634521248,v1:172.21.15.92:6836/3634521248] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5613bd320040 con 0x5613bd577520 2023-11-28T11:24:23.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.101+0000 7f02737fe700 1 --2- 172.21.15.92:0/325154092 >> [v2:172.21.15.92:6834/3634521248,v1:172.21.15.92:6836/3634521248] conn(0x5613bd577520 0x5613bd5778f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:23.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.102+0000 7f0252ffd700 10 client.6357 ms_handle_connect on v2:172.21.15.92:6834/3634521248 2023-11-28T11:24:23.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.102+0000 7f02737fe700 1 --2- 172.21.15.92:0/325154092 >> [v2:172.21.15.92:6834/3634521248,v1:172.21.15.92:6836/3634521248] conn(0x5613bd577520 0x5613bd5778f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a49e274483d21d59 server_cookie=1d1b56b06743c007 in_seq=0 out_seq=0 2023-11-28T11:24:23.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.102+0000 7f0252ffd700 1 -- 172.21.15.92:0/325154092 <== mds.0 v2:172.21.15.92:6834/3634521248 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x5613bd320040 con 0x5613bd577520 2023-11-28T11:24:23.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.102+0000 7f0252ffd700 10 client.6357 handle_command_reply: tid=0 2023-11-28T11:24:23.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.102+0000 7f028dde3b80 1 client.6357 shutdown 2023-11-28T11:24:23.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.102+0000 7f028dde3b80 1 -- 172.21.15.92:0/325154092 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5613bd571660 msgr2=0x5613bd5f4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:23.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.102+0000 7f028dde3b80 1 --2- 172.21.15.92:0/325154092 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5613bd571660 0x5613bd5f4a10 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f0264023040 tx=0x7f02640047c0 comp rx=0 tx=0).stop 2023-11-28T11:24:23.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.102+0000 7f028dde3b80 1 -- 172.21.15.92:0/325154092 shutdown_connections 2023-11-28T11:24:23.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.102+0000 7f028dde3b80 1 --2- 172.21.15.92:0/325154092 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5613bd58ef50 0x5613bd543230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:23.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.102+0000 7f028dde3b80 1 --2- 172.21.15.92:0/325154092 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5613bd571660 0x5613bd5f4a10 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:23.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.102+0000 7f028dde3b80 1 --2- 172.21.15.92:0/325154092 >> [v2:172.21.15.92:6834/3634521248,v1:172.21.15.92:6836/3634521248] conn(0x5613bd577520 0x5613bd5778f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:23.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.102+0000 7f028dde3b80 1 --2- 172.21.15.92:0/325154092 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5613bd585ac0 0x5613bd542cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:23.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.102+0000 7f0252ffd700 0 client.6357 ms_handle_reset on v2:172.21.15.92:6834/3634521248 2023-11-28T11:24:23.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.102+0000 7f028dde3b80 1 -- 172.21.15.92:0/325154092 >> 172.21.15.92:0/325154092 conn(0x5613bd53ed20 msgr2=0x5613bd58e180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:23.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.102+0000 7f028dde3b80 1 -- 172.21.15.92:0/325154092 shutdown_connections 2023-11-28T11:24:23.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.102+0000 7f028dde3b80 1 -- 172.21.15.92:0/325154092 wait complete. 2023-11-28T11:24:23.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.102+0000 7f028dde3b80 20 client.6357 trim_cache size 0 max 16384 2023-11-28T11:24:23.110 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-28T11:24:23.110 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "68956e0c-c16f-47cd-a220-f98364b0831d", 2023-11-28T11:24:23.111 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 0, 2023-11-28T11:24:23.111 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 6213, 2023-11-28T11:24:23.111 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-11-28T11:24:23.111 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-11-28T11:24:23.111 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-11-28T11:24:23.111 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 6.8425207840000004, 2023-11-28T11:24:23.111 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 32, 2023-11-28T11:24:23.111 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 68, 2023-11-28T11:24:23.111 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 68, 2023-11-28T11:24:23.111 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 12.849425659 2023-11-28T11:24:23.111 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-28T11:24:23.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.103+0000 7f027a3ff700 1 -- 172.21.15.92:0/1583310769 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f02600682f0 msgr2=0x7f026006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:23.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.103+0000 7f027a3ff700 1 --2- 172.21.15.92:0/1583310769 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f02600682f0 0x7f026006a7a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f02640099f0 tx=0x7f0264008040 comp rx=0 tx=0).stop 2023-11-28T11:24:23.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.103+0000 7f027a3ff700 1 -- 172.21.15.92:0/1583310769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02740d0d60 msgr2=0x7f0274157c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:23.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.103+0000 7f027a3ff700 1 --2- 172.21.15.92:0/1583310769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02740d0d60 0x7f0274157c40 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f0274072610 tx=0x7f026c004ad0 comp rx=0 tx=0).stop 2023-11-28T11:24:23.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.103+0000 7f027a3ff700 1 -- 172.21.15.92:0/1583310769 shutdown_connections 2023-11-28T11:24:23.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.103+0000 7f027a3ff700 1 --2- 172.21.15.92:0/1583310769 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f02600682f0 0x7f026006a7a0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:23.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.103+0000 7f027a3ff700 1 --2- 172.21.15.92:0/1583310769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02740d0d60 0x7f0274157c40 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:23.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.103+0000 7f027a3ff700 1 --2- 172.21.15.92:0/1583310769 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f02740b8510 0x7f02741f6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:23.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.103+0000 7f027a3ff700 1 --2- 172.21.15.92:0/1583310769 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f02740cd460 0x7f02741f2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:23.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.103+0000 7f027a3ff700 1 -- 172.21.15.92:0/1583310769 >> 172.21.15.92:0/1583310769 conn(0x7f02740719a0 msgr2=0x7f02740d2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:23.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.103+0000 7f027a3ff700 1 -- 172.21.15.92:0/1583310769 shutdown_connections 2023-11-28T11:24:23.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.103+0000 7f027a3ff700 1 -- 172.21.15.92:0/1583310769 wait complete. 2023-11-28T11:24:23.121 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T11:24:23.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.316+0000 7f06b6fff700 1 Processor -- start 2023-11-28T11:24:23.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.316+0000 7f06b6fff700 1 -- start start 2023-11-28T11:24:23.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.317+0000 7f06b6fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f06b00b8510 0x7f06b006b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:23.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.317+0000 7f06b6fff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06b00b88e0 con 0x7f06b00cd460 2023-11-28T11:24:23.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.317+0000 7f06b6fff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06b00cd8f0 con 0x7f06b00d0d60 2023-11-28T11:24:23.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.317+0000 7f06b6fff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06b00cdbd0 con 0x7f06b00b8510 2023-11-28T11:24:23.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.317+0000 7f06b4d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f06b00b8510 0x7f06b006b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:23.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.317+0000 7f06affff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f06b00cd460 0x7f06b006beb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35732/0 (socket says 172.21.15.92:35732) 2023-11-28T11:24:23.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.317+0000 7f06affff700 1 -- 172.21.15.92:0/1285873687 learned_addr learned my addr 172.21.15.92:0/1285873687 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:24:23.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.318+0000 7f06b4d9b700 1 -- 172.21.15.92:0/1285873687 >> v1:172.21.15.169:6789/0 conn(0x7f06b00d0d60 legacy=0x7f06b00d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:23.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.318+0000 7f06b4d9b700 1 -- 172.21.15.92:0/1285873687 >> v1:172.21.15.92:6789/0 conn(0x7f06b00cd460 legacy=0x7f06b006beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:23.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.318+0000 7f06b4d9b700 1 -- 172.21.15.92:0/1285873687 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06b00cfd10 con 0x7f06b00b8510 2023-11-28T11:24:23.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.318+0000 7f06af7fe700 1 -- 172.21.15.92:0/1285873687 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 683785270 0 0) 0x7f06b00b88e0 con 0x7f06b00cd460 2023-11-28T11:24:23.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.318+0000 7f06b4d9b700 1 --2- 172.21.15.92:0/1285873687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06b00b8510 0x7f06b006b970 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f06a4009d90 tx=0x7f06a400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e9db4cf0d1cd38a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:23.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.318+0000 7f06af7fe700 1 -- 172.21.15.92:0/1285873687 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a400e070 con 0x7f06b00b8510 2023-11-28T11:24:23.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.318+0000 7f06af7fe700 1 -- 172.21.15.92:0/1285873687 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a400bb90 con 0x7f06b00b8510 2023-11-28T11:24:23.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.318+0000 7f06af7fe700 1 -- 172.21.15.92:0/1285873687 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a4003a20 con 0x7f06b00b8510 2023-11-28T11:24:23.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.319+0000 7f06b6fff700 1 -- 172.21.15.92:0/1285873687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06b00b8510 msgr2=0x7f06b006b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:23.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.319+0000 7f06b6fff700 1 --2- 172.21.15.92:0/1285873687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06b00b8510 0x7f06b006b970 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f06a4009d90 tx=0x7f06a400b330 comp rx=0 tx=0).stop 2023-11-28T11:24:23.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.319+0000 7f06b6fff700 1 -- 172.21.15.92:0/1285873687 shutdown_connections 2023-11-28T11:24:23.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.319+0000 7f06b6fff700 1 --2- 172.21.15.92:0/1285873687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06b00b8510 0x7f06b006b970 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:23.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.319+0000 7f06b6fff700 1 -- 172.21.15.92:0/1285873687 >> 172.21.15.92:0/1285873687 conn(0x7f06b00719a0 msgr2=0x7f06b0071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:23.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.319+0000 7f06b6fff700 1 -- 172.21.15.92:0/1285873687 shutdown_connections 2023-11-28T11:24:23.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.319+0000 7f06b6fff700 1 -- 172.21.15.92:0/1285873687 wait complete. 2023-11-28T11:24:23.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.321+0000 7f06b6fff700 1 Processor -- start 2023-11-28T11:24:23.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.321+0000 7f06b6fff700 1 -- start start 2023-11-28T11:24:23.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.322+0000 7f06b6fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06b00b8510 0x7f06b015ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:23.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.322+0000 7f06b6fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06b00cd460 0x7f06b00b34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:23.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.322+0000 7f06b6fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06b00d0d60 0x7f06b00b39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:23.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.322+0000 7f06b6fff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b01e65f0 con 0x7f06b00cd460 2023-11-28T11:24:23.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.322+0000 7f06b6fff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b01f5310 con 0x7f06b00d0d60 2023-11-28T11:24:23.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.322+0000 7f06b6fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06b01e5970 con 0x7f06b00b8510 2023-11-28T11:24:23.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.322+0000 7f06affff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06b00cd460 0x7f06b00b34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:23.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.322+0000 7f06affff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06b00cd460 0x7f06b00b34a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55308/0 (socket says 172.21.15.92:55308) 2023-11-28T11:24:23.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.322+0000 7f06affff700 1 -- 172.21.15.92:0/2760995384 learned_addr learned my addr 172.21.15.92:0/2760995384 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:23.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.322+0000 7f06b4d9b700 1 --2- 172.21.15.92:0/2760995384 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06b00b8510 0x7f06b015ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:23.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.322+0000 7f06b559c700 1 --2- 172.21.15.92:0/2760995384 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06b00d0d60 0x7f06b00b39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:23.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.322+0000 7f06affff700 1 -- 172.21.15.92:0/2760995384 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06b00b8510 msgr2=0x7f06b015ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:23.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.322+0000 7f06affff700 1 --2- 172.21.15.92:0/2760995384 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06b00b8510 0x7f06b015ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:23.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.322+0000 7f06affff700 1 -- 172.21.15.92:0/2760995384 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06b00d0d60 msgr2=0x7f06b00b39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:23.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.322+0000 7f06affff700 1 --2- 172.21.15.92:0/2760995384 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06b00d0d60 0x7f06b00b39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:23.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.322+0000 7f06affff700 1 -- 172.21.15.92:0/2760995384 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a4009a70 con 0x7f06b00cd460 2023-11-28T11:24:23.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.322+0000 7f06b4d9b700 1 --2- 172.21.15.92:0/2760995384 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06b00b8510 0x7f06b015ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:24:23.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.322+0000 7f06b559c700 1 --2- 172.21.15.92:0/2760995384 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06b00d0d60 0x7f06b00b39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:24:23.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.322+0000 7f06affff700 1 --2- 172.21.15.92:0/2760995384 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06b00cd460 0x7f06b00b34a0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f069c00c550 tx=0x7f069c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:23.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.323+0000 7f06ad7fa700 1 -- 172.21.15.92:0/2760995384 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069c004c80 con 0x7f06b00cd460 2023-11-28T11:24:23.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.323+0000 7f06ad7fa700 1 -- 172.21.15.92:0/2760995384 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f069c004de0 con 0x7f06b00cd460 2023-11-28T11:24:23.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.323+0000 7f06ad7fa700 1 -- 172.21.15.92:0/2760995384 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069c007310 con 0x7f06b00cd460 2023-11-28T11:24:23.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.323+0000 7f06b6fff700 1 -- 172.21.15.92:0/2760995384 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06b00d29c0 con 0x7f06b00cd460 2023-11-28T11:24:23.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.323+0000 7f06b6fff700 1 -- 172.21.15.92:0/2760995384 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06b00cfff0 con 0x7f06b00cd460 2023-11-28T11:24:23.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.324+0000 7f06b6fff700 1 -- 172.21.15.92:0/2760995384 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06b00d2b50 con 0x7f06b00cd460 2023-11-28T11:24:23.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.325+0000 7f06ad7fa700 1 -- 172.21.15.92:0/2760995384 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f069c01a020 con 0x7f06b00cd460 2023-11-28T11:24:23.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.326+0000 7f06ad7fa700 1 --2- 172.21.15.92:0/2760995384 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f06940682f0 0x7f069406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:23.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.326+0000 7f06ad7fa700 1 -- 172.21.15.92:0/2760995384 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f069c085a40 con 0x7f06b00cd460 2023-11-28T11:24:23.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.326+0000 7f06b4d9b700 1 --2- 172.21.15.92:0/2760995384 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f06940682f0 0x7f069406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:23.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.327+0000 7f06b4d9b700 1 --2- 172.21.15.92:0/2760995384 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f06940682f0 0x7f069406a7a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f06b00cf500 tx=0x7f06a4011000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:23.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.330+0000 7f06ad7fa700 1 -- 172.21.15.92:0/2760995384 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f069c052070 con 0x7f06b00cd460 2023-11-28T11:24:23.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.564+0000 7f06b6fff700 1 -- 172.21.15.92:0/2760995384 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f06b01e6730 con 0x7f06b00cd460 2023-11-28T11:24:23.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.565+0000 7f06ad7fa700 1 -- 172.21.15.92:0/2760995384 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v32) v1 ==== 53+0+73 (secure 0 0 0) 0x7f06b01e6730 con 0x7f06b00cd460 2023-11-28T11:24:23.566 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T11:24:23.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.567+0000 7f06b6fff700 1 -- 172.21.15.92:0/2760995384 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f06940682f0 msgr2=0x7f069406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:23.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.567+0000 7f06b6fff700 1 --2- 172.21.15.92:0/2760995384 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f06940682f0 0x7f069406a7a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f06b00cf500 tx=0x7f06a4011000 comp rx=0 tx=0).stop 2023-11-28T11:24:23.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.567+0000 7f06b6fff700 1 -- 172.21.15.92:0/2760995384 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06b00cd460 msgr2=0x7f06b00b34a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:23.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.567+0000 7f06b6fff700 1 --2- 172.21.15.92:0/2760995384 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06b00cd460 0x7f06b00b34a0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f069c00c550 tx=0x7f069c00c910 comp rx=0 tx=0).stop 2023-11-28T11:24:23.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.567+0000 7f06b6fff700 1 -- 172.21.15.92:0/2760995384 shutdown_connections 2023-11-28T11:24:23.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.567+0000 7f06b6fff700 1 --2- 172.21.15.92:0/2760995384 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f06b00d0d60 0x7f06b00b39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:23.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.567+0000 7f06b6fff700 1 --2- 172.21.15.92:0/2760995384 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f06b00b8510 0x7f06b015ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:23.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.567+0000 7f06b6fff700 1 --2- 172.21.15.92:0/2760995384 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f06940682f0 0x7f069406a7a0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:23.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.567+0000 7f06b6fff700 1 --2- 172.21.15.92:0/2760995384 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06b00cd460 0x7f06b00b34a0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:23.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.567+0000 7f06b6fff700 1 -- 172.21.15.92:0/2760995384 >> 172.21.15.92:0/2760995384 conn(0x7f06b00719a0 msgr2=0x7f06b00ccd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:23.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.567+0000 7f06b6fff700 1 -- 172.21.15.92:0/2760995384 shutdown_connections 2023-11-28T11:24:23.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:23.567+0000 7f06b6fff700 1 -- 172.21.15.92:0/2760995384 wait complete. 2023-11-28T11:24:23.586 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-28T11:24:23.587 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T11:24:23.587 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi092.front.sepia.ceph.com 2023-11-28T11:24:23.587 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T11:24:23.587 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T11:24:23.587 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-28T11:24:23.587 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-28T11:24:23.587 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T11:24:23.588 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:24:23.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:24:23.588 DEBUG:teuthology.orchestra.run.smithi092:> ip addr 2023-11-28T11:24:23.636 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:24:23.636 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:24:23.637 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:24:23.637 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:24:23.637 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-11-28T11:24:23.637 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:24:23.637 INFO:teuthology.orchestra.run.smithi092.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:24:23.637 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b2 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:24:23.638 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f0 2023-11-28T11:24:23.638 INFO:teuthology.orchestra.run.smithi092.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:24:23.638 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b3 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:24:23.638 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f1 2023-11-28T11:24:23.638 INFO:teuthology.orchestra.run.smithi092.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:24:23.638 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:44 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:24:23.638 INFO:teuthology.orchestra.run.smithi092.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T11:24:23.639 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:45 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:24:23.639 INFO:teuthology.orchestra.run.smithi092.stdout: inet 172.21.15.92/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T11:24:23.639 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft 41791sec preferred_lft 41791sec 2023-11-28T11:24:23.639 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::ec4:7aff:fe8f:cd45/64 scope link 2023-11-28T11:24:23.639 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:24:23.639 INFO:teuthology.orchestra.run.smithi092.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T11:24:23.640 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 02:f9:a8:fe:83:2e brd ff:ff:ff:ff:ff:ff 2023-11-28T11:24:23.640 INFO:teuthology.orchestra.run.smithi092.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T11:24:23.640 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:24:23.640 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::3ca6:3fff:feb6:2c13/64 scope link 2023-11-28T11:24:23.640 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:24:23.640 INFO:teuthology.orchestra.run.smithi092.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T11:24:23.640 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 4e:8f:e4:5b:18:ce brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T11:24:23.641 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::4c8f:e4ff:fe5b:18ce/64 scope link 2023-11-28T11:24:23.641 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:24:23.641 INFO:teuthology.orchestra.run.smithi092.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T11:24:23.641 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 02:f9:a8:fe:83:2e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T11:24:23.641 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::f9:a8ff:fefe:832e/64 scope link 2023-11-28T11:24:23.641 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:24:23.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:24:23.642 DEBUG:teuthology.orchestra.run.smithi092:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T11:24:23.744 INFO:teuthology.orchestra.run.smithi092.stdout:1 2023-11-28T11:24:23.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:24:23.747 DEBUG:teuthology.orchestra.run.smithi092:> route 2023-11-28T11:24:23.796 INFO:teuthology.orchestra.run.smithi092.stdout:Kernel IP routing table 2023-11-28T11:24:23.796 INFO:teuthology.orchestra.run.smithi092.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T11:24:23.797 INFO:teuthology.orchestra.run.smithi092.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T11:24:23.797 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T11:24:23.797 INFO:teuthology.orchestra.run.smithi092.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T11:24:23.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:24:23.797 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-11-28T11:24:23.797 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-11-28T11:24:23.797 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T11:24:23.797 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T11:24:23.798 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T11:24:23.798 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-11-28T11:24:23.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:24:23.967 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-11-28T11:24:24.013 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T11:24:24.014 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T11:24:24.014 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:24:24.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:24:24.014 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set brx.0 up 2023-11-28T11:24:24.085 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:24:24.086 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-28T11:24:24.132 INFO:teuthology.orchestra.run.smithi092.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:24:24.132 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:24:24.133 DEBUG:teuthology.orchestra.run.smithi092:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-28T11:24:24.218 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-11-28T11:24:24.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:24:24.219 DEBUG:teuthology.orchestra.run.smithi092:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:24:24.365 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:24:24.365 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T11:24:24.365 INFO:teuthology.orchestra.run.smithi092.stdout:invalid new device string format 2023-11-28T11:24:24.381 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:24:24.381 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T11:24:24.381 INFO:teuthology.orchestra.run.smithi092.stdout:invalid new device string format 2023-11-28T11:24:24.381 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.169:3300,172.21.15.169:3301" 2023-11-28T11:24:24.382 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:24:24.403 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T11:24:24.404 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T11:24:24.404 INFO:teuthology.orchestra.run.smithi092.stdout:invalid new device string format 2023-11-28T11:24:24.404 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.169:3300,172.21.15.169:3301" 2023-11-28T11:24:24.404 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T11:24:24.404 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:6789,172.21.15.169:6789,172.21.15.169:6790" 2023-11-28T11:24:24.404 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.92:6789,172.21.15.169:6789,172.21.15.169:6790:/ 2023-11-28T11:24:24.404 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=68956e0c-c16f-47cd-a220-f98364b0831d" will pass to kernel 2023-11-28T11:24:24.408 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T11:24:24.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:24:24.408 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T11:24:24.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:24:24.479 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T11:24:24.578 INFO:teuthology.orchestra.run.smithi092.stdout:client6115 2023-11-28T11:24:24.578 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-28T11:24:24.578 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:24:24.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:24:24.625 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir_test) 2023-11-28T11:24:24.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:24:24.717 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir_test/file) 2023-11-28T11:24:24.765 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T11:24:24.766 DEBUG:teuthology.orchestra.run.smithi092:> import os 2023-11-28T11:24:24.766 DEBUG:teuthology.orchestra.run.smithi092:> import stat 2023-11-28T11:24:24.766 DEBUG:teuthology.orchestra.run.smithi092:> 2023-11-28T11:24:24.766 DEBUG:teuthology.orchestra.run.smithi092:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir_test/file").st_ino) 2023-11-28T11:24:24.766 DEBUG:teuthology.orchestra.run.smithi092:> ' 2023-11-28T11:24:24.880 INFO:teuthology.orchestra.run.smithi092.stdout:1099511627777 2023-11-28T11:24:25.065 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:24:25.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.304+0000 7ff9845ff700 1 Processor -- start 2023-11-28T11:24:25.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.304+0000 7ff9845ff700 1 -- start start 2023-11-28T11:24:25.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.305+0000 7ff9845ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff97c0b8510 0x7ff97c06b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:25.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.306+0000 7ff9845ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff97c0d0d60 0x7ff97c0d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:25.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.306+0000 7ff9845ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff97c0b88e0 con 0x7ff97c0cd460 2023-11-28T11:24:25.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.306+0000 7ff9845ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff97c0cdbd0 con 0x7ff97c0b8510 2023-11-28T11:24:25.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.306+0000 7ff9845ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff97c0d02d0 con 0x7ff97c0d0d60 2023-11-28T11:24:25.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.306+0000 7ff981b9a700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff97c0cd460 0x7ff97c06beb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35740/0 (socket says 172.21.15.92:35740) 2023-11-28T11:24:25.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.306+0000 7ff98239b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff97c0b8510 0x7ff97c06b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:25.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.306+0000 7ff981b9a700 1 -- 172.21.15.92:0/1269029113 learned_addr learned my addr 172.21.15.92:0/1269029113 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:24:25.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.306+0000 7ff982b9c700 1 --2- 172.21.15.92:0/1269029113 >> v2:172.21.15.169:3300/0 conn(0x7ff97c0d0d60 0x7ff97c0d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:25.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.306+0000 7ff98239b700 1 -- 172.21.15.92:0/1269029113 >> v2:172.21.15.169:3300/0 conn(0x7ff97c0d0d60 msgr2=0x7ff97c0d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:25.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.306+0000 7ff98239b700 1 --2- 172.21.15.92:0/1269029113 >> v2:172.21.15.169:3300/0 conn(0x7ff97c0d0d60 0x7ff97c0d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:25.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.306+0000 7ff98239b700 1 -- 172.21.15.92:0/1269029113 >> v1:172.21.15.92:6789/0 conn(0x7ff97c0cd460 legacy=0x7ff97c06beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:25.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.306+0000 7ff98239b700 1 -- 172.21.15.92:0/1269029113 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff97c0cd8f0 con 0x7ff97c0b8510 2023-11-28T11:24:25.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.306+0000 7ff981399700 1 -- 172.21.15.92:0/1269029113 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3661644491 0 0) 0x7ff97c0b88e0 con 0x7ff97c0cd460 2023-11-28T11:24:25.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.307+0000 7ff98239b700 1 --2- 172.21.15.92:0/1269029113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff97c0b8510 0x7ff97c06b970 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7ff97c0ce8c0 tx=0x7ff978009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96c6050169cf9d22 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:25.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.307+0000 7ff981399700 1 -- 172.21.15.92:0/1269029113 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff978011830 con 0x7ff97c0b8510 2023-11-28T11:24:25.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.307+0000 7ff981399700 1 -- 172.21.15.92:0/1269029113 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff978011990 con 0x7ff97c0b8510 2023-11-28T11:24:25.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.307+0000 7ff981399700 1 -- 172.21.15.92:0/1269029113 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff978011d20 con 0x7ff97c0b8510 2023-11-28T11:24:25.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.307+0000 7ff9845ff700 1 -- 172.21.15.92:0/1269029113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff97c0b8510 msgr2=0x7ff97c06b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:25.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.307+0000 7ff9845ff700 1 --2- 172.21.15.92:0/1269029113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff97c0b8510 0x7ff97c06b970 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7ff97c0ce8c0 tx=0x7ff978009b50 comp rx=0 tx=0).stop 2023-11-28T11:24:25.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.308+0000 7ff9845ff700 1 -- 172.21.15.92:0/1269029113 shutdown_connections 2023-11-28T11:24:25.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.308+0000 7ff9845ff700 1 --2- 172.21.15.92:0/1269029113 >> v2:172.21.15.169:3300/0 conn(0x7ff97c0d0d60 0x7ff97c0d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:25.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.308+0000 7ff9845ff700 1 --2- 172.21.15.92:0/1269029113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff97c0b8510 0x7ff97c06b970 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:25.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.308+0000 7ff9845ff700 1 -- 172.21.15.92:0/1269029113 >> 172.21.15.92:0/1269029113 conn(0x7ff97c0719a0 msgr2=0x7ff97c071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:25.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.308+0000 7ff9845ff700 1 -- 172.21.15.92:0/1269029113 shutdown_connections 2023-11-28T11:24:25.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.308+0000 7ff9845ff700 1 -- 172.21.15.92:0/1269029113 wait complete. 2023-11-28T11:24:25.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.309+0000 7ff9845ff700 1 Processor -- start 2023-11-28T11:24:25.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.309+0000 7ff9845ff700 1 -- start start 2023-11-28T11:24:25.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.310+0000 7ff9845ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff97c0b8510 0x7ff97c156300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:25.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.310+0000 7ff9845ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff97c0cd460 0x7ff97c0ca810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:25.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.310+0000 7ff9845ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff97c0d0d60 0x7ff97c1f1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:25.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.310+0000 7ff9845ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff97c1f2a10 con 0x7ff97c0b8510 2023-11-28T11:24:25.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.310+0000 7ff9845ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff97c1f2450 con 0x7ff97c0cd460 2023-11-28T11:24:25.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.310+0000 7ff9845ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff97c1f2730 con 0x7ff97c0d0d60 2023-11-28T11:24:25.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.311+0000 7ff98239b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff97c0b8510 0x7ff97c156300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:25.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.311+0000 7ff98239b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff97c0b8510 0x7ff97c156300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55312/0 (socket says 172.21.15.92:55312) 2023-11-28T11:24:25.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.311+0000 7ff98239b700 1 -- 172.21.15.92:0/1103817289 learned_addr learned my addr 172.21.15.92:0/1103817289 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:25.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.311+0000 7ff981b9a700 1 --2- 172.21.15.92:0/1103817289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff97c0cd460 0x7ff97c0ca810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:25.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.311+0000 7ff982b9c700 1 --2- 172.21.15.92:0/1103817289 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff97c0d0d60 0x7ff97c1f1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:25.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.311+0000 7ff98239b700 1 -- 172.21.15.92:0/1103817289 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff97c0d0d60 msgr2=0x7ff97c1f1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:25.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.311+0000 7ff98239b700 1 --2- 172.21.15.92:0/1103817289 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff97c0d0d60 0x7ff97c1f1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:25.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.311+0000 7ff98239b700 1 -- 172.21.15.92:0/1103817289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff97c0cd460 msgr2=0x7ff97c0ca810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:25.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.311+0000 7ff98239b700 1 --2- 172.21.15.92:0/1103817289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff97c0cd460 0x7ff97c0ca810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:25.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.311+0000 7ff98239b700 1 -- 172.21.15.92:0/1103817289 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9780094a0 con 0x7ff97c0b8510 2023-11-28T11:24:25.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.311+0000 7ff98239b700 1 --2- 172.21.15.92:0/1103817289 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff97c0b8510 0x7ff97c156300 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7ff9780090b0 tx=0x7ff978004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:25.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.311+0000 7ff972ffd700 1 -- 172.21.15.92:0/1103817289 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97801a730 con 0x7ff97c0b8510 2023-11-28T11:24:25.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.312+0000 7ff972ffd700 1 -- 172.21.15.92:0/1103817289 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff97801a890 con 0x7ff97c0b8510 2023-11-28T11:24:25.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.312+0000 7ff972ffd700 1 -- 172.21.15.92:0/1103817289 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97801ac40 con 0x7ff97c0b8510 2023-11-28T11:24:25.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.312+0000 7ff9845ff700 1 -- 172.21.15.92:0/1103817289 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff97c0cfff0 con 0x7ff97c0b8510 2023-11-28T11:24:25.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.312+0000 7ff9845ff700 1 -- 172.21.15.92:0/1103817289 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff97c0cfd10 con 0x7ff97c0b8510 2023-11-28T11:24:25.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.313+0000 7ff9845ff700 1 -- 172.21.15.92:0/1103817289 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff97c067880 con 0x7ff97c0b8510 2023-11-28T11:24:25.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.314+0000 7ff972ffd700 1 -- 172.21.15.92:0/1103817289 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff97800f050 con 0x7ff97c0b8510 2023-11-28T11:24:25.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.314+0000 7ff972ffd700 1 --2- 172.21.15.92:0/1103817289 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff9600682f0 0x7ff96006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:25.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.315+0000 7ff972ffd700 1 -- 172.21.15.92:0/1103817289 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6382+0+0 (secure 0 0 0) 0x7ff978086c30 con 0x7ff97c0b8510 2023-11-28T11:24:25.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.315+0000 7ff981b9a700 1 --2- 172.21.15.92:0/1103817289 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff9600682f0 0x7ff96006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:25.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.315+0000 7ff981b9a700 1 --2- 172.21.15.92:0/1103817289 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff9600682f0 0x7ff96006a7a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff97c0b4900 tx=0x7ff96c007de0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:25.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.319+0000 7ff972ffd700 1 -- 172.21.15.92:0/1103817289 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff978053260 con 0x7ff97c0b8510 2023-11-28T11:24:25.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.549+0000 7ff9845ff700 1 -- 172.21.15.92:0/1103817289 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff97c0d2940 con 0x7ff97c0b8510 2023-11-28T11:24:25.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.550+0000 7ff972ffd700 1 -- 172.21.15.92:0/1103817289 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4656 (secure 0 0 0) 0x7ff978029090 con 0x7ff97c0b8510 2023-11-28T11:24:25.551 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:24:25.551 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5525,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2067956596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2067956596},{"type":"v1","addr":"172.21.15.169:6835","nonce":2067956596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5971,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2367049160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2367049160},{"type":"v1","addr":"172.21.15.169:6834","nonce":2367049160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5977,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1424858585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":1424858585},{"type":"v1","addr":"172.21.15.92:6837","nonce":1424858585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:24:16.257826+0000","modified":"2023-11-28T11:24:19.539818+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6213},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6213":{"gid":6213,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6836/3634521248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3634521248},{"type":"v1","addr":"172.21.15.92:6836","nonce":3634521248}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T11:24:25.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.552+0000 7ff9845ff700 1 -- 172.21.15.92:0/1103817289 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff9600682f0 msgr2=0x7ff96006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:25.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.552+0000 7ff9845ff700 1 --2- 172.21.15.92:0/1103817289 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff9600682f0 0x7ff96006a7a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff97c0b4900 tx=0x7ff96c007de0 comp rx=0 tx=0).stop 2023-11-28T11:24:25.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.552+0000 7ff9845ff700 1 -- 172.21.15.92:0/1103817289 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff97c0b8510 msgr2=0x7ff97c156300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:25.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.552+0000 7ff9845ff700 1 --2- 172.21.15.92:0/1103817289 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff97c0b8510 0x7ff97c156300 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7ff9780090b0 tx=0x7ff978004750 comp rx=0 tx=0).stop 2023-11-28T11:24:25.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.552+0000 7ff9845ff700 1 -- 172.21.15.92:0/1103817289 shutdown_connections 2023-11-28T11:24:25.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.552+0000 7ff9845ff700 1 --2- 172.21.15.92:0/1103817289 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff97c0d0d60 0x7ff97c1f1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:25.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.552+0000 7ff9845ff700 1 --2- 172.21.15.92:0/1103817289 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff9600682f0 0x7ff96006a7a0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:25.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.552+0000 7ff9845ff700 1 --2- 172.21.15.92:0/1103817289 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff97c0b8510 0x7ff97c156300 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:25.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.552+0000 7ff9845ff700 1 --2- 172.21.15.92:0/1103817289 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff97c0cd460 0x7ff97c0ca810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:25.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.552+0000 7ff9845ff700 1 -- 172.21.15.92:0/1103817289 >> 172.21.15.92:0/1103817289 conn(0x7ff97c0719a0 msgr2=0x7ff97c0b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:25.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.553+0000 7ff9845ff700 1 -- 172.21.15.92:0/1103817289 shutdown_connections 2023-11-28T11:24:25.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.553+0000 7ff9845ff700 1 -- 172.21.15.92:0/1103817289 wait complete. 2023-11-28T11:24:25.554 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 32 2023-11-28T11:24:25.572 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2023-11-28T11:24:25.803 INFO:teuthology.orchestra.run.smithi092.stdout:{"message":"","return_code":0} 2023-11-28T11:24:25.804 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-28T11:24:25.804 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2023-11-28T11:24:25.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.892+0000 7f286b5aa0c0 1 Processor -- start 2023-11-28T11:24:25.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.892+0000 7f286b5aa0c0 1 -- start start 2023-11-28T11:24:25.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.893+0000 7f286b5aa0c0 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f284c002260 0x56476f7e0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:25.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.893+0000 7f286b5aa0c0 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56476f7382c0 0x56476f739350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:25.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.893+0000 7f286b5aa0c0 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56476f739890 0x56476f7dcb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:25.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.893+0000 7f286b5aa0c0 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56476f646fe0 con 0x56476f739890 2023-11-28T11:24:25.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.893+0000 7f286b5aa0c0 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x56476f755b20 con 0x7f284c002260 2023-11-28T11:24:25.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.893+0000 7f286b5aa0c0 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x56476f738ec0 con 0x56476f7382c0 2023-11-28T11:24:25.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.893+0000 7f285d39c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56476f739890 0x56476f7dcb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:25.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.893+0000 7f285d39c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56476f739890 0x56476f7dcb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55336/0 (socket says 172.21.15.92:55336) 2023-11-28T11:24:25.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.893+0000 7f285d39c700 1 -- 172.21.15.92:0/974324703 learned_addr learned my addr 172.21.15.92:0/974324703 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:25.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.893+0000 7f285cb9b700 1 --2- 172.21.15.92:0/974324703 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f284c002260 0x56476f7e0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:25.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.893+0000 7f2857fff700 1 --2- 172.21.15.92:0/974324703 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56476f7382c0 0x56476f739350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:25.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.893+0000 7f285d39c700 1 -- 172.21.15.92:0/974324703 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56476f7382c0 msgr2=0x56476f739350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:25.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.894+0000 7f285d39c700 1 --2- 172.21.15.92:0/974324703 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56476f7382c0 0x56476f739350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:25.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.894+0000 7f285d39c700 1 -- 172.21.15.92:0/974324703 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f284c002260 msgr2=0x56476f7e0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:25.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.894+0000 7f285d39c700 1 --2- 172.21.15.92:0/974324703 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f284c002260 0x56476f7e0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:25.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.894+0000 7f285d39c700 1 -- 172.21.15.92:0/974324703 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56476f652830 con 0x56476f739890 2023-11-28T11:24:25.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.894+0000 7f285cb9b700 1 --2- 172.21.15.92:0/974324703 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f284c002260 0x56476f7e0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:24:25.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.894+0000 7f2857fff700 1 --2- 172.21.15.92:0/974324703 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56476f7382c0 0x56476f739350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:24:25.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.894+0000 7f285d39c700 1 --2- 172.21.15.92:0/974324703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56476f739890 0x56476f7dcb80 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x56476f7120a0 tx=0x7f2858006060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2161017249f1f19 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:25.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.894+0000 7f2856ffd700 1 -- 172.21.15.92:0/974324703 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2858006b90 con 0x56476f739890 2023-11-28T11:24:25.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.894+0000 7f2856ffd700 1 -- 172.21.15.92:0/974324703 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2858006cf0 con 0x56476f739890 2023-11-28T11:24:25.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.894+0000 7f2856ffd700 1 -- 172.21.15.92:0/974324703 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2858003690 con 0x56476f739890 2023-11-28T11:24:25.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.894+0000 7f286b5aa0c0 1 -- 172.21.15.92:0/974324703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56476f739890 msgr2=0x56476f7dcb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:25.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.894+0000 7f286b5aa0c0 1 --2- 172.21.15.92:0/974324703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56476f739890 0x56476f7dcb80 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x56476f7120a0 tx=0x7f2858006060 comp rx=0 tx=0).stop 2023-11-28T11:24:25.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.894+0000 7f286b5aa0c0 1 -- 172.21.15.92:0/974324703 shutdown_connections 2023-11-28T11:24:25.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.894+0000 7f286b5aa0c0 1 --2- 172.21.15.92:0/974324703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56476f739890 0x56476f7dcb80 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:25.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.894+0000 7f286b5aa0c0 1 --2- 172.21.15.92:0/974324703 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f284c002260 0x56476f7e0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:25.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.894+0000 7f286b5aa0c0 1 --2- 172.21.15.92:0/974324703 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56476f7382c0 0x56476f739350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:25.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.894+0000 7f286b5aa0c0 1 -- 172.21.15.92:0/974324703 >> 172.21.15.92:0/974324703 conn(0x56476f70f5b0 msgr2=0x56476f70c850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:25.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.895+0000 7f286b5aa0c0 1 -- 172.21.15.92:0/974324703 shutdown_connections 2023-11-28T11:24:25.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.895+0000 7f286b5aa0c0 1 -- 172.21.15.92:0/974324703 wait complete. 2023-11-28T11:24:25.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.895+0000 7f286b5aa0c0 1 Processor -- start 2023-11-28T11:24:25.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.896+0000 7f286b5aa0c0 1 -- start start 2023-11-28T11:24:25.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.896+0000 7f286b5aa0c0 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56476f7382c0 0x56476f7db840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:25.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.896+0000 7f286b5aa0c0 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56476f739890 0x56476f7e4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:25.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.896+0000 7f286b5aa0c0 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f284c002260 0x56476f7e4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:25.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.896+0000 7f286b5aa0c0 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56476f7d7e40 con 0x7f284c002260 2023-11-28T11:24:25.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.896+0000 7f286b5aa0c0 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x56476f7d8610 con 0x56476f739890 2023-11-28T11:24:25.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.896+0000 7f286b5aa0c0 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x56476f74e0a0 con 0x56476f7382c0 2023-11-28T11:24:25.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.897+0000 7f285d39c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f284c002260 0x56476f7e4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:25.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.897+0000 7f285d39c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f284c002260 0x56476f7e4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55338/0 (socket says 172.21.15.92:55338) 2023-11-28T11:24:25.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.897+0000 7f285d39c700 1 -- 172.21.15.92:0/2779290078 learned_addr learned my addr 172.21.15.92:0/2779290078 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:25.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.897+0000 7f2857fff700 1 --2- 172.21.15.92:0/2779290078 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56476f739890 0x56476f7e4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:25.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.897+0000 7f285cb9b700 1 --2- 172.21.15.92:0/2779290078 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56476f7382c0 0x56476f7db840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:25.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.897+0000 7f285d39c700 1 -- 172.21.15.92:0/2779290078 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56476f7382c0 msgr2=0x56476f7db840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:25.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.897+0000 7f285d39c700 1 --2- 172.21.15.92:0/2779290078 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56476f7382c0 0x56476f7db840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:25.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.897+0000 7f285d39c700 1 -- 172.21.15.92:0/2779290078 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56476f739890 msgr2=0x56476f7e4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:25.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.897+0000 7f285d39c700 1 --2- 172.21.15.92:0/2779290078 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x56476f739890 0x56476f7e4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:25.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.897+0000 7f285d39c700 1 -- 172.21.15.92:0/2779290078 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56476f6794f0 con 0x7f284c002260 2023-11-28T11:24:25.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.897+0000 7f285d39c700 1 --2- 172.21.15.92:0/2779290078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f284c002260 0x56476f7e4bb0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x56476f7381f0 tx=0x7f2858001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:25.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.897+0000 7f2855ffb700 1 -- 172.21.15.92:0/2779290078 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2858023070 con 0x7f284c002260 2023-11-28T11:24:25.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.897+0000 7f2855ffb700 1 -- 172.21.15.92:0/2779290078 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f285801d620 con 0x7f284c002260 2023-11-28T11:24:25.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.897+0000 7f286b5aa0c0 1 -- 172.21.15.92:0/2779290078 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56476f65f360 con 0x7f284c002260 2023-11-28T11:24:25.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.898+0000 7f2855ffb700 1 -- 172.21.15.92:0/2779290078 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f285801d9d0 con 0x7f284c002260 2023-11-28T11:24:25.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.898+0000 7f286b5aa0c0 1 -- 172.21.15.92:0/2779290078 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56476f65f120 con 0x7f284c002260 2023-11-28T11:24:25.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.899+0000 7f2855ffb700 1 -- 172.21.15.92:0/2779290078 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f285801db30 con 0x7f284c002260 2023-11-28T11:24:25.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.900+0000 7f2855ffb700 1 --2- 172.21.15.92:0/2779290078 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f2840066c10 0x7f28400690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:25.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.900+0000 7f2855ffb700 1 -- 172.21.15.92:0/2779290078 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f2858085b40 con 0x7f284c002260 2023-11-28T11:24:25.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.900+0000 7f285cb9b700 1 --2- 172.21.15.92:0/2779290078 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f2840066c10 0x7f28400690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:25.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.901+0000 7f286b5aa0c0 1 --2- 172.21.15.92:0/2779290078 >> [v2:172.21.15.169:6800/116033,v1:172.21.15.169:6801/116033] conn(0x56476f7dbd80 0x56476f754230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:25.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.901+0000 7f286b5aa0c0 1 -- 172.21.15.92:0/2779290078 --> [v2:172.21.15.169:6800/116033,v1:172.21.15.169:6801/116033] -- osd_op(unknown.0.0:1 11.7 11:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68) v8 -- 0x56476f7deea0 con 0x56476f7dbd80 2023-11-28T11:24:25.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.901+0000 7f285cb9b700 1 --2- 172.21.15.92:0/2779290078 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f2840066c10 0x7f28400690c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f2850015040 tx=0x7f2850011000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:25.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.901+0000 7f2857fff700 1 --2- 172.21.15.92:0/2779290078 >> [v2:172.21.15.169:6800/116033,v1:172.21.15.169:6801/116033] conn(0x56476f7dbd80 0x56476f754230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:25.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.902+0000 7f2857fff700 1 --2- 172.21.15.92:0/2779290078 >> [v2:172.21.15.169:6800/116033,v1:172.21.15.169:6801/116033] conn(0x56476f7dbd80 0x56476f754230 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:25.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.902+0000 7f2857fff700 1 -- 172.21.15.92:0/2779290078 <== osd.5 v2:172.21.15.169:6800/116033 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (94) out=94b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+94 (crc 0 0 0) 0x7f28480098e0 con 0x56476f7dbd80 2023-11-28T11:24:25.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.902+0000 7f286b5aa0c0 1 -- 172.21.15.92:0/2779290078 >> [v2:172.21.15.169:6800/116033,v1:172.21.15.169:6801/116033] conn(0x56476f7dbd80 msgr2=0x56476f754230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:25.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.902+0000 7f286b5aa0c0 1 --2- 172.21.15.92:0/2779290078 >> [v2:172.21.15.169:6800/116033,v1:172.21.15.169:6801/116033] conn(0x56476f7dbd80 0x56476f754230 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:25.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.902+0000 7f286b5aa0c0 1 -- 172.21.15.92:0/2779290078 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f2840066c10 msgr2=0x7f28400690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:25.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.902+0000 7f286b5aa0c0 1 --2- 172.21.15.92:0/2779290078 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f2840066c10 0x7f28400690c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f2850015040 tx=0x7f2850011000 comp rx=0 tx=0).stop 2023-11-28T11:24:25.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.902+0000 7f286b5aa0c0 1 -- 172.21.15.92:0/2779290078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f284c002260 msgr2=0x56476f7e4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:25.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.902+0000 7f286b5aa0c0 1 --2- 172.21.15.92:0/2779290078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f284c002260 0x56476f7e4bb0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x56476f7381f0 tx=0x7f2858001d80 comp rx=0 tx=0).stop 2023-11-28T11:24:25.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.902+0000 7f285d39c700 1 -- 172.21.15.92:0/2779290078 reap_dead start 2023-11-28T11:24:25.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.902+0000 7f286b5aa0c0 1 -- 172.21.15.92:0/2779290078 shutdown_connections 2023-11-28T11:24:25.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.902+0000 7f286b5aa0c0 1 -- 172.21.15.92:0/2779290078 >> 172.21.15.92:0/2779290078 conn(0x56476f70f5b0 msgr2=0x56476f7e0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:25.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.903+0000 7f286b5aa0c0 1 -- 172.21.15.92:0/2779290078 shutdown_connections 2023-11-28T11:24:25.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:25.903+0000 7f286b5aa0c0 1 -- 172.21.15.92:0/2779290078 wait complete. 2023-11-28T11:24:25.907 INFO:teuthology.orchestra.run.smithi092.stdout:Xfiledir_test 2023-11-28T11:24:25.907 DEBUG:teuthology.orchestra.run.smithi092:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2023-11-28T11:24:26.006 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-28T11:24:26.006 INFO:teuthology.orchestra.run.smithi092.stdout: "ino": 1099511627777, 2023-11-28T11:24:26.007 INFO:teuthology.orchestra.run.smithi092.stdout: "ancestors": [ 2023-11-28T11:24:26.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:24:26.007 INFO:teuthology.orchestra.run.smithi092.stdout: "dirino": 1099511627776, 2023-11-28T11:24:26.007 INFO:teuthology.orchestra.run.smithi092.stdout: "dname": "file", 2023-11-28T11:24:26.007 INFO:teuthology.orchestra.run.smithi092.stdout: "version": 2 2023-11-28T11:24:26.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-28T11:24:26.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:24:26.008 INFO:teuthology.orchestra.run.smithi092.stdout: "dirino": 1, 2023-11-28T11:24:26.008 INFO:teuthology.orchestra.run.smithi092.stdout: "dname": "dir_test", 2023-11-28T11:24:26.008 INFO:teuthology.orchestra.run.smithi092.stdout: "version": 4 2023-11-28T11:24:26.008 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:24:26.008 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2023-11-28T11:24:26.009 INFO:teuthology.orchestra.run.smithi092.stdout: "pool": 11, 2023-11-28T11:24:26.009 INFO:teuthology.orchestra.run.smithi092.stdout: "old_pools": [] 2023-11-28T11:24:26.009 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-28T11:24:26.009 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:24:26.010 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'The backtrace is corrupted' 2023-11-28T11:24:26.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.098+0000 7f1bd46700c0 1 Processor -- start 2023-11-28T11:24:26.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.098+0000 7f1bd46700c0 1 -- start start 2023-11-28T11:24:26.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.099+0000 7f1bd46700c0 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bb00022b0 0x5651d42ef2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:26.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.099+0000 7f1bd46700c0 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5651d42f44b0 0x5651d42d58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:26.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.099+0000 7f1bd46700c0 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5651d4379dd0 0x5651d437af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:26.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.099+0000 7f1bd46700c0 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651d424f000 con 0x5651d4379dd0 2023-11-28T11:24:26.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.099+0000 7f1bd46700c0 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651d41e1fe0 con 0x7f1bb00022b0 2023-11-28T11:24:26.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.099+0000 7f1bd46700c0 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5651d42a7e70 con 0x5651d42f44b0 2023-11-28T11:24:26.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.099+0000 7f1bc659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5651d4379dd0 0x5651d437af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:26.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.099+0000 7f1bc659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5651d4379dd0 0x5651d437af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55360/0 (socket says 172.21.15.92:55360) 2023-11-28T11:24:26.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.099+0000 7f1bc659c700 1 -- 172.21.15.92:0/1410679516 learned_addr learned my addr 172.21.15.92:0/1410679516 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:26.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.099+0000 7f1bc559a700 1 --2- 172.21.15.92:0/1410679516 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5651d42f44b0 0x5651d42d58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:26.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.099+0000 7f1bc5d9b700 1 --2- 172.21.15.92:0/1410679516 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bb00022b0 0x5651d42ef2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:26.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.099+0000 7f1bc659c700 1 -- 172.21.15.92:0/1410679516 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5651d42f44b0 msgr2=0x5651d42d58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:26.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.099+0000 7f1bc659c700 1 --2- 172.21.15.92:0/1410679516 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5651d42f44b0 0x5651d42d58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.099+0000 7f1bc659c700 1 -- 172.21.15.92:0/1410679516 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bb00022b0 msgr2=0x5651d42ef2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:26.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.099+0000 7f1bc659c700 1 --2- 172.21.15.92:0/1410679516 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bb00022b0 0x5651d42ef2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.099+0000 7f1bc659c700 1 -- 172.21.15.92:0/1410679516 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651d41fa360 con 0x5651d4379dd0 2023-11-28T11:24:26.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.100+0000 7f1bc659c700 1 --2- 172.21.15.92:0/1410679516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5651d4379dd0 0x5651d437af00 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x5651d42aa940 tx=0x7f1bc0006f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ed4e066185f1b79 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:26.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.100+0000 7f1bb7fff700 1 -- 172.21.15.92:0/1410679516 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc0001e70 con 0x5651d4379dd0 2023-11-28T11:24:26.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.100+0000 7f1bb7fff700 1 -- 172.21.15.92:0/1410679516 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bc0001ff0 con 0x5651d4379dd0 2023-11-28T11:24:26.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.100+0000 7f1bb7fff700 1 -- 172.21.15.92:0/1410679516 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc0004670 con 0x5651d4379dd0 2023-11-28T11:24:26.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.100+0000 7f1bd46700c0 1 -- 172.21.15.92:0/1410679516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5651d4379dd0 msgr2=0x5651d437af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:26.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.100+0000 7f1bd46700c0 1 --2- 172.21.15.92:0/1410679516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5651d4379dd0 0x5651d437af00 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x5651d42aa940 tx=0x7f1bc0006f90 comp rx=0 tx=0).stop 2023-11-28T11:24:26.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.100+0000 7f1bd46700c0 1 -- 172.21.15.92:0/1410679516 shutdown_connections 2023-11-28T11:24:26.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.100+0000 7f1bd46700c0 1 --2- 172.21.15.92:0/1410679516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5651d4379dd0 0x5651d437af00 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.100+0000 7f1bd46700c0 1 --2- 172.21.15.92:0/1410679516 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f1bb00022b0 0x5651d42ef2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.100+0000 7f1bd46700c0 1 --2- 172.21.15.92:0/1410679516 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5651d42f44b0 0x5651d42d58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.100+0000 7f1bd46700c0 1 -- 172.21.15.92:0/1410679516 >> 172.21.15.92:0/1410679516 conn(0x5651d42f0f30 msgr2=0x5651d42d83a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:26.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.100+0000 7f1bd46700c0 1 -- 172.21.15.92:0/1410679516 shutdown_connections 2023-11-28T11:24:26.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.101+0000 7f1bd46700c0 1 -- 172.21.15.92:0/1410679516 wait complete. 2023-11-28T11:24:26.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.101+0000 7f1bd46700c0 1 Processor -- start 2023-11-28T11:24:26.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.101+0000 7f1bd46700c0 1 -- start start 2023-11-28T11:24:26.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.102+0000 7f1bd46700c0 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5651d42f44b0 0x5651d4380170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:26.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.102+0000 7f1bd46700c0 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5651d4379dd0 0x5651d4371940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:26.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.102+0000 7f1bd46700c0 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1bb00022b0 0x5651d436fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:26.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.102+0000 7f1bd46700c0 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651d43714b0 con 0x7f1bb00022b0 2023-11-28T11:24:26.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.102+0000 7f1bd46700c0 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651d42ed1e0 con 0x5651d42f44b0 2023-11-28T11:24:26.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.102+0000 7f1bd46700c0 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5651d4370e60 con 0x5651d4379dd0 2023-11-28T11:24:26.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.103+0000 7f1bc659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1bb00022b0 0x5651d436fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:26.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.103+0000 7f1bc659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1bb00022b0 0x5651d436fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55362/0 (socket says 172.21.15.92:55362) 2023-11-28T11:24:26.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.103+0000 7f1bc659c700 1 -- 172.21.15.92:0/3623410261 learned_addr learned my addr 172.21.15.92:0/3623410261 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:26.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.103+0000 7f1bc559a700 1 --2- 172.21.15.92:0/3623410261 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5651d4379dd0 0x5651d4371940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:26.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.103+0000 7f1bc5d9b700 1 --2- 172.21.15.92:0/3623410261 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5651d42f44b0 0x5651d4380170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:26.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.103+0000 7f1bc659c700 1 -- 172.21.15.92:0/3623410261 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5651d4379dd0 msgr2=0x5651d4371940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:26.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.103+0000 7f1bc659c700 1 --2- 172.21.15.92:0/3623410261 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5651d4379dd0 0x5651d4371940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.103+0000 7f1bc659c700 1 -- 172.21.15.92:0/3623410261 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5651d42f44b0 msgr2=0x5651d4380170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:26.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.103+0000 7f1bc659c700 1 --2- 172.21.15.92:0/3623410261 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5651d42f44b0 0x5651d4380170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.103+0000 7f1bc659c700 1 -- 172.21.15.92:0/3623410261 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651d42144f0 con 0x7f1bb00022b0 2023-11-28T11:24:26.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.103+0000 7f1bc659c700 1 --2- 172.21.15.92:0/3623410261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1bb00022b0 0x5651d436fdb0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f1bc000f000 tx=0x7f1bc00014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:26.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.103+0000 7f1bb6ffd700 1 -- 172.21.15.92:0/3623410261 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc0013620 con 0x7f1bb00022b0 2023-11-28T11:24:26.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.103+0000 7f1bb6ffd700 1 -- 172.21.15.92:0/3623410261 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bc0001e80 con 0x7f1bb00022b0 2023-11-28T11:24:26.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.103+0000 7f1bb6ffd700 1 -- 172.21.15.92:0/3623410261 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc0013980 con 0x7f1bb00022b0 2023-11-28T11:24:26.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.103+0000 7f1bd46700c0 1 -- 172.21.15.92:0/3623410261 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5651d42d6eb0 con 0x7f1bb00022b0 2023-11-28T11:24:26.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.104+0000 7f1bd46700c0 1 -- 172.21.15.92:0/3623410261 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5651d41ed830 con 0x7f1bb00022b0 2023-11-28T11:24:26.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.105+0000 7f1bb6ffd700 1 -- 172.21.15.92:0/3623410261 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1bc0025030 con 0x7f1bb00022b0 2023-11-28T11:24:26.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.106+0000 7f1bb6ffd700 1 --2- 172.21.15.92:0/3623410261 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f1ba8066c10 0x7f1ba80690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:26.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.106+0000 7f1bb6ffd700 1 -- 172.21.15.92:0/3623410261 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f1bc001b070 con 0x7f1bb00022b0 2023-11-28T11:24:26.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.106+0000 7f1bc5d9b700 1 --2- 172.21.15.92:0/3623410261 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f1ba8066c10 0x7f1ba80690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:26.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.106+0000 7f1bd46700c0 1 --2- 172.21.15.92:0/3623410261 >> [v2:172.21.15.169:6800/116033,v1:172.21.15.169:6801/116033] conn(0x5651d42ecc90 0x5651d43745d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:26.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.106+0000 7f1bd46700c0 1 -- 172.21.15.92:0/3623410261 --> [v2:172.21.15.169:6800/116033,v1:172.21.15.169:6801/116033] -- osd_op(unknown.0.0:1 11.7 11:e2e07930:::10000000001.00000000:head [setxattr parent (26) in=32b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68) v8 -- 0x5651d43702f0 con 0x5651d42ecc90 2023-11-28T11:24:26.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.107+0000 7f1bc559a700 1 --2- 172.21.15.92:0/3623410261 >> [v2:172.21.15.169:6800/116033,v1:172.21.15.169:6801/116033] conn(0x5651d42ecc90 0x5651d43745d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:26.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.107+0000 7f1bc5d9b700 1 --2- 172.21.15.92:0/3623410261 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f1ba8066c10 0x7f1ba80690c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f1bb8000c00 tx=0x7f1bb8010000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:26.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.107+0000 7f1bc559a700 1 --2- 172.21.15.92:0/3623410261 >> [v2:172.21.15.169:6800/116033,v1:172.21.15.169:6801/116033] conn(0x5651d42ecc90 0x5651d43745d0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:26.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.109+0000 7f1bc559a700 1 -- 172.21.15.92:0/3623410261 <== osd.5 v2:172.21.15.169:6800/116033 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (26)] v68'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f1bbc0097c0 con 0x5651d42ecc90 2023-11-28T11:24:26.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.109+0000 7f1bd46700c0 1 -- 172.21.15.92:0/3623410261 >> [v2:172.21.15.169:6800/116033,v1:172.21.15.169:6801/116033] conn(0x5651d42ecc90 msgr2=0x5651d43745d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:26.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.109+0000 7f1bd46700c0 1 --2- 172.21.15.92:0/3623410261 >> [v2:172.21.15.169:6800/116033,v1:172.21.15.169:6801/116033] conn(0x5651d42ecc90 0x5651d43745d0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.109+0000 7f1bd46700c0 1 -- 172.21.15.92:0/3623410261 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f1ba8066c10 msgr2=0x7f1ba80690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:26.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.109+0000 7f1bd46700c0 1 --2- 172.21.15.92:0/3623410261 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f1ba8066c10 0x7f1ba80690c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f1bb8000c00 tx=0x7f1bb8010000 comp rx=0 tx=0).stop 2023-11-28T11:24:26.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.109+0000 7f1bd46700c0 1 -- 172.21.15.92:0/3623410261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1bb00022b0 msgr2=0x5651d436fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:26.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.109+0000 7f1bd46700c0 1 --2- 172.21.15.92:0/3623410261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1bb00022b0 0x5651d436fdb0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f1bc000f000 tx=0x7f1bc00014a0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.109+0000 7f1bc659c700 1 -- 172.21.15.92:0/3623410261 reap_dead start 2023-11-28T11:24:26.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.109+0000 7f1bd46700c0 1 -- 172.21.15.92:0/3623410261 shutdown_connections 2023-11-28T11:24:26.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.109+0000 7f1bd46700c0 1 -- 172.21.15.92:0/3623410261 >> 172.21.15.92:0/3623410261 conn(0x5651d42f0f30 msgr2=0x5651d42f2630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:26.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.110+0000 7f1bd46700c0 1 -- 172.21.15.92:0/3623410261 shutdown_connections 2023-11-28T11:24:26.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.110+0000 7f1bd46700c0 1 -- 172.21.15.92:0/3623410261 wait complete. 2023-11-28T11:24:26.114 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub start / recursive 2023-11-28T11:24:26.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.314+0000 7ff6985ff700 1 Processor -- start 2023-11-28T11:24:26.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.314+0000 7ff6985ff700 1 -- start start 2023-11-28T11:24:26.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.315+0000 7ff6985ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff6900b8790 0x7ff6900cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:26.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.315+0000 7ff6985ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff6900cd460 0x7ff69006b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:26.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.316+0000 7ff6985ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff69006c2e0 con 0x7ff69006be80 2023-11-28T11:24:26.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.316+0000 7ff6985ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6900d00b0 con 0x7ff6900cd460 2023-11-28T11:24:26.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.316+0000 7ff6985ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6900b5240 con 0x7ff6900b8790 2023-11-28T11:24:26.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.316+0000 7ff696b9c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff69006be80 0x7ff6901602b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35766/0 (socket says 172.21.15.92:35766) 2023-11-28T11:24:26.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.316+0000 7ff696b9c700 1 -- 172.21.15.92:0/4062550111 learned_addr learned my addr 172.21.15.92:0/4062550111 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:24:26.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.316+0000 7ff695b9a700 1 --2- 172.21.15.92:0/4062550111 >> v2:172.21.15.92:3300/0 conn(0x7ff6900cd460 0x7ff69006b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:26.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.316+0000 7ff69639b700 1 --2- 172.21.15.92:0/4062550111 >> v2:172.21.15.169:3300/0 conn(0x7ff6900b8790 0x7ff6900cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:26.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.316+0000 7ff695399700 1 -- 172.21.15.92:0/4062550111 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1389094138 0 0) 0x7ff69006c2e0 con 0x7ff69006be80 2023-11-28T11:24:26.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.316+0000 7ff695b9a700 1 -- 172.21.15.92:0/4062550111 >> v2:172.21.15.169:3300/0 conn(0x7ff6900b8790 msgr2=0x7ff6900cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:26.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.316+0000 7ff695b9a700 1 --2- 172.21.15.92:0/4062550111 >> v2:172.21.15.169:3300/0 conn(0x7ff6900b8790 0x7ff6900cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.316+0000 7ff695b9a700 1 -- 172.21.15.92:0/4062550111 >> v1:172.21.15.92:6789/0 conn(0x7ff69006be80 legacy=0x7ff6901602b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:26.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.316+0000 7ff695b9a700 1 -- 172.21.15.92:0/4062550111 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6900b5520 con 0x7ff6900cd460 2023-11-28T11:24:26.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.317+0000 7ff695b9a700 1 --2- 172.21.15.92:0/4062550111 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6900cd460 0x7ff69006b940 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7ff680009840 tx=0x7ff680009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c374d175470a412 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:26.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.317+0000 7ff695399700 1 -- 172.21.15.92:0/4062550111 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff680011830 con 0x7ff6900cd460 2023-11-28T11:24:26.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.317+0000 7ff695399700 1 -- 172.21.15.92:0/4062550111 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff680011990 con 0x7ff6900cd460 2023-11-28T11:24:26.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.317+0000 7ff695399700 1 -- 172.21.15.92:0/4062550111 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff680011d20 con 0x7ff6900cd460 2023-11-28T11:24:26.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.317+0000 7ff6985ff700 1 -- 172.21.15.92:0/4062550111 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6900cd460 msgr2=0x7ff69006b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:26.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.317+0000 7ff6985ff700 1 --2- 172.21.15.92:0/4062550111 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6900cd460 0x7ff69006b940 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7ff680009840 tx=0x7ff680009b50 comp rx=0 tx=0).stop 2023-11-28T11:24:26.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.317+0000 7ff6985ff700 1 -- 172.21.15.92:0/4062550111 shutdown_connections 2023-11-28T11:24:26.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.317+0000 7ff6985ff700 1 --2- 172.21.15.92:0/4062550111 >> v2:172.21.15.169:3300/0 conn(0x7ff6900b8790 0x7ff6900cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.317+0000 7ff6985ff700 1 --2- 172.21.15.92:0/4062550111 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6900cd460 0x7ff69006b940 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.317+0000 7ff6985ff700 1 -- 172.21.15.92:0/4062550111 >> 172.21.15.92:0/4062550111 conn(0x7ff6900719a0 msgr2=0x7ff690071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:26.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.317+0000 7ff6985ff700 1 -- 172.21.15.92:0/4062550111 shutdown_connections 2023-11-28T11:24:26.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.317+0000 7ff6985ff700 1 -- 172.21.15.92:0/4062550111 wait complete. 2023-11-28T11:24:26.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.319+0000 7ff6985ff700 1 Processor -- start 2023-11-28T11:24:26.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.319+0000 7ff6985ff700 1 -- start start 2023-11-28T11:24:26.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.320+0000 7ff6985ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff69006be80 0x7ff690155a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:26.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.320+0000 7ff6985ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6900b8790 0x7ff6900b5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:26.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.320+0000 7ff6985ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6900cd460 0x7ff6900b6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:26.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.320+0000 7ff6985ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6900c6540 con 0x7ff6900b8790 2023-11-28T11:24:26.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.320+0000 7ff6985ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff690155f60 con 0x7ff6900cd460 2023-11-28T11:24:26.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.320+0000 7ff6985ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff690160d80 con 0x7ff69006be80 2023-11-28T11:24:26.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.321+0000 7ff695b9a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6900b8790 0x7ff6900b5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:26.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.321+0000 7ff695b9a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6900b8790 0x7ff6900b5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55390/0 (socket says 172.21.15.92:55390) 2023-11-28T11:24:26.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.321+0000 7ff695b9a700 1 -- 172.21.15.92:0/2487428271 learned_addr learned my addr 172.21.15.92:0/2487428271 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:26.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.321+0000 7ff69639b700 1 --2- 172.21.15.92:0/2487428271 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff69006be80 0x7ff690155a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:26.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.321+0000 7ff696b9c700 1 --2- 172.21.15.92:0/2487428271 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6900cd460 0x7ff6900b6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:26.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.321+0000 7ff695b9a700 1 -- 172.21.15.92:0/2487428271 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff69006be80 msgr2=0x7ff690155a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:26.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.321+0000 7ff695b9a700 1 --2- 172.21.15.92:0/2487428271 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff69006be80 0x7ff690155a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.321+0000 7ff695b9a700 1 -- 172.21.15.92:0/2487428271 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6900cd460 msgr2=0x7ff6900b6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:26.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.321+0000 7ff695b9a700 1 --2- 172.21.15.92:0/2487428271 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6900cd460 0x7ff6900b6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.321+0000 7ff695b9a700 1 -- 172.21.15.92:0/2487428271 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6800094a0 con 0x7ff6900b8790 2023-11-28T11:24:26.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.321+0000 7ff695b9a700 1 --2- 172.21.15.92:0/2487428271 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6900b8790 0x7ff6900b5cd0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7ff680009b90 tx=0x7ff680004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:26.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.321+0000 7ff686ffd700 1 -- 172.21.15.92:0/2487428271 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68001a8e0 con 0x7ff6900b8790 2023-11-28T11:24:26.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.321+0000 7ff686ffd700 1 -- 172.21.15.92:0/2487428271 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff68001aa40 con 0x7ff6900b8790 2023-11-28T11:24:26.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.321+0000 7ff686ffd700 1 -- 172.21.15.92:0/2487428271 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68001add0 con 0x7ff6900b8790 2023-11-28T11:24:26.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.322+0000 7ff6985ff700 1 -- 172.21.15.92:0/2487428271 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6900b5800 con 0x7ff6900b8790 2023-11-28T11:24:26.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.322+0000 7ff6985ff700 1 -- 172.21.15.92:0/2487428271 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6900d0a80 con 0x7ff6900b8790 2023-11-28T11:24:26.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.324+0000 7ff686ffd700 1 -- 172.21.15.92:0/2487428271 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff68000f050 con 0x7ff6900b8790 2023-11-28T11:24:26.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.324+0000 7ff686ffd700 1 --2- 172.21.15.92:0/2487428271 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff6740682f0 0x7ff67406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:26.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.325+0000 7ff686ffd700 1 -- 172.21.15.92:0/2487428271 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6382+0+0 (secure 0 0 0) 0x7ff680086d70 con 0x7ff6900b8790 2023-11-28T11:24:26.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.325+0000 7ff69639b700 1 --2- 172.21.15.92:0/2487428271 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff6740682f0 0x7ff67406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:26.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.325+0000 7ff69639b700 1 --2- 172.21.15.92:0/2487428271 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff6740682f0 0x7ff67406a7a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff68c009b70 tx=0x7ff68c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:26.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.326+0000 7ff6ac00ab80 1 Processor -- start 2023-11-28T11:24:26.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.326+0000 7ff6ac00ab80 1 -- start start 2023-11-28T11:24:26.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.327+0000 7ff6ac00ab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56123870a000 0x5612386c3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:26.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.327+0000 7ff6ac00ab80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561238700b70 0x56123870a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:26.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.327+0000 7ff6ac00ab80 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5612386ec710 0x5612386c4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:26.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.327+0000 7ff6ac00ab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x561238641e90 con 0x56123870a000 2023-11-28T11:24:26.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.327+0000 7ff6ac00ab80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612385ae1f0 con 0x561238700b70 2023-11-28T11:24:26.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.327+0000 7ff6ac00ab80 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x56123861ce60 con 0x5612386ec710 2023-11-28T11:24:26.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.327+0000 7ff69639b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56123870a000 0x5612386c3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:26.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.327+0000 7ff69639b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56123870a000 0x5612386c3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55394/0 (socket says 172.21.15.92:55394) 2023-11-28T11:24:26.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.328+0000 7ff69639b700 1 -- 172.21.15.92:0/1772017128 learned_addr learned my addr 172.21.15.92:0/1772017128 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:26.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.328+0000 7ff695b9a700 1 --2- 172.21.15.92:0/1772017128 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561238700b70 0x56123870a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:26.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.328+0000 7ff696b9c700 1 --2- 172.21.15.92:0/1772017128 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5612386ec710 0x5612386c4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:26.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.328+0000 7ff69639b700 1 -- 172.21.15.92:0/1772017128 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5612386ec710 msgr2=0x5612386c4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:26.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.328+0000 7ff69639b700 1 --2- 172.21.15.92:0/1772017128 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5612386ec710 0x5612386c4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.328+0000 7ff69639b700 1 -- 172.21.15.92:0/1772017128 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561238700b70 msgr2=0x56123870a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:26.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.328+0000 7ff69639b700 1 --2- 172.21.15.92:0/1772017128 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561238700b70 0x56123870a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.328+0000 7ff69639b700 1 -- 172.21.15.92:0/1772017128 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56123840a840 con 0x56123870a000 2023-11-28T11:24:26.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.328+0000 7ff69639b700 1 --2- 172.21.15.92:0/1772017128 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56123870a000 0x5612386c3aa0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7ff68c011ca0 tx=0x7ff68c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=795f5396dd5988e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:26.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.328+0000 7ff673fff700 1 -- 172.21.15.92:0/1772017128 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68c0158d0 con 0x56123870a000 2023-11-28T11:24:26.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.328+0000 7ff673fff700 1 -- 172.21.15.92:0/1772017128 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff68c015a30 con 0x56123870a000 2023-11-28T11:24:26.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.328+0000 7ff673fff700 1 -- 172.21.15.92:0/1772017128 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68c015da0 con 0x56123870a000 2023-11-28T11:24:26.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.329+0000 7ff6ac00ab80 1 -- 172.21.15.92:0/1772017128 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56123870a000 msgr2=0x5612386c3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:26.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.329+0000 7ff6ac00ab80 1 --2- 172.21.15.92:0/1772017128 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56123870a000 0x5612386c3aa0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7ff68c011ca0 tx=0x7ff68c014000 comp rx=0 tx=0).stop 2023-11-28T11:24:26.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.329+0000 7ff6ac00ab80 1 -- 172.21.15.92:0/1772017128 shutdown_connections 2023-11-28T11:24:26.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.329+0000 7ff6ac00ab80 1 --2- 172.21.15.92:0/1772017128 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5612386ec710 0x5612386c4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.329+0000 7ff6ac00ab80 1 --2- 172.21.15.92:0/1772017128 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56123870a000 0x5612386c3aa0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.329+0000 7ff6ac00ab80 1 --2- 172.21.15.92:0/1772017128 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561238700b70 0x56123870a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.329+0000 7ff6ac00ab80 1 -- 172.21.15.92:0/1772017128 >> 172.21.15.92:0/1772017128 conn(0x5612386b9dd0 msgr2=0x5612386c2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:26.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.329+0000 7ff6ac00ab80 1 -- 172.21.15.92:0/1772017128 shutdown_connections 2023-11-28T11:24:26.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.329+0000 7ff6ac00ab80 1 -- 172.21.15.92:0/1772017128 wait complete. 2023-11-28T11:24:26.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.330+0000 7ff6ac00ab80 1 -- start start 2023-11-28T11:24:26.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.330+0000 7ff6ac00ab80 1 Processor -- start 2023-11-28T11:24:26.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.331+0000 7ff6ac00ab80 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5612386ec710 0x5612386eb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:26.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.331+0000 7ff6ac00ab80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561238700b70 0x5612386ee750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:26.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.331+0000 7ff6ac00ab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56123870a000 0x5612387190f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:26.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.331+0000 7ff6ac00ab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612386161d0 con 0x56123870a000 2023-11-28T11:24:26.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.331+0000 7ff6ac00ab80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x56123840b440 con 0x561238700b70 2023-11-28T11:24:26.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.331+0000 7ff6ac00ab80 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5612385ae3f0 con 0x5612386ec710 2023-11-28T11:24:26.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.331+0000 7ff6ac00ab80 10 client.0 _ll_register_callbacks cb 0x5612385b44b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:24:26.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.331+0000 7ff696b9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56123870a000 0x5612387190f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:26.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.332+0000 7ff696b9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56123870a000 0x5612387190f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55396/0 (socket says 172.21.15.92:55396) 2023-11-28T11:24:26.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.332+0000 7ff696b9c700 1 -- 172.21.15.92:0/2918608358 learned_addr learned my addr 172.21.15.92:0/2918608358 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:26.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.332+0000 7ff69639b700 1 --2- 172.21.15.92:0/2918608358 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5612386ec710 0x5612386eb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:26.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.332+0000 7ff695b9a700 1 --2- 172.21.15.92:0/2918608358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561238700b70 0x5612386ee750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:26.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.332+0000 7ff696b9c700 1 -- 172.21.15.92:0/2918608358 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5612386ec710 msgr2=0x5612386eb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:26.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.332+0000 7ff696b9c700 1 --2- 172.21.15.92:0/2918608358 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5612386ec710 0x5612386eb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.332+0000 7ff696b9c700 1 -- 172.21.15.92:0/2918608358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561238700b70 msgr2=0x5612386ee750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:26.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.332+0000 7ff696b9c700 1 --2- 172.21.15.92:0/2918608358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561238700b70 0x5612386ee750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.332+0000 7ff696b9c700 1 -- 172.21.15.92:0/2918608358 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff68c0118f0 con 0x56123870a000 2023-11-28T11:24:26.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.332+0000 7ff695b9a700 1 --2- 172.21.15.92:0/2918608358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561238700b70 0x5612386ee750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:24:26.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.332+0000 7ff696b9c700 1 --2- 172.21.15.92:0/2918608358 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56123870a000 0x5612387190f0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7ff688007680 tx=0x7ff688007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=235fb7b7b4aea920 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:26.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.332+0000 7ff6717fa700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-11-28T11:24:26.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.332+0000 7ff6717fa700 1 -- 172.21.15.92:0/2918608358 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff688010ba0 con 0x56123870a000 2023-11-28T11:24:26.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.332+0000 7ff6717fa700 1 -- 172.21.15.92:0/2918608358 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(68..68 src has 1..68) v4 ==== 6382+0+0 (secure 0 0 0) 0x7ff688010d00 con 0x56123870a000 2023-11-28T11:24:26.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.332+0000 7ff6ac00ab80 1 -- 172.21.15.92:0/2918608358 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56123861a0e0 con 0x56123870a000 2023-11-28T11:24:26.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.333+0000 7ff6717fa700 1 -- 172.21.15.92:0/2918608358 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff688010ec0 con 0x56123870a000 2023-11-28T11:24:26.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.333+0000 7ff6ac00ab80 10 client.6408 fetch_fsmap learned FSMap version 32 2023-11-28T11:24:26.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.333+0000 7ff6ac00ab80 1 -- 172.21.15.92:0/2918608358 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x5612384f7150 con 0x56123870a000 2023-11-28T11:24:26.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.333+0000 7ff6717fa700 1 -- 172.21.15.92:0/2918608358 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56123861a0e0 con 0x56123870a000 2023-11-28T11:24:26.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.333+0000 7ff6717fa700 1 -- 172.21.15.92:0/2918608358 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 32) v1 ==== 3106+0+0 (secure 0 0 0) 0x7ff68801d3f0 con 0x56123870a000 2023-11-28T11:24:26.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.333+0000 7ff6ac00ab80 10 client.6408 fetch_fsmap finished waiting for FSMap version 32 2023-11-28T11:24:26.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.333+0000 7ff6ac00ab80 10 client.6408 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2023-11-28T11:24:26.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.333+0000 7ff6ac00ab80 20 client.6408 populate_metadata read hostname 'smithi092' 2023-11-28T11:24:26.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.333+0000 7ff6ac00ab80 1 --2- 172.21.15.92:0/2918608358 >> [v2:172.21.15.92:6834/3634521248,v1:172.21.15.92:6836/3634521248] conn(0x5612387197a0 0x56123871c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:26.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.333+0000 7ff6ac00ab80 4 client.6408 mds_command: new command op to 6213 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:24:26.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.333+0000 7ff6ac00ab80 1 -- 172.21.15.92:0/2918608358 --> [v2:172.21.15.92:6834/3634521248,v1:172.21.15.92:6836/3634521248] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561238465fa0 con 0x5612387197a0 2023-11-28T11:24:26.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.333+0000 7ff69639b700 1 --2- 172.21.15.92:0/2918608358 >> [v2:172.21.15.92:6834/3634521248,v1:172.21.15.92:6836/3634521248] conn(0x5612387197a0 0x56123871c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:26.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.334+0000 7ff69639b700 1 --2- 172.21.15.92:0/2918608358 >> [v2:172.21.15.92:6834/3634521248,v1:172.21.15.92:6836/3634521248] conn(0x5612387197a0 0x56123871c740 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=689799ffcfa1f5e2 server_cookie=b192be8a6beeb6df in_seq=0 out_seq=0 2023-11-28T11:24:26.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.334+0000 7ff6717fa700 10 client.6408 ms_handle_connect on v2:172.21.15.92:6834/3634521248 2023-11-28T11:24:26.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.334+0000 7ff6717fa700 1 -- 172.21.15.92:0/2918608358 <== mds.0 v2:172.21.15.92:6834/3634521248 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x561238465fa0 con 0x5612387197a0 2023-11-28T11:24:26.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.334+0000 7ff6717fa700 10 client.6408 handle_command_reply: tid=0 2023-11-28T11:24:26.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.334+0000 7ff6ac00ab80 1 client.6408 shutdown 2023-11-28T11:24:26.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.335+0000 7ff6ac00ab80 1 -- 172.21.15.92:0/2918608358 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56123870a000 msgr2=0x5612387190f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:26.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.335+0000 7ff6ac00ab80 1 --2- 172.21.15.92:0/2918608358 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56123870a000 0x5612387190f0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7ff688007680 tx=0x7ff688007a40 comp rx=0 tx=0).stop 2023-11-28T11:24:26.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.335+0000 7ff6ac00ab80 1 -- 172.21.15.92:0/2918608358 shutdown_connections 2023-11-28T11:24:26.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.335+0000 7ff6ac00ab80 1 --2- 172.21.15.92:0/2918608358 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56123870a000 0x5612387190f0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.335+0000 7ff6ac00ab80 1 --2- 172.21.15.92:0/2918608358 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5612386ec710 0x5612386eb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.335+0000 7ff6ac00ab80 1 --2- 172.21.15.92:0/2918608358 >> [v2:172.21.15.92:6834/3634521248,v1:172.21.15.92:6836/3634521248] conn(0x5612387197a0 0x56123871c740 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.335+0000 7ff6ac00ab80 1 --2- 172.21.15.92:0/2918608358 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561238700b70 0x5612386ee750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.335+0000 7ff6717fa700 0 client.6408 ms_handle_reset on v2:172.21.15.92:6834/3634521248 2023-11-28T11:24:26.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.335+0000 7ff6ac00ab80 1 -- 172.21.15.92:0/2918608358 >> 172.21.15.92:0/2918608358 conn(0x5612386b9dd0 msgr2=0x5612386ebde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:26.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.335+0000 7ff6ac00ab80 1 -- 172.21.15.92:0/2918608358 shutdown_connections 2023-11-28T11:24:26.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.335+0000 7ff6ac00ab80 1 -- 172.21.15.92:0/2918608358 wait complete. 2023-11-28T11:24:26.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.335+0000 7ff6ac00ab80 20 client.6408 trim_cache size 0 max 16384 2023-11-28T11:24:26.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.352+0000 7ff6ac00ab80 1 Processor -- start 2023-11-28T11:24:26.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.352+0000 7ff6ac00ab80 1 -- start start 2023-11-28T11:24:26.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.353+0000 7ff6ac00ab80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5612386ec710 0x5612386f27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:26.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.353+0000 7ff6ac00ab80 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561238700b70 0x5612386eb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:26.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.353+0000 7ff6ac00ab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56123870a000 0x5612387697c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:26.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.353+0000 7ff6ac00ab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612386bfc60 con 0x56123870a000 2023-11-28T11:24:26.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.353+0000 7ff6ac00ab80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612386f2f00 con 0x5612386ec710 2023-11-28T11:24:26.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.353+0000 7ff6ac00ab80 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x561238706710 con 0x561238700b70 2023-11-28T11:24:26.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.353+0000 7ff696b9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56123870a000 0x5612387697c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:26.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.353+0000 7ff696b9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56123870a000 0x5612387697c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55404/0 (socket says 172.21.15.92:55404) 2023-11-28T11:24:26.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.353+0000 7ff696b9c700 1 -- 172.21.15.92:0/2966589030 learned_addr learned my addr 172.21.15.92:0/2966589030 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:26.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.354+0000 7ff696b9c700 1 -- 172.21.15.92:0/2966589030 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561238700b70 msgr2=0x5612386eb330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:26.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.354+0000 7ff696b9c700 1 --2- 172.21.15.92:0/2966589030 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561238700b70 0x5612386eb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.354+0000 7ff696b9c700 1 -- 172.21.15.92:0/2966589030 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5612386ec710 msgr2=0x5612386f27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:26.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.354+0000 7ff696b9c700 1 --2- 172.21.15.92:0/2966589030 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5612386ec710 0x5612386f27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.354+0000 7ff696b9c700 1 -- 172.21.15.92:0/2966589030 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56123870a6a0 con 0x56123870a000 2023-11-28T11:24:26.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.354+0000 7ff696b9c700 1 --2- 172.21.15.92:0/2966589030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56123870a000 0x5612387697c0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7ff688001d50 tx=0x7ff68800c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a94c7a4212add7ed server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:26.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.354+0000 7ff684bff700 1 -- 172.21.15.92:0/2966589030 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68801d950 con 0x56123870a000 2023-11-28T11:24:26.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.354+0000 7ff684bff700 1 -- 172.21.15.92:0/2966589030 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff68801d3f0 con 0x56123870a000 2023-11-28T11:24:26.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.354+0000 7ff684bff700 1 -- 172.21.15.92:0/2966589030 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff688010ba0 con 0x56123870a000 2023-11-28T11:24:26.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.355+0000 7ff6ac00ab80 1 -- 172.21.15.92:0/2966589030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56123870a000 msgr2=0x5612387697c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:26.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.355+0000 7ff6ac00ab80 1 --2- 172.21.15.92:0/2966589030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56123870a000 0x5612387697c0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7ff688001d50 tx=0x7ff68800c000 comp rx=0 tx=0).stop 2023-11-28T11:24:26.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.355+0000 7ff6ac00ab80 1 -- 172.21.15.92:0/2966589030 shutdown_connections 2023-11-28T11:24:26.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.355+0000 7ff6ac00ab80 1 --2- 172.21.15.92:0/2966589030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56123870a000 0x5612387697c0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.355+0000 7ff6ac00ab80 1 --2- 172.21.15.92:0/2966589030 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5612386ec710 0x5612386f27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.355+0000 7ff6ac00ab80 1 --2- 172.21.15.92:0/2966589030 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x561238700b70 0x5612386eb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.355+0000 7ff6ac00ab80 1 -- 172.21.15.92:0/2966589030 >> 172.21.15.92:0/2966589030 conn(0x5612386b9dd0 msgr2=0x561238702d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:26.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.355+0000 7ff6ac00ab80 1 -- 172.21.15.92:0/2966589030 shutdown_connections 2023-11-28T11:24:26.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.355+0000 7ff6ac00ab80 1 -- 172.21.15.92:0/2966589030 wait complete. 2023-11-28T11:24:26.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.356+0000 7ff6ac00ab80 1 -- start start 2023-11-28T11:24:26.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.356+0000 7ff6ac00ab80 1 Processor -- start 2023-11-28T11:24:26.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.357+0000 7ff6ac00ab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612386ec710 0x5612386be580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:26.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.357+0000 7ff6ac00ab80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561238700b70 0x5612386ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:26.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.357+0000 7ff6ac00ab80 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56123870a000 0x5612386c2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:26.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.357+0000 7ff6ac00ab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612386beac0 con 0x5612386ec710 2023-11-28T11:24:26.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.357+0000 7ff6ac00ab80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x561238773770 con 0x561238700b70 2023-11-28T11:24:26.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.357+0000 7ff6ac00ab80 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5612387028a0 con 0x56123870a000 2023-11-28T11:24:26.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.357+0000 7ff6ac00ab80 10 client.0 _ll_register_callbacks cb 0x5612385b44b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:24:26.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.357+0000 7ff69639b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612386ec710 0x5612386be580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:26.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.357+0000 7ff69639b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612386ec710 0x5612386be580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55418/0 (socket says 172.21.15.92:55418) 2023-11-28T11:24:26.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.357+0000 7ff69639b700 1 -- 172.21.15.92:0/3364215941 learned_addr learned my addr 172.21.15.92:0/3364215941 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:26.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.357+0000 7ff696b9c700 1 --2- 172.21.15.92:0/3364215941 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56123870a000 0x5612386c2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:26.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.357+0000 7ff695b9a700 1 --2- 172.21.15.92:0/3364215941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561238700b70 0x5612386ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:26.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.357+0000 7ff69639b700 1 -- 172.21.15.92:0/3364215941 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56123870a000 msgr2=0x5612386c2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:26.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.357+0000 7ff69639b700 1 --2- 172.21.15.92:0/3364215941 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56123870a000 0x5612386c2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.357+0000 7ff69639b700 1 -- 172.21.15.92:0/3364215941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561238700b70 msgr2=0x5612386ffcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:26.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.357+0000 7ff69639b700 1 --2- 172.21.15.92:0/3364215941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561238700b70 0x5612386ffcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.357+0000 7ff69639b700 1 -- 172.21.15.92:0/3364215941 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5612384f7150 con 0x5612386ec710 2023-11-28T11:24:26.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.358+0000 7ff69639b700 1 --2- 172.21.15.92:0/3364215941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612386ec710 0x5612386be580 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7ff68c008010 tx=0x7ff68c009210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d19fe2d2e94cf181 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:26.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.358+0000 7ff6717fa700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-11-28T11:24:26.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.358+0000 7ff6717fa700 1 -- 172.21.15.92:0/3364215941 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68c02f040 con 0x5612386ec710 2023-11-28T11:24:26.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.358+0000 7ff6717fa700 1 -- 172.21.15.92:0/3364215941 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(68..68 src has 1..68) v4 ==== 6382+0+0 (secure 0 0 0) 0x7ff68c0363b0 con 0x5612386ec710 2023-11-28T11:24:26.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.358+0000 7ff6ac00ab80 1 -- 172.21.15.92:0/3364215941 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff68c015a30 con 0x5612386ec710 2023-11-28T11:24:26.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.358+0000 7ff6717fa700 1 -- 172.21.15.92:0/3364215941 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff68c006de0 con 0x5612386ec710 2023-11-28T11:24:26.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.358+0000 7ff6ac00ab80 10 client.6414 fetch_fsmap learned FSMap version 32 2023-11-28T11:24:26.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.358+0000 7ff6ac00ab80 1 -- 172.21.15.92:0/3364215941 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x5612385c3bc0 con 0x5612386ec710 2023-11-28T11:24:26.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.358+0000 7ff6717fa700 1 -- 172.21.15.92:0/3364215941 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff68c015a30 con 0x5612386ec710 2023-11-28T11:24:26.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.358+0000 7ff6717fa700 1 -- 172.21.15.92:0/3364215941 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 32) v1 ==== 3106+0+0 (secure 0 0 0) 0x7ff68c015ba0 con 0x5612386ec710 2023-11-28T11:24:26.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.358+0000 7ff6ac00ab80 10 client.6414 fetch_fsmap finished waiting for FSMap version 32 2023-11-28T11:24:26.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.358+0000 7ff6ac00ab80 10 client.6414 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2023-11-28T11:24:26.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.358+0000 7ff6ac00ab80 20 client.6414 populate_metadata read hostname 'smithi092' 2023-11-28T11:24:26.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.358+0000 7ff6ac00ab80 1 --2- 172.21.15.92:0/3364215941 >> [v2:172.21.15.92:6834/3634521248,v1:172.21.15.92:6836/3634521248] conn(0x5612386c32a0 0x5612386edb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:26.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.358+0000 7ff6ac00ab80 4 client.6414 mds_command: new command op to 6213 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}] 2023-11-28T11:24:26.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.358+0000 7ff6ac00ab80 1 -- 172.21.15.92:0/3364215941 --> [v2:172.21.15.92:6834/3634521248,v1:172.21.15.92:6836/3634521248] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}) v1 -- 0x56123849b0f0 con 0x5612386c32a0 2023-11-28T11:24:26.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.359+0000 7ff695b9a700 1 --2- 172.21.15.92:0/3364215941 >> [v2:172.21.15.92:6834/3634521248,v1:172.21.15.92:6836/3634521248] conn(0x5612386c32a0 0x5612386edb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:26.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.359+0000 7ff695b9a700 1 --2- 172.21.15.92:0/3364215941 >> [v2:172.21.15.92:6834/3634521248,v1:172.21.15.92:6836/3634521248] conn(0x5612386c32a0 0x5612386edb60 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=872b6cfdcb558ff8 server_cookie=449e0fd837cd696b in_seq=0 out_seq=0 2023-11-28T11:24:26.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.359+0000 7ff6717fa700 10 client.6414 ms_handle_connect on v2:172.21.15.92:6834/3634521248 2023-11-28T11:24:26.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.360+0000 7ff6717fa700 1 -- 172.21.15.92:0/3364215941 <== mds.0 v2:172.21.15.92:6834/3634521248 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56123849b0f0 con 0x5612386c32a0 2023-11-28T11:24:26.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.360+0000 7ff6717fa700 10 client.6414 handle_command_reply: tid=0 2023-11-28T11:24:26.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.360+0000 7ff6ac00ab80 1 client.6414 shutdown 2023-11-28T11:24:26.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.360+0000 7ff6ac00ab80 1 -- 172.21.15.92:0/3364215941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612386ec710 msgr2=0x5612386be580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:26.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.360+0000 7ff6ac00ab80 1 --2- 172.21.15.92:0/3364215941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612386ec710 0x5612386be580 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7ff68c008010 tx=0x7ff68c009210 comp rx=0 tx=0).stop 2023-11-28T11:24:26.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.360+0000 7ff6ac00ab80 1 -- 172.21.15.92:0/3364215941 shutdown_connections 2023-11-28T11:24:26.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.360+0000 7ff6ac00ab80 1 --2- 172.21.15.92:0/3364215941 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x56123870a000 0x5612386c2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.360+0000 7ff6ac00ab80 1 --2- 172.21.15.92:0/3364215941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5612386ec710 0x5612386be580 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.360+0000 7ff6ac00ab80 1 --2- 172.21.15.92:0/3364215941 >> [v2:172.21.15.92:6834/3634521248,v1:172.21.15.92:6836/3634521248] conn(0x5612386c32a0 0x5612386edb60 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.360+0000 7ff6ac00ab80 1 --2- 172.21.15.92:0/3364215941 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x561238700b70 0x5612386ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.360+0000 7ff6ac00ab80 1 -- 172.21.15.92:0/3364215941 >> 172.21.15.92:0/3364215941 conn(0x5612386b9dd0 msgr2=0x5612386f2040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:26.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.360+0000 7ff6717fa700 0 client.6414 ms_handle_reset on v2:172.21.15.92:6834/3634521248 2023-11-28T11:24:26.366 INFO:tasks.ceph.mds.a.smithi092.stderr:2023-11-28T11:24:26.360+0000 7f4f523ea700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6115=0-4194304@1} caps={6115=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6115 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x564291d91b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(11)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-28T11:24:26.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.360+0000 7ff6ac00ab80 1 -- 172.21.15.92:0/3364215941 shutdown_connections 2023-11-28T11:24:26.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.360+0000 7ff6ac00ab80 1 -- 172.21.15.92:0/3364215941 wait complete. 2023-11-28T11:24:26.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.361+0000 7ff6ac00ab80 20 client.6414 trim_cache size 0 max 16384 2023-11-28T11:24:26.367 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-28T11:24:26.367 INFO:teuthology.orchestra.run.smithi092.stdout: "return_code": 0, 2023-11-28T11:24:26.367 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_tag": "d9b00fb7-24bf-4caf-a0f8-67b0344fdc43", 2023-11-28T11:24:26.367 INFO:teuthology.orchestra.run.smithi092.stdout: "mode": "asynchronous" 2023-11-28T11:24:26.367 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-28T11:24:26.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.361+0000 7ff6985ff700 1 -- 172.21.15.92:0/2487428271 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff6740682f0 msgr2=0x7ff67406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:26.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.361+0000 7ff6985ff700 1 --2- 172.21.15.92:0/2487428271 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff6740682f0 0x7ff67406a7a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff68c009b70 tx=0x7ff68c008040 comp rx=0 tx=0).stop 2023-11-28T11:24:26.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.361+0000 7ff6985ff700 1 -- 172.21.15.92:0/2487428271 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6900b8790 msgr2=0x7ff6900b5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:26.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.361+0000 7ff6985ff700 1 --2- 172.21.15.92:0/2487428271 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6900b8790 0x7ff6900b5cd0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7ff680009b90 tx=0x7ff680004750 comp rx=0 tx=0).stop 2023-11-28T11:24:26.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.361+0000 7ff6985ff700 1 -- 172.21.15.92:0/2487428271 shutdown_connections 2023-11-28T11:24:26.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.361+0000 7ff6985ff700 1 --2- 172.21.15.92:0/2487428271 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff6900cd460 0x7ff6900b6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.361+0000 7ff6985ff700 1 --2- 172.21.15.92:0/2487428271 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff69006be80 0x7ff690155a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.361+0000 7ff6985ff700 1 --2- 172.21.15.92:0/2487428271 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff6740682f0 0x7ff67406a7a0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.361+0000 7ff6985ff700 1 --2- 172.21.15.92:0/2487428271 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6900b8790 0x7ff6900b5cd0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.361+0000 7ff6985ff700 1 -- 172.21.15.92:0/2487428271 >> 172.21.15.92:0/2487428271 conn(0x7ff6900719a0 msgr2=0x7ff6900d24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:26.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.361+0000 7ff6985ff700 1 -- 172.21.15.92:0/2487428271 shutdown_connections 2023-11-28T11:24:26.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.361+0000 7ff6985ff700 1 -- 172.21.15.92:0/2487428271 wait complete. 2023-11-28T11:24:26.379 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub status 2023-11-28T11:24:26.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.576+0000 7f99f85ff700 1 Processor -- start 2023-11-28T11:24:26.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.576+0000 7f99f85ff700 1 -- start start 2023-11-28T11:24:26.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.577+0000 7f99f85ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99f00b8510 0x7f99f006b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:26.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.577+0000 7f99f85ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f99f00cd460 0x7f99f006beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:26.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.577+0000 7f99f85ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99f00b88e0 con 0x7f99f006c3f0 2023-11-28T11:24:26.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.577+0000 7f99f85ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99f00cdbd0 con 0x7f99f00cd460 2023-11-28T11:24:26.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.577+0000 7f99f85ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99f006c830 con 0x7f99f00b8510 2023-11-28T11:24:26.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.578+0000 7f99f639b700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99f00b8510 0x7f99f006b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:26.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.578+0000 7f99f5b9a700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f99f00cd460 0x7f99f006beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:26.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.578+0000 7f99f5b9a700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f99f00cd460 0x7f99f006beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.92:45682/0 (socket says 172.21.15.92:45682) 2023-11-28T11:24:26.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.578+0000 7f99f639b700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99f00b8510 0x7f99f006b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.92:44864/0 (socket says 172.21.15.92:44864) 2023-11-28T11:24:26.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.578+0000 7f99f5b9a700 1 -- 172.21.15.92:0/3071350624 learned_addr learned my addr 172.21.15.92:0/3071350624 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:26.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.578+0000 7f99f6b9c700 1 --1- 172.21.15.92:0/3071350624 >> v1:172.21.15.169:6789/0 conn(0x7f99f006c3f0 0x7f99f015cc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.92:53000/0 (socket says 172.21.15.92:53000) 2023-11-28T11:24:26.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.578+0000 7f99f639b700 1 -- 172.21.15.92:0/3071350624 >> v2:172.21.15.169:3300/0 conn(0x7f99f00cd460 msgr2=0x7f99f006beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:26.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.578+0000 7f99f639b700 1 --2- 172.21.15.92:0/3071350624 >> v2:172.21.15.169:3300/0 conn(0x7f99f00cd460 0x7f99f006beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.578+0000 7f99f639b700 1 -- 172.21.15.92:0/3071350624 >> v1:172.21.15.169:6789/0 conn(0x7f99f006c3f0 legacy=0x7f99f015cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:26.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.578+0000 7f99f639b700 1 -- 172.21.15.92:0/3071350624 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f00cd8f0 con 0x7f99f00b8510 2023-11-28T11:24:26.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.578+0000 7f99f5b9a700 1 --2- 172.21.15.92:0/3071350624 >> v2:172.21.15.169:3300/0 conn(0x7f99f00cd460 0x7f99f006beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:24:26.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.579+0000 7f99f639b700 1 --2- 172.21.15.92:0/3071350624 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99f00b8510 0x7f99f006b970 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f99ec009840 tx=0x7f99ec009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b839c74cd1c7d4a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:26.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.579+0000 7f99f5399700 1 -- 172.21.15.92:0/3071350624 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ec011830 con 0x7f99f00b8510 2023-11-28T11:24:26.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.579+0000 7f99f5399700 1 -- 172.21.15.92:0/3071350624 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99ec011990 con 0x7f99f00b8510 2023-11-28T11:24:26.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.579+0000 7f99f5399700 1 -- 172.21.15.92:0/3071350624 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ec011d20 con 0x7f99f00b8510 2023-11-28T11:24:26.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.579+0000 7f99f85ff700 1 -- 172.21.15.92:0/3071350624 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99f00b8510 msgr2=0x7f99f006b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:26.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.579+0000 7f99f85ff700 1 --2- 172.21.15.92:0/3071350624 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99f00b8510 0x7f99f006b970 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f99ec009840 tx=0x7f99ec009b50 comp rx=0 tx=0).stop 2023-11-28T11:24:26.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.579+0000 7f99f85ff700 1 -- 172.21.15.92:0/3071350624 shutdown_connections 2023-11-28T11:24:26.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.579+0000 7f99f85ff700 1 --2- 172.21.15.92:0/3071350624 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99f00b8510 0x7f99f006b970 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.579+0000 7f99f85ff700 1 --2- 172.21.15.92:0/3071350624 >> v2:172.21.15.169:3300/0 conn(0x7f99f00cd460 0x7f99f006beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.579+0000 7f99f85ff700 1 -- 172.21.15.92:0/3071350624 >> 172.21.15.92:0/3071350624 conn(0x7f99f00719a0 msgr2=0x7f99f0071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:26.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.579+0000 7f99f85ff700 1 -- 172.21.15.92:0/3071350624 shutdown_connections 2023-11-28T11:24:26.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.580+0000 7f99f85ff700 1 -- 172.21.15.92:0/3071350624 wait complete. 2023-11-28T11:24:26.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.581+0000 7f99f85ff700 1 Processor -- start 2023-11-28T11:24:26.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.581+0000 7f99f85ff700 1 -- start start 2023-11-28T11:24:26.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.582+0000 7f99f85ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99f006c3f0 0x7f99f00c9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:26.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.582+0000 7f99f85ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99f00b8510 0x7f99f00cb740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:26.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.582+0000 7f99f85ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99f00cd460 0x7f99f00c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:26.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.582+0000 7f99f85ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f00cb2b0 con 0x7f99f00cd460 2023-11-28T11:24:26.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.582+0000 7f99f85ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f00cbc80 con 0x7f99f00b8510 2023-11-28T11:24:26.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.582+0000 7f99f85ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99f00ca420 con 0x7f99f006c3f0 2023-11-28T11:24:26.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.582+0000 7f99f6b9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99f00cd460 0x7f99f00c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:26.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.583+0000 7f99f6b9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99f00cd460 0x7f99f00c9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55426/0 (socket says 172.21.15.92:55426) 2023-11-28T11:24:26.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.583+0000 7f99f6b9c700 1 -- 172.21.15.92:0/569905934 learned_addr learned my addr 172.21.15.92:0/569905934 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:26.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.583+0000 7f99f639b700 1 --2- 172.21.15.92:0/569905934 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99f006c3f0 0x7f99f00c9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:26.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.583+0000 7f99f5b9a700 1 --2- 172.21.15.92:0/569905934 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99f00b8510 0x7f99f00cb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:26.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.583+0000 7f99f6b9c700 1 -- 172.21.15.92:0/569905934 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99f006c3f0 msgr2=0x7f99f00c9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:26.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.583+0000 7f99f6b9c700 1 --2- 172.21.15.92:0/569905934 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99f006c3f0 0x7f99f00c9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.583+0000 7f99f6b9c700 1 -- 172.21.15.92:0/569905934 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99f00b8510 msgr2=0x7f99f00cb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:26.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.583+0000 7f99f6b9c700 1 --2- 172.21.15.92:0/569905934 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99f00b8510 0x7f99f00cb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.583+0000 7f99f6b9c700 1 -- 172.21.15.92:0/569905934 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e0009550 con 0x7f99f00cd460 2023-11-28T11:24:26.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.583+0000 7f99f6b9c700 1 --2- 172.21.15.92:0/569905934 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99f00cd460 0x7f99f00c9ee0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f99e800cc00 tx=0x7f99e800cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:26.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.583+0000 7f99e6ffd700 1 -- 172.21.15.92:0/569905934 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e800aa40 con 0x7f99f00cd460 2023-11-28T11:24:26.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.583+0000 7f99e6ffd700 1 -- 172.21.15.92:0/569905934 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99e800aba0 con 0x7f99f00cd460 2023-11-28T11:24:26.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.583+0000 7f99f85ff700 1 -- 172.21.15.92:0/569905934 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99ec0094a0 con 0x7f99f00cd460 2023-11-28T11:24:26.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.583+0000 7f99e6ffd700 1 -- 172.21.15.92:0/569905934 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e8007070 con 0x7f99f00cd460 2023-11-28T11:24:26.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.584+0000 7f99f85ff700 1 -- 172.21.15.92:0/569905934 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99f00cfff0 con 0x7f99f00cd460 2023-11-28T11:24:26.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.586+0000 7f99e6ffd700 1 -- 172.21.15.92:0/569905934 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f99e8019030 con 0x7f99f00cd460 2023-11-28T11:24:26.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.587+0000 7f99e6ffd700 1 --2- 172.21.15.92:0/569905934 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f99d40682f0 0x7f99d406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:26.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.587+0000 7f99e6ffd700 1 -- 172.21.15.92:0/569905934 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f99e80859a0 con 0x7f99f00cd460 2023-11-28T11:24:26.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.587+0000 7f99f639b700 1 --2- 172.21.15.92:0/569905934 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f99d40682f0 0x7f99d406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:26.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.588+0000 7f99f639b700 1 --2- 172.21.15.92:0/569905934 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f99d40682f0 0x7f99d406a7a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f99ec009fd0 tx=0x7f99ec008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:26.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.589+0000 7f9a0be79b80 1 Processor -- start 2023-11-28T11:24:26.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.589+0000 7f9a0be79b80 1 -- start start 2023-11-28T11:24:26.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.590+0000 7f9a0be79b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5582ff430fa0 0x5582ff3eaa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:26.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.590+0000 7f9a0be79b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582ff427b10 0x5582ff431370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:26.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.590+0000 7f9a0be79b80 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5582ff4136b0 0x5582ff3ebbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:26.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.590+0000 7f9a0be79b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582ff368e30 con 0x5582ff427b10 2023-11-28T11:24:26.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.590+0000 7f9a0be79b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582ff2d5190 con 0x5582ff430fa0 2023-11-28T11:24:26.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.590+0000 7f9a0be79b80 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5582ff343e00 con 0x5582ff4136b0 2023-11-28T11:24:26.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.590+0000 7f99f5b9a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582ff427b10 0x5582ff431370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:26.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.590+0000 7f99f5b9a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582ff427b10 0x5582ff431370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55438/0 (socket says 172.21.15.92:55438) 2023-11-28T11:24:26.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.590+0000 7f99f5b9a700 1 -- 172.21.15.92:0/293032762 learned_addr learned my addr 172.21.15.92:0/293032762 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:26.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.590+0000 7f99f639b700 1 --2- 172.21.15.92:0/293032762 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5582ff430fa0 0x5582ff3eaa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:26.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.590+0000 7f99f6b9c700 1 --2- 172.21.15.92:0/293032762 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5582ff4136b0 0x5582ff3ebbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:26.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.590+0000 7f99f5b9a700 1 -- 172.21.15.92:0/293032762 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5582ff4136b0 msgr2=0x5582ff3ebbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:26.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.590+0000 7f99f5b9a700 1 --2- 172.21.15.92:0/293032762 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5582ff4136b0 0x5582ff3ebbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.590+0000 7f99f5b9a700 1 -- 172.21.15.92:0/293032762 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5582ff430fa0 msgr2=0x5582ff3eaa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:26.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.590+0000 7f99f5b9a700 1 --2- 172.21.15.92:0/293032762 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5582ff430fa0 0x5582ff3eaa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.590+0000 7f99f5b9a700 1 -- 172.21.15.92:0/293032762 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582ff1317e0 con 0x5582ff427b10 2023-11-28T11:24:26.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.591+0000 7f99f5b9a700 1 --2- 172.21.15.92:0/293032762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582ff427b10 0x5582ff431370 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f99e0009d60 tx=0x7f99e00115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb1a5a41b1d0fc41 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:26.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.591+0000 7f99d3fff700 1 -- 172.21.15.92:0/293032762 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e0009e50 con 0x5582ff427b10 2023-11-28T11:24:26.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.591+0000 7f99d3fff700 1 -- 172.21.15.92:0/293032762 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99e0011d60 con 0x5582ff427b10 2023-11-28T11:24:26.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.591+0000 7f99d3fff700 1 -- 172.21.15.92:0/293032762 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e0014640 con 0x5582ff427b10 2023-11-28T11:24:26.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.591+0000 7f9a0be79b80 1 -- 172.21.15.92:0/293032762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582ff427b10 msgr2=0x5582ff431370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:26.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.591+0000 7f9a0be79b80 1 --2- 172.21.15.92:0/293032762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582ff427b10 0x5582ff431370 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f99e0009d60 tx=0x7f99e00115a0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.592+0000 7f9a0be79b80 1 -- 172.21.15.92:0/293032762 shutdown_connections 2023-11-28T11:24:26.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.592+0000 7f9a0be79b80 1 --2- 172.21.15.92:0/293032762 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5582ff4136b0 0x5582ff3ebbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.592+0000 7f9a0be79b80 1 --2- 172.21.15.92:0/293032762 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5582ff430fa0 0x5582ff3eaa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.592+0000 7f9a0be79b80 1 --2- 172.21.15.92:0/293032762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582ff427b10 0x5582ff431370 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.592+0000 7f9a0be79b80 1 -- 172.21.15.92:0/293032762 >> 172.21.15.92:0/293032762 conn(0x5582ff3e0d70 msgr2=0x5582ff3e9f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:26.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.592+0000 7f9a0be79b80 1 -- 172.21.15.92:0/293032762 shutdown_connections 2023-11-28T11:24:26.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.592+0000 7f9a0be79b80 1 -- 172.21.15.92:0/293032762 wait complete. 2023-11-28T11:24:26.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.592+0000 7f9a0be79b80 1 -- start start 2023-11-28T11:24:26.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.593+0000 7f9a0be79b80 1 Processor -- start 2023-11-28T11:24:26.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.594+0000 7f9a0be79b80 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5582ff4136b0 0x5582ff419140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:26.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.594+0000 7f9a0be79b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582ff427b10 0x5582ff416fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:26.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.594+0000 7f9a0be79b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5582ff430fa0 0x5582ff403490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:26.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.594+0000 7f9a0be79b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582ff33d170 con 0x5582ff427b10 2023-11-28T11:24:26.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.594+0000 7f9a0be79b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582ff1323e0 con 0x5582ff430fa0 2023-11-28T11:24:26.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.594+0000 7f9a0be79b80 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5582ff2d5390 con 0x5582ff4136b0 2023-11-28T11:24:26.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.595+0000 7f9a0be79b80 10 client.0 _ll_register_callbacks cb 0x5582ff2db450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:24:26.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.595+0000 7f99f5b9a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582ff427b10 0x5582ff416fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:26.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.595+0000 7f99f5b9a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582ff427b10 0x5582ff416fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55442/0 (socket says 172.21.15.92:55442) 2023-11-28T11:24:26.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.595+0000 7f99f5b9a700 1 -- 172.21.15.92:0/3842074097 learned_addr learned my addr 172.21.15.92:0/3842074097 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:26.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.595+0000 7f99f639b700 1 --2- 172.21.15.92:0/3842074097 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5582ff4136b0 0x5582ff419140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:26.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.595+0000 7f99f6b9c700 1 --2- 172.21.15.92:0/3842074097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5582ff430fa0 0x5582ff403490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:26.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.595+0000 7f99f5b9a700 1 -- 172.21.15.92:0/3842074097 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5582ff4136b0 msgr2=0x5582ff419140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:26.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.595+0000 7f99f5b9a700 1 --2- 172.21.15.92:0/3842074097 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5582ff4136b0 0x5582ff419140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.595+0000 7f99f5b9a700 1 -- 172.21.15.92:0/3842074097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5582ff430fa0 msgr2=0x5582ff403490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:26.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.595+0000 7f99f5b9a700 1 --2- 172.21.15.92:0/3842074097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5582ff430fa0 0x5582ff403490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.595+0000 7f99f5b9a700 1 -- 172.21.15.92:0/3842074097 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99e0009930 con 0x5582ff427b10 2023-11-28T11:24:26.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.595+0000 7f99f6b9c700 1 --2- 172.21.15.92:0/3842074097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5582ff430fa0 0x5582ff403490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:24:26.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.595+0000 7f99f639b700 1 --2- 172.21.15.92:0/3842074097 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5582ff4136b0 0x5582ff419140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:24:26.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.595+0000 7f99f5b9a700 1 --2- 172.21.15.92:0/3842074097 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582ff427b10 0x5582ff416fd0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f99e0006d30 tx=0x7f99e00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd96342f6a22d803 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:26.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.595+0000 7f99d17fa700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-11-28T11:24:26.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.596+0000 7f99d17fa700 1 -- 172.21.15.92:0/3842074097 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e0004220 con 0x5582ff427b10 2023-11-28T11:24:26.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.596+0000 7f99d17fa700 1 -- 172.21.15.92:0/3842074097 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(68..68 src has 1..68) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f99e0029d50 con 0x5582ff427b10 2023-11-28T11:24:26.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.596+0000 7f9a0be79b80 1 -- 172.21.15.92:0/3842074097 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5582ff341080 con 0x5582ff427b10 2023-11-28T11:24:26.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.596+0000 7f99d17fa700 1 -- 172.21.15.92:0/3842074097 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99e0011ed0 con 0x5582ff427b10 2023-11-28T11:24:26.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.596+0000 7f9a0be79b80 10 client.6423 fetch_fsmap learned FSMap version 32 2023-11-28T11:24:26.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.596+0000 7f9a0be79b80 1 -- 172.21.15.92:0/3842074097 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x5582ff21e0f0 con 0x5582ff427b10 2023-11-28T11:24:26.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.596+0000 7f99d17fa700 1 -- 172.21.15.92:0/3842074097 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5582ff341080 con 0x5582ff427b10 2023-11-28T11:24:26.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.596+0000 7f99d17fa700 1 -- 172.21.15.92:0/3842074097 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 32) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f99e001b020 con 0x5582ff427b10 2023-11-28T11:24:26.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.596+0000 7f9a0be79b80 10 client.6423 fetch_fsmap finished waiting for FSMap version 32 2023-11-28T11:24:26.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.596+0000 7f9a0be79b80 10 client.6423 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2023-11-28T11:24:26.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.596+0000 7f9a0be79b80 20 client.6423 populate_metadata read hostname 'smithi092' 2023-11-28T11:24:26.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.596+0000 7f9a0be79b80 1 --2- 172.21.15.92:0/3842074097 >> [v2:172.21.15.92:6834/3634521248,v1:172.21.15.92:6836/3634521248] conn(0x5582ff417510 0x5582ff428980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:26.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.596+0000 7f9a0be79b80 4 client.6423 mds_command: new command op to 6213 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:24:26.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.596+0000 7f9a0be79b80 1 -- 172.21.15.92:0/3842074097 --> [v2:172.21.15.92:6834/3634521248,v1:172.21.15.92:6836/3634521248] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5582ff18cf40 con 0x5582ff417510 2023-11-28T11:24:26.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.597+0000 7f99f639b700 1 --2- 172.21.15.92:0/3842074097 >> [v2:172.21.15.92:6834/3634521248,v1:172.21.15.92:6836/3634521248] conn(0x5582ff417510 0x5582ff428980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:26.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.597+0000 7f99d17fa700 10 client.6423 ms_handle_connect on v2:172.21.15.92:6834/3634521248 2023-11-28T11:24:26.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.597+0000 7f99f639b700 1 --2- 172.21.15.92:0/3842074097 >> [v2:172.21.15.92:6834/3634521248,v1:172.21.15.92:6836/3634521248] conn(0x5582ff417510 0x5582ff428980 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ba89aca45f3c2a7 server_cookie=6550ea4e314b54b7 in_seq=0 out_seq=0 2023-11-28T11:24:26.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.598+0000 7f99d17fa700 1 -- 172.21.15.92:0/3842074097 <== mds.0 v2:172.21.15.92:6834/3634521248 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x5582ff18cf40 con 0x5582ff417510 2023-11-28T11:24:26.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.598+0000 7f99d17fa700 10 client.6423 handle_command_reply: tid=0 2023-11-28T11:24:26.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.598+0000 7f9a0be79b80 1 client.6423 shutdown 2023-11-28T11:24:26.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.598+0000 7f9a0be79b80 1 -- 172.21.15.92:0/3842074097 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582ff427b10 msgr2=0x5582ff416fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:26.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.598+0000 7f9a0be79b80 1 --2- 172.21.15.92:0/3842074097 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582ff427b10 0x5582ff416fd0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f99e0006d30 tx=0x7f99e00047c0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.598+0000 7f9a0be79b80 1 -- 172.21.15.92:0/3842074097 shutdown_connections 2023-11-28T11:24:26.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.598+0000 7f9a0be79b80 1 --2- 172.21.15.92:0/3842074097 >> [v2:172.21.15.92:6834/3634521248,v1:172.21.15.92:6836/3634521248] conn(0x5582ff417510 0x5582ff428980 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.598+0000 7f9a0be79b80 1 --2- 172.21.15.92:0/3842074097 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5582ff430fa0 0x5582ff403490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.598+0000 7f9a0be79b80 1 --2- 172.21.15.92:0/3842074097 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5582ff4136b0 0x5582ff419140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.598+0000 7f9a0be79b80 1 --2- 172.21.15.92:0/3842074097 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582ff427b10 0x5582ff416fd0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.598+0000 7f99d17fa700 0 client.6423 ms_handle_reset on v2:172.21.15.92:6834/3634521248 2023-11-28T11:24:26.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.598+0000 7f9a0be79b80 1 -- 172.21.15.92:0/3842074097 >> 172.21.15.92:0/3842074097 conn(0x5582ff3e0d70 msgr2=0x5582ff3e9f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:26.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.598+0000 7f9a0be79b80 1 -- 172.21.15.92:0/3842074097 shutdown_connections 2023-11-28T11:24:26.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.598+0000 7f9a0be79b80 1 -- 172.21.15.92:0/3842074097 wait complete. 2023-11-28T11:24:26.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.598+0000 7f9a0be79b80 20 client.6423 trim_cache size 0 max 16384 2023-11-28T11:24:26.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.615+0000 7f9a0be79b80 1 Processor -- start 2023-11-28T11:24:26.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.615+0000 7f9a0be79b80 1 -- start start 2023-11-28T11:24:26.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.616+0000 7f9a0be79b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5582ff4136b0 0x5582ff428ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:26.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.616+0000 7f9a0be79b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582ff427b10 0x5582ff4293e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:26.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.616+0000 7f9a0be79b80 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5582ff430fa0 0x5582ff3ebae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:26.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.616+0000 7f9a0be79b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582ff3ec930 con 0x5582ff427b10 2023-11-28T11:24:26.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.616+0000 7f9a0be79b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582ff3ed9a0 con 0x5582ff4136b0 2023-11-28T11:24:26.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.616+0000 7f9a0be79b80 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5582ff428270 con 0x5582ff430fa0 2023-11-28T11:24:26.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.617+0000 7f99f5b9a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582ff427b10 0x5582ff4293e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:26.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.617+0000 7f99f5b9a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582ff427b10 0x5582ff4293e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55444/0 (socket says 172.21.15.92:55444) 2023-11-28T11:24:26.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.617+0000 7f99f5b9a700 1 -- 172.21.15.92:0/707395670 learned_addr learned my addr 172.21.15.92:0/707395670 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:26.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.617+0000 7f99f6b9c700 1 --2- 172.21.15.92:0/707395670 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5582ff430fa0 0x5582ff3ebae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:26.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.617+0000 7f99f639b700 1 --2- 172.21.15.92:0/707395670 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5582ff4136b0 0x5582ff428ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:26.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.617+0000 7f99f5b9a700 1 -- 172.21.15.92:0/707395670 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5582ff430fa0 msgr2=0x5582ff3ebae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:26.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.617+0000 7f99f5b9a700 1 --2- 172.21.15.92:0/707395670 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5582ff430fa0 0x5582ff3ebae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.617+0000 7f99f5b9a700 1 -- 172.21.15.92:0/707395670 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5582ff4136b0 msgr2=0x5582ff428ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:26.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.617+0000 7f99f5b9a700 1 --2- 172.21.15.92:0/707395670 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5582ff4136b0 0x5582ff428ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.617+0000 7f99f5b9a700 1 -- 172.21.15.92:0/707395670 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582ff304330 con 0x5582ff427b10 2023-11-28T11:24:26.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.617+0000 7f99f6b9c700 1 --2- 172.21.15.92:0/707395670 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5582ff430fa0 0x5582ff3ebae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:24:26.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.617+0000 7f99f5b9a700 1 --2- 172.21.15.92:0/707395670 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582ff427b10 0x5582ff4293e0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f99e000b010 tx=0x7f99e0022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d8d3736ee13ccec server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:26.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.617+0000 7f99e4bff700 1 -- 172.21.15.92:0/707395670 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e0031620 con 0x5582ff427b10 2023-11-28T11:24:26.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.617+0000 7f99e4bff700 1 -- 172.21.15.92:0/707395670 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99e0011ed0 con 0x5582ff427b10 2023-11-28T11:24:26.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.617+0000 7f99e4bff700 1 -- 172.21.15.92:0/707395670 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e00319c0 con 0x5582ff427b10 2023-11-28T11:24:26.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.618+0000 7f9a0be79b80 1 -- 172.21.15.92:0/707395670 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582ff427b10 msgr2=0x5582ff4293e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:26.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.618+0000 7f9a0be79b80 1 --2- 172.21.15.92:0/707395670 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582ff427b10 0x5582ff4293e0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f99e000b010 tx=0x7f99e0022000 comp rx=0 tx=0).stop 2023-11-28T11:24:26.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.618+0000 7f9a0be79b80 1 -- 172.21.15.92:0/707395670 shutdown_connections 2023-11-28T11:24:26.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.618+0000 7f9a0be79b80 1 --2- 172.21.15.92:0/707395670 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5582ff430fa0 0x5582ff3ebae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.618+0000 7f9a0be79b80 1 --2- 172.21.15.92:0/707395670 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5582ff4136b0 0x5582ff428ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.618+0000 7f9a0be79b80 1 --2- 172.21.15.92:0/707395670 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582ff427b10 0x5582ff4293e0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.618+0000 7f9a0be79b80 1 -- 172.21.15.92:0/707395670 >> 172.21.15.92:0/707395670 conn(0x5582ff3e0d70 msgr2=0x5582ff418f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:26.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.618+0000 7f9a0be79b80 1 -- 172.21.15.92:0/707395670 shutdown_connections 2023-11-28T11:24:26.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.618+0000 7f9a0be79b80 1 -- 172.21.15.92:0/707395670 wait complete. 2023-11-28T11:24:26.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.619+0000 7f9a0be79b80 1 -- start start 2023-11-28T11:24:26.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.619+0000 7f9a0be79b80 1 Processor -- start 2023-11-28T11:24:26.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.621+0000 7f9a0be79b80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5582ff4136b0 0x5582ff42a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:26.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.621+0000 7f9a0be79b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582ff427b10 0x5582ff44ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:26.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.621+0000 7f9a0be79b80 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5582ff430fa0 0x5582ff415bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:26.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.621+0000 7f9a0be79b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582ff44f880 con 0x5582ff427b10 2023-11-28T11:24:26.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.621+0000 7f9a0be79b80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582ff429ca0 con 0x5582ff4136b0 2023-11-28T11:24:26.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.621+0000 7f9a0be79b80 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x5582ff416620 con 0x5582ff430fa0 2023-11-28T11:24:26.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.621+0000 7f9a0be79b80 10 client.0 _ll_register_callbacks cb 0x5582ff2db450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:24:26.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.621+0000 7f99f5b9a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582ff427b10 0x5582ff44ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:26.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.621+0000 7f99f5b9a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582ff427b10 0x5582ff44ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55460/0 (socket says 172.21.15.92:55460) 2023-11-28T11:24:26.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.621+0000 7f99f5b9a700 1 -- 172.21.15.92:0/1323042070 learned_addr learned my addr 172.21.15.92:0/1323042070 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:26.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.621+0000 7f99f6b9c700 1 --2- 172.21.15.92:0/1323042070 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5582ff430fa0 0x5582ff415bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:26.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.621+0000 7f99f639b700 1 --2- 172.21.15.92:0/1323042070 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5582ff4136b0 0x5582ff42a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:26.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.621+0000 7f99f5b9a700 1 -- 172.21.15.92:0/1323042070 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5582ff430fa0 msgr2=0x5582ff415bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:26.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.621+0000 7f99f5b9a700 1 --2- 172.21.15.92:0/1323042070 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5582ff430fa0 0x5582ff415bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.621+0000 7f99f5b9a700 1 -- 172.21.15.92:0/1323042070 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5582ff4136b0 msgr2=0x5582ff42a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:26.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.621+0000 7f99f5b9a700 1 --2- 172.21.15.92:0/1323042070 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5582ff4136b0 0x5582ff42a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.621+0000 7f99f5b9a700 1 -- 172.21.15.92:0/1323042070 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5582ff21e0f0 con 0x5582ff427b10 2023-11-28T11:24:26.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.622+0000 7f99f5b9a700 1 --2- 172.21.15.92:0/1323042070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582ff427b10 0x5582ff44ece0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f99e0004590 tx=0x7f99e00092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ca2284a217fdf30 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:26.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.622+0000 7f99d17fa700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-11-28T11:24:26.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.622+0000 7f99d17fa700 1 -- 172.21.15.92:0/1323042070 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e0031d20 con 0x5582ff427b10 2023-11-28T11:24:26.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.622+0000 7f99d17fa700 1 -- 172.21.15.92:0/1323042070 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(68..68 src has 1..68) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f99e0000e40 con 0x5582ff427b10 2023-11-28T11:24:26.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.622+0000 7f9a0be79b80 1 -- 172.21.15.92:0/1323042070 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f99e0011d60 con 0x5582ff427b10 2023-11-28T11:24:26.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.622+0000 7f99d17fa700 1 -- 172.21.15.92:0/1323042070 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99e0014bd0 con 0x5582ff427b10 2023-11-28T11:24:26.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.622+0000 7f9a0be79b80 10 client.6429 fetch_fsmap learned FSMap version 32 2023-11-28T11:24:26.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.622+0000 7f9a0be79b80 1 -- 172.21.15.92:0/1323042070 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x5582ff192220 con 0x5582ff427b10 2023-11-28T11:24:26.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.622+0000 7f99d17fa700 1 -- 172.21.15.92:0/1323042070 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99e0011d60 con 0x5582ff427b10 2023-11-28T11:24:26.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.622+0000 7f99d17fa700 1 -- 172.21.15.92:0/1323042070 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 32) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f99e001b040 con 0x5582ff427b10 2023-11-28T11:24:26.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.623+0000 7f9a0be79b80 10 client.6429 fetch_fsmap finished waiting for FSMap version 32 2023-11-28T11:24:26.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.623+0000 7f9a0be79b80 10 client.6429 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2023-11-28T11:24:26.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.623+0000 7f9a0be79b80 20 client.6429 populate_metadata read hostname 'smithi092' 2023-11-28T11:24:26.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.623+0000 7f9a0be79b80 1 --2- 172.21.15.92:0/1323042070 >> [v2:172.21.15.92:6834/3634521248,v1:172.21.15.92:6836/3634521248] conn(0x5582ff416100 0x5582ff42fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:26.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.623+0000 7f9a0be79b80 4 client.6429 mds_command: new command op to 6213 tid=0[{"prefix": "scrub status"}] 2023-11-28T11:24:26.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.623+0000 7f9a0be79b80 1 -- 172.21.15.92:0/1323042070 --> [v2:172.21.15.92:6834/3634521248,v1:172.21.15.92:6836/3634521248] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5582ff1c2090 con 0x5582ff416100 2023-11-28T11:24:26.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.623+0000 7f99f639b700 1 --2- 172.21.15.92:0/1323042070 >> [v2:172.21.15.92:6834/3634521248,v1:172.21.15.92:6836/3634521248] conn(0x5582ff416100 0x5582ff42fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:26.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.623+0000 7f99f639b700 1 --2- 172.21.15.92:0/1323042070 >> [v2:172.21.15.92:6834/3634521248,v1:172.21.15.92:6836/3634521248] conn(0x5582ff416100 0x5582ff42fa60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8e2922f024fad22 server_cookie=fb1e467114d04508 in_seq=0 out_seq=0 2023-11-28T11:24:26.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.623+0000 7f99d17fa700 10 client.6429 ms_handle_connect on v2:172.21.15.92:6834/3634521248 2023-11-28T11:24:26.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.623+0000 7f99d17fa700 1 -- 172.21.15.92:0/1323042070 <== mds.0 v2:172.21.15.92:6834/3634521248 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+253 (crc 0 0 0) 0x5582ff1c2090 con 0x5582ff416100 2023-11-28T11:24:26.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.623+0000 7f99d17fa700 10 client.6429 handle_command_reply: tid=0 2023-11-28T11:24:26.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.623+0000 7f9a0be79b80 1 client.6429 shutdown 2023-11-28T11:24:26.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.624+0000 7f9a0be79b80 1 -- 172.21.15.92:0/1323042070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582ff427b10 msgr2=0x5582ff44ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:26.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.624+0000 7f9a0be79b80 1 --2- 172.21.15.92:0/1323042070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582ff427b10 0x5582ff44ece0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f99e0004590 tx=0x7f99e00092c0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.624+0000 7f9a0be79b80 1 -- 172.21.15.92:0/1323042070 shutdown_connections 2023-11-28T11:24:26.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.624+0000 7f9a0be79b80 1 --2- 172.21.15.92:0/1323042070 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x5582ff430fa0 0x5582ff415bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.624+0000 7f9a0be79b80 1 --2- 172.21.15.92:0/1323042070 >> [v2:172.21.15.92:6834/3634521248,v1:172.21.15.92:6836/3634521248] conn(0x5582ff416100 0x5582ff42fa60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.624+0000 7f9a0be79b80 1 --2- 172.21.15.92:0/1323042070 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x5582ff4136b0 0x5582ff42a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.624+0000 7f9a0be79b80 1 --2- 172.21.15.92:0/1323042070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5582ff427b10 0x5582ff44ece0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.624+0000 7f99d17fa700 0 client.6429 ms_handle_reset on v2:172.21.15.92:6834/3634521248 2023-11-28T11:24:26.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.624+0000 7f9a0be79b80 1 -- 172.21.15.92:0/1323042070 >> 172.21.15.92:0/1323042070 conn(0x5582ff3e0d70 msgr2=0x5582ff3e6ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:26.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.624+0000 7f9a0be79b80 1 -- 172.21.15.92:0/1323042070 shutdown_connections 2023-11-28T11:24:26.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.624+0000 7f9a0be79b80 1 -- 172.21.15.92:0/1323042070 wait complete. 2023-11-28T11:24:26.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.624+0000 7f9a0be79b80 20 client.6429 trim_cache size 0 max 16384 2023-11-28T11:24:26.632 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-28T11:24:26.632 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-28T11:24:26.632 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": { 2023-11-28T11:24:26.632 INFO:teuthology.orchestra.run.smithi092.stdout: "d9b00fb7-24bf-4caf-a0f8-67b0344fdc43": { 2023-11-28T11:24:26.632 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/", 2023-11-28T11:24:26.632 INFO:teuthology.orchestra.run.smithi092.stdout: "tag": "d9b00fb7-24bf-4caf-a0f8-67b0344fdc43", 2023-11-28T11:24:26.633 INFO:teuthology.orchestra.run.smithi092.stdout: "options": "recursive" 2023-11-28T11:24:26.633 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:24:26.633 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:24:26.633 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-28T11:24:26.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.625+0000 7f99f85ff700 1 -- 172.21.15.92:0/569905934 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f99d40682f0 msgr2=0x7f99d406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:26.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.625+0000 7f99f85ff700 1 --2- 172.21.15.92:0/569905934 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f99d40682f0 0x7f99d406a7a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f99ec009fd0 tx=0x7f99ec008030 comp rx=0 tx=0).stop 2023-11-28T11:24:26.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.625+0000 7f99f85ff700 1 -- 172.21.15.92:0/569905934 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99f00cd460 msgr2=0x7f99f00c9ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:26.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.625+0000 7f99f85ff700 1 --2- 172.21.15.92:0/569905934 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99f00cd460 0x7f99f00c9ee0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f99e800cc00 tx=0x7f99e800cf10 comp rx=0 tx=0).stop 2023-11-28T11:24:26.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.625+0000 7f99f85ff700 1 -- 172.21.15.92:0/569905934 shutdown_connections 2023-11-28T11:24:26.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.625+0000 7f99f85ff700 1 --2- 172.21.15.92:0/569905934 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f99d40682f0 0x7f99d406a7a0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.625+0000 7f99f85ff700 1 --2- 172.21.15.92:0/569905934 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99f00cd460 0x7f99f00c9ee0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.625+0000 7f99f85ff700 1 --2- 172.21.15.92:0/569905934 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99f006c3f0 0x7f99f00c9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.625+0000 7f99f85ff700 1 --2- 172.21.15.92:0/569905934 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f99f00b8510 0x7f99f00cb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:26.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.625+0000 7f99f85ff700 1 -- 172.21.15.92:0/569905934 >> 172.21.15.92:0/569905934 conn(0x7f99f00719a0 msgr2=0x7f99f00cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:26.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.625+0000 7f99f85ff700 1 -- 172.21.15.92:0/569905934 shutdown_connections 2023-11-28T11:24:26.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:26.625+0000 7f99f85ff700 1 -- 172.21.15.92:0/569905934 wait complete. 2023-11-28T11:24:26.643 INFO:tasks.cephfs.filesystem:scrub status for tag:d9b00fb7-24bf-4caf-a0f8-67b0344fdc43 - {'path': '/', 'tag': 'd9b00fb7-24bf-4caf-a0f8-67b0344fdc43', 'options': 'recursive'} 2023-11-28T11:24:32.577 INFO:tasks.ceph.mon.a.smithi092.stderr:2023-11-28T11:24:32.576+0000 7f086abf9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-11-28T11:24:48.806 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:24:48.812 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:24:56.644 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub status 2023-11-28T11:24:56.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.850+0000 7f7e755ff700 1 Processor -- start 2023-11-28T11:24:56.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.850+0000 7f7e755ff700 1 -- start start 2023-11-28T11:24:56.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.851+0000 7f7e755ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7e700b8510 0x7f7e7006b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:56.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.852+0000 7f7e755ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e700b88e0 con 0x7f7e700cd460 2023-11-28T11:24:56.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.852+0000 7f7e755ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e700cd8f0 con 0x7f7e700d0d60 2023-11-28T11:24:56.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.852+0000 7f7e755ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e700cdbd0 con 0x7f7e700b8510 2023-11-28T11:24:56.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.852+0000 7f7e67fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f7e700cd460 0x7f7e7006beb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53710/0 (socket says 172.21.15.92:53710) 2023-11-28T11:24:56.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.852+0000 7f7e6effd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7e700b8510 0x7f7e7006b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:56.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.852+0000 7f7e67fff700 1 -- 172.21.15.92:0/372381555 learned_addr learned my addr 172.21.15.92:0/372381555 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:24:56.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.852+0000 7f7e6effd700 1 -- 172.21.15.92:0/372381555 >> v1:172.21.15.169:6789/0 conn(0x7f7e700d0d60 legacy=0x7f7e700d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:56.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.852+0000 7f7e6effd700 1 -- 172.21.15.92:0/372381555 >> v1:172.21.15.92:6789/0 conn(0x7f7e700cd460 legacy=0x7f7e7006beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:56.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.852+0000 7f7e6effd700 1 -- 172.21.15.92:0/372381555 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e700cfd10 con 0x7f7e700b8510 2023-11-28T11:24:56.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.852+0000 7f7e6e7fc700 1 -- 172.21.15.92:0/372381555 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2083423534 0 0) 0x7f7e700b88e0 con 0x7f7e700cd460 2023-11-28T11:24:56.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.853+0000 7f7e6effd700 1 --2- 172.21.15.92:0/372381555 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e700b8510 0x7f7e7006b970 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f7e60009840 tx=0x7f7e60009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80b03dbd628bdf2d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:56.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.853+0000 7f7e6e7fc700 1 -- 172.21.15.92:0/372381555 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e60004230 con 0x7f7e700b8510 2023-11-28T11:24:56.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.853+0000 7f7e6e7fc700 1 -- 172.21.15.92:0/372381555 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e60004390 con 0x7f7e700b8510 2023-11-28T11:24:56.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.853+0000 7f7e6e7fc700 1 -- 172.21.15.92:0/372381555 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e600036a0 con 0x7f7e700b8510 2023-11-28T11:24:56.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.853+0000 7f7e755ff700 1 -- 172.21.15.92:0/372381555 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e700b8510 msgr2=0x7f7e7006b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:56.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.853+0000 7f7e755ff700 1 --2- 172.21.15.92:0/372381555 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e700b8510 0x7f7e7006b970 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f7e60009840 tx=0x7f7e60009b50 comp rx=0 tx=0).stop 2023-11-28T11:24:56.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.853+0000 7f7e755ff700 1 -- 172.21.15.92:0/372381555 shutdown_connections 2023-11-28T11:24:56.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.853+0000 7f7e755ff700 1 --2- 172.21.15.92:0/372381555 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e700b8510 0x7f7e7006b970 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:56.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.853+0000 7f7e755ff700 1 -- 172.21.15.92:0/372381555 >> 172.21.15.92:0/372381555 conn(0x7f7e700719a0 msgr2=0x7f7e70071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:56.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.854+0000 7f7e755ff700 1 -- 172.21.15.92:0/372381555 shutdown_connections 2023-11-28T11:24:56.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.854+0000 7f7e755ff700 1 -- 172.21.15.92:0/372381555 wait complete. 2023-11-28T11:24:56.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.855+0000 7f7e755ff700 1 Processor -- start 2023-11-28T11:24:56.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.855+0000 7f7e755ff700 1 -- start start 2023-11-28T11:24:56.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.856+0000 7f7e755ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e700b8510 0x7f7e7015ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:56.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.856+0000 7f7e755ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e700cd460 0x7f7e701f22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:56.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.856+0000 7f7e755ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e700d0d60 0x7f7e701e3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:56.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.856+0000 7f7e755ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e701e5430 con 0x7f7e700cd460 2023-11-28T11:24:56.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.856+0000 7f7e755ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e701e4880 con 0x7f7e700d0d60 2023-11-28T11:24:56.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.856+0000 7f7e755ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e701f5610 con 0x7f7e700b8510 2023-11-28T11:24:56.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.856+0000 7f7e67fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e700cd460 0x7f7e701f22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:56.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.857+0000 7f7e67fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e700cd460 0x7f7e701f22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38700/0 (socket says 172.21.15.92:38700) 2023-11-28T11:24:56.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.857+0000 7f7e67fff700 1 -- 172.21.15.92:0/2272168408 learned_addr learned my addr 172.21.15.92:0/2272168408 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:56.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.857+0000 7f7e6effd700 1 --2- 172.21.15.92:0/2272168408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e700b8510 0x7f7e7015ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:56.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.857+0000 7f7e6f7fe700 1 --2- 172.21.15.92:0/2272168408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e700d0d60 0x7f7e701e3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:56.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.857+0000 7f7e67fff700 1 -- 172.21.15.92:0/2272168408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e700b8510 msgr2=0x7f7e7015ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:56.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.857+0000 7f7e67fff700 1 --2- 172.21.15.92:0/2272168408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e700b8510 0x7f7e7015ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:56.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.857+0000 7f7e67fff700 1 -- 172.21.15.92:0/2272168408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e700d0d60 msgr2=0x7f7e701e3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:56.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.857+0000 7f7e67fff700 1 --2- 172.21.15.92:0/2272168408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e700d0d60 0x7f7e701e3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:56.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.857+0000 7f7e67fff700 1 -- 172.21.15.92:0/2272168408 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e600094a0 con 0x7f7e700cd460 2023-11-28T11:24:56.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.857+0000 7f7e6f7fe700 1 --2- 172.21.15.92:0/2272168408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e700d0d60 0x7f7e701e3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:24:56.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.857+0000 7f7e67fff700 1 --2- 172.21.15.92:0/2272168408 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e700cd460 0x7f7e701f22c0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f7e5800c550 tx=0x7f7e5800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:56.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.857+0000 7f7e677fe700 1 -- 172.21.15.92:0/2272168408 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e58004c80 con 0x7f7e700cd460 2023-11-28T11:24:56.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.857+0000 7f7e677fe700 1 -- 172.21.15.92:0/2272168408 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e58004e00 con 0x7f7e700cd460 2023-11-28T11:24:56.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.857+0000 7f7e677fe700 1 -- 172.21.15.92:0/2272168408 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e58007310 con 0x7f7e700cd460 2023-11-28T11:24:56.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.857+0000 7f7e755ff700 1 -- 172.21.15.92:0/2272168408 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e700d29c0 con 0x7f7e700cd460 2023-11-28T11:24:56.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.858+0000 7f7e755ff700 1 -- 172.21.15.92:0/2272168408 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e700cfff0 con 0x7f7e700cd460 2023-11-28T11:24:56.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.860+0000 7f7e677fe700 1 -- 172.21.15.92:0/2272168408 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7e5801a020 con 0x7f7e700cd460 2023-11-28T11:24:56.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.861+0000 7f7e677fe700 1 --2- 172.21.15.92:0/2272168408 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7e500682f0 0x7f7e5006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:56.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.861+0000 7f7e677fe700 1 -- 172.21.15.92:0/2272168408 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6556+0+0 (secure 0 0 0) 0x7f7e580859d0 con 0x7f7e700cd460 2023-11-28T11:24:56.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.862+0000 7f7e6effd700 1 --2- 172.21.15.92:0/2272168408 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7e500682f0 0x7f7e5006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:56.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.862+0000 7f7e6effd700 1 --2- 172.21.15.92:0/2272168408 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7e500682f0 0x7f7e5006a7a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7e600090b0 tx=0x7f7e6001c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:56.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.863+0000 7f7e88f4db80 1 Processor -- start 2023-11-28T11:24:56.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.863+0000 7f7e88f4db80 1 -- start start 2023-11-28T11:24:56.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.864+0000 7f7e88f4db80 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f8fc076fa0 0x55f8fc030a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:56.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.864+0000 7f7e88f4db80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f8fc06db10 0x55f8fc077370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:56.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.864+0000 7f7e88f4db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8fc0596b0 0x55f8fc031bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:56.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.864+0000 7f7e88f4db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8fbfaee30 con 0x55f8fc0596b0 2023-11-28T11:24:56.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.864+0000 7f7e88f4db80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8fbf1b190 con 0x55f8fc06db10 2023-11-28T11:24:56.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.864+0000 7f7e88f4db80 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f8fbf89e00 con 0x55f8fc076fa0 2023-11-28T11:24:56.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.864+0000 7f7e6f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8fc0596b0 0x55f8fc031bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:56.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.864+0000 7f7e6f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8fc0596b0 0x55f8fc031bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38708/0 (socket says 172.21.15.92:38708) 2023-11-28T11:24:56.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.864+0000 7f7e6f7fe700 1 -- 172.21.15.92:0/1317732891 learned_addr learned my addr 172.21.15.92:0/1317732891 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:56.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.865+0000 7f7e6f7fe700 1 -- 172.21.15.92:0/1317732891 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f8fc076fa0 msgr2=0x55f8fc030a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:56.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.865+0000 7f7e6effd700 1 --2- 172.21.15.92:0/1317732891 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f8fc076fa0 0x55f8fc030a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:56.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.865+0000 7f7e67fff700 1 --2- 172.21.15.92:0/1317732891 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f8fc06db10 0x55f8fc077370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:56.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.865+0000 7f7e6f7fe700 1 --2- 172.21.15.92:0/1317732891 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f8fc076fa0 0x55f8fc030a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:56.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.865+0000 7f7e6f7fe700 1 -- 172.21.15.92:0/1317732891 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f8fc06db10 msgr2=0x55f8fc077370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:56.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.865+0000 7f7e6f7fe700 1 --2- 172.21.15.92:0/1317732891 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f8fc06db10 0x55f8fc077370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:56.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.865+0000 7f7e6f7fe700 1 -- 172.21.15.92:0/1317732891 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8fbd777e0 con 0x55f8fc0596b0 2023-11-28T11:24:56.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.865+0000 7f7e67fff700 1 --2- 172.21.15.92:0/1317732891 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f8fc06db10 0x55f8fc077370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:24:56.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.865+0000 7f7e6effd700 1 --2- 172.21.15.92:0/1317732891 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f8fc076fa0 0x55f8fc030a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:24:56.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.865+0000 7f7e6f7fe700 1 --2- 172.21.15.92:0/1317732891 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8fc0596b0 0x55f8fc031bb0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f7e6800c480 tx=0x7f7e6800c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6398fe8494790e35 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:56.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.865+0000 7f7e64ff9700 1 -- 172.21.15.92:0/1317732891 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e68011070 con 0x55f8fc0596b0 2023-11-28T11:24:56.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.865+0000 7f7e64ff9700 1 -- 172.21.15.92:0/1317732891 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e6800cb20 con 0x55f8fc0596b0 2023-11-28T11:24:56.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.865+0000 7f7e64ff9700 1 -- 172.21.15.92:0/1317732891 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e68016420 con 0x55f8fc0596b0 2023-11-28T11:24:56.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.866+0000 7f7e88f4db80 1 -- 172.21.15.92:0/1317732891 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8fc0596b0 msgr2=0x55f8fc031bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:56.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.866+0000 7f7e88f4db80 1 --2- 172.21.15.92:0/1317732891 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8fc0596b0 0x55f8fc031bb0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f7e6800c480 tx=0x7f7e6800c4b0 comp rx=0 tx=0).stop 2023-11-28T11:24:56.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.866+0000 7f7e88f4db80 1 -- 172.21.15.92:0/1317732891 shutdown_connections 2023-11-28T11:24:56.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.866+0000 7f7e88f4db80 1 --2- 172.21.15.92:0/1317732891 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8fc0596b0 0x55f8fc031bb0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:56.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.866+0000 7f7e88f4db80 1 --2- 172.21.15.92:0/1317732891 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f8fc076fa0 0x55f8fc030a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:56.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.866+0000 7f7e88f4db80 1 --2- 172.21.15.92:0/1317732891 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f8fc06db10 0x55f8fc077370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:56.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.866+0000 7f7e88f4db80 1 -- 172.21.15.92:0/1317732891 >> 172.21.15.92:0/1317732891 conn(0x55f8fc026d70 msgr2=0x55f8fc02ff70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:56.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.866+0000 7f7e88f4db80 1 -- 172.21.15.92:0/1317732891 shutdown_connections 2023-11-28T11:24:56.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.866+0000 7f7e88f4db80 1 -- 172.21.15.92:0/1317732891 wait complete. 2023-11-28T11:24:56.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.867+0000 7f7e88f4db80 1 -- start start 2023-11-28T11:24:56.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.867+0000 7f7e88f4db80 1 Processor -- start 2023-11-28T11:24:56.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.868+0000 7f7e88f4db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8fc0596b0 0x55f8fc07a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:56.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.868+0000 7f7e88f4db80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f8fc06db10 0x55f8fc0477c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:56.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.868+0000 7f7e88f4db80 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f8fc076fa0 0x55f8fc047d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:56.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.868+0000 7f7e88f4db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8fbf83170 con 0x55f8fc0596b0 2023-11-28T11:24:56.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.868+0000 7f7e88f4db80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8fbd783e0 con 0x55f8fc06db10 2023-11-28T11:24:56.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.868+0000 7f7e88f4db80 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f8fbf1b390 con 0x55f8fc076fa0 2023-11-28T11:24:56.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.868+0000 7f7e88f4db80 10 client.0 _ll_register_callbacks cb 0x55f8fbf21450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:24:56.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.868+0000 7f7e6effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8fc0596b0 0x55f8fc07a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:56.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.868+0000 7f7e6effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8fc0596b0 0x55f8fc07a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38712/0 (socket says 172.21.15.92:38712) 2023-11-28T11:24:56.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.868+0000 7f7e6effd700 1 -- 172.21.15.92:0/295097908 learned_addr learned my addr 172.21.15.92:0/295097908 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:56.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.868+0000 7f7e67fff700 1 --2- 172.21.15.92:0/295097908 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f8fc06db10 0x55f8fc0477c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:56.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.868+0000 7f7e6f7fe700 1 --2- 172.21.15.92:0/295097908 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f8fc076fa0 0x55f8fc047d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:56.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.869+0000 7f7e6effd700 1 -- 172.21.15.92:0/295097908 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f8fc076fa0 msgr2=0x55f8fc047d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:56.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.869+0000 7f7e6effd700 1 --2- 172.21.15.92:0/295097908 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f8fc076fa0 0x55f8fc047d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:56.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.869+0000 7f7e6effd700 1 -- 172.21.15.92:0/295097908 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f8fc06db10 msgr2=0x55f8fc0477c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:56.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.869+0000 7f7e6effd700 1 --2- 172.21.15.92:0/295097908 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f8fc06db10 0x55f8fc0477c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:56.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.869+0000 7f7e6effd700 1 -- 172.21.15.92:0/295097908 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e68009ca0 con 0x55f8fc0596b0 2023-11-28T11:24:56.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.869+0000 7f7e6effd700 1 --2- 172.21.15.92:0/295097908 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8fc0596b0 0x55f8fc07a410 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f7e60003ad0 tx=0x7f7e6000e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a99ae873cbbd070c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:56.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.869+0000 7f7e4dffb700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-11-28T11:24:56.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.869+0000 7f7e4dffb700 1 -- 172.21.15.92:0/295097908 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e60029050 con 0x55f8fc0596b0 2023-11-28T11:24:56.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.869+0000 7f7e4dffb700 1 -- 172.21.15.92:0/295097908 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 6556+0+0 (secure 0 0 0) 0x7f7e6001f070 con 0x55f8fc0596b0 2023-11-28T11:24:56.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.869+0000 7f7e88f4db80 1 -- 172.21.15.92:0/295097908 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f8fbf87080 con 0x55f8fc0596b0 2023-11-28T11:24:56.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.869+0000 7f7e4dffb700 1 -- 172.21.15.92:0/295097908 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e60011c00 con 0x55f8fc0596b0 2023-11-28T11:24:56.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.869+0000 7f7e88f4db80 10 client.6444 fetch_fsmap learned FSMap version 33 2023-11-28T11:24:56.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.869+0000 7f7e88f4db80 1 -- 172.21.15.92:0/295097908 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x55f8fbe640f0 con 0x55f8fc0596b0 2023-11-28T11:24:56.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.870+0000 7f7e4dffb700 1 -- 172.21.15.92:0/295097908 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f8fbf87080 con 0x55f8fc0596b0 2023-11-28T11:24:56.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.870+0000 7f7e4dffb700 1 -- 172.21.15.92:0/295097908 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 33) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f7e60037380 con 0x55f8fc0596b0 2023-11-28T11:24:56.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.870+0000 7f7e88f4db80 10 client.6444 fetch_fsmap finished waiting for FSMap version 33 2023-11-28T11:24:56.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.870+0000 7f7e88f4db80 10 client.6444 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2023-11-28T11:24:56.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.870+0000 7f7e88f4db80 20 client.6444 populate_metadata read hostname 'smithi092' 2023-11-28T11:24:56.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.870+0000 7f7e88f4db80 1 --2- 172.21.15.92:0/295097908 >> [v2:172.21.15.92:6834/3634521248,v1:172.21.15.92:6836/3634521248] conn(0x55f8fc06f5e0 0x55f8fc077b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:56.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.870+0000 7f7e88f4db80 4 client.6444 mds_command: new command op to 6213 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:24:56.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.870+0000 7f7e88f4db80 1 -- 172.21.15.92:0/295097908 --> [v2:172.21.15.92:6834/3634521248,v1:172.21.15.92:6836/3634521248] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f8fbdd2f40 con 0x55f8fc06f5e0 2023-11-28T11:24:56.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.870+0000 7f7e67fff700 1 --2- 172.21.15.92:0/295097908 >> [v2:172.21.15.92:6834/3634521248,v1:172.21.15.92:6836/3634521248] conn(0x55f8fc06f5e0 0x55f8fc077b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:56.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.870+0000 7f7e67fff700 1 --2- 172.21.15.92:0/295097908 >> [v2:172.21.15.92:6834/3634521248,v1:172.21.15.92:6836/3634521248] conn(0x55f8fc06f5e0 0x55f8fc077b30 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b20d321e3882dc7 server_cookie=484f7877119f082b in_seq=0 out_seq=0 2023-11-28T11:24:56.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.870+0000 7f7e4dffb700 10 client.6444 ms_handle_connect on v2:172.21.15.92:6834/3634521248 2023-11-28T11:24:56.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.871+0000 7f7e4dffb700 1 -- 172.21.15.92:0/295097908 <== mds.0 v2:172.21.15.92:6834/3634521248 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x55f8fbdd2f40 con 0x55f8fc06f5e0 2023-11-28T11:24:56.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.871+0000 7f7e4dffb700 10 client.6444 handle_command_reply: tid=0 2023-11-28T11:24:56.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.871+0000 7f7e88f4db80 1 client.6444 shutdown 2023-11-28T11:24:56.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.871+0000 7f7e88f4db80 1 -- 172.21.15.92:0/295097908 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8fc0596b0 msgr2=0x55f8fc07a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:56.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.871+0000 7f7e88f4db80 1 --2- 172.21.15.92:0/295097908 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8fc0596b0 0x55f8fc07a410 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f7e60003ad0 tx=0x7f7e6000e000 comp rx=0 tx=0).stop 2023-11-28T11:24:56.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.871+0000 7f7e88f4db80 1 -- 172.21.15.92:0/295097908 shutdown_connections 2023-11-28T11:24:56.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.871+0000 7f7e88f4db80 1 --2- 172.21.15.92:0/295097908 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f8fc076fa0 0x55f8fc047d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:56.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.871+0000 7f7e88f4db80 1 --2- 172.21.15.92:0/295097908 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8fc0596b0 0x55f8fc07a410 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:56.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.871+0000 7f7e88f4db80 1 --2- 172.21.15.92:0/295097908 >> [v2:172.21.15.92:6834/3634521248,v1:172.21.15.92:6836/3634521248] conn(0x55f8fc06f5e0 0x55f8fc077b30 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:56.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.871+0000 7f7e88f4db80 1 --2- 172.21.15.92:0/295097908 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f8fc06db10 0x55f8fc0477c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:56.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.871+0000 7f7e4dffb700 0 client.6444 ms_handle_reset on v2:172.21.15.92:6834/3634521248 2023-11-28T11:24:56.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.871+0000 7f7e88f4db80 1 -- 172.21.15.92:0/295097908 >> 172.21.15.92:0/295097908 conn(0x55f8fc026d70 msgr2=0x55f8fc02bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:56.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.871+0000 7f7e88f4db80 1 -- 172.21.15.92:0/295097908 shutdown_connections 2023-11-28T11:24:56.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.871+0000 7f7e88f4db80 1 -- 172.21.15.92:0/295097908 wait complete. 2023-11-28T11:24:56.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.872+0000 7f7e88f4db80 20 client.6444 trim_cache size 0 max 16384 2023-11-28T11:24:56.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.888+0000 7f7e88f4db80 1 Processor -- start 2023-11-28T11:24:56.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.888+0000 7f7e88f4db80 1 -- start start 2023-11-28T11:24:56.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.889+0000 7f7e88f4db80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f8fc0596b0 0x55f8fc072980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:56.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.889+0000 7f7e88f4db80 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f8fc06db10 0x55f8fc08f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:56.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.889+0000 7f7e88f4db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8fc076fa0 0x55f8fc0900e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:56.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.889+0000 7f7e88f4db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8fc0745f0 con 0x55f8fc076fa0 2023-11-28T11:24:56.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.889+0000 7f7e88f4db80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8fc077410 con 0x55f8fc0596b0 2023-11-28T11:24:56.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.889+0000 7f7e88f4db80 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f8fc024070 con 0x55f8fc06db10 2023-11-28T11:24:56.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.890+0000 7f7e6f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8fc076fa0 0x55f8fc0900e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:56.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.890+0000 7f7e6f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8fc076fa0 0x55f8fc0900e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38726/0 (socket says 172.21.15.92:38726) 2023-11-28T11:24:56.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.890+0000 7f7e6f7fe700 1 -- 172.21.15.92:0/2786074538 learned_addr learned my addr 172.21.15.92:0/2786074538 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:56.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.890+0000 7f7e67fff700 1 --2- 172.21.15.92:0/2786074538 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f8fc06db10 0x55f8fc08f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:56.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.890+0000 7f7e6effd700 1 --2- 172.21.15.92:0/2786074538 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f8fc0596b0 0x55f8fc072980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:56.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.890+0000 7f7e6f7fe700 1 -- 172.21.15.92:0/2786074538 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f8fc06db10 msgr2=0x55f8fc08f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:56.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.890+0000 7f7e6f7fe700 1 --2- 172.21.15.92:0/2786074538 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f8fc06db10 0x55f8fc08f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:56.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.890+0000 7f7e6f7fe700 1 -- 172.21.15.92:0/2786074538 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f8fc0596b0 msgr2=0x55f8fc072980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:56.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.890+0000 7f7e6f7fe700 1 --2- 172.21.15.92:0/2786074538 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f8fc0596b0 0x55f8fc072980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:56.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.890+0000 7f7e6f7fe700 1 -- 172.21.15.92:0/2786074538 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8fbf4a330 con 0x55f8fc076fa0 2023-11-28T11:24:56.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.890+0000 7f7e6f7fe700 1 --2- 172.21.15.92:0/2786074538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8fc076fa0 0x55f8fc0900e0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f7e6800c450 tx=0x7f7e6800e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f1f4b6f42a7a75f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:56.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.890+0000 7f7e657fa700 1 -- 172.21.15.92:0/2786074538 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6800cc90 con 0x55f8fc076fa0 2023-11-28T11:24:56.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.890+0000 7f7e657fa700 1 -- 172.21.15.92:0/2786074538 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e6800cdf0 con 0x55f8fc076fa0 2023-11-28T11:24:56.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.891+0000 7f7e657fa700 1 -- 172.21.15.92:0/2786074538 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e68020680 con 0x55f8fc076fa0 2023-11-28T11:24:56.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.891+0000 7f7e88f4db80 1 -- 172.21.15.92:0/2786074538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8fc076fa0 msgr2=0x55f8fc0900e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:56.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.891+0000 7f7e88f4db80 1 --2- 172.21.15.92:0/2786074538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8fc076fa0 0x55f8fc0900e0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f7e6800c450 tx=0x7f7e6800e000 comp rx=0 tx=0).stop 2023-11-28T11:24:56.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.891+0000 7f7e88f4db80 1 -- 172.21.15.92:0/2786074538 shutdown_connections 2023-11-28T11:24:56.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.891+0000 7f7e88f4db80 1 --2- 172.21.15.92:0/2786074538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8fc076fa0 0x55f8fc0900e0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:56.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.891+0000 7f7e88f4db80 1 --2- 172.21.15.92:0/2786074538 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f8fc0596b0 0x55f8fc072980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:56.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.891+0000 7f7e88f4db80 1 --2- 172.21.15.92:0/2786074538 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f8fc06db10 0x55f8fc08f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:56.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.891+0000 7f7e88f4db80 1 -- 172.21.15.92:0/2786074538 >> 172.21.15.92:0/2786074538 conn(0x55f8fc026d70 msgr2=0x55f8fc02e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:56.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.891+0000 7f7e88f4db80 1 -- 172.21.15.92:0/2786074538 shutdown_connections 2023-11-28T11:24:56.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.891+0000 7f7e88f4db80 1 -- 172.21.15.92:0/2786074538 wait complete. 2023-11-28T11:24:56.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.892+0000 7f7e88f4db80 1 -- start start 2023-11-28T11:24:56.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.892+0000 7f7e88f4db80 1 Processor -- start 2023-11-28T11:24:56.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.893+0000 7f7e88f4db80 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f8fc0596b0 0x55f8fc0d5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:56.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.893+0000 7f7e88f4db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8fc06db10 0x55f8fc094cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:56.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.893+0000 7f7e88f4db80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f8fc076fa0 0x55f8fc075cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:56.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.893+0000 7f7e88f4db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8fc06d960 con 0x55f8fc06db10 2023-11-28T11:24:56.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.893+0000 7f7e88f4db80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8fc0957b0 con 0x55f8fc076fa0 2023-11-28T11:24:56.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.893+0000 7f7e88f4db80 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f8fc06d430 con 0x55f8fc0596b0 2023-11-28T11:24:56.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.893+0000 7f7e88f4db80 10 client.0 _ll_register_callbacks cb 0x55f8fbf21450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:24:56.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.893+0000 7f7e67fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8fc06db10 0x55f8fc094cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:56.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.893+0000 7f7e67fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8fc06db10 0x55f8fc094cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38734/0 (socket says 172.21.15.92:38734) 2023-11-28T11:24:56.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.893+0000 7f7e67fff700 1 -- 172.21.15.92:0/1359807694 learned_addr learned my addr 172.21.15.92:0/1359807694 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:56.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.894+0000 7f7e6effd700 1 --2- 172.21.15.92:0/1359807694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f8fc0596b0 0x55f8fc0d5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:56.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.894+0000 7f7e6f7fe700 1 --2- 172.21.15.92:0/1359807694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f8fc076fa0 0x55f8fc075cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:56.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.894+0000 7f7e67fff700 1 -- 172.21.15.92:0/1359807694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f8fc0596b0 msgr2=0x55f8fc0d5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:56.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.894+0000 7f7e67fff700 1 --2- 172.21.15.92:0/1359807694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f8fc0596b0 0x55f8fc0d5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:56.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.894+0000 7f7e67fff700 1 -- 172.21.15.92:0/1359807694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f8fc076fa0 msgr2=0x55f8fc075cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:56.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.894+0000 7f7e67fff700 1 --2- 172.21.15.92:0/1359807694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f8fc076fa0 0x55f8fc075cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:56.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.894+0000 7f7e67fff700 1 -- 172.21.15.92:0/1359807694 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e6801a040 con 0x55f8fc06db10 2023-11-28T11:24:56.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.894+0000 7f7e67fff700 1 --2- 172.21.15.92:0/1359807694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8fc06db10 0x55f8fc094cc0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f7e580521a0 tx=0x7f7e58059040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79351a1e35b8ece4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:56.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.894+0000 7f7e4dffb700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-11-28T11:24:56.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.894+0000 7f7e4dffb700 1 -- 172.21.15.92:0/1359807694 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e58007740 con 0x55f8fc06db10 2023-11-28T11:24:56.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.894+0000 7f7e4dffb700 1 -- 172.21.15.92:0/1359807694 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 6556+0+0 (secure 0 0 0) 0x7f7e58055390 con 0x55f8fc06db10 2023-11-28T11:24:56.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.894+0000 7f7e88f4db80 1 -- 172.21.15.92:0/1359807694 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7e6800cb20 con 0x55f8fc06db10 2023-11-28T11:24:56.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.894+0000 7f7e4dffb700 1 -- 172.21.15.92:0/1359807694 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e580690a0 con 0x55f8fc06db10 2023-11-28T11:24:56.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.894+0000 7f7e88f4db80 10 client.6450 fetch_fsmap learned FSMap version 33 2023-11-28T11:24:56.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.894+0000 7f7e88f4db80 1 -- 172.21.15.92:0/1359807694 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x55f8fbdd8220 con 0x55f8fc06db10 2023-11-28T11:24:56.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.895+0000 7f7e4dffb700 1 -- 172.21.15.92:0/1359807694 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e6800cb20 con 0x55f8fc06db10 2023-11-28T11:24:56.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.895+0000 7f7e4dffb700 1 -- 172.21.15.92:0/1359807694 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 33) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f7e58070020 con 0x55f8fc06db10 2023-11-28T11:24:56.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.895+0000 7f7e88f4db80 10 client.6450 fetch_fsmap finished waiting for FSMap version 33 2023-11-28T11:24:56.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.895+0000 7f7e88f4db80 10 client.6450 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2023-11-28T11:24:56.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.895+0000 7f7e88f4db80 20 client.6450 populate_metadata read hostname 'smithi092' 2023-11-28T11:24:56.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.895+0000 7f7e88f4db80 1 --2- 172.21.15.92:0/1359807694 >> [v2:172.21.15.92:6834/3634521248,v1:172.21.15.92:6836/3634521248] conn(0x55f8fc047700 0x55f8fc045d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:56.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.895+0000 7f7e88f4db80 4 client.6450 mds_command: new command op to 6213 tid=0[{"prefix": "scrub status"}] 2023-11-28T11:24:56.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.895+0000 7f7e88f4db80 1 -- 172.21.15.92:0/1359807694 --> [v2:172.21.15.92:6834/3634521248,v1:172.21.15.92:6836/3634521248] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f8fbe08090 con 0x55f8fc047700 2023-11-28T11:24:56.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.895+0000 7f7e6effd700 1 --2- 172.21.15.92:0/1359807694 >> [v2:172.21.15.92:6834/3634521248,v1:172.21.15.92:6836/3634521248] conn(0x55f8fc047700 0x55f8fc045d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:56.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.895+0000 7f7e4dffb700 10 client.6450 ms_handle_connect on v2:172.21.15.92:6834/3634521248 2023-11-28T11:24:56.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.895+0000 7f7e6effd700 1 --2- 172.21.15.92:0/1359807694 >> [v2:172.21.15.92:6834/3634521248,v1:172.21.15.92:6836/3634521248] conn(0x55f8fc047700 0x55f8fc045d80 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=961366214ee10fab server_cookie=a570c82a890a52ad in_seq=0 out_seq=0 2023-11-28T11:24:56.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.896+0000 7f7e4dffb700 1 -- 172.21.15.92:0/1359807694 <== mds.0 v2:172.21.15.92:6834/3634521248 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55f8fbe08090 con 0x55f8fc047700 2023-11-28T11:24:56.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.896+0000 7f7e4dffb700 10 client.6450 handle_command_reply: tid=0 2023-11-28T11:24:56.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.896+0000 7f7e88f4db80 1 client.6450 shutdown 2023-11-28T11:24:56.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.896+0000 7f7e88f4db80 1 -- 172.21.15.92:0/1359807694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8fc06db10 msgr2=0x55f8fc094cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:56.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.896+0000 7f7e88f4db80 1 --2- 172.21.15.92:0/1359807694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8fc06db10 0x55f8fc094cc0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f7e580521a0 tx=0x7f7e58059040 comp rx=0 tx=0).stop 2023-11-28T11:24:56.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.896+0000 7f7e88f4db80 1 -- 172.21.15.92:0/1359807694 shutdown_connections 2023-11-28T11:24:56.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.896+0000 7f7e88f4db80 1 --2- 172.21.15.92:0/1359807694 >> [v2:172.21.15.92:6834/3634521248,v1:172.21.15.92:6836/3634521248] conn(0x55f8fc047700 0x55f8fc045d80 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:56.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.896+0000 7f7e88f4db80 1 --2- 172.21.15.92:0/1359807694 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55f8fc076fa0 0x55f8fc075cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:56.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.896+0000 7f7e88f4db80 1 --2- 172.21.15.92:0/1359807694 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55f8fc0596b0 0x55f8fc0d5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:56.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.896+0000 7f7e4dffb700 0 client.6450 ms_handle_reset on v2:172.21.15.92:6834/3634521248 2023-11-28T11:24:56.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.896+0000 7f7e88f4db80 1 --2- 172.21.15.92:0/1359807694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8fc06db10 0x55f8fc094cc0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:56.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.896+0000 7f7e88f4db80 1 -- 172.21.15.92:0/1359807694 >> 172.21.15.92:0/1359807694 conn(0x55f8fc026d70 msgr2=0x55f8fc02d660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:56.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.896+0000 7f7e88f4db80 1 -- 172.21.15.92:0/1359807694 shutdown_connections 2023-11-28T11:24:56.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.896+0000 7f7e88f4db80 1 -- 172.21.15.92:0/1359807694 wait complete. 2023-11-28T11:24:56.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.896+0000 7f7e88f4db80 20 client.6450 trim_cache size 0 max 16384 2023-11-28T11:24:56.904 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-28T11:24:56.904 INFO:teuthology.orchestra.run.smithi092.stdout: "status": "no active scrubs running", 2023-11-28T11:24:56.904 INFO:teuthology.orchestra.run.smithi092.stdout: "scrubs": {} 2023-11-28T11:24:56.904 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-28T11:24:56.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.897+0000 7f7e755ff700 1 -- 172.21.15.92:0/2272168408 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7e500682f0 msgr2=0x7f7e5006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:56.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.897+0000 7f7e755ff700 1 --2- 172.21.15.92:0/2272168408 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7e500682f0 0x7f7e5006a7a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7e600090b0 tx=0x7f7e6001c040 comp rx=0 tx=0).stop 2023-11-28T11:24:56.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.897+0000 7f7e755ff700 1 -- 172.21.15.92:0/2272168408 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e700cd460 msgr2=0x7f7e701f22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:56.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.897+0000 7f7e755ff700 1 --2- 172.21.15.92:0/2272168408 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e700cd460 0x7f7e701f22c0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f7e5800c550 tx=0x7f7e5800c860 comp rx=0 tx=0).stop 2023-11-28T11:24:56.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.897+0000 7f7e755ff700 1 -- 172.21.15.92:0/2272168408 shutdown_connections 2023-11-28T11:24:56.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.897+0000 7f7e755ff700 1 --2- 172.21.15.92:0/2272168408 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7e700d0d60 0x7f7e701e3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:56.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.897+0000 7f7e755ff700 1 --2- 172.21.15.92:0/2272168408 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7e700b8510 0x7f7e7015ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:56.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.897+0000 7f7e755ff700 1 --2- 172.21.15.92:0/2272168408 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7e500682f0 0x7f7e5006a7a0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:56.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.897+0000 7f7e755ff700 1 --2- 172.21.15.92:0/2272168408 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e700cd460 0x7f7e701f22c0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:56.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.897+0000 7f7e755ff700 1 -- 172.21.15.92:0/2272168408 >> 172.21.15.92:0/2272168408 conn(0x7f7e700719a0 msgr2=0x7f7e700b6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:56.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.897+0000 7f7e755ff700 1 -- 172.21.15.92:0/2272168408 shutdown_connections 2023-11-28T11:24:56.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:56.897+0000 7f7e755ff700 1 -- 172.21.15.92:0/2272168408 wait complete. 2023-11-28T11:24:56.914 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-28T11:24:56.915 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:24:57.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.111+0000 7f28aafff700 1 Processor -- start 2023-11-28T11:24:57.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.112+0000 7f28aafff700 1 -- start start 2023-11-28T11:24:57.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.113+0000 7f28aafff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f28a40cd460 0x7f28a406c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:57.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.113+0000 7f28aafff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28a40b88e0 con 0x7f28a40d0d60 2023-11-28T11:24:57.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.113+0000 7f28aafff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28a40cd8f0 con 0x7f28a40b8510 2023-11-28T11:24:57.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.113+0000 7f28aafff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28a40cdbd0 con 0x7f28a40cd460 2023-11-28T11:24:57.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.113+0000 7f28a3fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f28a40cd460 0x7f28a406c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:57.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.113+0000 7f28a959c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f28a40d0d60 0x7f28a40d1200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53718/0 (socket says 172.21.15.92:53718) 2023-11-28T11:24:57.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.113+0000 7f28a959c700 1 -- 172.21.15.92:0/2587605042 learned_addr learned my addr 172.21.15.92:0/2587605042 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:24:57.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.114+0000 7f28a3fff700 1 -- 172.21.15.92:0/2587605042 >> v1:172.21.15.169:6789/0 conn(0x7f28a40b8510 legacy=0x7f28a406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:57.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.114+0000 7f28a3fff700 1 -- 172.21.15.92:0/2587605042 >> v1:172.21.15.92:6789/0 conn(0x7f28a40d0d60 legacy=0x7f28a40d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:57.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.114+0000 7f28a3fff700 1 -- 172.21.15.92:0/2587605042 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a40cfd10 con 0x7f28a40cd460 2023-11-28T11:24:57.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.114+0000 7f28a37fe700 1 -- 172.21.15.92:0/2587605042 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3010428083 0 0) 0x7f28a40b88e0 con 0x7f28a40d0d60 2023-11-28T11:24:57.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.114+0000 7f28a3fff700 1 --2- 172.21.15.92:0/2587605042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28a40cd460 0x7f28a406c080 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f2890009840 tx=0x7f2890009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85501a5686a748a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:57.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.114+0000 7f28a37fe700 1 -- 172.21.15.92:0/2587605042 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2890011830 con 0x7f28a40cd460 2023-11-28T11:24:57.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.114+0000 7f28a37fe700 1 -- 172.21.15.92:0/2587605042 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2890011990 con 0x7f28a40cd460 2023-11-28T11:24:57.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.114+0000 7f28a37fe700 1 -- 172.21.15.92:0/2587605042 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2890011d20 con 0x7f28a40cd460 2023-11-28T11:24:57.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.115+0000 7f28aafff700 1 -- 172.21.15.92:0/2587605042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28a40cd460 msgr2=0x7f28a406c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:57.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.115+0000 7f28aafff700 1 --2- 172.21.15.92:0/2587605042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28a40cd460 0x7f28a406c080 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f2890009840 tx=0x7f2890009b50 comp rx=0 tx=0).stop 2023-11-28T11:24:57.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.115+0000 7f28aafff700 1 -- 172.21.15.92:0/2587605042 shutdown_connections 2023-11-28T11:24:57.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.115+0000 7f28aafff700 1 --2- 172.21.15.92:0/2587605042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28a40cd460 0x7f28a406c080 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:57.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.115+0000 7f28aafff700 1 -- 172.21.15.92:0/2587605042 >> 172.21.15.92:0/2587605042 conn(0x7f28a40719a0 msgr2=0x7f28a4071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:57.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.115+0000 7f28aafff700 1 -- 172.21.15.92:0/2587605042 shutdown_connections 2023-11-28T11:24:57.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.115+0000 7f28aafff700 1 -- 172.21.15.92:0/2587605042 wait complete. 2023-11-28T11:24:57.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.117+0000 7f28aafff700 1 Processor -- start 2023-11-28T11:24:57.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.117+0000 7f28aafff700 1 -- start start 2023-11-28T11:24:57.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.118+0000 7f28aafff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28a40b8510 0x7f28a415b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:57.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.118+0000 7f28aafff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28a40cd460 0x7f28a40b7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:57.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.118+0000 7f28aafff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28a40d0d60 0x7f28a40b77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:57.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.118+0000 7f28aafff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a41e6d70 con 0x7f28a40cd460 2023-11-28T11:24:57.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.118+0000 7f28aafff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a41f2080 con 0x7f28a40b8510 2023-11-28T11:24:57.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.118+0000 7f28aafff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28a415ba00 con 0x7f28a40d0d60 2023-11-28T11:24:57.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.118+0000 7f28a3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28a40cd460 0x7f28a40b7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:57.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.118+0000 7f28a3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28a40cd460 0x7f28a40b7280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38746/0 (socket says 172.21.15.92:38746) 2023-11-28T11:24:57.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.118+0000 7f28a3fff700 1 -- 172.21.15.92:0/2544049212 learned_addr learned my addr 172.21.15.92:0/2544049212 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:57.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.118+0000 7f28a8d9b700 1 --2- 172.21.15.92:0/2544049212 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28a40b8510 0x7f28a415b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:57.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.118+0000 7f28a959c700 1 --2- 172.21.15.92:0/2544049212 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28a40d0d60 0x7f28a40b77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:57.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.119+0000 7f28a3fff700 1 -- 172.21.15.92:0/2544049212 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28a40d0d60 msgr2=0x7f28a40b77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:57.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.119+0000 7f28a3fff700 1 --2- 172.21.15.92:0/2544049212 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28a40d0d60 0x7f28a40b77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:57.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.119+0000 7f28a3fff700 1 -- 172.21.15.92:0/2544049212 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28a40b8510 msgr2=0x7f28a415b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:57.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.119+0000 7f28a3fff700 1 --2- 172.21.15.92:0/2544049212 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28a40b8510 0x7f28a415b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:57.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.119+0000 7f28a3fff700 1 -- 172.21.15.92:0/2544049212 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28900094a0 con 0x7f28a40cd460 2023-11-28T11:24:57.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.119+0000 7f28a3fff700 1 --2- 172.21.15.92:0/2544049212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28a40cd460 0x7f28a40b7280 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f28a4072610 tx=0x7f2890004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:57.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.119+0000 7f28a17fa700 1 -- 172.21.15.92:0/2544049212 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289001a970 con 0x7f28a40cd460 2023-11-28T11:24:57.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.119+0000 7f28a17fa700 1 -- 172.21.15.92:0/2544049212 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f289001aad0 con 0x7f28a40cd460 2023-11-28T11:24:57.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.119+0000 7f28a17fa700 1 -- 172.21.15.92:0/2544049212 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289001ae40 con 0x7f28a40cd460 2023-11-28T11:24:57.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.119+0000 7f28aafff700 1 -- 172.21.15.92:0/2544049212 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28a40d29c0 con 0x7f28a40cd460 2023-11-28T11:24:57.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.119+0000 7f28aafff700 1 -- 172.21.15.92:0/2544049212 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28a40cfff0 con 0x7f28a40cd460 2023-11-28T11:24:57.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.120+0000 7f28aafff700 1 -- 172.21.15.92:0/2544049212 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28a40d2b50 con 0x7f28a40cd460 2023-11-28T11:24:57.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.121+0000 7f28a17fa700 1 -- 172.21.15.92:0/2544049212 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f289000f050 con 0x7f28a40cd460 2023-11-28T11:24:57.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.122+0000 7f28a17fa700 1 --2- 172.21.15.92:0/2544049212 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f28880682f0 0x7f288806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:57.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.122+0000 7f28a17fa700 1 -- 172.21.15.92:0/2544049212 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6556+0+0 (secure 0 0 0) 0x7f2890087370 con 0x7f28a40cd460 2023-11-28T11:24:57.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.122+0000 7f28a8d9b700 1 --2- 172.21.15.92:0/2544049212 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f28880682f0 0x7f288806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:57.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.123+0000 7f28a8d9b700 1 --2- 172.21.15.92:0/2544049212 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f28880682f0 0x7f288806a7a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f2898007fd0 tx=0x7f2898009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:57.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.126+0000 7f28a17fa700 1 -- 172.21.15.92:0/2544049212 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f28900538f0 con 0x7f28a40cd460 2023-11-28T11:24:57.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.361+0000 7f28aafff700 1 -- 172.21.15.92:0/2544049212 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f28a4067880 con 0x7f28a40cd460 2023-11-28T11:24:57.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.362+0000 7f28a17fa700 1 -- 172.21.15.92:0/2544049212 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4656 (secure 0 0 0) 0x7f2890056f10 con 0x7f28a40cd460 2023-11-28T11:24:57.363 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:24:57.363 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5525,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2067956596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2067956596},{"type":"v1","addr":"172.21.15.169:6835","nonce":2067956596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5971,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2367049160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2367049160},{"type":"v1","addr":"172.21.15.169:6834","nonce":2367049160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5977,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1424858585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":1424858585},{"type":"v1","addr":"172.21.15.92:6837","nonce":1424858585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:24:16.257826+0000","modified":"2023-11-28T11:24:32.577560+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6213},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6213":{"gid":6213,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6836/3634521248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3634521248},{"type":"v1","addr":"172.21.15.92:6836","nonce":3634521248}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T11:24:57.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.364+0000 7f28aafff700 1 -- 172.21.15.92:0/2544049212 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f28880682f0 msgr2=0x7f288806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:57.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.364+0000 7f28aafff700 1 --2- 172.21.15.92:0/2544049212 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f28880682f0 0x7f288806a7a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f2898007fd0 tx=0x7f2898009040 comp rx=0 tx=0).stop 2023-11-28T11:24:57.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.364+0000 7f28aafff700 1 -- 172.21.15.92:0/2544049212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28a40cd460 msgr2=0x7f28a40b7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:57.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.364+0000 7f28aafff700 1 --2- 172.21.15.92:0/2544049212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28a40cd460 0x7f28a40b7280 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f28a4072610 tx=0x7f2890004750 comp rx=0 tx=0).stop 2023-11-28T11:24:57.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.364+0000 7f28aafff700 1 -- 172.21.15.92:0/2544049212 shutdown_connections 2023-11-28T11:24:57.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.364+0000 7f28aafff700 1 --2- 172.21.15.92:0/2544049212 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f28a40d0d60 0x7f28a40b77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:57.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.364+0000 7f28aafff700 1 --2- 172.21.15.92:0/2544049212 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f28a40b8510 0x7f28a415b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:57.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.364+0000 7f28aafff700 1 --2- 172.21.15.92:0/2544049212 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f28880682f0 0x7f288806a7a0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:57.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.364+0000 7f28aafff700 1 --2- 172.21.15.92:0/2544049212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28a40cd460 0x7f28a40b7280 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:57.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.364+0000 7f28aafff700 1 -- 172.21.15.92:0/2544049212 >> 172.21.15.92:0/2544049212 conn(0x7f28a40719a0 msgr2=0x7f28a40cbef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:57.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.364+0000 7f28aafff700 1 -- 172.21.15.92:0/2544049212 shutdown_connections 2023-11-28T11:24:57.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.364+0000 7f28aafff700 1 -- 172.21.15.92:0/2544049212 wait complete. 2023-11-28T11:24:57.366 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 33 2023-11-28T11:24:57.384 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2023-11-28T11:24:57.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.581+0000 7f0d11bff700 1 Processor -- start 2023-11-28T11:24:57.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.581+0000 7f0d11bff700 1 -- start start 2023-11-28T11:24:57.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.582+0000 7f0d11bff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0d0c0b8510 0x7f0d0c06b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:57.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.582+0000 7f0d11bff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d0c0cd460 0x7f0d0c06beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:57.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.583+0000 7f0d11bff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d0c0b88e0 con 0x7f0d0c06c3f0 2023-11-28T11:24:57.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.583+0000 7f0d11bff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d0c0cdbd0 con 0x7f0d0c0b8510 2023-11-28T11:24:57.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.583+0000 7f0d11bff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0c0d02d0 con 0x7f0d0c0cd460 2023-11-28T11:24:57.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.583+0000 7f0d0b7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0d0c0b8510 0x7f0d0c06b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:57.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.583+0000 7f0d0b7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0d0c0b8510 0x7f0d0c06b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38756/0 (socket says 172.21.15.92:38756) 2023-11-28T11:24:57.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.583+0000 7f0d0b7fe700 1 -- 172.21.15.92:0/3011649973 learned_addr learned my addr 172.21.15.92:0/3011649973 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:57.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.583+0000 7f0d0affd700 1 --2- 172.21.15.92:0/3011649973 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d0c0cd460 0x7f0d0c06beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:57.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.583+0000 7f0d0b7fe700 1 -- 172.21.15.92:0/3011649973 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d0c0cd460 msgr2=0x7f0d0c06beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:57.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.583+0000 7f0d0b7fe700 1 --2- 172.21.15.92:0/3011649973 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d0c0cd460 0x7f0d0c06beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:57.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.583+0000 7f0d0b7fe700 1 -- 172.21.15.92:0/3011649973 >> v1:172.21.15.169:6789/0 conn(0x7f0d0c06c3f0 legacy=0x7f0d0c15cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:57.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.583+0000 7f0d0b7fe700 1 -- 172.21.15.92:0/3011649973 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d0c0cd8f0 con 0x7f0d0c0b8510 2023-11-28T11:24:57.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.583+0000 7f0d0affd700 1 --2- 172.21.15.92:0/3011649973 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d0c0cd460 0x7f0d0c06beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:24:57.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.584+0000 7f0d0b7fe700 1 --2- 172.21.15.92:0/3011649973 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d0c0b8510 0x7f0d0c06b970 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc009920 tx=0x7f0cfc009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bea77782ceaa7a0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:57.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.584+0000 7f0d0a7fc700 1 -- 172.21.15.92:0/3011649973 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc004230 con 0x7f0d0c0b8510 2023-11-28T11:24:57.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.584+0000 7f0d0a7fc700 1 -- 172.21.15.92:0/3011649973 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cfc004390 con 0x7f0d0c0b8510 2023-11-28T11:24:57.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.584+0000 7f0d0a7fc700 1 -- 172.21.15.92:0/3011649973 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc009e70 con 0x7f0d0c0b8510 2023-11-28T11:24:57.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.584+0000 7f0d11bff700 1 -- 172.21.15.92:0/3011649973 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d0c0b8510 msgr2=0x7f0d0c06b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:57.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.584+0000 7f0d11bff700 1 --2- 172.21.15.92:0/3011649973 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d0c0b8510 0x7f0d0c06b970 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc009920 tx=0x7f0cfc009c30 comp rx=0 tx=0).stop 2023-11-28T11:24:57.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.584+0000 7f0d11bff700 1 -- 172.21.15.92:0/3011649973 shutdown_connections 2023-11-28T11:24:57.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.584+0000 7f0d11bff700 1 --2- 172.21.15.92:0/3011649973 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d0c0b8510 0x7f0d0c06b970 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:57.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.584+0000 7f0d11bff700 1 --2- 172.21.15.92:0/3011649973 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d0c0cd460 0x7f0d0c06beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:57.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.584+0000 7f0d11bff700 1 -- 172.21.15.92:0/3011649973 >> 172.21.15.92:0/3011649973 conn(0x7f0d0c0719a0 msgr2=0x7f0d0c071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:57.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.584+0000 7f0d11bff700 1 -- 172.21.15.92:0/3011649973 shutdown_connections 2023-11-28T11:24:57.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.585+0000 7f0d11bff700 1 -- 172.21.15.92:0/3011649973 wait complete. 2023-11-28T11:24:57.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.586+0000 7f0d11bff700 1 Processor -- start 2023-11-28T11:24:57.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.586+0000 7f0d11bff700 1 -- start start 2023-11-28T11:24:57.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.587+0000 7f0d11bff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d0c06c3f0 0x7f0d0c0c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:57.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.587+0000 7f0d11bff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d0c0b8510 0x7f0d0c0c96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:57.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.587+0000 7f0d11bff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d0c0cd460 0x7f0d0c0c73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:57.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.587+0000 7f0d11bff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0c0c7920 con 0x7f0d0c0cd460 2023-11-28T11:24:57.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.587+0000 7f0d11bff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0c0c8650 con 0x7f0d0c0b8510 2023-11-28T11:24:57.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.587+0000 7f0d11bff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0c0c8930 con 0x7f0d0c06c3f0 2023-11-28T11:24:57.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.588+0000 7f0d0bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d0c0cd460 0x7f0d0c0c73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:57.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.588+0000 7f0d0bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d0c0cd460 0x7f0d0c0c73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38772/0 (socket says 172.21.15.92:38772) 2023-11-28T11:24:57.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.588+0000 7f0d0bfff700 1 -- 172.21.15.92:0/659716465 learned_addr learned my addr 172.21.15.92:0/659716465 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:57.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.588+0000 7f0d0b7fe700 1 --2- 172.21.15.92:0/659716465 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d0c06c3f0 0x7f0d0c0c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:57.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.588+0000 7f0d0affd700 1 --2- 172.21.15.92:0/659716465 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d0c0b8510 0x7f0d0c0c96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:57.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.588+0000 7f0d0bfff700 1 -- 172.21.15.92:0/659716465 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d0c06c3f0 msgr2=0x7f0d0c0c7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:57.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.588+0000 7f0d0bfff700 1 --2- 172.21.15.92:0/659716465 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d0c06c3f0 0x7f0d0c0c7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:57.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.588+0000 7f0d0bfff700 1 -- 172.21.15.92:0/659716465 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d0c0b8510 msgr2=0x7f0d0c0c96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:57.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.588+0000 7f0d0bfff700 1 --2- 172.21.15.92:0/659716465 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d0c0b8510 0x7f0d0c0c96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:57.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.588+0000 7f0d0bfff700 1 -- 172.21.15.92:0/659716465 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cfc009550 con 0x7f0d0c0cd460 2023-11-28T11:24:57.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.588+0000 7f0d0affd700 1 --2- 172.21.15.92:0/659716465 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d0c0b8510 0x7f0d0c0c96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:24:57.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.588+0000 7f0d0bfff700 1 --2- 172.21.15.92:0/659716465 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d0c0cd460 0x7f0d0c0c73e0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f0d0400cc10 tx=0x7f0d0400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:57.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.588+0000 7f0cf3fff700 1 -- 172.21.15.92:0/659716465 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0400abb0 con 0x7f0d0c0cd460 2023-11-28T11:24:57.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.588+0000 7f0cf3fff700 1 -- 172.21.15.92:0/659716465 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d0400ad10 con 0x7f0d0c0cd460 2023-11-28T11:24:57.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.588+0000 7f0cf3fff700 1 -- 172.21.15.92:0/659716465 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d04012660 con 0x7f0d0c0cd460 2023-11-28T11:24:57.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.588+0000 7f0d11bff700 1 -- 172.21.15.92:0/659716465 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d0c0cfff0 con 0x7f0d0c0cd460 2023-11-28T11:24:57.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.589+0000 7f0d11bff700 1 -- 172.21.15.92:0/659716465 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d0c0cfd10 con 0x7f0d0c0cd460 2023-11-28T11:24:57.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.591+0000 7f0cf3fff700 1 -- 172.21.15.92:0/659716465 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0d04022070 con 0x7f0d0c0cd460 2023-11-28T11:24:57.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.591+0000 7f0cf3fff700 1 --2- 172.21.15.92:0/659716465 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f0cf80682f0 0x7f0cf806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:57.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.592+0000 7f0cf3fff700 1 -- 172.21.15.92:0/659716465 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6556+0+0 (secure 0 0 0) 0x7f0d040865b0 con 0x7f0d0c0cd460 2023-11-28T11:24:57.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.592+0000 7f0d0b7fe700 1 --2- 172.21.15.92:0/659716465 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f0cf80682f0 0x7f0cf806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:57.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.592+0000 7f0d0b7fe700 1 --2- 172.21.15.92:0/659716465 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f0cf80682f0 0x7f0cf806a7a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc009130 tx=0x7f0cfc008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:57.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.594+0000 7f0d2551cb80 1 Processor -- start 2023-11-28T11:24:57.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.594+0000 7f0d2551cb80 1 -- start start 2023-11-28T11:24:57.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.595+0000 7f0d2551cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c11e9a7ff0 0x55c11e961a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:57.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.595+0000 7f0d2551cb80 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c11e99eb60 0x55c11e9a83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:57.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.595+0000 7f0d2551cb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c11e98a700 0x55c11e962c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:57.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.595+0000 7f0d2551cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c11e8dfe80 con 0x55c11e9a7ff0 2023-11-28T11:24:57.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.595+0000 7f0d2551cb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c11e84c1e0 con 0x55c11e98a700 2023-11-28T11:24:57.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.595+0000 7f0d2551cb80 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c11e8bae50 con 0x55c11e99eb60 2023-11-28T11:24:57.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.595+0000 7f0d0b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c11e9a7ff0 0x55c11e961a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:57.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.595+0000 7f0d0b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c11e9a7ff0 0x55c11e961a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38782/0 (socket says 172.21.15.92:38782) 2023-11-28T11:24:57.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.595+0000 7f0d0b7fe700 1 -- 172.21.15.92:0/1504936134 learned_addr learned my addr 172.21.15.92:0/1504936134 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:57.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.596+0000 7f0d0b7fe700 1 -- 172.21.15.92:0/1504936134 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c11e99eb60 msgr2=0x55c11e9a83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:57.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.596+0000 7f0d0b7fe700 1 --2- 172.21.15.92:0/1504936134 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c11e99eb60 0x55c11e9a83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:57.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.596+0000 7f0d0b7fe700 1 -- 172.21.15.92:0/1504936134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c11e98a700 msgr2=0x55c11e962c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:57.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.596+0000 7f0d0b7fe700 1 --2- 172.21.15.92:0/1504936134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c11e98a700 0x55c11e962c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:57.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.596+0000 7f0d0b7fe700 1 -- 172.21.15.92:0/1504936134 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c11e6a8830 con 0x55c11e9a7ff0 2023-11-28T11:24:57.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.596+0000 7f0d0b7fe700 1 --2- 172.21.15.92:0/1504936134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c11e9a7ff0 0x55c11e961a90 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc004360 tx=0x7f0cfc010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36bba481d76f5f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:57.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.596+0000 7f0cf17fa700 1 -- 172.21.15.92:0/1504936134 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc0036a0 con 0x55c11e9a7ff0 2023-11-28T11:24:57.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.596+0000 7f0cf17fa700 1 -- 172.21.15.92:0/1504936134 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cfc003800 con 0x55c11e9a7ff0 2023-11-28T11:24:57.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.596+0000 7f0cf17fa700 1 -- 172.21.15.92:0/1504936134 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc003bb0 con 0x55c11e9a7ff0 2023-11-28T11:24:57.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.597+0000 7f0d2551cb80 1 -- 172.21.15.92:0/1504936134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c11e9a7ff0 msgr2=0x55c11e961a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:57.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.597+0000 7f0d2551cb80 1 --2- 172.21.15.92:0/1504936134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c11e9a7ff0 0x55c11e961a90 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc004360 tx=0x7f0cfc010000 comp rx=0 tx=0).stop 2023-11-28T11:24:57.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.597+0000 7f0d2551cb80 1 -- 172.21.15.92:0/1504936134 shutdown_connections 2023-11-28T11:24:57.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.597+0000 7f0d2551cb80 1 --2- 172.21.15.92:0/1504936134 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c11e98a700 0x55c11e962c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:57.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.597+0000 7f0d2551cb80 1 --2- 172.21.15.92:0/1504936134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c11e9a7ff0 0x55c11e961a90 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:57.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.597+0000 7f0d2551cb80 1 --2- 172.21.15.92:0/1504936134 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c11e99eb60 0x55c11e9a83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:57.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.597+0000 7f0d2551cb80 1 -- 172.21.15.92:0/1504936134 >> 172.21.15.92:0/1504936134 conn(0x55c11e957dc0 msgr2=0x55c11e960fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:57.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.597+0000 7f0d2551cb80 1 -- 172.21.15.92:0/1504936134 shutdown_connections 2023-11-28T11:24:57.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.597+0000 7f0d2551cb80 1 -- 172.21.15.92:0/1504936134 wait complete. 2023-11-28T11:24:57.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.598+0000 7f0d2551cb80 1 -- start start 2023-11-28T11:24:57.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.598+0000 7f0d2551cb80 1 Processor -- start 2023-11-28T11:24:57.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.599+0000 7f0d2551cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c11e98a700 0x55c11e9a3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:57.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.599+0000 7f0d2551cb80 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c11e99eb60 0x55c11e9a6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:57.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.599+0000 7f0d2551cb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c11e9a7ff0 0x55c11e9a7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:57.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.599+0000 7f0d2551cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c11e8b41c0 con 0x55c11e98a700 2023-11-28T11:24:57.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.599+0000 7f0d2551cb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c11e6a9430 con 0x55c11e9a7ff0 2023-11-28T11:24:57.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.599+0000 7f0d2551cb80 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c11e84c3e0 con 0x55c11e99eb60 2023-11-28T11:24:57.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.599+0000 7f0d2551cb80 10 client.0 _ll_register_callbacks cb 0x55c11e8524a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:24:57.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.599+0000 7f0d0b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c11e98a700 0x55c11e9a3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:57.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.599+0000 7f0d0b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c11e98a700 0x55c11e9a3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38784/0 (socket says 172.21.15.92:38784) 2023-11-28T11:24:57.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.599+0000 7f0d0b7fe700 1 -- 172.21.15.92:0/2463596970 learned_addr learned my addr 172.21.15.92:0/2463596970 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:57.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.599+0000 7f0d0b7fe700 1 -- 172.21.15.92:0/2463596970 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c11e99eb60 msgr2=0x55c11e9a6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:57.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.599+0000 7f0d0b7fe700 1 --2- 172.21.15.92:0/2463596970 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c11e99eb60 0x55c11e9a6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:57.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.599+0000 7f0d0b7fe700 1 -- 172.21.15.92:0/2463596970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c11e9a7ff0 msgr2=0x55c11e9a7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:57.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.599+0000 7f0d0b7fe700 1 --2- 172.21.15.92:0/2463596970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c11e9a7ff0 0x55c11e9a7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:57.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.599+0000 7f0d0b7fe700 1 -- 172.21.15.92:0/2463596970 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d0c0cd8f0 con 0x55c11e98a700 2023-11-28T11:24:57.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.600+0000 7f0d0b7fe700 1 --2- 172.21.15.92:0/2463596970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c11e98a700 0x55c11e9a3340 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc00bb70 tx=0x7f0cfc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f53e054220a218c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:57.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.600+0000 7f0cea7fc700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-11-28T11:24:57.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.600+0000 7f0cea7fc700 1 -- 172.21.15.92:0/2463596970 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc036070 con 0x55c11e98a700 2023-11-28T11:24:57.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.600+0000 7f0cea7fc700 1 -- 172.21.15.92:0/2463596970 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 6556+0+0 (secure 0 0 0) 0x7f0cfc03ce20 con 0x55c11e98a700 2023-11-28T11:24:57.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.600+0000 7f0d2551cb80 1 -- 172.21.15.92:0/2463596970 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c11e8b80d0 con 0x55c11e98a700 2023-11-28T11:24:57.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.600+0000 7f0cea7fc700 1 -- 172.21.15.92:0/2463596970 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0cfc003d40 con 0x55c11e98a700 2023-11-28T11:24:57.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.600+0000 7f0d2551cb80 10 client.6471 fetch_fsmap learned FSMap version 33 2023-11-28T11:24:57.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.600+0000 7f0d2551cb80 1 -- 172.21.15.92:0/2463596970 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x55c11e795140 con 0x55c11e98a700 2023-11-28T11:24:57.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.600+0000 7f0cea7fc700 1 -- 172.21.15.92:0/2463596970 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c11e8b80d0 con 0x55c11e98a700 2023-11-28T11:24:57.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.600+0000 7f0cea7fc700 1 -- 172.21.15.92:0/2463596970 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 33) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f0cfc01c450 con 0x55c11e98a700 2023-11-28T11:24:57.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.600+0000 7f0d2551cb80 10 client.6471 fetch_fsmap finished waiting for FSMap version 33 2023-11-28T11:24:57.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.601+0000 7f0d2551cb80 10 client.6471 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T11:24:57.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.601+0000 7f0d2551cb80 20 client.6471 populate_metadata read hostname 'smithi092' 2023-11-28T11:24:57.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.601+0000 7f0d2551cb80 1 --2- 172.21.15.92:0/2463596970 >> [v2:172.21.15.92:6834/3634521248,v1:172.21.15.92:6836/3634521248] conn(0x55c11e9906d0 0x55c11e990aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:57.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.601+0000 7f0d2551cb80 4 client.6471 mds_command: new command op to 6213 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:24:57.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.601+0000 7f0d2551cb80 1 -- 172.21.15.92:0/2463596970 --> [v2:172.21.15.92:6834/3634521248,v1:172.21.15.92:6836/3634521248] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c11e703f90 con 0x55c11e9906d0 2023-11-28T11:24:57.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.601+0000 7f0d0affd700 1 --2- 172.21.15.92:0/2463596970 >> [v2:172.21.15.92:6834/3634521248,v1:172.21.15.92:6836/3634521248] conn(0x55c11e9906d0 0x55c11e990aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:57.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.601+0000 7f0d0affd700 1 --2- 172.21.15.92:0/2463596970 >> [v2:172.21.15.92:6834/3634521248,v1:172.21.15.92:6836/3634521248] conn(0x55c11e9906d0 0x55c11e990aa0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78a83b93771f7725 server_cookie=89586a2ff1e79a4d in_seq=0 out_seq=0 2023-11-28T11:24:57.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.601+0000 7f0cea7fc700 10 client.6471 ms_handle_connect on v2:172.21.15.92:6834/3634521248 2023-11-28T11:24:57.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.602+0000 7f0cea7fc700 1 -- 172.21.15.92:0/2463596970 <== mds.0 v2:172.21.15.92:6834/3634521248 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x55c11e703f90 con 0x55c11e9906d0 2023-11-28T11:24:57.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.602+0000 7f0cea7fc700 10 client.6471 handle_command_reply: tid=0 2023-11-28T11:24:57.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.602+0000 7f0d2551cb80 1 client.6471 shutdown 2023-11-28T11:24:57.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.602+0000 7f0d2551cb80 1 -- 172.21.15.92:0/2463596970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c11e98a700 msgr2=0x55c11e9a3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:57.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.602+0000 7f0d2551cb80 1 --2- 172.21.15.92:0/2463596970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c11e98a700 0x55c11e9a3340 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc00bb70 tx=0x7f0cfc0047c0 comp rx=0 tx=0).stop 2023-11-28T11:24:57.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.602+0000 7f0d2551cb80 1 -- 172.21.15.92:0/2463596970 shutdown_connections 2023-11-28T11:24:57.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.602+0000 7f0d2551cb80 1 --2- 172.21.15.92:0/2463596970 >> [v2:172.21.15.92:6834/3634521248,v1:172.21.15.92:6836/3634521248] conn(0x55c11e9906d0 0x55c11e990aa0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:57.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.602+0000 7f0d2551cb80 1 --2- 172.21.15.92:0/2463596970 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c11e9a7ff0 0x55c11e9a7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:57.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.602+0000 7f0d2551cb80 1 --2- 172.21.15.92:0/2463596970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c11e98a700 0x55c11e9a3340 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:57.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.602+0000 7f0cea7fc700 0 client.6471 ms_handle_reset on v2:172.21.15.92:6834/3634521248 2023-11-28T11:24:57.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.602+0000 7f0d2551cb80 1 --2- 172.21.15.92:0/2463596970 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c11e99eb60 0x55c11e9a6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:57.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.602+0000 7f0d2551cb80 1 -- 172.21.15.92:0/2463596970 >> 172.21.15.92:0/2463596970 conn(0x55c11e957dc0 msgr2=0x55c11e989cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:57.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.602+0000 7f0d2551cb80 1 -- 172.21.15.92:0/2463596970 shutdown_connections 2023-11-28T11:24:57.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.603+0000 7f0d2551cb80 1 -- 172.21.15.92:0/2463596970 wait complete. 2023-11-28T11:24:57.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.603+0000 7f0d2551cb80 20 client.6471 trim_cache size 0 max 16384 2023-11-28T11:24:57.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.616+0000 7f0d2551cb80 1 Processor -- start 2023-11-28T11:24:57.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.616+0000 7f0d2551cb80 1 -- start start 2023-11-28T11:24:57.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.617+0000 7f0d2551cb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c11e98a700 0x55c11e9a4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:57.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.617+0000 7f0d2551cb80 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c11e99eb60 0x55c11e990aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:57.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.617+0000 7f0d2551cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c11e9a7ff0 0x55c11e9a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:57.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.617+0000 7f0d2551cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c11e9a3640 con 0x55c11e9a7ff0 2023-11-28T11:24:57.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.617+0000 7f0d2551cb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c11e95ea00 con 0x55c11e98a700 2023-11-28T11:24:57.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.617+0000 7f0d2551cb80 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c11e957b30 con 0x55c11e99eb60 2023-11-28T11:24:57.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.617+0000 7f0d0bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c11e9a7ff0 0x55c11e9a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:57.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.617+0000 7f0d0bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c11e9a7ff0 0x55c11e9a6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38788/0 (socket says 172.21.15.92:38788) 2023-11-28T11:24:57.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.617+0000 7f0d0bfff700 1 -- 172.21.15.92:0/1022671139 learned_addr learned my addr 172.21.15.92:0/1022671139 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:57.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.617+0000 7f0d0b7fe700 1 --2- 172.21.15.92:0/1022671139 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c11e98a700 0x55c11e9a4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:57.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.617+0000 7f0d0affd700 1 --2- 172.21.15.92:0/1022671139 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c11e99eb60 0x55c11e990aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:57.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.617+0000 7f0d0bfff700 1 -- 172.21.15.92:0/1022671139 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c11e99eb60 msgr2=0x55c11e990aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:57.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.617+0000 7f0d0bfff700 1 --2- 172.21.15.92:0/1022671139 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c11e99eb60 0x55c11e990aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:57.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.617+0000 7f0d0bfff700 1 -- 172.21.15.92:0/1022671139 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c11e98a700 msgr2=0x55c11e9a4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:57.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.617+0000 7f0d0bfff700 1 --2- 172.21.15.92:0/1022671139 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c11e98a700 0x55c11e9a4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:57.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.617+0000 7f0d0bfff700 1 -- 172.21.15.92:0/1022671139 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c11e87b380 con 0x55c11e9a7ff0 2023-11-28T11:24:57.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.617+0000 7f0d0bfff700 1 --2- 172.21.15.92:0/1022671139 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c11e9a7ff0 0x55c11e9a6cc0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f0d04056490 tx=0x7f0d04056350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6a01a8e97e464a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:57.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.618+0000 7f0cf1ffb700 1 -- 172.21.15.92:0/1022671139 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0405e880 con 0x55c11e9a7ff0 2023-11-28T11:24:57.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.618+0000 7f0cf1ffb700 1 -- 172.21.15.92:0/1022671139 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d0405e3f0 con 0x55c11e9a7ff0 2023-11-28T11:24:57.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.618+0000 7f0cf1ffb700 1 -- 172.21.15.92:0/1022671139 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0405c0d0 con 0x55c11e9a7ff0 2023-11-28T11:24:57.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.618+0000 7f0d2551cb80 1 -- 172.21.15.92:0/1022671139 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c11e9a7ff0 msgr2=0x55c11e9a6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:57.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.618+0000 7f0d2551cb80 1 --2- 172.21.15.92:0/1022671139 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c11e9a7ff0 0x55c11e9a6cc0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f0d04056490 tx=0x7f0d04056350 comp rx=0 tx=0).stop 2023-11-28T11:24:57.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.618+0000 7f0d2551cb80 1 -- 172.21.15.92:0/1022671139 shutdown_connections 2023-11-28T11:24:57.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.618+0000 7f0d2551cb80 1 --2- 172.21.15.92:0/1022671139 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c11e9a7ff0 0x55c11e9a6cc0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:57.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.618+0000 7f0d2551cb80 1 --2- 172.21.15.92:0/1022671139 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c11e98a700 0x55c11e9a4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:57.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.618+0000 7f0d2551cb80 1 --2- 172.21.15.92:0/1022671139 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c11e99eb60 0x55c11e990aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:57.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.618+0000 7f0d2551cb80 1 -- 172.21.15.92:0/1022671139 >> 172.21.15.92:0/1022671139 conn(0x55c11e957dc0 msgr2=0x55c11e98e530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:57.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.618+0000 7f0d2551cb80 1 -- 172.21.15.92:0/1022671139 shutdown_connections 2023-11-28T11:24:57.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.618+0000 7f0d2551cb80 1 -- 172.21.15.92:0/1022671139 wait complete. 2023-11-28T11:24:57.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.619+0000 7f0d2551cb80 1 -- start start 2023-11-28T11:24:57.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.619+0000 7f0d2551cb80 1 Processor -- start 2023-11-28T11:24:57.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.620+0000 7f0d2551cb80 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c11e98a700 0x55c11e984330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:57.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.620+0000 7f0d2551cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c11e99eb60 0x55c11e9c96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:57.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.620+0000 7f0d2551cb80 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c11e9a7ff0 0x55c11ea0d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:57.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.620+0000 7f0d2551cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c11ea0c9e0 con 0x55c11e99eb60 2023-11-28T11:24:57.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.620+0000 7f0d2551cb80 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c11e984c30 con 0x55c11e98a700 2023-11-28T11:24:57.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.620+0000 7f0d2551cb80 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c11e9c1790 con 0x55c11e9a7ff0 2023-11-28T11:24:57.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.620+0000 7f0d2551cb80 10 client.0 _ll_register_callbacks cb 0x55c11e8524a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:24:57.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.620+0000 7f0d0affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c11e99eb60 0x55c11e9c96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:57.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.621+0000 7f0d0affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c11e99eb60 0x55c11e9c96a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38804/0 (socket says 172.21.15.92:38804) 2023-11-28T11:24:57.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.621+0000 7f0d0affd700 1 -- 172.21.15.92:0/4237885622 learned_addr learned my addr 172.21.15.92:0/4237885622 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:57.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.621+0000 7f0d0b7fe700 1 --2- 172.21.15.92:0/4237885622 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c11e98a700 0x55c11e984330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:57.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.621+0000 7f0d0bfff700 1 --2- 172.21.15.92:0/4237885622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c11e9a7ff0 0x55c11ea0d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:57.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.621+0000 7f0d0affd700 1 -- 172.21.15.92:0/4237885622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c11e9a7ff0 msgr2=0x55c11ea0d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:57.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.621+0000 7f0d0affd700 1 --2- 172.21.15.92:0/4237885622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c11e9a7ff0 0x55c11ea0d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:57.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.621+0000 7f0d0affd700 1 -- 172.21.15.92:0/4237885622 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c11e98a700 msgr2=0x55c11e984330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:57.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.621+0000 7f0d0affd700 1 --2- 172.21.15.92:0/4237885622 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c11e98a700 0x55c11e984330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:57.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.621+0000 7f0d0affd700 1 -- 172.21.15.92:0/4237885622 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d0c0cfd10 con 0x55c11e99eb60 2023-11-28T11:24:57.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.621+0000 7f0d0affd700 1 --2- 172.21.15.92:0/4237885622 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c11e99eb60 0x55c11e9c96a0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c1e84a0 tx=0x7f0d0000bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e1b8c556cc703cf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:57.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.621+0000 7f0cea7fc700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-11-28T11:24:57.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.621+0000 7f0cea7fc700 1 -- 172.21.15.92:0/4237885622 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d000172b0 con 0x55c11e99eb60 2023-11-28T11:24:57.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.621+0000 7f0cea7fc700 1 -- 172.21.15.92:0/4237885622 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 6556+0+0 (secure 0 0 0) 0x7f0d00017410 con 0x55c11e99eb60 2023-11-28T11:24:57.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.621+0000 7f0d2551cb80 1 -- 172.21.15.92:0/4237885622 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0cfc003800 con 0x55c11e99eb60 2023-11-28T11:24:57.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.622+0000 7f0cea7fc700 1 -- 172.21.15.92:0/4237885622 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d00020470 con 0x55c11e99eb60 2023-11-28T11:24:57.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.622+0000 7f0d2551cb80 10 client.6477 fetch_fsmap learned FSMap version 33 2023-11-28T11:24:57.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.622+0000 7f0d2551cb80 1 -- 172.21.15.92:0/4237885622 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x55c11e709270 con 0x55c11e99eb60 2023-11-28T11:24:57.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.622+0000 7f0cea7fc700 1 -- 172.21.15.92:0/4237885622 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cfc003800 con 0x55c11e99eb60 2023-11-28T11:24:57.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.622+0000 7f0cea7fc700 1 -- 172.21.15.92:0/4237885622 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 33) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f0d000205d0 con 0x55c11e99eb60 2023-11-28T11:24:57.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.622+0000 7f0d2551cb80 10 client.6477 fetch_fsmap finished waiting for FSMap version 33 2023-11-28T11:24:57.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.622+0000 7f0d2551cb80 10 client.6477 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T11:24:57.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.622+0000 7f0d2551cb80 20 client.6477 populate_metadata read hostname 'smithi092' 2023-11-28T11:24:57.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.622+0000 7f0d2551cb80 1 --2- 172.21.15.92:0/4237885622 >> [v2:172.21.15.92:6834/3634521248,v1:172.21.15.92:6836/3634521248] conn(0x55c11e98e4a0 0x55c11e98e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:57.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.622+0000 7f0d2551cb80 4 client.6477 mds_command: new command op to 6213 tid=0[{"prefix": "damage ls", "format": "json-pretty"}] 2023-11-28T11:24:57.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.622+0000 7f0d2551cb80 1 -- 172.21.15.92:0/4237885622 --> [v2:172.21.15.92:6834/3634521248,v1:172.21.15.92:6836/3634521248] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x55c11e7390e0 con 0x55c11e98e4a0 2023-11-28T11:24:57.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.622+0000 7f0d0b7fe700 1 --2- 172.21.15.92:0/4237885622 >> [v2:172.21.15.92:6834/3634521248,v1:172.21.15.92:6836/3634521248] conn(0x55c11e98e4a0 0x55c11e98e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:57.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.623+0000 7f0d0b7fe700 1 --2- 172.21.15.92:0/4237885622 >> [v2:172.21.15.92:6834/3634521248,v1:172.21.15.92:6836/3634521248] conn(0x55c11e98e4a0 0x55c11e98e870 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80a99c0afbe2ca0d server_cookie=b5e485290435409e in_seq=0 out_seq=0 2023-11-28T11:24:57.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.623+0000 7f0cea7fc700 10 client.6477 ms_handle_connect on v2:172.21.15.92:6834/3634521248 2023-11-28T11:24:57.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.623+0000 7f0cea7fc700 1 -- 172.21.15.92:0/4237885622 <== mds.0 v2:172.21.15.92:6834/3634521248 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+140 (crc 0 0 0) 0x55c11e7390e0 con 0x55c11e98e4a0 2023-11-28T11:24:57.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.623+0000 7f0cea7fc700 10 client.6477 handle_command_reply: tid=0 2023-11-28T11:24:57.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.623+0000 7f0d2551cb80 1 client.6477 shutdown 2023-11-28T11:24:57.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.623+0000 7f0d2551cb80 1 -- 172.21.15.92:0/4237885622 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c11e99eb60 msgr2=0x55c11e9c96a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:57.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.623+0000 7f0d2551cb80 1 --2- 172.21.15.92:0/4237885622 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c11e99eb60 0x55c11e9c96a0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c1e84a0 tx=0x7f0d0000bce0 comp rx=0 tx=0).stop 2023-11-28T11:24:57.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.623+0000 7f0d2551cb80 1 -- 172.21.15.92:0/4237885622 shutdown_connections 2023-11-28T11:24:57.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.623+0000 7f0d2551cb80 1 --2- 172.21.15.92:0/4237885622 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x55c11e9a7ff0 0x55c11ea0d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:57.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.623+0000 7f0d2551cb80 1 --2- 172.21.15.92:0/4237885622 >> [v2:172.21.15.92:6834/3634521248,v1:172.21.15.92:6836/3634521248] conn(0x55c11e98e4a0 0x55c11e98e870 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:57.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.623+0000 7f0d2551cb80 1 --2- 172.21.15.92:0/4237885622 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x55c11e98a700 0x55c11e984330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:57.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.623+0000 7f0d2551cb80 1 --2- 172.21.15.92:0/4237885622 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c11e99eb60 0x55c11e9c96a0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:57.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.623+0000 7f0cea7fc700 0 client.6477 ms_handle_reset on v2:172.21.15.92:6834/3634521248 2023-11-28T11:24:57.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.623+0000 7f0d2551cb80 1 -- 172.21.15.92:0/4237885622 >> 172.21.15.92:0/4237885622 conn(0x55c11e957dc0 msgr2=0x55c11e9a4b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:57.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.623+0000 7f0d2551cb80 1 -- 172.21.15.92:0/4237885622 shutdown_connections 2023-11-28T11:24:57.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.623+0000 7f0d2551cb80 1 -- 172.21.15.92:0/4237885622 wait complete. 2023-11-28T11:24:57.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.623+0000 7f0d2551cb80 20 client.6477 trim_cache size 0 max 16384 2023-11-28T11:24:57.630 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:24:57.630 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-11-28T11:24:57.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-28T11:24:57.631 INFO:teuthology.orchestra.run.smithi092.stdout: "damage_type": "backtrace", 2023-11-28T11:24:57.631 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 945585993, 2023-11-28T11:24:57.631 INFO:teuthology.orchestra.run.smithi092.stdout: "ino": 1099511627777, 2023-11-28T11:24:57.631 INFO:teuthology.orchestra.run.smithi092.stdout: "path": "/dir_test/file" 2023-11-28T11:24:57.631 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-28T11:24:57.631 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-11-28T11:24:57.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.624+0000 7f0d11bff700 1 -- 172.21.15.92:0/659716465 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f0cf80682f0 msgr2=0x7f0cf806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:57.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.624+0000 7f0d11bff700 1 --2- 172.21.15.92:0/659716465 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f0cf80682f0 0x7f0cf806a7a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc009130 tx=0x7f0cfc008030 comp rx=0 tx=0).stop 2023-11-28T11:24:57.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.624+0000 7f0d11bff700 1 -- 172.21.15.92:0/659716465 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d0c0cd460 msgr2=0x7f0d0c0c73e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:57.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.624+0000 7f0d11bff700 1 --2- 172.21.15.92:0/659716465 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d0c0cd460 0x7f0d0c0c73e0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f0d0400cc10 tx=0x7f0d0400a060 comp rx=0 tx=0).stop 2023-11-28T11:24:57.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.624+0000 7f0d11bff700 1 -- 172.21.15.92:0/659716465 shutdown_connections 2023-11-28T11:24:57.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.624+0000 7f0d11bff700 1 --2- 172.21.15.92:0/659716465 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f0cf80682f0 0x7f0cf806a7a0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:57.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.624+0000 7f0d11bff700 1 --2- 172.21.15.92:0/659716465 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d0c0cd460 0x7f0d0c0c73e0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:57.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.624+0000 7f0d11bff700 1 --2- 172.21.15.92:0/659716465 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0d0c06c3f0 0x7f0d0c0c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:57.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.624+0000 7f0d11bff700 1 --2- 172.21.15.92:0/659716465 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0d0c0b8510 0x7f0d0c0c96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:57.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.624+0000 7f0d11bff700 1 -- 172.21.15.92:0/659716465 >> 172.21.15.92:0/659716465 conn(0x7f0d0c0719a0 msgr2=0x7f0d0c0b3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:57.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.624+0000 7f0d11bff700 1 -- 172.21.15.92:0/659716465 shutdown_connections 2023-11-28T11:24:57.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.624+0000 7f0d11bff700 1 -- 172.21.15.92:0/659716465 wait complete. 2023-11-28T11:24:57.642 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T11:24:57.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.837+0000 7f7d1ffff700 1 Processor -- start 2023-11-28T11:24:57.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.837+0000 7f7d1ffff700 1 -- start start 2023-11-28T11:24:57.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.838+0000 7f7d1ffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7d180cd460 0x7f7d1806c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:57.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.838+0000 7f7d1ffff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d180d0d60 0x7f7d180d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:57.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.838+0000 7f7d1ffff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d180b88e0 con 0x7f7d180b8510 2023-11-28T11:24:57.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.838+0000 7f7d1ffff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d180cdbd0 con 0x7f7d180cd460 2023-11-28T11:24:57.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.838+0000 7f7d1ffff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d180d02d0 con 0x7f7d180d0d60 2023-11-28T11:24:57.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.838+0000 7f7d1d59a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7d180cd460 0x7f7d1806c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:57.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.839+0000 7f7d1d59a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7d180cd460 0x7f7d1806c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38820/0 (socket says 172.21.15.92:38820) 2023-11-28T11:24:57.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.839+0000 7f7d1d59a700 1 -- 172.21.15.92:0/4263993519 learned_addr learned my addr 172.21.15.92:0/4263993519 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:57.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.839+0000 7f7d1e59c700 1 --2- 172.21.15.92:0/4263993519 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d180d0d60 0x7f7d180d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:57.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.839+0000 7f7d1d59a700 1 -- 172.21.15.92:0/4263993519 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d180d0d60 msgr2=0x7f7d180d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:57.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.839+0000 7f7d1d59a700 1 --2- 172.21.15.92:0/4263993519 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d180d0d60 0x7f7d180d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:57.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.839+0000 7f7d1d59a700 1 -- 172.21.15.92:0/4263993519 >> v1:172.21.15.169:6789/0 conn(0x7f7d180b8510 legacy=0x7f7d1806b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:24:57.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.839+0000 7f7d1d59a700 1 -- 172.21.15.92:0/4263993519 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d180cd8f0 con 0x7f7d180cd460 2023-11-28T11:24:57.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.839+0000 7f7d1e59c700 1 --2- 172.21.15.92:0/4263993519 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d180d0d60 0x7f7d180d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:24:57.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.839+0000 7f7d1d59a700 1 --2- 172.21.15.92:0/4263993519 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d180cd460 0x7f7d1806c080 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f7d08009920 tx=0x7f7d08009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60b5ee140d6cb93a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:57.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.839+0000 7f7d1cd99700 1 -- 172.21.15.92:0/4263993519 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d08004230 con 0x7f7d180cd460 2023-11-28T11:24:57.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.840+0000 7f7d1cd99700 1 -- 172.21.15.92:0/4263993519 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d08004390 con 0x7f7d180cd460 2023-11-28T11:24:57.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.840+0000 7f7d1cd99700 1 -- 172.21.15.92:0/4263993519 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d08009e70 con 0x7f7d180cd460 2023-11-28T11:24:57.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.840+0000 7f7d1ffff700 1 -- 172.21.15.92:0/4263993519 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d180cd460 msgr2=0x7f7d1806c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:57.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.840+0000 7f7d1ffff700 1 --2- 172.21.15.92:0/4263993519 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d180cd460 0x7f7d1806c080 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f7d08009920 tx=0x7f7d08009c30 comp rx=0 tx=0).stop 2023-11-28T11:24:57.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.840+0000 7f7d1ffff700 1 -- 172.21.15.92:0/4263993519 shutdown_connections 2023-11-28T11:24:57.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.840+0000 7f7d1ffff700 1 --2- 172.21.15.92:0/4263993519 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d180d0d60 0x7f7d180d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:57.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.840+0000 7f7d1ffff700 1 --2- 172.21.15.92:0/4263993519 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d180cd460 0x7f7d1806c080 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:57.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.840+0000 7f7d1ffff700 1 -- 172.21.15.92:0/4263993519 >> 172.21.15.92:0/4263993519 conn(0x7f7d180719a0 msgr2=0x7f7d18071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:57.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.840+0000 7f7d1ffff700 1 -- 172.21.15.92:0/4263993519 shutdown_connections 2023-11-28T11:24:57.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.840+0000 7f7d1ffff700 1 -- 172.21.15.92:0/4263993519 wait complete. 2023-11-28T11:24:57.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.842+0000 7f7d1ffff700 1 Processor -- start 2023-11-28T11:24:57.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.842+0000 7f7d1ffff700 1 -- start start 2023-11-28T11:24:57.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.843+0000 7f7d1ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d180b8510 0x7f7d1814e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:57.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.843+0000 7f7d1ffff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d180cd460 0x7f7d181e3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:57.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.843+0000 7f7d1ffff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d180d0d60 0x7f7d181e1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:57.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.843+0000 7f7d1ffff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d181e2390 con 0x7f7d180b8510 2023-11-28T11:24:57.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.843+0000 7f7d1ffff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d181e2670 con 0x7f7d180d0d60 2023-11-28T11:24:57.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.843+0000 7f7d1ffff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d181e2950 con 0x7f7d180cd460 2023-11-28T11:24:57.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.843+0000 7f7d1dd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d180b8510 0x7f7d1814e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:57.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.843+0000 7f7d1dd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d180b8510 0x7f7d1814e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38826/0 (socket says 172.21.15.92:38826) 2023-11-28T11:24:57.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.843+0000 7f7d1dd9b700 1 -- 172.21.15.92:0/3161721449 learned_addr learned my addr 172.21.15.92:0/3161721449 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:24:57.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.843+0000 7f7d1d59a700 1 --2- 172.21.15.92:0/3161721449 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d180cd460 0x7f7d181e3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:57.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.843+0000 7f7d1e59c700 1 --2- 172.21.15.92:0/3161721449 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d180d0d60 0x7f7d181e1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:57.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.844+0000 7f7d1dd9b700 1 -- 172.21.15.92:0/3161721449 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d180cd460 msgr2=0x7f7d181e3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:57.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.844+0000 7f7d1dd9b700 1 --2- 172.21.15.92:0/3161721449 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d180cd460 0x7f7d181e3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:57.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.844+0000 7f7d1dd9b700 1 -- 172.21.15.92:0/3161721449 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d180d0d60 msgr2=0x7f7d181e1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:57.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.844+0000 7f7d1dd9b700 1 --2- 172.21.15.92:0/3161721449 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d180d0d60 0x7f7d181e1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:57.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.844+0000 7f7d1dd9b700 1 -- 172.21.15.92:0/3161721449 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d08009550 con 0x7f7d180b8510 2023-11-28T11:24:57.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.844+0000 7f7d1d59a700 1 --2- 172.21.15.92:0/3161721449 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d180cd460 0x7f7d181e3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:24:57.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.844+0000 7f7d1dd9b700 1 --2- 172.21.15.92:0/3161721449 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d180b8510 0x7f7d1814e3f0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f7d1400c4d0 tx=0x7f7d1400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:57.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.844+0000 7f7d0e7fc700 1 -- 172.21.15.92:0/3161721449 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d14011790 con 0x7f7d180b8510 2023-11-28T11:24:57.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.844+0000 7f7d0e7fc700 1 -- 172.21.15.92:0/3161721449 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d140118f0 con 0x7f7d180b8510 2023-11-28T11:24:57.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.844+0000 7f7d0e7fc700 1 -- 172.21.15.92:0/3161721449 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d14011ca0 con 0x7f7d180b8510 2023-11-28T11:24:57.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.844+0000 7f7d1ffff700 1 -- 172.21.15.92:0/3161721449 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d180cfff0 con 0x7f7d180b8510 2023-11-28T11:24:57.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.844+0000 7f7d1ffff700 1 -- 172.21.15.92:0/3161721449 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d180cfd10 con 0x7f7d180b8510 2023-11-28T11:24:57.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.845+0000 7f7d1ffff700 1 -- 172.21.15.92:0/3161721449 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d18067880 con 0x7f7d180b8510 2023-11-28T11:24:57.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.846+0000 7f7d0e7fc700 1 -- 172.21.15.92:0/3161721449 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7d14010050 con 0x7f7d180b8510 2023-11-28T11:24:57.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.847+0000 7f7d0e7fc700 1 --2- 172.21.15.92:0/3161721449 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7cfc0682f0 0x7f7cfc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:24:57.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.848+0000 7f7d0e7fc700 1 -- 172.21.15.92:0/3161721449 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6556+0+0 (secure 0 0 0) 0x7f7d14085cf0 con 0x7f7d180b8510 2023-11-28T11:24:57.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.848+0000 7f7d1d59a700 1 --2- 172.21.15.92:0/3161721449 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7cfc0682f0 0x7f7cfc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:24:57.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.848+0000 7f7d1d59a700 1 --2- 172.21.15.92:0/3161721449 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7cfc0682f0 0x7f7cfc06a7a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7d180b1de0 tx=0x7f7d08008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:24:57.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:57.852+0000 7f7d0e7fc700 1 -- 172.21.15.92:0/3161721449 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7d14052270 con 0x7f7d180b8510 2023-11-28T11:24:58.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:58.086+0000 7f7d1ffff700 1 -- 172.21.15.92:0/3161721449 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f7d180d2940 con 0x7f7d180b8510 2023-11-28T11:24:58.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:58.086+0000 7f7d0e7fc700 1 -- 172.21.15.92:0/3161721449 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f7d14055890 con 0x7f7d180b8510 2023-11-28T11:24:58.087 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:24:58.087 INFO:teuthology.orchestra.run.smithi092.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T11:24:58.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:58.088+0000 7f7d1ffff700 1 -- 172.21.15.92:0/3161721449 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7cfc0682f0 msgr2=0x7f7cfc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:58.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:58.088+0000 7f7d1ffff700 1 --2- 172.21.15.92:0/3161721449 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7cfc0682f0 0x7f7cfc06a7a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7d180b1de0 tx=0x7f7d08008030 comp rx=0 tx=0).stop 2023-11-28T11:24:58.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:58.088+0000 7f7d1ffff700 1 -- 172.21.15.92:0/3161721449 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d180b8510 msgr2=0x7f7d1814e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:24:58.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:58.088+0000 7f7d1ffff700 1 --2- 172.21.15.92:0/3161721449 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d180b8510 0x7f7d1814e3f0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f7d1400c4d0 tx=0x7f7d1400c890 comp rx=0 tx=0).stop 2023-11-28T11:24:58.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:58.088+0000 7f7d1ffff700 1 -- 172.21.15.92:0/3161721449 shutdown_connections 2023-11-28T11:24:58.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:58.088+0000 7f7d1ffff700 1 --2- 172.21.15.92:0/3161721449 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d180d0d60 0x7f7d181e1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:58.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:58.088+0000 7f7d1ffff700 1 --2- 172.21.15.92:0/3161721449 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7cfc0682f0 0x7f7cfc06a7a0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:58.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:58.088+0000 7f7d1ffff700 1 --2- 172.21.15.92:0/3161721449 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d180b8510 0x7f7d1814e3f0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:58.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:58.088+0000 7f7d1ffff700 1 --2- 172.21.15.92:0/3161721449 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d180cd460 0x7f7d181e3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:24:58.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:58.088+0000 7f7d1ffff700 1 -- 172.21.15.92:0/3161721449 >> 172.21.15.92:0/3161721449 conn(0x7f7d180719a0 msgr2=0x7f7d180cccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:24:58.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:58.088+0000 7f7d1ffff700 1 -- 172.21.15.92:0/3161721449 shutdown_connections 2023-11-28T11:24:58.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:24:58.089+0000 7f7d1ffff700 1 -- 172.21.15.92:0/3161721449 wait complete. 2023-11-28T11:24:58.107 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=20 retry_count=0)... 2023-11-28T11:25:03.108 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T11:25:03.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.323+0000 7fb5313ff700 1 Processor -- start 2023-11-28T11:25:03.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.324+0000 7fb5313ff700 1 -- start start 2023-11-28T11:25:03.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.325+0000 7fb5313ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb52c142e80 0x7fb52c06a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:03.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.325+0000 7fb5313ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb52c06a7b0 0x7fb52c0c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:03.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.325+0000 7fb5313ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb52c159b10 con 0x7fb52c0ca430 2023-11-28T11:25:03.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.325+0000 7fb5313ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb52c0c9380 con 0x7fb52c142e80 2023-11-28T11:25:03.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.325+0000 7fb5313ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb52c06ac30 con 0x7fb52c06a7b0 2023-11-28T11:25:03.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.325+0000 7fb52affd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb52c142e80 0x7fb52c06a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:03.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.325+0000 7fb52b7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb52c0ca430 0x7fb52c06d2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:57170/0 (socket says 172.21.15.92:57170) 2023-11-28T11:25:03.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.325+0000 7fb52b7fe700 1 -- 172.21.15.92:0/437130620 learned_addr learned my addr 172.21.15.92:0/437130620 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:25:03.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.325+0000 7fb52affd700 1 -- 172.21.15.92:0/437130620 >> v2:172.21.15.169:3300/0 conn(0x7fb52c06a7b0 msgr2=0x7fb52c0c9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:03.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.326+0000 7fb523fff700 1 --2- 172.21.15.92:0/437130620 >> v2:172.21.15.169:3300/0 conn(0x7fb52c06a7b0 0x7fb52c0c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:03.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.326+0000 7fb52affd700 1 --2- 172.21.15.92:0/437130620 >> v2:172.21.15.169:3300/0 conn(0x7fb52c06a7b0 0x7fb52c0c9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:03.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.326+0000 7fb52affd700 1 -- 172.21.15.92:0/437130620 >> v1:172.21.15.92:6789/0 conn(0x7fb52c0ca430 legacy=0x7fb52c06d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:03.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.326+0000 7fb52a7fc700 1 -- 172.21.15.92:0/437130620 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2682263278 0 0) 0x7fb52c159b10 con 0x7fb52c0ca430 2023-11-28T11:25:03.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.326+0000 7fb52affd700 1 -- 172.21.15.92:0/437130620 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb52c143250 con 0x7fb52c142e80 2023-11-28T11:25:03.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.326+0000 7fb523fff700 1 --2- 172.21.15.92:0/437130620 >> v2:172.21.15.169:3300/0 conn(0x7fb52c06a7b0 0x7fb52c0c9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:25:03.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.326+0000 7fb52affd700 1 --2- 172.21.15.92:0/437130620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb52c142e80 0x7fb52c06a270 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fb52c0cc220 tx=0x7fb51c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=701713f835e568e8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:03.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.326+0000 7fb52a7fc700 1 -- 172.21.15.92:0/437130620 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51c011830 con 0x7fb52c142e80 2023-11-28T11:25:03.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.326+0000 7fb52a7fc700 1 -- 172.21.15.92:0/437130620 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb51c011990 con 0x7fb52c142e80 2023-11-28T11:25:03.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.326+0000 7fb52a7fc700 1 -- 172.21.15.92:0/437130620 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51c011d20 con 0x7fb52c142e80 2023-11-28T11:25:03.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.327+0000 7fb5313ff700 1 -- 172.21.15.92:0/437130620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb52c142e80 msgr2=0x7fb52c06a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:03.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.327+0000 7fb5313ff700 1 --2- 172.21.15.92:0/437130620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb52c142e80 0x7fb52c06a270 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fb52c0cc220 tx=0x7fb51c009b20 comp rx=0 tx=0).stop 2023-11-28T11:25:03.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.327+0000 7fb5313ff700 1 -- 172.21.15.92:0/437130620 shutdown_connections 2023-11-28T11:25:03.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.327+0000 7fb5313ff700 1 --2- 172.21.15.92:0/437130620 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb52c142e80 0x7fb52c06a270 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:03.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.327+0000 7fb5313ff700 1 --2- 172.21.15.92:0/437130620 >> v2:172.21.15.169:3300/0 conn(0x7fb52c06a7b0 0x7fb52c0c9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:03.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.327+0000 7fb5313ff700 1 -- 172.21.15.92:0/437130620 >> 172.21.15.92:0/437130620 conn(0x7fb52c0cb490 msgr2=0x7fb52c0cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:03.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.327+0000 7fb5313ff700 1 -- 172.21.15.92:0/437130620 shutdown_connections 2023-11-28T11:25:03.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.327+0000 7fb5313ff700 1 -- 172.21.15.92:0/437130620 wait complete. 2023-11-28T11:25:03.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.329+0000 7fb5313ff700 1 Processor -- start 2023-11-28T11:25:03.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.329+0000 7fb5313ff700 1 -- start start 2023-11-28T11:25:03.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.331+0000 7fb5313ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb52c06a7b0 0x7fb52c0c60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:03.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.331+0000 7fb5313ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb52c0ca430 0x7fb52c156ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:03.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.331+0000 7fb5313ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb52c142e80 0x7fb52c0c3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:03.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.331+0000 7fb5313ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c0c6600 con 0x7fb52c06a7b0 2023-11-28T11:25:03.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.331+0000 7fb5313ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c157ac0 con 0x7fb52c0ca430 2023-11-28T11:25:03.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.331+0000 7fb5313ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c0c2890 con 0x7fb52c142e80 2023-11-28T11:25:03.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.331+0000 7fb52affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb52c06a7b0 0x7fb52c0c60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:03.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.331+0000 7fb52affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb52c06a7b0 0x7fb52c0c60c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46096/0 (socket says 172.21.15.92:46096) 2023-11-28T11:25:03.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.331+0000 7fb52affd700 1 -- 172.21.15.92:0/2455745162 learned_addr learned my addr 172.21.15.92:0/2455745162 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:03.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.331+0000 7fb523fff700 1 --2- 172.21.15.92:0/2455745162 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb52c0ca430 0x7fb52c156ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:03.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.331+0000 7fb52b7fe700 1 --2- 172.21.15.92:0/2455745162 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb52c142e80 0x7fb52c0c3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:03.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.331+0000 7fb52affd700 1 -- 172.21.15.92:0/2455745162 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb52c142e80 msgr2=0x7fb52c0c3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:03.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.331+0000 7fb52affd700 1 --2- 172.21.15.92:0/2455745162 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb52c142e80 0x7fb52c0c3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:03.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.331+0000 7fb52affd700 1 -- 172.21.15.92:0/2455745162 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb52c0ca430 msgr2=0x7fb52c156ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:03.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.331+0000 7fb52affd700 1 --2- 172.21.15.92:0/2455745162 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb52c0ca430 0x7fb52c156ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:03.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.331+0000 7fb52affd700 1 -- 172.21.15.92:0/2455745162 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb51c0094a0 con 0x7fb52c06a7b0 2023-11-28T11:25:03.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.331+0000 7fb523fff700 1 --2- 172.21.15.92:0/2455745162 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb52c0ca430 0x7fb52c156ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:25:03.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.332+0000 7fb52affd700 1 --2- 172.21.15.92:0/2455745162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb52c06a7b0 0x7fb52c0c60c0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fb51c014040 tx=0x7fb51c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:03.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.332+0000 7fb5237fe700 1 -- 172.21.15.92:0/2455745162 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51c0038a0 con 0x7fb52c06a7b0 2023-11-28T11:25:03.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.332+0000 7fb5237fe700 1 -- 172.21.15.92:0/2455745162 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb51c01e070 con 0x7fb52c06a7b0 2023-11-28T11:25:03.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.332+0000 7fb5313ff700 1 -- 172.21.15.92:0/2455745162 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb52c0cc5f0 con 0x7fb52c06a7b0 2023-11-28T11:25:03.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.332+0000 7fb5237fe700 1 -- 172.21.15.92:0/2455745162 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51c003c40 con 0x7fb52c06a7b0 2023-11-28T11:25:03.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.332+0000 7fb5313ff700 1 -- 172.21.15.92:0/2455745162 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb52c0cab00 con 0x7fb52c06a7b0 2023-11-28T11:25:03.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.333+0000 7fb5313ff700 1 -- 172.21.15.92:0/2455745162 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb52c0cc8d0 con 0x7fb52c06a7b0 2023-11-28T11:25:03.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.334+0000 7fb5237fe700 1 -- 172.21.15.92:0/2455745162 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb51c00f050 con 0x7fb52c06a7b0 2023-11-28T11:25:03.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.335+0000 7fb5237fe700 1 --2- 172.21.15.92:0/2455745162 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb50c0682f0 0x7fb50c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:03.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.335+0000 7fb5237fe700 1 -- 172.21.15.92:0/2455745162 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6556+0+0 (secure 0 0 0) 0x7fb51c0870a0 con 0x7fb52c06a7b0 2023-11-28T11:25:03.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.335+0000 7fb523fff700 1 --2- 172.21.15.92:0/2455745162 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb50c0682f0 0x7fb50c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:03.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.335+0000 7fb523fff700 1 --2- 172.21.15.92:0/2455745162 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb50c0682f0 0x7fb50c06a7a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb52c06e5e0 tx=0x7fb514008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:03.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.339+0000 7fb5237fe700 1 -- 172.21.15.92:0/2455745162 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb51c053620 con 0x7fb52c06a7b0 2023-11-28T11:25:03.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.573+0000 7fb5313ff700 1 -- 172.21.15.92:0/2455745162 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fb52c067880 con 0x7fb52c06a7b0 2023-11-28T11:25:03.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.573+0000 7fb5237fe700 1 -- 172.21.15.92:0/2455745162 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7fb51c015080 con 0x7fb52c06a7b0 2023-11-28T11:25:03.574 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:25:03.574 INFO:teuthology.orchestra.run.smithi092.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T11:25:03.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.575+0000 7fb5313ff700 1 -- 172.21.15.92:0/2455745162 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb50c0682f0 msgr2=0x7fb50c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:03.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.575+0000 7fb5313ff700 1 --2- 172.21.15.92:0/2455745162 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb50c0682f0 0x7fb50c06a7a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb52c06e5e0 tx=0x7fb514008040 comp rx=0 tx=0).stop 2023-11-28T11:25:03.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.575+0000 7fb5313ff700 1 -- 172.21.15.92:0/2455745162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb52c06a7b0 msgr2=0x7fb52c0c60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:03.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.575+0000 7fb5313ff700 1 --2- 172.21.15.92:0/2455745162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb52c06a7b0 0x7fb52c0c60c0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fb51c014040 tx=0x7fb51c004750 comp rx=0 tx=0).stop 2023-11-28T11:25:03.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.576+0000 7fb5313ff700 1 -- 172.21.15.92:0/2455745162 shutdown_connections 2023-11-28T11:25:03.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.576+0000 7fb5313ff700 1 --2- 172.21.15.92:0/2455745162 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb52c142e80 0x7fb52c0c3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:03.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.576+0000 7fb5313ff700 1 --2- 172.21.15.92:0/2455745162 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb50c0682f0 0x7fb50c06a7a0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:03.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.576+0000 7fb5313ff700 1 --2- 172.21.15.92:0/2455745162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb52c06a7b0 0x7fb52c0c60c0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:03.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.576+0000 7fb5313ff700 1 --2- 172.21.15.92:0/2455745162 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb52c0ca430 0x7fb52c156ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:03.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.576+0000 7fb5313ff700 1 -- 172.21.15.92:0/2455745162 >> 172.21.15.92:0/2455745162 conn(0x7fb52c0cb490 msgr2=0x7fb52c0afbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:03.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.576+0000 7fb5313ff700 1 -- 172.21.15.92:0/2455745162 shutdown_connections 2023-11-28T11:25:03.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:03.576+0000 7fb5313ff700 1 -- 172.21.15.92:0/2455745162 wait complete. 2023-11-28T11:25:03.594 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=20 retry_count=0)... 2023-11-28T11:25:08.596 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T11:25:08.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:08.797+0000 7f4d94dff700 1 Processor -- start 2023-11-28T11:25:08.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:08.798+0000 7f4d94dff700 1 -- start start 2023-11-28T11:25:08.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:08.799+0000 7f4d94dff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4d900b8510 0x7f4d9006b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:08.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:08.799+0000 7f4d94dff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d900cd460 0x7f4d9006beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:08.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:08.799+0000 7f4d94dff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d900b88e0 con 0x7f4d9006c3f0 2023-11-28T11:25:08.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:08.799+0000 7f4d94dff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d900cdbd0 con 0x7f4d900b8510 2023-11-28T11:25:08.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:08.799+0000 7f4d94dff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d900d02d0 con 0x7f4d900cd460 2023-11-28T11:25:08.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:08.799+0000 7f4d87fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d900cd460 0x7f4d9006beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:08.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:08.799+0000 7f4d8e59c700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4d900b8510 0x7f4d9006b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:08.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:08.799+0000 7f4d87fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d900cd460 0x7f4d9006beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.92:46304/0 (socket says 172.21.15.92:46304) 2023-11-28T11:25:08.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:08.799+0000 7f4d87fff700 1 -- 172.21.15.92:0/911750835 learned_addr learned my addr 172.21.15.92:0/911750835 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:08.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:08.800+0000 7f4d87fff700 1 -- 172.21.15.92:0/911750835 >> v2:172.21.15.169:3300/0 conn(0x7f4d900b8510 msgr2=0x7f4d9006b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:08.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:08.800+0000 7f4d87fff700 1 --2- 172.21.15.92:0/911750835 >> v2:172.21.15.169:3300/0 conn(0x7f4d900b8510 0x7f4d9006b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:08.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:08.800+0000 7f4d87fff700 1 -- 172.21.15.92:0/911750835 >> v1:172.21.15.169:6789/0 conn(0x7f4d9006c3f0 legacy=0x7f4d9015cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:08.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:08.800+0000 7f4d87fff700 1 -- 172.21.15.92:0/911750835 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d900cd8f0 con 0x7f4d900cd460 2023-11-28T11:25:08.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:08.800+0000 7f4d87fff700 1 --2- 172.21.15.92:0/911750835 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d900cd460 0x7f4d9006beb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4d780098f0 tx=0x7f4d78009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b06309a7aa97988 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:08.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:08.801+0000 7f4d8dd9b700 1 -- 172.21.15.92:0/911750835 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d78011830 con 0x7f4d900cd460 2023-11-28T11:25:08.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:08.801+0000 7f4d8dd9b700 1 -- 172.21.15.92:0/911750835 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d78011990 con 0x7f4d900cd460 2023-11-28T11:25:08.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:08.801+0000 7f4d8dd9b700 1 -- 172.21.15.92:0/911750835 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d78011d20 con 0x7f4d900cd460 2023-11-28T11:25:08.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:08.801+0000 7f4d94dff700 1 -- 172.21.15.92:0/911750835 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d900cd460 msgr2=0x7f4d9006beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:08.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:08.801+0000 7f4d94dff700 1 --2- 172.21.15.92:0/911750835 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d900cd460 0x7f4d9006beb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4d780098f0 tx=0x7f4d78009c00 comp rx=0 tx=0).stop 2023-11-28T11:25:08.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:08.801+0000 7f4d94dff700 1 -- 172.21.15.92:0/911750835 shutdown_connections 2023-11-28T11:25:08.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:08.801+0000 7f4d94dff700 1 --2- 172.21.15.92:0/911750835 >> v2:172.21.15.169:3300/0 conn(0x7f4d900b8510 0x7f4d9006b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:08.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:08.801+0000 7f4d94dff700 1 --2- 172.21.15.92:0/911750835 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d900cd460 0x7f4d9006beb0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:08.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:08.801+0000 7f4d94dff700 1 -- 172.21.15.92:0/911750835 >> 172.21.15.92:0/911750835 conn(0x7f4d900719a0 msgr2=0x7f4d90071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:08.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:08.801+0000 7f4d94dff700 1 -- 172.21.15.92:0/911750835 shutdown_connections 2023-11-28T11:25:08.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:08.801+0000 7f4d94dff700 1 -- 172.21.15.92:0/911750835 wait complete. 2023-11-28T11:25:08.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:08.803+0000 7f4d94dff700 1 Processor -- start 2023-11-28T11:25:08.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:08.803+0000 7f4d94dff700 1 -- start start 2023-11-28T11:25:08.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:08.804+0000 7f4d94dff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d9006c3f0 0x7f4d901575c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:08.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:08.804+0000 7f4d94dff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d900b8510 0x7f4d90155c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:08.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:08.804+0000 7f4d94dff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d900cd460 0x7f4d900b68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:08.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:08.804+0000 7f4d94dff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d901580b0 con 0x7f4d9006c3f0 2023-11-28T11:25:08.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:08.804+0000 7f4d94dff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d90157b00 con 0x7f4d900b8510 2023-11-28T11:25:08.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:08.804+0000 7f4d94dff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d90157dd0 con 0x7f4d900cd460 2023-11-28T11:25:08.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:08.804+0000 7f4d8e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d9006c3f0 0x7f4d901575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:08.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:08.804+0000 7f4d8e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d9006c3f0 0x7f4d901575c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46104/0 (socket says 172.21.15.92:46104) 2023-11-28T11:25:08.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:08.804+0000 7f4d8e59c700 1 -- 172.21.15.92:0/1329069890 learned_addr learned my addr 172.21.15.92:0/1329069890 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:08.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:08.804+0000 7f4d87fff700 1 --2- 172.21.15.92:0/1329069890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d900b8510 0x7f4d90155c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:08.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:08.804+0000 7f4d8ed9d700 1 --2- 172.21.15.92:0/1329069890 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d900cd460 0x7f4d900b68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:08.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:08.805+0000 7f4d8e59c700 1 -- 172.21.15.92:0/1329069890 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d900cd460 msgr2=0x7f4d900b68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:08.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:08.805+0000 7f4d8e59c700 1 --2- 172.21.15.92:0/1329069890 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d900cd460 0x7f4d900b68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:08.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:08.805+0000 7f4d8e59c700 1 -- 172.21.15.92:0/1329069890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d900b8510 msgr2=0x7f4d90155c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:08.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:08.805+0000 7f4d8e59c700 1 --2- 172.21.15.92:0/1329069890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d900b8510 0x7f4d90155c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:08.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:08.805+0000 7f4d8e59c700 1 -- 172.21.15.92:0/1329069890 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d78009550 con 0x7f4d9006c3f0 2023-11-28T11:25:08.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:08.805+0000 7f4d8e59c700 1 --2- 172.21.15.92:0/1329069890 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d9006c3f0 0x7f4d901575c0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f4d8000e9b0 tx=0x7f4d8000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:08.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:08.805+0000 7f4d86ffd700 1 -- 172.21.15.92:0/1329069890 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d80009710 con 0x7f4d9006c3f0 2023-11-28T11:25:08.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:08.805+0000 7f4d86ffd700 1 -- 172.21.15.92:0/1329069890 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d8000ee50 con 0x7f4d9006c3f0 2023-11-28T11:25:08.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:08.805+0000 7f4d94dff700 1 -- 172.21.15.92:0/1329069890 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d900cfff0 con 0x7f4d9006c3f0 2023-11-28T11:25:08.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:08.805+0000 7f4d86ffd700 1 -- 172.21.15.92:0/1329069890 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8000ca50 con 0x7f4d9006c3f0 2023-11-28T11:25:08.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:08.805+0000 7f4d94dff700 1 -- 172.21.15.92:0/1329069890 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d900cfd10 con 0x7f4d9006c3f0 2023-11-28T11:25:08.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:08.806+0000 7f4d94dff700 1 -- 172.21.15.92:0/1329069890 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d900cb2c0 con 0x7f4d9006c3f0 2023-11-28T11:25:08.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:08.807+0000 7f4d86ffd700 1 -- 172.21.15.92:0/1329069890 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4d80023070 con 0x7f4d9006c3f0 2023-11-28T11:25:08.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:08.808+0000 7f4d86ffd700 1 --2- 172.21.15.92:0/1329069890 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f4d700682f0 0x7f4d7006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:08.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:08.808+0000 7f4d86ffd700 1 -- 172.21.15.92:0/1329069890 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6556+0+0 (secure 0 0 0) 0x7f4d80086120 con 0x7f4d9006c3f0 2023-11-28T11:25:08.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:08.808+0000 7f4d87fff700 1 --2- 172.21.15.92:0/1329069890 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f4d700682f0 0x7f4d7006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:08.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:08.809+0000 7f4d87fff700 1 --2- 172.21.15.92:0/1329069890 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f4d700682f0 0x7f4d7006a7a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4d78009c40 tx=0x7f4d78008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:08.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:08.812+0000 7f4d86ffd700 1 -- 172.21.15.92:0/1329069890 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4d900cb2c0 con 0x7f4d9006c3f0 2023-11-28T11:25:09.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:09.044+0000 7f4d94dff700 1 -- 172.21.15.92:0/1329069890 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f4d900cb2c0 con 0x7f4d9006c3f0 2023-11-28T11:25:09.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:09.045+0000 7f4d86ffd700 1 -- 172.21.15.92:0/1329069890 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f4d900cb2c0 con 0x7f4d9006c3f0 2023-11-28T11:25:09.046 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:25:09.046 INFO:teuthology.orchestra.run.smithi092.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T11:25:09.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:09.047+0000 7f4d94dff700 1 -- 172.21.15.92:0/1329069890 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f4d700682f0 msgr2=0x7f4d7006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:09.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:09.047+0000 7f4d94dff700 1 --2- 172.21.15.92:0/1329069890 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f4d700682f0 0x7f4d7006a7a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4d78009c40 tx=0x7f4d78008030 comp rx=0 tx=0).stop 2023-11-28T11:25:09.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:09.047+0000 7f4d94dff700 1 -- 172.21.15.92:0/1329069890 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d9006c3f0 msgr2=0x7f4d901575c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:09.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:09.047+0000 7f4d94dff700 1 --2- 172.21.15.92:0/1329069890 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d9006c3f0 0x7f4d901575c0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f4d8000e9b0 tx=0x7f4d8000ecc0 comp rx=0 tx=0).stop 2023-11-28T11:25:09.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:09.047+0000 7f4d94dff700 1 -- 172.21.15.92:0/1329069890 shutdown_connections 2023-11-28T11:25:09.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:09.047+0000 7f4d94dff700 1 --2- 172.21.15.92:0/1329069890 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4d900cd460 0x7f4d900b68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:09.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:09.047+0000 7f4d94dff700 1 --2- 172.21.15.92:0/1329069890 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f4d700682f0 0x7f4d7006a7a0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:09.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:09.047+0000 7f4d94dff700 1 --2- 172.21.15.92:0/1329069890 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d9006c3f0 0x7f4d901575c0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:09.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:09.047+0000 7f4d94dff700 1 --2- 172.21.15.92:0/1329069890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4d900b8510 0x7f4d90155c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:09.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:09.047+0000 7f4d94dff700 1 -- 172.21.15.92:0/1329069890 >> 172.21.15.92:0/1329069890 conn(0x7f4d900719a0 msgr2=0x7f4d900b1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:09.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:09.047+0000 7f4d94dff700 1 -- 172.21.15.92:0/1329069890 shutdown_connections 2023-11-28T11:25:09.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:09.047+0000 7f4d94dff700 1 -- 172.21.15.92:0/1329069890 wait complete. 2023-11-28T11:25:09.066 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=20 retry_count=0)... 2023-11-28T11:25:14.068 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T11:25:14.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.285+0000 7f998d1ff700 1 Processor -- start 2023-11-28T11:25:14.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.285+0000 7f998d1ff700 1 -- start start 2023-11-28T11:25:14.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.286+0000 7f998d1ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99880b8510 0x7f998806b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:14.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.286+0000 7f998d1ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99880b88e0 con 0x7f99880d0d60 2023-11-28T11:25:14.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.286+0000 7f998d1ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99880cd8f0 con 0x7f99880cd460 2023-11-28T11:25:14.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.286+0000 7f998d1ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99880cdbd0 con 0x7f99880b8510 2023-11-28T11:25:14.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.286+0000 7f998759e700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f99880d0d60 0x7f99880d11a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:45394/0 (socket says 172.21.15.92:45394) 2023-11-28T11:25:14.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.286+0000 7f998759e700 1 -- 172.21.15.92:0/3046480472 learned_addr learned my addr 172.21.15.92:0/3046480472 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:25:14.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.286+0000 7f998659c700 1 -- 172.21.15.92:0/3046480472 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 745502051 0 0) 0x7f99880b88e0 con 0x7f99880d0d60 2023-11-28T11:25:14.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.287+0000 7f9986d9d700 1 --2- 172.21.15.92:0/3046480472 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99880b8510 0x7f998806b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:14.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.287+0000 7f998659c700 1 -- 172.21.15.92:0/3046480472 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99740035b0 con 0x7f99880d0d60 2023-11-28T11:25:14.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.287+0000 7f998659c700 1 -- 172.21.15.92:0/3046480472 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3758227057 0 0) 0x7f99740035b0 con 0x7f99880d0d60 2023-11-28T11:25:14.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.287+0000 7f998659c700 1 -- 172.21.15.92:0/3046480472 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99880b8510 msgr2=0x7f998806b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:14.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.287+0000 7f998659c700 1 --2- 172.21.15.92:0/3046480472 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99880b8510 0x7f998806b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:14.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.287+0000 7f998659c700 1 -- 172.21.15.92:0/3046480472 >> v1:172.21.15.169:6789/0 conn(0x7f99880cd460 legacy=0x7f998806beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:14.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.287+0000 7f998659c700 1 -- 172.21.15.92:0/3046480472 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99880cfd10 con 0x7f99880d0d60 2023-11-28T11:25:14.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.287+0000 7f998659c700 1 -- 172.21.15.92:0/3046480472 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532161587 0 0) 0x7f99800019f0 con 0x7f99880d0d60 2023-11-28T11:25:14.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.287+0000 7f998659c700 1 -- 172.21.15.92:0/3046480472 >> v1:172.21.15.92:6789/0 conn(0x7f99880d0d60 legacy=0x7f99880d11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:14.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.287+0000 7f998659c700 1 --2- 172.21.15.92:0/3046480472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9974003a30 0x7f9974003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:14.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.287+0000 7f998659c700 1 --2- 172.21.15.92:0/3046480472 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99880b8510 0x7f9974004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:14.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.287+0000 7f998659c700 1 --2- 172.21.15.92:0/3046480472 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9974004d90 0x7f9974005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:14.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.287+0000 7f998659c700 1 -- 172.21.15.92:0/3046480472 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99880cdbd0 con 0x7f9974003a30 2023-11-28T11:25:14.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.287+0000 7f998659c700 1 -- 172.21.15.92:0/3046480472 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997400baf0 con 0x7f9974004d90 2023-11-28T11:25:14.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.287+0000 7f998659c700 1 -- 172.21.15.92:0/3046480472 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f997400bcb0 con 0x7f99880b8510 2023-11-28T11:25:14.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.287+0000 7f9986d9d700 1 --2- 172.21.15.92:0/3046480472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9974003a30 0x7f9974003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:14.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.288+0000 7f997ffff700 1 --2- 172.21.15.92:0/3046480472 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99880b8510 0x7f9974004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:14.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.288+0000 7f998759e700 1 --2- 172.21.15.92:0/3046480472 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9974004d90 0x7f9974005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:14.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.288+0000 7f9986d9d700 1 -- 172.21.15.92:0/3046480472 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99880b8510 msgr2=0x7f9974004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:14.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.288+0000 7f9986d9d700 1 --2- 172.21.15.92:0/3046480472 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f99880b8510 0x7f9974004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:14.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.288+0000 7f9986d9d700 1 -- 172.21.15.92:0/3046480472 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9974004d90 msgr2=0x7f9974005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:14.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.288+0000 7f9986d9d700 1 --2- 172.21.15.92:0/3046480472 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9974004d90 0x7f9974005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:14.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.288+0000 7f9986d9d700 1 -- 172.21.15.92:0/3046480472 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99740035b0 con 0x7f9974003a30 2023-11-28T11:25:14.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.288+0000 7f998759e700 1 --2- 172.21.15.92:0/3046480472 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9974004d90 0x7f9974005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:25:14.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.288+0000 7f9986d9d700 1 --2- 172.21.15.92:0/3046480472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9974003a30 0x7f9974003e40 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f9978009b60 tx=0x7f9974005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=379e0428efceeb33 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:14.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.288+0000 7f998659c700 1 -- 172.21.15.92:0/3046480472 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997800e6c0 con 0x7f9974003a30 2023-11-28T11:25:14.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.288+0000 7f998659c700 1 -- 172.21.15.92:0/3046480472 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f997800e820 con 0x7f9974003a30 2023-11-28T11:25:14.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.288+0000 7f998659c700 1 -- 172.21.15.92:0/3046480472 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997800eb90 con 0x7f9974003a30 2023-11-28T11:25:14.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.289+0000 7f998d1ff700 1 -- 172.21.15.92:0/3046480472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9974003a30 msgr2=0x7f9974003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:14.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.289+0000 7f998d1ff700 1 --2- 172.21.15.92:0/3046480472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9974003a30 0x7f9974003e40 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f9978009b60 tx=0x7f9974005880 comp rx=0 tx=0).stop 2023-11-28T11:25:14.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.289+0000 7f998759e700 1 -- 172.21.15.92:0/3046480472 reap_dead start 2023-11-28T11:25:14.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.289+0000 7f998d1ff700 1 -- 172.21.15.92:0/3046480472 shutdown_connections 2023-11-28T11:25:14.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.289+0000 7f998d1ff700 1 -- 172.21.15.92:0/3046480472 >> 172.21.15.92:0/3046480472 conn(0x7f99880719a0 msgr2=0x7f9988071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:14.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.289+0000 7f998d1ff700 1 -- 172.21.15.92:0/3046480472 shutdown_connections 2023-11-28T11:25:14.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.289+0000 7f998d1ff700 1 -- 172.21.15.92:0/3046480472 wait complete. 2023-11-28T11:25:14.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.291+0000 7f998d1ff700 1 Processor -- start 2023-11-28T11:25:14.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.291+0000 7f998d1ff700 1 -- start start 2023-11-28T11:25:14.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.292+0000 7f998d1ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9974003a30 0x7f998815ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:14.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.292+0000 7f998d1ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f998815a6d0 0x7f99881e4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:14.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.292+0000 7f998d1ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99881e29f0 0x7f99881e7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:14.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.292+0000 7f998d1ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99881e3e30 con 0x7f99881e29f0 2023-11-28T11:25:14.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.292+0000 7f998d1ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99881e4790 con 0x7f9974003a30 2023-11-28T11:25:14.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.292+0000 7f998d1ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f998815ab40 con 0x7f998815a6d0 2023-11-28T11:25:14.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.292+0000 7f998759e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99881e29f0 0x7f99881e7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:14.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.292+0000 7f998759e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99881e29f0 0x7f99881e7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34676/0 (socket says 172.21.15.92:34676) 2023-11-28T11:25:14.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.292+0000 7f998759e700 1 -- 172.21.15.92:0/60048792 learned_addr learned my addr 172.21.15.92:0/60048792 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:14.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.292+0000 7f997ffff700 1 --2- 172.21.15.92:0/60048792 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f998815a6d0 0x7f99881e4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:14.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.292+0000 7f9986d9d700 1 --2- 172.21.15.92:0/60048792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9974003a30 0x7f998815ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:14.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.292+0000 7f998759e700 1 -- 172.21.15.92:0/60048792 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f998815a6d0 msgr2=0x7f99881e4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:14.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.292+0000 7f998759e700 1 --2- 172.21.15.92:0/60048792 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f998815a6d0 0x7f99881e4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:14.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.292+0000 7f998759e700 1 -- 172.21.15.92:0/60048792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9974003a30 msgr2=0x7f998815ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:14.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.292+0000 7f998759e700 1 --2- 172.21.15.92:0/60048792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9974003a30 0x7f998815ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:14.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.292+0000 7f998759e700 1 -- 172.21.15.92:0/60048792 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99880d29c0 con 0x7f99881e29f0 2023-11-28T11:25:14.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.292+0000 7f9986d9d700 1 --2- 172.21.15.92:0/60048792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9974003a30 0x7f998815ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:25:14.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.293+0000 7f998759e700 1 --2- 172.21.15.92:0/60048792 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99881e29f0 0x7f99881e7700 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f99880cdd70 tx=0x7f998000e4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:14.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.293+0000 7f997f7fe700 1 -- 172.21.15.92:0/60048792 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998000ed20 con 0x7f99881e29f0 2023-11-28T11:25:14.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.293+0000 7f997f7fe700 1 -- 172.21.15.92:0/60048792 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f998000eec0 con 0x7f99881e29f0 2023-11-28T11:25:14.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.293+0000 7f998d1ff700 1 -- 172.21.15.92:0/60048792 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99880cfff0 con 0x7f99881e29f0 2023-11-28T11:25:14.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.293+0000 7f997f7fe700 1 -- 172.21.15.92:0/60048792 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9980013660 con 0x7f99881e29f0 2023-11-28T11:25:14.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.293+0000 7f998d1ff700 1 -- 172.21.15.92:0/60048792 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99880d2b50 con 0x7f99881e29f0 2023-11-28T11:25:14.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.294+0000 7f998d1ff700 1 -- 172.21.15.92:0/60048792 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99881e3f70 con 0x7f99881e29f0 2023-11-28T11:25:14.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.295+0000 7f997f7fe700 1 -- 172.21.15.92:0/60048792 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f99800137c0 con 0x7f99881e29f0 2023-11-28T11:25:14.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.296+0000 7f997f7fe700 1 --2- 172.21.15.92:0/60048792 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f99680682f0 0x7f996806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:14.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.296+0000 7f997f7fe700 1 -- 172.21.15.92:0/60048792 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6556+0+0 (secure 0 0 0) 0x7f9980086770 con 0x7f99881e29f0 2023-11-28T11:25:14.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.296+0000 7f9986d9d700 1 --2- 172.21.15.92:0/60048792 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f99680682f0 0x7f996806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:14.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.297+0000 7f9986d9d700 1 --2- 172.21.15.92:0/60048792 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f99680682f0 0x7f996806a7a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f99880cd430 tx=0x7f9978008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:14.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.300+0000 7f997f7fe700 1 -- 172.21.15.92:0/60048792 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f99881e3f70 con 0x7f99881e29f0 2023-11-28T11:25:14.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.531+0000 7f998d1ff700 1 -- 172.21.15.92:0/60048792 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f99881e3f70 con 0x7f99881e29f0 2023-11-28T11:25:14.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.531+0000 7f997f7fe700 1 -- 172.21.15.92:0/60048792 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f99881e3f70 con 0x7f99881e29f0 2023-11-28T11:25:14.532 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:25:14.532 INFO:teuthology.orchestra.run.smithi092.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T11:25:14.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.533+0000 7f998d1ff700 1 -- 172.21.15.92:0/60048792 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f99680682f0 msgr2=0x7f996806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:14.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.533+0000 7f998d1ff700 1 --2- 172.21.15.92:0/60048792 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f99680682f0 0x7f996806a7a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f99880cd430 tx=0x7f9978008030 comp rx=0 tx=0).stop 2023-11-28T11:25:14.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.533+0000 7f998d1ff700 1 -- 172.21.15.92:0/60048792 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99881e29f0 msgr2=0x7f99881e7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:14.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.533+0000 7f998d1ff700 1 --2- 172.21.15.92:0/60048792 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99881e29f0 0x7f99881e7700 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f99880cdd70 tx=0x7f998000e4f0 comp rx=0 tx=0).stop 2023-11-28T11:25:14.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.534+0000 7f998d1ff700 1 -- 172.21.15.92:0/60048792 shutdown_connections 2023-11-28T11:25:14.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.534+0000 7f998d1ff700 1 --2- 172.21.15.92:0/60048792 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f99680682f0 0x7f996806a7a0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:14.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.534+0000 7f998d1ff700 1 --2- 172.21.15.92:0/60048792 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99881e29f0 0x7f99881e7700 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:14.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.534+0000 7f998d1ff700 1 --2- 172.21.15.92:0/60048792 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f9974003a30 0x7f998815ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:14.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.534+0000 7f998d1ff700 1 --2- 172.21.15.92:0/60048792 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f998815a6d0 0x7f99881e4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:14.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.534+0000 7f998d1ff700 1 -- 172.21.15.92:0/60048792 >> 172.21.15.92:0/60048792 conn(0x7f99880719a0 msgr2=0x7f99880b5ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:14.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.534+0000 7f998d1ff700 1 -- 172.21.15.92:0/60048792 shutdown_connections 2023-11-28T11:25:14.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:14.534+0000 7f998d1ff700 1 -- 172.21.15.92:0/60048792 wait complete. 2023-11-28T11:25:14.554 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=20 retry_count=0)... 2023-11-28T11:25:18.921 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:25:18.926 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:25:19.555 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T11:25:19.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.757+0000 7f5b089ff700 1 Processor -- start 2023-11-28T11:25:19.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.757+0000 7f5b089ff700 1 -- start start 2023-11-28T11:25:19.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.758+0000 7f5b089ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5b040b8510 0x7f5b0406b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:19.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.758+0000 7f5b089ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b040cd460 0x7f5b0406beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:19.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.758+0000 7f5b089ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b040b88e0 con 0x7f5b0406c3f0 2023-11-28T11:25:19.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.758+0000 7f5b089ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b040cdbd0 con 0x7f5b040b8510 2023-11-28T11:25:19.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.758+0000 7f5b089ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b040d02d0 con 0x7f5b040cd460 2023-11-28T11:25:19.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.759+0000 7f5b02d9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f5b0406c3f0 0x7f5b0415cc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:58950/0 (socket says 172.21.15.92:58950) 2023-11-28T11:25:19.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.759+0000 7f5b02d9d700 1 -- 172.21.15.92:0/389592670 learned_addr learned my addr 172.21.15.92:0/389592670 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:25:19.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.759+0000 7f5afbfff700 1 --2- 172.21.15.92:0/389592670 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b040cd460 0x7f5b0406beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:19.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.759+0000 7f5b0259c700 1 --2- 172.21.15.92:0/389592670 >> v2:172.21.15.169:3300/0 conn(0x7f5b040b8510 0x7f5b0406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:19.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.759+0000 7f5b01d9b700 1 -- 172.21.15.92:0/389592670 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1482744891 0 0) 0x7f5b040b88e0 con 0x7f5b0406c3f0 2023-11-28T11:25:19.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.759+0000 7f5b01d9b700 1 -- 172.21.15.92:0/389592670 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5af00035b0 con 0x7f5b0406c3f0 2023-11-28T11:25:19.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.759+0000 7f5b01d9b700 1 -- 172.21.15.92:0/389592670 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3051668384 0 0) 0x7f5af00035b0 con 0x7f5b0406c3f0 2023-11-28T11:25:19.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.759+0000 7f5b01d9b700 1 -- 172.21.15.92:0/389592670 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b040cd460 msgr2=0x7f5b0406beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:19.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.759+0000 7f5b01d9b700 1 --2- 172.21.15.92:0/389592670 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b040cd460 0x7f5b0406beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:19.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.759+0000 7f5b01d9b700 1 -- 172.21.15.92:0/389592670 >> v2:172.21.15.169:3300/0 conn(0x7f5b040b8510 msgr2=0x7f5b0406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:19.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.759+0000 7f5b01d9b700 1 --2- 172.21.15.92:0/389592670 >> v2:172.21.15.169:3300/0 conn(0x7f5b040b8510 0x7f5b0406b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:19.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.759+0000 7f5b01d9b700 1 -- 172.21.15.92:0/389592670 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b040cd8f0 con 0x7f5b0406c3f0 2023-11-28T11:25:19.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.759+0000 7f5b01d9b700 1 -- 172.21.15.92:0/389592670 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532161587 0 0) 0x7f5afc0019f0 con 0x7f5b0406c3f0 2023-11-28T11:25:19.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.759+0000 7f5b01d9b700 1 -- 172.21.15.92:0/389592670 >> v1:172.21.15.92:6789/0 conn(0x7f5b0406c3f0 legacy=0x7f5b0415cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:19.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.759+0000 7f5b01d9b700 1 --2- 172.21.15.92:0/389592670 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5af0003a30 0x7f5af0003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:19.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.759+0000 7f5b01d9b700 1 --2- 172.21.15.92:0/389592670 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5af0004850 0x7f5af0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:19.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.759+0000 7f5b01d9b700 1 --2- 172.21.15.92:0/389592670 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b040cd460 0x7f5af0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:19.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.760+0000 7f5b01d9b700 1 -- 172.21.15.92:0/389592670 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b040cdbd0 con 0x7f5af0004850 2023-11-28T11:25:19.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.760+0000 7f5b01d9b700 1 -- 172.21.15.92:0/389592670 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b040d02d0 con 0x7f5af0003a30 2023-11-28T11:25:19.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.760+0000 7f5b01d9b700 1 -- 172.21.15.92:0/389592670 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5af000ba10 con 0x7f5b040cd460 2023-11-28T11:25:19.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.760+0000 7f5afbfff700 1 --2- 172.21.15.92:0/389592670 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5af0004850 0x7f5af0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:19.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.760+0000 7f5b0259c700 1 --2- 172.21.15.92:0/389592670 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5af0003a30 0x7f5af0003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:19.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.760+0000 7f5b02d9d700 1 --2- 172.21.15.92:0/389592670 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b040cd460 0x7f5af0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:19.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.760+0000 7f5afbfff700 1 -- 172.21.15.92:0/389592670 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b040cd460 msgr2=0x7f5af0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:19.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.760+0000 7f5afbfff700 1 --2- 172.21.15.92:0/389592670 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b040cd460 0x7f5af0005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:19.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.760+0000 7f5afbfff700 1 -- 172.21.15.92:0/389592670 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5af0003a30 msgr2=0x7f5af0003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:19.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.760+0000 7f5afbfff700 1 --2- 172.21.15.92:0/389592670 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5af0003a30 0x7f5af0003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:19.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.760+0000 7f5afbfff700 1 -- 172.21.15.92:0/389592670 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af00035b0 con 0x7f5af0004850 2023-11-28T11:25:19.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.760+0000 7f5b02d9d700 1 --2- 172.21.15.92:0/389592670 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b040cd460 0x7f5af0005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:25:19.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.760+0000 7f5afbfff700 1 --2- 172.21.15.92:0/389592670 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5af0004850 0x7f5af0004cc0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f5aec009b60 tx=0x7f5af0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abd8d06c9f568253 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:19.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.761+0000 7f5b01d9b700 1 -- 172.21.15.92:0/389592670 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aec00e6c0 con 0x7f5af0004850 2023-11-28T11:25:19.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.761+0000 7f5b01d9b700 1 -- 172.21.15.92:0/389592670 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aec00e820 con 0x7f5af0004850 2023-11-28T11:25:19.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.761+0000 7f5b01d9b700 1 -- 172.21.15.92:0/389592670 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aec00eb90 con 0x7f5af0004850 2023-11-28T11:25:19.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.761+0000 7f5b089ff700 1 -- 172.21.15.92:0/389592670 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5af0004850 msgr2=0x7f5af0004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:19.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.761+0000 7f5b089ff700 1 --2- 172.21.15.92:0/389592670 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5af0004850 0x7f5af0004cc0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f5aec009b60 tx=0x7f5af0005880 comp rx=0 tx=0).stop 2023-11-28T11:25:19.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.761+0000 7f5b02d9d700 1 -- 172.21.15.92:0/389592670 reap_dead start 2023-11-28T11:25:19.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.761+0000 7f5b089ff700 1 -- 172.21.15.92:0/389592670 shutdown_connections 2023-11-28T11:25:19.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.761+0000 7f5b089ff700 1 -- 172.21.15.92:0/389592670 >> 172.21.15.92:0/389592670 conn(0x7f5b040719a0 msgr2=0x7f5b04071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:19.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.761+0000 7f5b089ff700 1 -- 172.21.15.92:0/389592670 shutdown_connections 2023-11-28T11:25:19.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.761+0000 7f5b089ff700 1 -- 172.21.15.92:0/389592670 wait complete. 2023-11-28T11:25:19.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.763+0000 7f5b089ff700 1 Processor -- start 2023-11-28T11:25:19.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.763+0000 7f5b089ff700 1 -- start start 2023-11-28T11:25:19.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.764+0000 7f5b089ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5af0004850 0x7f5b04156350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:19.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.764+0000 7f5b089ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b040cb8e0 0x7f5b040cbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:19.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.764+0000 7f5b089ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b041f2390 0x7f5b041f2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:19.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.764+0000 7f5b089ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b040cc5a0 con 0x7f5b041f2390 2023-11-28T11:25:19.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.764+0000 7f5b089ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b040cc880 con 0x7f5af0004850 2023-11-28T11:25:19.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.764+0000 7f5b089ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b040ccb60 con 0x7f5b040cb8e0 2023-11-28T11:25:19.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.764+0000 7f5b02d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b041f2390 0x7f5b041f2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:19.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.764+0000 7f5b02d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b041f2390 0x7f5b041f2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44556/0 (socket says 172.21.15.92:44556) 2023-11-28T11:25:19.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.764+0000 7f5b02d9d700 1 -- 172.21.15.92:0/2447541576 learned_addr learned my addr 172.21.15.92:0/2447541576 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:19.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.765+0000 7f5b0259c700 1 --2- 172.21.15.92:0/2447541576 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5af0004850 0x7f5b04156350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:19.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.765+0000 7f5afbfff700 1 --2- 172.21.15.92:0/2447541576 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b040cb8e0 0x7f5b040cbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:19.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.765+0000 7f5b02d9d700 1 -- 172.21.15.92:0/2447541576 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b040cb8e0 msgr2=0x7f5b040cbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:19.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.765+0000 7f5b02d9d700 1 --2- 172.21.15.92:0/2447541576 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b040cb8e0 0x7f5b040cbd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:19.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.765+0000 7f5b02d9d700 1 -- 172.21.15.92:0/2447541576 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5af0004850 msgr2=0x7f5b04156350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:19.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.765+0000 7f5b02d9d700 1 --2- 172.21.15.92:0/2447541576 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5af0004850 0x7f5b04156350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:19.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.765+0000 7f5b02d9d700 1 -- 172.21.15.92:0/2447541576 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b040cfff0 con 0x7f5b041f2390 2023-11-28T11:25:19.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.765+0000 7f5afbfff700 1 --2- 172.21.15.92:0/2447541576 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b040cb8e0 0x7f5b040cbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:25:19.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.765+0000 7f5b0259c700 1 --2- 172.21.15.92:0/2447541576 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5af0004850 0x7f5b04156350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:25:19.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.765+0000 7f5b02d9d700 1 --2- 172.21.15.92:0/2447541576 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b041f2390 0x7f5b041f2800 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f5afc001370 tx=0x7f5afc007c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:19.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.765+0000 7f5afaffd700 1 -- 172.21.15.92:0/2447541576 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5afc00ee20 con 0x7f5b041f2390 2023-11-28T11:25:19.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.765+0000 7f5afaffd700 1 -- 172.21.15.92:0/2447541576 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5afc013420 con 0x7f5b041f2390 2023-11-28T11:25:19.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.765+0000 7f5afaffd700 1 -- 172.21.15.92:0/2447541576 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5afc013790 con 0x7f5b041f2390 2023-11-28T11:25:19.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.765+0000 7f5b089ff700 1 -- 172.21.15.92:0/2447541576 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b040cfd10 con 0x7f5b041f2390 2023-11-28T11:25:19.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.765+0000 7f5b089ff700 1 -- 172.21.15.92:0/2447541576 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b040d29c0 con 0x7f5b041f2390 2023-11-28T11:25:19.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.766+0000 7f5b089ff700 1 -- 172.21.15.92:0/2447541576 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b04067880 con 0x7f5b041f2390 2023-11-28T11:25:19.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.767+0000 7f5afaffd700 1 -- 172.21.15.92:0/2447541576 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5afc0138f0 con 0x7f5b041f2390 2023-11-28T11:25:19.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.768+0000 7f5afaffd700 1 --2- 172.21.15.92:0/2447541576 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5ae40682f0 0x7f5ae406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:19.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.769+0000 7f5afaffd700 1 -- 172.21.15.92:0/2447541576 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6556+0+0 (secure 0 0 0) 0x7f5afc0867b0 con 0x7f5b041f2390 2023-11-28T11:25:19.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.769+0000 7f5b0259c700 1 --2- 172.21.15.92:0/2447541576 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5ae40682f0 0x7f5ae406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:19.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.769+0000 7f5b0259c700 1 --2- 172.21.15.92:0/2447541576 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5ae40682f0 0x7f5ae406a7a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5af4009bd0 tx=0x7f5af4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:19.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:19.773+0000 7f5afaffd700 1 -- 172.21.15.92:0/2447541576 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5afc052cd0 con 0x7f5b041f2390 2023-11-28T11:25:20.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.007+0000 7f5b089ff700 1 -- 172.21.15.92:0/2447541576 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f5b04072d30 con 0x7f5b041f2390 2023-11-28T11:25:20.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.008+0000 7f5afaffd700 1 -- 172.21.15.92:0/2447541576 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f5afc0562f0 con 0x7f5b041f2390 2023-11-28T11:25:20.010 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:25:20.010 INFO:teuthology.orchestra.run.smithi092.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T11:25:20.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.010+0000 7f5b089ff700 1 -- 172.21.15.92:0/2447541576 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5ae40682f0 msgr2=0x7f5ae406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:20.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.010+0000 7f5b089ff700 1 --2- 172.21.15.92:0/2447541576 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5ae40682f0 0x7f5ae406a7a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5af4009bd0 tx=0x7f5af4008040 comp rx=0 tx=0).stop 2023-11-28T11:25:20.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.010+0000 7f5b089ff700 1 -- 172.21.15.92:0/2447541576 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b041f2390 msgr2=0x7f5b041f2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:20.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.010+0000 7f5b089ff700 1 --2- 172.21.15.92:0/2447541576 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b041f2390 0x7f5b041f2800 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f5afc001370 tx=0x7f5afc007c90 comp rx=0 tx=0).stop 2023-11-28T11:25:20.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.010+0000 7f5b089ff700 1 -- 172.21.15.92:0/2447541576 shutdown_connections 2023-11-28T11:25:20.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.010+0000 7f5b089ff700 1 --2- 172.21.15.92:0/2447541576 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5ae40682f0 0x7f5ae406a7a0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:20.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.010+0000 7f5b089ff700 1 --2- 172.21.15.92:0/2447541576 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b041f2390 0x7f5b041f2800 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:20.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.010+0000 7f5b089ff700 1 --2- 172.21.15.92:0/2447541576 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5af0004850 0x7f5b04156350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:20.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.010+0000 7f5b089ff700 1 --2- 172.21.15.92:0/2447541576 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5b040cb8e0 0x7f5b040cbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:20.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.010+0000 7f5b089ff700 1 -- 172.21.15.92:0/2447541576 >> 172.21.15.92:0/2447541576 conn(0x7f5b040719a0 msgr2=0x7f5b040b4260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:20.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.010+0000 7f5b089ff700 1 -- 172.21.15.92:0/2447541576 shutdown_connections 2023-11-28T11:25:20.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.010+0000 7f5b089ff700 1 -- 172.21.15.92:0/2447541576 wait complete. 2023-11-28T11:25:20.031 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T11:25:20.031 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:25:20.133 DEBUG:teuthology.orchestra.run.smithi169:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:25:20.242 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T11:25:20.242 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-28T11:25:20.242 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:25:20.290 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-28T11:25:20.290 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:25:20.377 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:25:20.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.612+0000 7fbde57ff700 1 Processor -- start 2023-11-28T11:25:20.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.612+0000 7fbde57ff700 1 -- start start 2023-11-28T11:25:20.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.613+0000 7fbde57ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbde00b8510 0x7fbde006b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:20.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.613+0000 7fbde57ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fbde00d0d60 0x7fbde00d11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:20.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.613+0000 7fbde57ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbde00b88e0 con 0x7fbde00cd460 2023-11-28T11:25:20.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.613+0000 7fbde57ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbde00cdbd0 con 0x7fbde00d0d60 2023-11-28T11:25:20.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.613+0000 7fbde57ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbde00d02d0 con 0x7fbde00b8510 2023-11-28T11:25:20.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.613+0000 7fbdde7fc700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fbde00cd460 0x7fbde006beb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:58964/0 (socket says 172.21.15.92:58964) 2023-11-28T11:25:20.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.614+0000 7fbdde7fc700 1 -- 172.21.15.92:0/2211418890 learned_addr learned my addr 172.21.15.92:0/2211418890 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:25:20.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.614+0000 7fbddeffd700 1 --2- 172.21.15.92:0/2211418890 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbde00b8510 0x7fbde006b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:20.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.614+0000 7fbddf7fe700 1 --2- 172.21.15.92:0/2211418890 >> v2:172.21.15.169:3300/0 conn(0x7fbde00d0d60 0x7fbde00d11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:20.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.614+0000 7fbdddffb700 1 -- 172.21.15.92:0/2211418890 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1119021540 0 0) 0x7fbde00b88e0 con 0x7fbde00cd460 2023-11-28T11:25:20.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.614+0000 7fbdddffb700 1 -- 172.21.15.92:0/2211418890 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdc80035b0 con 0x7fbde00cd460 2023-11-28T11:25:20.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.614+0000 7fbdddffb700 1 -- 172.21.15.92:0/2211418890 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3265544074 0 0) 0x7fbdc80035b0 con 0x7fbde00cd460 2023-11-28T11:25:20.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.614+0000 7fbdddffb700 1 -- 172.21.15.92:0/2211418890 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbde00b8510 msgr2=0x7fbde006b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:20.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.614+0000 7fbdddffb700 1 --2- 172.21.15.92:0/2211418890 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbde00b8510 0x7fbde006b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:20.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.614+0000 7fbdddffb700 1 -- 172.21.15.92:0/2211418890 >> v2:172.21.15.169:3300/0 conn(0x7fbde00d0d60 msgr2=0x7fbde00d11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:20.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.614+0000 7fbdddffb700 1 --2- 172.21.15.92:0/2211418890 >> v2:172.21.15.169:3300/0 conn(0x7fbde00d0d60 0x7fbde00d11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:20.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.614+0000 7fbdddffb700 1 -- 172.21.15.92:0/2211418890 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde00cd8f0 con 0x7fbde00cd460 2023-11-28T11:25:20.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.614+0000 7fbdddffb700 1 -- 172.21.15.92:0/2211418890 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532161587 0 0) 0x7fbdd4001670 con 0x7fbde00cd460 2023-11-28T11:25:20.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.614+0000 7fbdddffb700 1 -- 172.21.15.92:0/2211418890 >> v1:172.21.15.92:6789/0 conn(0x7fbde00cd460 legacy=0x7fbde006beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:20.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.614+0000 7fbdddffb700 1 --2- 172.21.15.92:0/2211418890 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbdc8003a30 0x7fbdc8003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:20.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.614+0000 7fbdddffb700 1 --2- 172.21.15.92:0/2211418890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbdc8004850 0x7fbdc8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:20.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.614+0000 7fbdddffb700 1 --2- 172.21.15.92:0/2211418890 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbdc8005200 0x7fbdc800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:20.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.614+0000 7fbdddffb700 1 -- 172.21.15.92:0/2211418890 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde00cdbd0 con 0x7fbdc8003a30 2023-11-28T11:25:20.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.614+0000 7fbdddffb700 1 -- 172.21.15.92:0/2211418890 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde00d02d0 con 0x7fbdc8004850 2023-11-28T11:25:20.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.614+0000 7fbdddffb700 1 -- 172.21.15.92:0/2211418890 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc800bf50 con 0x7fbdc8005200 2023-11-28T11:25:20.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.614+0000 7fbddeffd700 1 --2- 172.21.15.92:0/2211418890 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbde00b8510 0x7fbde006b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:25:20.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.615+0000 7fbdde7fc700 1 --2- 172.21.15.92:0/2211418890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbdc8004850 0x7fbdc8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:20.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.615+0000 7fbddf7fe700 1 --2- 172.21.15.92:0/2211418890 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbdc8005200 0x7fbdc800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:20.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.615+0000 7fbddeffd700 1 --2- 172.21.15.92:0/2211418890 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbdc8003a30 0x7fbdc8003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:20.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.615+0000 7fbddf7fe700 1 -- 172.21.15.92:0/2211418890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbdc8004850 msgr2=0x7fbdc8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:20.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.615+0000 7fbddf7fe700 1 --2- 172.21.15.92:0/2211418890 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbdc8004850 0x7fbdc8004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:20.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.615+0000 7fbddf7fe700 1 -- 172.21.15.92:0/2211418890 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbdc8003a30 msgr2=0x7fbdc8003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:20.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.615+0000 7fbddf7fe700 1 --2- 172.21.15.92:0/2211418890 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbdc8003a30 0x7fbdc8003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:20.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.615+0000 7fbddf7fe700 1 -- 172.21.15.92:0/2211418890 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc80035b0 con 0x7fbdc8005200 2023-11-28T11:25:20.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.615+0000 7fbddeffd700 1 --2- 172.21.15.92:0/2211418890 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbdc8003a30 0x7fbdc8003e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:25:20.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.615+0000 7fbddf7fe700 1 --2- 172.21.15.92:0/2211418890 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbdc8005200 0x7fbdc800ba10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8009fc0 tx=0x7fbdc8005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b20f152130511f5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:20.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.616+0000 7fbdddffb700 1 -- 172.21.15.92:0/2211418890 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd800ec30 con 0x7fbdc8005200 2023-11-28T11:25:20.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.616+0000 7fbdddffb700 1 -- 172.21.15.92:0/2211418890 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdd800f040 con 0x7fbdc8005200 2023-11-28T11:25:20.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.616+0000 7fbdddffb700 1 -- 172.21.15.92:0/2211418890 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd8014480 con 0x7fbdc8005200 2023-11-28T11:25:20.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.616+0000 7fbde57ff700 1 -- 172.21.15.92:0/2211418890 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbdc8005200 msgr2=0x7fbdc800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:20.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.616+0000 7fbde57ff700 1 --2- 172.21.15.92:0/2211418890 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbdc8005200 0x7fbdc800ba10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8009fc0 tx=0x7fbdc8005880 comp rx=0 tx=0).stop 2023-11-28T11:25:20.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.616+0000 7fbddf7fe700 1 -- 172.21.15.92:0/2211418890 reap_dead start 2023-11-28T11:25:20.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.616+0000 7fbde57ff700 1 -- 172.21.15.92:0/2211418890 shutdown_connections 2023-11-28T11:25:20.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.616+0000 7fbde57ff700 1 -- 172.21.15.92:0/2211418890 >> 172.21.15.92:0/2211418890 conn(0x7fbde00719a0 msgr2=0x7fbde0071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:20.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.616+0000 7fbde57ff700 1 -- 172.21.15.92:0/2211418890 shutdown_connections 2023-11-28T11:25:20.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.616+0000 7fbde57ff700 1 -- 172.21.15.92:0/2211418890 wait complete. 2023-11-28T11:25:20.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.618+0000 7fbde57ff700 1 Processor -- start 2023-11-28T11:25:20.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.618+0000 7fbde57ff700 1 -- start start 2023-11-28T11:25:20.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.619+0000 7fbde57ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbdc8005200 0x7fbde014d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:20.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.619+0000 7fbde57ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbde0154480 0x7fbde00cc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:20.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.619+0000 7fbde57ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbde00b5cd0 0x7fbde00b6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:20.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.619+0000 7fbde57ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde015a610 con 0x7fbdc8005200 2023-11-28T11:25:20.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.619+0000 7fbde57ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde00b6680 con 0x7fbde0154480 2023-11-28T11:25:20.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.619+0000 7fbde57ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbde00cd2b0 con 0x7fbde00b5cd0 2023-11-28T11:25:20.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.619+0000 7fbddeffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbdc8005200 0x7fbde014d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:20.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.619+0000 7fbddeffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbdc8005200 0x7fbde014d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44570/0 (socket says 172.21.15.92:44570) 2023-11-28T11:25:20.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.619+0000 7fbddeffd700 1 -- 172.21.15.92:0/3430380690 learned_addr learned my addr 172.21.15.92:0/3430380690 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:20.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.620+0000 7fbdde7fc700 1 --2- 172.21.15.92:0/3430380690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbde0154480 0x7fbde00cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:20.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.620+0000 7fbddf7fe700 1 --2- 172.21.15.92:0/3430380690 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbde00b5cd0 0x7fbde00b6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:20.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.620+0000 7fbddeffd700 1 -- 172.21.15.92:0/3430380690 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbde00b5cd0 msgr2=0x7fbde00b6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:20.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.620+0000 7fbddeffd700 1 --2- 172.21.15.92:0/3430380690 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbde00b5cd0 0x7fbde00b6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:20.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.620+0000 7fbddeffd700 1 -- 172.21.15.92:0/3430380690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbde0154480 msgr2=0x7fbde00cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:20.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.620+0000 7fbddeffd700 1 --2- 172.21.15.92:0/3430380690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbde0154480 0x7fbde00cc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:20.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.620+0000 7fbddeffd700 1 -- 172.21.15.92:0/3430380690 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde00cfff0 con 0x7fbdc8005200 2023-11-28T11:25:20.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.620+0000 7fbddf7fe700 1 --2- 172.21.15.92:0/3430380690 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbde00b5cd0 0x7fbde00b6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:25:20.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.620+0000 7fbdde7fc700 1 --2- 172.21.15.92:0/3430380690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbde0154480 0x7fbde00cc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:25:20.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.620+0000 7fbddeffd700 1 --2- 172.21.15.92:0/3430380690 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbdc8005200 0x7fbde014d350 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0011c70 tx=0x7fbdd000f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:20.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.620+0000 7fbdc77fe700 1 -- 172.21.15.92:0/3430380690 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd0011e70 con 0x7fbdc8005200 2023-11-28T11:25:20.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.620+0000 7fbdc77fe700 1 -- 172.21.15.92:0/3430380690 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdd000fd40 con 0x7fbdc8005200 2023-11-28T11:25:20.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.620+0000 7fbdc77fe700 1 -- 172.21.15.92:0/3430380690 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd0014640 con 0x7fbdc8005200 2023-11-28T11:25:20.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.620+0000 7fbde57ff700 1 -- 172.21.15.92:0/3430380690 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbde00cfd10 con 0x7fbdc8005200 2023-11-28T11:25:20.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.620+0000 7fbde57ff700 1 -- 172.21.15.92:0/3430380690 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbde00d29c0 con 0x7fbdc8005200 2023-11-28T11:25:20.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.621+0000 7fbde57ff700 1 -- 172.21.15.92:0/3430380690 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbde0067880 con 0x7fbdc8005200 2023-11-28T11:25:20.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.622+0000 7fbdc77fe700 1 -- 172.21.15.92:0/3430380690 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbdd00147a0 con 0x7fbdc8005200 2023-11-28T11:25:20.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.623+0000 7fbdc77fe700 1 --2- 172.21.15.92:0/3430380690 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fbdcc0682f0 0x7fbdcc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:20.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.623+0000 7fbdc77fe700 1 -- 172.21.15.92:0/3430380690 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6556+0+0 (secure 0 0 0) 0x7fbdd0086b90 con 0x7fbdc8005200 2023-11-28T11:25:20.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.623+0000 7fbdde7fc700 1 --2- 172.21.15.92:0/3430380690 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fbdcc0682f0 0x7fbdcc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:20.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.624+0000 7fbdde7fc700 1 --2- 172.21.15.92:0/3430380690 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fbdcc0682f0 0x7fbdcc06a7a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fbde00d3e80 tx=0x7fbdd4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:20.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.627+0000 7fbdc77fe700 1 -- 172.21.15.92:0/3430380690 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbdd0053110 con 0x7fbdc8005200 2023-11-28T11:25:20.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.820+0000 7fbde57ff700 1 -- 172.21.15.92:0/3430380690 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbde00b4fd0 con 0x7fbdc8005200 2023-11-28T11:25:20.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.822+0000 7fbdc77fe700 1 -- 172.21.15.92:0/3430380690 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+18417 (secure 0 0 0) 0x7fbdd0022020 con 0x7fbdc8005200 2023-11-28T11:25:20.824 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:25:20.824 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":70,"fsid":"68956e0c-c16f-47cd-a220-f98364b0831d","created":"2023-11-28T11:20:28.697573+0000","modified":"2023-11-28T11:24:38.644982+0000","last_up_change":"2023-11-28T11:20:41.257011+0000","last_in_change":"2023-11-28T11:20:32.249767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:20:41.925237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:20:48.682497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:24:13.972356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-28T11:24:15.237693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6abe483f-0e03-4a26-b9cf-62041291747e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":115823},{"type":"v1","addr":"172.21.15.92:6809","nonce":115823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":115823},{"type":"v1","addr":"172.21.15.92:6813","nonce":115823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":115823},{"type":"v1","addr":"172.21.15.92:6821","nonce":115823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":115823},{"type":"v1","addr":"172.21.15.92:6817","nonce":115823}]},"public_addr":"172.21.15.92:6809/115823","cluster_addr":"172.21.15.92:6813/115823","heartbeat_back_addr":"172.21.15.92:6821/115823","heartbeat_front_addr":"172.21.15.92:6817/115823","state":["exists","up"]},{"osd":1,"uuid":"13c1de73-1a8c-4d7e-8a28-9dae52dfe089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":115824},{"type":"v1","addr":"172.21.15.92:6812","nonce":115824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":115824},{"type":"v1","addr":"172.21.15.92:6816","nonce":115824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":115824},{"type":"v1","addr":"172.21.15.92:6823","nonce":115824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":115824},{"type":"v1","addr":"172.21.15.92:6820","nonce":115824}]},"public_addr":"172.21.15.92:6812/115824","cluster_addr":"172.21.15.92:6816/115824","heartbeat_back_addr":"172.21.15.92:6823/115824","heartbeat_front_addr":"172.21.15.92:6820/115824","state":["exists","up"]},{"osd":2,"uuid":"25788a55-112f-4f00-810d-2bc37cf76778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":115821},{"type":"v1","addr":"172.21.15.92:6801","nonce":115821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":115821},{"type":"v1","addr":"172.21.15.92:6803","nonce":115821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":115821},{"type":"v1","addr":"172.21.15.92:6807","nonce":115821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":115821},{"type":"v1","addr":"172.21.15.92:6805","nonce":115821}]},"public_addr":"172.21.15.92:6801/115821","cluster_addr":"172.21.15.92:6803/115821","heartbeat_back_addr":"172.21.15.92:6807/115821","heartbeat_front_addr":"172.21.15.92:6805/115821","state":["exists","up"]},{"osd":3,"uuid":"adf54e24-33a9-41fe-9ce3-98a5ab6c158e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":115825},{"type":"v1","addr":"172.21.15.92:6825","nonce":115825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":115825},{"type":"v1","addr":"172.21.15.92:6827","nonce":115825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":115825},{"type":"v1","addr":"172.21.15.92:6831","nonce":115825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":115825},{"type":"v1","addr":"172.21.15.92:6829","nonce":115825}]},"public_addr":"172.21.15.92:6825/115825","cluster_addr":"172.21.15.92:6827/115825","heartbeat_back_addr":"172.21.15.92:6831/115825","heartbeat_front_addr":"172.21.15.92:6829/115825","state":["exists","up"]},{"osd":4,"uuid":"ccb9b747-f399-44ea-9a18-9db3152227e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":116035},{"type":"v1","addr":"172.21.15.169:6817","nonce":116035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":116035},{"type":"v1","addr":"172.21.15.169:6819","nonce":116035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":116035},{"type":"v1","addr":"172.21.15.169:6826","nonce":116035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":116035},{"type":"v1","addr":"172.21.15.169:6822","nonce":116035}]},"public_addr":"172.21.15.169:6817/116035","cluster_addr":"172.21.15.169:6819/116035","heartbeat_back_addr":"172.21.15.169:6826/116035","heartbeat_front_addr":"172.21.15.169:6822/116035","state":["exists","up"]},{"osd":5,"uuid":"308a1e63-75aa-4d7d-8a7c-cecc24714fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":116033},{"type":"v1","addr":"172.21.15.169:6801","nonce":116033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":116033},{"type":"v1","addr":"172.21.15.169:6803","nonce":116033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":116033},{"type":"v1","addr":"172.21.15.169:6807","nonce":116033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":116033},{"type":"v1","addr":"172.21.15.169:6805","nonce":116033}]},"public_addr":"172.21.15.169:6801/116033","cluster_addr":"172.21.15.169:6803/116033","heartbeat_back_addr":"172.21.15.169:6807/116033","heartbeat_front_addr":"172.21.15.169:6805/116033","state":["exists","up"]},{"osd":6,"uuid":"9c459b7b-b821-4b4b-b91e-6aaae271457c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":116034},{"type":"v1","addr":"172.21.15.169:6809","nonce":116034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":116034},{"type":"v1","addr":"172.21.15.169:6811","nonce":116034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":116034},{"type":"v1","addr":"172.21.15.169:6815","nonce":116034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":116034},{"type":"v1","addr":"172.21.15.169:6813","nonce":116034}]},"public_addr":"172.21.15.169:6809/116034","cluster_addr":"172.21.15.169:6811/116034","heartbeat_back_addr":"172.21.15.169:6815/116034","heartbeat_front_addr":"172.21.15.169:6813/116034","state":["exists","up"]},{"osd":7,"uuid":"1ca9aa4a-755d-4e28-a937-4d8dccab4df3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":116044},{"type":"v1","addr":"172.21.15.169:6823","nonce":116044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6825","nonce":116044},{"type":"v1","addr":"172.21.15.169:6827","nonce":116044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":116044},{"type":"v1","addr":"172.21.15.169:6831","nonce":116044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":116044},{"type":"v1","addr":"172.21.15.169:6829","nonce":116044}]},"public_addr":"172.21.15.169:6823/116044","cluster_addr":"172.21.15.169:6827/116044","heartbeat_back_addr":"172.21.15.169:6831/116044","heartbeat_front_addr":"172.21.15.169:6829/116044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.586781+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.869778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.508594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.860493+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.872267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.746933+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.611520+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.871446+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.10","mappings":[{"from":6,"to":5}]},{"pgid":"11.17","mappings":[{"from":6,"to":1}]},{"pgid":"11.29","mappings":[{"from":7,"to":0}]},{"pgid":"11.31","mappings":[{"from":2,"to":4}]},{"pgid":"11.33","mappings":[{"from":6,"to":1}]},{"pgid":"11.3e","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:25:20.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.824+0000 7fbde57ff700 1 -- 172.21.15.92:0/3430380690 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fbdcc0682f0 msgr2=0x7fbdcc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:20.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.824+0000 7fbde57ff700 1 --2- 172.21.15.92:0/3430380690 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fbdcc0682f0 0x7fbdcc06a7a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fbde00d3e80 tx=0x7fbdd4009040 comp rx=0 tx=0).stop 2023-11-28T11:25:20.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.824+0000 7fbde57ff700 1 -- 172.21.15.92:0/3430380690 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbdc8005200 msgr2=0x7fbde014d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:20.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.824+0000 7fbde57ff700 1 --2- 172.21.15.92:0/3430380690 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbdc8005200 0x7fbde014d350 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0011c70 tx=0x7fbdd000f520 comp rx=0 tx=0).stop 2023-11-28T11:25:20.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.824+0000 7fbde57ff700 1 -- 172.21.15.92:0/3430380690 shutdown_connections 2023-11-28T11:25:20.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.824+0000 7fbde57ff700 1 --2- 172.21.15.92:0/3430380690 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbde00b5cd0 0x7fbde00b6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:20.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.824+0000 7fbde57ff700 1 --2- 172.21.15.92:0/3430380690 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fbdcc0682f0 0x7fbdcc06a7a0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:20.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.824+0000 7fbde57ff700 1 --2- 172.21.15.92:0/3430380690 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbdc8005200 0x7fbde014d350 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:20.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.824+0000 7fbde57ff700 1 --2- 172.21.15.92:0/3430380690 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbde0154480 0x7fbde00cc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:20.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.824+0000 7fbde57ff700 1 -- 172.21.15.92:0/3430380690 >> 172.21.15.92:0/3430380690 conn(0x7fbde00719a0 msgr2=0x7fbde00b75f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:20.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.824+0000 7fbde57ff700 1 -- 172.21.15.92:0/3430380690 shutdown_connections 2023-11-28T11:25:20.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:20.824+0000 7fbde57ff700 1 -- 172.21.15.92:0/3430380690 wait complete. 2023-11-28T11:25:20.845 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:20:41.925237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:20:48.682497+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:24:13.972356+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:24:15.237693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:25:20.845 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:25:21.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.042+0000 7fbddffff700 1 Processor -- start 2023-11-28T11:25:21.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.043+0000 7fbddffff700 1 -- start start 2023-11-28T11:25:21.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.044+0000 7fbddffff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbdd80b8510 0x7fbdd806b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:21.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.044+0000 7fbddffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbdd80cd460 0x7fbdd806beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:21.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.044+0000 7fbddffff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdd80b88e0 con 0x7fbdd806c3f0 2023-11-28T11:25:21.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.044+0000 7fbddffff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdd80cdbd0 con 0x7fbdd80cd460 2023-11-28T11:25:21.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.044+0000 7fbddffff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd806c830 con 0x7fbdd80b8510 2023-11-28T11:25:21.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.044+0000 7fbddd59a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbdd80cd460 0x7fbdd806beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:21.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.044+0000 7fbdde59c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fbdd806c3f0 0x7fbdd815cc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:58978/0 (socket says 172.21.15.92:58978) 2023-11-28T11:25:21.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.044+0000 7fbdde59c700 1 -- 172.21.15.92:0/3805753823 learned_addr learned my addr 172.21.15.92:0/3805753823 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:25:21.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.044+0000 7fbdddd9b700 1 --2- 172.21.15.92:0/3805753823 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbdd80b8510 0x7fbdd806b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:21.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.045+0000 7fbddd59a700 1 -- 172.21.15.92:0/3805753823 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbdd80b8510 msgr2=0x7fbdd806b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:21.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.045+0000 7fbddd59a700 1 --2- 172.21.15.92:0/3805753823 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbdd80b8510 0x7fbdd806b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:21.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.045+0000 7fbddd59a700 1 -- 172.21.15.92:0/3805753823 >> v1:172.21.15.92:6789/0 conn(0x7fbdd806c3f0 legacy=0x7fbdd815cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:21.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.045+0000 7fbddd59a700 1 -- 172.21.15.92:0/3805753823 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd80cd8f0 con 0x7fbdd80cd460 2023-11-28T11:25:21.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.045+0000 7fbddcd99700 1 -- 172.21.15.92:0/3805753823 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1148071828 0 0) 0x7fbdd80b88e0 con 0x7fbdd806c3f0 2023-11-28T11:25:21.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.045+0000 7fbddd59a700 1 --2- 172.21.15.92:0/3805753823 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbdd80cd460 0x7fbdd806beb0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8009870 tx=0x7fbdc8009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=916eb063ae4447fb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:21.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.045+0000 7fbddcd99700 1 -- 172.21.15.92:0/3805753823 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc8011830 con 0x7fbdd80cd460 2023-11-28T11:25:21.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.045+0000 7fbddcd99700 1 -- 172.21.15.92:0/3805753823 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdc8011990 con 0x7fbdd80cd460 2023-11-28T11:25:21.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.045+0000 7fbddcd99700 1 -- 172.21.15.92:0/3805753823 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc8011d20 con 0x7fbdd80cd460 2023-11-28T11:25:21.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.046+0000 7fbddffff700 1 -- 172.21.15.92:0/3805753823 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbdd80cd460 msgr2=0x7fbdd806beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:21.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.046+0000 7fbddffff700 1 --2- 172.21.15.92:0/3805753823 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbdd80cd460 0x7fbdd806beb0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8009870 tx=0x7fbdc8009b80 comp rx=0 tx=0).stop 2023-11-28T11:25:21.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.046+0000 7fbddffff700 1 -- 172.21.15.92:0/3805753823 shutdown_connections 2023-11-28T11:25:21.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.046+0000 7fbddffff700 1 --2- 172.21.15.92:0/3805753823 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbdd80b8510 0x7fbdd806b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:21.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.046+0000 7fbddffff700 1 --2- 172.21.15.92:0/3805753823 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbdd80cd460 0x7fbdd806beb0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:21.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.046+0000 7fbddffff700 1 -- 172.21.15.92:0/3805753823 >> 172.21.15.92:0/3805753823 conn(0x7fbdd80719a0 msgr2=0x7fbdd8071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:21.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.046+0000 7fbddffff700 1 -- 172.21.15.92:0/3805753823 shutdown_connections 2023-11-28T11:25:21.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.046+0000 7fbddffff700 1 -- 172.21.15.92:0/3805753823 wait complete. 2023-11-28T11:25:21.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.048+0000 7fbddffff700 1 Processor -- start 2023-11-28T11:25:21.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.048+0000 7fbddffff700 1 -- start start 2023-11-28T11:25:21.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.048+0000 7fbddffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbdd806c3f0 0x7fbdd80aa430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:21.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.049+0000 7fbddffff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbdd80b8510 0x7fbdd815c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:21.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.049+0000 7fbddffff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbdd80cd460 0x7fbdd815c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:21.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.049+0000 7fbddffff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd81f2750 con 0x7fbdd806c3f0 2023-11-28T11:25:21.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.049+0000 7fbddffff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd80cc210 con 0x7fbdd80b8510 2023-11-28T11:25:21.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.049+0000 7fbddffff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd80cc4f0 con 0x7fbdd80cd460 2023-11-28T11:25:21.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.049+0000 7fbdddd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbdd806c3f0 0x7fbdd80aa430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:21.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.049+0000 7fbdddd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbdd806c3f0 0x7fbdd80aa430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44578/0 (socket says 172.21.15.92:44578) 2023-11-28T11:25:21.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.049+0000 7fbdddd9b700 1 -- 172.21.15.92:0/2912156382 learned_addr learned my addr 172.21.15.92:0/2912156382 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:21.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.049+0000 7fbdde59c700 1 --2- 172.21.15.92:0/2912156382 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbdd80cd460 0x7fbdd815c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:21.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.049+0000 7fbddd59a700 1 --2- 172.21.15.92:0/2912156382 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbdd80b8510 0x7fbdd815c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:21.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.049+0000 7fbdddd9b700 1 -- 172.21.15.92:0/2912156382 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbdd80cd460 msgr2=0x7fbdd815c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:21.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.049+0000 7fbdddd9b700 1 --2- 172.21.15.92:0/2912156382 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbdd80cd460 0x7fbdd815c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:21.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.049+0000 7fbdddd9b700 1 -- 172.21.15.92:0/2912156382 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbdd80b8510 msgr2=0x7fbdd815c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:21.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.049+0000 7fbdddd9b700 1 --2- 172.21.15.92:0/2912156382 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbdd80b8510 0x7fbdd815c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:21.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.049+0000 7fbdddd9b700 1 -- 172.21.15.92:0/2912156382 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc80094a0 con 0x7fbdd806c3f0 2023-11-28T11:25:21.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.050+0000 7fbdddd9b700 1 --2- 172.21.15.92:0/2912156382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbdd806c3f0 0x7fbdd80aa430 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fbdd400e9b0 tx=0x7fbdd400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:21.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.050+0000 7fbdce7fc700 1 -- 172.21.15.92:0/2912156382 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd4009710 con 0x7fbdd806c3f0 2023-11-28T11:25:21.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.050+0000 7fbdce7fc700 1 -- 172.21.15.92:0/2912156382 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdd400ee50 con 0x7fbdd806c3f0 2023-11-28T11:25:21.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.050+0000 7fbddffff700 1 -- 172.21.15.92:0/2912156382 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdd80cfff0 con 0x7fbdd806c3f0 2023-11-28T11:25:21.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.050+0000 7fbdce7fc700 1 -- 172.21.15.92:0/2912156382 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd400ca50 con 0x7fbdd806c3f0 2023-11-28T11:25:21.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.050+0000 7fbddffff700 1 -- 172.21.15.92:0/2912156382 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdd80cfd10 con 0x7fbdd806c3f0 2023-11-28T11:25:21.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.051+0000 7fbddffff700 1 -- 172.21.15.92:0/2912156382 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdd8067880 con 0x7fbdd806c3f0 2023-11-28T11:25:21.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.052+0000 7fbdce7fc700 1 -- 172.21.15.92:0/2912156382 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbdd4019030 con 0x7fbdd806c3f0 2023-11-28T11:25:21.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.052+0000 7fbdce7fc700 1 --2- 172.21.15.92:0/2912156382 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fbdbc0682f0 0x7fbdbc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:21.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.053+0000 7fbdce7fc700 1 -- 172.21.15.92:0/2912156382 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6556+0+0 (secure 0 0 0) 0x7fbdd4085e70 con 0x7fbdd806c3f0 2023-11-28T11:25:21.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.053+0000 7fbddd59a700 1 --2- 172.21.15.92:0/2912156382 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fbdbc0682f0 0x7fbdbc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:21.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.053+0000 7fbddd59a700 1 --2- 172.21.15.92:0/2912156382 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fbdbc0682f0 0x7fbdbc06a7a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fbdc80097c0 tx=0x7fbdc801c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:21.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.056+0000 7fbdce7fc700 1 -- 172.21.15.92:0/2912156382 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbdd4052390 con 0x7fbdd806c3f0 2023-11-28T11:25:21.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.246+0000 7fbddffff700 1 -- 172.21.15.92:0/2912156382 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbdd80d2940 con 0x7fbdd806c3f0 2023-11-28T11:25:21.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.246+0000 7fbdce7fc700 1 -- 172.21.15.92:0/2912156382 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbdd4021020 con 0x7fbdd806c3f0 2023-11-28T11:25:21.249 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-11-28T11:25:21.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.249+0000 7fbddffff700 1 -- 172.21.15.92:0/2912156382 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fbdbc0682f0 msgr2=0x7fbdbc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:21.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.249+0000 7fbddffff700 1 --2- 172.21.15.92:0/2912156382 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fbdbc0682f0 0x7fbdbc06a7a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fbdc80097c0 tx=0x7fbdc801c040 comp rx=0 tx=0).stop 2023-11-28T11:25:21.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.249+0000 7fbddffff700 1 -- 172.21.15.92:0/2912156382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbdd806c3f0 msgr2=0x7fbdd80aa430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:21.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.249+0000 7fbddffff700 1 --2- 172.21.15.92:0/2912156382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbdd806c3f0 0x7fbdd80aa430 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fbdd400e9b0 tx=0x7fbdd400ecc0 comp rx=0 tx=0).stop 2023-11-28T11:25:21.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.249+0000 7fbddffff700 1 -- 172.21.15.92:0/2912156382 shutdown_connections 2023-11-28T11:25:21.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.249+0000 7fbddffff700 1 --2- 172.21.15.92:0/2912156382 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbdd80cd460 0x7fbdd815c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:21.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.249+0000 7fbddffff700 1 --2- 172.21.15.92:0/2912156382 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fbdbc0682f0 0x7fbdbc06a7a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:21.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.249+0000 7fbddffff700 1 --2- 172.21.15.92:0/2912156382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbdd806c3f0 0x7fbdd80aa430 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:21.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.249+0000 7fbddffff700 1 --2- 172.21.15.92:0/2912156382 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbdd80b8510 0x7fbdd815c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:21.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.249+0000 7fbddffff700 1 -- 172.21.15.92:0/2912156382 >> 172.21.15.92:0/2912156382 conn(0x7fbdd80719a0 msgr2=0x7fbdd81e7150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:21.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.249+0000 7fbddffff700 1 -- 172.21.15.92:0/2912156382 shutdown_connections 2023-11-28T11:25:21.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.249+0000 7fbddffff700 1 -- 172.21.15.92:0/2912156382 wait complete. 2023-11-28T11:25:21.268 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:25:21.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.464+0000 7f869a1ff700 1 Processor -- start 2023-11-28T11:25:21.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.465+0000 7f869a1ff700 1 -- start start 2023-11-28T11:25:21.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.466+0000 7f869a1ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f86940b8510 0x7f869406b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:21.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.466+0000 7f869a1ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86940b88e0 con 0x7f86940cd460 2023-11-28T11:25:21.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.466+0000 7f869a1ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86940cd8f0 con 0x7f86940d0d60 2023-11-28T11:25:21.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.466+0000 7f869a1ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86940cdbd0 con 0x7f86940b8510 2023-11-28T11:25:21.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.466+0000 7f868bfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f86940cd460 0x7f869406beb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:58980/0 (socket says 172.21.15.92:58980) 2023-11-28T11:25:21.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.466+0000 7f868bfff700 1 -- 172.21.15.92:0/3481394959 learned_addr learned my addr 172.21.15.92:0/3481394959 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:25:21.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.466+0000 7f86937fe700 1 --2- 172.21.15.92:0/3481394959 >> v2:172.21.15.169:3300/0 conn(0x7f86940b8510 0x7f869406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:21.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.466+0000 7f8692ffd700 1 -- 172.21.15.92:0/3481394959 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3228508331 0 0) 0x7f86940b88e0 con 0x7f86940cd460 2023-11-28T11:25:21.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.466+0000 7f8692ffd700 1 -- 172.21.15.92:0/3481394959 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86800035b0 con 0x7f86940cd460 2023-11-28T11:25:21.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.467+0000 7f86937fe700 1 -- 172.21.15.92:0/3481394959 >> v1:172.21.15.169:6789/0 conn(0x7f86940d0d60 legacy=0x7f86940d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:21.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.467+0000 7f8692ffd700 1 -- 172.21.15.92:0/3481394959 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1481239404 0 0) 0x7f86940cd8f0 con 0x7f86940d0d60 2023-11-28T11:25:21.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.467+0000 7f86937fe700 1 -- 172.21.15.92:0/3481394959 >> v1:172.21.15.92:6789/0 conn(0x7f86940cd460 legacy=0x7f869406beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:21.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.467+0000 7f86937fe700 1 -- 172.21.15.92:0/3481394959 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86940cfd10 con 0x7f86940b8510 2023-11-28T11:25:21.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.467+0000 7f86937fe700 1 --2- 172.21.15.92:0/3481394959 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86940b8510 0x7f869406b970 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8684009840 tx=0x7f8684009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9ca385d2f4ff5e3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:21.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.467+0000 7f8692ffd700 1 -- 172.21.15.92:0/3481394959 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8684011830 con 0x7f86940b8510 2023-11-28T11:25:21.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.467+0000 7f8692ffd700 1 -- 172.21.15.92:0/3481394959 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8684011990 con 0x7f86940b8510 2023-11-28T11:25:21.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.467+0000 7f8692ffd700 1 -- 172.21.15.92:0/3481394959 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8684011d20 con 0x7f86940b8510 2023-11-28T11:25:21.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.468+0000 7f869a1ff700 1 -- 172.21.15.92:0/3481394959 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86940b8510 msgr2=0x7f869406b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:21.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.468+0000 7f869a1ff700 1 --2- 172.21.15.92:0/3481394959 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86940b8510 0x7f869406b970 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8684009840 tx=0x7f8684009b50 comp rx=0 tx=0).stop 2023-11-28T11:25:21.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.468+0000 7f869a1ff700 1 -- 172.21.15.92:0/3481394959 shutdown_connections 2023-11-28T11:25:21.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.468+0000 7f869a1ff700 1 --2- 172.21.15.92:0/3481394959 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86940b8510 0x7f869406b970 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:21.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.468+0000 7f869a1ff700 1 -- 172.21.15.92:0/3481394959 >> 172.21.15.92:0/3481394959 conn(0x7f86940719a0 msgr2=0x7f8694071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:21.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.468+0000 7f869a1ff700 1 -- 172.21.15.92:0/3481394959 shutdown_connections 2023-11-28T11:25:21.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.468+0000 7f869a1ff700 1 -- 172.21.15.92:0/3481394959 wait complete. 2023-11-28T11:25:21.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.470+0000 7f869a1ff700 1 Processor -- start 2023-11-28T11:25:21.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.470+0000 7f869a1ff700 1 -- start start 2023-11-28T11:25:21.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.470+0000 7f869a1ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86940b8510 0x7f86941f2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:21.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.471+0000 7f869a1ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86940cd460 0x7f86940b72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:21.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.471+0000 7f869a1ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86940d0d60 0x7f86940b7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:21.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.471+0000 7f869a1ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86941f2a10 con 0x7f86940cd460 2023-11-28T11:25:21.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.471+0000 7f869a1ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86941f2cf0 con 0x7f86940d0d60 2023-11-28T11:25:21.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.471+0000 7f869a1ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86941f2fd0 con 0x7f86940b8510 2023-11-28T11:25:21.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.471+0000 7f868bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86940cd460 0x7f86940b72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:21.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.471+0000 7f868bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86940cd460 0x7f86940b72d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44582/0 (socket says 172.21.15.92:44582) 2023-11-28T11:25:21.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.471+0000 7f868bfff700 1 -- 172.21.15.92:0/798221721 learned_addr learned my addr 172.21.15.92:0/798221721 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:21.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.471+0000 7f86937fe700 1 --2- 172.21.15.92:0/798221721 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86940b8510 0x7f86941f2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:21.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.471+0000 7f8693fff700 1 --2- 172.21.15.92:0/798221721 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86940d0d60 0x7f86940b7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:21.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.471+0000 7f868bfff700 1 -- 172.21.15.92:0/798221721 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86940b8510 msgr2=0x7f86941f2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:21.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.471+0000 7f868bfff700 1 --2- 172.21.15.92:0/798221721 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86940b8510 0x7f86941f2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:21.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.471+0000 7f868bfff700 1 -- 172.21.15.92:0/798221721 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86940d0d60 msgr2=0x7f86940b7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:21.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.471+0000 7f868bfff700 1 --2- 172.21.15.92:0/798221721 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86940d0d60 0x7f86940b7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:21.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.471+0000 7f868bfff700 1 -- 172.21.15.92:0/798221721 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86840094a0 con 0x7f86940cd460 2023-11-28T11:25:21.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.472+0000 7f868bfff700 1 --2- 172.21.15.92:0/798221721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86940cd460 0x7f86940b72d0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f867c00c500 tx=0x7f867c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:21.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.472+0000 7f8690ff9700 1 -- 172.21.15.92:0/798221721 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c005e30 con 0x7f86940cd460 2023-11-28T11:25:21.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.472+0000 7f8690ff9700 1 -- 172.21.15.92:0/798221721 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f867c0070d0 con 0x7f86940cd460 2023-11-28T11:25:21.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.472+0000 7f869a1ff700 1 -- 172.21.15.92:0/798221721 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86940d29c0 con 0x7f86940cd460 2023-11-28T11:25:21.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.472+0000 7f8690ff9700 1 -- 172.21.15.92:0/798221721 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c007480 con 0x7f86940cd460 2023-11-28T11:25:21.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.472+0000 7f869a1ff700 1 -- 172.21.15.92:0/798221721 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86940cfff0 con 0x7f86940cd460 2023-11-28T11:25:21.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.473+0000 7f869a1ff700 1 -- 172.21.15.92:0/798221721 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86940d2b50 con 0x7f86940cd460 2023-11-28T11:25:21.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.474+0000 7f8690ff9700 1 -- 172.21.15.92:0/798221721 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f867c019030 con 0x7f86940cd460 2023-11-28T11:25:21.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.475+0000 7f8690ff9700 1 --2- 172.21.15.92:0/798221721 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f86740682f0 0x7f867406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:21.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.475+0000 7f8690ff9700 1 -- 172.21.15.92:0/798221721 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6556+0+0 (secure 0 0 0) 0x7f867c0852b0 con 0x7f86940cd460 2023-11-28T11:25:21.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.475+0000 7f86937fe700 1 --2- 172.21.15.92:0/798221721 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f86740682f0 0x7f867406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:21.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.475+0000 7f86937fe700 1 --2- 172.21.15.92:0/798221721 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f86740682f0 0x7f867406a7a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8684009fd0 tx=0x7f868401c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:21.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.479+0000 7f8690ff9700 1 -- 172.21.15.92:0/798221721 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f867c051830 con 0x7f86940cd460 2023-11-28T11:25:21.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.668+0000 7f869a1ff700 1 -- 172.21.15.92:0/798221721 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8694067880 con 0x7f86940cd460 2023-11-28T11:25:21.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.669+0000 7f8690ff9700 1 -- 172.21.15.92:0/798221721 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v70) v1 ==== 92+0+10 (secure 0 0 0) 0x7f867c017020 con 0x7f86940cd460 2023-11-28T11:25:21.672 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-11-28T11:25:21.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.671+0000 7f869a1ff700 1 -- 172.21.15.92:0/798221721 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f86740682f0 msgr2=0x7f867406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:21.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.671+0000 7f869a1ff700 1 --2- 172.21.15.92:0/798221721 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f86740682f0 0x7f867406a7a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8684009fd0 tx=0x7f868401c040 comp rx=0 tx=0).stop 2023-11-28T11:25:21.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.671+0000 7f869a1ff700 1 -- 172.21.15.92:0/798221721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86940cd460 msgr2=0x7f86940b72d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:21.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.671+0000 7f869a1ff700 1 --2- 172.21.15.92:0/798221721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86940cd460 0x7f86940b72d0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f867c00c500 tx=0x7f867c00c810 comp rx=0 tx=0).stop 2023-11-28T11:25:21.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.671+0000 7f869a1ff700 1 -- 172.21.15.92:0/798221721 shutdown_connections 2023-11-28T11:25:21.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.671+0000 7f869a1ff700 1 --2- 172.21.15.92:0/798221721 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f86940d0d60 0x7f86940b7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:21.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.671+0000 7f869a1ff700 1 --2- 172.21.15.92:0/798221721 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f86940b8510 0x7f86941f2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:21.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.671+0000 7f869a1ff700 1 --2- 172.21.15.92:0/798221721 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f86740682f0 0x7f867406a7a0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:21.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.671+0000 7f869a1ff700 1 --2- 172.21.15.92:0/798221721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86940cd460 0x7f86940b72d0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:21.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.671+0000 7f869a1ff700 1 -- 172.21.15.92:0/798221721 >> 172.21.15.92:0/798221721 conn(0x7f86940719a0 msgr2=0x7f86940b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:21.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.671+0000 7f869a1ff700 1 -- 172.21.15.92:0/798221721 shutdown_connections 2023-11-28T11:25:21.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.671+0000 7f869a1ff700 1 -- 172.21.15.92:0/798221721 wait complete. 2023-11-28T11:25:21.691 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:25:21.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.890+0000 7f359a1ff700 1 Processor -- start 2023-11-28T11:25:21.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.890+0000 7f359a1ff700 1 -- start start 2023-11-28T11:25:21.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.891+0000 7f359a1ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f35940d1d70 0x7f3594167270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:21.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.891+0000 7f359a1ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f359406c790 con 0x7f35940cd460 2023-11-28T11:25:21.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.891+0000 7f359a1ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35940b88e0 con 0x7f35940b8510 2023-11-28T11:25:21.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.891+0000 7f359a1ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35940cdbd0 con 0x7f35940d1d70 2023-11-28T11:25:21.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.891+0000 7f358bfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f35940cd460 0x7f359406c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:58984/0 (socket says 172.21.15.92:58984) 2023-11-28T11:25:21.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.891+0000 7f3593fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f35940d1d70 0x7f3594167270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:21.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.891+0000 7f358bfff700 1 -- 172.21.15.92:0/1394073160 learned_addr learned my addr 172.21.15.92:0/1394073160 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:25:21.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.892+0000 7f3593fff700 1 -- 172.21.15.92:0/1394073160 >> v1:172.21.15.169:6789/0 conn(0x7f35940b8510 legacy=0x7f359406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:21.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.892+0000 7f3593fff700 1 -- 172.21.15.92:0/1394073160 >> v1:172.21.15.92:6789/0 conn(0x7f35940cd460 legacy=0x7f359406c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:21.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.892+0000 7f3593fff700 1 -- 172.21.15.92:0/1394073160 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35940cd8f0 con 0x7f35940d1d70 2023-11-28T11:25:21.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.891+0000 7f3592ffd700 1 -- 172.21.15.92:0/1394073160 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1044651186 0 0) 0x7f359406c790 con 0x7f35940cd460 2023-11-28T11:25:21.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.892+0000 7f3593fff700 1 --2- 172.21.15.92:0/1394073160 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35940d1d70 0x7f3594167270 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f358c009d90 tx=0x7f358c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bf5d02ffe9874dc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:21.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.892+0000 7f3592ffd700 1 -- 172.21.15.92:0/1394073160 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358c00e070 con 0x7f35940d1d70 2023-11-28T11:25:21.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.892+0000 7f3592ffd700 1 -- 172.21.15.92:0/1394073160 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f358c00bbd0 con 0x7f35940d1d70 2023-11-28T11:25:21.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.892+0000 7f3592ffd700 1 -- 172.21.15.92:0/1394073160 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358c012430 con 0x7f35940d1d70 2023-11-28T11:25:21.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.892+0000 7f359a1ff700 1 -- 172.21.15.92:0/1394073160 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35940d1d70 msgr2=0x7f3594167270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:21.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.892+0000 7f359a1ff700 1 --2- 172.21.15.92:0/1394073160 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35940d1d70 0x7f3594167270 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f358c009d90 tx=0x7f358c00b330 comp rx=0 tx=0).stop 2023-11-28T11:25:21.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.893+0000 7f359a1ff700 1 -- 172.21.15.92:0/1394073160 shutdown_connections 2023-11-28T11:25:21.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.893+0000 7f359a1ff700 1 --2- 172.21.15.92:0/1394073160 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35940d1d70 0x7f3594167270 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:21.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.893+0000 7f359a1ff700 1 -- 172.21.15.92:0/1394073160 >> 172.21.15.92:0/1394073160 conn(0x7f35940719a0 msgr2=0x7f3594071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:21.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.893+0000 7f359a1ff700 1 -- 172.21.15.92:0/1394073160 shutdown_connections 2023-11-28T11:25:21.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.893+0000 7f359a1ff700 1 -- 172.21.15.92:0/1394073160 wait complete. 2023-11-28T11:25:21.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.894+0000 7f359a1ff700 1 Processor -- start 2023-11-28T11:25:21.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.895+0000 7f359a1ff700 1 -- start start 2023-11-28T11:25:21.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.895+0000 7f359a1ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35940b8510 0x7f35941e6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:21.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.895+0000 7f359a1ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35940cd460 0x7f359415b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:21.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.895+0000 7f359a1ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35940d1d70 0x7f35941e5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:21.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.895+0000 7f359a1ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359415b7d0 con 0x7f35940b8510 2023-11-28T11:25:21.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.895+0000 7f359a1ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35941e4250 con 0x7f35940cd460 2023-11-28T11:25:21.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.895+0000 7f359a1ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35941e3490 con 0x7f35940d1d70 2023-11-28T11:25:21.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.896+0000 7f35937fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35940b8510 0x7f35941e6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:21.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.896+0000 7f35937fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35940b8510 0x7f35941e6790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44594/0 (socket says 172.21.15.92:44594) 2023-11-28T11:25:21.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.896+0000 7f35937fe700 1 -- 172.21.15.92:0/3536419377 learned_addr learned my addr 172.21.15.92:0/3536419377 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:21.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.896+0000 7f358bfff700 1 --2- 172.21.15.92:0/3536419377 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35940cd460 0x7f359415b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:21.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.896+0000 7f3593fff700 1 --2- 172.21.15.92:0/3536419377 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35940d1d70 0x7f35941e5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:21.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.896+0000 7f35937fe700 1 -- 172.21.15.92:0/3536419377 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35940d1d70 msgr2=0x7f35941e5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:21.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.896+0000 7f35937fe700 1 --2- 172.21.15.92:0/3536419377 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35940d1d70 0x7f35941e5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:21.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.896+0000 7f35937fe700 1 -- 172.21.15.92:0/3536419377 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35940cd460 msgr2=0x7f359415b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:21.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.896+0000 7f35937fe700 1 --2- 172.21.15.92:0/3536419377 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35940cd460 0x7f359415b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:21.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.896+0000 7f35937fe700 1 -- 172.21.15.92:0/3536419377 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f358c009a70 con 0x7f35940b8510 2023-11-28T11:25:21.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.896+0000 7f35937fe700 1 --2- 172.21.15.92:0/3536419377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35940b8510 0x7f35941e6790 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f358400c4e0 tx=0x7f358400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:21.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.897+0000 7f3590ff9700 1 -- 172.21.15.92:0/3536419377 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35840052d0 con 0x7f35940b8510 2023-11-28T11:25:21.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.897+0000 7f3590ff9700 1 -- 172.21.15.92:0/3536419377 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3584005430 con 0x7f35940b8510 2023-11-28T11:25:21.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.897+0000 7f3590ff9700 1 -- 172.21.15.92:0/3536419377 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35840038c0 con 0x7f35940b8510 2023-11-28T11:25:21.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.897+0000 7f359a1ff700 1 -- 172.21.15.92:0/3536419377 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35940cfff0 con 0x7f35940b8510 2023-11-28T11:25:21.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.897+0000 7f359a1ff700 1 -- 172.21.15.92:0/3536419377 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35940cfd10 con 0x7f35940b8510 2023-11-28T11:25:21.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.898+0000 7f359a1ff700 1 -- 172.21.15.92:0/3536419377 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3594067880 con 0x7f35940b8510 2023-11-28T11:25:21.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.899+0000 7f3590ff9700 1 -- 172.21.15.92:0/3536419377 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3584019030 con 0x7f35940b8510 2023-11-28T11:25:21.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.900+0000 7f3590ff9700 1 --2- 172.21.15.92:0/3536419377 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f35740682f0 0x7f357406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:21.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.900+0000 7f3590ff9700 1 -- 172.21.15.92:0/3536419377 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6556+0+0 (secure 0 0 0) 0x7f35840851f0 con 0x7f35940b8510 2023-11-28T11:25:21.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.900+0000 7f358bfff700 1 --2- 172.21.15.92:0/3536419377 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f35740682f0 0x7f357406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:21.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.901+0000 7f358bfff700 1 --2- 172.21.15.92:0/3536419377 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f35740682f0 0x7f357406a7a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f357c007b10 tx=0x7f357c007a60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:21.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:21.904+0000 7f3590ff9700 1 -- 172.21.15.92:0/3536419377 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3584051770 con 0x7f35940b8510 2023-11-28T11:25:22.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.093+0000 7f359a1ff700 1 -- 172.21.15.92:0/3536419377 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f35940d2940 con 0x7f35940b8510 2023-11-28T11:25:22.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.094+0000 7f3590ff9700 1 -- 172.21.15.92:0/3536419377 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v70) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3584054d90 con 0x7f35940b8510 2023-11-28T11:25:22.096 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-11-28T11:25:22.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.096+0000 7f359a1ff700 1 -- 172.21.15.92:0/3536419377 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f35740682f0 msgr2=0x7f357406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:22.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.096+0000 7f359a1ff700 1 --2- 172.21.15.92:0/3536419377 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f35740682f0 0x7f357406a7a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f357c007b10 tx=0x7f357c007a60 comp rx=0 tx=0).stop 2023-11-28T11:25:22.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.096+0000 7f359a1ff700 1 -- 172.21.15.92:0/3536419377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35940b8510 msgr2=0x7f35941e6790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:22.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.096+0000 7f359a1ff700 1 --2- 172.21.15.92:0/3536419377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35940b8510 0x7f35941e6790 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f358400c4e0 tx=0x7f358400c7f0 comp rx=0 tx=0).stop 2023-11-28T11:25:22.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.096+0000 7f359a1ff700 1 -- 172.21.15.92:0/3536419377 shutdown_connections 2023-11-28T11:25:22.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.096+0000 7f359a1ff700 1 --2- 172.21.15.92:0/3536419377 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f35940d1d70 0x7f35941e5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:22.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.096+0000 7f359a1ff700 1 --2- 172.21.15.92:0/3536419377 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f35740682f0 0x7f357406a7a0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:22.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.096+0000 7f359a1ff700 1 --2- 172.21.15.92:0/3536419377 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35940b8510 0x7f35941e6790 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:22.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.096+0000 7f359a1ff700 1 --2- 172.21.15.92:0/3536419377 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f35940cd460 0x7f359415b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:22.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.096+0000 7f359a1ff700 1 -- 172.21.15.92:0/3536419377 >> 172.21.15.92:0/3536419377 conn(0x7f35940719a0 msgr2=0x7f35940cbef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:22.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.096+0000 7f359a1ff700 1 -- 172.21.15.92:0/3536419377 shutdown_connections 2023-11-28T11:25:22.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.096+0000 7f359a1ff700 1 -- 172.21.15.92:0/3536419377 wait complete. 2023-11-28T11:25:22.116 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:25:22.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.314+0000 7fbedb1ff700 1 Processor -- start 2023-11-28T11:25:22.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.315+0000 7fbedb1ff700 1 -- start start 2023-11-28T11:25:22.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.316+0000 7fbedb1ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbed40b8510 0x7fbed406b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:22.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.316+0000 7fbedb1ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fbed40d0d60 0x7fbed40d11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:22.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.316+0000 7fbedb1ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbed40b88e0 con 0x7fbed40cd460 2023-11-28T11:25:22.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.316+0000 7fbedb1ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbed40cdbd0 con 0x7fbed40d0d60 2023-11-28T11:25:22.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.316+0000 7fbedb1ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbed40d02d0 con 0x7fbed40b8510 2023-11-28T11:25:22.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.316+0000 7fbed979c700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fbed40d0d60 0x7fbed40d11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:22.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.316+0000 7fbed8f9b700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbed40b8510 0x7fbed406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:22.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.316+0000 7fbed979c700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fbed40d0d60 0x7fbed40d11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.92:51272/0 (socket says 172.21.15.92:51272) 2023-11-28T11:25:22.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.316+0000 7fbed979c700 1 -- 172.21.15.92:0/869111855 learned_addr learned my addr 172.21.15.92:0/869111855 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:22.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.316+0000 7fbed3fff700 1 --1- 172.21.15.92:0/869111855 >> v1:172.21.15.169:6789/0 conn(0x7fbed40cd460 0x7fbed406beb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.92:53800/0 (socket says 172.21.15.92:53800) 2023-11-28T11:25:22.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.317+0000 7fbed979c700 1 -- 172.21.15.92:0/869111855 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbed40b8510 msgr2=0x7fbed406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:22.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.317+0000 7fbed979c700 1 --2- 172.21.15.92:0/869111855 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbed40b8510 0x7fbed406b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:22.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.317+0000 7fbed979c700 1 -- 172.21.15.92:0/869111855 >> v1:172.21.15.169:6789/0 conn(0x7fbed40cd460 legacy=0x7fbed406beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:22.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.317+0000 7fbed979c700 1 -- 172.21.15.92:0/869111855 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed40cd8f0 con 0x7fbed40d0d60 2023-11-28T11:25:22.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.317+0000 7fbed37fe700 1 -- 172.21.15.92:0/869111855 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3814188246 0 0) 0x7fbed40b88e0 con 0x7fbed40cd460 2023-11-28T11:25:22.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.317+0000 7fbed979c700 1 --2- 172.21.15.92:0/869111855 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbed40d0d60 0x7fbed40d11a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fbecc009e80 tx=0x7fbecc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42a2a0b9fb42eeb7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:22.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.317+0000 7fbed37fe700 1 -- 172.21.15.92:0/869111855 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbecc00e070 con 0x7fbed40d0d60 2023-11-28T11:25:22.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.317+0000 7fbed37fe700 1 -- 172.21.15.92:0/869111855 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbecc00b8d0 con 0x7fbed40d0d60 2023-11-28T11:25:22.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.318+0000 7fbed37fe700 1 -- 172.21.15.92:0/869111855 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbecc00bc80 con 0x7fbed40d0d60 2023-11-28T11:25:22.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.318+0000 7fbedb1ff700 1 -- 172.21.15.92:0/869111855 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbed40d0d60 msgr2=0x7fbed40d11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:22.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.318+0000 7fbedb1ff700 1 --2- 172.21.15.92:0/869111855 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbed40d0d60 0x7fbed40d11a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fbecc009e80 tx=0x7fbecc00b330 comp rx=0 tx=0).stop 2023-11-28T11:25:22.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.318+0000 7fbedb1ff700 1 -- 172.21.15.92:0/869111855 shutdown_connections 2023-11-28T11:25:22.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.318+0000 7fbedb1ff700 1 --2- 172.21.15.92:0/869111855 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbed40d0d60 0x7fbed40d11a0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:22.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.318+0000 7fbedb1ff700 1 --2- 172.21.15.92:0/869111855 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbed40b8510 0x7fbed406b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:22.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.318+0000 7fbedb1ff700 1 -- 172.21.15.92:0/869111855 >> 172.21.15.92:0/869111855 conn(0x7fbed40719a0 msgr2=0x7fbed4071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:22.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.318+0000 7fbedb1ff700 1 -- 172.21.15.92:0/869111855 shutdown_connections 2023-11-28T11:25:22.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.318+0000 7fbedb1ff700 1 -- 172.21.15.92:0/869111855 wait complete. 2023-11-28T11:25:22.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.320+0000 7fbedb1ff700 1 Processor -- start 2023-11-28T11:25:22.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.320+0000 7fbedb1ff700 1 -- start start 2023-11-28T11:25:22.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.321+0000 7fbedb1ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbed40b8510 0x7fbed40b1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:22.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.321+0000 7fbedb1ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbed40cd460 0x7fbed40b6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:22.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.321+0000 7fbedb1ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbed40d0d60 0x7fbed415a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:22.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.321+0000 7fbedb1ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed415b4c0 con 0x7fbed40b8510 2023-11-28T11:25:22.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.321+0000 7fbedb1ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed40a9840 con 0x7fbed40d0d60 2023-11-28T11:25:22.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.321+0000 7fbedb1ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbed41e2b00 con 0x7fbed40cd460 2023-11-28T11:25:22.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.321+0000 7fbed8f9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbed40b8510 0x7fbed40b1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:22.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.321+0000 7fbed8f9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbed40b8510 0x7fbed40b1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44602/0 (socket says 172.21.15.92:44602) 2023-11-28T11:25:22.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.321+0000 7fbed8f9b700 1 -- 172.21.15.92:0/2048660827 learned_addr learned my addr 172.21.15.92:0/2048660827 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:22.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.321+0000 7fbed979c700 1 --2- 172.21.15.92:0/2048660827 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbed40d0d60 0x7fbed415a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:22.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.321+0000 7fbed3fff700 1 --2- 172.21.15.92:0/2048660827 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbed40cd460 0x7fbed40b6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:22.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.322+0000 7fbed8f9b700 1 -- 172.21.15.92:0/2048660827 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbed40cd460 msgr2=0x7fbed40b6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:22.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.322+0000 7fbed8f9b700 1 --2- 172.21.15.92:0/2048660827 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbed40cd460 0x7fbed40b6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:22.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.322+0000 7fbed8f9b700 1 -- 172.21.15.92:0/2048660827 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbed40d0d60 msgr2=0x7fbed415a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:22.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.322+0000 7fbed8f9b700 1 --2- 172.21.15.92:0/2048660827 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbed40d0d60 0x7fbed415a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:22.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.322+0000 7fbed8f9b700 1 -- 172.21.15.92:0/2048660827 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbecc009b30 con 0x7fbed40b8510 2023-11-28T11:25:22.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.322+0000 7fbed3fff700 1 --2- 172.21.15.92:0/2048660827 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbed40cd460 0x7fbed40b6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:25:22.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.322+0000 7fbed8f9b700 1 --2- 172.21.15.92:0/2048660827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbed40b8510 0x7fbed40b1f20 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fbec800e9b0 tx=0x7fbec800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:22.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.322+0000 7fbed17fa700 1 -- 172.21.15.92:0/2048660827 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec8009710 con 0x7fbed40b8510 2023-11-28T11:25:22.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.322+0000 7fbed17fa700 1 -- 172.21.15.92:0/2048660827 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec800c850 con 0x7fbed40b8510 2023-11-28T11:25:22.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.322+0000 7fbed17fa700 1 -- 172.21.15.92:0/2048660827 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec800cbc0 con 0x7fbed40b8510 2023-11-28T11:25:22.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.322+0000 7fbedb1ff700 1 -- 172.21.15.92:0/2048660827 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbed40cfff0 con 0x7fbed40b8510 2023-11-28T11:25:22.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.322+0000 7fbedb1ff700 1 -- 172.21.15.92:0/2048660827 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbed40cfd10 con 0x7fbed40b8510 2023-11-28T11:25:22.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.323+0000 7fbedb1ff700 1 -- 172.21.15.92:0/2048660827 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbed4067880 con 0x7fbed40b8510 2023-11-28T11:25:22.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.324+0000 7fbed17fa700 1 -- 172.21.15.92:0/2048660827 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbec8019030 con 0x7fbed40b8510 2023-11-28T11:25:22.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.325+0000 7fbed17fa700 1 --2- 172.21.15.92:0/2048660827 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fbeb80682f0 0x7fbeb806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:22.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.325+0000 7fbed17fa700 1 -- 172.21.15.92:0/2048660827 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6556+0+0 (secure 0 0 0) 0x7fbec8085f80 con 0x7fbed40b8510 2023-11-28T11:25:22.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.325+0000 7fbed3fff700 1 --2- 172.21.15.92:0/2048660827 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fbeb80682f0 0x7fbeb806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:22.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.326+0000 7fbed3fff700 1 --2- 172.21.15.92:0/2048660827 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fbeb80682f0 0x7fbeb806a7a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fbec0000f60 tx=0x7fbec0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:22.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.329+0000 7fbed17fa700 1 -- 172.21.15.92:0/2048660827 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbec8052500 con 0x7fbed40b8510 2023-11-28T11:25:22.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.520+0000 7fbedb1ff700 1 -- 172.21.15.92:0/2048660827 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbed40d2940 con 0x7fbed40b8510 2023-11-28T11:25:22.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.521+0000 7fbed17fa700 1 -- 172.21.15.92:0/2048660827 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v70) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbec80210a0 con 0x7fbed40b8510 2023-11-28T11:25:22.524 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-11-28T11:25:22.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.523+0000 7fbedb1ff700 1 -- 172.21.15.92:0/2048660827 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fbeb80682f0 msgr2=0x7fbeb806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:22.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.523+0000 7fbedb1ff700 1 --2- 172.21.15.92:0/2048660827 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fbeb80682f0 0x7fbeb806a7a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fbec0000f60 tx=0x7fbec0009040 comp rx=0 tx=0).stop 2023-11-28T11:25:22.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.523+0000 7fbedb1ff700 1 -- 172.21.15.92:0/2048660827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbed40b8510 msgr2=0x7fbed40b1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:22.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.523+0000 7fbedb1ff700 1 --2- 172.21.15.92:0/2048660827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbed40b8510 0x7fbed40b1f20 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fbec800e9b0 tx=0x7fbec800ed70 comp rx=0 tx=0).stop 2023-11-28T11:25:22.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.523+0000 7fbedb1ff700 1 -- 172.21.15.92:0/2048660827 shutdown_connections 2023-11-28T11:25:22.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.523+0000 7fbedb1ff700 1 --2- 172.21.15.92:0/2048660827 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fbed40d0d60 0x7fbed415a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:22.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.523+0000 7fbedb1ff700 1 --2- 172.21.15.92:0/2048660827 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fbeb80682f0 0x7fbeb806a7a0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:22.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.523+0000 7fbedb1ff700 1 --2- 172.21.15.92:0/2048660827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbed40b8510 0x7fbed40b1f20 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:22.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.523+0000 7fbedb1ff700 1 --2- 172.21.15.92:0/2048660827 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fbed40cd460 0x7fbed40b6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:22.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.523+0000 7fbedb1ff700 1 -- 172.21.15.92:0/2048660827 >> 172.21.15.92:0/2048660827 conn(0x7fbed40719a0 msgr2=0x7fbed40b4280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:22.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.523+0000 7fbedb1ff700 1 -- 172.21.15.92:0/2048660827 shutdown_connections 2023-11-28T11:25:22.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.523+0000 7fbedb1ff700 1 -- 172.21.15.92:0/2048660827 wait complete. 2023-11-28T11:25:22.544 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T11:25:22.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.739+0000 7ff231fff700 1 Processor -- start 2023-11-28T11:25:22.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.739+0000 7ff231fff700 1 -- start start 2023-11-28T11:25:22.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.740+0000 7ff231fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff22c0b8510 0x7ff22c06b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:22.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.740+0000 7ff231fff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff22c0cd460 0x7ff22c06beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:22.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.740+0000 7ff231fff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff22c0b88e0 con 0x7ff22c06c3f0 2023-11-28T11:25:22.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.740+0000 7ff231fff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff22c0cdbd0 con 0x7ff22c0b8510 2023-11-28T11:25:22.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.740+0000 7ff231fff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff22c0d02d0 con 0x7ff22c0cd460 2023-11-28T11:25:22.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.740+0000 7ff22b7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff22c0b8510 0x7ff22c06b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:22.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.740+0000 7ff22b7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff22c0b8510 0x7ff22c06b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44612/0 (socket says 172.21.15.92:44612) 2023-11-28T11:25:22.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.740+0000 7ff22b7fe700 1 -- 172.21.15.92:0/1758049729 learned_addr learned my addr 172.21.15.92:0/1758049729 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:22.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.740+0000 7ff223fff700 1 --2- 172.21.15.92:0/1758049729 >> v2:172.21.15.169:3300/0 conn(0x7ff22c0cd460 0x7ff22c06beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:22.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.741+0000 7ff22b7fe700 1 -- 172.21.15.92:0/1758049729 >> v2:172.21.15.169:3300/0 conn(0x7ff22c0cd460 msgr2=0x7ff22c06beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:22.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.741+0000 7ff22b7fe700 1 --2- 172.21.15.92:0/1758049729 >> v2:172.21.15.169:3300/0 conn(0x7ff22c0cd460 0x7ff22c06beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:22.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.741+0000 7ff22b7fe700 1 -- 172.21.15.92:0/1758049729 >> v1:172.21.15.169:6789/0 conn(0x7ff22c06c3f0 legacy=0x7ff22c15cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:22.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.741+0000 7ff22b7fe700 1 -- 172.21.15.92:0/1758049729 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff22c0cd8f0 con 0x7ff22c0b8510 2023-11-28T11:25:22.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.741+0000 7ff22b7fe700 1 --2- 172.21.15.92:0/1758049729 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff22c0b8510 0x7ff22c06b970 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7ff21c009920 tx=0x7ff21c009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d43aa3635a928f72 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:22.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.741+0000 7ff22affd700 1 -- 172.21.15.92:0/1758049729 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21c004230 con 0x7ff22c0b8510 2023-11-28T11:25:22.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.741+0000 7ff22affd700 1 -- 172.21.15.92:0/1758049729 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff21c004390 con 0x7ff22c0b8510 2023-11-28T11:25:22.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.741+0000 7ff22affd700 1 -- 172.21.15.92:0/1758049729 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21c009e70 con 0x7ff22c0b8510 2023-11-28T11:25:22.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.742+0000 7ff231fff700 1 -- 172.21.15.92:0/1758049729 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff22c0b8510 msgr2=0x7ff22c06b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:22.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.742+0000 7ff231fff700 1 --2- 172.21.15.92:0/1758049729 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff22c0b8510 0x7ff22c06b970 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7ff21c009920 tx=0x7ff21c009c30 comp rx=0 tx=0).stop 2023-11-28T11:25:22.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.742+0000 7ff231fff700 1 -- 172.21.15.92:0/1758049729 shutdown_connections 2023-11-28T11:25:22.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.742+0000 7ff231fff700 1 --2- 172.21.15.92:0/1758049729 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff22c0b8510 0x7ff22c06b970 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:22.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.742+0000 7ff231fff700 1 --2- 172.21.15.92:0/1758049729 >> v2:172.21.15.169:3300/0 conn(0x7ff22c0cd460 0x7ff22c06beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:22.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.742+0000 7ff231fff700 1 -- 172.21.15.92:0/1758049729 >> 172.21.15.92:0/1758049729 conn(0x7ff22c0719a0 msgr2=0x7ff22c071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:22.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.742+0000 7ff231fff700 1 -- 172.21.15.92:0/1758049729 shutdown_connections 2023-11-28T11:25:22.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.742+0000 7ff231fff700 1 -- 172.21.15.92:0/1758049729 wait complete. 2023-11-28T11:25:22.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.743+0000 7ff231fff700 1 Processor -- start 2023-11-28T11:25:22.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.744+0000 7ff231fff700 1 -- start start 2023-11-28T11:25:22.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.745+0000 7ff231fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff22c06c3f0 0x7ff22c1e3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:22.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.745+0000 7ff231fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff22c0b8510 0x7ff22c15a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:22.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.745+0000 7ff231fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff22c0cd460 0x7ff22c1e71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:22.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.745+0000 7ff231fff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff22c1e2380 con 0x7ff22c06c3f0 2023-11-28T11:25:22.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.745+0000 7ff231fff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff22c1e2660 con 0x7ff22c0cd460 2023-11-28T11:25:22.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.745+0000 7ff231fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff22c1e2940 con 0x7ff22c0b8510 2023-11-28T11:25:22.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.745+0000 7ff22b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff22c06c3f0 0x7ff22c1e3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:22.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.745+0000 7ff22b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff22c06c3f0 0x7ff22c1e3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44624/0 (socket says 172.21.15.92:44624) 2023-11-28T11:25:22.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.745+0000 7ff22b7fe700 1 -- 172.21.15.92:0/1297675643 learned_addr learned my addr 172.21.15.92:0/1297675643 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:22.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.745+0000 7ff22bfff700 1 --2- 172.21.15.92:0/1297675643 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff22c0cd460 0x7ff22c1e71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:22.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.745+0000 7ff223fff700 1 --2- 172.21.15.92:0/1297675643 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff22c0b8510 0x7ff22c15a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:22.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.745+0000 7ff22b7fe700 1 -- 172.21.15.92:0/1297675643 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff22c0b8510 msgr2=0x7ff22c15a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:22.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.745+0000 7ff22b7fe700 1 --2- 172.21.15.92:0/1297675643 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff22c0b8510 0x7ff22c15a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:22.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.745+0000 7ff22b7fe700 1 -- 172.21.15.92:0/1297675643 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff22c0cd460 msgr2=0x7ff22c1e71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:22.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.745+0000 7ff22b7fe700 1 --2- 172.21.15.92:0/1297675643 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff22c0cd460 0x7ff22c1e71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:22.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.745+0000 7ff22b7fe700 1 -- 172.21.15.92:0/1297675643 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff21c009550 con 0x7ff22c06c3f0 2023-11-28T11:25:22.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.745+0000 7ff22bfff700 1 --2- 172.21.15.92:0/1297675643 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff22c0cd460 0x7ff22c1e71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:25:22.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.746+0000 7ff22b7fe700 1 --2- 172.21.15.92:0/1297675643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff22c06c3f0 0x7ff22c1e3420 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7ff21c009870 tx=0x7ff21c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:22.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.746+0000 7ff228ff9700 1 -- 172.21.15.92:0/1297675643 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21c0036a0 con 0x7ff22c06c3f0 2023-11-28T11:25:22.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.746+0000 7ff228ff9700 1 -- 172.21.15.92:0/1297675643 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff21c003800 con 0x7ff22c06c3f0 2023-11-28T11:25:22.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.746+0000 7ff231fff700 1 -- 172.21.15.92:0/1297675643 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff22c0cfff0 con 0x7ff22c06c3f0 2023-11-28T11:25:22.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.746+0000 7ff228ff9700 1 -- 172.21.15.92:0/1297675643 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21c003b90 con 0x7ff22c06c3f0 2023-11-28T11:25:22.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.746+0000 7ff231fff700 1 -- 172.21.15.92:0/1297675643 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff22c0cfd10 con 0x7ff22c06c3f0 2023-11-28T11:25:22.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.747+0000 7ff231fff700 1 -- 172.21.15.92:0/1297675643 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff22c067880 con 0x7ff22c06c3f0 2023-11-28T11:25:22.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.748+0000 7ff228ff9700 1 -- 172.21.15.92:0/1297675643 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff21c017020 con 0x7ff22c06c3f0 2023-11-28T11:25:22.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.749+0000 7ff228ff9700 1 --2- 172.21.15.92:0/1297675643 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff20c0682f0 0x7ff20c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:22.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.749+0000 7ff228ff9700 1 -- 172.21.15.92:0/1297675643 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6556+0+0 (secure 0 0 0) 0x7ff21c087c80 con 0x7ff22c06c3f0 2023-11-28T11:25:22.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.749+0000 7ff223fff700 1 --2- 172.21.15.92:0/1297675643 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff20c0682f0 0x7ff20c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:22.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.750+0000 7ff223fff700 1 --2- 172.21.15.92:0/1297675643 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff20c0682f0 0x7ff20c06a7a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff22c1e6ea0 tx=0x7ff214008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:22.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.755+0000 7ff228ff9700 1 -- 172.21.15.92:0/1297675643 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff21c0542f0 con 0x7ff22c06c3f0 2023-11-28T11:25:22.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.960+0000 7ff231fff700 1 -- 172.21.15.92:0/1297675643 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff22c0d2940 con 0x7ff22c06c3f0 2023-11-28T11:25:22.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.961+0000 7ff228ff9700 1 -- 172.21.15.92:0/1297675643 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v70) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff21c057910 con 0x7ff22c06c3f0 2023-11-28T11:25:22.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.963+0000 7ff231fff700 1 -- 172.21.15.92:0/1297675643 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff20c0682f0 msgr2=0x7ff20c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:22.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.963+0000 7ff231fff700 1 --2- 172.21.15.92:0/1297675643 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff20c0682f0 0x7ff20c06a7a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff22c1e6ea0 tx=0x7ff214008040 comp rx=0 tx=0).stop 2023-11-28T11:25:22.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.963+0000 7ff231fff700 1 -- 172.21.15.92:0/1297675643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff22c06c3f0 msgr2=0x7ff22c1e3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:22.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.963+0000 7ff231fff700 1 --2- 172.21.15.92:0/1297675643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff22c06c3f0 0x7ff22c1e3420 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7ff21c009870 tx=0x7ff21c004750 comp rx=0 tx=0).stop 2023-11-28T11:25:22.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.963+0000 7ff231fff700 1 -- 172.21.15.92:0/1297675643 shutdown_connections 2023-11-28T11:25:22.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.963+0000 7ff231fff700 1 --2- 172.21.15.92:0/1297675643 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff22c0cd460 0x7ff22c1e71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:22.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.963+0000 7ff231fff700 1 --2- 172.21.15.92:0/1297675643 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff20c0682f0 0x7ff20c06a7a0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:22.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.963+0000 7ff231fff700 1 --2- 172.21.15.92:0/1297675643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff22c06c3f0 0x7ff22c1e3420 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:22.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.963+0000 7ff231fff700 1 --2- 172.21.15.92:0/1297675643 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff22c0b8510 0x7ff22c15a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:22.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.963+0000 7ff231fff700 1 -- 172.21.15.92:0/1297675643 >> 172.21.15.92:0/1297675643 conn(0x7ff22c0719a0 msgr2=0x7ff22c0b6850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:22.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.963+0000 7ff231fff700 1 -- 172.21.15.92:0/1297675643 shutdown_connections 2023-11-28T11:25:22.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:22.963+0000 7ff231fff700 1 -- 172.21.15.92:0/1297675643 wait complete. 2023-11-28T11:25:22.966 INFO:teuthology.orchestra.run.smithi092.stderr:listed 0 entries 2023-11-28T11:25:22.983 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-28T11:25:22.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:25:22.984 DEBUG:teuthology.orchestra.run.smithi092:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-28T11:25:23.085 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2023-11-28T11:25:23.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:25:23.086 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T11:25:23.143 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-28T11:25:23.143 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:25:23.193 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348296k,nr_inodes=4087074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275620k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:25:23.193 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-28T11:25:23.193 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:25:23.240 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348288k,nr_inodes=4087072,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275616k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:25:23.240 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-28T11:25:23.240 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:25:23.288 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348296k,nr_inodes=4087074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275620k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:25:23.288 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:25:23.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.481+0000 7f91053ff700 1 Processor -- start 2023-11-28T11:25:23.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.481+0000 7f91053ff700 1 -- start start 2023-11-28T11:25:23.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.482+0000 7f91053ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f91000b8510 0x7f910006b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:23.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.483+0000 7f91053ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91000d0d60 0x7f91000d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:23.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.483+0000 7f91053ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91000b88e0 con 0x7f91000cd460 2023-11-28T11:25:23.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.483+0000 7f91053ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91000cdbd0 con 0x7f91000b8510 2023-11-28T11:25:23.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.483+0000 7f91053ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91000d02d0 con 0x7f91000d0d60 2023-11-28T11:25:23.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.483+0000 7f90feffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f91000b8510 0x7f910006b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:23.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.483+0000 7f90f67fc700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f91000cd460 0x7f910006beb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:58994/0 (socket says 172.21.15.92:58994) 2023-11-28T11:25:23.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.483+0000 7f90f67fc700 1 -- 172.21.15.92:0/2811329220 learned_addr learned my addr 172.21.15.92:0/2811329220 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:25:23.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.483+0000 7f90ff7fe700 1 --2- 172.21.15.92:0/2811329220 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91000d0d60 0x7f91000d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:23.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.483+0000 7f90fe7fc700 1 -- 172.21.15.92:0/2811329220 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4177062559 0 0) 0x7f91000b88e0 con 0x7f91000cd460 2023-11-28T11:25:23.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.483+0000 7f90fe7fc700 1 -- 172.21.15.92:0/2811329220 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90e00035b0 con 0x7f91000cd460 2023-11-28T11:25:23.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.483+0000 7f90feffd700 1 -- 172.21.15.92:0/2811329220 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91000d0d60 msgr2=0x7f91000d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:23.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.483+0000 7f90feffd700 1 --2- 172.21.15.92:0/2811329220 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91000d0d60 0x7f91000d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:23.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.483+0000 7f90feffd700 1 -- 172.21.15.92:0/2811329220 >> v1:172.21.15.92:6789/0 conn(0x7f91000cd460 legacy=0x7f910006beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:23.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.483+0000 7f90feffd700 1 -- 172.21.15.92:0/2811329220 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91000cd8f0 con 0x7f91000b8510 2023-11-28T11:25:23.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.484+0000 7f90feffd700 1 --2- 172.21.15.92:0/2811329220 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91000b8510 0x7f910006b970 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f90f0009870 tx=0x7f90f0009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d94da8190d7ad19d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:23.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.484+0000 7f90fe7fc700 1 -- 172.21.15.92:0/2811329220 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f0011830 con 0x7f91000b8510 2023-11-28T11:25:23.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.484+0000 7f90fe7fc700 1 -- 172.21.15.92:0/2811329220 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90f0011990 con 0x7f91000b8510 2023-11-28T11:25:23.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.484+0000 7f90fe7fc700 1 -- 172.21.15.92:0/2811329220 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f0011d20 con 0x7f91000b8510 2023-11-28T11:25:23.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.484+0000 7f91053ff700 1 -- 172.21.15.92:0/2811329220 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91000b8510 msgr2=0x7f910006b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:23.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.484+0000 7f91053ff700 1 --2- 172.21.15.92:0/2811329220 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91000b8510 0x7f910006b970 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f90f0009870 tx=0x7f90f0009b80 comp rx=0 tx=0).stop 2023-11-28T11:25:23.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.485+0000 7f91053ff700 1 -- 172.21.15.92:0/2811329220 shutdown_connections 2023-11-28T11:25:23.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.485+0000 7f91053ff700 1 --2- 172.21.15.92:0/2811329220 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91000d0d60 0x7f91000d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:23.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.485+0000 7f91053ff700 1 --2- 172.21.15.92:0/2811329220 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91000b8510 0x7f910006b970 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:23.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.485+0000 7f91053ff700 1 -- 172.21.15.92:0/2811329220 >> 172.21.15.92:0/2811329220 conn(0x7f91000719a0 msgr2=0x7f9100071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:23.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.485+0000 7f91053ff700 1 -- 172.21.15.92:0/2811329220 shutdown_connections 2023-11-28T11:25:23.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.485+0000 7f91053ff700 1 -- 172.21.15.92:0/2811329220 wait complete. 2023-11-28T11:25:23.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.486+0000 7f91053ff700 1 Processor -- start 2023-11-28T11:25:23.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.487+0000 7f91053ff700 1 -- start start 2023-11-28T11:25:23.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.487+0000 7f91053ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91000b8510 0x7f91000ccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:23.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.487+0000 7f91053ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91000cd460 0x7f9100157950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:23.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.487+0000 7f91053ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91000d0d60 0x7f9100150640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:23.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.487+0000 7f91053ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9100157e90 con 0x7f91000b8510 2023-11-28T11:25:23.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.487+0000 7f91053ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9100158130 con 0x7f91000d0d60 2023-11-28T11:25:23.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.488+0000 7f91053ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9100158410 con 0x7f91000cd460 2023-11-28T11:25:23.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.488+0000 7f90feffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91000b8510 0x7f91000ccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:23.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.488+0000 7f90feffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91000b8510 0x7f91000ccbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44652/0 (socket says 172.21.15.92:44652) 2023-11-28T11:25:23.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.488+0000 7f90feffd700 1 -- 172.21.15.92:0/3356948178 learned_addr learned my addr 172.21.15.92:0/3356948178 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:23.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.488+0000 7f90f67fc700 1 --2- 172.21.15.92:0/3356948178 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91000cd460 0x7f9100157950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:23.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.488+0000 7f90feffd700 1 -- 172.21.15.92:0/3356948178 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91000cd460 msgr2=0x7f9100157950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:23.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.488+0000 7f90ff7fe700 1 --2- 172.21.15.92:0/3356948178 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91000d0d60 0x7f9100150640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:23.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.488+0000 7f90feffd700 1 --2- 172.21.15.92:0/3356948178 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91000cd460 0x7f9100157950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:23.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.488+0000 7f90feffd700 1 -- 172.21.15.92:0/3356948178 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91000d0d60 msgr2=0x7f9100150640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:23.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.488+0000 7f90feffd700 1 --2- 172.21.15.92:0/3356948178 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91000d0d60 0x7f9100150640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:23.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.488+0000 7f90feffd700 1 -- 172.21.15.92:0/3356948178 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f00094a0 con 0x7f91000b8510 2023-11-28T11:25:23.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.488+0000 7f90f67fc700 1 --2- 172.21.15.92:0/3356948178 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91000cd460 0x7f9100157950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:25:23.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.488+0000 7f90ff7fe700 1 --2- 172.21.15.92:0/3356948178 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91000d0d60 0x7f9100150640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:25:23.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.488+0000 7f90feffd700 1 --2- 172.21.15.92:0/3356948178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91000b8510 0x7f91000ccbb0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f90f0000c00 tx=0x7f90f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:23.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.489+0000 7f90f7fff700 1 -- 172.21.15.92:0/3356948178 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f00036a0 con 0x7f91000b8510 2023-11-28T11:25:23.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.489+0000 7f90f7fff700 1 -- 172.21.15.92:0/3356948178 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90f001f070 con 0x7f91000b8510 2023-11-28T11:25:23.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.489+0000 7f90f7fff700 1 -- 172.21.15.92:0/3356948178 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f0003a40 con 0x7f91000b8510 2023-11-28T11:25:23.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.489+0000 7f91053ff700 1 -- 172.21.15.92:0/3356948178 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91000cfff0 con 0x7f91000b8510 2023-11-28T11:25:23.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.489+0000 7f91053ff700 1 -- 172.21.15.92:0/3356948178 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91000cfd10 con 0x7f91000b8510 2023-11-28T11:25:23.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.490+0000 7f91053ff700 1 -- 172.21.15.92:0/3356948178 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9100067880 con 0x7f91000b8510 2023-11-28T11:25:23.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.491+0000 7f90f7fff700 1 -- 172.21.15.92:0/3356948178 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f90f000f050 con 0x7f91000b8510 2023-11-28T11:25:23.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.491+0000 7f90f7fff700 1 --2- 172.21.15.92:0/3356948178 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f90e40682f0 0x7f90e406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:23.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.492+0000 7f90f7fff700 1 -- 172.21.15.92:0/3356948178 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6556+0+0 (secure 0 0 0) 0x7f90f0021070 con 0x7f91000b8510 2023-11-28T11:25:23.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.492+0000 7f90f67fc700 1 --2- 172.21.15.92:0/3356948178 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f90e40682f0 0x7f90e406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:23.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.492+0000 7f90f67fc700 1 --2- 172.21.15.92:0/3356948178 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f90e40682f0 0x7f90e406a7a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f90e80011e0 tx=0x7f90e8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:23.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.496+0000 7f90f7fff700 1 -- 172.21.15.92:0/3356948178 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f90f0053dc0 con 0x7f91000b8510 2023-11-28T11:25:23.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.731+0000 7f91053ff700 1 -- 172.21.15.92:0/3356948178 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f91001e3d80 con 0x7f91000b8510 2023-11-28T11:25:23.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.731+0000 7f90f7fff700 1 -- 172.21.15.92:0/3356948178 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4656 (secure 0 0 0) 0x7f90f0015090 con 0x7f91000b8510 2023-11-28T11:25:23.733 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:25:23.734 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5525,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2067956596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2067956596},{"type":"v1","addr":"172.21.15.169:6835","nonce":2067956596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5971,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2367049160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2367049160},{"type":"v1","addr":"172.21.15.169:6834","nonce":2367049160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5977,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1424858585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":1424858585},{"type":"v1","addr":"172.21.15.92:6837","nonce":1424858585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:24:16.257826+0000","modified":"2023-11-28T11:24:32.577560+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6213},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6213":{"gid":6213,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6836/3634521248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3634521248},{"type":"v1","addr":"172.21.15.92:6836","nonce":3634521248}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T11:25:23.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.734+0000 7f91053ff700 1 -- 172.21.15.92:0/3356948178 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f90e40682f0 msgr2=0x7f90e406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:23.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.734+0000 7f91053ff700 1 --2- 172.21.15.92:0/3356948178 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f90e40682f0 0x7f90e406a7a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f90e80011e0 tx=0x7f90e8009040 comp rx=0 tx=0).stop 2023-11-28T11:25:23.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.734+0000 7f91053ff700 1 -- 172.21.15.92:0/3356948178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91000b8510 msgr2=0x7f91000ccbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:23.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.734+0000 7f91053ff700 1 --2- 172.21.15.92:0/3356948178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91000b8510 0x7f91000ccbb0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f90f0000c00 tx=0x7f90f0004750 comp rx=0 tx=0).stop 2023-11-28T11:25:23.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.734+0000 7f91053ff700 1 -- 172.21.15.92:0/3356948178 shutdown_connections 2023-11-28T11:25:23.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.734+0000 7f91053ff700 1 --2- 172.21.15.92:0/3356948178 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f91000d0d60 0x7f9100150640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:23.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.734+0000 7f91053ff700 1 --2- 172.21.15.92:0/3356948178 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f90e40682f0 0x7f90e406a7a0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:23.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.734+0000 7f91053ff700 1 --2- 172.21.15.92:0/3356948178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91000b8510 0x7f91000ccbb0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:23.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.734+0000 7f91053ff700 1 --2- 172.21.15.92:0/3356948178 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f91000cd460 0x7f9100157950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:23.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.734+0000 7f91053ff700 1 -- 172.21.15.92:0/3356948178 >> 172.21.15.92:0/3356948178 conn(0x7f91000719a0 msgr2=0x7f91000b66b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:23.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.734+0000 7f91053ff700 1 -- 172.21.15.92:0/3356948178 shutdown_connections 2023-11-28T11:25:23.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.734+0000 7f91053ff700 1 -- 172.21.15.92:0/3356948178 wait complete. 2023-11-28T11:25:23.739 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 33 2023-11-28T11:25:23.754 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:25:23.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.949+0000 7f83a43ff700 1 Processor -- start 2023-11-28T11:25:23.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.949+0000 7f83a43ff700 1 -- start start 2023-11-28T11:25:23.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.950+0000 7f83a43ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f839c0b8510 0x7f839c06b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:23.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.950+0000 7f83a43ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f839c0b88e0 con 0x7f839c0d0d60 2023-11-28T11:25:23.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.950+0000 7f83a43ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f839c0cd8f0 con 0x7f839c0cd460 2023-11-28T11:25:23.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.950+0000 7f83a43ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f839c0cdbd0 con 0x7f839c0b8510 2023-11-28T11:25:23.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.951+0000 7f83a299c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f839c0d0d60 0x7f839c0d11a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59008/0 (socket says 172.21.15.92:59008) 2023-11-28T11:25:23.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.951+0000 7f83a299c700 1 -- 172.21.15.92:0/545528877 learned_addr learned my addr 172.21.15.92:0/545528877 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:25:23.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.951+0000 7f83a219b700 1 --2- 172.21.15.92:0/545528877 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f839c0b8510 0x7f839c06b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:23.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.951+0000 7f83a1199700 1 -- 172.21.15.92:0/545528877 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2459355717 0 0) 0x7f839c0b88e0 con 0x7f839c0d0d60 2023-11-28T11:25:23.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.951+0000 7f83a1199700 1 -- 172.21.15.92:0/545528877 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83880035b0 con 0x7f839c0d0d60 2023-11-28T11:25:23.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.951+0000 7f83a1199700 1 -- 172.21.15.92:0/545528877 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1764928991 0 0) 0x7f839c0cd8f0 con 0x7f839c0cd460 2023-11-28T11:25:23.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.951+0000 7f83a1199700 1 -- 172.21.15.92:0/545528877 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f839c0b88e0 con 0x7f839c0cd460 2023-11-28T11:25:23.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.951+0000 7f83a1199700 1 -- 172.21.15.92:0/545528877 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2421514889 0 0) 0x7f83880035b0 con 0x7f839c0d0d60 2023-11-28T11:25:23.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.951+0000 7f83a1199700 1 -- 172.21.15.92:0/545528877 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f839c0b8510 msgr2=0x7f839c06b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:23.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.951+0000 7f83a1199700 1 --2- 172.21.15.92:0/545528877 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f839c0b8510 0x7f839c06b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:23.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.951+0000 7f83a1199700 1 -- 172.21.15.92:0/545528877 >> v1:172.21.15.169:6789/0 conn(0x7f839c0cd460 legacy=0x7f839c06beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:23.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.951+0000 7f83a1199700 1 -- 172.21.15.92:0/545528877 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f839c0cfd10 con 0x7f839c0d0d60 2023-11-28T11:25:23.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.951+0000 7f83a1199700 1 -- 172.21.15.92:0/545528877 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532161587 0 0) 0x7f83940019f0 con 0x7f839c0d0d60 2023-11-28T11:25:23.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.951+0000 7f83a219b700 1 --2- 172.21.15.92:0/545528877 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f839c0b8510 0x7f839c06b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:25:23.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.951+0000 7f83a1199700 1 -- 172.21.15.92:0/545528877 >> v1:172.21.15.92:6789/0 conn(0x7f839c0d0d60 legacy=0x7f839c0d11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:23.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.952+0000 7f83a1199700 1 --2- 172.21.15.92:0/545528877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8388003bc0 0x7f8388003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:23.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.952+0000 7f83a1199700 1 --2- 172.21.15.92:0/545528877 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f839c0b8510 0x7f8388004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:23.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.952+0000 7f83a1199700 1 --2- 172.21.15.92:0/545528877 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8388004d90 0x7f8388005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:23.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.952+0000 7f83a1199700 1 -- 172.21.15.92:0/545528877 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f839c0cdbd0 con 0x7f8388003bc0 2023-11-28T11:25:23.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.952+0000 7f83a1199700 1 -- 172.21.15.92:0/545528877 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f838800bac0 con 0x7f8388004d90 2023-11-28T11:25:23.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.952+0000 7f83a1199700 1 -- 172.21.15.92:0/545528877 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f838800bc00 con 0x7f839c0b8510 2023-11-28T11:25:23.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.952+0000 7f83a219b700 1 --2- 172.21.15.92:0/545528877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8388003bc0 0x7f8388003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:23.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.952+0000 7f83a299c700 1 --2- 172.21.15.92:0/545528877 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8388004d90 0x7f8388005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:23.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.952+0000 7f83a199a700 1 --2- 172.21.15.92:0/545528877 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f839c0b8510 0x7f8388004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:23.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.952+0000 7f83a219b700 1 -- 172.21.15.92:0/545528877 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f839c0b8510 msgr2=0x7f8388004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:23.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.952+0000 7f83a219b700 1 --2- 172.21.15.92:0/545528877 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f839c0b8510 0x7f8388004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:23.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.952+0000 7f83a219b700 1 -- 172.21.15.92:0/545528877 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8388004d90 msgr2=0x7f8388005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:23.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.952+0000 7f83a219b700 1 --2- 172.21.15.92:0/545528877 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8388004d90 0x7f8388005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:23.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.952+0000 7f83a219b700 1 -- 172.21.15.92:0/545528877 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83980094a0 con 0x7f8388003bc0 2023-11-28T11:25:23.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.952+0000 7f83a299c700 1 --2- 172.21.15.92:0/545528877 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f8388004d90 0x7f8388005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:25:23.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.952+0000 7f83a219b700 1 --2- 172.21.15.92:0/545528877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8388003bc0 0x7f8388003fb0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f8398009d30 tx=0x7f8388005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2c34732d9e5166c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:23.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.953+0000 7f83a1199700 1 -- 172.21.15.92:0/545528877 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8398009e80 con 0x7f8388003bc0 2023-11-28T11:25:23.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.953+0000 7f83a1199700 1 -- 172.21.15.92:0/545528877 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f839800e9a0 con 0x7f8388003bc0 2023-11-28T11:25:23.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.953+0000 7f83a1199700 1 -- 172.21.15.92:0/545528877 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f839800ed10 con 0x7f8388003bc0 2023-11-28T11:25:23.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.953+0000 7f83a43ff700 1 -- 172.21.15.92:0/545528877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8388003bc0 msgr2=0x7f8388003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:23.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.953+0000 7f83a43ff700 1 --2- 172.21.15.92:0/545528877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8388003bc0 0x7f8388003fb0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f8398009d30 tx=0x7f8388005880 comp rx=0 tx=0).stop 2023-11-28T11:25:23.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.953+0000 7f83a299c700 1 -- 172.21.15.92:0/545528877 reap_dead start 2023-11-28T11:25:23.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.953+0000 7f83a43ff700 1 -- 172.21.15.92:0/545528877 shutdown_connections 2023-11-28T11:25:23.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.953+0000 7f83a43ff700 1 -- 172.21.15.92:0/545528877 >> 172.21.15.92:0/545528877 conn(0x7f839c0719a0 msgr2=0x7f839c071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:23.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.953+0000 7f83a43ff700 1 -- 172.21.15.92:0/545528877 shutdown_connections 2023-11-28T11:25:23.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.953+0000 7f83a43ff700 1 -- 172.21.15.92:0/545528877 wait complete. 2023-11-28T11:25:23.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.955+0000 7f83a43ff700 1 Processor -- start 2023-11-28T11:25:23.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.955+0000 7f83a43ff700 1 -- start start 2023-11-28T11:25:23.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.956+0000 7f83a43ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8388003bc0 0x7f839c154330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:23.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.956+0000 7f83a43ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f839c0c9ee0 0x7f839c15b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:23.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.956+0000 7f83a43ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f839c1f24f0 0x7f839c1f2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:23.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.956+0000 7f83a43ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f839c154870 con 0x7f8388003bc0 2023-11-28T11:25:23.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.956+0000 7f83a43ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f839c0ca3e0 con 0x7f839c0c9ee0 2023-11-28T11:25:23.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.956+0000 7f83a43ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f839c0ca6c0 con 0x7f839c1f24f0 2023-11-28T11:25:23.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.956+0000 7f83a219b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8388003bc0 0x7f839c154330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:23.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.956+0000 7f83a219b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8388003bc0 0x7f839c154330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44664/0 (socket says 172.21.15.92:44664) 2023-11-28T11:25:23.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.956+0000 7f83a219b700 1 -- 172.21.15.92:0/154257514 learned_addr learned my addr 172.21.15.92:0/154257514 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:23.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.957+0000 7f83a219b700 1 -- 172.21.15.92:0/154257514 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f839c1f24f0 msgr2=0x7f839c1f2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:23.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.957+0000 7f83a219b700 1 --2- 172.21.15.92:0/154257514 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f839c1f24f0 0x7f839c1f2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:23.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.957+0000 7f83a219b700 1 -- 172.21.15.92:0/154257514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f839c0c9ee0 msgr2=0x7f839c15b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:23.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.957+0000 7f83a219b700 1 --2- 172.21.15.92:0/154257514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f839c0c9ee0 0x7f839c15b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:23.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.957+0000 7f83a219b700 1 -- 172.21.15.92:0/154257514 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f839c0d29c0 con 0x7f8388003bc0 2023-11-28T11:25:23.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.957+0000 7f83a219b700 1 --2- 172.21.15.92:0/154257514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8388003bc0 0x7f839c154330 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f839800e790 tx=0x7f83980047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:23.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.957+0000 7f8392ffd700 1 -- 172.21.15.92:0/154257514 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8398003ab0 con 0x7f8388003bc0 2023-11-28T11:25:23.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.957+0000 7f8392ffd700 1 -- 172.21.15.92:0/154257514 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8398003c30 con 0x7f8388003bc0 2023-11-28T11:25:23.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.957+0000 7f83a43ff700 1 -- 172.21.15.92:0/154257514 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f839c0cfff0 con 0x7f8388003bc0 2023-11-28T11:25:23.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.957+0000 7f8392ffd700 1 -- 172.21.15.92:0/154257514 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8398009e80 con 0x7f8388003bc0 2023-11-28T11:25:23.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.957+0000 7f83a43ff700 1 -- 172.21.15.92:0/154257514 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f839c0d2b50 con 0x7f8388003bc0 2023-11-28T11:25:23.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.959+0000 7f83a43ff700 1 -- 172.21.15.92:0/154257514 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f839c0cdda0 con 0x7f8388003bc0 2023-11-28T11:25:23.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.959+0000 7f8392ffd700 1 -- 172.21.15.92:0/154257514 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8398025070 con 0x7f8388003bc0 2023-11-28T11:25:23.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.960+0000 7f8392ffd700 1 --2- 172.21.15.92:0/154257514 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f83800682f0 0x7f838006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:23.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.960+0000 7f8392ffd700 1 -- 172.21.15.92:0/154257514 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6556+0+0 (secure 0 0 0) 0x7f8398087a20 con 0x7f8388003bc0 2023-11-28T11:25:23.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.960+0000 7f83a199a700 1 --2- 172.21.15.92:0/154257514 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f83800682f0 0x7f838006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:23.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.961+0000 7f83a199a700 1 --2- 172.21.15.92:0/154257514 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f83800682f0 0x7f838006a7a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f838c007b70 tx=0x7f838c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:23.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:23.965+0000 7f8392ffd700 1 -- 172.21.15.92:0/154257514 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8398053fe0 con 0x7f8388003bc0 2023-11-28T11:25:24.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.155+0000 7f83a43ff700 1 -- 172.21.15.92:0/154257514 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f839c067880 con 0x7f8388003bc0 2023-11-28T11:25:24.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.157+0000 7f8392ffd700 1 -- 172.21.15.92:0/154257514 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+18417 (secure 0 0 0) 0x7f8398057600 con 0x7f8388003bc0 2023-11-28T11:25:24.159 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:25:24.159 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":70,"fsid":"68956e0c-c16f-47cd-a220-f98364b0831d","created":"2023-11-28T11:20:28.697573+0000","modified":"2023-11-28T11:24:38.644982+0000","last_up_change":"2023-11-28T11:20:41.257011+0000","last_in_change":"2023-11-28T11:20:32.249767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:20:41.925237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:20:48.682497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:24:13.972356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-28T11:24:15.237693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6abe483f-0e03-4a26-b9cf-62041291747e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":115823},{"type":"v1","addr":"172.21.15.92:6809","nonce":115823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":115823},{"type":"v1","addr":"172.21.15.92:6813","nonce":115823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":115823},{"type":"v1","addr":"172.21.15.92:6821","nonce":115823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":115823},{"type":"v1","addr":"172.21.15.92:6817","nonce":115823}]},"public_addr":"172.21.15.92:6809/115823","cluster_addr":"172.21.15.92:6813/115823","heartbeat_back_addr":"172.21.15.92:6821/115823","heartbeat_front_addr":"172.21.15.92:6817/115823","state":["exists","up"]},{"osd":1,"uuid":"13c1de73-1a8c-4d7e-8a28-9dae52dfe089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":115824},{"type":"v1","addr":"172.21.15.92:6812","nonce":115824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":115824},{"type":"v1","addr":"172.21.15.92:6816","nonce":115824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":115824},{"type":"v1","addr":"172.21.15.92:6823","nonce":115824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":115824},{"type":"v1","addr":"172.21.15.92:6820","nonce":115824}]},"public_addr":"172.21.15.92:6812/115824","cluster_addr":"172.21.15.92:6816/115824","heartbeat_back_addr":"172.21.15.92:6823/115824","heartbeat_front_addr":"172.21.15.92:6820/115824","state":["exists","up"]},{"osd":2,"uuid":"25788a55-112f-4f00-810d-2bc37cf76778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":115821},{"type":"v1","addr":"172.21.15.92:6801","nonce":115821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":115821},{"type":"v1","addr":"172.21.15.92:6803","nonce":115821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":115821},{"type":"v1","addr":"172.21.15.92:6807","nonce":115821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":115821},{"type":"v1","addr":"172.21.15.92:6805","nonce":115821}]},"public_addr":"172.21.15.92:6801/115821","cluster_addr":"172.21.15.92:6803/115821","heartbeat_back_addr":"172.21.15.92:6807/115821","heartbeat_front_addr":"172.21.15.92:6805/115821","state":["exists","up"]},{"osd":3,"uuid":"adf54e24-33a9-41fe-9ce3-98a5ab6c158e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":115825},{"type":"v1","addr":"172.21.15.92:6825","nonce":115825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":115825},{"type":"v1","addr":"172.21.15.92:6827","nonce":115825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":115825},{"type":"v1","addr":"172.21.15.92:6831","nonce":115825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":115825},{"type":"v1","addr":"172.21.15.92:6829","nonce":115825}]},"public_addr":"172.21.15.92:6825/115825","cluster_addr":"172.21.15.92:6827/115825","heartbeat_back_addr":"172.21.15.92:6831/115825","heartbeat_front_addr":"172.21.15.92:6829/115825","state":["exists","up"]},{"osd":4,"uuid":"ccb9b747-f399-44ea-9a18-9db3152227e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":116035},{"type":"v1","addr":"172.21.15.169:6817","nonce":116035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":116035},{"type":"v1","addr":"172.21.15.169:6819","nonce":116035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":116035},{"type":"v1","addr":"172.21.15.169:6826","nonce":116035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":116035},{"type":"v1","addr":"172.21.15.169:6822","nonce":116035}]},"public_addr":"172.21.15.169:6817/116035","cluster_addr":"172.21.15.169:6819/116035","heartbeat_back_addr":"172.21.15.169:6826/116035","heartbeat_front_addr":"172.21.15.169:6822/116035","state":["exists","up"]},{"osd":5,"uuid":"308a1e63-75aa-4d7d-8a7c-cecc24714fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":116033},{"type":"v1","addr":"172.21.15.169:6801","nonce":116033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":116033},{"type":"v1","addr":"172.21.15.169:6803","nonce":116033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":116033},{"type":"v1","addr":"172.21.15.169:6807","nonce":116033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":116033},{"type":"v1","addr":"172.21.15.169:6805","nonce":116033}]},"public_addr":"172.21.15.169:6801/116033","cluster_addr":"172.21.15.169:6803/116033","heartbeat_back_addr":"172.21.15.169:6807/116033","heartbeat_front_addr":"172.21.15.169:6805/116033","state":["exists","up"]},{"osd":6,"uuid":"9c459b7b-b821-4b4b-b91e-6aaae271457c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":116034},{"type":"v1","addr":"172.21.15.169:6809","nonce":116034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":116034},{"type":"v1","addr":"172.21.15.169:6811","nonce":116034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":116034},{"type":"v1","addr":"172.21.15.169:6815","nonce":116034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":116034},{"type":"v1","addr":"172.21.15.169:6813","nonce":116034}]},"public_addr":"172.21.15.169:6809/116034","cluster_addr":"172.21.15.169:6811/116034","heartbeat_back_addr":"172.21.15.169:6815/116034","heartbeat_front_addr":"172.21.15.169:6813/116034","state":["exists","up"]},{"osd":7,"uuid":"1ca9aa4a-755d-4e28-a937-4d8dccab4df3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":116044},{"type":"v1","addr":"172.21.15.169:6823","nonce":116044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6825","nonce":116044},{"type":"v1","addr":"172.21.15.169:6827","nonce":116044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":116044},{"type":"v1","addr":"172.21.15.169:6831","nonce":116044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":116044},{"type":"v1","addr":"172.21.15.169:6829","nonce":116044}]},"public_addr":"172.21.15.169:6823/116044","cluster_addr":"172.21.15.169:6827/116044","heartbeat_back_addr":"172.21.15.169:6831/116044","heartbeat_front_addr":"172.21.15.169:6829/116044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.586781+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.869778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.508594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.860493+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.872267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.746933+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.611520+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.871446+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.10","mappings":[{"from":6,"to":5}]},{"pgid":"11.17","mappings":[{"from":6,"to":1}]},{"pgid":"11.29","mappings":[{"from":7,"to":0}]},{"pgid":"11.31","mappings":[{"from":2,"to":4}]},{"pgid":"11.33","mappings":[{"from":6,"to":1}]},{"pgid":"11.3e","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:25:24.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.159+0000 7f83a43ff700 1 -- 172.21.15.92:0/154257514 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f83800682f0 msgr2=0x7f838006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:24.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.159+0000 7f83a43ff700 1 --2- 172.21.15.92:0/154257514 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f83800682f0 0x7f838006a7a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f838c007b70 tx=0x7f838c009040 comp rx=0 tx=0).stop 2023-11-28T11:25:24.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.159+0000 7f83a43ff700 1 -- 172.21.15.92:0/154257514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8388003bc0 msgr2=0x7f839c154330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:24.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.159+0000 7f83a43ff700 1 --2- 172.21.15.92:0/154257514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8388003bc0 0x7f839c154330 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f839800e790 tx=0x7f83980047c0 comp rx=0 tx=0).stop 2023-11-28T11:25:24.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.159+0000 7f83a43ff700 1 -- 172.21.15.92:0/154257514 shutdown_connections 2023-11-28T11:25:24.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.159+0000 7f83a43ff700 1 --2- 172.21.15.92:0/154257514 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f839c1f24f0 0x7f839c1f2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:24.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.159+0000 7f83a43ff700 1 --2- 172.21.15.92:0/154257514 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f83800682f0 0x7f838006a7a0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:24.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.159+0000 7f83a43ff700 1 --2- 172.21.15.92:0/154257514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8388003bc0 0x7f839c154330 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:24.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.159+0000 7f83a43ff700 1 --2- 172.21.15.92:0/154257514 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f839c0c9ee0 0x7f839c15b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:24.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.159+0000 7f83a43ff700 1 -- 172.21.15.92:0/154257514 >> 172.21.15.92:0/154257514 conn(0x7f839c0719a0 msgr2=0x7f839c0b1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:24.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.159+0000 7f83a43ff700 1 -- 172.21.15.92:0/154257514 shutdown_connections 2023-11-28T11:25:24.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.159+0000 7f83a43ff700 1 -- 172.21.15.92:0/154257514 wait complete. 2023-11-28T11:25:24.180 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:20:41.925237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:20:48.682497+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:24:13.972356+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:24:15.237693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:25:24.181 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:25:24.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.380+0000 7fc49b3ff700 1 Processor -- start 2023-11-28T11:25:24.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.380+0000 7fc49b3ff700 1 -- start start 2023-11-28T11:25:24.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.381+0000 7fc49b3ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc4940cd460 0x7fc49406c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:24.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.381+0000 7fc49b3ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4940d0d60 0x7fc4940d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:24.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.381+0000 7fc49b3ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4940b88e0 con 0x7fc4940b8510 2023-11-28T11:25:24.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.381+0000 7fc49b3ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4940cdbd0 con 0x7fc4940cd460 2023-11-28T11:25:24.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.381+0000 7fc49b3ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4940d02d0 con 0x7fc4940d0d60 2023-11-28T11:25:24.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.382+0000 7fc49899a700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc4940cd460 0x7fc49406c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:24.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.382+0000 7fc49999c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4940d0d60 0x7fc4940d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:24.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.382+0000 7fc49999c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4940d0d60 0x7fc4940d1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.92:39222/0 (socket says 172.21.15.92:39222) 2023-11-28T11:25:24.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.382+0000 7fc49919b700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fc4940b8510 0x7fc49406b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.92:53826/0 (socket says 172.21.15.92:53826) 2023-11-28T11:25:24.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.382+0000 7fc49899a700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc4940cd460 0x7fc49406c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.92:51336/0 (socket says 172.21.15.92:51336) 2023-11-28T11:25:24.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.382+0000 7fc49999c700 1 -- 172.21.15.92:0/3865569990 learned_addr learned my addr 172.21.15.92:0/3865569990 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:24.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.382+0000 7fc49999c700 1 -- 172.21.15.92:0/3865569990 >> v2:172.21.15.169:3300/0 conn(0x7fc4940cd460 msgr2=0x7fc49406c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:24.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.382+0000 7fc49999c700 1 --2- 172.21.15.92:0/3865569990 >> v2:172.21.15.169:3300/0 conn(0x7fc4940cd460 0x7fc49406c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:24.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.382+0000 7fc49999c700 1 -- 172.21.15.92:0/3865569990 >> v1:172.21.15.169:6789/0 conn(0x7fc4940b8510 legacy=0x7fc49406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:24.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.382+0000 7fc49999c700 1 -- 172.21.15.92:0/3865569990 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4940cd8f0 con 0x7fc4940d0d60 2023-11-28T11:25:24.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.382+0000 7fc49999c700 1 --2- 172.21.15.92:0/3865569990 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4940d0d60 0x7fc4940d1200 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc48c009e50 tx=0x7fc48c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4c55389bacc8a82 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:24.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.383+0000 7fc48bfff700 1 -- 172.21.15.92:0/3865569990 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc48c00e070 con 0x7fc4940d0d60 2023-11-28T11:25:24.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.383+0000 7fc48bfff700 1 -- 172.21.15.92:0/3865569990 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc48c00b910 con 0x7fc4940d0d60 2023-11-28T11:25:24.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.383+0000 7fc48bfff700 1 -- 172.21.15.92:0/3865569990 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc48c00bcc0 con 0x7fc4940d0d60 2023-11-28T11:25:24.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.383+0000 7fc49b3ff700 1 -- 172.21.15.92:0/3865569990 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4940d0d60 msgr2=0x7fc4940d1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:24.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.383+0000 7fc49b3ff700 1 --2- 172.21.15.92:0/3865569990 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4940d0d60 0x7fc4940d1200 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc48c009e50 tx=0x7fc48c00b330 comp rx=0 tx=0).stop 2023-11-28T11:25:24.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.383+0000 7fc49b3ff700 1 -- 172.21.15.92:0/3865569990 shutdown_connections 2023-11-28T11:25:24.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.383+0000 7fc49b3ff700 1 --2- 172.21.15.92:0/3865569990 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4940d0d60 0x7fc4940d1200 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:24.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.383+0000 7fc49b3ff700 1 --2- 172.21.15.92:0/3865569990 >> v2:172.21.15.169:3300/0 conn(0x7fc4940cd460 0x7fc49406c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:24.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.383+0000 7fc49b3ff700 1 -- 172.21.15.92:0/3865569990 >> 172.21.15.92:0/3865569990 conn(0x7fc4940719a0 msgr2=0x7fc494071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:24.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.383+0000 7fc49b3ff700 1 -- 172.21.15.92:0/3865569990 shutdown_connections 2023-11-28T11:25:24.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.383+0000 7fc49b3ff700 1 -- 172.21.15.92:0/3865569990 wait complete. 2023-11-28T11:25:24.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.385+0000 7fc49b3ff700 1 Processor -- start 2023-11-28T11:25:24.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.385+0000 7fc49b3ff700 1 -- start start 2023-11-28T11:25:24.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.386+0000 7fc49b3ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4940b8510 0x7fc4940ccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:24.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.386+0000 7fc49b3ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4940cd460 0x7fc4941594b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:24.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.386+0000 7fc49b3ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4940d0d60 0x7fc4941e2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:24.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.386+0000 7fc49b3ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4941e1d90 con 0x7fc4940b8510 2023-11-28T11:25:24.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.386+0000 7fc49b3ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4941e2070 con 0x7fc4940cd460 2023-11-28T11:25:24.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.386+0000 7fc49b3ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4941e2350 con 0x7fc4940d0d60 2023-11-28T11:25:24.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.386+0000 7fc49919b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4940b8510 0x7fc4940ccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:24.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.386+0000 7fc49919b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4940b8510 0x7fc4940ccbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44670/0 (socket says 172.21.15.92:44670) 2023-11-28T11:25:24.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.386+0000 7fc49919b700 1 -- 172.21.15.92:0/3209597720 learned_addr learned my addr 172.21.15.92:0/3209597720 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:24.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.387+0000 7fc49899a700 1 --2- 172.21.15.92:0/3209597720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4940cd460 0x7fc4941594b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:24.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.387+0000 7fc49999c700 1 --2- 172.21.15.92:0/3209597720 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4940d0d60 0x7fc4941e2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:24.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.387+0000 7fc49919b700 1 -- 172.21.15.92:0/3209597720 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4940d0d60 msgr2=0x7fc4941e2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:24.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.387+0000 7fc49919b700 1 --2- 172.21.15.92:0/3209597720 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4940d0d60 0x7fc4941e2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:24.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.387+0000 7fc49919b700 1 -- 172.21.15.92:0/3209597720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4940cd460 msgr2=0x7fc4941594b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:24.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.387+0000 7fc49919b700 1 --2- 172.21.15.92:0/3209597720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4940cd460 0x7fc4941594b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:24.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.387+0000 7fc49919b700 1 -- 172.21.15.92:0/3209597720 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc48c009b30 con 0x7fc4940b8510 2023-11-28T11:25:24.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.387+0000 7fc49919b700 1 --2- 172.21.15.92:0/3209597720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4940b8510 0x7fc4940ccbb0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fc49000c520 tx=0x7fc49000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:24.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.387+0000 7fc489ffb700 1 -- 172.21.15.92:0/3209597720 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc490005cb0 con 0x7fc4940b8510 2023-11-28T11:25:24.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.387+0000 7fc489ffb700 1 -- 172.21.15.92:0/3209597720 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc490005e10 con 0x7fc4940b8510 2023-11-28T11:25:24.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.387+0000 7fc49b3ff700 1 -- 172.21.15.92:0/3209597720 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4940cfff0 con 0x7fc4940b8510 2023-11-28T11:25:24.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.387+0000 7fc489ffb700 1 -- 172.21.15.92:0/3209597720 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4900072d0 con 0x7fc4940b8510 2023-11-28T11:25:24.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.388+0000 7fc49b3ff700 1 -- 172.21.15.92:0/3209597720 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4940cfd10 con 0x7fc4940b8510 2023-11-28T11:25:24.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.389+0000 7fc49b3ff700 1 -- 172.21.15.92:0/3209597720 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4940ce6e0 con 0x7fc4940b8510 2023-11-28T11:25:24.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.390+0000 7fc489ffb700 1 -- 172.21.15.92:0/3209597720 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc490019030 con 0x7fc4940b8510 2023-11-28T11:25:24.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.390+0000 7fc489ffb700 1 --2- 172.21.15.92:0/3209597720 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc4780682f0 0x7fc47806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:24.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.391+0000 7fc489ffb700 1 -- 172.21.15.92:0/3209597720 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6556+0+0 (secure 0 0 0) 0x7fc490084fb0 con 0x7fc4940b8510 2023-11-28T11:25:24.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.391+0000 7fc49899a700 1 --2- 172.21.15.92:0/3209597720 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc4780682f0 0x7fc47806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:24.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.391+0000 7fc49899a700 1 --2- 172.21.15.92:0/3209597720 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc4780682f0 0x7fc47806a7a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc49408af10 tx=0x7fc484009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:24.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.394+0000 7fc489ffb700 1 -- 172.21.15.92:0/3209597720 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc49004b5e0 con 0x7fc4940b8510 2023-11-28T11:25:24.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.584+0000 7fc49b3ff700 1 -- 172.21.15.92:0/3209597720 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc494067880 con 0x7fc4940b8510 2023-11-28T11:25:24.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.585+0000 7fc489ffb700 1 -- 172.21.15.92:0/3209597720 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc490017020 con 0x7fc4940b8510 2023-11-28T11:25:24.587 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-11-28T11:25:24.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.587+0000 7fc49b3ff700 1 -- 172.21.15.92:0/3209597720 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc4780682f0 msgr2=0x7fc47806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:24.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.587+0000 7fc49b3ff700 1 --2- 172.21.15.92:0/3209597720 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc4780682f0 0x7fc47806a7a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc49408af10 tx=0x7fc484009210 comp rx=0 tx=0).stop 2023-11-28T11:25:24.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.587+0000 7fc49b3ff700 1 -- 172.21.15.92:0/3209597720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4940b8510 msgr2=0x7fc4940ccbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:24.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.587+0000 7fc49b3ff700 1 --2- 172.21.15.92:0/3209597720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4940b8510 0x7fc4940ccbb0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fc49000c520 tx=0x7fc49000c830 comp rx=0 tx=0).stop 2023-11-28T11:25:24.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.587+0000 7fc49b3ff700 1 -- 172.21.15.92:0/3209597720 shutdown_connections 2023-11-28T11:25:24.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.587+0000 7fc49b3ff700 1 --2- 172.21.15.92:0/3209597720 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc4940d0d60 0x7fc4941e2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:24.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.587+0000 7fc49b3ff700 1 --2- 172.21.15.92:0/3209597720 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc4780682f0 0x7fc47806a7a0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:24.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.587+0000 7fc49b3ff700 1 --2- 172.21.15.92:0/3209597720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4940b8510 0x7fc4940ccbb0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:24.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.587+0000 7fc49b3ff700 1 --2- 172.21.15.92:0/3209597720 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc4940cd460 0x7fc4941594b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:24.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.587+0000 7fc49b3ff700 1 -- 172.21.15.92:0/3209597720 >> 172.21.15.92:0/3209597720 conn(0x7fc4940719a0 msgr2=0x7fc4940b7260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:24.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.587+0000 7fc49b3ff700 1 -- 172.21.15.92:0/3209597720 shutdown_connections 2023-11-28T11:25:24.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.587+0000 7fc49b3ff700 1 -- 172.21.15.92:0/3209597720 wait complete. 2023-11-28T11:25:24.607 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:25:24.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.805+0000 7fe7443ff700 1 Processor -- start 2023-11-28T11:25:24.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.805+0000 7fe7443ff700 1 -- start start 2023-11-28T11:25:24.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.806+0000 7fe7443ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe73c0d2df0 0x7fe73c0cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:24.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.806+0000 7fe7443ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe73c0d1950 con 0x7fe73c0b4fc0 2023-11-28T11:25:24.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.806+0000 7fe7443ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe73c0b2500 con 0x7fe73c0b1c60 2023-11-28T11:25:24.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.806+0000 7fe7443ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe73c0d3d30 con 0x7fe73c0d2df0 2023-11-28T11:25:24.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.807+0000 7fe74199a700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe73c0b4fc0 0x7fe73c0d31c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59012/0 (socket says 172.21.15.92:59012) 2023-11-28T11:25:24.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.807+0000 7fe74199a700 1 -- 172.21.15.92:0/3110598059 learned_addr learned my addr 172.21.15.92:0/3110598059 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:25:24.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.807+0000 7fe74219b700 1 --2- 172.21.15.92:0/3110598059 >> v2:172.21.15.169:3300/0 conn(0x7fe73c0d2df0 0x7fe73c0cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:24.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.807+0000 7fe741199700 1 -- 172.21.15.92:0/3110598059 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1900231713 0 0) 0x7fe73c0d1950 con 0x7fe73c0b4fc0 2023-11-28T11:25:24.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.807+0000 7fe741199700 1 -- 172.21.15.92:0/3110598059 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7280035b0 con 0x7fe73c0b4fc0 2023-11-28T11:25:24.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.807+0000 7fe741199700 1 -- 172.21.15.92:0/3110598059 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2952726168 0 0) 0x7fe7280035b0 con 0x7fe73c0b4fc0 2023-11-28T11:25:24.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.807+0000 7fe741199700 1 -- 172.21.15.92:0/3110598059 >> v2:172.21.15.169:3300/0 conn(0x7fe73c0d2df0 msgr2=0x7fe73c0cd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:24.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.807+0000 7fe741199700 1 --2- 172.21.15.92:0/3110598059 >> v2:172.21.15.169:3300/0 conn(0x7fe73c0d2df0 0x7fe73c0cd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:24.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.807+0000 7fe741199700 1 -- 172.21.15.92:0/3110598059 >> v1:172.21.15.169:6789/0 conn(0x7fe73c0b1c60 legacy=0x7fe73c0cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:24.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.807+0000 7fe741199700 1 -- 172.21.15.92:0/3110598059 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe73c0d3a50 con 0x7fe73c0b4fc0 2023-11-28T11:25:24.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.807+0000 7fe741199700 1 -- 172.21.15.92:0/3110598059 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532161587 0 0) 0x7fe72c001670 con 0x7fe73c0b4fc0 2023-11-28T11:25:24.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.807+0000 7fe741199700 1 -- 172.21.15.92:0/3110598059 >> v1:172.21.15.92:6789/0 conn(0x7fe73c0b4fc0 legacy=0x7fe73c0d31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:24.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.808+0000 7fe741199700 1 --2- 172.21.15.92:0/3110598059 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe728003a00 0x7fe728003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:24.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.808+0000 7fe741199700 1 --2- 172.21.15.92:0/3110598059 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe728004850 0x7fe728004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:24.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.808+0000 7fe741199700 1 --2- 172.21.15.92:0/3110598059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe728005200 0x7fe72800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:24.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.808+0000 7fe741199700 1 -- 172.21.15.92:0/3110598059 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe73c0d3d30 con 0x7fe728005200 2023-11-28T11:25:24.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.808+0000 7fe741199700 1 -- 172.21.15.92:0/3110598059 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe72800c080 con 0x7fe728003a00 2023-11-28T11:25:24.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.808+0000 7fe741199700 1 -- 172.21.15.92:0/3110598059 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe72800c240 con 0x7fe728004850 2023-11-28T11:25:24.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.808+0000 7fe74219b700 1 --2- 172.21.15.92:0/3110598059 >> v2:172.21.15.169:3300/0 conn(0x7fe73c0d2df0 0x7fe73c0cd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:25:24.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.808+0000 7fe74299c700 1 --2- 172.21.15.92:0/3110598059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe728005200 0x7fe72800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:24.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.808+0000 7fe74219b700 1 --2- 172.21.15.92:0/3110598059 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe728003a00 0x7fe728003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:24.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.808+0000 7fe74199a700 1 --2- 172.21.15.92:0/3110598059 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe728004850 0x7fe728004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:24.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.808+0000 7fe74299c700 1 -- 172.21.15.92:0/3110598059 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe728004850 msgr2=0x7fe728004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:24.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.808+0000 7fe74299c700 1 --2- 172.21.15.92:0/3110598059 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe728004850 0x7fe728004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:24.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.808+0000 7fe74299c700 1 -- 172.21.15.92:0/3110598059 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe728003a00 msgr2=0x7fe728003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:24.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.808+0000 7fe74299c700 1 --2- 172.21.15.92:0/3110598059 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe728003a00 0x7fe728003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:24.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.808+0000 7fe74299c700 1 -- 172.21.15.92:0/3110598059 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7280035b0 con 0x7fe728005200 2023-11-28T11:25:24.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.808+0000 7fe74199a700 1 --2- 172.21.15.92:0/3110598059 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe728004850 0x7fe728004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:25:24.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.808+0000 7fe74299c700 1 -- 172.21.15.92:0/3110598059 reap_dead start 2023-11-28T11:25:24.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.808+0000 7fe74299c700 1 --2- 172.21.15.92:0/3110598059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe728005200 0x7fe72800ba10 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fe73400c240 tx=0x7fe728005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8afb762eda609f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:24.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.809+0000 7fe741199700 1 -- 172.21.15.92:0/3110598059 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe734001df0 con 0x7fe728005200 2023-11-28T11:25:24.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.809+0000 7fe741199700 1 -- 172.21.15.92:0/3110598059 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe73400cdc0 con 0x7fe728005200 2023-11-28T11:25:24.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.809+0000 7fe741199700 1 -- 172.21.15.92:0/3110598059 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe734003c40 con 0x7fe728005200 2023-11-28T11:25:24.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.809+0000 7fe7443ff700 1 -- 172.21.15.92:0/3110598059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe728005200 msgr2=0x7fe72800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:24.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.809+0000 7fe7443ff700 1 --2- 172.21.15.92:0/3110598059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe728005200 0x7fe72800ba10 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fe73400c240 tx=0x7fe728005880 comp rx=0 tx=0).stop 2023-11-28T11:25:24.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.809+0000 7fe7443ff700 1 -- 172.21.15.92:0/3110598059 shutdown_connections 2023-11-28T11:25:24.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.809+0000 7fe7443ff700 1 --2- 172.21.15.92:0/3110598059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe728005200 0x7fe72800ba10 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:24.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.809+0000 7fe7443ff700 1 -- 172.21.15.92:0/3110598059 >> 172.21.15.92:0/3110598059 conn(0x7fe73c0719a0 msgr2=0x7fe73c071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:24.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.809+0000 7fe7443ff700 1 -- 172.21.15.92:0/3110598059 shutdown_connections 2023-11-28T11:25:24.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.809+0000 7fe7443ff700 1 -- 172.21.15.92:0/3110598059 wait complete. 2023-11-28T11:25:24.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.812+0000 7fe7443ff700 1 Processor -- start 2023-11-28T11:25:24.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.812+0000 7fe7443ff700 1 -- start start 2023-11-28T11:25:24.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.813+0000 7fe7443ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe728005200 0x7fe73c1f52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:24.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.813+0000 7fe7443ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe73c1f2540 0x7fe73c1e70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:24.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.813+0000 7fe7443ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe73c1f29b0 0x7fe73c1e75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:24.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.813+0000 7fe7443ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe73c1f5810 con 0x7fe73c1f29b0 2023-11-28T11:25:24.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.813+0000 7fe7443ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe73c1e4d50 con 0x7fe73c1f2540 2023-11-28T11:25:24.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.813+0000 7fe7443ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe73c1e5030 con 0x7fe728005200 2023-11-28T11:25:24.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.813+0000 7fe74299c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe73c1f29b0 0x7fe73c1e75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:24.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.813+0000 7fe74299c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe73c1f29b0 0x7fe73c1e75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44688/0 (socket says 172.21.15.92:44688) 2023-11-28T11:25:24.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.813+0000 7fe74299c700 1 -- 172.21.15.92:0/2745088478 learned_addr learned my addr 172.21.15.92:0/2745088478 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:24.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.814+0000 7fe74199a700 1 --2- 172.21.15.92:0/2745088478 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe73c1f2540 0x7fe73c1e70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:24.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.814+0000 7fe74219b700 1 --2- 172.21.15.92:0/2745088478 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe728005200 0x7fe73c1f52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:24.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.814+0000 7fe74299c700 1 -- 172.21.15.92:0/2745088478 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe728005200 msgr2=0x7fe73c1f52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:24.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.814+0000 7fe74299c700 1 --2- 172.21.15.92:0/2745088478 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe728005200 0x7fe73c1f52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:24.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.814+0000 7fe74299c700 1 -- 172.21.15.92:0/2745088478 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe73c1f2540 msgr2=0x7fe73c1e70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:24.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.814+0000 7fe74299c700 1 --2- 172.21.15.92:0/2745088478 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe73c1f2540 0x7fe73c1e70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:24.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.814+0000 7fe74299c700 1 -- 172.21.15.92:0/2745088478 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe73c0b88f0 con 0x7fe73c1f29b0 2023-11-28T11:25:24.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.814+0000 7fe74199a700 1 --2- 172.21.15.92:0/2745088478 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe73c1f2540 0x7fe73c1e70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:25:24.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.814+0000 7fe74299c700 1 --2- 172.21.15.92:0/2745088478 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe73c1f29b0 0x7fe73c1e75f0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fe73c0ce4e0 tx=0x7fe7340014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:24.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.814+0000 7fe732ffd700 1 -- 172.21.15.92:0/2745088478 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe734003ea0 con 0x7fe73c1f29b0 2023-11-28T11:25:24.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.814+0000 7fe732ffd700 1 -- 172.21.15.92:0/2745088478 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7340052e0 con 0x7fe73c1f29b0 2023-11-28T11:25:24.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.814+0000 7fe732ffd700 1 -- 172.21.15.92:0/2745088478 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe734005670 con 0x7fe73c1f29b0 2023-11-28T11:25:24.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.814+0000 7fe7443ff700 1 -- 172.21.15.92:0/2745088478 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe73c0d15e0 con 0x7fe73c1f29b0 2023-11-28T11:25:24.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.815+0000 7fe7443ff700 1 -- 172.21.15.92:0/2745088478 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe73c0b21b0 con 0x7fe73c1f29b0 2023-11-28T11:25:24.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.816+0000 7fe7443ff700 1 -- 172.21.15.92:0/2745088478 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe73c067170 con 0x7fe73c1f29b0 2023-11-28T11:25:24.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.816+0000 7fe732ffd700 1 -- 172.21.15.92:0/2745088478 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe734010050 con 0x7fe73c1f29b0 2023-11-28T11:25:24.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.817+0000 7fe732ffd700 1 --2- 172.21.15.92:0/2745088478 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe7200682f0 0x7fe72006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:24.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.817+0000 7fe732ffd700 1 -- 172.21.15.92:0/2745088478 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6556+0+0 (secure 0 0 0) 0x7fe734086d40 con 0x7fe73c1f29b0 2023-11-28T11:25:24.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.818+0000 7fe74219b700 1 --2- 172.21.15.92:0/2745088478 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe7200682f0 0x7fe72006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:24.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.818+0000 7fe74219b700 1 --2- 172.21.15.92:0/2745088478 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe7200682f0 0x7fe72006a7a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe73c0b2c00 tx=0x7fe738009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:24.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:24.822+0000 7fe732ffd700 1 -- 172.21.15.92:0/2745088478 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe7340532c0 con 0x7fe73c1f29b0 2023-11-28T11:25:25.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.012+0000 7fe7443ff700 1 -- 172.21.15.92:0/2745088478 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe73c067170 con 0x7fe73c1f29b0 2023-11-28T11:25:25.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.012+0000 7fe732ffd700 1 -- 172.21.15.92:0/2745088478 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v70) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe734016020 con 0x7fe73c1f29b0 2023-11-28T11:25:25.015 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-11-28T11:25:25.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.015+0000 7fe7443ff700 1 -- 172.21.15.92:0/2745088478 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe7200682f0 msgr2=0x7fe72006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:25.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.015+0000 7fe7443ff700 1 --2- 172.21.15.92:0/2745088478 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe7200682f0 0x7fe72006a7a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe73c0b2c00 tx=0x7fe738009210 comp rx=0 tx=0).stop 2023-11-28T11:25:25.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.015+0000 7fe7443ff700 1 -- 172.21.15.92:0/2745088478 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe73c1f29b0 msgr2=0x7fe73c1e75f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:25.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.015+0000 7fe7443ff700 1 --2- 172.21.15.92:0/2745088478 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe73c1f29b0 0x7fe73c1e75f0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fe73c0ce4e0 tx=0x7fe7340014a0 comp rx=0 tx=0).stop 2023-11-28T11:25:25.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.015+0000 7fe7443ff700 1 -- 172.21.15.92:0/2745088478 shutdown_connections 2023-11-28T11:25:25.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.015+0000 7fe7443ff700 1 --2- 172.21.15.92:0/2745088478 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe7200682f0 0x7fe72006a7a0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:25.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.015+0000 7fe7443ff700 1 --2- 172.21.15.92:0/2745088478 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe73c1f29b0 0x7fe73c1e75f0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:25.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.015+0000 7fe7443ff700 1 --2- 172.21.15.92:0/2745088478 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe728005200 0x7fe73c1f52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:25.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.015+0000 7fe7443ff700 1 --2- 172.21.15.92:0/2745088478 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe73c1f2540 0x7fe73c1e70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:25.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.015+0000 7fe7443ff700 1 -- 172.21.15.92:0/2745088478 >> 172.21.15.92:0/2745088478 conn(0x7fe73c0719a0 msgr2=0x7fe73c0b4230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:25.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.015+0000 7fe7443ff700 1 -- 172.21.15.92:0/2745088478 shutdown_connections 2023-11-28T11:25:25.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.015+0000 7fe7443ff700 1 -- 172.21.15.92:0/2745088478 wait complete. 2023-11-28T11:25:25.035 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:25:25.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.231+0000 7f378e1ff700 1 Processor -- start 2023-11-28T11:25:25.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.231+0000 7f378e1ff700 1 -- start start 2023-11-28T11:25:25.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.232+0000 7f378e1ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f37880d1d70 0x7f3788167270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:25.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.232+0000 7f378e1ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f378806c790 con 0x7f37880b8510 2023-11-28T11:25:25.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.232+0000 7f378e1ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37880b88e0 con 0x7f37880cd460 2023-11-28T11:25:25.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.232+0000 7f378e1ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37880cdbd0 con 0x7f37880d1d70 2023-11-28T11:25:25.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.232+0000 7f3787fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f37880d1d70 0x7f3788167270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:25.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.232+0000 7f37877fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f37880b8510 0x7f378806b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59016/0 (socket says 172.21.15.92:59016) 2023-11-28T11:25:25.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.232+0000 7f37877fe700 1 -- 172.21.15.92:0/4014932281 learned_addr learned my addr 172.21.15.92:0/4014932281 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:25:25.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.233+0000 7f3787fff700 1 -- 172.21.15.92:0/4014932281 >> v1:172.21.15.169:6789/0 conn(0x7f37880cd460 legacy=0x7f378806c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:25.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.233+0000 7f3787fff700 1 -- 172.21.15.92:0/4014932281 >> v1:172.21.15.92:6789/0 conn(0x7f37880b8510 legacy=0x7f378806b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:25.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.233+0000 7f3787fff700 1 -- 172.21.15.92:0/4014932281 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37880cd8f0 con 0x7f37880d1d70 2023-11-28T11:25:25.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.233+0000 7f37867fc700 1 -- 172.21.15.92:0/4014932281 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4129750110 0 0) 0x7f378806c790 con 0x7f37880b8510 2023-11-28T11:25:25.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.233+0000 7f3787fff700 1 --2- 172.21.15.92:0/4014932281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37880d1d70 0x7f3788167270 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f3780009d90 tx=0x7f378000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc3c9a37ac72ae96 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:25.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.233+0000 7f37867fc700 1 -- 172.21.15.92:0/4014932281 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378000e070 con 0x7f37880d1d70 2023-11-28T11:25:25.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.234+0000 7f37867fc700 1 -- 172.21.15.92:0/4014932281 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f378000bbd0 con 0x7f37880d1d70 2023-11-28T11:25:25.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.234+0000 7f37867fc700 1 -- 172.21.15.92:0/4014932281 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3780012430 con 0x7f37880d1d70 2023-11-28T11:25:25.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.234+0000 7f378e1ff700 1 -- 172.21.15.92:0/4014932281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37880d1d70 msgr2=0x7f3788167270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:25.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.234+0000 7f378e1ff700 1 --2- 172.21.15.92:0/4014932281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37880d1d70 0x7f3788167270 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f3780009d90 tx=0x7f378000b330 comp rx=0 tx=0).stop 2023-11-28T11:25:25.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.234+0000 7f378e1ff700 1 -- 172.21.15.92:0/4014932281 shutdown_connections 2023-11-28T11:25:25.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.234+0000 7f378e1ff700 1 --2- 172.21.15.92:0/4014932281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37880d1d70 0x7f3788167270 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:25.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.234+0000 7f378e1ff700 1 -- 172.21.15.92:0/4014932281 >> 172.21.15.92:0/4014932281 conn(0x7f37880719a0 msgr2=0x7f3788071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:25.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.234+0000 7f378e1ff700 1 -- 172.21.15.92:0/4014932281 shutdown_connections 2023-11-28T11:25:25.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.234+0000 7f378e1ff700 1 -- 172.21.15.92:0/4014932281 wait complete. 2023-11-28T11:25:25.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.236+0000 7f378e1ff700 1 Processor -- start 2023-11-28T11:25:25.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.236+0000 7f378e1ff700 1 -- start start 2023-11-28T11:25:25.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.237+0000 7f378e1ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37880b8510 0x7f37881e7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:25.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.237+0000 7f378e1ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37880cd460 0x7f37881e4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:25.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.237+0000 7f378e1ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37880d1d70 0x7f378815a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:25.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.237+0000 7f378e1ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37881e6680 con 0x7f37880cd460 2023-11-28T11:25:25.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.237+0000 7f378e1ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37881e5a00 con 0x7f37880d1d70 2023-11-28T11:25:25.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.237+0000 7f378e1ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37881e5ce0 con 0x7f37880b8510 2023-11-28T11:25:25.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.237+0000 7f3786ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37880cd460 0x7f37881e4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:25.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.237+0000 7f37877fe700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37880b8510 0x7f37881e7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:25.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.237+0000 7f3786ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37880cd460 0x7f37881e4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44706/0 (socket says 172.21.15.92:44706) 2023-11-28T11:25:25.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.237+0000 7f3786ffd700 1 -- 172.21.15.92:0/3145174795 learned_addr learned my addr 172.21.15.92:0/3145174795 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:25.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.237+0000 7f3787fff700 1 --2- 172.21.15.92:0/3145174795 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37880d1d70 0x7f378815a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:25.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.238+0000 7f3786ffd700 1 -- 172.21.15.92:0/3145174795 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37880b8510 msgr2=0x7f37881e7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:25.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.238+0000 7f3786ffd700 1 --2- 172.21.15.92:0/3145174795 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37880b8510 0x7f37881e7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:25.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.238+0000 7f3786ffd700 1 -- 172.21.15.92:0/3145174795 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37880d1d70 msgr2=0x7f378815a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:25.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.238+0000 7f3786ffd700 1 --2- 172.21.15.92:0/3145174795 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37880d1d70 0x7f378815a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:25.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.238+0000 7f3786ffd700 1 -- 172.21.15.92:0/3145174795 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3780009a70 con 0x7f37880cd460 2023-11-28T11:25:25.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.238+0000 7f3786ffd700 1 --2- 172.21.15.92:0/3145174795 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37880cd460 0x7f37881e4fb0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f377c00c480 tx=0x7f377c00c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:25.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.238+0000 7f376ffff700 1 -- 172.21.15.92:0/3145174795 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f377c0052d0 con 0x7f37880cd460 2023-11-28T11:25:25.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.238+0000 7f376ffff700 1 -- 172.21.15.92:0/3145174795 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f377c00ce90 con 0x7f37880cd460 2023-11-28T11:25:25.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.238+0000 7f378e1ff700 1 -- 172.21.15.92:0/3145174795 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37880cfff0 con 0x7f37880cd460 2023-11-28T11:25:25.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.238+0000 7f376ffff700 1 -- 172.21.15.92:0/3145174795 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f377c003680 con 0x7f37880cd460 2023-11-28T11:25:25.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.238+0000 7f378e1ff700 1 -- 172.21.15.92:0/3145174795 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37880cfd10 con 0x7f37880cd460 2023-11-28T11:25:25.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.239+0000 7f378e1ff700 1 -- 172.21.15.92:0/3145174795 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37881e67c0 con 0x7f37880cd460 2023-11-28T11:25:25.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.240+0000 7f376ffff700 1 -- 172.21.15.92:0/3145174795 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f377c019030 con 0x7f37880cd460 2023-11-28T11:25:25.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.241+0000 7f376ffff700 1 --2- 172.21.15.92:0/3145174795 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f37740682f0 0x7f377406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:25.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.241+0000 7f376ffff700 1 -- 172.21.15.92:0/3145174795 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6556+0+0 (secure 0 0 0) 0x7f377c085190 con 0x7f37880cd460 2023-11-28T11:25:25.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.241+0000 7f37877fe700 1 --2- 172.21.15.92:0/3145174795 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f37740682f0 0x7f377406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:25.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.242+0000 7f37877fe700 1 --2- 172.21.15.92:0/3145174795 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f37740682f0 0x7f377406a7a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3778007b10 tx=0x7f3778007a60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:25.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.245+0000 7f376ffff700 1 -- 172.21.15.92:0/3145174795 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f37881e67c0 con 0x7f37880cd460 2023-11-28T11:25:25.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.441+0000 7f378e1ff700 1 -- 172.21.15.92:0/3145174795 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f37881e67c0 con 0x7f37880cd460 2023-11-28T11:25:25.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.441+0000 7f376ffff700 1 -- 172.21.15.92:0/3145174795 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v70) v1 ==== 104+0+11 (secure 0 0 0) 0x7f37881e67c0 con 0x7f37880cd460 2023-11-28T11:25:25.444 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-11-28T11:25:25.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.443+0000 7f378e1ff700 1 -- 172.21.15.92:0/3145174795 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f37740682f0 msgr2=0x7f377406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:25.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.443+0000 7f378e1ff700 1 --2- 172.21.15.92:0/3145174795 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f37740682f0 0x7f377406a7a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3778007b10 tx=0x7f3778007a60 comp rx=0 tx=0).stop 2023-11-28T11:25:25.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.443+0000 7f378e1ff700 1 -- 172.21.15.92:0/3145174795 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37880cd460 msgr2=0x7f37881e4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:25.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.443+0000 7f378e1ff700 1 --2- 172.21.15.92:0/3145174795 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37880cd460 0x7f37881e4fb0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f377c00c480 tx=0x7f377c00c790 comp rx=0 tx=0).stop 2023-11-28T11:25:25.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.444+0000 7f378e1ff700 1 -- 172.21.15.92:0/3145174795 shutdown_connections 2023-11-28T11:25:25.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.444+0000 7f378e1ff700 1 --2- 172.21.15.92:0/3145174795 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f37880d1d70 0x7f378815a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:25.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.444+0000 7f378e1ff700 1 --2- 172.21.15.92:0/3145174795 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f37880b8510 0x7f37881e7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:25.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.444+0000 7f378e1ff700 1 --2- 172.21.15.92:0/3145174795 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f37740682f0 0x7f377406a7a0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:25.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.444+0000 7f378e1ff700 1 --2- 172.21.15.92:0/3145174795 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37880cd460 0x7f37881e4fb0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:25.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.444+0000 7f378e1ff700 1 -- 172.21.15.92:0/3145174795 >> 172.21.15.92:0/3145174795 conn(0x7f37880719a0 msgr2=0x7f37880b6a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:25.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.444+0000 7f378e1ff700 1 -- 172.21.15.92:0/3145174795 shutdown_connections 2023-11-28T11:25:25.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.444+0000 7f378e1ff700 1 -- 172.21.15.92:0/3145174795 wait complete. 2023-11-28T11:25:25.465 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:25:25.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.664+0000 7f4e6b1ff700 1 Processor -- start 2023-11-28T11:25:25.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.664+0000 7f4e6b1ff700 1 -- start start 2023-11-28T11:25:25.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.665+0000 7f4e6b1ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4e640b8510 0x7f4e6406b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:25.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.665+0000 7f4e6b1ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f4e640cd460 0x7f4e6406beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:25.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.665+0000 7f4e6b1ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4e6406c3f0 0x7f4e6415cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:25.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.665+0000 7f4e6b1ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e640cdbd0 con 0x7f4e6406c3f0 2023-11-28T11:25:25.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.665+0000 7f4e6b1ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e6406c830 con 0x7f4e640cd460 2023-11-28T11:25:25.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.665+0000 7f4e6b1ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e640d02d0 con 0x7f4e640b8510 2023-11-28T11:25:25.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.666+0000 7f4e6979c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4e6406c3f0 0x7f4e6415cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:25.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.666+0000 7f4e6979c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4e6406c3f0 0x7f4e6415cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44710/0 (socket says 172.21.15.92:44710) 2023-11-28T11:25:25.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.666+0000 7f4e6979c700 1 -- 172.21.15.92:0/859069148 learned_addr learned my addr 172.21.15.92:0/859069148 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:25.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.666+0000 7f4e68f9b700 1 --2- 172.21.15.92:0/859069148 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4e640b8510 0x7f4e6406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:25.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.666+0000 7f4e63fff700 1 --2- 172.21.15.92:0/859069148 >> v2:172.21.15.169:3300/0 conn(0x7f4e640cd460 0x7f4e6406beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:25.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.666+0000 7f4e6979c700 1 -- 172.21.15.92:0/859069148 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4e640b8510 msgr2=0x7f4e6406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:25.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.666+0000 7f4e6979c700 1 --2- 172.21.15.92:0/859069148 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4e640b8510 0x7f4e6406b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:25.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.666+0000 7f4e6979c700 1 -- 172.21.15.92:0/859069148 >> v2:172.21.15.169:3300/0 conn(0x7f4e640cd460 msgr2=0x7f4e6406beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:25.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.666+0000 7f4e6979c700 1 --2- 172.21.15.92:0/859069148 >> v2:172.21.15.169:3300/0 conn(0x7f4e640cd460 0x7f4e6406beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:25.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.666+0000 7f4e6979c700 1 -- 172.21.15.92:0/859069148 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e640b88e0 con 0x7f4e6406c3f0 2023-11-28T11:25:25.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.666+0000 7f4e63fff700 1 --2- 172.21.15.92:0/859069148 >> v2:172.21.15.169:3300/0 conn(0x7f4e640cd460 0x7f4e6406beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:25:25.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.667+0000 7f4e6979c700 1 --2- 172.21.15.92:0/859069148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e6406c3f0 0x7f4e6415cc60 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f4e6408ad80 tx=0x7f4e5c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4be09636dbcae554 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:25.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.667+0000 7f4e637fe700 1 -- 172.21.15.92:0/859069148 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c00e070 con 0x7f4e6406c3f0 2023-11-28T11:25:25.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.667+0000 7f4e637fe700 1 -- 172.21.15.92:0/859069148 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e5c00b890 con 0x7f4e6406c3f0 2023-11-28T11:25:25.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.667+0000 7f4e637fe700 1 -- 172.21.15.92:0/859069148 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c00bc40 con 0x7f4e6406c3f0 2023-11-28T11:25:25.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.667+0000 7f4e6b1ff700 1 -- 172.21.15.92:0/859069148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e6406c3f0 msgr2=0x7f4e6415cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:25.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.667+0000 7f4e6b1ff700 1 --2- 172.21.15.92:0/859069148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e6406c3f0 0x7f4e6415cc60 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f4e6408ad80 tx=0x7f4e5c00b330 comp rx=0 tx=0).stop 2023-11-28T11:25:25.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.667+0000 7f4e6b1ff700 1 -- 172.21.15.92:0/859069148 shutdown_connections 2023-11-28T11:25:25.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.667+0000 7f4e6b1ff700 1 --2- 172.21.15.92:0/859069148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e6406c3f0 0x7f4e6415cc60 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:25.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.667+0000 7f4e6b1ff700 1 --2- 172.21.15.92:0/859069148 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4e640b8510 0x7f4e6406b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:25.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.667+0000 7f4e6b1ff700 1 --2- 172.21.15.92:0/859069148 >> v2:172.21.15.169:3300/0 conn(0x7f4e640cd460 0x7f4e6406beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:25.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.667+0000 7f4e6b1ff700 1 -- 172.21.15.92:0/859069148 >> 172.21.15.92:0/859069148 conn(0x7f4e640719a0 msgr2=0x7f4e64071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:25.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.668+0000 7f4e6b1ff700 1 -- 172.21.15.92:0/859069148 shutdown_connections 2023-11-28T11:25:25.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.668+0000 7f4e6b1ff700 1 -- 172.21.15.92:0/859069148 wait complete. 2023-11-28T11:25:25.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.669+0000 7f4e6b1ff700 1 Processor -- start 2023-11-28T11:25:25.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.670+0000 7f4e6b1ff700 1 -- start start 2023-11-28T11:25:25.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.670+0000 7f4e6b1ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4e6406c3f0 0x7f4e640c7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:25.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.670+0000 7f4e6b1ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e640b8510 0x7f4e641e3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:25.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.670+0000 7f4e6b1ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e640cd460 0x7f4e641e1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:25.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.670+0000 7f4e6b1ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e640c8ed0 con 0x7f4e640b8510 2023-11-28T11:25:25.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.670+0000 7f4e6b1ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e640c82c0 con 0x7f4e640cd460 2023-11-28T11:25:25.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.670+0000 7f4e6b1ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e640c85a0 con 0x7f4e6406c3f0 2023-11-28T11:25:25.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.671+0000 7f4e63fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e640b8510 0x7f4e641e3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:25.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.671+0000 7f4e63fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e640b8510 0x7f4e641e3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44716/0 (socket says 172.21.15.92:44716) 2023-11-28T11:25:25.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.671+0000 7f4e63fff700 1 -- 172.21.15.92:0/1158531986 learned_addr learned my addr 172.21.15.92:0/1158531986 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:25.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.671+0000 7f4e68f9b700 1 --2- 172.21.15.92:0/1158531986 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4e6406c3f0 0x7f4e640c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:25.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.671+0000 7f4e6979c700 1 --2- 172.21.15.92:0/1158531986 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e640cd460 0x7f4e641e1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:25.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.671+0000 7f4e63fff700 1 -- 172.21.15.92:0/1158531986 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4e6406c3f0 msgr2=0x7f4e640c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:25.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.671+0000 7f4e63fff700 1 --2- 172.21.15.92:0/1158531986 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4e6406c3f0 0x7f4e640c7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:25.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.671+0000 7f4e63fff700 1 -- 172.21.15.92:0/1158531986 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e640cd460 msgr2=0x7f4e641e1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:25.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.671+0000 7f4e63fff700 1 --2- 172.21.15.92:0/1158531986 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e640cd460 0x7f4e641e1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:25.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.671+0000 7f4e63fff700 1 -- 172.21.15.92:0/1158531986 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e5c009b30 con 0x7f4e640b8510 2023-11-28T11:25:25.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.671+0000 7f4e68f9b700 1 --2- 172.21.15.92:0/1158531986 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4e6406c3f0 0x7f4e640c7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:25:25.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.671+0000 7f4e63fff700 1 --2- 172.21.15.92:0/1158531986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e640b8510 0x7f4e641e3ec0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f4e5000e810 tx=0x7f4e5000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:25.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.672+0000 7f4e617fa700 1 -- 172.21.15.92:0/1158531986 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5000c020 con 0x7f4e640b8510 2023-11-28T11:25:25.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.672+0000 7f4e617fa700 1 -- 172.21.15.92:0/1158531986 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e50004250 con 0x7f4e640b8510 2023-11-28T11:25:25.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.672+0000 7f4e617fa700 1 -- 172.21.15.92:0/1158531986 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e500036a0 con 0x7f4e640b8510 2023-11-28T11:25:25.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.672+0000 7f4e6b1ff700 1 -- 172.21.15.92:0/1158531986 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e640cfd10 con 0x7f4e640b8510 2023-11-28T11:25:25.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.672+0000 7f4e6b1ff700 1 -- 172.21.15.92:0/1158531986 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e640cd8f0 con 0x7f4e640b8510 2023-11-28T11:25:25.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.673+0000 7f4e6b1ff700 1 -- 172.21.15.92:0/1158531986 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e640cfff0 con 0x7f4e640b8510 2023-11-28T11:25:25.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.674+0000 7f4e617fa700 1 -- 172.21.15.92:0/1158531986 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4e50003800 con 0x7f4e640b8510 2023-11-28T11:25:25.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.675+0000 7f4e617fa700 1 --2- 172.21.15.92:0/1158531986 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f4e480682f0 0x7f4e4806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:25.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.675+0000 7f4e617fa700 1 -- 172.21.15.92:0/1158531986 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6556+0+0 (secure 0 0 0) 0x7f4e50085b90 con 0x7f4e640b8510 2023-11-28T11:25:25.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.675+0000 7f4e68f9b700 1 --2- 172.21.15.92:0/1158531986 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f4e480682f0 0x7f4e4806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:25.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.676+0000 7f4e68f9b700 1 --2- 172.21.15.92:0/1158531986 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f4e480682f0 0x7f4e4806a7a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4e58009bb0 tx=0x7f4e58008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:25.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.679+0000 7f4e617fa700 1 -- 172.21.15.92:0/1158531986 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4e500521a0 con 0x7f4e640b8510 2023-11-28T11:25:25.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.869+0000 7f4e6b1ff700 1 -- 172.21.15.92:0/1158531986 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4e64067880 con 0x7f4e640b8510 2023-11-28T11:25:25.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.870+0000 7f4e617fa700 1 -- 172.21.15.92:0/1158531986 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v70) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4e5000f040 con 0x7f4e640b8510 2023-11-28T11:25:25.872 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-11-28T11:25:25.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.872+0000 7f4e6b1ff700 1 -- 172.21.15.92:0/1158531986 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f4e480682f0 msgr2=0x7f4e4806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:25.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.872+0000 7f4e6b1ff700 1 --2- 172.21.15.92:0/1158531986 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f4e480682f0 0x7f4e4806a7a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4e58009bb0 tx=0x7f4e58008040 comp rx=0 tx=0).stop 2023-11-28T11:25:25.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.872+0000 7f4e6b1ff700 1 -- 172.21.15.92:0/1158531986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e640b8510 msgr2=0x7f4e641e3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:25.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.872+0000 7f4e6b1ff700 1 --2- 172.21.15.92:0/1158531986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e640b8510 0x7f4e641e3ec0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f4e5000e810 tx=0x7f4e5000ebd0 comp rx=0 tx=0).stop 2023-11-28T11:25:25.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.872+0000 7f4e6b1ff700 1 -- 172.21.15.92:0/1158531986 shutdown_connections 2023-11-28T11:25:25.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.872+0000 7f4e6b1ff700 1 --2- 172.21.15.92:0/1158531986 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f4e640cd460 0x7f4e641e1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:25.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.872+0000 7f4e6b1ff700 1 --2- 172.21.15.92:0/1158531986 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f4e6406c3f0 0x7f4e640c7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:25.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.872+0000 7f4e6b1ff700 1 --2- 172.21.15.92:0/1158531986 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f4e480682f0 0x7f4e4806a7a0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:25.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.872+0000 7f4e6b1ff700 1 --2- 172.21.15.92:0/1158531986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e640b8510 0x7f4e641e3ec0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:25.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.872+0000 7f4e6b1ff700 1 -- 172.21.15.92:0/1158531986 >> 172.21.15.92:0/1158531986 conn(0x7f4e640719a0 msgr2=0x7f4e640d2130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:25.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.872+0000 7f4e6b1ff700 1 -- 172.21.15.92:0/1158531986 shutdown_connections 2023-11-28T11:25:25.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:25.872+0000 7f4e6b1ff700 1 -- 172.21.15.92:0/1158531986 wait complete. 2023-11-28T11:25:25.894 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:25:26.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.088+0000 7f0a0c1ff700 1 Processor -- start 2023-11-28T11:25:26.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.088+0000 7f0a0c1ff700 1 -- start start 2023-11-28T11:25:26.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.090+0000 7f0a0c1ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0a040cd460 0x7f0a0406c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:26.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.090+0000 7f0a0c1ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a040b88e0 con 0x7f0a040d0d60 2023-11-28T11:25:26.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.090+0000 7f0a0c1ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a040cd8f0 con 0x7f0a040b8510 2023-11-28T11:25:26.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.090+0000 7f0a0c1ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a040cdbd0 con 0x7f0a040cd460 2023-11-28T11:25:26.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.090+0000 7f0a0a79c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f0a040d0d60 0x7f0a040d1200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59020/0 (socket says 172.21.15.92:59020) 2023-11-28T11:25:26.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.090+0000 7f0a0a79c700 1 -- 172.21.15.92:0/1939660242 learned_addr learned my addr 172.21.15.92:0/1939660242 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:25:26.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.090+0000 7f0a0979a700 1 --2- 172.21.15.92:0/1939660242 >> v2:172.21.15.169:3300/0 conn(0x7f0a040cd460 0x7f0a0406c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:26.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.090+0000 7f0a08f99700 1 -- 172.21.15.92:0/1939660242 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1224184060 0 0) 0x7f0a040b88e0 con 0x7f0a040d0d60 2023-11-28T11:25:26.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.090+0000 7f0a08f99700 1 -- 172.21.15.92:0/1939660242 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09f00035b0 con 0x7f0a040d0d60 2023-11-28T11:25:26.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.091+0000 7f0a08f99700 1 -- 172.21.15.92:0/1939660242 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1719491631 0 0) 0x7f09f00035b0 con 0x7f0a040d0d60 2023-11-28T11:25:26.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.091+0000 7f0a08f99700 1 -- 172.21.15.92:0/1939660242 >> v2:172.21.15.169:3300/0 conn(0x7f0a040cd460 msgr2=0x7f0a0406c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:26.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.091+0000 7f0a08f99700 1 --2- 172.21.15.92:0/1939660242 >> v2:172.21.15.169:3300/0 conn(0x7f0a040cd460 0x7f0a0406c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:26.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.091+0000 7f0a08f99700 1 -- 172.21.15.92:0/1939660242 >> v1:172.21.15.169:6789/0 conn(0x7f0a040b8510 legacy=0x7f0a0406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:26.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.091+0000 7f0a08f99700 1 -- 172.21.15.92:0/1939660242 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a040cfd10 con 0x7f0a040d0d60 2023-11-28T11:25:26.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.091+0000 7f0a08f99700 1 -- 172.21.15.92:0/1939660242 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532161587 0 0) 0x7f09fc0019f0 con 0x7f0a040d0d60 2023-11-28T11:25:26.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.091+0000 7f0a08f99700 1 -- 172.21.15.92:0/1939660242 >> v1:172.21.15.92:6789/0 conn(0x7f0a040d0d60 legacy=0x7f0a040d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:26.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.091+0000 7f0a08f99700 1 --2- 172.21.15.92:0/1939660242 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f09f0003a00 0x7f09f0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:26.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.091+0000 7f0a08f99700 1 --2- 172.21.15.92:0/1939660242 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09f0004850 0x7f09f0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:26.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.091+0000 7f0a08f99700 1 --2- 172.21.15.92:0/1939660242 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09f0005200 0x7f09f000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:26.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.091+0000 7f0a08f99700 1 -- 172.21.15.92:0/1939660242 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a040cdbd0 con 0x7f09f0004850 2023-11-28T11:25:26.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.091+0000 7f0a08f99700 1 -- 172.21.15.92:0/1939660242 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f000c080 con 0x7f09f0005200 2023-11-28T11:25:26.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.091+0000 7f0a08f99700 1 -- 172.21.15.92:0/1939660242 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09f000c240 con 0x7f09f0003a00 2023-11-28T11:25:26.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.091+0000 7f0a0979a700 1 --2- 172.21.15.92:0/1939660242 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09f0004850 0x7f09f0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:26.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.091+0000 7f0a09f9b700 1 --2- 172.21.15.92:0/1939660242 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f09f0003a00 0x7f09f0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:26.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.091+0000 7f0a0a79c700 1 --2- 172.21.15.92:0/1939660242 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09f0005200 0x7f09f000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:26.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.091+0000 7f0a0979a700 1 -- 172.21.15.92:0/1939660242 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f09f0003a00 msgr2=0x7f09f0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:26.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.091+0000 7f0a0979a700 1 --2- 172.21.15.92:0/1939660242 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f09f0003a00 0x7f09f0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:26.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.092+0000 7f0a0979a700 1 -- 172.21.15.92:0/1939660242 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09f0005200 msgr2=0x7f09f000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:26.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.092+0000 7f0a0979a700 1 --2- 172.21.15.92:0/1939660242 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f09f0005200 0x7f09f000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:26.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.092+0000 7f0a0979a700 1 -- 172.21.15.92:0/1939660242 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09f00035b0 con 0x7f09f0004850 2023-11-28T11:25:26.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.092+0000 7f0a0a79c700 1 -- 172.21.15.92:0/1939660242 reap_dead start 2023-11-28T11:25:26.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.092+0000 7f0a0979a700 1 --2- 172.21.15.92:0/1939660242 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09f0004850 0x7f09f0004cc0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f0a040b1de0 tx=0x7f09f0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c93ea7313badc614 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:26.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.092+0000 7f0a08f99700 1 -- 172.21.15.92:0/1939660242 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f400e6c0 con 0x7f09f0004850 2023-11-28T11:25:26.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.092+0000 7f0a08f99700 1 -- 172.21.15.92:0/1939660242 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09f400e820 con 0x7f09f0004850 2023-11-28T11:25:26.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.092+0000 7f0a08f99700 1 -- 172.21.15.92:0/1939660242 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f400ebb0 con 0x7f09f0004850 2023-11-28T11:25:26.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.092+0000 7f0a0c1ff700 1 -- 172.21.15.92:0/1939660242 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09f0004850 msgr2=0x7f09f0004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:26.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.092+0000 7f0a0c1ff700 1 --2- 172.21.15.92:0/1939660242 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09f0004850 0x7f09f0004cc0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f0a040b1de0 tx=0x7f09f0005880 comp rx=0 tx=0).stop 2023-11-28T11:25:26.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.093+0000 7f0a0c1ff700 1 -- 172.21.15.92:0/1939660242 shutdown_connections 2023-11-28T11:25:26.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.093+0000 7f0a0c1ff700 1 --2- 172.21.15.92:0/1939660242 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09f0004850 0x7f09f0004cc0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:26.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.093+0000 7f0a0c1ff700 1 -- 172.21.15.92:0/1939660242 >> 172.21.15.92:0/1939660242 conn(0x7f0a040719a0 msgr2=0x7f0a04071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:26.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.093+0000 7f0a0c1ff700 1 -- 172.21.15.92:0/1939660242 shutdown_connections 2023-11-28T11:25:26.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.093+0000 7f0a0c1ff700 1 -- 172.21.15.92:0/1939660242 wait complete. 2023-11-28T11:25:26.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.094+0000 7f0a0c1ff700 1 Processor -- start 2023-11-28T11:25:26.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.095+0000 7f0a0c1ff700 1 -- start start 2023-11-28T11:25:26.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.095+0000 7f0a0c1ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09f0004850 0x7f0a041e5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:26.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.095+0000 7f0a0c1ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a041f24a0 0x7f0a040cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:26.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.095+0000 7f0a0c1ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0a0415ade0 0x7f0a041e29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:26.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.096+0000 7f0a0c1ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a041e3800 con 0x7f09f0004850 2023-11-28T11:25:26.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.096+0000 7f0a0c1ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a041e3ae0 con 0x7f0a041f24a0 2023-11-28T11:25:26.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.096+0000 7f0a0c1ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a040caa90 con 0x7f0a0415ade0 2023-11-28T11:25:26.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.096+0000 7f0a09f9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09f0004850 0x7f0a041e5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:26.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.096+0000 7f0a09f9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09f0004850 0x7f0a041e5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44736/0 (socket says 172.21.15.92:44736) 2023-11-28T11:25:26.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.096+0000 7f0a09f9b700 1 -- 172.21.15.92:0/1227660979 learned_addr learned my addr 172.21.15.92:0/1227660979 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:26.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.096+0000 7f0a0979a700 1 --2- 172.21.15.92:0/1227660979 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a041f24a0 0x7f0a040cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:26.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.096+0000 7f0a0a79c700 1 --2- 172.21.15.92:0/1227660979 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0a0415ade0 0x7f0a041e29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:26.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.096+0000 7f0a09f9b700 1 -- 172.21.15.92:0/1227660979 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0a0415ade0 msgr2=0x7f0a041e29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:26.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.096+0000 7f0a09f9b700 1 --2- 172.21.15.92:0/1227660979 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0a0415ade0 0x7f0a041e29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:26.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.096+0000 7f0a09f9b700 1 -- 172.21.15.92:0/1227660979 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a041f24a0 msgr2=0x7f0a040cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:26.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.096+0000 7f0a09f9b700 1 --2- 172.21.15.92:0/1227660979 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a041f24a0 0x7f0a040cc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:26.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.096+0000 7f0a09f9b700 1 -- 172.21.15.92:0/1227660979 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a040d29c0 con 0x7f09f0004850 2023-11-28T11:25:26.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.096+0000 7f0a09f9b700 1 --2- 172.21.15.92:0/1227660979 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09f0004850 0x7f0a041e5280 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f0a0000f770 tx=0x7f0a0000fa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:26.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.097+0000 7f09fa7fc700 1 -- 172.21.15.92:0/1227660979 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a000052e0 con 0x7f09f0004850 2023-11-28T11:25:26.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.097+0000 7f09fa7fc700 1 -- 172.21.15.92:0/1227660979 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a00005440 con 0x7f09f0004850 2023-11-28T11:25:26.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.097+0000 7f0a0c1ff700 1 -- 172.21.15.92:0/1227660979 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a040cfff0 con 0x7f09f0004850 2023-11-28T11:25:26.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.097+0000 7f09fa7fc700 1 -- 172.21.15.92:0/1227660979 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a000038c0 con 0x7f09f0004850 2023-11-28T11:25:26.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.097+0000 7f0a0c1ff700 1 -- 172.21.15.92:0/1227660979 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a040d2b50 con 0x7f09f0004850 2023-11-28T11:25:26.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.098+0000 7f0a0c1ff700 1 -- 172.21.15.92:0/1227660979 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a041e3940 con 0x7f09f0004850 2023-11-28T11:25:26.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.099+0000 7f09fa7fc700 1 -- 172.21.15.92:0/1227660979 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0a00003a20 con 0x7f09f0004850 2023-11-28T11:25:26.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.100+0000 7f09fa7fc700 1 --2- 172.21.15.92:0/1227660979 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f09e80682f0 0x7f09e806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:26.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.100+0000 7f09fa7fc700 1 -- 172.21.15.92:0/1227660979 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6556+0+0 (secure 0 0 0) 0x7f0a00085d50 con 0x7f09f0004850 2023-11-28T11:25:26.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.100+0000 7f0a0979a700 1 --2- 172.21.15.92:0/1227660979 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f09e80682f0 0x7f09e806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:26.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.101+0000 7f0a0979a700 1 --2- 172.21.15.92:0/1227660979 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f09e80682f0 0x7f09e806a7a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f09f40090b0 tx=0x7f09f401e040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:26.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.104+0000 7f09fa7fc700 1 -- 172.21.15.92:0/1227660979 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0a041e3940 con 0x7f09f0004850 2023-11-28T11:25:26.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.337+0000 7f0a0c1ff700 1 -- 172.21.15.92:0/1227660979 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0a040cac50 con 0x7f09f0004850 2023-11-28T11:25:26.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.338+0000 7f09fa7fc700 1 -- 172.21.15.92:0/1227660979 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4656 (secure 0 0 0) 0x7f0a00052380 con 0x7f09f0004850 2023-11-28T11:25:26.340 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:25:26.340 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5525,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2067956596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2067956596},{"type":"v1","addr":"172.21.15.169:6835","nonce":2067956596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5971,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2367049160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2367049160},{"type":"v1","addr":"172.21.15.169:6834","nonce":2367049160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5977,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1424858585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":1424858585},{"type":"v1","addr":"172.21.15.92:6837","nonce":1424858585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:24:16.257826+0000","modified":"2023-11-28T11:24:32.577560+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6213},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6213":{"gid":6213,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6836/3634521248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3634521248},{"type":"v1","addr":"172.21.15.92:6836","nonce":3634521248}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T11:25:26.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.340+0000 7f0a0c1ff700 1 -- 172.21.15.92:0/1227660979 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f09e80682f0 msgr2=0x7f09e806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:26.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.340+0000 7f0a0c1ff700 1 --2- 172.21.15.92:0/1227660979 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f09e80682f0 0x7f09e806a7a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f09f40090b0 tx=0x7f09f401e040 comp rx=0 tx=0).stop 2023-11-28T11:25:26.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.340+0000 7f0a0c1ff700 1 -- 172.21.15.92:0/1227660979 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09f0004850 msgr2=0x7f0a041e5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:26.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.340+0000 7f0a0c1ff700 1 --2- 172.21.15.92:0/1227660979 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09f0004850 0x7f0a041e5280 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f0a0000f770 tx=0x7f0a0000fa80 comp rx=0 tx=0).stop 2023-11-28T11:25:26.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.340+0000 7f0a0c1ff700 1 -- 172.21.15.92:0/1227660979 shutdown_connections 2023-11-28T11:25:26.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.340+0000 7f0a0c1ff700 1 --2- 172.21.15.92:0/1227660979 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0a0415ade0 0x7f0a041e29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:26.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.340+0000 7f0a0c1ff700 1 --2- 172.21.15.92:0/1227660979 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f09e80682f0 0x7f09e806a7a0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:26.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.340+0000 7f0a0c1ff700 1 --2- 172.21.15.92:0/1227660979 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09f0004850 0x7f0a041e5280 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:26.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.340+0000 7f0a0c1ff700 1 --2- 172.21.15.92:0/1227660979 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0a041f24a0 0x7f0a040cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:26.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.340+0000 7f0a0c1ff700 1 -- 172.21.15.92:0/1227660979 >> 172.21.15.92:0/1227660979 conn(0x7f0a040719a0 msgr2=0x7f0a0406b970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:26.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.340+0000 7f0a0c1ff700 1 -- 172.21.15.92:0/1227660979 shutdown_connections 2023-11-28T11:25:26.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.340+0000 7f0a0c1ff700 1 -- 172.21.15.92:0/1227660979 wait complete. 2023-11-28T11:25:26.346 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 33 2023-11-28T11:25:26.360 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:25:26.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.555+0000 7f5a8a7ff700 1 Processor -- start 2023-11-28T11:25:26.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.556+0000 7f5a8a7ff700 1 -- start start 2023-11-28T11:25:26.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.557+0000 7f5a8a7ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5a840b8510 0x7f5a8406b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:26.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.557+0000 7f5a8a7ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5a840cd460 0x7f5a8406beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:26.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.557+0000 7f5a8a7ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a840b88e0 con 0x7f5a8406c3f0 2023-11-28T11:25:26.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.557+0000 7f5a8a7ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a840cdbd0 con 0x7f5a840b8510 2023-11-28T11:25:26.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.557+0000 7f5a8a7ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a840d02d0 con 0x7f5a840cd460 2023-11-28T11:25:26.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.557+0000 7f5a88d9c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f5a8406c3f0 0x7f5a8415cc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59022/0 (socket says 172.21.15.92:59022) 2023-11-28T11:25:26.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.557+0000 7f5a83fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5a840b8510 0x7f5a8406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:26.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.557+0000 7f5a88d9c700 1 -- 172.21.15.92:0/2875991820 learned_addr learned my addr 172.21.15.92:0/2875991820 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:25:26.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.557+0000 7f5a837fe700 1 --2- 172.21.15.92:0/2875991820 >> v2:172.21.15.169:3300/0 conn(0x7f5a840cd460 0x7f5a8406beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:26.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.557+0000 7f5a82ffd700 1 -- 172.21.15.92:0/2875991820 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 706813754 0 0) 0x7f5a840b88e0 con 0x7f5a8406c3f0 2023-11-28T11:25:26.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.557+0000 7f5a83fff700 1 -- 172.21.15.92:0/2875991820 >> v2:172.21.15.169:3300/0 conn(0x7f5a840cd460 msgr2=0x7f5a8406beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:26.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.558+0000 7f5a83fff700 1 --2- 172.21.15.92:0/2875991820 >> v2:172.21.15.169:3300/0 conn(0x7f5a840cd460 0x7f5a8406beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:26.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.558+0000 7f5a83fff700 1 -- 172.21.15.92:0/2875991820 >> v1:172.21.15.92:6789/0 conn(0x7f5a8406c3f0 legacy=0x7f5a8415cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:26.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.558+0000 7f5a83fff700 1 -- 172.21.15.92:0/2875991820 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a840cd8f0 con 0x7f5a840b8510 2023-11-28T11:25:26.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.558+0000 7f5a83fff700 1 --2- 172.21.15.92:0/2875991820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a840b8510 0x7f5a8406b970 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f5a74009870 tx=0x7f5a74009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d54f4613b9c9e913 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:26.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.558+0000 7f5a82ffd700 1 -- 172.21.15.92:0/2875991820 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a74011830 con 0x7f5a840b8510 2023-11-28T11:25:26.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.558+0000 7f5a82ffd700 1 -- 172.21.15.92:0/2875991820 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a74011990 con 0x7f5a840b8510 2023-11-28T11:25:26.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.558+0000 7f5a82ffd700 1 -- 172.21.15.92:0/2875991820 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a74011d20 con 0x7f5a840b8510 2023-11-28T11:25:26.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.559+0000 7f5a8a7ff700 1 -- 172.21.15.92:0/2875991820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a840b8510 msgr2=0x7f5a8406b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:26.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.559+0000 7f5a8a7ff700 1 --2- 172.21.15.92:0/2875991820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a840b8510 0x7f5a8406b970 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f5a74009870 tx=0x7f5a74009b80 comp rx=0 tx=0).stop 2023-11-28T11:25:26.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.559+0000 7f5a8a7ff700 1 -- 172.21.15.92:0/2875991820 shutdown_connections 2023-11-28T11:25:26.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.559+0000 7f5a8a7ff700 1 --2- 172.21.15.92:0/2875991820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a840b8510 0x7f5a8406b970 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:26.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.559+0000 7f5a8a7ff700 1 --2- 172.21.15.92:0/2875991820 >> v2:172.21.15.169:3300/0 conn(0x7f5a840cd460 0x7f5a8406beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:26.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.559+0000 7f5a8a7ff700 1 -- 172.21.15.92:0/2875991820 >> 172.21.15.92:0/2875991820 conn(0x7f5a840719a0 msgr2=0x7f5a84071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:26.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.559+0000 7f5a8a7ff700 1 -- 172.21.15.92:0/2875991820 shutdown_connections 2023-11-28T11:25:26.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.559+0000 7f5a8a7ff700 1 -- 172.21.15.92:0/2875991820 wait complete. 2023-11-28T11:25:26.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.560+0000 7f5a8a7ff700 1 Processor -- start 2023-11-28T11:25:26.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.561+0000 7f5a8a7ff700 1 -- start start 2023-11-28T11:25:26.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.562+0000 7f5a8a7ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a8406c3f0 0x7f5a841578e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:26.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.562+0000 7f5a8a7ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a840b8510 0x7f5a841f1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:26.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.562+0000 7f5a8a7ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a840cd460 0x7f5a840c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:26.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.562+0000 7f5a8a7ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a84158e70 con 0x7f5a840cd460 2023-11-28T11:25:26.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.562+0000 7f5a8a7ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a84158250 con 0x7f5a840b8510 2023-11-28T11:25:26.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.562+0000 7f5a8a7ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a84158530 con 0x7f5a8406c3f0 2023-11-28T11:25:26.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.562+0000 7f5a88d9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a840cd460 0x7f5a840c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:26.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.562+0000 7f5a88d9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a840cd460 0x7f5a840c9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44756/0 (socket says 172.21.15.92:44756) 2023-11-28T11:25:26.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.562+0000 7f5a88d9c700 1 -- 172.21.15.92:0/2631369237 learned_addr learned my addr 172.21.15.92:0/2631369237 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:26.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.562+0000 7f5a83fff700 1 --2- 172.21.15.92:0/2631369237 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a8406c3f0 0x7f5a841578e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:26.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.562+0000 7f5a837fe700 1 --2- 172.21.15.92:0/2631369237 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a840b8510 0x7f5a841f1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:26.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.562+0000 7f5a88d9c700 1 -- 172.21.15.92:0/2631369237 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a8406c3f0 msgr2=0x7f5a841578e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:26.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.562+0000 7f5a88d9c700 1 --2- 172.21.15.92:0/2631369237 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a8406c3f0 0x7f5a841578e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:26.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.562+0000 7f5a88d9c700 1 -- 172.21.15.92:0/2631369237 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a840b8510 msgr2=0x7f5a841f1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:26.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.562+0000 7f5a88d9c700 1 --2- 172.21.15.92:0/2631369237 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a840b8510 0x7f5a841f1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:26.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.562+0000 7f5a88d9c700 1 -- 172.21.15.92:0/2631369237 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a740094a0 con 0x7f5a840cd460 2023-11-28T11:25:26.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.563+0000 7f5a88d9c700 1 --2- 172.21.15.92:0/2631369237 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a840cd460 0x7f5a840c9ee0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c002230 tx=0x7f5a7c00cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:26.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.563+0000 7f5a80ff9700 1 -- 172.21.15.92:0/2631369237 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7c00aac0 con 0x7f5a840cd460 2023-11-28T11:25:26.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.563+0000 7f5a80ff9700 1 -- 172.21.15.92:0/2631369237 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a7c00ac40 con 0x7f5a840cd460 2023-11-28T11:25:26.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.563+0000 7f5a8a7ff700 1 -- 172.21.15.92:0/2631369237 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a840cfff0 con 0x7f5a840cd460 2023-11-28T11:25:26.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.563+0000 7f5a80ff9700 1 -- 172.21.15.92:0/2631369237 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7c0070d0 con 0x7f5a840cd460 2023-11-28T11:25:26.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.563+0000 7f5a8a7ff700 1 -- 172.21.15.92:0/2631369237 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a840cfd10 con 0x7f5a840cd460 2023-11-28T11:25:26.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.564+0000 7f5a8a7ff700 1 -- 172.21.15.92:0/2631369237 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a84158a50 con 0x7f5a840cd460 2023-11-28T11:25:26.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.565+0000 7f5a80ff9700 1 -- 172.21.15.92:0/2631369237 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5a7c01a030 con 0x7f5a840cd460 2023-11-28T11:25:26.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.565+0000 7f5a80ff9700 1 --2- 172.21.15.92:0/2631369237 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5a70068320 0x7f5a7006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:26.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.566+0000 7f5a80ff9700 1 -- 172.21.15.92:0/2631369237 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6556+0+0 (secure 0 0 0) 0x7f5a7c085a90 con 0x7f5a840cd460 2023-11-28T11:25:26.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.566+0000 7f5a83fff700 1 --2- 172.21.15.92:0/2631369237 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5a70068320 0x7f5a7006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:26.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.566+0000 7f5a83fff700 1 --2- 172.21.15.92:0/2631369237 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5a70068320 0x7f5a7006a7d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5a740097c0 tx=0x7f5a74008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:26.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.569+0000 7f5a80ff9700 1 -- 172.21.15.92:0/2631369237 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5a84158a50 con 0x7f5a840cd460 2023-11-28T11:25:26.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.763+0000 7f5a8a7ff700 1 -- 172.21.15.92:0/2631369237 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5a84158c40 con 0x7f5a840cd460 2023-11-28T11:25:26.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.764+0000 7f5a80ff9700 1 -- 172.21.15.92:0/2631369237 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+18417 (secure 0 0 0) 0x7f5a84158c40 con 0x7f5a840cd460 2023-11-28T11:25:26.766 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:25:26.767 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":70,"fsid":"68956e0c-c16f-47cd-a220-f98364b0831d","created":"2023-11-28T11:20:28.697573+0000","modified":"2023-11-28T11:24:38.644982+0000","last_up_change":"2023-11-28T11:20:41.257011+0000","last_in_change":"2023-11-28T11:20:32.249767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:20:41.925237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:20:48.682497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:24:13.972356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-28T11:24:15.237693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6abe483f-0e03-4a26-b9cf-62041291747e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":115823},{"type":"v1","addr":"172.21.15.92:6809","nonce":115823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":115823},{"type":"v1","addr":"172.21.15.92:6813","nonce":115823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":115823},{"type":"v1","addr":"172.21.15.92:6821","nonce":115823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":115823},{"type":"v1","addr":"172.21.15.92:6817","nonce":115823}]},"public_addr":"172.21.15.92:6809/115823","cluster_addr":"172.21.15.92:6813/115823","heartbeat_back_addr":"172.21.15.92:6821/115823","heartbeat_front_addr":"172.21.15.92:6817/115823","state":["exists","up"]},{"osd":1,"uuid":"13c1de73-1a8c-4d7e-8a28-9dae52dfe089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":115824},{"type":"v1","addr":"172.21.15.92:6812","nonce":115824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":115824},{"type":"v1","addr":"172.21.15.92:6816","nonce":115824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":115824},{"type":"v1","addr":"172.21.15.92:6823","nonce":115824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":115824},{"type":"v1","addr":"172.21.15.92:6820","nonce":115824}]},"public_addr":"172.21.15.92:6812/115824","cluster_addr":"172.21.15.92:6816/115824","heartbeat_back_addr":"172.21.15.92:6823/115824","heartbeat_front_addr":"172.21.15.92:6820/115824","state":["exists","up"]},{"osd":2,"uuid":"25788a55-112f-4f00-810d-2bc37cf76778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":115821},{"type":"v1","addr":"172.21.15.92:6801","nonce":115821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":115821},{"type":"v1","addr":"172.21.15.92:6803","nonce":115821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":115821},{"type":"v1","addr":"172.21.15.92:6807","nonce":115821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":115821},{"type":"v1","addr":"172.21.15.92:6805","nonce":115821}]},"public_addr":"172.21.15.92:6801/115821","cluster_addr":"172.21.15.92:6803/115821","heartbeat_back_addr":"172.21.15.92:6807/115821","heartbeat_front_addr":"172.21.15.92:6805/115821","state":["exists","up"]},{"osd":3,"uuid":"adf54e24-33a9-41fe-9ce3-98a5ab6c158e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":115825},{"type":"v1","addr":"172.21.15.92:6825","nonce":115825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":115825},{"type":"v1","addr":"172.21.15.92:6827","nonce":115825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":115825},{"type":"v1","addr":"172.21.15.92:6831","nonce":115825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":115825},{"type":"v1","addr":"172.21.15.92:6829","nonce":115825}]},"public_addr":"172.21.15.92:6825/115825","cluster_addr":"172.21.15.92:6827/115825","heartbeat_back_addr":"172.21.15.92:6831/115825","heartbeat_front_addr":"172.21.15.92:6829/115825","state":["exists","up"]},{"osd":4,"uuid":"ccb9b747-f399-44ea-9a18-9db3152227e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":116035},{"type":"v1","addr":"172.21.15.169:6817","nonce":116035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":116035},{"type":"v1","addr":"172.21.15.169:6819","nonce":116035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":116035},{"type":"v1","addr":"172.21.15.169:6826","nonce":116035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":116035},{"type":"v1","addr":"172.21.15.169:6822","nonce":116035}]},"public_addr":"172.21.15.169:6817/116035","cluster_addr":"172.21.15.169:6819/116035","heartbeat_back_addr":"172.21.15.169:6826/116035","heartbeat_front_addr":"172.21.15.169:6822/116035","state":["exists","up"]},{"osd":5,"uuid":"308a1e63-75aa-4d7d-8a7c-cecc24714fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":116033},{"type":"v1","addr":"172.21.15.169:6801","nonce":116033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":116033},{"type":"v1","addr":"172.21.15.169:6803","nonce":116033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":116033},{"type":"v1","addr":"172.21.15.169:6807","nonce":116033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":116033},{"type":"v1","addr":"172.21.15.169:6805","nonce":116033}]},"public_addr":"172.21.15.169:6801/116033","cluster_addr":"172.21.15.169:6803/116033","heartbeat_back_addr":"172.21.15.169:6807/116033","heartbeat_front_addr":"172.21.15.169:6805/116033","state":["exists","up"]},{"osd":6,"uuid":"9c459b7b-b821-4b4b-b91e-6aaae271457c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":116034},{"type":"v1","addr":"172.21.15.169:6809","nonce":116034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":116034},{"type":"v1","addr":"172.21.15.169:6811","nonce":116034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":116034},{"type":"v1","addr":"172.21.15.169:6815","nonce":116034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":116034},{"type":"v1","addr":"172.21.15.169:6813","nonce":116034}]},"public_addr":"172.21.15.169:6809/116034","cluster_addr":"172.21.15.169:6811/116034","heartbeat_back_addr":"172.21.15.169:6815/116034","heartbeat_front_addr":"172.21.15.169:6813/116034","state":["exists","up"]},{"osd":7,"uuid":"1ca9aa4a-755d-4e28-a937-4d8dccab4df3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":116044},{"type":"v1","addr":"172.21.15.169:6823","nonce":116044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6825","nonce":116044},{"type":"v1","addr":"172.21.15.169:6827","nonce":116044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":116044},{"type":"v1","addr":"172.21.15.169:6831","nonce":116044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":116044},{"type":"v1","addr":"172.21.15.169:6829","nonce":116044}]},"public_addr":"172.21.15.169:6823/116044","cluster_addr":"172.21.15.169:6827/116044","heartbeat_back_addr":"172.21.15.169:6831/116044","heartbeat_front_addr":"172.21.15.169:6829/116044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.586781+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.869778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.508594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.860493+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.872267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.746933+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.611520+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.871446+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.10","mappings":[{"from":6,"to":5}]},{"pgid":"11.17","mappings":[{"from":6,"to":1}]},{"pgid":"11.29","mappings":[{"from":7,"to":0}]},{"pgid":"11.31","mappings":[{"from":2,"to":4}]},{"pgid":"11.33","mappings":[{"from":6,"to":1}]},{"pgid":"11.3e","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:25:26.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.766+0000 7f5a8a7ff700 1 -- 172.21.15.92:0/2631369237 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5a70068320 msgr2=0x7f5a7006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:26.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.766+0000 7f5a8a7ff700 1 --2- 172.21.15.92:0/2631369237 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5a70068320 0x7f5a7006a7d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5a740097c0 tx=0x7f5a74008030 comp rx=0 tx=0).stop 2023-11-28T11:25:26.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.766+0000 7f5a8a7ff700 1 -- 172.21.15.92:0/2631369237 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a840cd460 msgr2=0x7f5a840c9ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:26.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.766+0000 7f5a8a7ff700 1 --2- 172.21.15.92:0/2631369237 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a840cd460 0x7f5a840c9ee0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c002230 tx=0x7f5a7c00cf50 comp rx=0 tx=0).stop 2023-11-28T11:25:26.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.766+0000 7f5a8a7ff700 1 -- 172.21.15.92:0/2631369237 shutdown_connections 2023-11-28T11:25:26.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.766+0000 7f5a8a7ff700 1 --2- 172.21.15.92:0/2631369237 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5a70068320 0x7f5a7006a7d0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:26.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.766+0000 7f5a8a7ff700 1 --2- 172.21.15.92:0/2631369237 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a840cd460 0x7f5a840c9ee0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:26.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.767+0000 7f5a8a7ff700 1 --2- 172.21.15.92:0/2631369237 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5a8406c3f0 0x7f5a841578e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:26.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.767+0000 7f5a8a7ff700 1 --2- 172.21.15.92:0/2631369237 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5a840b8510 0x7f5a841f1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:26.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.767+0000 7f5a8a7ff700 1 -- 172.21.15.92:0/2631369237 >> 172.21.15.92:0/2631369237 conn(0x7f5a840719a0 msgr2=0x7f5a840b41b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:26.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.767+0000 7f5a8a7ff700 1 -- 172.21.15.92:0/2631369237 shutdown_connections 2023-11-28T11:25:26.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.767+0000 7f5a8a7ff700 1 -- 172.21.15.92:0/2631369237 wait complete. 2023-11-28T11:25:26.787 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T11:25:26.787 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:25:26.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.989+0000 7fa88b1ff700 1 Processor -- start 2023-11-28T11:25:26.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.990+0000 7fa88b1ff700 1 -- start start 2023-11-28T11:25:26.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.991+0000 7fa88b1ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa8840cb890 0x7fa88406ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:26.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.991+0000 7fa88b1ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa88415a810 0x7fa88415afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:26.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.991+0000 7fa88b1ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa884072190 con 0x7fa88406e970 2023-11-28T11:25:26.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.991+0000 7fa88b1ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa88406f280 con 0x7fa8840cb890 2023-11-28T11:25:26.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.991+0000 7fa88b1ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa884071610 con 0x7fa88415a810 2023-11-28T11:25:26.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.991+0000 7fa88979c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa88415a810 0x7fa88415afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:26.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.991+0000 7fa883fff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa8840cb890 0x7fa88406ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:26.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.991+0000 7fa88979c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa88415a810 0x7fa88415afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.92:39292/0 (socket says 172.21.15.92:39292) 2023-11-28T11:25:26.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.991+0000 7fa88979c700 1 -- 172.21.15.92:0/2050908672 learned_addr learned my addr 172.21.15.92:0/2050908672 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:26.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.992+0000 7fa883fff700 1 -- 172.21.15.92:0/2050908672 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa88415a810 msgr2=0x7fa88415afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:26.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.992+0000 7fa883fff700 1 --2- 172.21.15.92:0/2050908672 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa88415a810 0x7fa88415afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:26.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.992+0000 7fa883fff700 1 -- 172.21.15.92:0/2050908672 >> v1:172.21.15.169:6789/0 conn(0x7fa88406e970 legacy=0x7fa88415e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:26.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.992+0000 7fa883fff700 1 -- 172.21.15.92:0/2050908672 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8840718f0 con 0x7fa8840cb890 2023-11-28T11:25:26.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.992+0000 7fa88979c700 1 --2- 172.21.15.92:0/2050908672 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa88415a810 0x7fa88415afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:25:26.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.992+0000 7fa883fff700 1 --2- 172.21.15.92:0/2050908672 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8840cb890 0x7fa88406ed40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa870009840 tx=0x7fa870009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a81ec69a5888f027 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:26.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.992+0000 7fa8837fe700 1 -- 172.21.15.92:0/2050908672 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa870011830 con 0x7fa8840cb890 2023-11-28T11:25:26.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.992+0000 7fa8837fe700 1 -- 172.21.15.92:0/2050908672 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa870011990 con 0x7fa8840cb890 2023-11-28T11:25:26.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.992+0000 7fa8837fe700 1 -- 172.21.15.92:0/2050908672 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa870011d20 con 0x7fa8840cb890 2023-11-28T11:25:26.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.993+0000 7fa88b1ff700 1 -- 172.21.15.92:0/2050908672 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8840cb890 msgr2=0x7fa88406ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:27.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.993+0000 7fa88b1ff700 1 --2- 172.21.15.92:0/2050908672 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8840cb890 0x7fa88406ed40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa870009840 tx=0x7fa870009b50 comp rx=0 tx=0).stop 2023-11-28T11:25:27.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.993+0000 7fa88b1ff700 1 -- 172.21.15.92:0/2050908672 shutdown_connections 2023-11-28T11:25:27.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.993+0000 7fa88b1ff700 1 --2- 172.21.15.92:0/2050908672 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa88415a810 0x7fa88415afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:27.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.993+0000 7fa88b1ff700 1 --2- 172.21.15.92:0/2050908672 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8840cb890 0x7fa88406ed40 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:27.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.993+0000 7fa88b1ff700 1 -- 172.21.15.92:0/2050908672 >> 172.21.15.92:0/2050908672 conn(0x7fa8840b0db0 msgr2=0x7fa8840b1180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:27.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.993+0000 7fa88b1ff700 1 -- 172.21.15.92:0/2050908672 shutdown_connections 2023-11-28T11:25:27.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.993+0000 7fa88b1ff700 1 -- 172.21.15.92:0/2050908672 wait complete. 2023-11-28T11:25:27.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.995+0000 7fa88b1ff700 1 Processor -- start 2023-11-28T11:25:27.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.995+0000 7fa88b1ff700 1 -- start start 2023-11-28T11:25:27.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.996+0000 7fa88b1ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa88406e970 0x7fa884158b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:27.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.996+0000 7fa88b1ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa88415a810 0x7fa884159310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:27.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.996+0000 7fa88b1ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8840acc50 0x7fa884159850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:27.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.996+0000 7fa88b1ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8840ae5a0 con 0x7fa8840acc50 2023-11-28T11:25:27.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.996+0000 7fa88b1ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8840ac830 con 0x7fa88406e970 2023-11-28T11:25:27.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.996+0000 7fa88b1ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8840ab8d0 con 0x7fa88415a810 2023-11-28T11:25:27.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.996+0000 7fa88979c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8840acc50 0x7fa884159850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:27.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.996+0000 7fa88979c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8840acc50 0x7fa884159850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44762/0 (socket says 172.21.15.92:44762) 2023-11-28T11:25:27.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.996+0000 7fa88979c700 1 -- 172.21.15.92:0/1309772766 learned_addr learned my addr 172.21.15.92:0/1309772766 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:27.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.996+0000 7fa883fff700 1 --2- 172.21.15.92:0/1309772766 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa88415a810 0x7fa884159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:27.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.996+0000 7fa888f9b700 1 --2- 172.21.15.92:0/1309772766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa88406e970 0x7fa884158b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:27.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.996+0000 7fa88979c700 1 -- 172.21.15.92:0/1309772766 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa88415a810 msgr2=0x7fa884159310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:27.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.996+0000 7fa88979c700 1 --2- 172.21.15.92:0/1309772766 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa88415a810 0x7fa884159310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:27.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.996+0000 7fa88979c700 1 -- 172.21.15.92:0/1309772766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa88406e970 msgr2=0x7fa884158b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:27.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.996+0000 7fa88979c700 1 --2- 172.21.15.92:0/1309772766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa88406e970 0x7fa884158b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:27.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.996+0000 7fa88979c700 1 -- 172.21.15.92:0/1309772766 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa87c009b30 con 0x7fa8840acc50 2023-11-28T11:25:27.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.996+0000 7fa888f9b700 1 --2- 172.21.15.92:0/1309772766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa88406e970 0x7fa884158b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:25:27.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.997+0000 7fa88979c700 1 --2- 172.21.15.92:0/1309772766 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8840acc50 0x7fa884159850 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fa87c00efc0 tx=0x7fa87c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:27.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.997+0000 7fa8817fa700 1 -- 172.21.15.92:0/1309772766 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87c018070 con 0x7fa8840acc50 2023-11-28T11:25:27.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.997+0000 7fa8817fa700 1 -- 172.21.15.92:0/1309772766 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa87c00f040 con 0x7fa8840acc50 2023-11-28T11:25:27.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.997+0000 7fa88b1ff700 1 -- 172.21.15.92:0/1309772766 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8700094a0 con 0x7fa8840acc50 2023-11-28T11:25:27.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.997+0000 7fa8817fa700 1 -- 172.21.15.92:0/1309772766 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87c003c40 con 0x7fa8840acc50 2023-11-28T11:25:27.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.997+0000 7fa88b1ff700 1 -- 172.21.15.92:0/1309772766 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa884071eb0 con 0x7fa8840acc50 2023-11-28T11:25:27.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.998+0000 7fa88b1ff700 1 -- 172.21.15.92:0/1309772766 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa884071bd0 con 0x7fa8840acc50 2023-11-28T11:25:27.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:26.999+0000 7fa8817fa700 1 -- 172.21.15.92:0/1309772766 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa87c00c930 con 0x7fa8840acc50 2023-11-28T11:25:27.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.000+0000 7fa8817fa700 1 --2- 172.21.15.92:0/1309772766 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa8680682f0 0x7fa86806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:27.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.000+0000 7fa8817fa700 1 -- 172.21.15.92:0/1309772766 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6556+0+0 (secure 0 0 0) 0x7fa87c086290 con 0x7fa8840acc50 2023-11-28T11:25:27.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.000+0000 7fa888f9b700 1 --2- 172.21.15.92:0/1309772766 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa8680682f0 0x7fa86806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:27.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.001+0000 7fa888f9b700 1 --2- 172.21.15.92:0/1309772766 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa8680682f0 0x7fa86806a7a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa878000f60 tx=0x7fa878009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:27.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.004+0000 7fa8817fa700 1 -- 172.21.15.92:0/1309772766 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa87c052790 con 0x7fa8840acc50 2023-11-28T11:25:27.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.235+0000 7fa88b1ff700 1 -- 172.21.15.92:0/1309772766 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa884071270 con 0x7fa8840acc50 2023-11-28T11:25:27.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.235+0000 7fa8817fa700 1 -- 172.21.15.92:0/1309772766 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4656 (secure 0 0 0) 0x7fa87c009ce0 con 0x7fa8840acc50 2023-11-28T11:25:27.238 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:25:27.238 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5525,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2067956596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2067956596},{"type":"v1","addr":"172.21.15.169:6835","nonce":2067956596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5971,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2367049160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2367049160},{"type":"v1","addr":"172.21.15.169:6834","nonce":2367049160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5977,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1424858585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":1424858585},{"type":"v1","addr":"172.21.15.92:6837","nonce":1424858585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:24:16.257826+0000","modified":"2023-11-28T11:24:32.577560+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6213},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6213":{"gid":6213,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6836/3634521248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3634521248},{"type":"v1","addr":"172.21.15.92:6836","nonce":3634521248}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T11:25:27.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.238+0000 7fa88b1ff700 1 -- 172.21.15.92:0/1309772766 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa8680682f0 msgr2=0x7fa86806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:27.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.238+0000 7fa88b1ff700 1 --2- 172.21.15.92:0/1309772766 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa8680682f0 0x7fa86806a7a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa878000f60 tx=0x7fa878009040 comp rx=0 tx=0).stop 2023-11-28T11:25:27.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.238+0000 7fa88b1ff700 1 -- 172.21.15.92:0/1309772766 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8840acc50 msgr2=0x7fa884159850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:27.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.238+0000 7fa88b1ff700 1 --2- 172.21.15.92:0/1309772766 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8840acc50 0x7fa884159850 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fa87c00efc0 tx=0x7fa87c00c3a0 comp rx=0 tx=0).stop 2023-11-28T11:25:27.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.238+0000 7fa88b1ff700 1 -- 172.21.15.92:0/1309772766 shutdown_connections 2023-11-28T11:25:27.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.238+0000 7fa88b1ff700 1 --2- 172.21.15.92:0/1309772766 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa8680682f0 0x7fa86806a7a0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:27.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.238+0000 7fa88b1ff700 1 --2- 172.21.15.92:0/1309772766 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8840acc50 0x7fa884159850 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:27.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.238+0000 7fa88b1ff700 1 --2- 172.21.15.92:0/1309772766 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa88406e970 0x7fa884158b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:27.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.238+0000 7fa88b1ff700 1 --2- 172.21.15.92:0/1309772766 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa88415a810 0x7fa884159310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:27.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.238+0000 7fa88b1ff700 1 -- 172.21.15.92:0/1309772766 >> 172.21.15.92:0/1309772766 conn(0x7fa8840b0db0 msgr2=0x7fa8840b1180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:27.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.238+0000 7fa88b1ff700 1 -- 172.21.15.92:0/1309772766 shutdown_connections 2023-11-28T11:25:27.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.238+0000 7fa88b1ff700 1 -- 172.21.15.92:0/1309772766 wait complete. 2023-11-28T11:25:27.241 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 33 2023-11-28T11:25:27.259 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:25:27.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.453+0000 7faf1a9ff700 1 Processor -- start 2023-11-28T11:25:27.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.453+0000 7faf1a9ff700 1 -- start start 2023-11-28T11:25:27.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.454+0000 7faf1a9ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf140cd460 0x7faf1406c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:27.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.454+0000 7faf1a9ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf140b88e0 con 0x7faf140d0d60 2023-11-28T11:25:27.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.454+0000 7faf1a9ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf140cd8f0 con 0x7faf140b8510 2023-11-28T11:25:27.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.454+0000 7faf1a9ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf140cdbd0 con 0x7faf140cd460 2023-11-28T11:25:27.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.455+0000 7faf18f9c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7faf140d0d60 0x7faf140d11e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59036/0 (socket says 172.21.15.92:59036) 2023-11-28T11:25:27.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.455+0000 7faf18f9c700 1 -- 172.21.15.92:0/705775477 learned_addr learned my addr 172.21.15.92:0/705775477 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:25:27.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.455+0000 7faf137fe700 1 --2- 172.21.15.92:0/705775477 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf140cd460 0x7faf1406c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:27.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.455+0000 7faf12ffd700 1 -- 172.21.15.92:0/705775477 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2778652218 0 0) 0x7faf140b88e0 con 0x7faf140d0d60 2023-11-28T11:25:27.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.455+0000 7faf12ffd700 1 -- 172.21.15.92:0/705775477 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faefc0035b0 con 0x7faf140d0d60 2023-11-28T11:25:27.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.455+0000 7faf12ffd700 1 -- 172.21.15.92:0/705775477 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 740828529 0 0) 0x7faf140cd8f0 con 0x7faf140b8510 2023-11-28T11:25:27.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.455+0000 7faf12ffd700 1 -- 172.21.15.92:0/705775477 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf140b88e0 con 0x7faf140b8510 2023-11-28T11:25:27.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.455+0000 7faf12ffd700 1 -- 172.21.15.92:0/705775477 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1809914435 0 0) 0x7faefc0035b0 con 0x7faf140d0d60 2023-11-28T11:25:27.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.455+0000 7faf137fe700 1 -- 172.21.15.92:0/705775477 >> v1:172.21.15.169:6789/0 conn(0x7faf140b8510 legacy=0x7faf1406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:27.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.455+0000 7faf137fe700 1 -- 172.21.15.92:0/705775477 >> v1:172.21.15.92:6789/0 conn(0x7faf140d0d60 legacy=0x7faf140d11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:27.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.455+0000 7faf137fe700 1 -- 172.21.15.92:0/705775477 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf140cfd10 con 0x7faf140cd460 2023-11-28T11:25:27.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.456+0000 7faf137fe700 1 --2- 172.21.15.92:0/705775477 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf140cd460 0x7faf1406c080 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7faf08009870 tx=0x7faf08009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=690c4534a4f792ac server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:27.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.456+0000 7faf12ffd700 1 -- 172.21.15.92:0/705775477 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0c0019f0 con 0x7faf140cd460 2023-11-28T11:25:27.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.456+0000 7faf12ffd700 1 -- 172.21.15.92:0/705775477 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf08004030 con 0x7faf140cd460 2023-11-28T11:25:27.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.456+0000 7faf12ffd700 1 -- 172.21.15.92:0/705775477 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf080043c0 con 0x7faf140cd460 2023-11-28T11:25:27.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.456+0000 7faf1a9ff700 1 -- 172.21.15.92:0/705775477 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf140cd460 msgr2=0x7faf1406c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:27.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.456+0000 7faf1a9ff700 1 --2- 172.21.15.92:0/705775477 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf140cd460 0x7faf1406c080 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7faf08009870 tx=0x7faf08009b80 comp rx=0 tx=0).stop 2023-11-28T11:25:27.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.457+0000 7faf1a9ff700 1 -- 172.21.15.92:0/705775477 shutdown_connections 2023-11-28T11:25:27.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.457+0000 7faf1a9ff700 1 --2- 172.21.15.92:0/705775477 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf140cd460 0x7faf1406c080 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:27.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.457+0000 7faf1a9ff700 1 -- 172.21.15.92:0/705775477 >> 172.21.15.92:0/705775477 conn(0x7faf140719a0 msgr2=0x7faf14071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:27.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.457+0000 7faf1a9ff700 1 -- 172.21.15.92:0/705775477 shutdown_connections 2023-11-28T11:25:27.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.457+0000 7faf1a9ff700 1 -- 172.21.15.92:0/705775477 wait complete. 2023-11-28T11:25:27.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.458+0000 7faf1a9ff700 1 Processor -- start 2023-11-28T11:25:27.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.459+0000 7faf1a9ff700 1 -- start start 2023-11-28T11:25:27.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.459+0000 7faf1a9ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf140b8510 0x7faf1414b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:27.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.459+0000 7faf1a9ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf140cd460 0x7faf140b6e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:27.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.459+0000 7faf1a9ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf140d0d60 0x7faf140b7390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:27.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.459+0000 7faf1a9ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf140ccd90 con 0x7faf140cd460 2023-11-28T11:25:27.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.459+0000 7faf1a9ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf1414b870 con 0x7faf140b8510 2023-11-28T11:25:27.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.459+0000 7faf1a9ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf140cbcf0 con 0x7faf140d0d60 2023-11-28T11:25:27.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.460+0000 7faf137fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf140cd460 0x7faf140b6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:27.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.460+0000 7faf137fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf140cd460 0x7faf140b6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44768/0 (socket says 172.21.15.92:44768) 2023-11-28T11:25:27.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.460+0000 7faf137fe700 1 -- 172.21.15.92:0/827157706 learned_addr learned my addr 172.21.15.92:0/827157706 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:27.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.460+0000 7faf13fff700 1 --2- 172.21.15.92:0/827157706 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf140b8510 0x7faf1414b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:27.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.460+0000 7faf18f9c700 1 --2- 172.21.15.92:0/827157706 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf140d0d60 0x7faf140b7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:27.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.460+0000 7faf137fe700 1 -- 172.21.15.92:0/827157706 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf140d0d60 msgr2=0x7faf140b7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:27.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.460+0000 7faf137fe700 1 --2- 172.21.15.92:0/827157706 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf140d0d60 0x7faf140b7390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:27.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.460+0000 7faf137fe700 1 -- 172.21.15.92:0/827157706 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf140b8510 msgr2=0x7faf1414b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:27.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.460+0000 7faf137fe700 1 --2- 172.21.15.92:0/827157706 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf140b8510 0x7faf1414b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:27.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.460+0000 7faf137fe700 1 -- 172.21.15.92:0/827157706 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf080094a0 con 0x7faf140cd460 2023-11-28T11:25:27.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.460+0000 7faf137fe700 1 --2- 172.21.15.92:0/827157706 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf140cd460 0x7faf140b6e50 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7faf08000c00 tx=0x7faf08004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:27.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.461+0000 7faf10ff9700 1 -- 172.21.15.92:0/827157706 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf08003e60 con 0x7faf140cd460 2023-11-28T11:25:27.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.461+0000 7faf10ff9700 1 -- 172.21.15.92:0/827157706 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf0801f070 con 0x7faf140cd460 2023-11-28T11:25:27.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.461+0000 7faf1a9ff700 1 -- 172.21.15.92:0/827157706 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf140d29c0 con 0x7faf140cd460 2023-11-28T11:25:27.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.461+0000 7faf10ff9700 1 -- 172.21.15.92:0/827157706 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf08011850 con 0x7faf140cd460 2023-11-28T11:25:27.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.461+0000 7faf1a9ff700 1 -- 172.21.15.92:0/827157706 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf140cfff0 con 0x7faf140cd460 2023-11-28T11:25:27.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.462+0000 7faf1a9ff700 1 -- 172.21.15.92:0/827157706 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf140d2b50 con 0x7faf140cd460 2023-11-28T11:25:27.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.463+0000 7faf10ff9700 1 -- 172.21.15.92:0/827157706 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7faf08017020 con 0x7faf140cd460 2023-11-28T11:25:27.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.464+0000 7faf10ff9700 1 --2- 172.21.15.92:0/827157706 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7faf000682f0 0x7faf0006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:27.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.464+0000 7faf10ff9700 1 -- 172.21.15.92:0/827157706 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6556+0+0 (secure 0 0 0) 0x7faf08021070 con 0x7faf140cd460 2023-11-28T11:25:27.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.464+0000 7faf13fff700 1 --2- 172.21.15.92:0/827157706 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7faf000682f0 0x7faf0006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:27.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.464+0000 7faf13fff700 1 --2- 172.21.15.92:0/827157706 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7faf000682f0 0x7faf0006a7a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7faf040011e0 tx=0x7faf04009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:27.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.468+0000 7faf10ff9700 1 -- 172.21.15.92:0/827157706 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7faf08053ce0 con 0x7faf140cd460 2023-11-28T11:25:27.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.701+0000 7faf1a9ff700 1 -- 172.21.15.92:0/827157706 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf14067880 con 0x7faf140cd460 2023-11-28T11:25:27.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.701+0000 7faf10ff9700 1 -- 172.21.15.92:0/827157706 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4656 (secure 0 0 0) 0x7faf08057300 con 0x7faf140cd460 2023-11-28T11:25:27.705 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:25:27.705 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5525,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2067956596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2067956596},{"type":"v1","addr":"172.21.15.169:6835","nonce":2067956596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5971,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2367049160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2367049160},{"type":"v1","addr":"172.21.15.169:6834","nonce":2367049160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5977,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1424858585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":1424858585},{"type":"v1","addr":"172.21.15.92:6837","nonce":1424858585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:24:16.257826+0000","modified":"2023-11-28T11:24:32.577560+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6213},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6213":{"gid":6213,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6836/3634521248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3634521248},{"type":"v1","addr":"172.21.15.92:6836","nonce":3634521248}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T11:25:27.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.704+0000 7faf1a9ff700 1 -- 172.21.15.92:0/827157706 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7faf000682f0 msgr2=0x7faf0006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:27.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.704+0000 7faf1a9ff700 1 --2- 172.21.15.92:0/827157706 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7faf000682f0 0x7faf0006a7a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7faf040011e0 tx=0x7faf04009040 comp rx=0 tx=0).stop 2023-11-28T11:25:27.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.704+0000 7faf1a9ff700 1 -- 172.21.15.92:0/827157706 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf140cd460 msgr2=0x7faf140b6e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:27.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.704+0000 7faf1a9ff700 1 --2- 172.21.15.92:0/827157706 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf140cd460 0x7faf140b6e50 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7faf08000c00 tx=0x7faf08004750 comp rx=0 tx=0).stop 2023-11-28T11:25:27.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.704+0000 7faf1a9ff700 1 -- 172.21.15.92:0/827157706 shutdown_connections 2023-11-28T11:25:27.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.704+0000 7faf1a9ff700 1 --2- 172.21.15.92:0/827157706 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7faf140d0d60 0x7faf140b7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:27.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.704+0000 7faf1a9ff700 1 --2- 172.21.15.92:0/827157706 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7faf140b8510 0x7faf1414b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:27.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.704+0000 7faf1a9ff700 1 --2- 172.21.15.92:0/827157706 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7faf000682f0 0x7faf0006a7a0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:27.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.704+0000 7faf1a9ff700 1 --2- 172.21.15.92:0/827157706 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf140cd460 0x7faf140b6e50 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:27.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.704+0000 7faf1a9ff700 1 -- 172.21.15.92:0/827157706 >> 172.21.15.92:0/827157706 conn(0x7faf140719a0 msgr2=0x7faf140d11a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:27.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.704+0000 7faf1a9ff700 1 -- 172.21.15.92:0/827157706 shutdown_connections 2023-11-28T11:25:27.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.704+0000 7faf1a9ff700 1 -- 172.21.15.92:0/827157706 wait complete. 2023-11-28T11:25:27.710 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 33 2023-11-28T11:25:27.724 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:25:27.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.919+0000 7f7d949ff700 1 Processor -- start 2023-11-28T11:25:27.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.919+0000 7f7d949ff700 1 -- start start 2023-11-28T11:25:27.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.920+0000 7f7d949ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d900b8510 0x7f7d9006b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:27.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.920+0000 7f7d949ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d900b88e0 con 0x7f7d900cd460 2023-11-28T11:25:27.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.920+0000 7f7d949ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d900cd8f0 con 0x7f7d900d0d60 2023-11-28T11:25:27.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.920+0000 7f7d949ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d900cdbd0 con 0x7f7d900b8510 2023-11-28T11:25:27.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.921+0000 7f7d8dd9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f7d900cd460 0x7f7d9006beb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59046/0 (socket says 172.21.15.92:59046) 2023-11-28T11:25:27.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.921+0000 7f7d8dd9b700 1 -- 172.21.15.92:0/742749693 learned_addr learned my addr 172.21.15.92:0/742749693 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:25:27.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.921+0000 7f7d8d59a700 1 -- 172.21.15.92:0/742749693 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4005054510 0 0) 0x7f7d900b88e0 con 0x7f7d900cd460 2023-11-28T11:25:27.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.921+0000 7f7d8d59a700 1 -- 172.21.15.92:0/742749693 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d780035b0 con 0x7f7d900cd460 2023-11-28T11:25:27.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.921+0000 7f7d8e59c700 1 --2- 172.21.15.92:0/742749693 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d900b8510 0x7f7d9006b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:27.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.921+0000 7f7d8d59a700 1 -- 172.21.15.92:0/742749693 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 697609613 0 0) 0x7f7d780035b0 con 0x7f7d900cd460 2023-11-28T11:25:27.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.921+0000 7f7d8d59a700 1 -- 172.21.15.92:0/742749693 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d900b8510 msgr2=0x7f7d9006b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:27.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.921+0000 7f7d8d59a700 1 --2- 172.21.15.92:0/742749693 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d900b8510 0x7f7d9006b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:27.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.921+0000 7f7d8d59a700 1 -- 172.21.15.92:0/742749693 >> v1:172.21.15.169:6789/0 conn(0x7f7d900d0d60 legacy=0x7f7d900d11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:27.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.921+0000 7f7d8d59a700 1 -- 172.21.15.92:0/742749693 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d900cfd10 con 0x7f7d900cd460 2023-11-28T11:25:27.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.921+0000 7f7d8d59a700 1 -- 172.21.15.92:0/742749693 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532161587 0 0) 0x7f7d84001670 con 0x7f7d900cd460 2023-11-28T11:25:27.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.921+0000 7f7d8d59a700 1 -- 172.21.15.92:0/742749693 >> v1:172.21.15.92:6789/0 conn(0x7f7d900cd460 legacy=0x7f7d9006beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:27.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.922+0000 7f7d8d59a700 1 --2- 172.21.15.92:0/742749693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d78003a30 0x7f7d78003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:27.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.922+0000 7f7d8d59a700 1 --2- 172.21.15.92:0/742749693 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d78004850 0x7f7d78004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:27.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.922+0000 7f7d8d59a700 1 --2- 172.21.15.92:0/742749693 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d900b8510 0x7f7d78005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:27.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.922+0000 7f7d8d59a700 1 -- 172.21.15.92:0/742749693 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d900cdbd0 con 0x7f7d78003a30 2023-11-28T11:25:27.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.922+0000 7f7d8d59a700 1 -- 172.21.15.92:0/742749693 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7800baf0 con 0x7f7d78004850 2023-11-28T11:25:27.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.922+0000 7f7d8d59a700 1 -- 172.21.15.92:0/742749693 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7800bcb0 con 0x7f7d900b8510 2023-11-28T11:25:27.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.922+0000 7f7d8e59c700 1 --2- 172.21.15.92:0/742749693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d78003a30 0x7f7d78003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:27.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.922+0000 7f7d8ed9d700 1 --2- 172.21.15.92:0/742749693 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d900b8510 0x7f7d78005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:27.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.922+0000 7f7d8dd9b700 1 --2- 172.21.15.92:0/742749693 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d78004850 0x7f7d78004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:27.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.922+0000 7f7d8e59c700 1 -- 172.21.15.92:0/742749693 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d900b8510 msgr2=0x7f7d78005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:27.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.922+0000 7f7d8e59c700 1 --2- 172.21.15.92:0/742749693 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d900b8510 0x7f7d78005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:27.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.922+0000 7f7d8e59c700 1 -- 172.21.15.92:0/742749693 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d78004850 msgr2=0x7f7d78004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:27.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.922+0000 7f7d8e59c700 1 --2- 172.21.15.92:0/742749693 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d78004850 0x7f7d78004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:27.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.922+0000 7f7d8e59c700 1 -- 172.21.15.92:0/742749693 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d780035b0 con 0x7f7d78003a30 2023-11-28T11:25:27.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.923+0000 7f7d8e59c700 1 --2- 172.21.15.92:0/742749693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d78003a30 0x7f7d78003e40 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f7d80009b60 tx=0x7f7d78005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc1e6c9672603bc0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:27.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.923+0000 7f7d8d59a700 1 -- 172.21.15.92:0/742749693 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8000e6c0 con 0x7f7d78003a30 2023-11-28T11:25:27.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.923+0000 7f7d8d59a700 1 -- 172.21.15.92:0/742749693 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d8000e820 con 0x7f7d78003a30 2023-11-28T11:25:27.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.923+0000 7f7d8d59a700 1 -- 172.21.15.92:0/742749693 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8000eb90 con 0x7f7d78003a30 2023-11-28T11:25:27.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.923+0000 7f7d949ff700 1 -- 172.21.15.92:0/742749693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d78003a30 msgr2=0x7f7d78003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:27.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.923+0000 7f7d949ff700 1 --2- 172.21.15.92:0/742749693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d78003a30 0x7f7d78003e40 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f7d80009b60 tx=0x7f7d78005880 comp rx=0 tx=0).stop 2023-11-28T11:25:27.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.923+0000 7f7d8ed9d700 1 -- 172.21.15.92:0/742749693 reap_dead start 2023-11-28T11:25:27.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.923+0000 7f7d949ff700 1 -- 172.21.15.92:0/742749693 shutdown_connections 2023-11-28T11:25:27.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.923+0000 7f7d949ff700 1 -- 172.21.15.92:0/742749693 >> 172.21.15.92:0/742749693 conn(0x7f7d900719a0 msgr2=0x7f7d90071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:27.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.923+0000 7f7d949ff700 1 -- 172.21.15.92:0/742749693 shutdown_connections 2023-11-28T11:25:27.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.923+0000 7f7d949ff700 1 -- 172.21.15.92:0/742749693 wait complete. 2023-11-28T11:25:27.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.925+0000 7f7d949ff700 1 Processor -- start 2023-11-28T11:25:27.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.925+0000 7f7d949ff700 1 -- start start 2023-11-28T11:25:27.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.926+0000 7f7d949ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d78003a30 0x7f7d901534d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:27.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.926+0000 7f7d949ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d90153d10 0x7f7d900cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:27.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.926+0000 7f7d949ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d900cccf0 0x7f7d900ca4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:27.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.926+0000 7f7d949ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d900cd160 con 0x7f7d78003a30 2023-11-28T11:25:27.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.926+0000 7f7d949ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d90159c50 con 0x7f7d90153d10 2023-11-28T11:25:27.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.926+0000 7f7d949ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d900b1c70 con 0x7f7d900cccf0 2023-11-28T11:25:27.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.926+0000 7f7d8e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d78003a30 0x7f7d901534d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:27.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.926+0000 7f7d8e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d78003a30 0x7f7d901534d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44800/0 (socket says 172.21.15.92:44800) 2023-11-28T11:25:27.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.926+0000 7f7d8e59c700 1 -- 172.21.15.92:0/2483614575 learned_addr learned my addr 172.21.15.92:0/2483614575 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:27.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.927+0000 7f7d8e59c700 1 -- 172.21.15.92:0/2483614575 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d900cccf0 msgr2=0x7f7d900ca4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:27.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.927+0000 7f7d8e59c700 1 --2- 172.21.15.92:0/2483614575 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d900cccf0 0x7f7d900ca4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:27.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.927+0000 7f7d8e59c700 1 -- 172.21.15.92:0/2483614575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d90153d10 msgr2=0x7f7d900cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:27.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.927+0000 7f7d8dd9b700 1 --2- 172.21.15.92:0/2483614575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d90153d10 0x7f7d900cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:27.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.927+0000 7f7d8e59c700 1 --2- 172.21.15.92:0/2483614575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d90153d10 0x7f7d900cc7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:27.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.927+0000 7f7d8e59c700 1 -- 172.21.15.92:0/2483614575 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d900d29c0 con 0x7f7d78003a30 2023-11-28T11:25:27.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.927+0000 7f7d8e59c700 1 --2- 172.21.15.92:0/2483614575 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d78003a30 0x7f7d901534d0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f7d7800bac0 tx=0x7f7d800047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:27.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.927+0000 7f7d7effd700 1 -- 172.21.15.92:0/2483614575 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8000e6c0 con 0x7f7d78003a30 2023-11-28T11:25:27.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.927+0000 7f7d7effd700 1 -- 172.21.15.92:0/2483614575 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d80003ab0 con 0x7f7d78003a30 2023-11-28T11:25:27.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.927+0000 7f7d949ff700 1 -- 172.21.15.92:0/2483614575 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d900cfff0 con 0x7f7d78003a30 2023-11-28T11:25:27.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.927+0000 7f7d7effd700 1 -- 172.21.15.92:0/2483614575 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d80003e60 con 0x7f7d78003a30 2023-11-28T11:25:27.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.927+0000 7f7d949ff700 1 -- 172.21.15.92:0/2483614575 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d900d2b50 con 0x7f7d78003a30 2023-11-28T11:25:27.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.928+0000 7f7d949ff700 1 -- 172.21.15.92:0/2483614575 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d900cd100 con 0x7f7d78003a30 2023-11-28T11:25:27.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.929+0000 7f7d7effd700 1 -- 172.21.15.92:0/2483614575 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7d80025070 con 0x7f7d78003a30 2023-11-28T11:25:27.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.930+0000 7f7d7effd700 1 --2- 172.21.15.92:0/2483614575 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7d700682f0 0x7f7d7006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:27.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.931+0000 7f7d7effd700 1 -- 172.21.15.92:0/2483614575 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6556+0+0 (secure 0 0 0) 0x7f7d80087700 con 0x7f7d78003a30 2023-11-28T11:25:27.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.931+0000 7f7d8dd9b700 1 --2- 172.21.15.92:0/2483614575 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7d700682f0 0x7f7d7006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:27.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.931+0000 7f7d8dd9b700 1 --2- 172.21.15.92:0/2483614575 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7d700682f0 0x7f7d7006a7a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7d84000c00 tx=0x7f7d84001b70 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:27.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:27.935+0000 7f7d7effd700 1 -- 172.21.15.92:0/2483614575 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7d80053c80 con 0x7f7d78003a30 2023-11-28T11:25:28.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.128+0000 7f7d949ff700 1 -- 172.21.15.92:0/2483614575 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7d90067880 con 0x7f7d78003a30 2023-11-28T11:25:28.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.129+0000 7f7d7effd700 1 -- 172.21.15.92:0/2483614575 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+18417 (secure 0 0 0) 0x7f7d800572a0 con 0x7f7d78003a30 2023-11-28T11:25:28.132 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:25:28.132 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":70,"fsid":"68956e0c-c16f-47cd-a220-f98364b0831d","created":"2023-11-28T11:20:28.697573+0000","modified":"2023-11-28T11:24:38.644982+0000","last_up_change":"2023-11-28T11:20:41.257011+0000","last_in_change":"2023-11-28T11:20:32.249767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:20:41.925237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:20:48.682497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:24:13.972356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-28T11:24:15.237693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6abe483f-0e03-4a26-b9cf-62041291747e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":115823},{"type":"v1","addr":"172.21.15.92:6809","nonce":115823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":115823},{"type":"v1","addr":"172.21.15.92:6813","nonce":115823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":115823},{"type":"v1","addr":"172.21.15.92:6821","nonce":115823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":115823},{"type":"v1","addr":"172.21.15.92:6817","nonce":115823}]},"public_addr":"172.21.15.92:6809/115823","cluster_addr":"172.21.15.92:6813/115823","heartbeat_back_addr":"172.21.15.92:6821/115823","heartbeat_front_addr":"172.21.15.92:6817/115823","state":["exists","up"]},{"osd":1,"uuid":"13c1de73-1a8c-4d7e-8a28-9dae52dfe089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":115824},{"type":"v1","addr":"172.21.15.92:6812","nonce":115824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":115824},{"type":"v1","addr":"172.21.15.92:6816","nonce":115824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":115824},{"type":"v1","addr":"172.21.15.92:6823","nonce":115824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":115824},{"type":"v1","addr":"172.21.15.92:6820","nonce":115824}]},"public_addr":"172.21.15.92:6812/115824","cluster_addr":"172.21.15.92:6816/115824","heartbeat_back_addr":"172.21.15.92:6823/115824","heartbeat_front_addr":"172.21.15.92:6820/115824","state":["exists","up"]},{"osd":2,"uuid":"25788a55-112f-4f00-810d-2bc37cf76778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":115821},{"type":"v1","addr":"172.21.15.92:6801","nonce":115821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":115821},{"type":"v1","addr":"172.21.15.92:6803","nonce":115821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":115821},{"type":"v1","addr":"172.21.15.92:6807","nonce":115821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":115821},{"type":"v1","addr":"172.21.15.92:6805","nonce":115821}]},"public_addr":"172.21.15.92:6801/115821","cluster_addr":"172.21.15.92:6803/115821","heartbeat_back_addr":"172.21.15.92:6807/115821","heartbeat_front_addr":"172.21.15.92:6805/115821","state":["exists","up"]},{"osd":3,"uuid":"adf54e24-33a9-41fe-9ce3-98a5ab6c158e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":115825},{"type":"v1","addr":"172.21.15.92:6825","nonce":115825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":115825},{"type":"v1","addr":"172.21.15.92:6827","nonce":115825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":115825},{"type":"v1","addr":"172.21.15.92:6831","nonce":115825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":115825},{"type":"v1","addr":"172.21.15.92:6829","nonce":115825}]},"public_addr":"172.21.15.92:6825/115825","cluster_addr":"172.21.15.92:6827/115825","heartbeat_back_addr":"172.21.15.92:6831/115825","heartbeat_front_addr":"172.21.15.92:6829/115825","state":["exists","up"]},{"osd":4,"uuid":"ccb9b747-f399-44ea-9a18-9db3152227e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":116035},{"type":"v1","addr":"172.21.15.169:6817","nonce":116035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":116035},{"type":"v1","addr":"172.21.15.169:6819","nonce":116035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":116035},{"type":"v1","addr":"172.21.15.169:6826","nonce":116035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":116035},{"type":"v1","addr":"172.21.15.169:6822","nonce":116035}]},"public_addr":"172.21.15.169:6817/116035","cluster_addr":"172.21.15.169:6819/116035","heartbeat_back_addr":"172.21.15.169:6826/116035","heartbeat_front_addr":"172.21.15.169:6822/116035","state":["exists","up"]},{"osd":5,"uuid":"308a1e63-75aa-4d7d-8a7c-cecc24714fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":116033},{"type":"v1","addr":"172.21.15.169:6801","nonce":116033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":116033},{"type":"v1","addr":"172.21.15.169:6803","nonce":116033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":116033},{"type":"v1","addr":"172.21.15.169:6807","nonce":116033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":116033},{"type":"v1","addr":"172.21.15.169:6805","nonce":116033}]},"public_addr":"172.21.15.169:6801/116033","cluster_addr":"172.21.15.169:6803/116033","heartbeat_back_addr":"172.21.15.169:6807/116033","heartbeat_front_addr":"172.21.15.169:6805/116033","state":["exists","up"]},{"osd":6,"uuid":"9c459b7b-b821-4b4b-b91e-6aaae271457c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":116034},{"type":"v1","addr":"172.21.15.169:6809","nonce":116034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":116034},{"type":"v1","addr":"172.21.15.169:6811","nonce":116034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":116034},{"type":"v1","addr":"172.21.15.169:6815","nonce":116034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":116034},{"type":"v1","addr":"172.21.15.169:6813","nonce":116034}]},"public_addr":"172.21.15.169:6809/116034","cluster_addr":"172.21.15.169:6811/116034","heartbeat_back_addr":"172.21.15.169:6815/116034","heartbeat_front_addr":"172.21.15.169:6813/116034","state":["exists","up"]},{"osd":7,"uuid":"1ca9aa4a-755d-4e28-a937-4d8dccab4df3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":116044},{"type":"v1","addr":"172.21.15.169:6823","nonce":116044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6825","nonce":116044},{"type":"v1","addr":"172.21.15.169:6827","nonce":116044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":116044},{"type":"v1","addr":"172.21.15.169:6831","nonce":116044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":116044},{"type":"v1","addr":"172.21.15.169:6829","nonce":116044}]},"public_addr":"172.21.15.169:6823/116044","cluster_addr":"172.21.15.169:6827/116044","heartbeat_back_addr":"172.21.15.169:6831/116044","heartbeat_front_addr":"172.21.15.169:6829/116044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.586781+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.869778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.508594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.860493+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.872267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.746933+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.611520+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.871446+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.10","mappings":[{"from":6,"to":5}]},{"pgid":"11.17","mappings":[{"from":6,"to":1}]},{"pgid":"11.29","mappings":[{"from":7,"to":0}]},{"pgid":"11.31","mappings":[{"from":2,"to":4}]},{"pgid":"11.33","mappings":[{"from":6,"to":1}]},{"pgid":"11.3e","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:25:28.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.131+0000 7f7d949ff700 1 -- 172.21.15.92:0/2483614575 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7d700682f0 msgr2=0x7f7d7006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:28.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.131+0000 7f7d949ff700 1 --2- 172.21.15.92:0/2483614575 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7d700682f0 0x7f7d7006a7a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7d84000c00 tx=0x7f7d84001b70 comp rx=0 tx=0).stop 2023-11-28T11:25:28.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.131+0000 7f7d949ff700 1 -- 172.21.15.92:0/2483614575 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d78003a30 msgr2=0x7f7d901534d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:28.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.131+0000 7f7d949ff700 1 --2- 172.21.15.92:0/2483614575 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d78003a30 0x7f7d901534d0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f7d7800bac0 tx=0x7f7d800047c0 comp rx=0 tx=0).stop 2023-11-28T11:25:28.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.132+0000 7f7d949ff700 1 -- 172.21.15.92:0/2483614575 shutdown_connections 2023-11-28T11:25:28.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.132+0000 7f7d949ff700 1 --2- 172.21.15.92:0/2483614575 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f7d900cccf0 0x7f7d900ca4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:28.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.132+0000 7f7d949ff700 1 --2- 172.21.15.92:0/2483614575 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f7d700682f0 0x7f7d7006a7a0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:28.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.132+0000 7f7d949ff700 1 --2- 172.21.15.92:0/2483614575 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d78003a30 0x7f7d901534d0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:28.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.132+0000 7f7d949ff700 1 --2- 172.21.15.92:0/2483614575 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f7d90153d10 0x7f7d900cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:28.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.132+0000 7f7d949ff700 1 -- 172.21.15.92:0/2483614575 >> 172.21.15.92:0/2483614575 conn(0x7f7d900719a0 msgr2=0x7f7d900d1520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:28.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.132+0000 7f7d949ff700 1 -- 172.21.15.92:0/2483614575 shutdown_connections 2023-11-28T11:25:28.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.132+0000 7f7d949ff700 1 -- 172.21.15.92:0/2483614575 wait complete. 2023-11-28T11:25:28.152 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T11:25:28.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.353+0000 7fa4c89ff700 1 Processor -- start 2023-11-28T11:25:28.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.353+0000 7fa4c89ff700 1 -- start start 2023-11-28T11:25:28.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.353+0000 7fa4c89ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa4c40b8510 0x7fa4c406b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:28.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.353+0000 7fa4c89ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4c40d0d60 0x7fa4c40d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:28.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.353+0000 7fa4c89ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4c40b88e0 con 0x7fa4c40cd460 2023-11-28T11:25:28.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.353+0000 7fa4c89ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4c40cdbd0 con 0x7fa4c40b8510 2023-11-28T11:25:28.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.353+0000 7fa4c89ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c40d02d0 con 0x7fa4c40d0d60 2023-11-28T11:25:28.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.354+0000 7fa4c2d9d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4c40d0d60 0x7fa4c40d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:28.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.354+0000 7fa4c259c700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa4c40b8510 0x7fa4c406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:28.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.354+0000 7fa4bbfff700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fa4c40cd460 0x7fa4c406beb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.92:53904/0 (socket says 172.21.15.92:53904) 2023-11-28T11:25:28.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.354+0000 7fa4c2d9d700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4c40d0d60 0x7fa4c40d1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.92:39358/0 (socket says 172.21.15.92:39358) 2023-11-28T11:25:28.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.354+0000 7fa4bbfff700 1 -- 172.21.15.92:0/3465538807 learned_addr learned my addr 172.21.15.92:0/3465538807 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:25:28.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.355+0000 7fa4c2d9d700 1 -- 172.21.15.92:0/3465538807 >> v2:172.21.15.169:3300/0 conn(0x7fa4c40b8510 msgr2=0x7fa4c406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:28.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.355+0000 7fa4c2d9d700 1 --2- 172.21.15.92:0/3465538807 >> v2:172.21.15.169:3300/0 conn(0x7fa4c40b8510 0x7fa4c406b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:28.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.355+0000 7fa4c2d9d700 1 -- 172.21.15.92:0/3465538807 >> v1:172.21.15.169:6789/0 conn(0x7fa4c40cd460 legacy=0x7fa4c406beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:28.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.354+0000 7fa4c1d9b700 1 -- 172.21.15.92:0/3465538807 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1501028330 0 0) 0x7fa4c40b88e0 con 0x7fa4c40cd460 2023-11-28T11:25:28.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.355+0000 7fa4c2d9d700 1 -- 172.21.15.92:0/3465538807 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4c40cd8f0 con 0x7fa4c40d0d60 2023-11-28T11:25:28.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.355+0000 7fa4c2d9d700 1 --2- 172.21.15.92:0/3465538807 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4c40d0d60 0x7fa4c40d1200 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc009d90 tx=0x7fa4bc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92e56c38dcb4d5b3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:28.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.355+0000 7fa4c1d9b700 1 -- 172.21.15.92:0/3465538807 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc00e070 con 0x7fa4c40d0d60 2023-11-28T11:25:28.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.355+0000 7fa4c1d9b700 1 -- 172.21.15.92:0/3465538807 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4bc00bb20 con 0x7fa4c40d0d60 2023-11-28T11:25:28.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.355+0000 7fa4c1d9b700 1 -- 172.21.15.92:0/3465538807 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc012430 con 0x7fa4c40d0d60 2023-11-28T11:25:28.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.356+0000 7fa4c89ff700 1 -- 172.21.15.92:0/3465538807 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4c40d0d60 msgr2=0x7fa4c40d1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:28.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.356+0000 7fa4c89ff700 1 --2- 172.21.15.92:0/3465538807 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4c40d0d60 0x7fa4c40d1200 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc009d90 tx=0x7fa4bc00b330 comp rx=0 tx=0).stop 2023-11-28T11:25:28.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.356+0000 7fa4c89ff700 1 -- 172.21.15.92:0/3465538807 shutdown_connections 2023-11-28T11:25:28.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.356+0000 7fa4c89ff700 1 --2- 172.21.15.92:0/3465538807 >> v2:172.21.15.169:3300/0 conn(0x7fa4c40b8510 0x7fa4c406b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:28.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.356+0000 7fa4c89ff700 1 --2- 172.21.15.92:0/3465538807 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4c40d0d60 0x7fa4c40d1200 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:28.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.356+0000 7fa4c89ff700 1 -- 172.21.15.92:0/3465538807 >> 172.21.15.92:0/3465538807 conn(0x7fa4c40719a0 msgr2=0x7fa4c4071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:28.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.356+0000 7fa4c89ff700 1 -- 172.21.15.92:0/3465538807 shutdown_connections 2023-11-28T11:25:28.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.356+0000 7fa4c89ff700 1 -- 172.21.15.92:0/3465538807 wait complete. 2023-11-28T11:25:28.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.357+0000 7fa4c89ff700 1 Processor -- start 2023-11-28T11:25:28.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.358+0000 7fa4c89ff700 1 -- start start 2023-11-28T11:25:28.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.358+0000 7fa4c89ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4c40b8510 0x7fa4c41f4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:28.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.358+0000 7fa4c89ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4c40cd460 0x7fa4c40cc720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:28.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.358+0000 7fa4c89ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4c40d0d60 0x7fa4c40ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:28.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.358+0000 7fa4c89ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c41f5060 con 0x7fa4c40cd460 2023-11-28T11:25:28.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.358+0000 7fa4c89ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c415ae90 con 0x7fa4c40d0d60 2023-11-28T11:25:28.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.358+0000 7fa4c89ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c41f1ce0 con 0x7fa4c40b8510 2023-11-28T11:25:28.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.359+0000 7fa4bbfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4c40cd460 0x7fa4c40cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:28.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.359+0000 7fa4c259c700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4c40b8510 0x7fa4c41f4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:28.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.359+0000 7fa4bbfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4c40cd460 0x7fa4c40cc720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44816/0 (socket says 172.21.15.92:44816) 2023-11-28T11:25:28.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.359+0000 7fa4bbfff700 1 -- 172.21.15.92:0/1332908802 learned_addr learned my addr 172.21.15.92:0/1332908802 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:28.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.359+0000 7fa4c2d9d700 1 --2- 172.21.15.92:0/1332908802 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4c40d0d60 0x7fa4c40ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:28.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.359+0000 7fa4bbfff700 1 -- 172.21.15.92:0/1332908802 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4c40b8510 msgr2=0x7fa4c41f4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:28.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.359+0000 7fa4bbfff700 1 --2- 172.21.15.92:0/1332908802 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4c40b8510 0x7fa4c41f4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:28.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.359+0000 7fa4bbfff700 1 -- 172.21.15.92:0/1332908802 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4c40d0d60 msgr2=0x7fa4c40ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:28.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.359+0000 7fa4bbfff700 1 --2- 172.21.15.92:0/1332908802 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4c40d0d60 0x7fa4c40ccc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:28.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.359+0000 7fa4bbfff700 1 -- 172.21.15.92:0/1332908802 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4bc009a70 con 0x7fa4c40cd460 2023-11-28T11:25:28.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.359+0000 7fa4c259c700 1 --2- 172.21.15.92:0/1332908802 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4c40b8510 0x7fa4c41f4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:25:28.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.359+0000 7fa4bbfff700 1 --2- 172.21.15.92:0/1332908802 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4c40cd460 0x7fa4c40cc720 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac00c520 tx=0x7fa4ac00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:28.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.360+0000 7fa4baffd700 1 -- 172.21.15.92:0/1332908802 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac004c80 con 0x7fa4c40cd460 2023-11-28T11:25:28.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.360+0000 7fa4baffd700 1 -- 172.21.15.92:0/1332908802 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4ac004e00 con 0x7fa4c40cd460 2023-11-28T11:25:28.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.360+0000 7fa4baffd700 1 -- 172.21.15.92:0/1332908802 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac007310 con 0x7fa4c40cd460 2023-11-28T11:25:28.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.360+0000 7fa4c89ff700 1 -- 172.21.15.92:0/1332908802 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4c40cfff0 con 0x7fa4c40cd460 2023-11-28T11:25:28.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.360+0000 7fa4c89ff700 1 -- 172.21.15.92:0/1332908802 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4c40cfd10 con 0x7fa4c40cd460 2023-11-28T11:25:28.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.361+0000 7fa4c89ff700 1 -- 172.21.15.92:0/1332908802 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4c41f1e70 con 0x7fa4c40cd460 2023-11-28T11:25:28.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.362+0000 7fa4baffd700 1 -- 172.21.15.92:0/1332908802 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa4ac01a020 con 0x7fa4c40cd460 2023-11-28T11:25:28.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.362+0000 7fa4baffd700 1 --2- 172.21.15.92:0/1332908802 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa4a40682f0 0x7fa4a406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:28.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.363+0000 7fa4baffd700 1 -- 172.21.15.92:0/1332908802 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6556+0+0 (secure 0 0 0) 0x7fa4ac085ad0 con 0x7fa4c40cd460 2023-11-28T11:25:28.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.363+0000 7fa4c259c700 1 --2- 172.21.15.92:0/1332908802 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa4a40682f0 0x7fa4a406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:28.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.363+0000 7fa4c259c700 1 --2- 172.21.15.92:0/1332908802 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa4a40682f0 0x7fa4a406a7a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa4b4009fd0 tx=0x7fa4b4009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:28.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.368+0000 7fa4baffd700 1 -- 172.21.15.92:0/1332908802 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa4ac052100 con 0x7fa4c40cd460 2023-11-28T11:25:28.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:28.601+0000 7fa4c89ff700 1 -- 172.21.15.92:0/1332908802 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fa4c4067880 con 0x7fa4c40cd460 2023-11-28T11:25:29.555 INFO:tasks.ceph.mon.a.smithi092.stderr:2023-11-28T11:25:29.554+0000 7f086abf9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T11:25:29.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.558+0000 7fa4baffd700 1 -- 172.21.15.92:0/1332908802 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v34) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa4ac055720 con 0x7fa4c40cd460 2023-11-28T11:25:29.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.562+0000 7fa4c89ff700 1 -- 172.21.15.92:0/1332908802 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa4a40682f0 msgr2=0x7fa4a406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:29.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.562+0000 7fa4c89ff700 1 --2- 172.21.15.92:0/1332908802 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa4a40682f0 0x7fa4a406a7a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa4b4009fd0 tx=0x7fa4b4009210 comp rx=0 tx=0).stop 2023-11-28T11:25:29.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.562+0000 7fa4c89ff700 1 -- 172.21.15.92:0/1332908802 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4c40cd460 msgr2=0x7fa4c40cc720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:29.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.562+0000 7fa4c89ff700 1 --2- 172.21.15.92:0/1332908802 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4c40cd460 0x7fa4c40cc720 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac00c520 tx=0x7fa4ac00c830 comp rx=0 tx=0).stop 2023-11-28T11:25:29.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.562+0000 7fa4c89ff700 1 -- 172.21.15.92:0/1332908802 shutdown_connections 2023-11-28T11:25:29.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.562+0000 7fa4c89ff700 1 --2- 172.21.15.92:0/1332908802 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa4c40d0d60 0x7fa4c40ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:29.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.562+0000 7fa4c89ff700 1 --2- 172.21.15.92:0/1332908802 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa4c40b8510 0x7fa4c41f4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:29.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.562+0000 7fa4c89ff700 1 --2- 172.21.15.92:0/1332908802 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa4a40682f0 0x7fa4a406a7a0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:29.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.562+0000 7fa4c89ff700 1 --2- 172.21.15.92:0/1332908802 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4c40cd460 0x7fa4c40cc720 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:29.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.562+0000 7fa4c89ff700 1 -- 172.21.15.92:0/1332908802 >> 172.21.15.92:0/1332908802 conn(0x7fa4c40719a0 msgr2=0x7fa4c40b3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:29.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.562+0000 7fa4c89ff700 1 -- 172.21.15.92:0/1332908802 shutdown_connections 2023-11-28T11:25:29.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.562+0000 7fa4c89ff700 1 -- 172.21.15.92:0/1332908802 wait complete. 2023-11-28T11:25:29.564 INFO:teuthology.orchestra.run.smithi092.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T11:25:29.573 INFO:tasks.ceph.mds.a.smithi092.stderr: -7359> 2023-11-28T11:24:20.747+0000 7f4f55bf1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T11:25:29.578 INFO:tasks.ceph.mds.a.smithi092.stderr: -3799> 2023-11-28T11:24:26.360+0000 7f4f523ea700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6115=0-4194304@1} caps={6115=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6115 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x564291d91b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(11)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-28T11:25:29.586 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T11:25:29.629 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-11-28T11:25:29.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.790+0000 7ff5c5bff700 1 Processor -- start 2023-11-28T11:25:29.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.790+0000 7ff5c5bff700 1 -- start start 2023-11-28T11:25:29.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.791+0000 7ff5c5bff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7ff5c00b8510 0x7ff5c006b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:29.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.791+0000 7ff5c5bff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5c00d0d60 0x7ff5c00d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:29.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.791+0000 7ff5c5bff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5c00b88e0 con 0x7ff5c00cd460 2023-11-28T11:25:29.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.791+0000 7ff5c5bff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5c00cdbd0 con 0x7ff5c00b8510 2023-11-28T11:25:29.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.791+0000 7ff5c5bff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c00d02d0 con 0x7ff5c00d0d60 2023-11-28T11:25:29.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.791+0000 7ff5beffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff5c00cd460 0x7ff5c006beb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36170/0 (socket says 172.21.15.92:36170) 2023-11-28T11:25:29.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.791+0000 7ff5beffd700 1 -- 172.21.15.92:0/3245001731 learned_addr learned my addr 172.21.15.92:0/3245001731 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:25:29.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.791+0000 7ff5bffff700 1 --2- 172.21.15.92:0/3245001731 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5c00d0d60 0x7ff5c00d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:29.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.791+0000 7ff5bf7fe700 1 --2- 172.21.15.92:0/3245001731 >> v2:172.21.15.169:3300/0 conn(0x7ff5c00b8510 0x7ff5c006b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:29.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.792+0000 7ff5be7fc700 1 -- 172.21.15.92:0/3245001731 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 27974498 0 0) 0x7ff5c00b88e0 con 0x7ff5c00cd460 2023-11-28T11:25:29.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.792+0000 7ff5be7fc700 1 -- 172.21.15.92:0/3245001731 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5a80035b0 con 0x7ff5c00cd460 2023-11-28T11:25:29.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.792+0000 7ff5bffff700 1 -- 172.21.15.92:0/3245001731 >> v2:172.21.15.169:3300/0 conn(0x7ff5c00b8510 msgr2=0x7ff5c006b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:29.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.792+0000 7ff5bffff700 1 --2- 172.21.15.92:0/3245001731 >> v2:172.21.15.169:3300/0 conn(0x7ff5c00b8510 0x7ff5c006b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:29.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.792+0000 7ff5bffff700 1 -- 172.21.15.92:0/3245001731 >> v1:172.21.15.92:6789/0 conn(0x7ff5c00cd460 legacy=0x7ff5c006beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:29.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.792+0000 7ff5bffff700 1 -- 172.21.15.92:0/3245001731 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c00cd8f0 con 0x7ff5c00d0d60 2023-11-28T11:25:29.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.792+0000 7ff5bffff700 1 --2- 172.21.15.92:0/3245001731 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5c00d0d60 0x7ff5c00d1200 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8009d90 tx=0x7ff5b800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aeb308471cff79e9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:29.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.792+0000 7ff5be7fc700 1 -- 172.21.15.92:0/3245001731 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b800e070 con 0x7ff5c00d0d60 2023-11-28T11:25:29.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.792+0000 7ff5be7fc700 1 -- 172.21.15.92:0/3245001731 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5b800bb20 con 0x7ff5c00d0d60 2023-11-28T11:25:29.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.792+0000 7ff5be7fc700 1 -- 172.21.15.92:0/3245001731 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b8012430 con 0x7ff5c00d0d60 2023-11-28T11:25:29.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.793+0000 7ff5c5bff700 1 -- 172.21.15.92:0/3245001731 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5c00d0d60 msgr2=0x7ff5c00d1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:29.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.793+0000 7ff5c5bff700 1 --2- 172.21.15.92:0/3245001731 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5c00d0d60 0x7ff5c00d1200 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8009d90 tx=0x7ff5b800b330 comp rx=0 tx=0).stop 2023-11-28T11:25:29.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.793+0000 7ff5c5bff700 1 -- 172.21.15.92:0/3245001731 shutdown_connections 2023-11-28T11:25:29.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.793+0000 7ff5c5bff700 1 --2- 172.21.15.92:0/3245001731 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5c00d0d60 0x7ff5c00d1200 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:29.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.793+0000 7ff5c5bff700 1 --2- 172.21.15.92:0/3245001731 >> v2:172.21.15.169:3300/0 conn(0x7ff5c00b8510 0x7ff5c006b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:29.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.793+0000 7ff5c5bff700 1 -- 172.21.15.92:0/3245001731 >> 172.21.15.92:0/3245001731 conn(0x7ff5c00719a0 msgr2=0x7ff5c0071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:29.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.793+0000 7ff5c5bff700 1 -- 172.21.15.92:0/3245001731 shutdown_connections 2023-11-28T11:25:29.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.793+0000 7ff5c5bff700 1 -- 172.21.15.92:0/3245001731 wait complete. 2023-11-28T11:25:29.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.795+0000 7ff5c5bff700 1 Processor -- start 2023-11-28T11:25:29.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.795+0000 7ff5c5bff700 1 -- start start 2023-11-28T11:25:29.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.796+0000 7ff5c5bff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5c00b8510 0x7ff5c0157cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:29.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.796+0000 7ff5c5bff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5c00cd460 0x7ff5c00c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:29.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.796+0000 7ff5c5bff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5c00d0d60 0x7ff5c00cc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:29.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.796+0000 7ff5c5bff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c00caba0 con 0x7ff5c00b8510 2023-11-28T11:25:29.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.796+0000 7ff5c5bff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c00cae80 con 0x7ff5c00cd460 2023-11-28T11:25:29.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.796+0000 7ff5c5bff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c01e4a90 con 0x7ff5c00d0d60 2023-11-28T11:25:29.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.796+0000 7ff5bf7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5c00b8510 0x7ff5c0157cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:29.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.796+0000 7ff5bf7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5c00b8510 0x7ff5c0157cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51206/0 (socket says 172.21.15.92:51206) 2023-11-28T11:25:29.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.796+0000 7ff5bf7fe700 1 -- 172.21.15.92:0/4218699208 learned_addr learned my addr 172.21.15.92:0/4218699208 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:29.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.797+0000 7ff5bf7fe700 1 -- 172.21.15.92:0/4218699208 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5c00d0d60 msgr2=0x7ff5c00cc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:29.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.797+0000 7ff5bf7fe700 1 --2- 172.21.15.92:0/4218699208 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5c00d0d60 0x7ff5c00cc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:29.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.797+0000 7ff5bf7fe700 1 -- 172.21.15.92:0/4218699208 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5c00cd460 msgr2=0x7ff5c00c9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:29.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.797+0000 7ff5bf7fe700 1 --2- 172.21.15.92:0/4218699208 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5c00cd460 0x7ff5c00c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:29.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.797+0000 7ff5bf7fe700 1 -- 172.21.15.92:0/4218699208 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5b8009a70 con 0x7ff5c00b8510 2023-11-28T11:25:29.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.797+0000 7ff5bf7fe700 1 --2- 172.21.15.92:0/4218699208 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5c00b8510 0x7ff5c0157cf0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7ff5b000e9b0 tx=0x7ff5b000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:29.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.797+0000 7ff5a7fff700 1 -- 172.21.15.92:0/4218699208 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b0009710 con 0x7ff5c00b8510 2023-11-28T11:25:29.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.797+0000 7ff5a7fff700 1 -- 172.21.15.92:0/4218699208 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5b000c850 con 0x7ff5c00b8510 2023-11-28T11:25:29.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.797+0000 7ff5a7fff700 1 -- 172.21.15.92:0/4218699208 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b000cbc0 con 0x7ff5c00b8510 2023-11-28T11:25:29.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.797+0000 7ff5c5bff700 1 -- 172.21.15.92:0/4218699208 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5c00cfff0 con 0x7ff5c00b8510 2023-11-28T11:25:29.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.797+0000 7ff5c5bff700 1 -- 172.21.15.92:0/4218699208 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5c00cfd10 con 0x7ff5c00b8510 2023-11-28T11:25:29.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.798+0000 7ff5c5bff700 1 -- 172.21.15.92:0/4218699208 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5c00ce400 con 0x7ff5c00b8510 2023-11-28T11:25:29.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.799+0000 7ff5a7fff700 1 -- 172.21.15.92:0/4218699208 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff5b0019030 con 0x7ff5c00b8510 2023-11-28T11:25:29.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.800+0000 7ff5a7fff700 1 --2- 172.21.15.92:0/4218699208 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff5ac0682f0 0x7ff5ac06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:29.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.800+0000 7ff5a7fff700 1 -- 172.21.15.92:0/4218699208 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6642+0+0 (secure 0 0 0) 0x7ff5b00860a0 con 0x7ff5c00b8510 2023-11-28T11:25:29.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.800+0000 7ff5beffd700 1 --2- 172.21.15.92:0/4218699208 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff5ac0682f0 0x7ff5ac06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:29.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.801+0000 7ff5beffd700 1 --2- 172.21.15.92:0/4218699208 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff5ac0682f0 0x7ff5ac06a7a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff5c00b11a0 tx=0x7ff5b40079c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:29.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:29.804+0000 7ff5a7fff700 1 -- 172.21.15.92:0/4218699208 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff5b0052530 con 0x7ff5c00b8510 2023-11-28T11:25:30.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.039+0000 7ff5c5bff700 1 -- 172.21.15.92:0/4218699208 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff5c0067880 con 0x7ff5c00b8510 2023-11-28T11:25:30.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.560+0000 7ff5a7fff700 1 -- 172.21.15.92:0/4218699208 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v35) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff5b00170a0 con 0x7ff5c00b8510 2023-11-28T11:25:30.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.563+0000 7ff5c5bff700 1 -- 172.21.15.92:0/4218699208 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff5ac0682f0 msgr2=0x7ff5ac06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:30.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.563+0000 7ff5c5bff700 1 --2- 172.21.15.92:0/4218699208 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff5ac0682f0 0x7ff5ac06a7a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff5c00b11a0 tx=0x7ff5b40079c0 comp rx=0 tx=0).stop 2023-11-28T11:25:30.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.563+0000 7ff5c5bff700 1 -- 172.21.15.92:0/4218699208 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5c00b8510 msgr2=0x7ff5c0157cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:30.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.563+0000 7ff5c5bff700 1 --2- 172.21.15.92:0/4218699208 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5c00b8510 0x7ff5c0157cf0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7ff5b000e9b0 tx=0x7ff5b000ed70 comp rx=0 tx=0).stop 2023-11-28T11:25:30.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.563+0000 7ff5c5bff700 1 -- 172.21.15.92:0/4218699208 shutdown_connections 2023-11-28T11:25:30.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.563+0000 7ff5c5bff700 1 --2- 172.21.15.92:0/4218699208 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7ff5c00d0d60 0x7ff5c00cc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:30.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.563+0000 7ff5c5bff700 1 --2- 172.21.15.92:0/4218699208 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7ff5ac0682f0 0x7ff5ac06a7a0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:30.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.563+0000 7ff5c5bff700 1 --2- 172.21.15.92:0/4218699208 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5c00b8510 0x7ff5c0157cf0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:30.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.563+0000 7ff5c5bff700 1 --2- 172.21.15.92:0/4218699208 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7ff5c00cd460 0x7ff5c00c9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:30.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.563+0000 7ff5c5bff700 1 -- 172.21.15.92:0/4218699208 >> 172.21.15.92:0/4218699208 conn(0x7ff5c00719a0 msgr2=0x7ff5c00b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:30.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.563+0000 7ff5c5bff700 1 -- 172.21.15.92:0/4218699208 shutdown_connections 2023-11-28T11:25:30.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.563+0000 7ff5c5bff700 1 -- 172.21.15.92:0/4218699208 wait complete. 2023-11-28T11:25:30.584 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T11:25:30.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.827+0000 7f2d99fff700 1 Processor -- start 2023-11-28T11:25:30.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.827+0000 7f2d99fff700 1 -- start start 2023-11-28T11:25:30.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.828+0000 7f2d99fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d940cd460 0x7f2d9406b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:30.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.828+0000 7f2d99fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2d9406be80 0x7f2d940d0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:30.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.828+0000 7f2d99fff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d940b5240 con 0x7f2d940b8790 2023-11-28T11:25:30.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.828+0000 7f2d99fff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d9406c300 con 0x7f2d9406be80 2023-11-28T11:25:30.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.828+0000 7f2d99fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d940b5520 con 0x7f2d940cd460 2023-11-28T11:25:30.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.828+0000 7f2d937fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f2d940b8790 0x7f2d940cfb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36178/0 (socket says 172.21.15.92:36178) 2023-11-28T11:25:30.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.828+0000 7f2d937fe700 1 -- 172.21.15.92:0/1449087553 learned_addr learned my addr 172.21.15.92:0/1449087553 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:25:30.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.828+0000 7f2d93fff700 1 --2- 172.21.15.92:0/1449087553 >> v2:172.21.15.92:3300/0 conn(0x7f2d9406be80 0x7f2d940d0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:30.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.829+0000 7f2d8bfff700 1 --2- 172.21.15.92:0/1449087553 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d940cd460 0x7f2d9406b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:30.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.829+0000 7f2d92ffd700 1 -- 172.21.15.92:0/1449087553 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 816236486 0 0) 0x7f2d940b5240 con 0x7f2d940b8790 2023-11-28T11:25:30.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.829+0000 7f2d92ffd700 1 -- 172.21.15.92:0/1449087553 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d800035b0 con 0x7f2d940b8790 2023-11-28T11:25:30.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.829+0000 7f2d93fff700 1 -- 172.21.15.92:0/1449087553 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d940cd460 msgr2=0x7f2d9406b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:30.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.829+0000 7f2d93fff700 1 --2- 172.21.15.92:0/1449087553 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d940cd460 0x7f2d9406b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:30.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.829+0000 7f2d93fff700 1 -- 172.21.15.92:0/1449087553 >> v1:172.21.15.92:6789/0 conn(0x7f2d940b8790 legacy=0x7f2d940cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:30.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.829+0000 7f2d93fff700 1 -- 172.21.15.92:0/1449087553 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d940d0a80 con 0x7f2d9406be80 2023-11-28T11:25:30.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.829+0000 7f2d93fff700 1 --2- 172.21.15.92:0/1449087553 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d9406be80 0x7f2d940d0fa0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c009d90 tx=0x7f2d8c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39a2802fd0925c4b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:30.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.829+0000 7f2d92ffd700 1 -- 172.21.15.92:0/1449087553 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c00e070 con 0x7f2d9406be80 2023-11-28T11:25:30.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.829+0000 7f2d92ffd700 1 -- 172.21.15.92:0/1449087553 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d8c00bb20 con 0x7f2d9406be80 2023-11-28T11:25:30.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.829+0000 7f2d92ffd700 1 -- 172.21.15.92:0/1449087553 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c00be90 con 0x7f2d9406be80 2023-11-28T11:25:30.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.830+0000 7f2d99fff700 1 -- 172.21.15.92:0/1449087553 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d9406be80 msgr2=0x7f2d940d0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:30.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.830+0000 7f2d99fff700 1 --2- 172.21.15.92:0/1449087553 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d9406be80 0x7f2d940d0fa0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c009d90 tx=0x7f2d8c00b330 comp rx=0 tx=0).stop 2023-11-28T11:25:30.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.830+0000 7f2d99fff700 1 -- 172.21.15.92:0/1449087553 shutdown_connections 2023-11-28T11:25:30.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.830+0000 7f2d99fff700 1 --2- 172.21.15.92:0/1449087553 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d9406be80 0x7f2d940d0fa0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:30.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.830+0000 7f2d99fff700 1 --2- 172.21.15.92:0/1449087553 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d940cd460 0x7f2d9406b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:30.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.830+0000 7f2d99fff700 1 -- 172.21.15.92:0/1449087553 >> 172.21.15.92:0/1449087553 conn(0x7f2d940719a0 msgr2=0x7f2d94071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:30.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.830+0000 7f2d99fff700 1 -- 172.21.15.92:0/1449087553 shutdown_connections 2023-11-28T11:25:30.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.830+0000 7f2d99fff700 1 -- 172.21.15.92:0/1449087553 wait complete. 2023-11-28T11:25:30.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.832+0000 7f2d99fff700 1 Processor -- start 2023-11-28T11:25:30.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.832+0000 7f2d99fff700 1 -- start start 2023-11-28T11:25:30.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.833+0000 7f2d99fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d9406be80 0x7f2d941542a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:30.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.833+0000 7f2d99fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d940b8790 0x7f2d940c8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:30.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.833+0000 7f2d99fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d940cd460 0x7f2d940c86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:30.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.833+0000 7f2d99fff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d940c9ee0 con 0x7f2d9406be80 2023-11-28T11:25:30.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.833+0000 7f2d99fff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d940c79a0 con 0x7f2d940b8790 2023-11-28T11:25:30.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.833+0000 7f2d99fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d940c7c80 con 0x7f2d940cd460 2023-11-28T11:25:30.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.833+0000 7f2d937fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d9406be80 0x7f2d941542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:30.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.833+0000 7f2d937fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d9406be80 0x7f2d941542a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51218/0 (socket says 172.21.15.92:51218) 2023-11-28T11:25:30.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.833+0000 7f2d937fe700 1 -- 172.21.15.92:0/84390649 learned_addr learned my addr 172.21.15.92:0/84390649 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:30.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.833+0000 7f2d8bfff700 1 --2- 172.21.15.92:0/84390649 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d940b8790 0x7f2d940c8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:30.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.833+0000 7f2d93fff700 1 --2- 172.21.15.92:0/84390649 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d940cd460 0x7f2d940c86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:30.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.834+0000 7f2d937fe700 1 -- 172.21.15.92:0/84390649 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d940cd460 msgr2=0x7f2d940c86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:30.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.834+0000 7f2d937fe700 1 --2- 172.21.15.92:0/84390649 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d940cd460 0x7f2d940c86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:30.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.834+0000 7f2d937fe700 1 -- 172.21.15.92:0/84390649 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d940b8790 msgr2=0x7f2d940c8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:30.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.834+0000 7f2d937fe700 1 --2- 172.21.15.92:0/84390649 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d940b8790 0x7f2d940c8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:30.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.834+0000 7f2d937fe700 1 -- 172.21.15.92:0/84390649 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d8c009a70 con 0x7f2d9406be80 2023-11-28T11:25:30.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.834+0000 7f2d937fe700 1 --2- 172.21.15.92:0/84390649 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d9406be80 0x7f2d941542a0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f2d8400c4d0 tx=0x7f2d8400c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:30.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.834+0000 7f2d90ff9700 1 -- 172.21.15.92:0/84390649 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d84005d70 con 0x7f2d9406be80 2023-11-28T11:25:30.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.834+0000 7f2d90ff9700 1 -- 172.21.15.92:0/84390649 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d840070d0 con 0x7f2d9406be80 2023-11-28T11:25:30.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.834+0000 7f2d90ff9700 1 -- 172.21.15.92:0/84390649 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d84007440 con 0x7f2d9406be80 2023-11-28T11:25:30.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.834+0000 7f2d99fff700 1 -- 172.21.15.92:0/84390649 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d940d14e0 con 0x7f2d9406be80 2023-11-28T11:25:30.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.834+0000 7f2d99fff700 1 -- 172.21.15.92:0/84390649 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d940b5800 con 0x7f2d9406be80 2023-11-28T11:25:30.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.835+0000 7f2d99fff700 1 -- 172.21.15.92:0/84390649 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d940d1670 con 0x7f2d9406be80 2023-11-28T11:25:30.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.836+0000 7f2d90ff9700 1 -- 172.21.15.92:0/84390649 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2d84019030 con 0x7f2d9406be80 2023-11-28T11:25:30.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.837+0000 7f2d90ff9700 1 --2- 172.21.15.92:0/84390649 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f2d740682c0 0x7f2d7406a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:30.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.837+0000 7f2d90ff9700 1 -- 172.21.15.92:0/84390649 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f2d84085130 con 0x7f2d9406be80 2023-11-28T11:25:30.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.837+0000 7f2d8bfff700 1 --2- 172.21.15.92:0/84390649 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f2d740682c0 0x7f2d7406a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:30.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.838+0000 7f2d8bfff700 1 --2- 172.21.15.92:0/84390649 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f2d740682c0 0x7f2d7406a770 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f2d940b2130 tx=0x7f2d7c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:30.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:30.841+0000 7f2d90ff9700 1 -- 172.21.15.92:0/84390649 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2d8404b5e0 con 0x7f2d9406be80 2023-11-28T11:25:31.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:31.032+0000 7f2d99fff700 1 -- 172.21.15.92:0/84390649 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2d94067880 con 0x7f2d9406be80 2023-11-28T11:25:31.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:31.798+0000 7f2d90ff9700 1 -- 172.21.15.92:0/84390649 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v72) v1 ==== 181+0+0 (secure 0 0 0) 0x7f2d840170d0 con 0x7f2d9406be80 2023-11-28T11:25:31.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:31.803+0000 7f2d99fff700 1 -- 172.21.15.92:0/84390649 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f2d740682c0 msgr2=0x7f2d7406a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:31.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:31.803+0000 7f2d99fff700 1 --2- 172.21.15.92:0/84390649 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f2d740682c0 0x7f2d7406a770 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f2d940b2130 tx=0x7f2d7c008040 comp rx=0 tx=0).stop 2023-11-28T11:25:31.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:31.803+0000 7f2d99fff700 1 -- 172.21.15.92:0/84390649 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d9406be80 msgr2=0x7f2d941542a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:31.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:31.803+0000 7f2d99fff700 1 --2- 172.21.15.92:0/84390649 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d9406be80 0x7f2d941542a0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f2d8400c4d0 tx=0x7f2d8400c7e0 comp rx=0 tx=0).stop 2023-11-28T11:25:31.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:31.803+0000 7f2d99fff700 1 -- 172.21.15.92:0/84390649 shutdown_connections 2023-11-28T11:25:31.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:31.803+0000 7f2d99fff700 1 --2- 172.21.15.92:0/84390649 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f2d940cd460 0x7f2d940c86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:31.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:31.803+0000 7f2d99fff700 1 --2- 172.21.15.92:0/84390649 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f2d740682c0 0x7f2d7406a770 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:31.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:31.803+0000 7f2d99fff700 1 --2- 172.21.15.92:0/84390649 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d9406be80 0x7f2d941542a0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:31.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:31.803+0000 7f2d99fff700 1 --2- 172.21.15.92:0/84390649 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f2d940b8790 0x7f2d940c8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:31.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:31.803+0000 7f2d99fff700 1 -- 172.21.15.92:0/84390649 >> 172.21.15.92:0/84390649 conn(0x7f2d940719a0 msgr2=0x7f2d9415b330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:31.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:31.803+0000 7f2d99fff700 1 -- 172.21.15.92:0/84390649 shutdown_connections 2023-11-28T11:25:31.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:31.803+0000 7f2d99fff700 1 -- 172.21.15.92:0/84390649 wait complete. 2023-11-28T11:25:31.805 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' removed 2023-11-28T11:25:31.826 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T11:25:32.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.033+0000 7efe263ff700 1 Processor -- start 2023-11-28T11:25:32.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.033+0000 7efe263ff700 1 -- start start 2023-11-28T11:25:32.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.034+0000 7efe263ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe200cd460 0x7efe2006b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:32.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.034+0000 7efe263ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7efe2006be80 0x7efe200d0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:32.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.034+0000 7efe263ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe200b5240 con 0x7efe200b8790 2023-11-28T11:25:32.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.034+0000 7efe263ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe2006c300 con 0x7efe2006be80 2023-11-28T11:25:32.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.034+0000 7efe263ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe200b5520 con 0x7efe200cd460 2023-11-28T11:25:32.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.034+0000 7efe1ffff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7efe200b8790 0x7efe200cfb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36188/0 (socket says 172.21.15.92:36188) 2023-11-28T11:25:32.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.034+0000 7efe1ffff700 1 -- 172.21.15.92:0/3621981311 learned_addr learned my addr 172.21.15.92:0/3621981311 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:25:32.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.034+0000 7efe2499c700 1 --2- 172.21.15.92:0/3621981311 >> v2:172.21.15.92:3300/0 conn(0x7efe2006be80 0x7efe200d0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:32.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.035+0000 7efe1f7fe700 1 --2- 172.21.15.92:0/3621981311 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe200cd460 0x7efe2006b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:32.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.035+0000 7efe1effd700 1 -- 172.21.15.92:0/3621981311 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2950619744 0 0) 0x7efe200b5240 con 0x7efe200b8790 2023-11-28T11:25:32.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.035+0000 7efe1effd700 1 -- 172.21.15.92:0/3621981311 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe080035b0 con 0x7efe200b8790 2023-11-28T11:25:32.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.035+0000 7efe2499c700 1 -- 172.21.15.92:0/3621981311 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe200cd460 msgr2=0x7efe2006b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:32.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.035+0000 7efe2499c700 1 --2- 172.21.15.92:0/3621981311 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe200cd460 0x7efe2006b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:32.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.035+0000 7efe2499c700 1 -- 172.21.15.92:0/3621981311 >> v1:172.21.15.92:6789/0 conn(0x7efe200b8790 legacy=0x7efe200cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:32.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.035+0000 7efe2499c700 1 -- 172.21.15.92:0/3621981311 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe200d0a80 con 0x7efe2006be80 2023-11-28T11:25:32.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.035+0000 7efe2499c700 1 --2- 172.21.15.92:0/3621981311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe2006be80 0x7efe200d0fa0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7efe18009d90 tx=0x7efe1800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea68eec8f489a3af server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:32.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.035+0000 7efe1effd700 1 -- 172.21.15.92:0/3621981311 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1800e070 con 0x7efe2006be80 2023-11-28T11:25:32.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.035+0000 7efe1effd700 1 -- 172.21.15.92:0/3621981311 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe1800bb20 con 0x7efe2006be80 2023-11-28T11:25:32.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.035+0000 7efe1effd700 1 -- 172.21.15.92:0/3621981311 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe18012430 con 0x7efe2006be80 2023-11-28T11:25:32.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.036+0000 7efe263ff700 1 -- 172.21.15.92:0/3621981311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe2006be80 msgr2=0x7efe200d0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:32.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.036+0000 7efe263ff700 1 --2- 172.21.15.92:0/3621981311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe2006be80 0x7efe200d0fa0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7efe18009d90 tx=0x7efe1800b330 comp rx=0 tx=0).stop 2023-11-28T11:25:32.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.036+0000 7efe263ff700 1 -- 172.21.15.92:0/3621981311 shutdown_connections 2023-11-28T11:25:32.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.036+0000 7efe263ff700 1 --2- 172.21.15.92:0/3621981311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe2006be80 0x7efe200d0fa0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:32.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.036+0000 7efe263ff700 1 --2- 172.21.15.92:0/3621981311 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe200cd460 0x7efe2006b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:32.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.036+0000 7efe263ff700 1 -- 172.21.15.92:0/3621981311 >> 172.21.15.92:0/3621981311 conn(0x7efe200719a0 msgr2=0x7efe20071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:32.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.036+0000 7efe263ff700 1 -- 172.21.15.92:0/3621981311 shutdown_connections 2023-11-28T11:25:32.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.036+0000 7efe263ff700 1 -- 172.21.15.92:0/3621981311 wait complete. 2023-11-28T11:25:32.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.038+0000 7efe263ff700 1 Processor -- start 2023-11-28T11:25:32.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.038+0000 7efe263ff700 1 -- start start 2023-11-28T11:25:32.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.039+0000 7efe263ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe2006be80 0x7efe200c5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:32.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.039+0000 7efe263ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe200b8790 0x7efe200b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:32.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.039+0000 7efe263ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe200cd460 0x7efe200b2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:32.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.039+0000 7efe263ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe200ca700 con 0x7efe200cd460 2023-11-28T11:25:32.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.039+0000 7efe263ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe200c9510 con 0x7efe2006be80 2023-11-28T11:25:32.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.039+0000 7efe263ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe200ca080 con 0x7efe200b8790 2023-11-28T11:25:32.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.039+0000 7efe2499c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe200cd460 0x7efe200b2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:32.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.039+0000 7efe2499c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe200cd460 0x7efe200b2010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51246/0 (socket says 172.21.15.92:51246) 2023-11-28T11:25:32.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.039+0000 7efe2499c700 1 -- 172.21.15.92:0/2466307689 learned_addr learned my addr 172.21.15.92:0/2466307689 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:32.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.039+0000 7efe1f7fe700 1 --2- 172.21.15.92:0/2466307689 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe200b8790 0x7efe200b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:32.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.039+0000 7efe1ffff700 1 --2- 172.21.15.92:0/2466307689 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe2006be80 0x7efe200c5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:32.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.039+0000 7efe2499c700 1 -- 172.21.15.92:0/2466307689 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe200b8790 msgr2=0x7efe200b1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:32.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.039+0000 7efe2499c700 1 --2- 172.21.15.92:0/2466307689 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe200b8790 0x7efe200b1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:32.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.039+0000 7efe2499c700 1 -- 172.21.15.92:0/2466307689 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe2006be80 msgr2=0x7efe200c5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:32.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.039+0000 7efe2499c700 1 --2- 172.21.15.92:0/2466307689 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe2006be80 0x7efe200c5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:32.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.039+0000 7efe2499c700 1 -- 172.21.15.92:0/2466307689 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe18009a70 con 0x7efe200cd460 2023-11-28T11:25:32.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.039+0000 7efe1f7fe700 1 --2- 172.21.15.92:0/2466307689 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe200b8790 0x7efe200b1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:25:32.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.039+0000 7efe1ffff700 1 --2- 172.21.15.92:0/2466307689 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe2006be80 0x7efe200c5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:25:32.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.040+0000 7efe2499c700 1 --2- 172.21.15.92:0/2466307689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe200cd460 0x7efe200b2010 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7efe18014040 tx=0x7efe18004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:32.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.040+0000 7efe1cff9700 1 -- 172.21.15.92:0/2466307689 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe18012430 con 0x7efe200cd460 2023-11-28T11:25:32.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.040+0000 7efe1cff9700 1 -- 172.21.15.92:0/2466307689 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe18020070 con 0x7efe200cd460 2023-11-28T11:25:32.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.040+0000 7efe263ff700 1 -- 172.21.15.92:0/2466307689 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe200d14e0 con 0x7efe200cd460 2023-11-28T11:25:32.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.040+0000 7efe1cff9700 1 -- 172.21.15.92:0/2466307689 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe18003ea0 con 0x7efe200cd460 2023-11-28T11:25:32.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.040+0000 7efe263ff700 1 -- 172.21.15.92:0/2466307689 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe200b5800 con 0x7efe200cd460 2023-11-28T11:25:32.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.041+0000 7efe263ff700 1 -- 172.21.15.92:0/2466307689 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe200d1670 con 0x7efe200cd460 2023-11-28T11:25:32.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.042+0000 7efe1cff9700 1 -- 172.21.15.92:0/2466307689 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7efe1802e070 con 0x7efe200cd460 2023-11-28T11:25:32.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.043+0000 7efe1cff9700 1 --2- 172.21.15.92:0/2466307689 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7efe0c0682f0 0x7efe0c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:32.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.043+0000 7efe1cff9700 1 -- 172.21.15.92:0/2466307689 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6204+0+0 (secure 0 0 0) 0x7efe18022070 con 0x7efe200cd460 2023-11-28T11:25:32.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.043+0000 7efe1ffff700 1 --2- 172.21.15.92:0/2466307689 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7efe0c0682f0 0x7efe0c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:32.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.044+0000 7efe1ffff700 1 --2- 172.21.15.92:0/2466307689 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7efe0c0682f0 0x7efe0c06a7a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7efe100011e0 tx=0x7efe10009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:32.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.047+0000 7efe1cff9700 1 -- 172.21.15.92:0/2466307689 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7efe180542d0 con 0x7efe200cd460 2023-11-28T11:25:32.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.239+0000 7efe263ff700 1 -- 172.21.15.92:0/2466307689 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7efe200c9ee0 con 0x7efe200cd460 2023-11-28T11:25:32.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.800+0000 7efe1cff9700 1 -- 172.21.15.92:0/2466307689 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v73) v1 ==== 169+0+0 (secure 0 0 0) 0x7efe200c9ee0 con 0x7efe200cd460 2023-11-28T11:25:32.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.804+0000 7efe263ff700 1 -- 172.21.15.92:0/2466307689 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7efe0c0682f0 msgr2=0x7efe0c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:32.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.804+0000 7efe263ff700 1 --2- 172.21.15.92:0/2466307689 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7efe0c0682f0 0x7efe0c06a7a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7efe100011e0 tx=0x7efe10009040 comp rx=0 tx=0).stop 2023-11-28T11:25:32.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.804+0000 7efe263ff700 1 -- 172.21.15.92:0/2466307689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe200cd460 msgr2=0x7efe200b2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:32.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.804+0000 7efe263ff700 1 --2- 172.21.15.92:0/2466307689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe200cd460 0x7efe200b2010 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7efe18014040 tx=0x7efe18004ad0 comp rx=0 tx=0).stop 2023-11-28T11:25:32.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.805+0000 7efe263ff700 1 -- 172.21.15.92:0/2466307689 shutdown_connections 2023-11-28T11:25:32.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.805+0000 7efe263ff700 1 --2- 172.21.15.92:0/2466307689 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7efe0c0682f0 0x7efe0c06a7a0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:32.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.805+0000 7efe263ff700 1 --2- 172.21.15.92:0/2466307689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe200cd460 0x7efe200b2010 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:32.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.805+0000 7efe263ff700 1 --2- 172.21.15.92:0/2466307689 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7efe2006be80 0x7efe200c5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:32.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.805+0000 7efe263ff700 1 --2- 172.21.15.92:0/2466307689 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7efe200b8790 0x7efe200b1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:32.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.805+0000 7efe263ff700 1 -- 172.21.15.92:0/2466307689 >> 172.21.15.92:0/2466307689 conn(0x7efe200719a0 msgr2=0x7efe200b7930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:32.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.805+0000 7efe263ff700 1 -- 172.21.15.92:0/2466307689 shutdown_connections 2023-11-28T11:25:32.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:32.805+0000 7efe263ff700 1 -- 172.21.15.92:0/2466307689 wait complete. 2023-11-28T11:25:32.808 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' removed 2023-11-28T11:25:32.828 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-28T11:25:33.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.028+0000 7fb3bd1ff700 1 Processor -- start 2023-11-28T11:25:33.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.028+0000 7fb3bd1ff700 1 -- start start 2023-11-28T11:25:33.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.029+0000 7fb3bd1ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb3b80cd460 0x7fb3b806c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:33.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.029+0000 7fb3bd1ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3b80d0d60 0x7fb3b80d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:33.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.029+0000 7fb3bd1ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3b80b88e0 con 0x7fb3b80b8510 2023-11-28T11:25:33.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.029+0000 7fb3bd1ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3b80cdbd0 con 0x7fb3b80cd460 2023-11-28T11:25:33.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.029+0000 7fb3bd1ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b80d02d0 con 0x7fb3b80d0d60 2023-11-28T11:25:33.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.030+0000 7fb3b6d9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb3b80b8510 0x7fb3b806b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36196/0 (socket says 172.21.15.92:36196) 2023-11-28T11:25:33.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.030+0000 7fb3b6d9d700 1 -- 172.21.15.92:0/3838884834 learned_addr learned my addr 172.21.15.92:0/3838884834 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:25:33.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.030+0000 7fb3b759e700 1 --2- 172.21.15.92:0/3838884834 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3b80d0d60 0x7fb3b80d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:33.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.030+0000 7fb3b659c700 1 --2- 172.21.15.92:0/3838884834 >> v2:172.21.15.169:3300/0 conn(0x7fb3b80cd460 0x7fb3b806c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:33.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.030+0000 7fb3b5d9b700 1 -- 172.21.15.92:0/3838884834 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3407704402 0 0) 0x7fb3b80b88e0 con 0x7fb3b80b8510 2023-11-28T11:25:33.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.030+0000 7fb3b5d9b700 1 -- 172.21.15.92:0/3838884834 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3a00035b0 con 0x7fb3b80b8510 2023-11-28T11:25:33.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.030+0000 7fb3b5d9b700 1 -- 172.21.15.92:0/3838884834 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3787802043 0 0) 0x7fb3a00035b0 con 0x7fb3b80b8510 2023-11-28T11:25:33.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.030+0000 7fb3b5d9b700 1 -- 172.21.15.92:0/3838884834 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3b80d0d60 msgr2=0x7fb3b80d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:33.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.030+0000 7fb3b5d9b700 1 --2- 172.21.15.92:0/3838884834 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3b80d0d60 0x7fb3b80d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:33.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.030+0000 7fb3b5d9b700 1 -- 172.21.15.92:0/3838884834 >> v2:172.21.15.169:3300/0 conn(0x7fb3b80cd460 msgr2=0x7fb3b806c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:33.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.030+0000 7fb3b5d9b700 1 --2- 172.21.15.92:0/3838884834 >> v2:172.21.15.169:3300/0 conn(0x7fb3b80cd460 0x7fb3b806c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:33.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.030+0000 7fb3b5d9b700 1 -- 172.21.15.92:0/3838884834 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b80cd8f0 con 0x7fb3b80b8510 2023-11-28T11:25:33.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.030+0000 7fb3b5d9b700 1 -- 172.21.15.92:0/3838884834 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532161587 0 0) 0x7fb3a8001670 con 0x7fb3b80b8510 2023-11-28T11:25:33.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.030+0000 7fb3b5d9b700 1 -- 172.21.15.92:0/3838884834 >> v1:172.21.15.92:6789/0 conn(0x7fb3b80b8510 legacy=0x7fb3b806b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:33.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.031+0000 7fb3b5d9b700 1 --2- 172.21.15.92:0/3838884834 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3a0003a30 0x7fb3a0003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:33.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.031+0000 7fb3b5d9b700 1 --2- 172.21.15.92:0/3838884834 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3a0004850 0x7fb3a0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:33.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.031+0000 7fb3b5d9b700 1 --2- 172.21.15.92:0/3838884834 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3b80d0d60 0x7fb3a0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:33.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.031+0000 7fb3b5d9b700 1 -- 172.21.15.92:0/3838884834 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b80cdbd0 con 0x7fb3a0004850 2023-11-28T11:25:33.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.031+0000 7fb3b5d9b700 1 -- 172.21.15.92:0/3838884834 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b80d02d0 con 0x7fb3a0003a30 2023-11-28T11:25:33.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.031+0000 7fb3b5d9b700 1 -- 172.21.15.92:0/3838884834 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a000ba10 con 0x7fb3b80d0d60 2023-11-28T11:25:33.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.031+0000 7fb3b659c700 1 --2- 172.21.15.92:0/3838884834 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3a0004850 0x7fb3a0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:33.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.031+0000 7fb3b6d9d700 1 --2- 172.21.15.92:0/3838884834 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3a0003a30 0x7fb3a0003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:33.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.031+0000 7fb3b759e700 1 --2- 172.21.15.92:0/3838884834 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3b80d0d60 0x7fb3a0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:33.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.031+0000 7fb3b659c700 1 -- 172.21.15.92:0/3838884834 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3b80d0d60 msgr2=0x7fb3a0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:33.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.031+0000 7fb3b659c700 1 --2- 172.21.15.92:0/3838884834 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3b80d0d60 0x7fb3a0005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:33.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.031+0000 7fb3b659c700 1 -- 172.21.15.92:0/3838884834 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3a0003a30 msgr2=0x7fb3a0003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:33.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.031+0000 7fb3b659c700 1 --2- 172.21.15.92:0/3838884834 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3a0003a30 0x7fb3a0003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:33.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.031+0000 7fb3b659c700 1 -- 172.21.15.92:0/3838884834 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a00035b0 con 0x7fb3a0004850 2023-11-28T11:25:33.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.031+0000 7fb3b6d9d700 1 --2- 172.21.15.92:0/3838884834 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3a0003a30 0x7fb3a0003e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:25:33.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.031+0000 7fb3b659c700 1 --2- 172.21.15.92:0/3838884834 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3a0004850 0x7fb3a0004cc0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fb3b80b1de0 tx=0x7fb3a0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=466d22c7b4597301 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:33.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.032+0000 7fb3b5d9b700 1 -- 172.21.15.92:0/3838884834 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac00e6c0 con 0x7fb3a0004850 2023-11-28T11:25:33.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.032+0000 7fb3b5d9b700 1 -- 172.21.15.92:0/3838884834 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3ac00e820 con 0x7fb3a0004850 2023-11-28T11:25:33.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.032+0000 7fb3b5d9b700 1 -- 172.21.15.92:0/3838884834 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac00ebb0 con 0x7fb3a0004850 2023-11-28T11:25:33.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.032+0000 7fb3bd1ff700 1 -- 172.21.15.92:0/3838884834 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3a0004850 msgr2=0x7fb3a0004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:33.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.032+0000 7fb3bd1ff700 1 --2- 172.21.15.92:0/3838884834 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3a0004850 0x7fb3a0004cc0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fb3b80b1de0 tx=0x7fb3a0005880 comp rx=0 tx=0).stop 2023-11-28T11:25:33.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.032+0000 7fb3b759e700 1 -- 172.21.15.92:0/3838884834 reap_dead start 2023-11-28T11:25:33.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.032+0000 7fb3bd1ff700 1 -- 172.21.15.92:0/3838884834 shutdown_connections 2023-11-28T11:25:33.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.032+0000 7fb3bd1ff700 1 -- 172.21.15.92:0/3838884834 >> 172.21.15.92:0/3838884834 conn(0x7fb3b80719a0 msgr2=0x7fb3b8071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:33.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.032+0000 7fb3bd1ff700 1 -- 172.21.15.92:0/3838884834 shutdown_connections 2023-11-28T11:25:33.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.032+0000 7fb3bd1ff700 1 -- 172.21.15.92:0/3838884834 wait complete. 2023-11-28T11:25:33.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.034+0000 7fb3bd1ff700 1 Processor -- start 2023-11-28T11:25:33.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.034+0000 7fb3bd1ff700 1 -- start start 2023-11-28T11:25:33.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.035+0000 7fb3bd1ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3a0004850 0x7fb3b80cc190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:33.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.035+0000 7fb3bd1ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3b81f3420 0x7fb3b80c8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:33.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.035+0000 7fb3bd1ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3b81544e0 0x7fb3b80c9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:33.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.035+0000 7fb3bd1ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b81f2750 con 0x7fb3b81544e0 2023-11-28T11:25:33.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.035+0000 7fb3bd1ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b80cc6d0 con 0x7fb3a0004850 2023-11-28T11:25:33.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.035+0000 7fb3bd1ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b815b6b0 con 0x7fb3b81f3420 2023-11-28T11:25:33.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.035+0000 7fb3b759e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3b81544e0 0x7fb3b80c9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:33.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.035+0000 7fb3b759e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3b81544e0 0x7fb3b80c9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51260/0 (socket says 172.21.15.92:51260) 2023-11-28T11:25:33.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.035+0000 7fb3b759e700 1 -- 172.21.15.92:0/774054002 learned_addr learned my addr 172.21.15.92:0/774054002 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:33.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.036+0000 7fb3b759e700 1 -- 172.21.15.92:0/774054002 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3b81f3420 msgr2=0x7fb3b80c8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:33.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.036+0000 7fb3b759e700 1 --2- 172.21.15.92:0/774054002 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3b81f3420 0x7fb3b80c8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:33.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.036+0000 7fb3b759e700 1 -- 172.21.15.92:0/774054002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3a0004850 msgr2=0x7fb3b80cc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:33.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.036+0000 7fb3b759e700 1 --2- 172.21.15.92:0/774054002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3a0004850 0x7fb3b80cc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:33.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.036+0000 7fb3b759e700 1 -- 172.21.15.92:0/774054002 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b80cfff0 con 0x7fb3b81544e0 2023-11-28T11:25:33.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.036+0000 7fb3b759e700 1 --2- 172.21.15.92:0/774054002 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3b81544e0 0x7fb3b80c9350 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fb3b80ce8c0 tx=0x7fb3b00114b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:33.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.036+0000 7fb39f7fe700 1 -- 172.21.15.92:0/774054002 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b000e070 con 0x7fb3b81544e0 2023-11-28T11:25:33.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.036+0000 7fb39f7fe700 1 -- 172.21.15.92:0/774054002 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3b0003cf0 con 0x7fb3b81544e0 2023-11-28T11:25:33.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.036+0000 7fb3bd1ff700 1 -- 172.21.15.92:0/774054002 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3b80cfd10 con 0x7fb3b81544e0 2023-11-28T11:25:33.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.036+0000 7fb39f7fe700 1 -- 172.21.15.92:0/774054002 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b0003ea0 con 0x7fb3b81544e0 2023-11-28T11:25:33.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.036+0000 7fb3bd1ff700 1 -- 172.21.15.92:0/774054002 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3b80d29c0 con 0x7fb3b81544e0 2023-11-28T11:25:33.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.037+0000 7fb3bd1ff700 1 -- 172.21.15.92:0/774054002 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3b8067880 con 0x7fb3b81544e0 2023-11-28T11:25:33.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.038+0000 7fb39f7fe700 1 -- 172.21.15.92:0/774054002 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb3b0011a10 con 0x7fb3b81544e0 2023-11-28T11:25:33.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.039+0000 7fb39f7fe700 1 --2- 172.21.15.92:0/774054002 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb3a40682f0 0x7fb3a406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:33.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.039+0000 7fb39f7fe700 1 -- 172.21.15.92:0/774054002 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fb3b0085fd0 con 0x7fb3b81544e0 2023-11-28T11:25:33.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.039+0000 7fb3b6d9d700 1 --2- 172.21.15.92:0/774054002 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb3a40682f0 0x7fb3a406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:33.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.040+0000 7fb3b6d9d700 1 --2- 172.21.15.92:0/774054002 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb3a40682f0 0x7fb3a406a7a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8001bb0 tx=0x7fb3a8000f40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:33.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.043+0000 7fb39f7fe700 1 -- 172.21.15.92:0/774054002 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb3b00529f0 con 0x7fb3b81544e0 2023-11-28T11:25:33.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.235+0000 7fb3bd1ff700 1 -- 172.21.15.92:0/774054002 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fb3b80c66e0 con 0x7fb3b81544e0 2023-11-28T11:25:33.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.239+0000 7fb39f7fe700 1 -- 172.21.15.92:0/774054002 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb3b0056010 con 0x7fb3b81544e0 2023-11-28T11:25:33.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.241+0000 7fb3bd1ff700 1 -- 172.21.15.92:0/774054002 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb3a40682f0 msgr2=0x7fb3a406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:33.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.241+0000 7fb3bd1ff700 1 --2- 172.21.15.92:0/774054002 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb3a40682f0 0x7fb3a406a7a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8001bb0 tx=0x7fb3a8000f40 comp rx=0 tx=0).stop 2023-11-28T11:25:33.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.241+0000 7fb3bd1ff700 1 -- 172.21.15.92:0/774054002 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3b81544e0 msgr2=0x7fb3b80c9350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:33.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.241+0000 7fb3bd1ff700 1 --2- 172.21.15.92:0/774054002 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3b81544e0 0x7fb3b80c9350 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fb3b80ce8c0 tx=0x7fb3b00114b0 comp rx=0 tx=0).stop 2023-11-28T11:25:33.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.242+0000 7fb3bd1ff700 1 -- 172.21.15.92:0/774054002 shutdown_connections 2023-11-28T11:25:33.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.242+0000 7fb3bd1ff700 1 --2- 172.21.15.92:0/774054002 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb3a40682f0 0x7fb3a406a7a0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:33.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.242+0000 7fb3bd1ff700 1 --2- 172.21.15.92:0/774054002 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3b81544e0 0x7fb3b80c9350 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:33.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.242+0000 7fb3bd1ff700 1 --2- 172.21.15.92:0/774054002 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb3a0004850 0x7fb3b80cc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:33.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.242+0000 7fb3bd1ff700 1 --2- 172.21.15.92:0/774054002 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb3b81f3420 0x7fb3b80c8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:33.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.242+0000 7fb3bd1ff700 1 -- 172.21.15.92:0/774054002 >> 172.21.15.92:0/774054002 conn(0x7fb3b80719a0 msgr2=0x7fb3b80d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:33.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.242+0000 7fb3bd1ff700 1 -- 172.21.15.92:0/774054002 shutdown_connections 2023-11-28T11:25:33.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.242+0000 7fb3bd1ff700 1 -- 172.21.15.92:0/774054002 wait complete. 2023-11-28T11:25:33.261 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2023-11-28T11:25:33.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.463+0000 7fa67d7ff700 1 Processor -- start 2023-11-28T11:25:33.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.463+0000 7fa67d7ff700 1 -- start start 2023-11-28T11:25:33.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.465+0000 7fa67d7ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa6780d2df0 0x7fa6780cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:33.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.465+0000 7fa67d7ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fa6780b1c60 0x7fa6780cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:33.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.465+0000 7fa67d7ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6780d1950 con 0x7fa6780b4fc0 2023-11-28T11:25:33.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.465+0000 7fa67d7ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6780d3d30 con 0x7fa6780d2df0 2023-11-28T11:25:33.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.465+0000 7fa67d7ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6780b2500 con 0x7fa6780b1c60 2023-11-28T11:25:33.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.465+0000 7fa676ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa6780d2df0 0x7fa6780cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:33.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.465+0000 7fa676ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa6780d2df0 0x7fa6780cd620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51272/0 (socket says 172.21.15.92:51272) 2023-11-28T11:25:33.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.465+0000 7fa676ffd700 1 -- 172.21.15.92:0/2537942607 learned_addr learned my addr 172.21.15.92:0/2537942607 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:33.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.466+0000 7fa6777fe700 1 --2- 172.21.15.92:0/2537942607 >> v2:172.21.15.169:3300/0 conn(0x7fa6780b1c60 0x7fa6780cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:33.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.466+0000 7fa676ffd700 1 -- 172.21.15.92:0/2537942607 >> v2:172.21.15.169:3300/0 conn(0x7fa6780b1c60 msgr2=0x7fa6780cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:33.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.466+0000 7fa676ffd700 1 --2- 172.21.15.92:0/2537942607 >> v2:172.21.15.169:3300/0 conn(0x7fa6780b1c60 0x7fa6780cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:33.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.466+0000 7fa676ffd700 1 -- 172.21.15.92:0/2537942607 >> v1:172.21.15.169:6789/0 conn(0x7fa6780b4fc0 legacy=0x7fa6780d31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:33.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.466+0000 7fa676ffd700 1 -- 172.21.15.92:0/2537942607 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6780d3a50 con 0x7fa6780d2df0 2023-11-28T11:25:33.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.466+0000 7fa676ffd700 1 --2- 172.21.15.92:0/2537942607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6780d2df0 0x7fa6780cd620 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fa67808c100 tx=0x7fa668009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc4bf7c68dc3a556 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:33.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.466+0000 7fa675ffb700 1 -- 172.21.15.92:0/2537942607 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa668004230 con 0x7fa6780d2df0 2023-11-28T11:25:33.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.466+0000 7fa675ffb700 1 -- 172.21.15.92:0/2537942607 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa668004390 con 0x7fa6780d2df0 2023-11-28T11:25:33.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.466+0000 7fa675ffb700 1 -- 172.21.15.92:0/2537942607 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6680036a0 con 0x7fa6780d2df0 2023-11-28T11:25:33.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.467+0000 7fa67d7ff700 1 -- 172.21.15.92:0/2537942607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6780d2df0 msgr2=0x7fa6780cd620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:33.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.467+0000 7fa67d7ff700 1 --2- 172.21.15.92:0/2537942607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6780d2df0 0x7fa6780cd620 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fa67808c100 tx=0x7fa668009bd0 comp rx=0 tx=0).stop 2023-11-28T11:25:33.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.467+0000 7fa67d7ff700 1 -- 172.21.15.92:0/2537942607 shutdown_connections 2023-11-28T11:25:33.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.467+0000 7fa67d7ff700 1 --2- 172.21.15.92:0/2537942607 >> v2:172.21.15.169:3300/0 conn(0x7fa6780b1c60 0x7fa6780cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:33.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.467+0000 7fa67d7ff700 1 --2- 172.21.15.92:0/2537942607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6780d2df0 0x7fa6780cd620 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:33.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.467+0000 7fa67d7ff700 1 -- 172.21.15.92:0/2537942607 >> 172.21.15.92:0/2537942607 conn(0x7fa6780719a0 msgr2=0x7fa678071da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:33.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.467+0000 7fa67d7ff700 1 -- 172.21.15.92:0/2537942607 shutdown_connections 2023-11-28T11:25:33.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.467+0000 7fa67d7ff700 1 -- 172.21.15.92:0/2537942607 wait complete. 2023-11-28T11:25:33.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.469+0000 7fa67d7ff700 1 Processor -- start 2023-11-28T11:25:33.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.469+0000 7fa67d7ff700 1 -- start start 2023-11-28T11:25:33.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.470+0000 7fa67d7ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6780b1c60 0x7fa678153e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:33.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.470+0000 7fa67d7ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6780b4fc0 0x7fa6780b6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:33.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.470+0000 7fa67d7ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6780d2df0 0x7fa6780b7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:33.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.470+0000 7fa67d7ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa678154350 con 0x7fa6780b1c60 2023-11-28T11:25:33.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.470+0000 7fa67d7ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa67815cde0 con 0x7fa6780b4fc0 2023-11-28T11:25:33.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.470+0000 7fa67d7ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa67815d0c0 con 0x7fa6780d2df0 2023-11-28T11:25:33.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.471+0000 7fa676ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6780b1c60 0x7fa678153e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:33.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.471+0000 7fa676ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6780b1c60 0x7fa678153e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51274/0 (socket says 172.21.15.92:51274) 2023-11-28T11:25:33.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.471+0000 7fa676ffd700 1 -- 172.21.15.92:0/1530421456 learned_addr learned my addr 172.21.15.92:0/1530421456 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:33.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.471+0000 7fa6767fc700 1 --2- 172.21.15.92:0/1530421456 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6780b4fc0 0x7fa6780b6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:33.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.471+0000 7fa676ffd700 1 -- 172.21.15.92:0/1530421456 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6780d2df0 msgr2=0x7fa6780b7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:33.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.471+0000 7fa6777fe700 1 --2- 172.21.15.92:0/1530421456 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6780d2df0 0x7fa6780b7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:33.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.471+0000 7fa676ffd700 1 --2- 172.21.15.92:0/1530421456 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6780d2df0 0x7fa6780b7510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:33.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.471+0000 7fa676ffd700 1 -- 172.21.15.92:0/1530421456 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6780b4fc0 msgr2=0x7fa6780b6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:33.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.471+0000 7fa676ffd700 1 --2- 172.21.15.92:0/1530421456 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6780b4fc0 0x7fa6780b6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:33.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.471+0000 7fa676ffd700 1 -- 172.21.15.92:0/1530421456 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa668009550 con 0x7fa6780b1c60 2023-11-28T11:25:33.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.471+0000 7fa6777fe700 1 --2- 172.21.15.92:0/1530421456 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6780d2df0 0x7fa6780b7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:25:33.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.471+0000 7fa676ffd700 1 --2- 172.21.15.92:0/1530421456 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6780b1c60 0x7fa678153e10 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fa668009130 tx=0x7fa668004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:33.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.472+0000 7fa65f7fe700 1 -- 172.21.15.92:0/1530421456 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa668011630 con 0x7fa6780b1c60 2023-11-28T11:25:33.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.472+0000 7fa65f7fe700 1 -- 172.21.15.92:0/1530421456 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa668011790 con 0x7fa6780b1c60 2023-11-28T11:25:33.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.472+0000 7fa65f7fe700 1 -- 172.21.15.92:0/1530421456 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa668011b40 con 0x7fa6780b1c60 2023-11-28T11:25:33.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.472+0000 7fa67d7ff700 1 -- 172.21.15.92:0/1530421456 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6780b88f0 con 0x7fa6780b1c60 2023-11-28T11:25:33.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.472+0000 7fa67d7ff700 1 -- 172.21.15.92:0/1530421456 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6780d15e0 con 0x7fa6780b1c60 2023-11-28T11:25:33.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.473+0000 7fa67d7ff700 1 -- 172.21.15.92:0/1530421456 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6780b21b0 con 0x7fa6780b1c60 2023-11-28T11:25:33.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.474+0000 7fa65f7fe700 1 -- 172.21.15.92:0/1530421456 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa668017020 con 0x7fa6780b1c60 2023-11-28T11:25:33.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.474+0000 7fa65f7fe700 1 --2- 172.21.15.92:0/1530421456 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa6640682f0 0x7fa66406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:33.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.475+0000 7fa65f7fe700 1 -- 172.21.15.92:0/1530421456 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fa668086ff0 con 0x7fa6780b1c60 2023-11-28T11:25:33.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.475+0000 7fa6767fc700 1 --2- 172.21.15.92:0/1530421456 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa6640682f0 0x7fa66406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:33.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.475+0000 7fa6767fc700 1 --2- 172.21.15.92:0/1530421456 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa6640682f0 0x7fa66406a7a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa67815fcb0 tx=0x7fa66c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:33.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.479+0000 7fa65f7fe700 1 -- 172.21.15.92:0/1530421456 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa6680b85e0 con 0x7fa6780b1c60 2023-11-28T11:25:33.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.742+0000 7fa67d7ff700 1 -- 172.21.15.92:0/1530421456 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) v1 -- 0x7fa678067170 con 0x7fa6780b1c60 2023-11-28T11:25:33.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.812+0000 7fa65f7fe700 1 -- 172.21.15.92:0/1530421456 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v250) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa668053940 con 0x7fa6780b1c60 2023-11-28T11:25:33.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.814+0000 7fa67d7ff700 1 -- 172.21.15.92:0/1530421456 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa6640682f0 msgr2=0x7fa66406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:33.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.814+0000 7fa67d7ff700 1 --2- 172.21.15.92:0/1530421456 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa6640682f0 0x7fa66406a7a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa67815fcb0 tx=0x7fa66c008040 comp rx=0 tx=0).stop 2023-11-28T11:25:33.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.814+0000 7fa67d7ff700 1 -- 172.21.15.92:0/1530421456 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6780b1c60 msgr2=0x7fa678153e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:33.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.814+0000 7fa67d7ff700 1 --2- 172.21.15.92:0/1530421456 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6780b1c60 0x7fa678153e10 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fa668009130 tx=0x7fa668004750 comp rx=0 tx=0).stop 2023-11-28T11:25:33.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.815+0000 7fa67d7ff700 1 -- 172.21.15.92:0/1530421456 shutdown_connections 2023-11-28T11:25:33.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.815+0000 7fa67d7ff700 1 --2- 172.21.15.92:0/1530421456 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6780d2df0 0x7fa6780b7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:33.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.815+0000 7fa67d7ff700 1 --2- 172.21.15.92:0/1530421456 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa6640682f0 0x7fa66406a7a0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:33.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.815+0000 7fa67d7ff700 1 --2- 172.21.15.92:0/1530421456 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6780b1c60 0x7fa678153e10 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:33.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.815+0000 7fa67d7ff700 1 --2- 172.21.15.92:0/1530421456 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6780b4fc0 0x7fa6780b6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:33.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.815+0000 7fa67d7ff700 1 -- 172.21.15.92:0/1530421456 >> 172.21.15.92:0/1530421456 conn(0x7fa6780719a0 msgr2=0x7fa6780b3100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:33.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.815+0000 7fa67d7ff700 1 -- 172.21.15.92:0/1530421456 shutdown_connections 2023-11-28T11:25:33.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:33.815+0000 7fa67d7ff700 1 -- 172.21.15.92:0/1530421456 wait complete. 2023-11-28T11:25:33.835 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ERROR 2023-11-28T11:25:33.835 INFO:tasks.cephfs_test_runner: 2023-11-28T11:25:33.835 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-28T11:25:33.835 INFO:tasks.cephfs_test_runner:ERROR: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T11:25:33.835 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared 2023-11-28T11:25:33.835 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-28T11:25:33.835 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-28T11:25:33.836 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_neesingh-rh_ceph_e67b67438c1e5e13734a7203ff9e57e30b8e4d32/qa/tasks/cephfs/test_forward_scrub.py", line 500, in test_health_status_after_backtrace_repair 2023-11-28T11:25:33.836 INFO:tasks.cephfs_test_runner: self.wait_until_true(lambda: self._check_for_presence_of_MDS_damage(), timeout=20) 2023-11-28T11:25:33.836 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_neesingh-rh_ceph_e67b67438c1e5e13734a7203ff9e57e30b8e4d32/qa/tasks/ceph_test_case.py", line 212, in wait_until_true 2023-11-28T11:25:33.836 INFO:tasks.cephfs_test_runner: raise TestTimeoutError("Timed out after {0}s and {1} retries".format(elapsed, retry_count)) 2023-11-28T11:25:33.836 INFO:tasks.cephfs_test_runner:tasks.ceph_test_case.TestTimeoutError: Timed out after 20s and 0 retries 2023-11-28T11:25:33.836 INFO:tasks.cephfs_test_runner: 2023-11-28T11:25:33.836 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-28T11:25:33.836 INFO:tasks.cephfs_test_runner:Ran 3 tests in 222.902s 2023-11-28T11:25:33.836 INFO:tasks.cephfs_test_runner: 2023-11-28T11:25:33.836 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2023-11-28T11:25:33.836 INFO:tasks.cephfs_test_runner: 2023-11-28T11:25:33.836 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-28T11:25:33.837 INFO:tasks.cephfs_test_runner:ERROR: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T11:25:33.837 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared 2023-11-28T11:25:33.837 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-28T11:25:33.837 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-28T11:25:33.837 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_neesingh-rh_ceph_e67b67438c1e5e13734a7203ff9e57e30b8e4d32/qa/tasks/cephfs/test_forward_scrub.py", line 500, in test_health_status_after_backtrace_repair 2023-11-28T11:25:33.837 INFO:tasks.cephfs_test_runner: self.wait_until_true(lambda: self._check_for_presence_of_MDS_damage(), timeout=20) 2023-11-28T11:25:33.837 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_neesingh-rh_ceph_e67b67438c1e5e13734a7203ff9e57e30b8e4d32/qa/tasks/ceph_test_case.py", line 212, in wait_until_true 2023-11-28T11:25:33.837 INFO:tasks.cephfs_test_runner: raise TestTimeoutError("Timed out after {0}s and {1} retries".format(elapsed, retry_count)) 2023-11-28T11:25:33.837 INFO:tasks.cephfs_test_runner:tasks.ceph_test_case.TestTimeoutError: Timed out after 20s and 0 retries 2023-11-28T11:25:33.837 INFO:tasks.cephfs_test_runner: 2023-11-28T11:25:33.838 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_neesingh-rh_ceph_e67b67438c1e5e13734a7203ff9e57e30b8e4d32/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T11:25:33.905 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=861142adecb64dd698fcaea137ec43ab Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_neesingh-rh_ceph_e67b67438c1e5e13734a7203ff9e57e30b8e4d32/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T11:25:33.907 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-11-28T11:25:33.916 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-11-28T11:25:33.924 INFO:tasks.kclient:Unmounting kernel clients... 2023-11-28T11:25:33.925 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-28T11:25:33.925 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:25:33.976 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348288k,nr_inodes=4087072,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275616k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:25:33.976 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-28T11:25:33.977 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:25:34.063 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348288k,nr_inodes=4087072,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275616k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:25:34.064 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-28T11:25:34.064 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:25:34.116 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348296k,nr_inodes=4087074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275620k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:25:34.116 DEBUG:teuthology.orchestra.run.smithi169:> set -ex 2023-11-28T11:25:34.116 DEBUG:teuthology.orchestra.run.smithi169:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:25:34.164 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348296k,nr_inodes=4087074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275620k,mode=700,uid=1000,gid=1285 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:25:34.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:25:34.165 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-11-28T11:25:34.212 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T11:25:34.212 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T11:25:34.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:25:34.214 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T11:25:34.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:25:34.285 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T11:25:34.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:25:34.357 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link delete ceph-brx 2023-11-28T11:25:34.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:25:34.481 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-11-28T11:25:34.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:25:34.532 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link delete ceph-brx 2023-11-28T11:25:34.606 INFO:teuthology.orchestra.run.smithi092.stderr:Cannot find device "ceph-brx" 2023-11-28T11:25:34.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:25:34.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:25:34.610 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2023-11-28T11:25:34.662 INFO:teuthology.orchestra.run.smithi169.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-28T11:25:34.662 INFO:teuthology.orchestra.run.smithi169.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-28T11:25:34.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:25:34.663 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-28T11:25:34.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:25:34.740 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-28T11:25:34.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:25:34.818 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link delete ceph-brx 2023-11-28T11:25:34.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:25:34.946 DEBUG:teuthology.orchestra.run.smithi169:> ip netns list 2023-11-28T11:25:34.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:25:34.995 DEBUG:teuthology.orchestra.run.smithi169:> sudo ip link delete ceph-brx 2023-11-28T11:25:35.070 INFO:teuthology.orchestra.run.smithi169.stderr:Cannot find device "ceph-brx" 2023-11-28T11:25:35.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:25:35.073 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-11-28T11:25:35.084 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-28T11:25:35.085 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:25:35.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.282+0000 7fd5049ff700 1 Processor -- start 2023-11-28T11:25:35.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.282+0000 7fd5049ff700 1 -- start start 2023-11-28T11:25:35.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.283+0000 7fd5049ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd500142e80 0x7fd50006a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:35.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.283+0000 7fd5049ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd50006a7b0 0x7fd5000c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:35.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.283+0000 7fd5049ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd5000ca430 0x7fd50006d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:35.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.284+0000 7fd5049ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5000c9380 con 0x7fd500142e80 2023-11-28T11:25:35.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.284+0000 7fd5049ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd50006ac00 con 0x7fd5000ca430 2023-11-28T11:25:35.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.284+0000 7fd5049ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd500159b10 con 0x7fd50006a7b0 2023-11-28T11:25:35.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.284+0000 7fd4fe59c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd500142e80 0x7fd50006a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:35.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.284+0000 7fd4fe59c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd500142e80 0x7fd50006a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51284/0 (socket says 172.21.15.92:51284) 2023-11-28T11:25:35.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.284+0000 7fd4fe59c700 1 -- 172.21.15.92:0/3512529434 learned_addr learned my addr 172.21.15.92:0/3512529434 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:35.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.284+0000 7fd4fed9d700 1 --2- 172.21.15.92:0/3512529434 >> v2:172.21.15.169:3300/0 conn(0x7fd5000ca430 0x7fd50006d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:35.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.284+0000 7fd4fdd9b700 1 --2- 172.21.15.92:0/3512529434 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd50006a7b0 0x7fd5000c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:35.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.284+0000 7fd4fe59c700 1 -- 172.21.15.92:0/3512529434 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd50006a7b0 msgr2=0x7fd5000c9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:35.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.284+0000 7fd4fe59c700 1 --2- 172.21.15.92:0/3512529434 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd50006a7b0 0x7fd5000c9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:35.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.284+0000 7fd4fe59c700 1 -- 172.21.15.92:0/3512529434 >> v2:172.21.15.169:3300/0 conn(0x7fd5000ca430 msgr2=0x7fd50006d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:35.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.284+0000 7fd4fe59c700 1 --2- 172.21.15.92:0/3512529434 >> v2:172.21.15.169:3300/0 conn(0x7fd5000ca430 0x7fd50006d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:35.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.284+0000 7fd4fe59c700 1 -- 172.21.15.92:0/3512529434 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5000ca890 con 0x7fd500142e80 2023-11-28T11:25:35.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.284+0000 7fd4fdd9b700 1 --2- 172.21.15.92:0/3512529434 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd50006a7b0 0x7fd5000c9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:25:35.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.284+0000 7fd4fed9d700 1 --2- 172.21.15.92:0/3512529434 >> v2:172.21.15.169:3300/0 conn(0x7fd5000ca430 0x7fd50006d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:25:35.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.284+0000 7fd4fe59c700 1 --2- 172.21.15.92:0/3512529434 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd500142e80 0x7fd50006a270 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fd50015d120 tx=0x7fd4f0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b843e8be03d7f69 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:35.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.285+0000 7fd4fd59a700 1 -- 172.21.15.92:0/3512529434 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f000b870 con 0x7fd500142e80 2023-11-28T11:25:35.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.285+0000 7fd4fd59a700 1 -- 172.21.15.92:0/3512529434 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4f000b9d0 con 0x7fd500142e80 2023-11-28T11:25:35.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.285+0000 7fd4fd59a700 1 -- 172.21.15.92:0/3512529434 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f000bd60 con 0x7fd500142e80 2023-11-28T11:25:35.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.285+0000 7fd5049ff700 1 -- 172.21.15.92:0/3512529434 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd500142e80 msgr2=0x7fd50006a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:35.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.285+0000 7fd5049ff700 1 --2- 172.21.15.92:0/3512529434 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd500142e80 0x7fd50006a270 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fd50015d120 tx=0x7fd4f0009bd0 comp rx=0 tx=0).stop 2023-11-28T11:25:35.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.285+0000 7fd5049ff700 1 -- 172.21.15.92:0/3512529434 shutdown_connections 2023-11-28T11:25:35.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.285+0000 7fd5049ff700 1 --2- 172.21.15.92:0/3512529434 >> v2:172.21.15.169:3300/0 conn(0x7fd5000ca430 0x7fd50006d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:35.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.285+0000 7fd5049ff700 1 --2- 172.21.15.92:0/3512529434 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd500142e80 0x7fd50006a270 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:35.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.285+0000 7fd5049ff700 1 --2- 172.21.15.92:0/3512529434 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd50006a7b0 0x7fd5000c9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:35.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.285+0000 7fd5049ff700 1 -- 172.21.15.92:0/3512529434 >> 172.21.15.92:0/3512529434 conn(0x7fd5000cb490 msgr2=0x7fd5000cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:35.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.285+0000 7fd5049ff700 1 -- 172.21.15.92:0/3512529434 shutdown_connections 2023-11-28T11:25:35.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.285+0000 7fd5049ff700 1 -- 172.21.15.92:0/3512529434 wait complete. 2023-11-28T11:25:35.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.287+0000 7fd5049ff700 1 Processor -- start 2023-11-28T11:25:35.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.287+0000 7fd5049ff700 1 -- start start 2023-11-28T11:25:35.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.288+0000 7fd5049ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd50006a7b0 0x7fd500140070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:35.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.288+0000 7fd5049ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5000ca430 0x7fd5000b1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:35.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.288+0000 7fd5049ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd500142e80 0x7fd5000c2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:35.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.288+0000 7fd5049ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5000b4830 con 0x7fd50006a7b0 2023-11-28T11:25:35.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.288+0000 7fd5049ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5000c22e0 con 0x7fd500142e80 2023-11-28T11:25:35.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.288+0000 7fd5049ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd500140a00 con 0x7fd5000ca430 2023-11-28T11:25:35.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.288+0000 7fd4fe59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd50006a7b0 0x7fd500140070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:35.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.288+0000 7fd4fe59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd50006a7b0 0x7fd500140070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51298/0 (socket says 172.21.15.92:51298) 2023-11-28T11:25:35.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.288+0000 7fd4fe59c700 1 -- 172.21.15.92:0/3596361691 learned_addr learned my addr 172.21.15.92:0/3596361691 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:35.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.288+0000 7fd4fdd9b700 1 --2- 172.21.15.92:0/3596361691 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5000ca430 0x7fd5000b1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:35.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.288+0000 7fd4fed9d700 1 --2- 172.21.15.92:0/3596361691 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd500142e80 0x7fd5000c2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:35.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.289+0000 7fd4fe59c700 1 -- 172.21.15.92:0/3596361691 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5000ca430 msgr2=0x7fd5000b1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:35.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.289+0000 7fd4fe59c700 1 --2- 172.21.15.92:0/3596361691 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5000ca430 0x7fd5000b1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:35.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.289+0000 7fd4fe59c700 1 -- 172.21.15.92:0/3596361691 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd500142e80 msgr2=0x7fd5000c2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:35.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.289+0000 7fd4fe59c700 1 --2- 172.21.15.92:0/3596361691 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd500142e80 0x7fd5000c2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:35.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.289+0000 7fd4fe59c700 1 -- 172.21.15.92:0/3596361691 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f0009550 con 0x7fd50006a7b0 2023-11-28T11:25:35.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.289+0000 7fd4fe59c700 1 --2- 172.21.15.92:0/3596361691 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd50006a7b0 0x7fd500140070 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0014040 tx=0x7fd4f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:35.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.289+0000 7fd4eeffd700 1 -- 172.21.15.92:0/3596361691 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f0004520 con 0x7fd50006a7b0 2023-11-28T11:25:35.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.289+0000 7fd4eeffd700 1 -- 172.21.15.92:0/3596361691 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4f001e070 con 0x7fd50006a7b0 2023-11-28T11:25:35.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.289+0000 7fd5049ff700 1 -- 172.21.15.92:0/3596361691 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5000cab00 con 0x7fd50006a7b0 2023-11-28T11:25:35.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.289+0000 7fd4eeffd700 1 -- 172.21.15.92:0/3596361691 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f00038e0 con 0x7fd50006a7b0 2023-11-28T11:25:35.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.289+0000 7fd5049ff700 1 -- 172.21.15.92:0/3596361691 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd500143250 con 0x7fd50006a7b0 2023-11-28T11:25:35.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.290+0000 7fd5049ff700 1 -- 172.21.15.92:0/3596361691 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5000cc5f0 con 0x7fd50006a7b0 2023-11-28T11:25:35.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.291+0000 7fd4eeffd700 1 -- 172.21.15.92:0/3596361691 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd4f000f050 con 0x7fd50006a7b0 2023-11-28T11:25:35.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.292+0000 7fd4eeffd700 1 --2- 172.21.15.92:0/3596361691 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd4e00682f0 0x7fd4e006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:35.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.292+0000 7fd4eeffd700 1 -- 172.21.15.92:0/3596361691 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fd4f0020070 con 0x7fd50006a7b0 2023-11-28T11:25:35.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.292+0000 7fd4fdd9b700 1 --2- 172.21.15.92:0/3596361691 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd4e00682f0 0x7fd4e006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:35.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.293+0000 7fd4fdd9b700 1 --2- 172.21.15.92:0/3596361691 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd4e00682f0 0x7fd4e006a7a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd5001f5990 tx=0x7fd4f4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:35.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.297+0000 7fd4eeffd700 1 -- 172.21.15.92:0/3596361691 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd4f0053ba0 con 0x7fd50006a7b0 2023-11-28T11:25:35.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.483+0000 7fd5049ff700 1 -- 172.21.15.92:0/3596361691 --> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd500067880 con 0x7fd4e00682f0 2023-11-28T11:25:35.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.486+0000 7fd4eeffd700 1 -- 172.21.15.92:0/3596361691 <== mgr.4103 v2:172.21.15.92:6832/115482 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72584 (secure 0 0 0) 0x7fd500067880 con 0x7fd4e00682f0 2023-11-28T11:25:35.488 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:25:35.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.488+0000 7fd5049ff700 1 -- 172.21.15.92:0/3596361691 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd4e00682f0 msgr2=0x7fd4e006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:35.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.488+0000 7fd5049ff700 1 --2- 172.21.15.92:0/3596361691 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd4e00682f0 0x7fd4e006a7a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd5001f5990 tx=0x7fd4f4008040 comp rx=0 tx=0).stop 2023-11-28T11:25:35.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.488+0000 7fd5049ff700 1 -- 172.21.15.92:0/3596361691 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd50006a7b0 msgr2=0x7fd500140070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:35.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.488+0000 7fd5049ff700 1 --2- 172.21.15.92:0/3596361691 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd50006a7b0 0x7fd500140070 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0014040 tx=0x7fd4f0004750 comp rx=0 tx=0).stop 2023-11-28T11:25:35.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.488+0000 7fd5049ff700 1 -- 172.21.15.92:0/3596361691 shutdown_connections 2023-11-28T11:25:35.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.488+0000 7fd5049ff700 1 --2- 172.21.15.92:0/3596361691 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd500142e80 0x7fd5000c2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:35.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.488+0000 7fd5049ff700 1 --2- 172.21.15.92:0/3596361691 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd4e00682f0 0x7fd4e006a7a0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:35.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.488+0000 7fd5049ff700 1 --2- 172.21.15.92:0/3596361691 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd50006a7b0 0x7fd500140070 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:35.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.488+0000 7fd5049ff700 1 --2- 172.21.15.92:0/3596361691 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd5000ca430 0x7fd5000b1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:35.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.488+0000 7fd5049ff700 1 -- 172.21.15.92:0/3596361691 >> 172.21.15.92:0/3596361691 conn(0x7fd5000cb490 msgr2=0x7fd50006dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:35.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.489+0000 7fd5049ff700 1 -- 172.21.15.92:0/3596361691 shutdown_connections 2023-11-28T11:25:35.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.489+0000 7fd5049ff700 1 -- 172.21.15.92:0/3596361691 wait complete. 2023-11-28T11:25:35.491 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-11-28T11:25:35.507 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":212,"stamp":"2023-11-28T11:25:33.921843+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":85920,"kb_used_data":7968,"kb_used_omap":0,"kb_used_meta":77888,"kb_avail":754888800,"statfs":{"total":773094113280,"available":773006131200,"internally_reserved":0,"allocated":8159232,"data_stored":4465040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":79757312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12572,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-6,"num_read_kb":-5,"num_write":-80,"num_write_kb":-40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001218"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":59,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-28T11:25:28.608290+0000","last_change":"2023-11-28T11:20:51.020016+0000","last_active":"2023-11-28T11:25:28.608290+0000","last_peered":"2023-11-28T11:25:28.608290+0000","last_clean":"2023-11-28T11:25:28.608290+0000","last_became_active":"2023-11-28T11:20:51.019492+0000","last_became_peered":"2023-11-28T11:20:51.019492+0000","last_unstale":"2023-11-28T11:25:28.608290+0000","last_undegraded":"2023-11-28T11:25:28.608290+0000","last_fullsized":"2023-11-28T11:25:28.608290+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:27.884538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":63,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-28T11:25:28.608862+0000","last_change":"2023-11-28T11:20:50.521908+0000","last_active":"2023-11-28T11:25:28.608862+0000","last_peered":"2023-11-28T11:25:28.608862+0000","last_clean":"2023-11-28T11:25:28.608862+0000","last_became_active":"2023-11-28T11:20:50.521059+0000","last_became_peered":"2023-11-28T11:20:50.521059+0000","last_unstale":"2023-11-28T11:25:28.608862+0000","last_undegraded":"2023-11-28T11:25:28.608862+0000","last_fullsized":"2023-11-28T11:25:28.608862+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:05.302498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":55,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-28T11:25:01.970276+0000","last_change":"2023-11-28T11:20:50.619004+0000","last_active":"2023-11-28T11:25:01.970276+0000","last_peered":"2023-11-28T11:25:01.970276+0000","last_clean":"2023-11-28T11:25:01.970276+0000","last_became_active":"2023-11-28T11:20:50.618315+0000","last_became_peered":"2023-11-28T11:20:50.618315+0000","last_unstale":"2023-11-28T11:25:01.970276+0000","last_undegraded":"2023-11-28T11:25:01.970276+0000","last_fullsized":"2023-11-28T11:25:01.970276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:27.884538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":63,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-28T11:25:28.608864+0000","last_change":"2023-11-28T11:20:50.522582+0000","last_active":"2023-11-28T11:25:28.608864+0000","last_peered":"2023-11-28T11:25:28.608864+0000","last_clean":"2023-11-28T11:25:28.608864+0000","last_became_active":"2023-11-28T11:20:50.522157+0000","last_became_peered":"2023-11-28T11:20:50.522157+0000","last_unstale":"2023-11-28T11:25:28.608864+0000","last_undegraded":"2023-11-28T11:25:28.608864+0000","last_fullsized":"2023-11-28T11:25:28.608864+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:27.884538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":209,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-28T11:24:46.975560+0000","last_change":"2023-11-28T11:20:43.294845+0000","last_active":"2023-11-28T11:24:46.975560+0000","last_peered":"2023-11-28T11:24:46.975560+0000","last_clean":"2023-11-28T11:24:46.975560+0000","last_became_active":"2023-11-28T11:20:43.294483+0000","last_became_peered":"2023-11-28T11:20:43.294483+0000","last_unstale":"2023-11-28T11:24:46.975560+0000","last_undegraded":"2023-11-28T11:24:46.975560+0000","last_fullsized":"2023-11-28T11:24:46.975560+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:42.270512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:42.270512+0000","last_clean_scrub_stamp":"2023-11-28T11:20:42.270512+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:17:03.376680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":56,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-28T11:24:51.976108+0000","last_change":"2023-11-28T11:20:50.526865+0000","last_active":"2023-11-28T11:24:51.976108+0000","last_peered":"2023-11-28T11:24:51.976108+0000","last_clean":"2023-11-28T11:24:51.976108+0000","last_became_active":"2023-11-28T11:20:50.526576+0000","last_became_peered":"2023-11-28T11:20:50.526576+0000","last_unstale":"2023-11-28T11:24:51.976108+0000","last_undegraded":"2023-11-28T11:24:51.976108+0000","last_fullsized":"2023-11-28T11:24:51.976108+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:09:18.460112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":56,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-28T11:25:01.977537+0000","last_change":"2023-11-28T11:20:51.020043+0000","last_active":"2023-11-28T11:25:01.977537+0000","last_peered":"2023-11-28T11:25:01.977537+0000","last_clean":"2023-11-28T11:25:01.977537+0000","last_became_active":"2023-11-28T11:20:51.019702+0000","last_became_peered":"2023-11-28T11:20:51.019702+0000","last_unstale":"2023-11-28T11:25:01.977537+0000","last_undegraded":"2023-11-28T11:25:01.977537+0000","last_fullsized":"2023-11-28T11:25:01.977537+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:26:20.246873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":59,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-28T11:25:28.607964+0000","last_change":"2023-11-28T11:20:50.524251+0000","last_active":"2023-11-28T11:25:28.607964+0000","last_peered":"2023-11-28T11:25:28.607964+0000","last_clean":"2023-11-28T11:25:28.607964+0000","last_became_active":"2023-11-28T11:20:50.524012+0000","last_became_peered":"2023-11-28T11:20:50.524012+0000","last_unstale":"2023-11-28T11:25:28.607964+0000","last_undegraded":"2023-11-28T11:25:28.607964+0000","last_fullsized":"2023-11-28T11:25:28.607964+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:05.302498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":55,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-28T11:25:11.979176+0000","last_change":"2023-11-28T11:20:50.524838+0000","last_active":"2023-11-28T11:25:11.979176+0000","last_peered":"2023-11-28T11:25:11.979176+0000","last_clean":"2023-11-28T11:25:11.979176+0000","last_became_active":"2023-11-28T11:20:50.524402+0000","last_became_peered":"2023-11-28T11:20:50.524402+0000","last_unstale":"2023-11-28T11:25:11.979176+0000","last_undegraded":"2023-11-28T11:25:11.979176+0000","last_fullsized":"2023-11-28T11:25:11.979176+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:22.699447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574910,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11032,"kb_used_data":1424,"kb_used_omap":0,"kb_used_meta":9600,"kb_avail":94360808,"statfs":{"total":96636764160,"available":96625467392,"internally_reserved":0,"allocated":1458176,"data_stored":996952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9830400},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.623,"15min":0.623},"min":{"1min":0.46899999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.89600000000000002,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.504},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.65000000000000002,"15min":0.64900000000000002},"min":{"1min":0.47399999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.89800000000000002,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.46200000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65100000000000002,"15min":0.64500000000000002},"min":{"1min":0.44800000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.90100000000000002,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.624},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.66700000000000004,"15min":0.67500000000000004},"min":{"1min":0.497,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.80000000000000004,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.53200000000000003}]},{"osd":2,"last update":"Tue Nov 28 11:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.72099999999999997,"15min":0.72299999999999998},"min":{"1min":0.57199999999999995,"5min":0.495,"15min":0.495},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.58599999999999997}]},{"osd":3,"last update":"Tue Nov 28 11:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.65500000000000003,"15min":0.66300000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.90900000000000003,"5min":1.004,"15min":1.004},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69999999999999996,"15min":0.70499999999999996},"min":{"1min":0.52300000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.83999999999999997,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.746}]},{"osd":4,"last update":"Tue Nov 28 11:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.61699999999999999,"15min":0.65700000000000003},"min":{"1min":0.36399999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.873,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.69099999999999995,"15min":0.69999999999999996},"min":{"1min":0.42899999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.97099999999999997,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.77000000000000002}]},{"osd":5,"last update":"Tue Nov 28 11:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.66900000000000004,"15min":0.68100000000000005},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.90100000000000002,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.68899999999999995,"15min":0.70999999999999996},"min":{"1min":0.40999999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.92500000000000004,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.70899999999999996}]},{"osd":6,"last update":"Tue Nov 28 11:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.74199999999999999,"15min":0.77200000000000002},"min":{"1min":0.45700000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.92300000000000004,"5min":1.633,"15min":1.633},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.70599999999999996,"15min":0.745},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.92200000000000004,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.84199999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574910,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9816,"kb_used_data":848,"kb_used_omap":0,"kb_used_meta":8960,"kb_avail":94362024,"statfs":{"total":96636764160,"available":96626712576,"internally_reserved":0,"allocated":868352,"data_stored":406698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9175040},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.63700000000000001,"15min":0.64100000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.167,"5min":1.409,"15min":1.409},"last":0.875},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.63800000000000001,"15min":0.63900000000000001},"min":{"1min":0.47099999999999997,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89700000000000002,"5min":1.244,"15min":1.244},"last":0.71699999999999997}]},{"osd":1,"last update":"Tue Nov 28 11:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.64200000000000002,"15min":0.63200000000000001},"min":{"1min":0.437,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.67100000000000004,"15min":0.68300000000000005},"min":{"1min":0.49199999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.02,"5min":1.038,"15min":1.038},"last":0.83899999999999997}]},{"osd":2,"last update":"Tue Nov 28 11:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.63,"15min":0.629},"min":{"1min":0.45400000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.73099999999999998,"15min":0.77300000000000002},"min":{"1min":0.51600000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.198,"5min":1.996,"15min":1.996},"last":0.73899999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.69999999999999996,"15min":0.70099999999999996},"min":{"1min":0.58299999999999996,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.66300000000000003,"15min":0.68100000000000005},"min":{"1min":0.501,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.92300000000000004,"5min":1.008,"15min":1.008},"last":0.75700000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.68899999999999995,"15min":0.72599999999999998},"min":{"1min":0.39800000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.1439999999999999,"5min":2.048,"15min":2.048},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.68999999999999995,"15min":0.71899999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.61699999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.65200000000000002,"15min":0.69099999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.86399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.754,"15min":0.80800000000000005},"min":{"1min":0.46000000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.99099999999999999,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":0.82899999999999996}]},{"osd":7,"last update":"Tue Nov 28 11:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.75,"15min":0.77400000000000002},"min":{"1min":0.56200000000000006,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.0009999999999999,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.76700000000000002,"15min":0.79400000000000004},"min":{"1min":0.58299999999999996,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.006,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.85799999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9964,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94361876,"statfs":{"total":96636764160,"available":96626561024,"internally_reserved":0,"allocated":888832,"data_stored":418996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.42499999999999999,"15min":0.45600000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.67900000000000005,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.45000000000000001,"15min":0.45900000000000002},"min":{"1min":0.33100000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.59799999999999998,"5min":0.999,"15min":0.999},"last":0.53800000000000003}]},{"osd":2,"last update":"Tue Nov 28 11:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.48499999999999999,"15min":0.51000000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.64600000000000002,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.47699999999999998,"15min":0.501},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.57299999999999995,"5min":1.002,"15min":1.002},"last":0.68700000000000006}]},{"osd":3,"last update":"Tue Nov 28 11:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.52800000000000002,"15min":0.55000000000000004},"min":{"1min":0.39100000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.627,"5min":1.198,"15min":1.198},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.51200000000000001,"15min":0.52100000000000002},"min":{"1min":0.38600000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.626,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.60499999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.73499999999999999,"15min":0.753},"min":{"1min":0.48799999999999999,"5min":0.438,"15min":0.438},"max":{"1min":0.91300000000000003,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.73399999999999999,"15min":0.753},"min":{"1min":0.498,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0780000000000001,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.71499999999999997}]},{"osd":5,"last update":"Tue Nov 28 11:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.746,"15min":0.76600000000000001},"min":{"1min":0.53500000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.98299999999999998,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.746,"15min":0.76800000000000002},"min":{"1min":0.50800000000000001,"5min":0.5,"15min":0.5},"max":{"1min":0.86299999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.72599999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.76300000000000001,"15min":0.78100000000000003},"min":{"1min":0.55300000000000005,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.99399999999999999,"5min":1.177,"15min":1.177},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.77900000000000003,"15min":0.80300000000000005},"min":{"1min":0.53000000000000003,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.97799999999999998,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.89800000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.76100000000000001},"min":{"1min":0.56200000000000006,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.92000000000000004,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.77900000000000003,"15min":0.79800000000000004},"min":{"1min":0.50800000000000001,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0089999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.80300000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574910,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":15188,"kb_used_data":1420,"kb_used_omap":0,"kb_used_meta":13760,"kb_avail":94356652,"statfs":{"total":96636764160,"available":96621211648,"internally_reserved":0,"allocated":1454080,"data_stored":996974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14090240},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.42299999999999999,"15min":0.44},"min":{"1min":0.27600000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.59199999999999997,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.53400000000000003,"15min":0.62},"min":{"1min":0.29799999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.875,"5min":3.8420000000000001,"15min":3.8420000000000001},"last":0.44600000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.48099999999999998,"15min":0.501},"min":{"1min":0.32200000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70899999999999996,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.57199999999999995,"15min":0.66000000000000003},"min":{"1min":0.33500000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.60899999999999999,"5min":3.8940000000000001,"15min":3.8940000000000001},"last":0.40100000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.54300000000000004,"15min":0.622},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.72699999999999998,"5min":3.681,"15min":3.681},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55600000000000005,"15min":0.58599999999999997},"min":{"1min":0.42499999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.67800000000000005,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.43099999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.80300000000000005,"15min":0.90300000000000002},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.96299999999999997,"5min":3.9790000000000001,"15min":3.9790000000000001},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.81599999999999995,"15min":0.92400000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.97899999999999998,"5min":3.5779999999999998,"15min":3.5779999999999998},"last":0.73199999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.77300000000000002,"15min":0.81399999999999995},"min":{"1min":0.56000000000000005,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.0069999999999999,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.874},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.82499999999999996,"15min":0.93999999999999995},"min":{"1min":0.50600000000000001,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1,"5min":3.9409999999999998,"15min":3.9409999999999998},"last":0.71999999999999997}]},{"osd":6,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.81999999999999995,"15min":0.92600000000000005},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0469999999999999,"5min":3.6160000000000001,"15min":3.6160000000000001},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.78400000000000003,"15min":0.83299999999999996},"min":{"1min":0.55200000000000005,"5min":0.501,"15min":0.501},"max":{"1min":0.94799999999999995,"5min":1.363,"15min":1.363},"last":0.70899999999999996}]},{"osd":7,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.88,"15min":1.0109999999999999},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.149,"5min":4.0099999999999998,"15min":4.0099999999999998},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.85699999999999998,"15min":0.97199999999999998},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.075,"5min":3.653,"15min":3.653},"last":0.79200000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574908,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9828,"kb_used_data":860,"kb_used_omap":0,"kb_used_meta":8960,"kb_avail":94362012,"statfs":{"total":96636764160,"available":96626700288,"internally_reserved":0,"allocated":880640,"data_stored":418396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9175040},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.46899999999999997,"15min":0.54400000000000004},"min":{"1min":0.255,"5min":0.219,"15min":0.219},"max":{"1min":0.56499999999999995,"5min":4.1109999999999998,"15min":4.1109999999999998},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.442,"5min":0.51700000000000002,"15min":0.58299999999999996},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.75800000000000001,"5min":3.6230000000000002,"15min":3.6230000000000002},"last":0.40200000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.61099999999999999,"15min":0.66600000000000004},"min":{"1min":0.40400000000000003,"5min":0.371,"15min":0.371},"max":{"1min":0.77700000000000002,"5min":3.6640000000000001,"15min":3.6640000000000001},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.502,"5min":0.58399999999999996,"15min":0.67500000000000004},"min":{"1min":0.35299999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.73199999999999998,"5min":4.0830000000000002,"15min":4.0830000000000002},"last":0.51500000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.55200000000000005,"15min":0.63600000000000001},"min":{"1min":0.35399999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.78200000000000003,"5min":4.1529999999999996,"15min":4.1529999999999996},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.55300000000000005,"15min":0.57999999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.68799999999999994,"5min":1.304,"15min":1.304},"last":0.46100000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.77500000000000002,"15min":0.80500000000000005},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0569999999999999,"5min":1.649,"15min":1.649},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.80300000000000005,"15min":0.89900000000000002},"min":{"1min":0.53300000000000003,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.98099999999999998,"5min":4.0060000000000002,"15min":4.0060000000000002},"last":0.54800000000000004}]},{"osd":5,"last update":"Tue Nov 28 11:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.81599999999999995,"15min":0.91000000000000003},"min":{"1min":0.504,"5min":0.5,"15min":0.5},"max":{"1min":0.94399999999999995,"5min":4.0529999999999999,"15min":4.0529999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.78400000000000003,"15min":0.80800000000000005},"min":{"1min":0.56200000000000006,"5min":0.497,"15min":0.497},"max":{"1min":1.125,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.71599999999999997}]},{"osd":6,"last update":"Tue Nov 28 11:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.79900000000000004,"15min":0.83599999999999997},"min":{"1min":0.51600000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.083,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.82899999999999996,"15min":0.85499999999999998},"min":{"1min":0.57399999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.155,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.63300000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.77100000000000002,"15min":0.80200000000000005},"min":{"1min":0.56999999999999995,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0109999999999999,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.80100000000000005,"15min":0.82699999999999996},"min":{"1min":0.52900000000000003,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1060000000000001,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.65500000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574909,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10008,"kb_used_data":848,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361832,"statfs":{"total":96636764160,"available":96626515968,"internally_reserved":0,"allocated":868352,"data_stored":406718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.39900000000000002,"15min":0.40400000000000003},"min":{"1min":0.28599999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.55000000000000004,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.45800000000000002,"15min":0.46100000000000002},"min":{"1min":0.30399999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.67800000000000005,"5min":1.157,"15min":1.157},"last":0.57699999999999996}]},{"osd":1,"last update":"Tue Nov 28 11:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.50700000000000001,"15min":0.57099999999999995},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.71099999999999997,"5min":4.8019999999999996,"15min":4.8019999999999996},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.52700000000000002,"15min":0.58499999999999996},"min":{"1min":0.33400000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.625,"5min":4.7539999999999996,"15min":4.7539999999999996},"last":0.51800000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.57699999999999996,"15min":0.64900000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.66900000000000004,"5min":4.8419999999999996,"15min":4.8419999999999996},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.502,"5min":0.57799999999999996,"15min":0.64600000000000002},"min":{"1min":0.34999999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.71199999999999997,"5min":4.7919999999999998,"15min":4.7919999999999998},"last":0.60299999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.76400000000000001,"15min":0.82899999999999996},"min":{"1min":0.49199999999999999,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.93500000000000005,"5min":4.8869999999999996,"15min":4.8869999999999996},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.80800000000000005,"15min":0.89000000000000001},"min":{"1min":0.54200000000000004,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.89900000000000002,"5min":5.2460000000000004,"15min":5.2460000000000004},"last":0.88400000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.81699999999999995,"15min":0.90000000000000002},"min":{"1min":0.54300000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.96799999999999997,"5min":5.0759999999999996,"15min":5.0759999999999996},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.79400000000000004,"15min":0.86299999999999999},"min":{"1min":0.51300000000000001,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.88700000000000001,"5min":5.3010000000000002,"15min":5.3010000000000002},"last":0.90300000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.80900000000000005,"15min":0.876},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.91400000000000003,"5min":5.3399999999999999,"15min":5.3399999999999999},"last":1.004},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.80100000000000005,"15min":0.86099999999999999},"min":{"1min":0.53800000000000003,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.91600000000000004,"5min":4.8330000000000002,"15min":4.8330000000000002},"last":0.96199999999999997}]},{"osd":7,"last update":"Tue Nov 28 11:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.80600000000000005,"15min":0.86899999999999999},"min":{"1min":0.52800000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.90500000000000003,"5min":4.8600000000000003,"15min":4.8600000000000003},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.83499999999999996,"15min":0.92300000000000004},"min":{"1min":0.56399999999999995,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.90100000000000002,"5min":5.1909999999999998,"15min":5.1909999999999998},"last":0.97899999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574909,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10196,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":9344,"kb_avail":94361644,"statfs":{"total":96636764160,"available":96626323456,"internally_reserved":0,"allocated":864256,"data_stored":406676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9568256},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.66900000000000004,"15min":0.68200000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.93999999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.67100000000000004,"15min":0.66400000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.97199999999999998,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.61299999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.67400000000000004,"15min":0.65200000000000002},"min":{"1min":0.40000000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.70599999999999996,"15min":0.70899999999999996},"min":{"1min":0.496,"5min":0.379,"15min":0.379},"max":{"1min":1.0700000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.499}]},{"osd":2,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.74099999999999999,"15min":0.745},"min":{"1min":0.47899999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.71399999999999997,"15min":0.70299999999999996},"min":{"1min":0.46600000000000003,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.64300000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.69099999999999995,"15min":0.68500000000000005},"min":{"1min":0.498,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.141,"5min":1.179,"15min":1.179},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.77300000000000002,"15min":0.78100000000000003},"min":{"1min":0.55000000000000004,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.298,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.73299999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.67500000000000004,"15min":0.68600000000000005},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.1140000000000001,"5min":1.161,"15min":1.161},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.66400000000000003,"15min":0.66600000000000004},"min":{"1min":0.437,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.27,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.71699999999999997}]},{"osd":6,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.71299999999999997,"15min":0.73099999999999998},"min":{"1min":0.438,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.359,"5min":1.514,"15min":1.514},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.72399999999999998,"15min":0.73599999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.70899999999999996}]},{"osd":7,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.70199999999999996,"15min":0.70299999999999996},"min":{"1min":0.47999999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.80100000000000005,"15min":0.83199999999999996},"min":{"1min":0.56200000000000006,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.3999999999999999,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":0.69599999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574909,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9888,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94361952,"statfs":{"total":96636764160,"available":96626638848,"internally_reserved":0,"allocated":876544,"data_stored":413630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.64700000000000002,"15min":0.63100000000000001},"min":{"1min":0.42299999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.85799999999999998,"5min":1.032,"15min":1.032},"last":0.745},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.68799999999999994,"15min":0.69599999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.91300000000000003,"5min":1.02,"15min":1.02},"last":0.65800000000000003}]},{"osd":1,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.67700000000000005,"15min":0.66200000000000003},"min":{"1min":0.48699999999999999,"5min":0.442,"15min":0.442},"max":{"1min":0.93600000000000005,"5min":1.355,"15min":1.355},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.68700000000000006,"15min":0.69399999999999995},"min":{"1min":0.48599999999999999,"5min":0.442,"15min":0.442},"max":{"1min":0.83399999999999996,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.70299999999999996}]},{"osd":2,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69399999999999995,"15min":0.69799999999999995},"min":{"1min":0.52200000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.93999999999999995,"5min":1.198,"15min":1.198},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.72199999999999998,"15min":0.73199999999999998},"min":{"1min":0.47799999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.75900000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.73399999999999999,"15min":0.751},"min":{"1min":0.50800000000000001,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.88700000000000001,"5min":1.141,"15min":1.141},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.72399999999999998,"15min":0.72599999999999998},"min":{"1min":0.55700000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.84399999999999997}]},{"osd":4,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.67800000000000005,"15min":0.71399999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.84099999999999997,"5min":2.5139999999999998,"15min":2.5139999999999998},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.70599999999999996,"15min":0.72999999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.98699999999999999,"5min":1.198,"15min":1.198},"last":0.85899999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.71899999999999997,"15min":0.73099999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.71899999999999997,"15min":0.72699999999999998},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.88900000000000001,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":0.77900000000000003}]},{"osd":7,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.77300000000000002,"15min":0.81299999999999994},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.96499999999999997,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.79100000000000004,"15min":0.81999999999999995},"min":{"1min":0.51600000000000001,"5min":0.496,"15min":0.496},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.88}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:25:35.509 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:25:35.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.708+0000 7f89215ff700 1 Processor -- start 2023-11-28T11:25:35.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.709+0000 7f89215ff700 1 -- start start 2023-11-28T11:25:35.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.710+0000 7f89215ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f891c0b8510 0x7f891c06b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:35.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.710+0000 7f89215ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f891c0cd460 0x7f891c06beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:35.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.710+0000 7f89215ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f891c0b88e0 con 0x7f891c06c3f0 2023-11-28T11:25:35.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.710+0000 7f89215ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f891c0cdbd0 con 0x7f891c0b8510 2023-11-28T11:25:35.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.710+0000 7f89215ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f891c0d02d0 con 0x7f891c0cd460 2023-11-28T11:25:35.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.710+0000 7f891affd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f891c0b8510 0x7f891c06b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:35.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.710+0000 7f891affd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f891c0b8510 0x7f891c06b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51300/0 (socket says 172.21.15.92:51300) 2023-11-28T11:25:35.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.710+0000 7f891affd700 1 -- 172.21.15.92:0/1626923806 learned_addr learned my addr 172.21.15.92:0/1626923806 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:35.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.710+0000 7f8913fff700 1 --2- 172.21.15.92:0/1626923806 >> v2:172.21.15.169:3300/0 conn(0x7f891c0cd460 0x7f891c06beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:35.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.711+0000 7f891affd700 1 -- 172.21.15.92:0/1626923806 >> v2:172.21.15.169:3300/0 conn(0x7f891c0cd460 msgr2=0x7f891c06beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:35.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.711+0000 7f891affd700 1 --2- 172.21.15.92:0/1626923806 >> v2:172.21.15.169:3300/0 conn(0x7f891c0cd460 0x7f891c06beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:35.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.711+0000 7f891affd700 1 -- 172.21.15.92:0/1626923806 >> v1:172.21.15.169:6789/0 conn(0x7f891c06c3f0 legacy=0x7f891c15cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:35.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.711+0000 7f891affd700 1 -- 172.21.15.92:0/1626923806 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f891c0cd8f0 con 0x7f891c0b8510 2023-11-28T11:25:35.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.711+0000 7f891affd700 1 --2- 172.21.15.92:0/1626923806 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f891c0b8510 0x7f891c06b970 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f890c009920 tx=0x7f890c009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92f2859114763130 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:35.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.711+0000 7f891a7fc700 1 -- 172.21.15.92:0/1626923806 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f890c011830 con 0x7f891c0b8510 2023-11-28T11:25:35.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.711+0000 7f891a7fc700 1 -- 172.21.15.92:0/1626923806 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f890c011990 con 0x7f891c0b8510 2023-11-28T11:25:35.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.711+0000 7f891a7fc700 1 -- 172.21.15.92:0/1626923806 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f890c011d20 con 0x7f891c0b8510 2023-11-28T11:25:35.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.712+0000 7f89215ff700 1 -- 172.21.15.92:0/1626923806 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f891c0b8510 msgr2=0x7f891c06b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:35.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.712+0000 7f89215ff700 1 --2- 172.21.15.92:0/1626923806 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f891c0b8510 0x7f891c06b970 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f890c009920 tx=0x7f890c009c30 comp rx=0 tx=0).stop 2023-11-28T11:25:35.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.712+0000 7f89215ff700 1 -- 172.21.15.92:0/1626923806 shutdown_connections 2023-11-28T11:25:35.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.712+0000 7f89215ff700 1 --2- 172.21.15.92:0/1626923806 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f891c0b8510 0x7f891c06b970 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:35.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.712+0000 7f89215ff700 1 --2- 172.21.15.92:0/1626923806 >> v2:172.21.15.169:3300/0 conn(0x7f891c0cd460 0x7f891c06beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:35.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.712+0000 7f89215ff700 1 -- 172.21.15.92:0/1626923806 >> 172.21.15.92:0/1626923806 conn(0x7f891c0719a0 msgr2=0x7f891c071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:35.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.712+0000 7f89215ff700 1 -- 172.21.15.92:0/1626923806 shutdown_connections 2023-11-28T11:25:35.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.712+0000 7f89215ff700 1 -- 172.21.15.92:0/1626923806 wait complete. 2023-11-28T11:25:35.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.714+0000 7f89215ff700 1 Processor -- start 2023-11-28T11:25:35.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.714+0000 7f89215ff700 1 -- start start 2023-11-28T11:25:35.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.716+0000 7f89215ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f891c06c3f0 0x7f891c1553f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:35.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.716+0000 7f89215ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f891c0b8510 0x7f891c1f2f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:35.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.716+0000 7f89215ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f891c0cd460 0x7f891c0aab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:35.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.716+0000 7f89215ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891c155930 con 0x7f891c0b8510 2023-11-28T11:25:35.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.716+0000 7f89215ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891c0a9840 con 0x7f891c0cd460 2023-11-28T11:25:35.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.716+0000 7f89215ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f891c1f34d0 con 0x7f891c06c3f0 2023-11-28T11:25:35.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.716+0000 7f8913fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f891c0b8510 0x7f891c1f2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:35.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.716+0000 7f8913fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f891c0b8510 0x7f891c1f2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51310/0 (socket says 172.21.15.92:51310) 2023-11-28T11:25:35.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.716+0000 7f8913fff700 1 -- 172.21.15.92:0/590642981 learned_addr learned my addr 172.21.15.92:0/590642981 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:35.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.716+0000 7f891affd700 1 --2- 172.21.15.92:0/590642981 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f891c06c3f0 0x7f891c1553f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:35.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.716+0000 7f891b7fe700 1 --2- 172.21.15.92:0/590642981 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f891c0cd460 0x7f891c0aab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:35.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.716+0000 7f8913fff700 1 -- 172.21.15.92:0/590642981 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f891c06c3f0 msgr2=0x7f891c1553f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:35.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.716+0000 7f8913fff700 1 --2- 172.21.15.92:0/590642981 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f891c06c3f0 0x7f891c1553f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:35.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.716+0000 7f8913fff700 1 -- 172.21.15.92:0/590642981 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f891c0cd460 msgr2=0x7f891c0aab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:35.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.716+0000 7f8913fff700 1 --2- 172.21.15.92:0/590642981 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f891c0cd460 0x7f891c0aab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:35.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.716+0000 7f8913fff700 1 -- 172.21.15.92:0/590642981 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f890c009550 con 0x7f891c0b8510 2023-11-28T11:25:35.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.716+0000 7f891b7fe700 1 --2- 172.21.15.92:0/590642981 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f891c0cd460 0x7f891c0aab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:25:35.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.717+0000 7f8913fff700 1 --2- 172.21.15.92:0/590642981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f891c0b8510 0x7f891c1f2f90 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f890400e870 tx=0x7f890400eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:35.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.717+0000 7f89137fe700 1 -- 172.21.15.92:0/590642981 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8904009530 con 0x7f891c0b8510 2023-11-28T11:25:35.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.717+0000 7f89137fe700 1 -- 172.21.15.92:0/590642981 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8904013680 con 0x7f891c0b8510 2023-11-28T11:25:35.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.717+0000 7f89215ff700 1 -- 172.21.15.92:0/590642981 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f891c0cfff0 con 0x7f891c0b8510 2023-11-28T11:25:35.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.717+0000 7f89137fe700 1 -- 172.21.15.92:0/590642981 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8904013a10 con 0x7f891c0b8510 2023-11-28T11:25:35.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.717+0000 7f89215ff700 1 -- 172.21.15.92:0/590642981 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f891c0cfd10 con 0x7f891c0b8510 2023-11-28T11:25:35.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.718+0000 7f89215ff700 1 -- 172.21.15.92:0/590642981 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f891c067880 con 0x7f891c0b8510 2023-11-28T11:25:35.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.719+0000 7f89137fe700 1 -- 172.21.15.92:0/590642981 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8904019030 con 0x7f891c0b8510 2023-11-28T11:25:35.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.720+0000 7f89137fe700 1 --2- 172.21.15.92:0/590642981 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f88fc0682f0 0x7f88fc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:35.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.720+0000 7f89137fe700 1 -- 172.21.15.92:0/590642981 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f89040858b0 con 0x7f891c0b8510 2023-11-28T11:25:35.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.720+0000 7f891affd700 1 --2- 172.21.15.92:0/590642981 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f88fc0682f0 0x7f88fc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:35.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.721+0000 7f891affd700 1 --2- 172.21.15.92:0/590642981 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f88fc0682f0 0x7f88fc06a7a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f890c000c00 tx=0x7f890c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:35.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.724+0000 7f89137fe700 1 -- 172.21.15.92:0/590642981 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8904052200 con 0x7f891c0b8510 2023-11-28T11:25:35.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.914+0000 7f89215ff700 1 -- 172.21.15.92:0/590642981 --> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f891c0d2940 con 0x7f88fc0682f0 2023-11-28T11:25:35.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.917+0000 7f89137fe700 1 -- 172.21.15.92:0/590642981 <== mgr.4103 v2:172.21.15.92:6832/115482 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72584 (secure 0 0 0) 0x7f891c0d2940 con 0x7f88fc0682f0 2023-11-28T11:25:35.919 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:25:35.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.919+0000 7f89215ff700 1 -- 172.21.15.92:0/590642981 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f88fc0682f0 msgr2=0x7f88fc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:35.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.919+0000 7f89215ff700 1 --2- 172.21.15.92:0/590642981 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f88fc0682f0 0x7f88fc06a7a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f890c000c00 tx=0x7f890c008030 comp rx=0 tx=0).stop 2023-11-28T11:25:35.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.919+0000 7f89215ff700 1 -- 172.21.15.92:0/590642981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f891c0b8510 msgr2=0x7f891c1f2f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:35.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.919+0000 7f89215ff700 1 --2- 172.21.15.92:0/590642981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f891c0b8510 0x7f891c1f2f90 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f890400e870 tx=0x7f890400eb80 comp rx=0 tx=0).stop 2023-11-28T11:25:35.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.920+0000 7f89215ff700 1 -- 172.21.15.92:0/590642981 shutdown_connections 2023-11-28T11:25:35.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.920+0000 7f89215ff700 1 --2- 172.21.15.92:0/590642981 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f891c0cd460 0x7f891c0aab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:35.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.920+0000 7f89215ff700 1 --2- 172.21.15.92:0/590642981 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f891c06c3f0 0x7f891c1553f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:35.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.920+0000 7f89215ff700 1 --2- 172.21.15.92:0/590642981 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f88fc0682f0 0x7f88fc06a7a0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:35.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.920+0000 7f89215ff700 1 --2- 172.21.15.92:0/590642981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f891c0b8510 0x7f891c1f2f90 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:35.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.920+0000 7f89215ff700 1 -- 172.21.15.92:0/590642981 >> 172.21.15.92:0/590642981 conn(0x7f891c0719a0 msgr2=0x7f891c0b5d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:35.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.920+0000 7f89215ff700 1 -- 172.21.15.92:0/590642981 shutdown_connections 2023-11-28T11:25:35.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:35.920+0000 7f89215ff700 1 -- 172.21.15.92:0/590642981 wait complete. 2023-11-28T11:25:35.924 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-11-28T11:25:35.939 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":212,"stamp":"2023-11-28T11:25:33.921843+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":85920,"kb_used_data":7968,"kb_used_omap":0,"kb_used_meta":77888,"kb_avail":754888800,"statfs":{"total":773094113280,"available":773006131200,"internally_reserved":0,"allocated":8159232,"data_stored":4465040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":79757312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12572,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-6,"num_read_kb":-5,"num_write":-80,"num_write_kb":-40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001218"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":59,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-28T11:25:28.608290+0000","last_change":"2023-11-28T11:20:51.020016+0000","last_active":"2023-11-28T11:25:28.608290+0000","last_peered":"2023-11-28T11:25:28.608290+0000","last_clean":"2023-11-28T11:25:28.608290+0000","last_became_active":"2023-11-28T11:20:51.019492+0000","last_became_peered":"2023-11-28T11:20:51.019492+0000","last_unstale":"2023-11-28T11:25:28.608290+0000","last_undegraded":"2023-11-28T11:25:28.608290+0000","last_fullsized":"2023-11-28T11:25:28.608290+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:27.884538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":63,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-28T11:25:28.608862+0000","last_change":"2023-11-28T11:20:50.521908+0000","last_active":"2023-11-28T11:25:28.608862+0000","last_peered":"2023-11-28T11:25:28.608862+0000","last_clean":"2023-11-28T11:25:28.608862+0000","last_became_active":"2023-11-28T11:20:50.521059+0000","last_became_peered":"2023-11-28T11:20:50.521059+0000","last_unstale":"2023-11-28T11:25:28.608862+0000","last_undegraded":"2023-11-28T11:25:28.608862+0000","last_fullsized":"2023-11-28T11:25:28.608862+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:05.302498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":55,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-28T11:25:01.970276+0000","last_change":"2023-11-28T11:20:50.619004+0000","last_active":"2023-11-28T11:25:01.970276+0000","last_peered":"2023-11-28T11:25:01.970276+0000","last_clean":"2023-11-28T11:25:01.970276+0000","last_became_active":"2023-11-28T11:20:50.618315+0000","last_became_peered":"2023-11-28T11:20:50.618315+0000","last_unstale":"2023-11-28T11:25:01.970276+0000","last_undegraded":"2023-11-28T11:25:01.970276+0000","last_fullsized":"2023-11-28T11:25:01.970276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:27.884538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":63,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-28T11:25:28.608864+0000","last_change":"2023-11-28T11:20:50.522582+0000","last_active":"2023-11-28T11:25:28.608864+0000","last_peered":"2023-11-28T11:25:28.608864+0000","last_clean":"2023-11-28T11:25:28.608864+0000","last_became_active":"2023-11-28T11:20:50.522157+0000","last_became_peered":"2023-11-28T11:20:50.522157+0000","last_unstale":"2023-11-28T11:25:28.608864+0000","last_undegraded":"2023-11-28T11:25:28.608864+0000","last_fullsized":"2023-11-28T11:25:28.608864+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:27.884538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":209,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-28T11:24:46.975560+0000","last_change":"2023-11-28T11:20:43.294845+0000","last_active":"2023-11-28T11:24:46.975560+0000","last_peered":"2023-11-28T11:24:46.975560+0000","last_clean":"2023-11-28T11:24:46.975560+0000","last_became_active":"2023-11-28T11:20:43.294483+0000","last_became_peered":"2023-11-28T11:20:43.294483+0000","last_unstale":"2023-11-28T11:24:46.975560+0000","last_undegraded":"2023-11-28T11:24:46.975560+0000","last_fullsized":"2023-11-28T11:24:46.975560+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:42.270512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:42.270512+0000","last_clean_scrub_stamp":"2023-11-28T11:20:42.270512+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:17:03.376680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":56,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-28T11:24:51.976108+0000","last_change":"2023-11-28T11:20:50.526865+0000","last_active":"2023-11-28T11:24:51.976108+0000","last_peered":"2023-11-28T11:24:51.976108+0000","last_clean":"2023-11-28T11:24:51.976108+0000","last_became_active":"2023-11-28T11:20:50.526576+0000","last_became_peered":"2023-11-28T11:20:50.526576+0000","last_unstale":"2023-11-28T11:24:51.976108+0000","last_undegraded":"2023-11-28T11:24:51.976108+0000","last_fullsized":"2023-11-28T11:24:51.976108+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:09:18.460112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":56,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-28T11:25:01.977537+0000","last_change":"2023-11-28T11:20:51.020043+0000","last_active":"2023-11-28T11:25:01.977537+0000","last_peered":"2023-11-28T11:25:01.977537+0000","last_clean":"2023-11-28T11:25:01.977537+0000","last_became_active":"2023-11-28T11:20:51.019702+0000","last_became_peered":"2023-11-28T11:20:51.019702+0000","last_unstale":"2023-11-28T11:25:01.977537+0000","last_undegraded":"2023-11-28T11:25:01.977537+0000","last_fullsized":"2023-11-28T11:25:01.977537+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:26:20.246873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":59,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-28T11:25:28.607964+0000","last_change":"2023-11-28T11:20:50.524251+0000","last_active":"2023-11-28T11:25:28.607964+0000","last_peered":"2023-11-28T11:25:28.607964+0000","last_clean":"2023-11-28T11:25:28.607964+0000","last_became_active":"2023-11-28T11:20:50.524012+0000","last_became_peered":"2023-11-28T11:20:50.524012+0000","last_unstale":"2023-11-28T11:25:28.607964+0000","last_undegraded":"2023-11-28T11:25:28.607964+0000","last_fullsized":"2023-11-28T11:25:28.607964+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:05.302498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":55,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-28T11:25:11.979176+0000","last_change":"2023-11-28T11:20:50.524838+0000","last_active":"2023-11-28T11:25:11.979176+0000","last_peered":"2023-11-28T11:25:11.979176+0000","last_clean":"2023-11-28T11:25:11.979176+0000","last_became_active":"2023-11-28T11:20:50.524402+0000","last_became_peered":"2023-11-28T11:20:50.524402+0000","last_unstale":"2023-11-28T11:25:11.979176+0000","last_undegraded":"2023-11-28T11:25:11.979176+0000","last_fullsized":"2023-11-28T11:25:11.979176+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:22.699447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574910,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11032,"kb_used_data":1424,"kb_used_omap":0,"kb_used_meta":9600,"kb_avail":94360808,"statfs":{"total":96636764160,"available":96625467392,"internally_reserved":0,"allocated":1458176,"data_stored":996952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9830400},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.623,"15min":0.623},"min":{"1min":0.46899999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.89600000000000002,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.504},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.65000000000000002,"15min":0.64900000000000002},"min":{"1min":0.47399999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.89800000000000002,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.46200000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65100000000000002,"15min":0.64500000000000002},"min":{"1min":0.44800000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.90100000000000002,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.624},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.66700000000000004,"15min":0.67500000000000004},"min":{"1min":0.497,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.80000000000000004,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.53200000000000003}]},{"osd":2,"last update":"Tue Nov 28 11:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.72099999999999997,"15min":0.72299999999999998},"min":{"1min":0.57199999999999995,"5min":0.495,"15min":0.495},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.58599999999999997}]},{"osd":3,"last update":"Tue Nov 28 11:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.65500000000000003,"15min":0.66300000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.90900000000000003,"5min":1.004,"15min":1.004},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69999999999999996,"15min":0.70499999999999996},"min":{"1min":0.52300000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.83999999999999997,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.746}]},{"osd":4,"last update":"Tue Nov 28 11:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.61699999999999999,"15min":0.65700000000000003},"min":{"1min":0.36399999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.873,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.69099999999999995,"15min":0.69999999999999996},"min":{"1min":0.42899999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.97099999999999997,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.77000000000000002}]},{"osd":5,"last update":"Tue Nov 28 11:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.66900000000000004,"15min":0.68100000000000005},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.90100000000000002,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.68899999999999995,"15min":0.70999999999999996},"min":{"1min":0.40999999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.92500000000000004,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.70899999999999996}]},{"osd":6,"last update":"Tue Nov 28 11:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.74199999999999999,"15min":0.77200000000000002},"min":{"1min":0.45700000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.92300000000000004,"5min":1.633,"15min":1.633},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.70599999999999996,"15min":0.745},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.92200000000000004,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.84199999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574910,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9816,"kb_used_data":848,"kb_used_omap":0,"kb_used_meta":8960,"kb_avail":94362024,"statfs":{"total":96636764160,"available":96626712576,"internally_reserved":0,"allocated":868352,"data_stored":406698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9175040},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.63700000000000001,"15min":0.64100000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.167,"5min":1.409,"15min":1.409},"last":0.875},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.63800000000000001,"15min":0.63900000000000001},"min":{"1min":0.47099999999999997,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89700000000000002,"5min":1.244,"15min":1.244},"last":0.71699999999999997}]},{"osd":1,"last update":"Tue Nov 28 11:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.64200000000000002,"15min":0.63200000000000001},"min":{"1min":0.437,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.67100000000000004,"15min":0.68300000000000005},"min":{"1min":0.49199999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.02,"5min":1.038,"15min":1.038},"last":0.83899999999999997}]},{"osd":2,"last update":"Tue Nov 28 11:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.63,"15min":0.629},"min":{"1min":0.45400000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.73099999999999998,"15min":0.77300000000000002},"min":{"1min":0.51600000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.198,"5min":1.996,"15min":1.996},"last":0.73899999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.69999999999999996,"15min":0.70099999999999996},"min":{"1min":0.58299999999999996,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.66300000000000003,"15min":0.68100000000000005},"min":{"1min":0.501,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.92300000000000004,"5min":1.008,"15min":1.008},"last":0.75700000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.68899999999999995,"15min":0.72599999999999998},"min":{"1min":0.39800000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.1439999999999999,"5min":2.048,"15min":2.048},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.68999999999999995,"15min":0.71899999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.61699999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.65200000000000002,"15min":0.69099999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.86399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.754,"15min":0.80800000000000005},"min":{"1min":0.46000000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.99099999999999999,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":0.82899999999999996}]},{"osd":7,"last update":"Tue Nov 28 11:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.75,"15min":0.77400000000000002},"min":{"1min":0.56200000000000006,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.0009999999999999,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.76700000000000002,"15min":0.79400000000000004},"min":{"1min":0.58299999999999996,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.006,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.85799999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9964,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94361876,"statfs":{"total":96636764160,"available":96626561024,"internally_reserved":0,"allocated":888832,"data_stored":418996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.42499999999999999,"15min":0.45600000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.67900000000000005,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.45000000000000001,"15min":0.45900000000000002},"min":{"1min":0.33100000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.59799999999999998,"5min":0.999,"15min":0.999},"last":0.53800000000000003}]},{"osd":2,"last update":"Tue Nov 28 11:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.48499999999999999,"15min":0.51000000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.64600000000000002,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.47699999999999998,"15min":0.501},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.57299999999999995,"5min":1.002,"15min":1.002},"last":0.68700000000000006}]},{"osd":3,"last update":"Tue Nov 28 11:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.52800000000000002,"15min":0.55000000000000004},"min":{"1min":0.39100000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.627,"5min":1.198,"15min":1.198},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.51200000000000001,"15min":0.52100000000000002},"min":{"1min":0.38600000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.626,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.60499999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.73499999999999999,"15min":0.753},"min":{"1min":0.48799999999999999,"5min":0.438,"15min":0.438},"max":{"1min":0.91300000000000003,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.73399999999999999,"15min":0.753},"min":{"1min":0.498,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0780000000000001,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.71499999999999997}]},{"osd":5,"last update":"Tue Nov 28 11:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.746,"15min":0.76600000000000001},"min":{"1min":0.53500000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.98299999999999998,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.746,"15min":0.76800000000000002},"min":{"1min":0.50800000000000001,"5min":0.5,"15min":0.5},"max":{"1min":0.86299999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.72599999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.76300000000000001,"15min":0.78100000000000003},"min":{"1min":0.55300000000000005,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.99399999999999999,"5min":1.177,"15min":1.177},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.77900000000000003,"15min":0.80300000000000005},"min":{"1min":0.53000000000000003,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.97799999999999998,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.89800000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.76100000000000001},"min":{"1min":0.56200000000000006,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.92000000000000004,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.77900000000000003,"15min":0.79800000000000004},"min":{"1min":0.50800000000000001,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0089999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.80300000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574910,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":15188,"kb_used_data":1420,"kb_used_omap":0,"kb_used_meta":13760,"kb_avail":94356652,"statfs":{"total":96636764160,"available":96621211648,"internally_reserved":0,"allocated":1454080,"data_stored":996974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14090240},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.42299999999999999,"15min":0.44},"min":{"1min":0.27600000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.59199999999999997,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.53400000000000003,"15min":0.62},"min":{"1min":0.29799999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.875,"5min":3.8420000000000001,"15min":3.8420000000000001},"last":0.44600000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.48099999999999998,"15min":0.501},"min":{"1min":0.32200000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70899999999999996,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.57199999999999995,"15min":0.66000000000000003},"min":{"1min":0.33500000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.60899999999999999,"5min":3.8940000000000001,"15min":3.8940000000000001},"last":0.40100000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.54300000000000004,"15min":0.622},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.72699999999999998,"5min":3.681,"15min":3.681},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55600000000000005,"15min":0.58599999999999997},"min":{"1min":0.42499999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.67800000000000005,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.43099999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.80300000000000005,"15min":0.90300000000000002},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.96299999999999997,"5min":3.9790000000000001,"15min":3.9790000000000001},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.81599999999999995,"15min":0.92400000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.97899999999999998,"5min":3.5779999999999998,"15min":3.5779999999999998},"last":0.73199999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.77300000000000002,"15min":0.81399999999999995},"min":{"1min":0.56000000000000005,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.0069999999999999,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.874},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.82499999999999996,"15min":0.93999999999999995},"min":{"1min":0.50600000000000001,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1,"5min":3.9409999999999998,"15min":3.9409999999999998},"last":0.71999999999999997}]},{"osd":6,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.81999999999999995,"15min":0.92600000000000005},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0469999999999999,"5min":3.6160000000000001,"15min":3.6160000000000001},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.78400000000000003,"15min":0.83299999999999996},"min":{"1min":0.55200000000000005,"5min":0.501,"15min":0.501},"max":{"1min":0.94799999999999995,"5min":1.363,"15min":1.363},"last":0.70899999999999996}]},{"osd":7,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.88,"15min":1.0109999999999999},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.149,"5min":4.0099999999999998,"15min":4.0099999999999998},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.85699999999999998,"15min":0.97199999999999998},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.075,"5min":3.653,"15min":3.653},"last":0.79200000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574908,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9828,"kb_used_data":860,"kb_used_omap":0,"kb_used_meta":8960,"kb_avail":94362012,"statfs":{"total":96636764160,"available":96626700288,"internally_reserved":0,"allocated":880640,"data_stored":418396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9175040},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.46899999999999997,"15min":0.54400000000000004},"min":{"1min":0.255,"5min":0.219,"15min":0.219},"max":{"1min":0.56499999999999995,"5min":4.1109999999999998,"15min":4.1109999999999998},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.442,"5min":0.51700000000000002,"15min":0.58299999999999996},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.75800000000000001,"5min":3.6230000000000002,"15min":3.6230000000000002},"last":0.40200000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.61099999999999999,"15min":0.66600000000000004},"min":{"1min":0.40400000000000003,"5min":0.371,"15min":0.371},"max":{"1min":0.77700000000000002,"5min":3.6640000000000001,"15min":3.6640000000000001},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.502,"5min":0.58399999999999996,"15min":0.67500000000000004},"min":{"1min":0.35299999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.73199999999999998,"5min":4.0830000000000002,"15min":4.0830000000000002},"last":0.51500000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.55200000000000005,"15min":0.63600000000000001},"min":{"1min":0.35399999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.78200000000000003,"5min":4.1529999999999996,"15min":4.1529999999999996},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.55300000000000005,"15min":0.57999999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.68799999999999994,"5min":1.304,"15min":1.304},"last":0.46100000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.77500000000000002,"15min":0.80500000000000005},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0569999999999999,"5min":1.649,"15min":1.649},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.80300000000000005,"15min":0.89900000000000002},"min":{"1min":0.53300000000000003,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.98099999999999998,"5min":4.0060000000000002,"15min":4.0060000000000002},"last":0.54800000000000004}]},{"osd":5,"last update":"Tue Nov 28 11:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.81599999999999995,"15min":0.91000000000000003},"min":{"1min":0.504,"5min":0.5,"15min":0.5},"max":{"1min":0.94399999999999995,"5min":4.0529999999999999,"15min":4.0529999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.78400000000000003,"15min":0.80800000000000005},"min":{"1min":0.56200000000000006,"5min":0.497,"15min":0.497},"max":{"1min":1.125,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.71599999999999997}]},{"osd":6,"last update":"Tue Nov 28 11:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.79900000000000004,"15min":0.83599999999999997},"min":{"1min":0.51600000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.083,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.82899999999999996,"15min":0.85499999999999998},"min":{"1min":0.57399999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.155,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.63300000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.77100000000000002,"15min":0.80200000000000005},"min":{"1min":0.56999999999999995,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0109999999999999,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.80100000000000005,"15min":0.82699999999999996},"min":{"1min":0.52900000000000003,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1060000000000001,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.65500000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574909,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10008,"kb_used_data":848,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361832,"statfs":{"total":96636764160,"available":96626515968,"internally_reserved":0,"allocated":868352,"data_stored":406718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.39900000000000002,"15min":0.40400000000000003},"min":{"1min":0.28599999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.55000000000000004,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.45800000000000002,"15min":0.46100000000000002},"min":{"1min":0.30399999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.67800000000000005,"5min":1.157,"15min":1.157},"last":0.57699999999999996}]},{"osd":1,"last update":"Tue Nov 28 11:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.50700000000000001,"15min":0.57099999999999995},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.71099999999999997,"5min":4.8019999999999996,"15min":4.8019999999999996},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.52700000000000002,"15min":0.58499999999999996},"min":{"1min":0.33400000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.625,"5min":4.7539999999999996,"15min":4.7539999999999996},"last":0.51800000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.57699999999999996,"15min":0.64900000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.66900000000000004,"5min":4.8419999999999996,"15min":4.8419999999999996},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.502,"5min":0.57799999999999996,"15min":0.64600000000000002},"min":{"1min":0.34999999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.71199999999999997,"5min":4.7919999999999998,"15min":4.7919999999999998},"last":0.60299999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.76400000000000001,"15min":0.82899999999999996},"min":{"1min":0.49199999999999999,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.93500000000000005,"5min":4.8869999999999996,"15min":4.8869999999999996},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.80800000000000005,"15min":0.89000000000000001},"min":{"1min":0.54200000000000004,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.89900000000000002,"5min":5.2460000000000004,"15min":5.2460000000000004},"last":0.88400000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.81699999999999995,"15min":0.90000000000000002},"min":{"1min":0.54300000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.96799999999999997,"5min":5.0759999999999996,"15min":5.0759999999999996},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.79400000000000004,"15min":0.86299999999999999},"min":{"1min":0.51300000000000001,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.88700000000000001,"5min":5.3010000000000002,"15min":5.3010000000000002},"last":0.90300000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.80900000000000005,"15min":0.876},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.91400000000000003,"5min":5.3399999999999999,"15min":5.3399999999999999},"last":1.004},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.80100000000000005,"15min":0.86099999999999999},"min":{"1min":0.53800000000000003,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.91600000000000004,"5min":4.8330000000000002,"15min":4.8330000000000002},"last":0.96199999999999997}]},{"osd":7,"last update":"Tue Nov 28 11:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.80600000000000005,"15min":0.86899999999999999},"min":{"1min":0.52800000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.90500000000000003,"5min":4.8600000000000003,"15min":4.8600000000000003},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.83499999999999996,"15min":0.92300000000000004},"min":{"1min":0.56399999999999995,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.90100000000000002,"5min":5.1909999999999998,"15min":5.1909999999999998},"last":0.97899999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574909,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10196,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":9344,"kb_avail":94361644,"statfs":{"total":96636764160,"available":96626323456,"internally_reserved":0,"allocated":864256,"data_stored":406676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9568256},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.66900000000000004,"15min":0.68200000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.93999999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.67100000000000004,"15min":0.66400000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.97199999999999998,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.61299999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.67400000000000004,"15min":0.65200000000000002},"min":{"1min":0.40000000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.70599999999999996,"15min":0.70899999999999996},"min":{"1min":0.496,"5min":0.379,"15min":0.379},"max":{"1min":1.0700000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.499}]},{"osd":2,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.74099999999999999,"15min":0.745},"min":{"1min":0.47899999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.71399999999999997,"15min":0.70299999999999996},"min":{"1min":0.46600000000000003,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.64300000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.69099999999999995,"15min":0.68500000000000005},"min":{"1min":0.498,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.141,"5min":1.179,"15min":1.179},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.77300000000000002,"15min":0.78100000000000003},"min":{"1min":0.55000000000000004,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.298,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.73299999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.67500000000000004,"15min":0.68600000000000005},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.1140000000000001,"5min":1.161,"15min":1.161},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.66400000000000003,"15min":0.66600000000000004},"min":{"1min":0.437,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.27,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.71699999999999997}]},{"osd":6,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.71299999999999997,"15min":0.73099999999999998},"min":{"1min":0.438,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.359,"5min":1.514,"15min":1.514},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.72399999999999998,"15min":0.73599999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.70899999999999996}]},{"osd":7,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.70199999999999996,"15min":0.70299999999999996},"min":{"1min":0.47999999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.80100000000000005,"15min":0.83199999999999996},"min":{"1min":0.56200000000000006,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.3999999999999999,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":0.69599999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574909,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9888,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94361952,"statfs":{"total":96636764160,"available":96626638848,"internally_reserved":0,"allocated":876544,"data_stored":413630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.64700000000000002,"15min":0.63100000000000001},"min":{"1min":0.42299999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.85799999999999998,"5min":1.032,"15min":1.032},"last":0.745},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.68799999999999994,"15min":0.69599999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.91300000000000003,"5min":1.02,"15min":1.02},"last":0.65800000000000003}]},{"osd":1,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.67700000000000005,"15min":0.66200000000000003},"min":{"1min":0.48699999999999999,"5min":0.442,"15min":0.442},"max":{"1min":0.93600000000000005,"5min":1.355,"15min":1.355},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.68700000000000006,"15min":0.69399999999999995},"min":{"1min":0.48599999999999999,"5min":0.442,"15min":0.442},"max":{"1min":0.83399999999999996,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.70299999999999996}]},{"osd":2,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69399999999999995,"15min":0.69799999999999995},"min":{"1min":0.52200000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.93999999999999995,"5min":1.198,"15min":1.198},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.72199999999999998,"15min":0.73199999999999998},"min":{"1min":0.47799999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.75900000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.73399999999999999,"15min":0.751},"min":{"1min":0.50800000000000001,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.88700000000000001,"5min":1.141,"15min":1.141},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.72399999999999998,"15min":0.72599999999999998},"min":{"1min":0.55700000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.84399999999999997}]},{"osd":4,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.67800000000000005,"15min":0.71399999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.84099999999999997,"5min":2.5139999999999998,"15min":2.5139999999999998},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.70599999999999996,"15min":0.72999999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.98699999999999999,"5min":1.198,"15min":1.198},"last":0.85899999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.71899999999999997,"15min":0.73099999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.71899999999999997,"15min":0.72699999999999998},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.88900000000000001,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":0.77900000000000003}]},{"osd":7,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.77300000000000002,"15min":0.81299999999999994},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.96499999999999997,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.79100000000000004,"15min":0.81999999999999995},"min":{"1min":0.51600000000000001,"5min":0.496,"15min":0.496},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.88}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:25:35.942 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-28T11:25:35.942 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:25:36.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.138+0000 7fb1a17ff700 1 Processor -- start 2023-11-28T11:25:36.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.138+0000 7fb1a17ff700 1 -- start start 2023-11-28T11:25:36.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.139+0000 7fb1a17ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb19c0b8510 0x7fb19c06b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:36.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.139+0000 7fb1a17ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb19c0b88e0 con 0x7fb19c0d0d60 2023-11-28T11:25:36.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.139+0000 7fb1a17ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb19c0cd8f0 con 0x7fb19c0cd460 2023-11-28T11:25:36.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.139+0000 7fb1a17ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb19c0cdbd0 con 0x7fb19c0b8510 2023-11-28T11:25:36.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.140+0000 7fb19b7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb19c0d0d60 0x7fb19c0d11a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36202/0 (socket says 172.21.15.92:36202) 2023-11-28T11:25:36.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.140+0000 7fb19b7fe700 1 -- 172.21.15.92:0/1779237459 learned_addr learned my addr 172.21.15.92:0/1779237459 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:25:36.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.140+0000 7fb19affd700 1 --2- 172.21.15.92:0/1779237459 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb19c0b8510 0x7fb19c06b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:36.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.140+0000 7fb199ffb700 1 -- 172.21.15.92:0/1779237459 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2626961309 0 0) 0x7fb19c0b88e0 con 0x7fb19c0d0d60 2023-11-28T11:25:36.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.140+0000 7fb199ffb700 1 -- 172.21.15.92:0/1779237459 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1840035b0 con 0x7fb19c0d0d60 2023-11-28T11:25:36.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.141+0000 7fb199ffb700 1 -- 172.21.15.92:0/1779237459 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 950661999 0 0) 0x7fb19c0cd8f0 con 0x7fb19c0cd460 2023-11-28T11:25:36.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.141+0000 7fb19affd700 1 -- 172.21.15.92:0/1779237459 >> v1:172.21.15.169:6789/0 conn(0x7fb19c0cd460 legacy=0x7fb19c06beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:36.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.141+0000 7fb19affd700 1 -- 172.21.15.92:0/1779237459 >> v1:172.21.15.92:6789/0 conn(0x7fb19c0d0d60 legacy=0x7fb19c0d11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:36.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.141+0000 7fb19affd700 1 -- 172.21.15.92:0/1779237459 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb19c0cfd10 con 0x7fb19c0b8510 2023-11-28T11:25:36.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.141+0000 7fb19affd700 1 --2- 172.21.15.92:0/1779237459 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb19c0b8510 0x7fb19c06b970 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb18c009870 tx=0x7fb18c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43bf07c65633dd6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:36.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.141+0000 7fb199ffb700 1 -- 172.21.15.92:0/1779237459 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb18c011830 con 0x7fb19c0b8510 2023-11-28T11:25:36.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.141+0000 7fb199ffb700 1 -- 172.21.15.92:0/1779237459 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb18c011990 con 0x7fb19c0b8510 2023-11-28T11:25:36.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.141+0000 7fb199ffb700 1 -- 172.21.15.92:0/1779237459 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb18c011d20 con 0x7fb19c0b8510 2023-11-28T11:25:36.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.142+0000 7fb1a17ff700 1 -- 172.21.15.92:0/1779237459 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb19c0b8510 msgr2=0x7fb19c06b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:36.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.142+0000 7fb1a17ff700 1 --2- 172.21.15.92:0/1779237459 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb19c0b8510 0x7fb19c06b970 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb18c009870 tx=0x7fb18c009b80 comp rx=0 tx=0).stop 2023-11-28T11:25:36.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.142+0000 7fb1a17ff700 1 -- 172.21.15.92:0/1779237459 shutdown_connections 2023-11-28T11:25:36.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.142+0000 7fb1a17ff700 1 --2- 172.21.15.92:0/1779237459 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb19c0b8510 0x7fb19c06b970 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:36.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.142+0000 7fb1a17ff700 1 -- 172.21.15.92:0/1779237459 >> 172.21.15.92:0/1779237459 conn(0x7fb19c0719a0 msgr2=0x7fb19c071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:36.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.142+0000 7fb1a17ff700 1 -- 172.21.15.92:0/1779237459 shutdown_connections 2023-11-28T11:25:36.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.142+0000 7fb1a17ff700 1 -- 172.21.15.92:0/1779237459 wait complete. 2023-11-28T11:25:36.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.144+0000 7fb1a17ff700 1 Processor -- start 2023-11-28T11:25:36.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.144+0000 7fb1a17ff700 1 -- start start 2023-11-28T11:25:36.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.145+0000 7fb1a17ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb19c0b8510 0x7fb19c14e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:36.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.145+0000 7fb1a17ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb19c0cd460 0x7fb19c06b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:36.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.145+0000 7fb1a17ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb19c0d0d60 0x7fb19c06bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:36.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.145+0000 7fb1a17ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19c0cbef0 con 0x7fb19c0d0d60 2023-11-28T11:25:36.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.145+0000 7fb1a17ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19c1f67f0 con 0x7fb19c0cd460 2023-11-28T11:25:36.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.145+0000 7fb1a17ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb19c0cc1b0 con 0x7fb19c0b8510 2023-11-28T11:25:36.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.145+0000 7fb19b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb19c0d0d60 0x7fb19c06bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:36.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.145+0000 7fb19b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb19c0d0d60 0x7fb19c06bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51322/0 (socket says 172.21.15.92:51322) 2023-11-28T11:25:36.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.145+0000 7fb19b7fe700 1 -- 172.21.15.92:0/4108381041 learned_addr learned my addr 172.21.15.92:0/4108381041 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:36.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.145+0000 7fb19affd700 1 --2- 172.21.15.92:0/4108381041 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb19c0b8510 0x7fb19c14e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:36.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.145+0000 7fb19a7fc700 1 --2- 172.21.15.92:0/4108381041 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb19c0cd460 0x7fb19c06b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:36.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.145+0000 7fb19b7fe700 1 -- 172.21.15.92:0/4108381041 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb19c0b8510 msgr2=0x7fb19c14e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:36.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.145+0000 7fb19b7fe700 1 --2- 172.21.15.92:0/4108381041 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb19c0b8510 0x7fb19c14e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:36.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.145+0000 7fb19b7fe700 1 -- 172.21.15.92:0/4108381041 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb19c0cd460 msgr2=0x7fb19c06b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:36.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.145+0000 7fb19b7fe700 1 --2- 172.21.15.92:0/4108381041 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb19c0cd460 0x7fb19c06b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:36.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.145+0000 7fb19b7fe700 1 -- 172.21.15.92:0/4108381041 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb18c0094a0 con 0x7fb19c0d0d60 2023-11-28T11:25:36.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.145+0000 7fb19affd700 1 --2- 172.21.15.92:0/4108381041 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb19c0b8510 0x7fb19c14e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:25:36.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.145+0000 7fb19a7fc700 1 --2- 172.21.15.92:0/4108381041 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb19c0cd460 0x7fb19c06b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:25:36.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.146+0000 7fb19b7fe700 1 --2- 172.21.15.92:0/4108381041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb19c0d0d60 0x7fb19c06bf20 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fb19400caa0 tx=0x7fb19400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:36.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.146+0000 7fb1837fe700 1 -- 172.21.15.92:0/4108381041 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19400aa80 con 0x7fb19c0d0d60 2023-11-28T11:25:36.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.146+0000 7fb1837fe700 1 -- 172.21.15.92:0/4108381041 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb19400abe0 con 0x7fb19c0d0d60 2023-11-28T11:25:36.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.146+0000 7fb1a17ff700 1 -- 172.21.15.92:0/4108381041 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb19c0d29c0 con 0x7fb19c0d0d60 2023-11-28T11:25:36.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.146+0000 7fb1837fe700 1 -- 172.21.15.92:0/4108381041 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb194012400 con 0x7fb19c0d0d60 2023-11-28T11:25:36.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.146+0000 7fb1a17ff700 1 -- 172.21.15.92:0/4108381041 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb19c0cfff0 con 0x7fb19c0d0d60 2023-11-28T11:25:36.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.147+0000 7fb1a17ff700 1 -- 172.21.15.92:0/4108381041 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb19c0d2b50 con 0x7fb19c0d0d60 2023-11-28T11:25:36.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.148+0000 7fb1837fe700 1 -- 172.21.15.92:0/4108381041 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb194019030 con 0x7fb19c0d0d60 2023-11-28T11:25:36.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.149+0000 7fb1837fe700 1 --2- 172.21.15.92:0/4108381041 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb1880682f0 0x7fb18806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:36.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.149+0000 7fb1837fe700 1 -- 172.21.15.92:0/4108381041 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fb1940856d0 con 0x7fb19c0d0d60 2023-11-28T11:25:36.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.149+0000 7fb19affd700 1 --2- 172.21.15.92:0/4108381041 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb1880682f0 0x7fb18806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:36.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.150+0000 7fb19affd700 1 --2- 172.21.15.92:0/4108381041 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb1880682f0 0x7fb18806a7a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb18c000c00 tx=0x7fb18c01c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:36.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.153+0000 7fb1837fe700 1 -- 172.21.15.92:0/4108381041 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb1940520f0 con 0x7fb19c0d0d60 2023-11-28T11:25:36.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.342+0000 7fb1a17ff700 1 -- 172.21.15.92:0/4108381041 --> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb19c067880 con 0x7fb1880682f0 2023-11-28T11:25:36.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.345+0000 7fb1837fe700 1 -- 172.21.15.92:0/4108381041 <== mgr.4103 v2:172.21.15.92:6832/115482 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72584 (secure 0 0 0) 0x7fb19c067880 con 0x7fb1880682f0 2023-11-28T11:25:36.348 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:25:36.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.347+0000 7fb1a17ff700 1 -- 172.21.15.92:0/4108381041 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb1880682f0 msgr2=0x7fb18806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:36.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.347+0000 7fb1a17ff700 1 --2- 172.21.15.92:0/4108381041 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb1880682f0 0x7fb18806a7a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb18c000c00 tx=0x7fb18c01c040 comp rx=0 tx=0).stop 2023-11-28T11:25:36.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.347+0000 7fb1a17ff700 1 -- 172.21.15.92:0/4108381041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb19c0d0d60 msgr2=0x7fb19c06bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:36.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.347+0000 7fb1a17ff700 1 --2- 172.21.15.92:0/4108381041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb19c0d0d60 0x7fb19c06bf20 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fb19400caa0 tx=0x7fb19400ce60 comp rx=0 tx=0).stop 2023-11-28T11:25:36.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.347+0000 7fb1a17ff700 1 -- 172.21.15.92:0/4108381041 shutdown_connections 2023-11-28T11:25:36.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.347+0000 7fb1a17ff700 1 --2- 172.21.15.92:0/4108381041 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb1880682f0 0x7fb18806a7a0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:36.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.347+0000 7fb1a17ff700 1 --2- 172.21.15.92:0/4108381041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb19c0d0d60 0x7fb19c06bf20 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:36.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.347+0000 7fb1a17ff700 1 --2- 172.21.15.92:0/4108381041 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb19c0b8510 0x7fb19c14e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:36.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.347+0000 7fb1a17ff700 1 --2- 172.21.15.92:0/4108381041 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb19c0cd460 0x7fb19c06b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:36.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.347+0000 7fb1a17ff700 1 -- 172.21.15.92:0/4108381041 >> 172.21.15.92:0/4108381041 conn(0x7fb19c0719a0 msgr2=0x7fb19c0b29f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:36.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.348+0000 7fb1a17ff700 1 -- 172.21.15.92:0/4108381041 shutdown_connections 2023-11-28T11:25:36.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.348+0000 7fb1a17ff700 1 -- 172.21.15.92:0/4108381041 wait complete. 2023-11-28T11:25:36.354 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-11-28T11:25:36.367 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":213,"stamp":"2023-11-28T11:25:35.922146+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":85920,"kb_used_data":7968,"kb_used_omap":0,"kb_used_meta":77888,"kb_avail":754888800,"statfs":{"total":773094113280,"available":773006131200,"internally_reserved":0,"allocated":8159232,"data_stored":4465040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":79757312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12572,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-6,"num_read_kb":-5,"num_write":-80,"num_write_kb":-40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.316071"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":59,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-28T11:25:28.608290+0000","last_change":"2023-11-28T11:20:51.020016+0000","last_active":"2023-11-28T11:25:28.608290+0000","last_peered":"2023-11-28T11:25:28.608290+0000","last_clean":"2023-11-28T11:25:28.608290+0000","last_became_active":"2023-11-28T11:20:51.019492+0000","last_became_peered":"2023-11-28T11:20:51.019492+0000","last_unstale":"2023-11-28T11:25:28.608290+0000","last_undegraded":"2023-11-28T11:25:28.608290+0000","last_fullsized":"2023-11-28T11:25:28.608290+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:27.884538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":63,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-28T11:25:28.608862+0000","last_change":"2023-11-28T11:20:50.521908+0000","last_active":"2023-11-28T11:25:28.608862+0000","last_peered":"2023-11-28T11:25:28.608862+0000","last_clean":"2023-11-28T11:25:28.608862+0000","last_became_active":"2023-11-28T11:20:50.521059+0000","last_became_peered":"2023-11-28T11:20:50.521059+0000","last_unstale":"2023-11-28T11:25:28.608862+0000","last_undegraded":"2023-11-28T11:25:28.608862+0000","last_fullsized":"2023-11-28T11:25:28.608862+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:05.302498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":55,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-28T11:25:01.970276+0000","last_change":"2023-11-28T11:20:50.619004+0000","last_active":"2023-11-28T11:25:01.970276+0000","last_peered":"2023-11-28T11:25:01.970276+0000","last_clean":"2023-11-28T11:25:01.970276+0000","last_became_active":"2023-11-28T11:20:50.618315+0000","last_became_peered":"2023-11-28T11:20:50.618315+0000","last_unstale":"2023-11-28T11:25:01.970276+0000","last_undegraded":"2023-11-28T11:25:01.970276+0000","last_fullsized":"2023-11-28T11:25:01.970276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:27.884538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":63,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-28T11:25:28.608864+0000","last_change":"2023-11-28T11:20:50.522582+0000","last_active":"2023-11-28T11:25:28.608864+0000","last_peered":"2023-11-28T11:25:28.608864+0000","last_clean":"2023-11-28T11:25:28.608864+0000","last_became_active":"2023-11-28T11:20:50.522157+0000","last_became_peered":"2023-11-28T11:20:50.522157+0000","last_unstale":"2023-11-28T11:25:28.608864+0000","last_undegraded":"2023-11-28T11:25:28.608864+0000","last_fullsized":"2023-11-28T11:25:28.608864+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:27.884538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":209,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-28T11:24:46.975560+0000","last_change":"2023-11-28T11:20:43.294845+0000","last_active":"2023-11-28T11:24:46.975560+0000","last_peered":"2023-11-28T11:24:46.975560+0000","last_clean":"2023-11-28T11:24:46.975560+0000","last_became_active":"2023-11-28T11:20:43.294483+0000","last_became_peered":"2023-11-28T11:20:43.294483+0000","last_unstale":"2023-11-28T11:24:46.975560+0000","last_undegraded":"2023-11-28T11:24:46.975560+0000","last_fullsized":"2023-11-28T11:24:46.975560+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:42.270512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:42.270512+0000","last_clean_scrub_stamp":"2023-11-28T11:20:42.270512+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:17:03.376680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":56,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-28T11:24:51.976108+0000","last_change":"2023-11-28T11:20:50.526865+0000","last_active":"2023-11-28T11:24:51.976108+0000","last_peered":"2023-11-28T11:24:51.976108+0000","last_clean":"2023-11-28T11:24:51.976108+0000","last_became_active":"2023-11-28T11:20:50.526576+0000","last_became_peered":"2023-11-28T11:20:50.526576+0000","last_unstale":"2023-11-28T11:24:51.976108+0000","last_undegraded":"2023-11-28T11:24:51.976108+0000","last_fullsized":"2023-11-28T11:24:51.976108+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:09:18.460112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":56,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-28T11:25:01.977537+0000","last_change":"2023-11-28T11:20:51.020043+0000","last_active":"2023-11-28T11:25:01.977537+0000","last_peered":"2023-11-28T11:25:01.977537+0000","last_clean":"2023-11-28T11:25:01.977537+0000","last_became_active":"2023-11-28T11:20:51.019702+0000","last_became_peered":"2023-11-28T11:20:51.019702+0000","last_unstale":"2023-11-28T11:25:01.977537+0000","last_undegraded":"2023-11-28T11:25:01.977537+0000","last_fullsized":"2023-11-28T11:25:01.977537+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:26:20.246873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":59,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-28T11:25:28.607964+0000","last_change":"2023-11-28T11:20:50.524251+0000","last_active":"2023-11-28T11:25:28.607964+0000","last_peered":"2023-11-28T11:25:28.607964+0000","last_clean":"2023-11-28T11:25:28.607964+0000","last_became_active":"2023-11-28T11:20:50.524012+0000","last_became_peered":"2023-11-28T11:20:50.524012+0000","last_unstale":"2023-11-28T11:25:28.607964+0000","last_undegraded":"2023-11-28T11:25:28.607964+0000","last_fullsized":"2023-11-28T11:25:28.607964+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:05.302498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":55,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-28T11:25:11.979176+0000","last_change":"2023-11-28T11:20:50.524838+0000","last_active":"2023-11-28T11:25:11.979176+0000","last_peered":"2023-11-28T11:25:11.979176+0000","last_clean":"2023-11-28T11:25:11.979176+0000","last_became_active":"2023-11-28T11:20:50.524402+0000","last_became_peered":"2023-11-28T11:20:50.524402+0000","last_unstale":"2023-11-28T11:25:11.979176+0000","last_undegraded":"2023-11-28T11:25:11.979176+0000","last_fullsized":"2023-11-28T11:25:11.979176+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:22.699447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574910,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11032,"kb_used_data":1424,"kb_used_omap":0,"kb_used_meta":9600,"kb_avail":94360808,"statfs":{"total":96636764160,"available":96625467392,"internally_reserved":0,"allocated":1458176,"data_stored":996952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9830400},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.623,"15min":0.623},"min":{"1min":0.46899999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.89600000000000002,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.504},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.65000000000000002,"15min":0.64900000000000002},"min":{"1min":0.47399999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.89800000000000002,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.46200000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65100000000000002,"15min":0.64500000000000002},"min":{"1min":0.44800000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.90100000000000002,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.624},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.66700000000000004,"15min":0.67500000000000004},"min":{"1min":0.497,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.80000000000000004,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.53200000000000003}]},{"osd":2,"last update":"Tue Nov 28 11:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.72099999999999997,"15min":0.72299999999999998},"min":{"1min":0.57199999999999995,"5min":0.495,"15min":0.495},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.58599999999999997}]},{"osd":3,"last update":"Tue Nov 28 11:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.65500000000000003,"15min":0.66300000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.90900000000000003,"5min":1.004,"15min":1.004},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69999999999999996,"15min":0.70499999999999996},"min":{"1min":0.52300000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.83999999999999997,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.746}]},{"osd":4,"last update":"Tue Nov 28 11:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.61699999999999999,"15min":0.65700000000000003},"min":{"1min":0.36399999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.873,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.69099999999999995,"15min":0.69999999999999996},"min":{"1min":0.42899999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.97099999999999997,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.77000000000000002}]},{"osd":5,"last update":"Tue Nov 28 11:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.66900000000000004,"15min":0.68100000000000005},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.90100000000000002,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.68899999999999995,"15min":0.70999999999999996},"min":{"1min":0.40999999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.92500000000000004,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.70899999999999996}]},{"osd":6,"last update":"Tue Nov 28 11:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.74199999999999999,"15min":0.77200000000000002},"min":{"1min":0.45700000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.92300000000000004,"5min":1.633,"15min":1.633},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.70599999999999996,"15min":0.745},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.92200000000000004,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.84199999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574910,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9816,"kb_used_data":848,"kb_used_omap":0,"kb_used_meta":8960,"kb_avail":94362024,"statfs":{"total":96636764160,"available":96626712576,"internally_reserved":0,"allocated":868352,"data_stored":406698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9175040},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.63700000000000001,"15min":0.64100000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.167,"5min":1.409,"15min":1.409},"last":0.875},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.63800000000000001,"15min":0.63900000000000001},"min":{"1min":0.47099999999999997,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89700000000000002,"5min":1.244,"15min":1.244},"last":0.71699999999999997}]},{"osd":1,"last update":"Tue Nov 28 11:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.64200000000000002,"15min":0.63200000000000001},"min":{"1min":0.437,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.67100000000000004,"15min":0.68300000000000005},"min":{"1min":0.49199999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.02,"5min":1.038,"15min":1.038},"last":0.83899999999999997}]},{"osd":2,"last update":"Tue Nov 28 11:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.63,"15min":0.629},"min":{"1min":0.45400000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.73099999999999998,"15min":0.77300000000000002},"min":{"1min":0.51600000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.198,"5min":1.996,"15min":1.996},"last":0.73899999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.69999999999999996,"15min":0.70099999999999996},"min":{"1min":0.58299999999999996,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.66300000000000003,"15min":0.68100000000000005},"min":{"1min":0.501,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.92300000000000004,"5min":1.008,"15min":1.008},"last":0.75700000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.68899999999999995,"15min":0.72599999999999998},"min":{"1min":0.39800000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.1439999999999999,"5min":2.048,"15min":2.048},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.68999999999999995,"15min":0.71899999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.61699999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.65200000000000002,"15min":0.69099999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.86399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.754,"15min":0.80800000000000005},"min":{"1min":0.46000000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.99099999999999999,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":0.82899999999999996}]},{"osd":7,"last update":"Tue Nov 28 11:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.75,"15min":0.77400000000000002},"min":{"1min":0.56200000000000006,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.0009999999999999,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.76700000000000002,"15min":0.79400000000000004},"min":{"1min":0.58299999999999996,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.006,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.85799999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9964,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94361876,"statfs":{"total":96636764160,"available":96626561024,"internally_reserved":0,"allocated":888832,"data_stored":418996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.42499999999999999,"15min":0.45600000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.67900000000000005,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.45000000000000001,"15min":0.45900000000000002},"min":{"1min":0.33100000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.59799999999999998,"5min":0.999,"15min":0.999},"last":0.53800000000000003}]},{"osd":2,"last update":"Tue Nov 28 11:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.48499999999999999,"15min":0.51000000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.64600000000000002,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.47699999999999998,"15min":0.501},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.57299999999999995,"5min":1.002,"15min":1.002},"last":0.68700000000000006}]},{"osd":3,"last update":"Tue Nov 28 11:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.52800000000000002,"15min":0.55000000000000004},"min":{"1min":0.39100000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.627,"5min":1.198,"15min":1.198},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.51200000000000001,"15min":0.52100000000000002},"min":{"1min":0.38600000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.626,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.60499999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.73499999999999999,"15min":0.753},"min":{"1min":0.48799999999999999,"5min":0.438,"15min":0.438},"max":{"1min":0.91300000000000003,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.73399999999999999,"15min":0.753},"min":{"1min":0.498,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0780000000000001,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.71499999999999997}]},{"osd":5,"last update":"Tue Nov 28 11:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.746,"15min":0.76600000000000001},"min":{"1min":0.53500000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.98299999999999998,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.746,"15min":0.76800000000000002},"min":{"1min":0.50800000000000001,"5min":0.5,"15min":0.5},"max":{"1min":0.86299999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.72599999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.76300000000000001,"15min":0.78100000000000003},"min":{"1min":0.55300000000000005,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.99399999999999999,"5min":1.177,"15min":1.177},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.77900000000000003,"15min":0.80300000000000005},"min":{"1min":0.53000000000000003,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.97799999999999998,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.89800000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.76100000000000001},"min":{"1min":0.56200000000000006,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.92000000000000004,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.77900000000000003,"15min":0.79800000000000004},"min":{"1min":0.50800000000000001,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0089999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.80300000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574910,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":15188,"kb_used_data":1420,"kb_used_omap":0,"kb_used_meta":13760,"kb_avail":94356652,"statfs":{"total":96636764160,"available":96621211648,"internally_reserved":0,"allocated":1454080,"data_stored":996974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14090240},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.42299999999999999,"15min":0.44},"min":{"1min":0.27600000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.59199999999999997,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.53400000000000003,"15min":0.62},"min":{"1min":0.29799999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.875,"5min":3.8420000000000001,"15min":3.8420000000000001},"last":0.44600000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.48099999999999998,"15min":0.501},"min":{"1min":0.32200000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70899999999999996,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.57199999999999995,"15min":0.66000000000000003},"min":{"1min":0.33500000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.60899999999999999,"5min":3.8940000000000001,"15min":3.8940000000000001},"last":0.40100000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.54300000000000004,"15min":0.622},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.72699999999999998,"5min":3.681,"15min":3.681},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55600000000000005,"15min":0.58599999999999997},"min":{"1min":0.42499999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.67800000000000005,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.43099999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.80300000000000005,"15min":0.90300000000000002},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.96299999999999997,"5min":3.9790000000000001,"15min":3.9790000000000001},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.81599999999999995,"15min":0.92400000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.97899999999999998,"5min":3.5779999999999998,"15min":3.5779999999999998},"last":0.73199999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.77300000000000002,"15min":0.81399999999999995},"min":{"1min":0.56000000000000005,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.0069999999999999,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.874},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.82499999999999996,"15min":0.93999999999999995},"min":{"1min":0.50600000000000001,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1,"5min":3.9409999999999998,"15min":3.9409999999999998},"last":0.71999999999999997}]},{"osd":6,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.81999999999999995,"15min":0.92600000000000005},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0469999999999999,"5min":3.6160000000000001,"15min":3.6160000000000001},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.78400000000000003,"15min":0.83299999999999996},"min":{"1min":0.55200000000000005,"5min":0.501,"15min":0.501},"max":{"1min":0.94799999999999995,"5min":1.363,"15min":1.363},"last":0.70899999999999996}]},{"osd":7,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.88,"15min":1.0109999999999999},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.149,"5min":4.0099999999999998,"15min":4.0099999999999998},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.85699999999999998,"15min":0.97199999999999998},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.075,"5min":3.653,"15min":3.653},"last":0.79200000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574908,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9828,"kb_used_data":860,"kb_used_omap":0,"kb_used_meta":8960,"kb_avail":94362012,"statfs":{"total":96636764160,"available":96626700288,"internally_reserved":0,"allocated":880640,"data_stored":418396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9175040},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.46899999999999997,"15min":0.54400000000000004},"min":{"1min":0.255,"5min":0.219,"15min":0.219},"max":{"1min":0.56499999999999995,"5min":4.1109999999999998,"15min":4.1109999999999998},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.442,"5min":0.51700000000000002,"15min":0.58299999999999996},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.75800000000000001,"5min":3.6230000000000002,"15min":3.6230000000000002},"last":0.40200000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.61099999999999999,"15min":0.66600000000000004},"min":{"1min":0.40400000000000003,"5min":0.371,"15min":0.371},"max":{"1min":0.77700000000000002,"5min":3.6640000000000001,"15min":3.6640000000000001},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.502,"5min":0.58399999999999996,"15min":0.67500000000000004},"min":{"1min":0.35299999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.73199999999999998,"5min":4.0830000000000002,"15min":4.0830000000000002},"last":0.51500000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.55200000000000005,"15min":0.63600000000000001},"min":{"1min":0.35399999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.78200000000000003,"5min":4.1529999999999996,"15min":4.1529999999999996},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.55300000000000005,"15min":0.57999999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.68799999999999994,"5min":1.304,"15min":1.304},"last":0.46100000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.77500000000000002,"15min":0.80500000000000005},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0569999999999999,"5min":1.649,"15min":1.649},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.80300000000000005,"15min":0.89900000000000002},"min":{"1min":0.53300000000000003,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.98099999999999998,"5min":4.0060000000000002,"15min":4.0060000000000002},"last":0.54800000000000004}]},{"osd":5,"last update":"Tue Nov 28 11:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.81599999999999995,"15min":0.91000000000000003},"min":{"1min":0.504,"5min":0.5,"15min":0.5},"max":{"1min":0.94399999999999995,"5min":4.0529999999999999,"15min":4.0529999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.78400000000000003,"15min":0.80800000000000005},"min":{"1min":0.56200000000000006,"5min":0.497,"15min":0.497},"max":{"1min":1.125,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.71599999999999997}]},{"osd":6,"last update":"Tue Nov 28 11:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.79900000000000004,"15min":0.83599999999999997},"min":{"1min":0.51600000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.083,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.82899999999999996,"15min":0.85499999999999998},"min":{"1min":0.57399999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.155,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.63300000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.77100000000000002,"15min":0.80200000000000005},"min":{"1min":0.56999999999999995,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0109999999999999,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.80100000000000005,"15min":0.82699999999999996},"min":{"1min":0.52900000000000003,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1060000000000001,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.65500000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574909,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10008,"kb_used_data":848,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361832,"statfs":{"total":96636764160,"available":96626515968,"internally_reserved":0,"allocated":868352,"data_stored":406718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.39900000000000002,"15min":0.40400000000000003},"min":{"1min":0.28599999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.55000000000000004,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.45800000000000002,"15min":0.46100000000000002},"min":{"1min":0.30399999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.67800000000000005,"5min":1.157,"15min":1.157},"last":0.57699999999999996}]},{"osd":1,"last update":"Tue Nov 28 11:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.50700000000000001,"15min":0.57099999999999995},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.71099999999999997,"5min":4.8019999999999996,"15min":4.8019999999999996},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.52700000000000002,"15min":0.58499999999999996},"min":{"1min":0.33400000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.625,"5min":4.7539999999999996,"15min":4.7539999999999996},"last":0.51800000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.57699999999999996,"15min":0.64900000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.66900000000000004,"5min":4.8419999999999996,"15min":4.8419999999999996},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.502,"5min":0.57799999999999996,"15min":0.64600000000000002},"min":{"1min":0.34999999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.71199999999999997,"5min":4.7919999999999998,"15min":4.7919999999999998},"last":0.60299999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.76400000000000001,"15min":0.82899999999999996},"min":{"1min":0.49199999999999999,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.93500000000000005,"5min":4.8869999999999996,"15min":4.8869999999999996},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.80800000000000005,"15min":0.89000000000000001},"min":{"1min":0.54200000000000004,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.89900000000000002,"5min":5.2460000000000004,"15min":5.2460000000000004},"last":0.88400000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.81699999999999995,"15min":0.90000000000000002},"min":{"1min":0.54300000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.96799999999999997,"5min":5.0759999999999996,"15min":5.0759999999999996},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.79400000000000004,"15min":0.86299999999999999},"min":{"1min":0.51300000000000001,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.88700000000000001,"5min":5.3010000000000002,"15min":5.3010000000000002},"last":0.90300000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.80900000000000005,"15min":0.876},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.91400000000000003,"5min":5.3399999999999999,"15min":5.3399999999999999},"last":1.004},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.80100000000000005,"15min":0.86099999999999999},"min":{"1min":0.53800000000000003,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.91600000000000004,"5min":4.8330000000000002,"15min":4.8330000000000002},"last":0.96199999999999997}]},{"osd":7,"last update":"Tue Nov 28 11:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.80600000000000005,"15min":0.86899999999999999},"min":{"1min":0.52800000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.90500000000000003,"5min":4.8600000000000003,"15min":4.8600000000000003},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.83499999999999996,"15min":0.92300000000000004},"min":{"1min":0.56399999999999995,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.90100000000000002,"5min":5.1909999999999998,"15min":5.1909999999999998},"last":0.97899999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574909,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10196,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":9344,"kb_avail":94361644,"statfs":{"total":96636764160,"available":96626323456,"internally_reserved":0,"allocated":864256,"data_stored":406676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9568256},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.66900000000000004,"15min":0.68200000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.93999999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.67100000000000004,"15min":0.66400000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.97199999999999998,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.61299999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.67400000000000004,"15min":0.65200000000000002},"min":{"1min":0.40000000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.70599999999999996,"15min":0.70899999999999996},"min":{"1min":0.496,"5min":0.379,"15min":0.379},"max":{"1min":1.0700000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.499}]},{"osd":2,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.74099999999999999,"15min":0.745},"min":{"1min":0.47899999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.71399999999999997,"15min":0.70299999999999996},"min":{"1min":0.46600000000000003,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.64300000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.69099999999999995,"15min":0.68500000000000005},"min":{"1min":0.498,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.141,"5min":1.179,"15min":1.179},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.77300000000000002,"15min":0.78100000000000003},"min":{"1min":0.55000000000000004,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.298,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.73299999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.67500000000000004,"15min":0.68600000000000005},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.1140000000000001,"5min":1.161,"15min":1.161},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.66400000000000003,"15min":0.66600000000000004},"min":{"1min":0.437,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.27,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.71699999999999997}]},{"osd":6,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.71299999999999997,"15min":0.73099999999999998},"min":{"1min":0.438,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.359,"5min":1.514,"15min":1.514},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.72399999999999998,"15min":0.73599999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.70899999999999996}]},{"osd":7,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.70199999999999996,"15min":0.70299999999999996},"min":{"1min":0.47999999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.80100000000000005,"15min":0.83199999999999996},"min":{"1min":0.56200000000000006,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.3999999999999999,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":0.69599999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574909,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9888,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94361952,"statfs":{"total":96636764160,"available":96626638848,"internally_reserved":0,"allocated":876544,"data_stored":413630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.64700000000000002,"15min":0.63100000000000001},"min":{"1min":0.42299999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.85799999999999998,"5min":1.032,"15min":1.032},"last":0.745},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.68799999999999994,"15min":0.69599999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.91300000000000003,"5min":1.02,"15min":1.02},"last":0.65800000000000003}]},{"osd":1,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.67700000000000005,"15min":0.66200000000000003},"min":{"1min":0.48699999999999999,"5min":0.442,"15min":0.442},"max":{"1min":0.93600000000000005,"5min":1.355,"15min":1.355},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.68700000000000006,"15min":0.69399999999999995},"min":{"1min":0.48599999999999999,"5min":0.442,"15min":0.442},"max":{"1min":0.83399999999999996,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.70299999999999996}]},{"osd":2,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69399999999999995,"15min":0.69799999999999995},"min":{"1min":0.52200000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.93999999999999995,"5min":1.198,"15min":1.198},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.72199999999999998,"15min":0.73199999999999998},"min":{"1min":0.47799999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.75900000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.73399999999999999,"15min":0.751},"min":{"1min":0.50800000000000001,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.88700000000000001,"5min":1.141,"15min":1.141},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.72399999999999998,"15min":0.72599999999999998},"min":{"1min":0.55700000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.84399999999999997}]},{"osd":4,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.67800000000000005,"15min":0.71399999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.84099999999999997,"5min":2.5139999999999998,"15min":2.5139999999999998},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.70599999999999996,"15min":0.72999999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.98699999999999999,"5min":1.198,"15min":1.198},"last":0.85899999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.71899999999999997,"15min":0.73099999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.71899999999999997,"15min":0.72699999999999998},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.88900000000000001,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":0.77900000000000003}]},{"osd":7,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.77300000000000002,"15min":0.81299999999999994},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.96499999999999997,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.79100000000000004,"15min":0.81999999999999995},"min":{"1min":0.51600000000000001,"5min":0.496,"15min":0.496},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.88}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:25:36.425 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:25:36.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.618+0000 7f47ac3ff700 1 Processor -- start 2023-11-28T11:25:36.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.619+0000 7f47ac3ff700 1 -- start start 2023-11-28T11:25:36.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.620+0000 7f47ac3ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f47a40b8510 0x7f47a406b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:36.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.620+0000 7f47ac3ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f47a40d0d60 0x7f47a40d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:36.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.620+0000 7f47ac3ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47a40b88e0 con 0x7f47a40cd460 2023-11-28T11:25:36.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.620+0000 7f47ac3ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47a40cdbd0 con 0x7f47a40d0d60 2023-11-28T11:25:36.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.620+0000 7f47ac3ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47a40d02d0 con 0x7f47a40b8510 2023-11-28T11:25:36.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.620+0000 7f47aa99c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f47a40d0d60 0x7f47a40d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:36.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.620+0000 7f47aa99c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f47a40d0d60 0x7f47a40d1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51332/0 (socket says 172.21.15.92:51332) 2023-11-28T11:25:36.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.620+0000 7f47aa99c700 1 -- 172.21.15.92:0/2431586798 learned_addr learned my addr 172.21.15.92:0/2431586798 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:36.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.620+0000 7f47aa19b700 1 --2- 172.21.15.92:0/2431586798 >> v2:172.21.15.169:3300/0 conn(0x7f47a40b8510 0x7f47a406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:36.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.620+0000 7f47aa99c700 1 -- 172.21.15.92:0/2431586798 >> v2:172.21.15.169:3300/0 conn(0x7f47a40b8510 msgr2=0x7f47a406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:36.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.620+0000 7f47aa99c700 1 --2- 172.21.15.92:0/2431586798 >> v2:172.21.15.169:3300/0 conn(0x7f47a40b8510 0x7f47a406b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:36.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.620+0000 7f47aa99c700 1 -- 172.21.15.92:0/2431586798 >> v1:172.21.15.169:6789/0 conn(0x7f47a40cd460 legacy=0x7f47a406beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:36.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.620+0000 7f47aa99c700 1 -- 172.21.15.92:0/2431586798 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a40cd8f0 con 0x7f47a40d0d60 2023-11-28T11:25:36.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.621+0000 7f47aa99c700 1 --2- 172.21.15.92:0/2431586798 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47a40d0d60 0x7f47a40d1200 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f47a40b5a50 tx=0x7f479c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59c3fe313dbc317c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:36.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.621+0000 7f47a9199700 1 -- 172.21.15.92:0/2431586798 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479c00e070 con 0x7f47a40d0d60 2023-11-28T11:25:36.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.621+0000 7f47a9199700 1 -- 172.21.15.92:0/2431586798 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f479c003a20 con 0x7f47a40d0d60 2023-11-28T11:25:36.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.621+0000 7f47a9199700 1 -- 172.21.15.92:0/2431586798 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479c003dd0 con 0x7f47a40d0d60 2023-11-28T11:25:36.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.621+0000 7f47ac3ff700 1 -- 172.21.15.92:0/2431586798 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47a40d0d60 msgr2=0x7f47a40d1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:36.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.621+0000 7f47ac3ff700 1 --2- 172.21.15.92:0/2431586798 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47a40d0d60 0x7f47a40d1200 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f47a40b5a50 tx=0x7f479c00b330 comp rx=0 tx=0).stop 2023-11-28T11:25:36.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.622+0000 7f47ac3ff700 1 -- 172.21.15.92:0/2431586798 shutdown_connections 2023-11-28T11:25:36.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.622+0000 7f47ac3ff700 1 --2- 172.21.15.92:0/2431586798 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47a40d0d60 0x7f47a40d1200 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:36.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.622+0000 7f47ac3ff700 1 --2- 172.21.15.92:0/2431586798 >> v2:172.21.15.169:3300/0 conn(0x7f47a40b8510 0x7f47a406b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:36.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.622+0000 7f47ac3ff700 1 -- 172.21.15.92:0/2431586798 >> 172.21.15.92:0/2431586798 conn(0x7f47a40719a0 msgr2=0x7f47a4071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:36.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.622+0000 7f47ac3ff700 1 -- 172.21.15.92:0/2431586798 shutdown_connections 2023-11-28T11:25:36.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.622+0000 7f47ac3ff700 1 -- 172.21.15.92:0/2431586798 wait complete. 2023-11-28T11:25:36.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.623+0000 7f47ac3ff700 1 Processor -- start 2023-11-28T11:25:36.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.623+0000 7f47ac3ff700 1 -- start start 2023-11-28T11:25:36.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.624+0000 7f47ac3ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47a40b8510 0x7f47a41e7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:36.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.624+0000 7f47ac3ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47a40cd460 0x7f47a41e4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:36.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.624+0000 7f47ac3ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47a40d0d60 0x7f47a41e5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:36.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.624+0000 7f47ac3ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a415b8e0 con 0x7f47a40b8510 2023-11-28T11:25:36.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.624+0000 7f47ac3ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a41e5860 con 0x7f47a40d0d60 2023-11-28T11:25:36.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.624+0000 7f47ac3ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47a41e5b40 con 0x7f47a40cd460 2023-11-28T11:25:36.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.625+0000 7f47aa19b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47a40b8510 0x7f47a41e7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:36.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.625+0000 7f47aa19b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47a40b8510 0x7f47a41e7690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51338/0 (socket says 172.21.15.92:51338) 2023-11-28T11:25:36.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.625+0000 7f47aa19b700 1 -- 172.21.15.92:0/1414637517 learned_addr learned my addr 172.21.15.92:0/1414637517 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:36.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.625+0000 7f47aa99c700 1 --2- 172.21.15.92:0/1414637517 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47a40d0d60 0x7f47a41e5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:36.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.625+0000 7f47a999a700 1 --2- 172.21.15.92:0/1414637517 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47a40cd460 0x7f47a41e4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:36.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.625+0000 7f47aa19b700 1 -- 172.21.15.92:0/1414637517 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47a40cd460 msgr2=0x7f47a41e4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:36.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.625+0000 7f47aa19b700 1 --2- 172.21.15.92:0/1414637517 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47a40cd460 0x7f47a41e4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:36.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.625+0000 7f47aa19b700 1 -- 172.21.15.92:0/1414637517 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47a40d0d60 msgr2=0x7f47a41e5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:36.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.625+0000 7f47aa19b700 1 --2- 172.21.15.92:0/1414637517 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47a40d0d60 0x7f47a41e5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:36.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.625+0000 7f47aa19b700 1 -- 172.21.15.92:0/1414637517 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f479c009b30 con 0x7f47a40b8510 2023-11-28T11:25:36.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.625+0000 7f47aa19b700 1 --2- 172.21.15.92:0/1414637517 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47a40b8510 0x7f47a41e7690 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f47a000e810 tx=0x7f47a000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:36.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.625+0000 7f479affd700 1 -- 172.21.15.92:0/1414637517 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a000c070 con 0x7f47a40b8510 2023-11-28T11:25:36.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.626+0000 7f479affd700 1 -- 172.21.15.92:0/1414637517 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47a0013610 con 0x7f47a40b8510 2023-11-28T11:25:36.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.626+0000 7f479affd700 1 -- 172.21.15.92:0/1414637517 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a00139c0 con 0x7f47a40b8510 2023-11-28T11:25:36.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.626+0000 7f47ac3ff700 1 -- 172.21.15.92:0/1414637517 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47a40cfff0 con 0x7f47a40b8510 2023-11-28T11:25:36.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.626+0000 7f47ac3ff700 1 -- 172.21.15.92:0/1414637517 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47a40cfd10 con 0x7f47a40b8510 2023-11-28T11:25:36.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.627+0000 7f47ac3ff700 1 -- 172.21.15.92:0/1414637517 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47a41e6750 con 0x7f47a40b8510 2023-11-28T11:25:36.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.628+0000 7f479affd700 1 -- 172.21.15.92:0/1414637517 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f47a0023070 con 0x7f47a40b8510 2023-11-28T11:25:36.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.628+0000 7f479affd700 1 --2- 172.21.15.92:0/1414637517 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f47880682f0 0x7f478806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:36.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.629+0000 7f479affd700 1 -- 172.21.15.92:0/1414637517 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f47a0085900 con 0x7f47a40b8510 2023-11-28T11:25:36.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.629+0000 7f47a999a700 1 --2- 172.21.15.92:0/1414637517 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f47880682f0 0x7f478806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:36.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.629+0000 7f47a999a700 1 --2- 172.21.15.92:0/1414637517 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f47880682f0 0x7f478806a7a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f47a40b32f0 tx=0x7f4794009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:36.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.633+0000 7f479affd700 1 -- 172.21.15.92:0/1414637517 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f47a00521d0 con 0x7f47a40b8510 2023-11-28T11:25:36.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.829+0000 7f47ac3ff700 1 -- 172.21.15.92:0/1414637517 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f47a4067880 con 0x7f47a40b8510 2023-11-28T11:25:36.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.830+0000 7f479affd700 1 -- 172.21.15.92:0/1414637517 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+14602 (secure 0 0 0) 0x7f47a000f040 con 0x7f47a40b8510 2023-11-28T11:25:36.831 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:25:36.831 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":73,"fsid":"68956e0c-c16f-47cd-a220-f98364b0831d","created":"2023-11-28T11:20:28.697573+0000","modified":"2023-11-28T11:25:32.793156+0000","last_up_change":"2023-11-28T11:20:41.257011+0000","last_in_change":"2023-11-28T11:20:32.249767+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T11:20:41.925237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T11:20:48.682497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6abe483f-0e03-4a26-b9cf-62041291747e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":115823},{"type":"v1","addr":"172.21.15.92:6809","nonce":115823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":115823},{"type":"v1","addr":"172.21.15.92:6813","nonce":115823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":115823},{"type":"v1","addr":"172.21.15.92:6821","nonce":115823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":115823},{"type":"v1","addr":"172.21.15.92:6817","nonce":115823}]},"public_addr":"172.21.15.92:6809/115823","cluster_addr":"172.21.15.92:6813/115823","heartbeat_back_addr":"172.21.15.92:6821/115823","heartbeat_front_addr":"172.21.15.92:6817/115823","state":["exists","up"]},{"osd":1,"uuid":"13c1de73-1a8c-4d7e-8a28-9dae52dfe089","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":115824},{"type":"v1","addr":"172.21.15.92:6812","nonce":115824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":115824},{"type":"v1","addr":"172.21.15.92:6816","nonce":115824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":115824},{"type":"v1","addr":"172.21.15.92:6823","nonce":115824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":115824},{"type":"v1","addr":"172.21.15.92:6820","nonce":115824}]},"public_addr":"172.21.15.92:6812/115824","cluster_addr":"172.21.15.92:6816/115824","heartbeat_back_addr":"172.21.15.92:6823/115824","heartbeat_front_addr":"172.21.15.92:6820/115824","state":["exists","up"]},{"osd":2,"uuid":"25788a55-112f-4f00-810d-2bc37cf76778","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":115821},{"type":"v1","addr":"172.21.15.92:6801","nonce":115821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":115821},{"type":"v1","addr":"172.21.15.92:6803","nonce":115821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":115821},{"type":"v1","addr":"172.21.15.92:6807","nonce":115821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":115821},{"type":"v1","addr":"172.21.15.92:6805","nonce":115821}]},"public_addr":"172.21.15.92:6801/115821","cluster_addr":"172.21.15.92:6803/115821","heartbeat_back_addr":"172.21.15.92:6807/115821","heartbeat_front_addr":"172.21.15.92:6805/115821","state":["exists","up"]},{"osd":3,"uuid":"adf54e24-33a9-41fe-9ce3-98a5ab6c158e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":115825},{"type":"v1","addr":"172.21.15.92:6825","nonce":115825}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":115825},{"type":"v1","addr":"172.21.15.92:6827","nonce":115825}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":115825},{"type":"v1","addr":"172.21.15.92:6831","nonce":115825}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":115825},{"type":"v1","addr":"172.21.15.92:6829","nonce":115825}]},"public_addr":"172.21.15.92:6825/115825","cluster_addr":"172.21.15.92:6827/115825","heartbeat_back_addr":"172.21.15.92:6831/115825","heartbeat_front_addr":"172.21.15.92:6829/115825","state":["exists","up"]},{"osd":4,"uuid":"ccb9b747-f399-44ea-9a18-9db3152227e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6816","nonce":116035},{"type":"v1","addr":"172.21.15.169:6817","nonce":116035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6818","nonce":116035},{"type":"v1","addr":"172.21.15.169:6819","nonce":116035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6824","nonce":116035},{"type":"v1","addr":"172.21.15.169:6826","nonce":116035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6820","nonce":116035},{"type":"v1","addr":"172.21.15.169:6822","nonce":116035}]},"public_addr":"172.21.15.169:6817/116035","cluster_addr":"172.21.15.169:6819/116035","heartbeat_back_addr":"172.21.15.169:6826/116035","heartbeat_front_addr":"172.21.15.169:6822/116035","state":["exists","up"]},{"osd":5,"uuid":"308a1e63-75aa-4d7d-8a7c-cecc24714fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6800","nonce":116033},{"type":"v1","addr":"172.21.15.169:6801","nonce":116033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6802","nonce":116033},{"type":"v1","addr":"172.21.15.169:6803","nonce":116033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6806","nonce":116033},{"type":"v1","addr":"172.21.15.169:6807","nonce":116033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6804","nonce":116033},{"type":"v1","addr":"172.21.15.169:6805","nonce":116033}]},"public_addr":"172.21.15.169:6801/116033","cluster_addr":"172.21.15.169:6803/116033","heartbeat_back_addr":"172.21.15.169:6807/116033","heartbeat_front_addr":"172.21.15.169:6805/116033","state":["exists","up"]},{"osd":6,"uuid":"9c459b7b-b821-4b4b-b91e-6aaae271457c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6808","nonce":116034},{"type":"v1","addr":"172.21.15.169:6809","nonce":116034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6810","nonce":116034},{"type":"v1","addr":"172.21.15.169:6811","nonce":116034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6814","nonce":116034},{"type":"v1","addr":"172.21.15.169:6815","nonce":116034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6812","nonce":116034},{"type":"v1","addr":"172.21.15.169:6813","nonce":116034}]},"public_addr":"172.21.15.169:6809/116034","cluster_addr":"172.21.15.169:6811/116034","heartbeat_back_addr":"172.21.15.169:6815/116034","heartbeat_front_addr":"172.21.15.169:6813/116034","state":["exists","up"]},{"osd":7,"uuid":"1ca9aa4a-755d-4e28-a937-4d8dccab4df3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6821","nonce":116044},{"type":"v1","addr":"172.21.15.169:6823","nonce":116044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6825","nonce":116044},{"type":"v1","addr":"172.21.15.169:6827","nonce":116044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6830","nonce":116044},{"type":"v1","addr":"172.21.15.169:6831","nonce":116044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6828","nonce":116044},{"type":"v1","addr":"172.21.15.169:6829","nonce":116044}]},"public_addr":"172.21.15.169:6823/116044","cluster_addr":"172.21.15.169:6827/116044","heartbeat_back_addr":"172.21.15.169:6831/116044","heartbeat_front_addr":"172.21.15.169:6829/116044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.586781+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.869778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.508594+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.860493+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.872267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.746933+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.611520+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:20:37.871446+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:6836/3634521248":"2023-11-29T11:25:28.603022+0000","172.21.15.92:6834/3634521248":"2023-11-29T11:25:28.603022+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:25:36.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.832+0000 7f47ac3ff700 1 -- 172.21.15.92:0/1414637517 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f47880682f0 msgr2=0x7f478806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:36.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.832+0000 7f47ac3ff700 1 --2- 172.21.15.92:0/1414637517 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f47880682f0 0x7f478806a7a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f47a40b32f0 tx=0x7f4794009040 comp rx=0 tx=0).stop 2023-11-28T11:25:36.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.832+0000 7f47ac3ff700 1 -- 172.21.15.92:0/1414637517 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47a40b8510 msgr2=0x7f47a41e7690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:36.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.832+0000 7f47ac3ff700 1 --2- 172.21.15.92:0/1414637517 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47a40b8510 0x7f47a41e7690 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f47a000e810 tx=0x7f47a000eb20 comp rx=0 tx=0).stop 2023-11-28T11:25:36.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.833+0000 7f47ac3ff700 1 -- 172.21.15.92:0/1414637517 shutdown_connections 2023-11-28T11:25:36.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.833+0000 7f47ac3ff700 1 --2- 172.21.15.92:0/1414637517 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f47a40d0d60 0x7f47a41e5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:36.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.833+0000 7f47ac3ff700 1 --2- 172.21.15.92:0/1414637517 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f47880682f0 0x7f478806a7a0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:36.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.833+0000 7f47ac3ff700 1 --2- 172.21.15.92:0/1414637517 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47a40b8510 0x7f47a41e7690 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:36.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.833+0000 7f47ac3ff700 1 --2- 172.21.15.92:0/1414637517 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f47a40cd460 0x7f47a41e4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:36.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.833+0000 7f47ac3ff700 1 -- 172.21.15.92:0/1414637517 >> 172.21.15.92:0/1414637517 conn(0x7f47a40719a0 msgr2=0x7f47a40b5d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:36.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.833+0000 7f47ac3ff700 1 -- 172.21.15.92:0/1414637517 shutdown_connections 2023-11-28T11:25:36.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:36.833+0000 7f47ac3ff700 1 -- 172.21.15.92:0/1414637517 wait complete. 2023-11-28T11:25:37.852 INFO:tasks.ceph:Scrubbing osd.0 2023-11-28T11:25:37.853 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-11-28T11:25:38.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.079+0000 7fa01fbff700 1 Processor -- start 2023-11-28T11:25:38.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.079+0000 7fa01fbff700 1 -- start start 2023-11-28T11:25:38.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.081+0000 7fa01fbff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa0180cd460 0x7fa01806b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:38.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.081+0000 7fa01fbff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa01806be80 0x7fa0180d0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:38.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.081+0000 7fa01fbff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0180b5240 con 0x7fa0180b8790 2023-11-28T11:25:38.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.081+0000 7fa01fbff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa01806c300 con 0x7fa0180cd460 2023-11-28T11:25:38.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.081+0000 7fa01fbff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0180b5520 con 0x7fa01806be80 2023-11-28T11:25:38.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.081+0000 7fa01d99b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fa0180b8790 0x7fa0180cfb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36208/0 (socket says 172.21.15.92:36208) 2023-11-28T11:25:38.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.081+0000 7fa01d99b700 1 -- 172.21.15.92:0/298293492 learned_addr learned my addr 172.21.15.92:0/298293492 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:25:38.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.081+0000 7fa01d19a700 1 --2- 172.21.15.92:0/298293492 >> v2:172.21.15.92:3300/0 conn(0x7fa0180cd460 0x7fa01806b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:38.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.081+0000 7fa01e19c700 1 --2- 172.21.15.92:0/298293492 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa01806be80 0x7fa0180d0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:38.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.081+0000 7fa01d19a700 1 -- 172.21.15.92:0/298293492 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa01806be80 msgr2=0x7fa0180d0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:38.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.081+0000 7fa01d19a700 1 --2- 172.21.15.92:0/298293492 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa01806be80 0x7fa0180d0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:38.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.081+0000 7fa01d19a700 1 -- 172.21.15.92:0/298293492 >> v1:172.21.15.92:6789/0 conn(0x7fa0180b8790 legacy=0x7fa0180cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:38.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.081+0000 7fa01d19a700 1 -- 172.21.15.92:0/298293492 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0180d0a80 con 0x7fa0180cd460 2023-11-28T11:25:38.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.081+0000 7fa01e19c700 1 --2- 172.21.15.92:0/298293492 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa01806be80 0x7fa0180d0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:25:38.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.081+0000 7fa01c999700 1 -- 172.21.15.92:0/298293492 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1759625449 0 0) 0x7fa0180b5240 con 0x7fa0180b8790 2023-11-28T11:25:38.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.082+0000 7fa01d19a700 1 --2- 172.21.15.92:0/298293492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0180cd460 0x7fa01806b940 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fa008009870 tx=0x7fa008009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0f72abc2253510e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:38.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.082+0000 7fa01c999700 1 -- 172.21.15.92:0/298293492 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa008011830 con 0x7fa0180cd460 2023-11-28T11:25:38.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.082+0000 7fa01c999700 1 -- 172.21.15.92:0/298293492 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa008011990 con 0x7fa0180cd460 2023-11-28T11:25:38.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.082+0000 7fa01c999700 1 -- 172.21.15.92:0/298293492 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa008011d20 con 0x7fa0180cd460 2023-11-28T11:25:38.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.082+0000 7fa01fbff700 1 -- 172.21.15.92:0/298293492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0180cd460 msgr2=0x7fa01806b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:38.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.082+0000 7fa01fbff700 1 --2- 172.21.15.92:0/298293492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0180cd460 0x7fa01806b940 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fa008009870 tx=0x7fa008009b80 comp rx=0 tx=0).stop 2023-11-28T11:25:38.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.083+0000 7fa01fbff700 1 -- 172.21.15.92:0/298293492 shutdown_connections 2023-11-28T11:25:38.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.083+0000 7fa01fbff700 1 --2- 172.21.15.92:0/298293492 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa01806be80 0x7fa0180d0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:38.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.083+0000 7fa01fbff700 1 --2- 172.21.15.92:0/298293492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0180cd460 0x7fa01806b940 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:38.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.083+0000 7fa01fbff700 1 -- 172.21.15.92:0/298293492 >> 172.21.15.92:0/298293492 conn(0x7fa0180719a0 msgr2=0x7fa018071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:38.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.083+0000 7fa01fbff700 1 -- 172.21.15.92:0/298293492 shutdown_connections 2023-11-28T11:25:38.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.083+0000 7fa01fbff700 1 -- 172.21.15.92:0/298293492 wait complete. 2023-11-28T11:25:38.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.084+0000 7fa01fbff700 1 Processor -- start 2023-11-28T11:25:38.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.084+0000 7fa01fbff700 1 -- start start 2023-11-28T11:25:38.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.085+0000 7fa01fbff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa01806be80 0x7fa0180ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:38.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.085+0000 7fa01fbff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0180b8790 0x7fa018150970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:38.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.085+0000 7fa01fbff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0180cd460 0x7fa018153ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:38.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.085+0000 7fa01fbff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa018153080 con 0x7fa01806be80 2023-11-28T11:25:38.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.085+0000 7fa01fbff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0180cd270 con 0x7fa0180cd460 2023-11-28T11:25:38.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.085+0000 7fa01fbff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0180cb850 con 0x7fa0180b8790 2023-11-28T11:25:38.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.086+0000 7fa01d99b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa01806be80 0x7fa0180ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:38.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.086+0000 7fa01d99b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa01806be80 0x7fa0180ccd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51356/0 (socket says 172.21.15.92:51356) 2023-11-28T11:25:38.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.086+0000 7fa01d99b700 1 -- 172.21.15.92:0/2162586505 learned_addr learned my addr 172.21.15.92:0/2162586505 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:38.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.086+0000 7fa01d19a700 1 --2- 172.21.15.92:0/2162586505 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0180b8790 0x7fa018150970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:38.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.086+0000 7fa01e19c700 1 --2- 172.21.15.92:0/2162586505 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0180cd460 0x7fa018153ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:38.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.086+0000 7fa01d99b700 1 -- 172.21.15.92:0/2162586505 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0180b8790 msgr2=0x7fa018150970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:38.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.086+0000 7fa01d99b700 1 --2- 172.21.15.92:0/2162586505 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa0180b8790 0x7fa018150970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:38.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.086+0000 7fa01d99b700 1 -- 172.21.15.92:0/2162586505 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0180cd460 msgr2=0x7fa018153ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:38.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.086+0000 7fa01d99b700 1 --2- 172.21.15.92:0/2162586505 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa0180cd460 0x7fa018153ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:38.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.086+0000 7fa01d99b700 1 -- 172.21.15.92:0/2162586505 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0080094a0 con 0x7fa01806be80 2023-11-28T11:25:38.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.086+0000 7fa01d99b700 1 --2- 172.21.15.92:0/2162586505 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa01806be80 0x7fa0180ccd30 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fa0140011e0 tx=0x7fa01400ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:38.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.087+0000 7fa00e7fc700 1 -- 172.21.15.92:0/2162586505 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa014004c80 con 0x7fa01806be80 2023-11-28T11:25:38.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.087+0000 7fa00e7fc700 1 -- 172.21.15.92:0/2162586505 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa014004de0 con 0x7fa01806be80 2023-11-28T11:25:38.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.087+0000 7fa00e7fc700 1 -- 172.21.15.92:0/2162586505 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa014007310 con 0x7fa01806be80 2023-11-28T11:25:38.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.087+0000 7fa01fbff700 1 -- 172.21.15.92:0/2162586505 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0180d14e0 con 0x7fa01806be80 2023-11-28T11:25:38.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.087+0000 7fa01fbff700 1 -- 172.21.15.92:0/2162586505 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0180b5800 con 0x7fa01806be80 2023-11-28T11:25:38.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.088+0000 7fa01fbff700 1 -- 172.21.15.92:0/2162586505 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa0180d1670 con 0x7fa01806be80 2023-11-28T11:25:38.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.089+0000 7fa00e7fc700 1 -- 172.21.15.92:0/2162586505 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa01401a020 con 0x7fa01806be80 2023-11-28T11:25:38.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.090+0000 7fa00e7fc700 1 --2- 172.21.15.92:0/2162586505 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9ffc0682f0 0x7f9ffc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:38.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.090+0000 7fa00e7fc700 1 -- 172.21.15.92:0/2162586505 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fa014085760 con 0x7fa01806be80 2023-11-28T11:25:38.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.090+0000 7fa01d19a700 1 --2- 172.21.15.92:0/2162586505 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9ffc0682f0 0x7f9ffc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:38.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.090+0000 7fa00e7fc700 1 --2- 172.21.15.92:0/2162586505 >> [v2:172.21.15.92:6808/115823,v1:172.21.15.92:6809/115823] conn(0x7f9ffc06eb90 0x7f9ffc070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:38.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.090+0000 7fa00e7fc700 1 -- 172.21.15.92:0/2162586505 --> [v2:172.21.15.92:6808/115823,v1:172.21.15.92:6809/115823] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9ffc0716b0 con 0x7f9ffc06eb90 2023-11-28T11:25:38.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.090+0000 7fa00e7fc700 1 -- 172.21.15.92:0/2162586505 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa014087d40 con 0x7fa01806be80 2023-11-28T11:25:38.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.090+0000 7fa01e19c700 1 --2- 172.21.15.92:0/2162586505 >> [v2:172.21.15.92:6808/115823,v1:172.21.15.92:6809/115823] conn(0x7f9ffc06eb90 0x7f9ffc070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:38.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.090+0000 7fa01d19a700 1 --2- 172.21.15.92:0/2162586505 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9ffc0682f0 0x7f9ffc06a7a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa008000c00 tx=0x7fa008008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:38.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.091+0000 7fa01e19c700 1 --2- 172.21.15.92:0/2162586505 >> [v2:172.21.15.92:6808/115823,v1:172.21.15.92:6809/115823] conn(0x7f9ffc06eb90 0x7f9ffc070fe0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:38.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.091+0000 7fa00e7fc700 1 -- 172.21.15.92:0/2162586505 <== osd.0 v2:172.21.15.92:6808/115823 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f9ffc0716b0 con 0x7f9ffc06eb90 2023-11-28T11:25:38.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.106+0000 7fa01fbff700 1 -- 172.21.15.92:0/2162586505 --> [v2:172.21.15.92:6808/115823,v1:172.21.15.92:6809/115823] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa0180d3d80 con 0x7f9ffc06eb90 2023-11-28T11:25:38.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.106+0000 7fa00e7fc700 1 -- 172.21.15.92:0/2162586505 <== osd.0 v2:172.21.15.92:6808/115823 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fa0180d3d80 con 0x7f9ffc06eb90 2023-11-28T11:25:38.112 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-28T11:25:38.112 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T11:25:38.112 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-28T11:25:38.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.106+0000 7fa01fbff700 1 -- 172.21.15.92:0/2162586505 >> [v2:172.21.15.92:6808/115823,v1:172.21.15.92:6809/115823] conn(0x7f9ffc06eb90 msgr2=0x7f9ffc070fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:38.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.106+0000 7fa01fbff700 1 --2- 172.21.15.92:0/2162586505 >> [v2:172.21.15.92:6808/115823,v1:172.21.15.92:6809/115823] conn(0x7f9ffc06eb90 0x7f9ffc070fe0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:38.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.107+0000 7fa01fbff700 1 -- 172.21.15.92:0/2162586505 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9ffc0682f0 msgr2=0x7f9ffc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:38.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.107+0000 7fa01fbff700 1 --2- 172.21.15.92:0/2162586505 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f9ffc0682f0 0x7f9ffc06a7a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa008000c00 tx=0x7fa008008030 comp rx=0 tx=0).stop 2023-11-28T11:25:38.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.107+0000 7fa01fbff700 1 -- 172.21.15.92:0/2162586505 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa01806be80 msgr2=0x7fa0180ccd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:38.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.107+0000 7fa01fbff700 1 --2- 172.21.15.92:0/2162586505 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa01806be80 0x7fa0180ccd30 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fa0140011e0 tx=0x7fa01400ca30 comp rx=0 tx=0).stop 2023-11-28T11:25:38.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.107+0000 7fa01e19c700 1 -- 172.21.15.92:0/2162586505 reap_dead start 2023-11-28T11:25:38.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.107+0000 7fa01fbff700 1 -- 172.21.15.92:0/2162586505 shutdown_connections 2023-11-28T11:25:38.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.107+0000 7fa01fbff700 1 -- 172.21.15.92:0/2162586505 >> 172.21.15.92:0/2162586505 conn(0x7fa0180719a0 msgr2=0x7fa0180b7080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:38.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.107+0000 7fa01fbff700 1 -- 172.21.15.92:0/2162586505 shutdown_connections 2023-11-28T11:25:38.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.107+0000 7fa01fbff700 1 -- 172.21.15.92:0/2162586505 wait complete. 2023-11-28T11:25:38.126 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-11-28T11:25:38.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.322+0000 7f07513ff700 1 Processor -- start 2023-11-28T11:25:38.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.322+0000 7f07513ff700 1 -- start start 2023-11-28T11:25:38.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.323+0000 7f07513ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f074c0b8510 0x7f074c06b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:38.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.323+0000 7f07513ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f074c0cd460 0x7f074c06beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:38.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.323+0000 7f07513ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f074c0b88e0 con 0x7f074c06c3f0 2023-11-28T11:25:38.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.323+0000 7f07513ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f074c0cdbd0 con 0x7f074c0cd460 2023-11-28T11:25:38.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.323+0000 7f07513ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f074c0d02d0 con 0x7f074c0b8510 2023-11-28T11:25:38.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.323+0000 7f074a7fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f074c0cd460 0x7f074c06beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:38.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.323+0000 7f074a7fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f074c0cd460 0x7f074c06beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51362/0 (socket says 172.21.15.92:51362) 2023-11-28T11:25:38.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.323+0000 7f074a7fc700 1 -- 172.21.15.92:0/2940419209 learned_addr learned my addr 172.21.15.92:0/2940419209 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:38.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.323+0000 7f074affd700 1 --2- 172.21.15.92:0/2940419209 >> v2:172.21.15.169:3300/0 conn(0x7f074c0b8510 0x7f074c06b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:38.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.324+0000 7f074a7fc700 1 -- 172.21.15.92:0/2940419209 >> v2:172.21.15.169:3300/0 conn(0x7f074c0b8510 msgr2=0x7f074c06b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:38.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.324+0000 7f074a7fc700 1 --2- 172.21.15.92:0/2940419209 >> v2:172.21.15.169:3300/0 conn(0x7f074c0b8510 0x7f074c06b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:38.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.324+0000 7f074a7fc700 1 -- 172.21.15.92:0/2940419209 >> v1:172.21.15.169:6789/0 conn(0x7f074c06c3f0 legacy=0x7f074c15cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:38.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.324+0000 7f074a7fc700 1 -- 172.21.15.92:0/2940419209 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f074c0cd8f0 con 0x7f074c0cd460 2023-11-28T11:25:38.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.324+0000 7f074a7fc700 1 --2- 172.21.15.92:0/2940419209 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f074c0cd460 0x7f074c06beb0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f0740009920 tx=0x7f0740009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=970bbd9eab6f9371 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:38.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.324+0000 7f0749ffb700 1 -- 172.21.15.92:0/2940419209 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0740011830 con 0x7f074c0cd460 2023-11-28T11:25:38.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.324+0000 7f0749ffb700 1 -- 172.21.15.92:0/2940419209 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0740011990 con 0x7f074c0cd460 2023-11-28T11:25:38.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.324+0000 7f0749ffb700 1 -- 172.21.15.92:0/2940419209 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0740011d20 con 0x7f074c0cd460 2023-11-28T11:25:38.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.325+0000 7f07513ff700 1 -- 172.21.15.92:0/2940419209 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f074c0cd460 msgr2=0x7f074c06beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:38.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.325+0000 7f07513ff700 1 --2- 172.21.15.92:0/2940419209 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f074c0cd460 0x7f074c06beb0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f0740009920 tx=0x7f0740009c30 comp rx=0 tx=0).stop 2023-11-28T11:25:38.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.325+0000 7f07513ff700 1 -- 172.21.15.92:0/2940419209 shutdown_connections 2023-11-28T11:25:38.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.325+0000 7f07513ff700 1 --2- 172.21.15.92:0/2940419209 >> v2:172.21.15.169:3300/0 conn(0x7f074c0b8510 0x7f074c06b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:38.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.325+0000 7f07513ff700 1 --2- 172.21.15.92:0/2940419209 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f074c0cd460 0x7f074c06beb0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:38.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.325+0000 7f07513ff700 1 -- 172.21.15.92:0/2940419209 >> 172.21.15.92:0/2940419209 conn(0x7f074c0719a0 msgr2=0x7f074c071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:38.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.325+0000 7f07513ff700 1 -- 172.21.15.92:0/2940419209 shutdown_connections 2023-11-28T11:25:38.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.325+0000 7f07513ff700 1 -- 172.21.15.92:0/2940419209 wait complete. 2023-11-28T11:25:38.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.327+0000 7f07513ff700 1 Processor -- start 2023-11-28T11:25:38.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.327+0000 7f07513ff700 1 -- start start 2023-11-28T11:25:38.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.328+0000 7f07513ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f074c06c3f0 0x7f074c0cb470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:38.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.328+0000 7f07513ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f074c0b8510 0x7f074c1e2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:38.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.328+0000 7f07513ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f074c0cd460 0x7f074c0c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:38.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.328+0000 7f07513ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f074c0c9490 con 0x7f074c0b8510 2023-11-28T11:25:38.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.328+0000 7f07513ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f074c0cbb50 con 0x7f074c0cd460 2023-11-28T11:25:38.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.328+0000 7f07513ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f074c0cbe30 con 0x7f074c06c3f0 2023-11-28T11:25:38.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.328+0000 7f074a7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f074c0b8510 0x7f074c1e2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:38.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.328+0000 7f074a7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f074c0b8510 0x7f074c1e2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51370/0 (socket says 172.21.15.92:51370) 2023-11-28T11:25:38.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.328+0000 7f074a7fc700 1 -- 172.21.15.92:0/582564680 learned_addr learned my addr 172.21.15.92:0/582564680 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:38.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.328+0000 7f074affd700 1 --2- 172.21.15.92:0/582564680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f074c06c3f0 0x7f074c0cb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:38.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.328+0000 7f074b7fe700 1 --2- 172.21.15.92:0/582564680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f074c0cd460 0x7f074c0c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:38.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.328+0000 7f074a7fc700 1 -- 172.21.15.92:0/582564680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f074c06c3f0 msgr2=0x7f074c0cb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:38.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.328+0000 7f074a7fc700 1 --2- 172.21.15.92:0/582564680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f074c06c3f0 0x7f074c0cb470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:38.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.328+0000 7f074a7fc700 1 -- 172.21.15.92:0/582564680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f074c0cd460 msgr2=0x7f074c0c8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:38.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.328+0000 7f074a7fc700 1 --2- 172.21.15.92:0/582564680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f074c0cd460 0x7f074c0c8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:38.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.329+0000 7f074a7fc700 1 -- 172.21.15.92:0/582564680 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0740009550 con 0x7f074c0b8510 2023-11-28T11:25:38.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.329+0000 7f074a7fc700 1 --2- 172.21.15.92:0/582564680 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f074c0b8510 0x7f074c1e2a00 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f0740000c00 tx=0x7f0740004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:38.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.329+0000 7f07337fe700 1 -- 172.21.15.92:0/582564680 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0740004510 con 0x7f074c0b8510 2023-11-28T11:25:38.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.329+0000 7f07337fe700 1 -- 172.21.15.92:0/582564680 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f074001e070 con 0x7f074c0b8510 2023-11-28T11:25:38.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.329+0000 7f07337fe700 1 -- 172.21.15.92:0/582564680 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07400038e0 con 0x7f074c0b8510 2023-11-28T11:25:38.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.329+0000 7f07513ff700 1 -- 172.21.15.92:0/582564680 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f074c0cfff0 con 0x7f074c0b8510 2023-11-28T11:25:38.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.329+0000 7f07513ff700 1 -- 172.21.15.92:0/582564680 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f074c0cfd10 con 0x7f074c0b8510 2023-11-28T11:25:38.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.330+0000 7f07513ff700 1 -- 172.21.15.92:0/582564680 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f074c067880 con 0x7f074c0b8510 2023-11-28T11:25:38.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.331+0000 7f07337fe700 1 -- 172.21.15.92:0/582564680 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f074000f050 con 0x7f074c0b8510 2023-11-28T11:25:38.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.332+0000 7f07337fe700 1 --2- 172.21.15.92:0/582564680 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f07380682f0 0x7f073806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:38.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.332+0000 7f07337fe700 1 -- 172.21.15.92:0/582564680 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f0740086c20 con 0x7f074c0b8510 2023-11-28T11:25:38.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.333+0000 7f074affd700 1 --2- 172.21.15.92:0/582564680 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f07380682f0 0x7f073806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:38.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.333+0000 7f074affd700 1 --2- 172.21.15.92:0/582564680 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f07380682f0 0x7f073806a7a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f073c009be0 tx=0x7f073c009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:38.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.337+0000 7f07337fe700 1 -- 172.21.15.92:0/582564680 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f07400534f0 con 0x7f074c0b8510 2023-11-28T11:25:38.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.525+0000 7f07513ff700 1 -- 172.21.15.92:0/582564680 --> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f074c0cbfa0 con 0x7f07380682f0 2023-11-28T11:25:38.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.525+0000 7f07337fe700 1 -- 172.21.15.92:0/582564680 <== mgr.4103 v2:172.21.15.92:6832/115482 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f074c0cbfa0 con 0x7f07380682f0 2023-11-28T11:25:38.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.527+0000 7f07513ff700 1 -- 172.21.15.92:0/582564680 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f07380682f0 msgr2=0x7f073806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:38.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.527+0000 7f07513ff700 1 --2- 172.21.15.92:0/582564680 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f07380682f0 0x7f073806a7a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f073c009be0 tx=0x7f073c009210 comp rx=0 tx=0).stop 2023-11-28T11:25:38.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.527+0000 7f07513ff700 1 -- 172.21.15.92:0/582564680 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f074c0b8510 msgr2=0x7f074c1e2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:38.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.527+0000 7f07513ff700 1 --2- 172.21.15.92:0/582564680 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f074c0b8510 0x7f074c1e2a00 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f0740000c00 tx=0x7f0740004750 comp rx=0 tx=0).stop 2023-11-28T11:25:38.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.527+0000 7f07513ff700 1 -- 172.21.15.92:0/582564680 shutdown_connections 2023-11-28T11:25:38.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.527+0000 7f07513ff700 1 --2- 172.21.15.92:0/582564680 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f074c0cd460 0x7f074c0c8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:38.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.527+0000 7f07513ff700 1 --2- 172.21.15.92:0/582564680 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f074c06c3f0 0x7f074c0cb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:38.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.527+0000 7f07513ff700 1 --2- 172.21.15.92:0/582564680 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f07380682f0 0x7f073806a7a0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:38.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.527+0000 7f07513ff700 1 --2- 172.21.15.92:0/582564680 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f074c0b8510 0x7f074c1e2a00 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:38.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.527+0000 7f07513ff700 1 -- 172.21.15.92:0/582564680 >> 172.21.15.92:0/582564680 conn(0x7f074c0719a0 msgr2=0x7f074c0b5d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:38.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.527+0000 7f07513ff700 1 -- 172.21.15.92:0/582564680 shutdown_connections 2023-11-28T11:25:38.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.527+0000 7f07513ff700 1 -- 172.21.15.92:0/582564680 wait complete. 2023-11-28T11:25:38.530 INFO:teuthology.orchestra.run.smithi092.stderr:instructed osd(s) 0 to deep-scrub 2023-11-28T11:25:38.546 INFO:tasks.ceph:Scrubbing osd.1 2023-11-28T11:25:38.547 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-11-28T11:25:38.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.745+0000 7f16d23ff700 1 Processor -- start 2023-11-28T11:25:38.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.746+0000 7f16d23ff700 1 -- start start 2023-11-28T11:25:38.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.747+0000 7f16d23ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f16cc0b8790 0x7f16cc0cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:38.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.747+0000 7f16d23ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16cc0cd460 0x7f16cc06b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:38.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.747+0000 7f16d23ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16cc0d00b0 con 0x7f16cc06be80 2023-11-28T11:25:38.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.747+0000 7f16d23ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16cc06c300 con 0x7f16cc0b8790 2023-11-28T11:25:38.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.747+0000 7f16d23ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16cc0b5240 con 0x7f16cc0cd460 2023-11-28T11:25:38.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.748+0000 7f16cbfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f16cc0b8790 0x7f16cc0cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:38.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.748+0000 7f16cbfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f16cc0b8790 0x7f16cc0cfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51386/0 (socket says 172.21.15.92:51386) 2023-11-28T11:25:38.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.748+0000 7f16cbfff700 1 -- 172.21.15.92:0/546242017 learned_addr learned my addr 172.21.15.92:0/546242017 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:38.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.748+0000 7f16cb7fe700 1 --2- 172.21.15.92:0/546242017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16cc0cd460 0x7f16cc06b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:38.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.748+0000 7f16cbfff700 1 -- 172.21.15.92:0/546242017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16cc0cd460 msgr2=0x7f16cc06b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:38.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.748+0000 7f16cbfff700 1 --2- 172.21.15.92:0/546242017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16cc0cd460 0x7f16cc06b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:38.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.748+0000 7f16cbfff700 1 -- 172.21.15.92:0/546242017 >> v1:172.21.15.169:6789/0 conn(0x7f16cc06be80 legacy=0x7f16cc1602b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:38.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.748+0000 7f16cbfff700 1 -- 172.21.15.92:0/546242017 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16cc0b5520 con 0x7f16cc0b8790 2023-11-28T11:25:38.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.748+0000 7f16cbfff700 1 --2- 172.21.15.92:0/546242017 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16cc0b8790 0x7f16cc0cfb70 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f16bc0098f0 tx=0x7f16bc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a3a7474fe7eebf8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:38.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.749+0000 7f16caffd700 1 -- 172.21.15.92:0/546242017 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16bc00b8a0 con 0x7f16cc0b8790 2023-11-28T11:25:38.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.749+0000 7f16caffd700 1 -- 172.21.15.92:0/546242017 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16bc00ba00 con 0x7f16cc0b8790 2023-11-28T11:25:38.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.749+0000 7f16caffd700 1 -- 172.21.15.92:0/546242017 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16bc00bd90 con 0x7f16cc0b8790 2023-11-28T11:25:38.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.749+0000 7f16d23ff700 1 -- 172.21.15.92:0/546242017 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16cc0b8790 msgr2=0x7f16cc0cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:38.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.749+0000 7f16d23ff700 1 --2- 172.21.15.92:0/546242017 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16cc0b8790 0x7f16cc0cfb70 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f16bc0098f0 tx=0x7f16bc009c00 comp rx=0 tx=0).stop 2023-11-28T11:25:38.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.749+0000 7f16d23ff700 1 -- 172.21.15.92:0/546242017 shutdown_connections 2023-11-28T11:25:38.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.749+0000 7f16d23ff700 1 --2- 172.21.15.92:0/546242017 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16cc0b8790 0x7f16cc0cfb70 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:38.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.749+0000 7f16d23ff700 1 --2- 172.21.15.92:0/546242017 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16cc0cd460 0x7f16cc06b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:38.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.749+0000 7f16d23ff700 1 -- 172.21.15.92:0/546242017 >> 172.21.15.92:0/546242017 conn(0x7f16cc0719a0 msgr2=0x7f16cc071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:38.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.749+0000 7f16d23ff700 1 -- 172.21.15.92:0/546242017 shutdown_connections 2023-11-28T11:25:38.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.749+0000 7f16d23ff700 1 -- 172.21.15.92:0/546242017 wait complete. 2023-11-28T11:25:38.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.751+0000 7f16d23ff700 1 Processor -- start 2023-11-28T11:25:38.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.751+0000 7f16d23ff700 1 -- start start 2023-11-28T11:25:38.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.752+0000 7f16d23ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16cc06be80 0x7f16cc1f2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:38.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.752+0000 7f16d23ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16cc0b8790 0x7f16cc1f5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:38.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.752+0000 7f16d23ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16cc0cd460 0x7f16cc0b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:38.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.752+0000 7f16d23ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16cc0a9950 con 0x7f16cc0cd460 2023-11-28T11:25:38.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.752+0000 7f16d23ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16cc1f62a0 con 0x7f16cc06be80 2023-11-28T11:25:38.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.752+0000 7f16d23ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16cc1f6580 con 0x7f16cc0b8790 2023-11-28T11:25:38.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.752+0000 7f16d099c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16cc0cd460 0x7f16cc0b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:38.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.752+0000 7f16d099c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16cc0cd460 0x7f16cc0b62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51390/0 (socket says 172.21.15.92:51390) 2023-11-28T11:25:38.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.752+0000 7f16d099c700 1 -- 172.21.15.92:0/3116647138 learned_addr learned my addr 172.21.15.92:0/3116647138 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:38.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.752+0000 7f16cb7fe700 1 --2- 172.21.15.92:0/3116647138 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16cc0b8790 0x7f16cc1f5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:38.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.752+0000 7f16cbfff700 1 --2- 172.21.15.92:0/3116647138 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16cc06be80 0x7f16cc1f2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:38.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.753+0000 7f16d099c700 1 -- 172.21.15.92:0/3116647138 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16cc0b8790 msgr2=0x7f16cc1f5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:38.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.753+0000 7f16d099c700 1 --2- 172.21.15.92:0/3116647138 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f16cc0b8790 0x7f16cc1f5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:38.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.753+0000 7f16d099c700 1 -- 172.21.15.92:0/3116647138 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16cc06be80 msgr2=0x7f16cc1f2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:38.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.753+0000 7f16d099c700 1 --2- 172.21.15.92:0/3116647138 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f16cc06be80 0x7f16cc1f2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:38.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.753+0000 7f16d099c700 1 -- 172.21.15.92:0/3116647138 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16bc009550 con 0x7f16cc0cd460 2023-11-28T11:25:38.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.753+0000 7f16d099c700 1 --2- 172.21.15.92:0/3116647138 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16cc0cd460 0x7f16cc0b62b0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f16c400cc50 tx=0x7f16c400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:38.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.753+0000 7f16c8ff9700 1 -- 172.21.15.92:0/3116647138 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c400ad80 con 0x7f16cc0cd460 2023-11-28T11:25:38.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.753+0000 7f16c8ff9700 1 -- 172.21.15.92:0/3116647138 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16c400ce50 con 0x7f16cc0cd460 2023-11-28T11:25:38.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.753+0000 7f16c8ff9700 1 -- 172.21.15.92:0/3116647138 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c4003c40 con 0x7f16cc0cd460 2023-11-28T11:25:38.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.753+0000 7f16d23ff700 1 -- 172.21.15.92:0/3116647138 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16cc0b5800 con 0x7f16cc0cd460 2023-11-28T11:25:38.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.753+0000 7f16d23ff700 1 -- 172.21.15.92:0/3116647138 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16cc0d0a80 con 0x7f16cc0cd460 2023-11-28T11:25:38.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.754+0000 7f16d23ff700 1 -- 172.21.15.92:0/3116647138 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f16cc0d1fb0 con 0x7f16cc0cd460 2023-11-28T11:25:38.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.755+0000 7f16c8ff9700 1 -- 172.21.15.92:0/3116647138 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f16c4010050 con 0x7f16cc0cd460 2023-11-28T11:25:38.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.756+0000 7f16c8ff9700 1 --2- 172.21.15.92:0/3116647138 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f16b80682f0 0x7f16b806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:38.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.756+0000 7f16c8ff9700 1 -- 172.21.15.92:0/3116647138 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f16c4085b30 con 0x7f16cc0cd460 2023-11-28T11:25:38.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.756+0000 7f16cbfff700 1 --2- 172.21.15.92:0/3116647138 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f16b80682f0 0x7f16b806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:38.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.756+0000 7f16c8ff9700 1 --2- 172.21.15.92:0/3116647138 >> [v2:172.21.15.92:6810/115824,v1:172.21.15.92:6812/115824] conn(0x7f16b806eb90 0x7f16b8070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:38.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.756+0000 7f16c8ff9700 1 -- 172.21.15.92:0/3116647138 --> [v2:172.21.15.92:6810/115824,v1:172.21.15.92:6812/115824] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f16b80716b0 con 0x7f16b806eb90 2023-11-28T11:25:38.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.756+0000 7f16c8ff9700 1 -- 172.21.15.92:0/3116647138 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16c408a2a0 con 0x7f16cc0cd460 2023-11-28T11:25:38.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.757+0000 7f16cb7fe700 1 --2- 172.21.15.92:0/3116647138 >> [v2:172.21.15.92:6810/115824,v1:172.21.15.92:6812/115824] conn(0x7f16b806eb90 0x7f16b8070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:38.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.757+0000 7f16cbfff700 1 --2- 172.21.15.92:0/3116647138 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f16b80682f0 0x7f16b806a7a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f16bc009c40 tx=0x7f16bc008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:38.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.757+0000 7f16cb7fe700 1 --2- 172.21.15.92:0/3116647138 >> [v2:172.21.15.92:6810/115824,v1:172.21.15.92:6812/115824] conn(0x7f16b806eb90 0x7f16b8070fe0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:38.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.758+0000 7f16c8ff9700 1 -- 172.21.15.92:0/3116647138 <== osd.1 v2:172.21.15.92:6810/115824 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f16b80716b0 con 0x7f16b806eb90 2023-11-28T11:25:38.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.773+0000 7f16d23ff700 1 -- 172.21.15.92:0/3116647138 --> [v2:172.21.15.92:6810/115824,v1:172.21.15.92:6812/115824] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f16cc067880 con 0x7f16b806eb90 2023-11-28T11:25:38.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.773+0000 7f16c8ff9700 1 -- 172.21.15.92:0/3116647138 <== osd.1 v2:172.21.15.92:6810/115824 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f16cc067880 con 0x7f16b806eb90 2023-11-28T11:25:38.774 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-28T11:25:38.775 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T11:25:38.775 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-28T11:25:38.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.773+0000 7f16d23ff700 1 -- 172.21.15.92:0/3116647138 >> [v2:172.21.15.92:6810/115824,v1:172.21.15.92:6812/115824] conn(0x7f16b806eb90 msgr2=0x7f16b8070fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:38.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.773+0000 7f16d23ff700 1 --2- 172.21.15.92:0/3116647138 >> [v2:172.21.15.92:6810/115824,v1:172.21.15.92:6812/115824] conn(0x7f16b806eb90 0x7f16b8070fe0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:38.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.773+0000 7f16d23ff700 1 -- 172.21.15.92:0/3116647138 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f16b80682f0 msgr2=0x7f16b806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:38.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.773+0000 7f16d23ff700 1 --2- 172.21.15.92:0/3116647138 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f16b80682f0 0x7f16b806a7a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f16bc009c40 tx=0x7f16bc008030 comp rx=0 tx=0).stop 2023-11-28T11:25:38.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.773+0000 7f16d23ff700 1 -- 172.21.15.92:0/3116647138 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16cc0cd460 msgr2=0x7f16cc0b62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:38.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.773+0000 7f16d23ff700 1 --2- 172.21.15.92:0/3116647138 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16cc0cd460 0x7f16cc0b62b0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f16c400cc50 tx=0x7f16c400a340 comp rx=0 tx=0).stop 2023-11-28T11:25:38.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.773+0000 7f16d099c700 1 -- 172.21.15.92:0/3116647138 reap_dead start 2023-11-28T11:25:38.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.774+0000 7f16d23ff700 1 -- 172.21.15.92:0/3116647138 shutdown_connections 2023-11-28T11:25:38.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.774+0000 7f16d23ff700 1 -- 172.21.15.92:0/3116647138 >> 172.21.15.92:0/3116647138 conn(0x7f16cc0719a0 msgr2=0x7f16cc0b6fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:38.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.774+0000 7f16d23ff700 1 -- 172.21.15.92:0/3116647138 shutdown_connections 2023-11-28T11:25:38.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.774+0000 7f16d23ff700 1 -- 172.21.15.92:0/3116647138 wait complete. 2023-11-28T11:25:38.792 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-11-28T11:25:38.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.988+0000 7f52f1bff700 1 Processor -- start 2023-11-28T11:25:38.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.989+0000 7f52f1bff700 1 -- start start 2023-11-28T11:25:38.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.990+0000 7f52f1bff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f52ec0b8510 0x7f52ec06b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:38.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.990+0000 7f52f1bff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52ec0cd460 0x7f52ec06beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:38.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.990+0000 7f52f1bff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52ec0b88e0 con 0x7f52ec06c3f0 2023-11-28T11:25:38.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.990+0000 7f52f1bff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52ec0cdbd0 con 0x7f52ec0b8510 2023-11-28T11:25:38.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.990+0000 7f52f1bff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52ec0d02d0 con 0x7f52ec0cd460 2023-11-28T11:25:38.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.990+0000 7f52ebfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f52ec06c3f0 0x7f52ec15cc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56570/0 (socket says 172.21.15.92:56570) 2023-11-28T11:25:38.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.990+0000 7f52ebfff700 1 -- 172.21.15.92:0/1620284966 learned_addr learned my addr 172.21.15.92:0/1620284966 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:25:38.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.990+0000 7f52e3fff700 1 --2- 172.21.15.92:0/1620284966 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52ec0cd460 0x7f52ec06beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:38.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.990+0000 7f52eb7fe700 1 --2- 172.21.15.92:0/1620284966 >> v2:172.21.15.169:3300/0 conn(0x7f52ec0b8510 0x7f52ec06b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:38.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.991+0000 7f52eaffd700 1 -- 172.21.15.92:0/1620284966 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 652086881 0 0) 0x7f52ec0b88e0 con 0x7f52ec06c3f0 2023-11-28T11:25:38.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.991+0000 7f52eaffd700 1 -- 172.21.15.92:0/1620284966 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52d80035b0 con 0x7f52ec06c3f0 2023-11-28T11:25:38.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.991+0000 7f52eaffd700 1 -- 172.21.15.92:0/1620284966 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2976384285 0 0) 0x7f52d80035b0 con 0x7f52ec06c3f0 2023-11-28T11:25:38.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.991+0000 7f52eaffd700 1 -- 172.21.15.92:0/1620284966 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52ec0cd460 msgr2=0x7f52ec06beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:38.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.991+0000 7f52eaffd700 1 --2- 172.21.15.92:0/1620284966 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52ec0cd460 0x7f52ec06beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:38.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.991+0000 7f52eaffd700 1 -- 172.21.15.92:0/1620284966 >> v2:172.21.15.169:3300/0 conn(0x7f52ec0b8510 msgr2=0x7f52ec06b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:38.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.991+0000 7f52eaffd700 1 --2- 172.21.15.92:0/1620284966 >> v2:172.21.15.169:3300/0 conn(0x7f52ec0b8510 0x7f52ec06b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:38.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.991+0000 7f52eaffd700 1 -- 172.21.15.92:0/1620284966 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52ec0cd8f0 con 0x7f52ec06c3f0 2023-11-28T11:25:38.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.991+0000 7f52eaffd700 1 -- 172.21.15.92:0/1620284966 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532161587 0 0) 0x7f52e40019f0 con 0x7f52ec06c3f0 2023-11-28T11:25:38.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.991+0000 7f52eaffd700 1 -- 172.21.15.92:0/1620284966 >> v1:172.21.15.92:6789/0 conn(0x7f52ec06c3f0 legacy=0x7f52ec15cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:38.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.991+0000 7f52eaffd700 1 --2- 172.21.15.92:0/1620284966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52d8003a30 0x7f52d8003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:38.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.991+0000 7f52eaffd700 1 --2- 172.21.15.92:0/1620284966 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52ec0cd460 0x7f52d8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:38.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.991+0000 7f52eaffd700 1 --2- 172.21.15.92:0/1620284966 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52d8004d90 0x7f52d8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:38.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.991+0000 7f52eaffd700 1 -- 172.21.15.92:0/1620284966 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ec0cdbd0 con 0x7f52d8003a30 2023-11-28T11:25:38.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.991+0000 7f52eaffd700 1 -- 172.21.15.92:0/1620284966 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ec0d02d0 con 0x7f52d8004d90 2023-11-28T11:25:38.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.991+0000 7f52eaffd700 1 -- 172.21.15.92:0/1620284966 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52d800ba10 con 0x7f52ec0cd460 2023-11-28T11:25:38.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.991+0000 7f52eb7fe700 1 --2- 172.21.15.92:0/1620284966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52d8003a30 0x7f52d8003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:38.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.991+0000 7f52ebfff700 1 --2- 172.21.15.92:0/1620284966 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52d8004d90 0x7f52d8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:38.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.991+0000 7f52e3fff700 1 --2- 172.21.15.92:0/1620284966 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52ec0cd460 0x7f52d8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:38.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.992+0000 7f52eb7fe700 1 -- 172.21.15.92:0/1620284966 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52ec0cd460 msgr2=0x7f52d8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:38.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.992+0000 7f52eb7fe700 1 --2- 172.21.15.92:0/1620284966 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52ec0cd460 0x7f52d8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:38.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.992+0000 7f52eb7fe700 1 -- 172.21.15.92:0/1620284966 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52d8004d90 msgr2=0x7f52d8005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:38.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.992+0000 7f52eb7fe700 1 --2- 172.21.15.92:0/1620284966 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52d8004d90 0x7f52d8005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:38.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.992+0000 7f52eb7fe700 1 -- 172.21.15.92:0/1620284966 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d80035b0 con 0x7f52d8003a30 2023-11-28T11:25:38.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.992+0000 7f52ebfff700 1 --2- 172.21.15.92:0/1620284966 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52d8004d90 0x7f52d8005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:25:38.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.992+0000 7f52eb7fe700 1 --2- 172.21.15.92:0/1620284966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52d8003a30 0x7f52d8003e40 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f52dc009b60 tx=0x7f52d8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a7c4aad82262489 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:38.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.992+0000 7f52eaffd700 1 -- 172.21.15.92:0/1620284966 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc00e660 con 0x7f52d8003a30 2023-11-28T11:25:38.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.992+0000 7f52eaffd700 1 -- 172.21.15.92:0/1620284966 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52dc00e7c0 con 0x7f52d8003a30 2023-11-28T11:25:38.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.992+0000 7f52eaffd700 1 -- 172.21.15.92:0/1620284966 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc00eb30 con 0x7f52d8003a30 2023-11-28T11:25:39.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.993+0000 7f52f1bff700 1 -- 172.21.15.92:0/1620284966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52d8003a30 msgr2=0x7f52d8003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:39.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.993+0000 7f52f1bff700 1 --2- 172.21.15.92:0/1620284966 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52d8003a30 0x7f52d8003e40 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f52dc009b60 tx=0x7f52d8005880 comp rx=0 tx=0).stop 2023-11-28T11:25:39.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.993+0000 7f52ebfff700 1 -- 172.21.15.92:0/1620284966 reap_dead start 2023-11-28T11:25:39.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.993+0000 7f52f1bff700 1 -- 172.21.15.92:0/1620284966 shutdown_connections 2023-11-28T11:25:39.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.993+0000 7f52f1bff700 1 -- 172.21.15.92:0/1620284966 >> 172.21.15.92:0/1620284966 conn(0x7f52ec0719a0 msgr2=0x7f52ec071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:39.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.993+0000 7f52f1bff700 1 -- 172.21.15.92:0/1620284966 shutdown_connections 2023-11-28T11:25:39.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.993+0000 7f52f1bff700 1 -- 172.21.15.92:0/1620284966 wait complete. 2023-11-28T11:25:39.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.995+0000 7f52f1bff700 1 Processor -- start 2023-11-28T11:25:39.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.995+0000 7f52f1bff700 1 -- start start 2023-11-28T11:25:39.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.996+0000 7f52f1bff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52d8003a30 0x7f52ec1e1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:39.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.996+0000 7f52f1bff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52ec0d1620 0x7f52ec0d1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:39.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.996+0000 7f52f1bff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52ec0c8820 0x7f52ec0c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:39.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.996+0000 7f52f1bff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ec0c9200 con 0x7f52d8003a30 2023-11-28T11:25:39.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.996+0000 7f52f1bff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ec156810 con 0x7f52ec0d1620 2023-11-28T11:25:39.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.996+0000 7f52f1bff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52ec0d2190 con 0x7f52ec0c8820 2023-11-28T11:25:39.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.996+0000 7f52eb7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52d8003a30 0x7f52ec1e1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:39.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.996+0000 7f52eb7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52d8003a30 0x7f52ec1e1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38402/0 (socket says 172.21.15.92:38402) 2023-11-28T11:25:39.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.996+0000 7f52eb7fe700 1 -- 172.21.15.92:0/3206109782 learned_addr learned my addr 172.21.15.92:0/3206109782 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:39.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.996+0000 7f52eb7fe700 1 -- 172.21.15.92:0/3206109782 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52ec0c8820 msgr2=0x7f52ec0c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:39.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.996+0000 7f52e3fff700 1 --2- 172.21.15.92:0/3206109782 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52ec0d1620 0x7f52ec0d1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:39.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.996+0000 7f52ebfff700 1 --2- 172.21.15.92:0/3206109782 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52ec0c8820 0x7f52ec0c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:39.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.996+0000 7f52eb7fe700 1 --2- 172.21.15.92:0/3206109782 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52ec0c8820 0x7f52ec0c8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:39.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.996+0000 7f52eb7fe700 1 -- 172.21.15.92:0/3206109782 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52ec0d1620 msgr2=0x7f52ec0d1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:39.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.996+0000 7f52eb7fe700 1 --2- 172.21.15.92:0/3206109782 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52ec0d1620 0x7f52ec0d1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:39.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.996+0000 7f52eb7fe700 1 -- 172.21.15.92:0/3206109782 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52ec0cfff0 con 0x7f52d8003a30 2023-11-28T11:25:39.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.996+0000 7f52e3fff700 1 --2- 172.21.15.92:0/3206109782 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52ec0d1620 0x7f52ec0d1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:25:39.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.997+0000 7f52eb7fe700 1 --2- 172.21.15.92:0/3206109782 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52d8003a30 0x7f52ec1e1b70 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f52dc0090b0 tx=0x7f52dc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:39.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.997+0000 7f52e8ff9700 1 -- 172.21.15.92:0/3206109782 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc00e660 con 0x7f52d8003a30 2023-11-28T11:25:39.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.997+0000 7f52e8ff9700 1 -- 172.21.15.92:0/3206109782 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52dc003ad0 con 0x7f52d8003a30 2023-11-28T11:25:39.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.997+0000 7f52f1bff700 1 -- 172.21.15.92:0/3206109782 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52ec0cfd10 con 0x7f52d8003a30 2023-11-28T11:25:39.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.997+0000 7f52e8ff9700 1 -- 172.21.15.92:0/3206109782 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc003e80 con 0x7f52d8003a30 2023-11-28T11:25:39.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.997+0000 7f52f1bff700 1 -- 172.21.15.92:0/3206109782 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52ec0d29c0 con 0x7f52d8003a30 2023-11-28T11:25:39.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:38.998+0000 7f52f1bff700 1 -- 172.21.15.92:0/3206109782 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52ec067880 con 0x7f52d8003a30 2023-11-28T11:25:39.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.000+0000 7f52e8ff9700 1 -- 172.21.15.92:0/3206109782 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f52dc019030 con 0x7f52d8003a30 2023-11-28T11:25:39.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.000+0000 7f52e8ff9700 1 --2- 172.21.15.92:0/3206109782 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f52cc0682f0 0x7f52cc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:39.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.000+0000 7f52e8ff9700 1 -- 172.21.15.92:0/3206109782 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f52dc0878e0 con 0x7f52d8003a30 2023-11-28T11:25:39.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.001+0000 7f52e3fff700 1 --2- 172.21.15.92:0/3206109782 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f52cc0682f0 0x7f52cc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:39.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.001+0000 7f52e3fff700 1 --2- 172.21.15.92:0/3206109782 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f52cc0682f0 0x7f52cc06a7a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f52ec1f5460 tx=0x7f52d4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:39.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.004+0000 7f52e8ff9700 1 -- 172.21.15.92:0/3206109782 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f52dc054400 con 0x7f52d8003a30 2023-11-28T11:25:39.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.194+0000 7f52f1bff700 1 -- 172.21.15.92:0/3206109782 --> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f52ec0ce680 con 0x7f52cc0682f0 2023-11-28T11:25:39.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.195+0000 7f52e8ff9700 1 -- 172.21.15.92:0/3206109782 <== mgr.4103 v2:172.21.15.92:6832/115482 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f52ec0ce680 con 0x7f52cc0682f0 2023-11-28T11:25:39.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.197+0000 7f52f1bff700 1 -- 172.21.15.92:0/3206109782 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f52cc0682f0 msgr2=0x7f52cc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:39.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.197+0000 7f52f1bff700 1 --2- 172.21.15.92:0/3206109782 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f52cc0682f0 0x7f52cc06a7a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f52ec1f5460 tx=0x7f52d4008040 comp rx=0 tx=0).stop 2023-11-28T11:25:39.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.197+0000 7f52f1bff700 1 -- 172.21.15.92:0/3206109782 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52d8003a30 msgr2=0x7f52ec1e1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:39.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.197+0000 7f52f1bff700 1 --2- 172.21.15.92:0/3206109782 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52d8003a30 0x7f52ec1e1b70 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f52dc0090b0 tx=0x7f52dc0047c0 comp rx=0 tx=0).stop 2023-11-28T11:25:39.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.197+0000 7f52f1bff700 1 -- 172.21.15.92:0/3206109782 shutdown_connections 2023-11-28T11:25:39.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.197+0000 7f52f1bff700 1 --2- 172.21.15.92:0/3206109782 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f52ec0c8820 0x7f52ec0c8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:39.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.197+0000 7f52f1bff700 1 --2- 172.21.15.92:0/3206109782 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f52cc0682f0 0x7f52cc06a7a0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:39.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.197+0000 7f52f1bff700 1 --2- 172.21.15.92:0/3206109782 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52d8003a30 0x7f52ec1e1b70 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:39.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.197+0000 7f52f1bff700 1 --2- 172.21.15.92:0/3206109782 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f52ec0d1620 0x7f52ec0d1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:39.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.197+0000 7f52f1bff700 1 -- 172.21.15.92:0/3206109782 >> 172.21.15.92:0/3206109782 conn(0x7f52ec0719a0 msgr2=0x7f52ec0b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:39.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.197+0000 7f52f1bff700 1 -- 172.21.15.92:0/3206109782 shutdown_connections 2023-11-28T11:25:39.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.197+0000 7f52f1bff700 1 -- 172.21.15.92:0/3206109782 wait complete. 2023-11-28T11:25:39.200 INFO:teuthology.orchestra.run.smithi092.stderr:instructed osd(s) 1 to deep-scrub 2023-11-28T11:25:39.216 INFO:tasks.ceph:Scrubbing osd.2 2023-11-28T11:25:39.216 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-11-28T11:25:39.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.418+0000 7f58c89ff700 1 Processor -- start 2023-11-28T11:25:39.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.418+0000 7f58c89ff700 1 -- start start 2023-11-28T11:25:39.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.419+0000 7f58c89ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f58c40cd460 0x7f58c406b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:39.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.419+0000 7f58c89ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58c40b5240 con 0x7f58c406be80 2023-11-28T11:25:39.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.419+0000 7f58c89ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58c40b5520 con 0x7f58c40b8790 2023-11-28T11:25:39.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.419+0000 7f58c89ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58c406c300 con 0x7f58c40cd460 2023-11-28T11:25:39.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.419+0000 7f58c2d9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f58c406be80 0x7f58c40d0fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56580/0 (socket says 172.21.15.92:56580) 2023-11-28T11:25:39.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.419+0000 7f58c2d9d700 1 -- 172.21.15.92:0/1405889421 learned_addr learned my addr 172.21.15.92:0/1405889421 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:25:39.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.419+0000 7f58bbfff700 1 --2- 172.21.15.92:0/1405889421 >> v2:172.21.15.169:3300/0 conn(0x7f58c40cd460 0x7f58c406b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:39.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.420+0000 7f58c1d9b700 1 -- 172.21.15.92:0/1405889421 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 772255866 0 0) 0x7f58c40b5240 con 0x7f58c406be80 2023-11-28T11:25:39.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.420+0000 7f58c1d9b700 1 -- 172.21.15.92:0/1405889421 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58b00035b0 con 0x7f58c406be80 2023-11-28T11:25:39.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.420+0000 7f58bbfff700 1 -- 172.21.15.92:0/1405889421 >> v1:172.21.15.169:6789/0 conn(0x7f58c40b8790 legacy=0x7f58c40cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:39.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.420+0000 7f58bbfff700 1 -- 172.21.15.92:0/1405889421 >> v1:172.21.15.92:6789/0 conn(0x7f58c406be80 legacy=0x7f58c40d0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:39.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.420+0000 7f58bbfff700 1 -- 172.21.15.92:0/1405889421 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c40d0a80 con 0x7f58c40cd460 2023-11-28T11:25:39.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.420+0000 7f58bbfff700 1 --2- 172.21.15.92:0/1405889421 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58c40cd460 0x7f58c406b940 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f58ac009870 tx=0x7f58ac009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1caf498dfd814da server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:39.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.420+0000 7f58c1d9b700 1 -- 172.21.15.92:0/1405889421 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac011830 con 0x7f58c40cd460 2023-11-28T11:25:39.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.420+0000 7f58c1d9b700 1 -- 172.21.15.92:0/1405889421 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58ac011990 con 0x7f58c40cd460 2023-11-28T11:25:39.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.420+0000 7f58c1d9b700 1 -- 172.21.15.92:0/1405889421 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac011d20 con 0x7f58c40cd460 2023-11-28T11:25:39.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.421+0000 7f58c89ff700 1 -- 172.21.15.92:0/1405889421 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58c40cd460 msgr2=0x7f58c406b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:39.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.421+0000 7f58c89ff700 1 --2- 172.21.15.92:0/1405889421 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58c40cd460 0x7f58c406b940 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f58ac009870 tx=0x7f58ac009b80 comp rx=0 tx=0).stop 2023-11-28T11:25:39.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.421+0000 7f58c89ff700 1 -- 172.21.15.92:0/1405889421 shutdown_connections 2023-11-28T11:25:39.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.421+0000 7f58c89ff700 1 --2- 172.21.15.92:0/1405889421 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58c40cd460 0x7f58c406b940 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:39.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.421+0000 7f58c89ff700 1 -- 172.21.15.92:0/1405889421 >> 172.21.15.92:0/1405889421 conn(0x7f58c40719a0 msgr2=0x7f58c4071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:39.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.421+0000 7f58c89ff700 1 -- 172.21.15.92:0/1405889421 shutdown_connections 2023-11-28T11:25:39.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.421+0000 7f58c89ff700 1 -- 172.21.15.92:0/1405889421 wait complete. 2023-11-28T11:25:39.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.423+0000 7f58c89ff700 1 Processor -- start 2023-11-28T11:25:39.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.423+0000 7f58c89ff700 1 -- start start 2023-11-28T11:25:39.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.424+0000 7f58c89ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58c406be80 0x7f58c415cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:39.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.424+0000 7f58c89ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58c40b8790 0x7f58c4156fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:39.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.424+0000 7f58c89ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58c40cd460 0x7f58c415e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:39.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.424+0000 7f58c89ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c40ccd40 con 0x7f58c40cd460 2023-11-28T11:25:39.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.424+0000 7f58c89ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c4157500 con 0x7f58c40b8790 2023-11-28T11:25:39.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.424+0000 7f58c89ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58c415d380 con 0x7f58c406be80 2023-11-28T11:25:39.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.424+0000 7f58c2d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58c40cd460 0x7f58c415e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:39.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.424+0000 7f58c2d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58c40cd460 0x7f58c415e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38404/0 (socket says 172.21.15.92:38404) 2023-11-28T11:25:39.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.424+0000 7f58c2d9d700 1 -- 172.21.15.92:0/3561553850 learned_addr learned my addr 172.21.15.92:0/3561553850 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:39.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.424+0000 7f58c259c700 1 --2- 172.21.15.92:0/3561553850 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58c406be80 0x7f58c415cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:39.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.424+0000 7f58bbfff700 1 --2- 172.21.15.92:0/3561553850 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58c40b8790 0x7f58c4156fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:39.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.425+0000 7f58c2d9d700 1 -- 172.21.15.92:0/3561553850 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58c406be80 msgr2=0x7f58c415cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:39.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.425+0000 7f58c2d9d700 1 --2- 172.21.15.92:0/3561553850 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f58c406be80 0x7f58c415cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:39.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.425+0000 7f58c2d9d700 1 -- 172.21.15.92:0/3561553850 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58c40b8790 msgr2=0x7f58c4156fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:39.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.425+0000 7f58c2d9d700 1 --2- 172.21.15.92:0/3561553850 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f58c40b8790 0x7f58c4156fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:39.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.425+0000 7f58c2d9d700 1 -- 172.21.15.92:0/3561553850 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ac0094a0 con 0x7f58c40cd460 2023-11-28T11:25:39.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.425+0000 7f58c2d9d700 1 --2- 172.21.15.92:0/3561553850 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58c40cd460 0x7f58c415e420 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f58bc00cb40 tx=0x7f58bc00ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:39.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.425+0000 7f58baffd700 1 -- 172.21.15.92:0/3561553850 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58bc00a9e0 con 0x7f58c40cd460 2023-11-28T11:25:39.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.425+0000 7f58baffd700 1 -- 172.21.15.92:0/3561553850 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58bc00ab40 con 0x7f58c40cd460 2023-11-28T11:25:39.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.425+0000 7f58c89ff700 1 -- 172.21.15.92:0/3561553850 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58c40d16b0 con 0x7f58c40cd460 2023-11-28T11:25:39.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.425+0000 7f58baffd700 1 -- 172.21.15.92:0/3561553850 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58bc012400 con 0x7f58c40cd460 2023-11-28T11:25:39.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.425+0000 7f58c89ff700 1 -- 172.21.15.92:0/3561553850 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58c40b5800 con 0x7f58c40cd460 2023-11-28T11:25:39.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.426+0000 7f58c89ff700 1 -- 172.21.15.92:0/3561553850 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f58c40d1840 con 0x7f58c40cd460 2023-11-28T11:25:39.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.427+0000 7f58baffd700 1 -- 172.21.15.92:0/3561553850 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f58bc019030 con 0x7f58c40cd460 2023-11-28T11:25:39.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.428+0000 7f58baffd700 1 --2- 172.21.15.92:0/3561553850 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f58a40682f0 0x7f58a406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:39.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.428+0000 7f58baffd700 1 -- 172.21.15.92:0/3561553850 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f58bc085590 con 0x7f58c40cd460 2023-11-28T11:25:39.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.428+0000 7f58c259c700 1 --2- 172.21.15.92:0/3561553850 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f58a40682f0 0x7f58a406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:39.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.428+0000 7f58baffd700 1 --2- 172.21.15.92:0/3561553850 >> [v2:172.21.15.92:6800/115821,v1:172.21.15.92:6801/115821] conn(0x7f58a406eb90 0x7f58a4070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:39.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.428+0000 7f58baffd700 1 -- 172.21.15.92:0/3561553850 --> [v2:172.21.15.92:6800/115821,v1:172.21.15.92:6801/115821] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f58a40716b0 con 0x7f58a406eb90 2023-11-28T11:25:39.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.428+0000 7f58baffd700 1 -- 172.21.15.92:0/3561553850 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58bc087b70 con 0x7f58c40cd460 2023-11-28T11:25:39.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.429+0000 7f58bbfff700 1 --2- 172.21.15.92:0/3561553850 >> [v2:172.21.15.92:6800/115821,v1:172.21.15.92:6801/115821] conn(0x7f58a406eb90 0x7f58a4070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:39.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.429+0000 7f58c259c700 1 --2- 172.21.15.92:0/3561553850 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f58a40682f0 0x7f58a406a7a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f58b4007fd0 tx=0x7f58b4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:39.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.429+0000 7f58bbfff700 1 --2- 172.21.15.92:0/3561553850 >> [v2:172.21.15.92:6800/115821,v1:172.21.15.92:6801/115821] conn(0x7f58a406eb90 0x7f58a4070fe0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:39.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.430+0000 7f58baffd700 1 -- 172.21.15.92:0/3561553850 <== osd.2 v2:172.21.15.92:6800/115821 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f58a40716b0 con 0x7f58a406eb90 2023-11-28T11:25:39.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.445+0000 7f58c89ff700 1 -- 172.21.15.92:0/3561553850 --> [v2:172.21.15.92:6800/115821,v1:172.21.15.92:6801/115821] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f58c4067880 con 0x7f58a406eb90 2023-11-28T11:25:39.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.445+0000 7f58baffd700 1 -- 172.21.15.92:0/3561553850 <== osd.2 v2:172.21.15.92:6800/115821 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f58c4067880 con 0x7f58a406eb90 2023-11-28T11:25:39.446 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-28T11:25:39.447 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T11:25:39.447 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-28T11:25:39.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.445+0000 7f58c89ff700 1 -- 172.21.15.92:0/3561553850 >> [v2:172.21.15.92:6800/115821,v1:172.21.15.92:6801/115821] conn(0x7f58a406eb90 msgr2=0x7f58a4070fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:39.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.445+0000 7f58c89ff700 1 --2- 172.21.15.92:0/3561553850 >> [v2:172.21.15.92:6800/115821,v1:172.21.15.92:6801/115821] conn(0x7f58a406eb90 0x7f58a4070fe0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:39.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.446+0000 7f58c89ff700 1 -- 172.21.15.92:0/3561553850 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f58a40682f0 msgr2=0x7f58a406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:39.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.446+0000 7f58c89ff700 1 --2- 172.21.15.92:0/3561553850 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f58a40682f0 0x7f58a406a7a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f58b4007fd0 tx=0x7f58b4009040 comp rx=0 tx=0).stop 2023-11-28T11:25:39.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.446+0000 7f58c89ff700 1 -- 172.21.15.92:0/3561553850 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58c40cd460 msgr2=0x7f58c415e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:39.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.446+0000 7f58c89ff700 1 --2- 172.21.15.92:0/3561553850 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58c40cd460 0x7f58c415e420 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f58bc00cb40 tx=0x7f58bc00ce50 comp rx=0 tx=0).stop 2023-11-28T11:25:39.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.446+0000 7f58c2d9d700 1 -- 172.21.15.92:0/3561553850 reap_dead start 2023-11-28T11:25:39.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.446+0000 7f58c89ff700 1 -- 172.21.15.92:0/3561553850 shutdown_connections 2023-11-28T11:25:39.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.446+0000 7f58c89ff700 1 -- 172.21.15.92:0/3561553850 >> 172.21.15.92:0/3561553850 conn(0x7f58c40719a0 msgr2=0x7f58c40b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:39.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.446+0000 7f58c89ff700 1 -- 172.21.15.92:0/3561553850 shutdown_connections 2023-11-28T11:25:39.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.446+0000 7f58c89ff700 1 -- 172.21.15.92:0/3561553850 wait complete. 2023-11-28T11:25:39.464 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-11-28T11:25:39.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.662+0000 7fb6e6dff700 1 Processor -- start 2023-11-28T11:25:39.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.662+0000 7fb6e6dff700 1 -- start start 2023-11-28T11:25:39.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.663+0000 7fb6e6dff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb6e00b8510 0x7fb6e006b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:39.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.663+0000 7fb6e6dff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fb6e00d0d60 0x7fb6e00d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:39.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.663+0000 7fb6e6dff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6e00b88e0 con 0x7fb6e00cd460 2023-11-28T11:25:39.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.663+0000 7fb6e6dff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6e00cdbd0 con 0x7fb6e00b8510 2023-11-28T11:25:39.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.663+0000 7fb6e6dff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6e00d02d0 con 0x7fb6e00d0d60 2023-11-28T11:25:39.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.663+0000 7fb6e4b9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb6e00b8510 0x7fb6e006b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:39.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.663+0000 7fb6dffff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb6e00cd460 0x7fb6e006beb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56588/0 (socket says 172.21.15.92:56588) 2023-11-28T11:25:39.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.663+0000 7fb6dffff700 1 -- 172.21.15.92:0/1208035053 learned_addr learned my addr 172.21.15.92:0/1208035053 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:25:39.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.663+0000 7fb6e539c700 1 --2- 172.21.15.92:0/1208035053 >> v2:172.21.15.169:3300/0 conn(0x7fb6e00d0d60 0x7fb6e00d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:39.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.664+0000 7fb6e4b9b700 1 -- 172.21.15.92:0/1208035053 >> v2:172.21.15.169:3300/0 conn(0x7fb6e00d0d60 msgr2=0x7fb6e00d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:39.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.664+0000 7fb6e4b9b700 1 --2- 172.21.15.92:0/1208035053 >> v2:172.21.15.169:3300/0 conn(0x7fb6e00d0d60 0x7fb6e00d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:39.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.664+0000 7fb6e4b9b700 1 -- 172.21.15.92:0/1208035053 >> v1:172.21.15.92:6789/0 conn(0x7fb6e00cd460 legacy=0x7fb6e006beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:39.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.664+0000 7fb6e4b9b700 1 -- 172.21.15.92:0/1208035053 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6e00cd8f0 con 0x7fb6e00b8510 2023-11-28T11:25:39.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.664+0000 7fb6df7fe700 1 -- 172.21.15.92:0/1208035053 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 502211255 0 0) 0x7fb6e00b88e0 con 0x7fb6e00cd460 2023-11-28T11:25:39.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.664+0000 7fb6e4b9b700 1 --2- 172.21.15.92:0/1208035053 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6e00b8510 0x7fb6e006b970 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fb6e00ce8c0 tx=0x7fb6d4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fc242d7965c3cab server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:39.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.664+0000 7fb6df7fe700 1 -- 172.21.15.92:0/1208035053 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d4011830 con 0x7fb6e00b8510 2023-11-28T11:25:39.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.664+0000 7fb6df7fe700 1 -- 172.21.15.92:0/1208035053 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6d4011990 con 0x7fb6e00b8510 2023-11-28T11:25:39.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.664+0000 7fb6df7fe700 1 -- 172.21.15.92:0/1208035053 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d4011d20 con 0x7fb6e00b8510 2023-11-28T11:25:39.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.665+0000 7fb6e6dff700 1 -- 172.21.15.92:0/1208035053 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6e00b8510 msgr2=0x7fb6e006b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:39.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.665+0000 7fb6e6dff700 1 --2- 172.21.15.92:0/1208035053 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6e00b8510 0x7fb6e006b970 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fb6e00ce8c0 tx=0x7fb6d4009b50 comp rx=0 tx=0).stop 2023-11-28T11:25:39.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.665+0000 7fb6e6dff700 1 -- 172.21.15.92:0/1208035053 shutdown_connections 2023-11-28T11:25:39.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.665+0000 7fb6e6dff700 1 --2- 172.21.15.92:0/1208035053 >> v2:172.21.15.169:3300/0 conn(0x7fb6e00d0d60 0x7fb6e00d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:39.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.665+0000 7fb6e6dff700 1 --2- 172.21.15.92:0/1208035053 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6e00b8510 0x7fb6e006b970 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:39.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.665+0000 7fb6e6dff700 1 -- 172.21.15.92:0/1208035053 >> 172.21.15.92:0/1208035053 conn(0x7fb6e00719a0 msgr2=0x7fb6e0071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:39.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.665+0000 7fb6e6dff700 1 -- 172.21.15.92:0/1208035053 shutdown_connections 2023-11-28T11:25:39.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.665+0000 7fb6e6dff700 1 -- 172.21.15.92:0/1208035053 wait complete. 2023-11-28T11:25:39.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.667+0000 7fb6e6dff700 1 Processor -- start 2023-11-28T11:25:39.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.667+0000 7fb6e6dff700 1 -- start start 2023-11-28T11:25:39.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.668+0000 7fb6e6dff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6e00b8510 0x7fb6e0159070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:39.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.668+0000 7fb6e6dff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6e00cd460 0x7fb6e01e5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:39.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.668+0000 7fb6e6dff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6e00d0d60 0x7fb6e00b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:39.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.668+0000 7fb6e6dff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e01595b0 con 0x7fb6e00d0d60 2023-11-28T11:25:39.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.668+0000 7fb6e6dff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e01e5940 con 0x7fb6e00b8510 2023-11-28T11:25:39.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.668+0000 7fb6e6dff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e01e5c20 con 0x7fb6e00cd460 2023-11-28T11:25:39.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.668+0000 7fb6e539c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6e00d0d60 0x7fb6e00b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:39.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.668+0000 7fb6e539c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6e00d0d60 0x7fb6e00b31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38414/0 (socket says 172.21.15.92:38414) 2023-11-28T11:25:39.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.668+0000 7fb6e539c700 1 -- 172.21.15.92:0/4056394364 learned_addr learned my addr 172.21.15.92:0/4056394364 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:39.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.668+0000 7fb6e4b9b700 1 --2- 172.21.15.92:0/4056394364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6e00b8510 0x7fb6e0159070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:39.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.668+0000 7fb6e539c700 1 -- 172.21.15.92:0/4056394364 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6e00cd460 msgr2=0x7fb6e01e5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:39.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.668+0000 7fb6e539c700 1 --2- 172.21.15.92:0/4056394364 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6e00cd460 0x7fb6e01e5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:39.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.668+0000 7fb6e539c700 1 -- 172.21.15.92:0/4056394364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6e00b8510 msgr2=0x7fb6e0159070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:39.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.668+0000 7fb6e539c700 1 --2- 172.21.15.92:0/4056394364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6e00b8510 0x7fb6e0159070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:39.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.668+0000 7fb6e539c700 1 -- 172.21.15.92:0/4056394364 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d40094a0 con 0x7fb6e00d0d60 2023-11-28T11:25:39.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.669+0000 7fb6e539c700 1 --2- 172.21.15.92:0/4056394364 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6e00d0d60 0x7fb6e00b31f0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fb6d8007e60 tx=0x7fb6d800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:39.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.669+0000 7fb6dd7fa700 1 -- 172.21.15.92:0/4056394364 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d800ede0 con 0x7fb6e00d0d60 2023-11-28T11:25:39.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.669+0000 7fb6dd7fa700 1 -- 172.21.15.92:0/4056394364 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6d8014420 con 0x7fb6e00d0d60 2023-11-28T11:25:39.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.669+0000 7fb6e6dff700 1 -- 172.21.15.92:0/4056394364 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6e00cfff0 con 0x7fb6e00d0d60 2023-11-28T11:25:39.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.669+0000 7fb6dd7fa700 1 -- 172.21.15.92:0/4056394364 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d80147d0 con 0x7fb6e00d0d60 2023-11-28T11:25:39.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.669+0000 7fb6e6dff700 1 -- 172.21.15.92:0/4056394364 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6e00cfd10 con 0x7fb6e00d0d60 2023-11-28T11:25:39.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.670+0000 7fb6e6dff700 1 -- 172.21.15.92:0/4056394364 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6e01e6550 con 0x7fb6e00d0d60 2023-11-28T11:25:39.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.671+0000 7fb6dd7fa700 1 -- 172.21.15.92:0/4056394364 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb6d8012050 con 0x7fb6e00d0d60 2023-11-28T11:25:39.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.672+0000 7fb6dd7fa700 1 --2- 172.21.15.92:0/4056394364 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb6c40682f0 0x7fb6c406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:39.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.672+0000 7fb6dd7fa700 1 -- 172.21.15.92:0/4056394364 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fb6d80860f0 con 0x7fb6e00d0d60 2023-11-28T11:25:39.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.672+0000 7fb6e4b9b700 1 --2- 172.21.15.92:0/4056394364 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb6c40682f0 0x7fb6c406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:39.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.673+0000 7fb6e4b9b700 1 --2- 172.21.15.92:0/4056394364 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb6c40682f0 0x7fb6c406a7a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb6d40090b0 tx=0x7fb6d4008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:39.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.676+0000 7fb6dd7fa700 1 -- 172.21.15.92:0/4056394364 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb6d8052b10 con 0x7fb6e00d0d60 2023-11-28T11:25:39.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.866+0000 7fb6e6dff700 1 -- 172.21.15.92:0/4056394364 --> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fb6e01e32a0 con 0x7fb6c40682f0 2023-11-28T11:25:39.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.866+0000 7fb6dd7fa700 1 -- 172.21.15.92:0/4056394364 <== mgr.4103 v2:172.21.15.92:6832/115482 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fb6e01e32a0 con 0x7fb6c40682f0 2023-11-28T11:25:39.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.868+0000 7fb6e6dff700 1 -- 172.21.15.92:0/4056394364 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb6c40682f0 msgr2=0x7fb6c406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:39.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.868+0000 7fb6e6dff700 1 --2- 172.21.15.92:0/4056394364 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb6c40682f0 0x7fb6c406a7a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb6d40090b0 tx=0x7fb6d4008030 comp rx=0 tx=0).stop 2023-11-28T11:25:39.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.868+0000 7fb6e6dff700 1 -- 172.21.15.92:0/4056394364 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6e00d0d60 msgr2=0x7fb6e00b31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:39.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.868+0000 7fb6e6dff700 1 --2- 172.21.15.92:0/4056394364 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6e00d0d60 0x7fb6e00b31f0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fb6d8007e60 tx=0x7fb6d800e3c0 comp rx=0 tx=0).stop 2023-11-28T11:25:39.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.869+0000 7fb6e6dff700 1 -- 172.21.15.92:0/4056394364 shutdown_connections 2023-11-28T11:25:39.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.869+0000 7fb6e6dff700 1 --2- 172.21.15.92:0/4056394364 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fb6c40682f0 0x7fb6c406a7a0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:39.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.869+0000 7fb6e6dff700 1 --2- 172.21.15.92:0/4056394364 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6e00d0d60 0x7fb6e00b31f0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:39.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.869+0000 7fb6e6dff700 1 --2- 172.21.15.92:0/4056394364 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fb6e00b8510 0x7fb6e0159070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:39.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.869+0000 7fb6e6dff700 1 --2- 172.21.15.92:0/4056394364 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fb6e00cd460 0x7fb6e01e5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:39.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.869+0000 7fb6e6dff700 1 -- 172.21.15.92:0/4056394364 >> 172.21.15.92:0/4056394364 conn(0x7fb6e00719a0 msgr2=0x7fb6e00b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:39.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.869+0000 7fb6e6dff700 1 -- 172.21.15.92:0/4056394364 shutdown_connections 2023-11-28T11:25:39.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:39.869+0000 7fb6e6dff700 1 -- 172.21.15.92:0/4056394364 wait complete. 2023-11-28T11:25:39.872 INFO:teuthology.orchestra.run.smithi092.stderr:instructed osd(s) 2 to deep-scrub 2023-11-28T11:25:39.888 INFO:tasks.ceph:Scrubbing osd.3 2023-11-28T11:25:39.888 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-11-28T11:25:40.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.087+0000 7fa6cd3ff700 1 Processor -- start 2023-11-28T11:25:40.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.087+0000 7fa6cd3ff700 1 -- start start 2023-11-28T11:25:40.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.088+0000 7fa6cd3ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6c80b8790 0x7fa6c80cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:40.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.088+0000 7fa6cd3ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6c80d00b0 con 0x7fa6c80cd460 2023-11-28T11:25:40.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.088+0000 7fa6cd3ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6c80b5240 con 0x7fa6c806c050 2023-11-28T11:25:40.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.088+0000 7fa6cd3ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c80b5520 con 0x7fa6c80b8790 2023-11-28T11:25:40.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.089+0000 7fa6c77fe700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7fa6c806c050 0x7fa6c80d0fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.92:42684/0 (socket says 172.21.15.92:42684) 2023-11-28T11:25:40.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.089+0000 7fa6c77fe700 1 -- 172.21.15.92:0/3609588623 learned_addr learned my addr 172.21.15.92:0/3609588623 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:25:40.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.089+0000 7fa6bffff700 1 --1- 172.21.15.92:0/3609588623 >> v1:172.21.15.92:6789/0 conn(0x7fa6c80cd460 0x7fa6c806b940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56598/0 (socket says 172.21.15.92:56598) 2023-11-28T11:25:40.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.089+0000 7fa6c6ffd700 1 --2- 172.21.15.92:0/3609588623 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6c80b8790 0x7fa6c80cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:40.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.089+0000 7fa6c67fc700 1 -- 172.21.15.92:0/3609588623 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3446040814 0 0) 0x7fa6c80b5240 con 0x7fa6c806c050 2023-11-28T11:25:40.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.089+0000 7fa6c67fc700 1 -- 172.21.15.92:0/3609588623 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6b40035b0 con 0x7fa6c806c050 2023-11-28T11:25:40.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.089+0000 7fa6c67fc700 1 -- 172.21.15.92:0/3609588623 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2929025135 0 0) 0x7fa6c80d00b0 con 0x7fa6c80cd460 2023-11-28T11:25:40.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.089+0000 7fa6c67fc700 1 -- 172.21.15.92:0/3609588623 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6c80b5240 con 0x7fa6c80cd460 2023-11-28T11:25:40.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.089+0000 7fa6c6ffd700 1 -- 172.21.15.92:0/3609588623 >> v1:172.21.15.169:6789/0 conn(0x7fa6c806c050 legacy=0x7fa6c80d0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:40.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.089+0000 7fa6c6ffd700 1 -- 172.21.15.92:0/3609588623 >> v1:172.21.15.92:6789/0 conn(0x7fa6c80cd460 legacy=0x7fa6c806b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:40.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.090+0000 7fa6c6ffd700 1 -- 172.21.15.92:0/3609588623 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c80d0a80 con 0x7fa6c80b8790 2023-11-28T11:25:40.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.090+0000 7fa6c6ffd700 1 --2- 172.21.15.92:0/3609588623 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6c80b8790 0x7fa6c80cfb70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8009870 tx=0x7fa6b8009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e787d7fcf0cea5c3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:40.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.090+0000 7fa6c67fc700 1 -- 172.21.15.92:0/3609588623 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b8011830 con 0x7fa6c80b8790 2023-11-28T11:25:40.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.090+0000 7fa6c67fc700 1 -- 172.21.15.92:0/3609588623 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6b8011990 con 0x7fa6c80b8790 2023-11-28T11:25:40.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.090+0000 7fa6c67fc700 1 -- 172.21.15.92:0/3609588623 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b8011d20 con 0x7fa6c80b8790 2023-11-28T11:25:40.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.091+0000 7fa6cd3ff700 1 -- 172.21.15.92:0/3609588623 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6c80b8790 msgr2=0x7fa6c80cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:40.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.091+0000 7fa6cd3ff700 1 --2- 172.21.15.92:0/3609588623 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6c80b8790 0x7fa6c80cfb70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8009870 tx=0x7fa6b8009b80 comp rx=0 tx=0).stop 2023-11-28T11:25:40.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.091+0000 7fa6cd3ff700 1 -- 172.21.15.92:0/3609588623 shutdown_connections 2023-11-28T11:25:40.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.091+0000 7fa6cd3ff700 1 --2- 172.21.15.92:0/3609588623 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6c80b8790 0x7fa6c80cfb70 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:40.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.091+0000 7fa6cd3ff700 1 -- 172.21.15.92:0/3609588623 >> 172.21.15.92:0/3609588623 conn(0x7fa6c80719a0 msgr2=0x7fa6c8071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:40.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.091+0000 7fa6cd3ff700 1 -- 172.21.15.92:0/3609588623 shutdown_connections 2023-11-28T11:25:40.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.091+0000 7fa6cd3ff700 1 -- 172.21.15.92:0/3609588623 wait complete. 2023-11-28T11:25:40.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.092+0000 7fa6cd3ff700 1 Processor -- start 2023-11-28T11:25:40.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.093+0000 7fa6cd3ff700 1 -- start start 2023-11-28T11:25:40.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.093+0000 7fa6cd3ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6c806c050 0x7fa6c80c8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:40.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.093+0000 7fa6cd3ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6c80b8790 0x7fa6c80b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:40.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.094+0000 7fa6cd3ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6c80cd460 0x7fa6c80b4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:40.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.094+0000 7fa6cd3ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c8151ad0 con 0x7fa6c80cd460 2023-11-28T11:25:40.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.094+0000 7fa6cd3ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c8152650 con 0x7fa6c806c050 2023-11-28T11:25:40.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.094+0000 7fa6cd3ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c8151d20 con 0x7fa6c80b8790 2023-11-28T11:25:40.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.094+0000 7fa6c77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6c80cd460 0x7fa6c80b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:40.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.094+0000 7fa6c77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6c80cd460 0x7fa6c80b4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38426/0 (socket says 172.21.15.92:38426) 2023-11-28T11:25:40.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.094+0000 7fa6c77fe700 1 -- 172.21.15.92:0/299276536 learned_addr learned my addr 172.21.15.92:0/299276536 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:40.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.094+0000 7fa6c6ffd700 1 --2- 172.21.15.92:0/299276536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6c806c050 0x7fa6c80c8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:40.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.094+0000 7fa6bffff700 1 --2- 172.21.15.92:0/299276536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6c80b8790 0x7fa6c80b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:40.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.094+0000 7fa6c77fe700 1 -- 172.21.15.92:0/299276536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6c80b8790 msgr2=0x7fa6c80b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:40.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.094+0000 7fa6c77fe700 1 --2- 172.21.15.92:0/299276536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6c80b8790 0x7fa6c80b3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:40.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.094+0000 7fa6c77fe700 1 -- 172.21.15.92:0/299276536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6c806c050 msgr2=0x7fa6c80c8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:40.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.094+0000 7fa6c77fe700 1 --2- 172.21.15.92:0/299276536 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa6c806c050 0x7fa6c80c8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:40.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.094+0000 7fa6c77fe700 1 -- 172.21.15.92:0/299276536 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b80094a0 con 0x7fa6c80cd460 2023-11-28T11:25:40.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.094+0000 7fa6bffff700 1 --2- 172.21.15.92:0/299276536 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa6c80b8790 0x7fa6c80b3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:25:40.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.095+0000 7fa6c77fe700 1 --2- 172.21.15.92:0/299276536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6c80cd460 0x7fa6c80b4320 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fa6c000cba0 tx=0x7fa6c000ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:40.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.095+0000 7fa6bf7fe700 1 -- 172.21.15.92:0/299276536 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c000aa30 con 0x7fa6c80cd460 2023-11-28T11:25:40.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.095+0000 7fa6bf7fe700 1 -- 172.21.15.92:0/299276536 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6c000ab90 con 0x7fa6c80cd460 2023-11-28T11:25:40.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.095+0000 7fa6bf7fe700 1 -- 172.21.15.92:0/299276536 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c0012400 con 0x7fa6c80cd460 2023-11-28T11:25:40.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.095+0000 7fa6cd3ff700 1 -- 172.21.15.92:0/299276536 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6c80d16b0 con 0x7fa6c80cd460 2023-11-28T11:25:40.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.095+0000 7fa6cd3ff700 1 -- 172.21.15.92:0/299276536 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6c80b5800 con 0x7fa6c80cd460 2023-11-28T11:25:40.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.096+0000 7fa6cd3ff700 1 -- 172.21.15.92:0/299276536 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa6c80d1840 con 0x7fa6c80cd460 2023-11-28T11:25:40.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.097+0000 7fa6bf7fe700 1 -- 172.21.15.92:0/299276536 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa6c0019030 con 0x7fa6c80cd460 2023-11-28T11:25:40.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.098+0000 7fa6bf7fe700 1 --2- 172.21.15.92:0/299276536 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa6a80682f0 0x7fa6a806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:40.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.098+0000 7fa6bf7fe700 1 -- 172.21.15.92:0/299276536 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fa6c0085ea0 con 0x7fa6c80cd460 2023-11-28T11:25:40.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.098+0000 7fa6c6ffd700 1 --2- 172.21.15.92:0/299276536 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa6a80682f0 0x7fa6a806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:40.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.098+0000 7fa6bf7fe700 1 --2- 172.21.15.92:0/299276536 >> [v2:172.21.15.92:6824/115825,v1:172.21.15.92:6825/115825] conn(0x7fa6a806eb90 0x7fa6a8070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:40.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.098+0000 7fa6bf7fe700 1 -- 172.21.15.92:0/299276536 --> [v2:172.21.15.92:6824/115825,v1:172.21.15.92:6825/115825] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa6a80716b0 con 0x7fa6a806eb90 2023-11-28T11:25:40.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.098+0000 7fa6bf7fe700 1 -- 172.21.15.92:0/299276536 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6c0086250 con 0x7fa6c80cd460 2023-11-28T11:25:40.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.098+0000 7fa6bffff700 1 --2- 172.21.15.92:0/299276536 >> [v2:172.21.15.92:6824/115825,v1:172.21.15.92:6825/115825] conn(0x7fa6a806eb90 0x7fa6a8070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:40.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.099+0000 7fa6c6ffd700 1 --2- 172.21.15.92:0/299276536 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa6a80682f0 0x7fa6a806a7a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa6b80097c0 tx=0x7fa6b801c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:40.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.099+0000 7fa6bffff700 1 --2- 172.21.15.92:0/299276536 >> [v2:172.21.15.92:6824/115825,v1:172.21.15.92:6825/115825] conn(0x7fa6a806eb90 0x7fa6a8070fe0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:40.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.100+0000 7fa6bf7fe700 1 -- 172.21.15.92:0/299276536 <== osd.3 v2:172.21.15.92:6824/115825 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fa6a80716b0 con 0x7fa6a806eb90 2023-11-28T11:25:40.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.115+0000 7fa6cd3ff700 1 -- 172.21.15.92:0/299276536 --> [v2:172.21.15.92:6824/115825,v1:172.21.15.92:6825/115825] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa6c8067880 con 0x7fa6a806eb90 2023-11-28T11:25:40.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.115+0000 7fa6bf7fe700 1 -- 172.21.15.92:0/299276536 <== osd.3 v2:172.21.15.92:6824/115825 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fa6c8067880 con 0x7fa6a806eb90 2023-11-28T11:25:40.119 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-28T11:25:40.119 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T11:25:40.119 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-28T11:25:40.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.115+0000 7fa6cd3ff700 1 -- 172.21.15.92:0/299276536 >> [v2:172.21.15.92:6824/115825,v1:172.21.15.92:6825/115825] conn(0x7fa6a806eb90 msgr2=0x7fa6a8070fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:40.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.115+0000 7fa6cd3ff700 1 --2- 172.21.15.92:0/299276536 >> [v2:172.21.15.92:6824/115825,v1:172.21.15.92:6825/115825] conn(0x7fa6a806eb90 0x7fa6a8070fe0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:40.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.115+0000 7fa6cd3ff700 1 -- 172.21.15.92:0/299276536 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa6a80682f0 msgr2=0x7fa6a806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:40.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.115+0000 7fa6cd3ff700 1 --2- 172.21.15.92:0/299276536 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa6a80682f0 0x7fa6a806a7a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa6b80097c0 tx=0x7fa6b801c040 comp rx=0 tx=0).stop 2023-11-28T11:25:40.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.115+0000 7fa6cd3ff700 1 -- 172.21.15.92:0/299276536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6c80cd460 msgr2=0x7fa6c80b4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:40.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.115+0000 7fa6cd3ff700 1 --2- 172.21.15.92:0/299276536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6c80cd460 0x7fa6c80b4320 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fa6c000cba0 tx=0x7fa6c000ceb0 comp rx=0 tx=0).stop 2023-11-28T11:25:40.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.115+0000 7fa6c77fe700 1 -- 172.21.15.92:0/299276536 reap_dead start 2023-11-28T11:25:40.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.115+0000 7fa6cd3ff700 1 -- 172.21.15.92:0/299276536 shutdown_connections 2023-11-28T11:25:40.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.115+0000 7fa6cd3ff700 1 -- 172.21.15.92:0/299276536 >> 172.21.15.92:0/299276536 conn(0x7fa6c80719a0 msgr2=0x7fa6c806b940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:40.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.116+0000 7fa6cd3ff700 1 -- 172.21.15.92:0/299276536 shutdown_connections 2023-11-28T11:25:40.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.116+0000 7fa6cd3ff700 1 -- 172.21.15.92:0/299276536 wait complete. 2023-11-28T11:25:40.133 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-11-28T11:25:40.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.329+0000 7fd6a09ff700 1 Processor -- start 2023-11-28T11:25:40.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.329+0000 7fd6a09ff700 1 -- start start 2023-11-28T11:25:40.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.330+0000 7fd6a09ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd69c0cd460 0x7fd69c06c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:40.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.330+0000 7fd6a09ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fd69c0d0d60 0x7fd69c0d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:40.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.330+0000 7fd6a09ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd69c0b88e0 con 0x7fd69c0b8510 2023-11-28T11:25:40.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.330+0000 7fd6a09ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd69c0cdbd0 con 0x7fd69c0cd460 2023-11-28T11:25:40.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.330+0000 7fd6a09ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd69c0d02d0 con 0x7fd69c0d0d60 2023-11-28T11:25:40.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.331+0000 7fd699d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd69c0cd460 0x7fd69c06c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:40.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.331+0000 7fd699d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd69c0cd460 0x7fd69c06c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38440/0 (socket says 172.21.15.92:38440) 2023-11-28T11:25:40.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.331+0000 7fd699d9b700 1 -- 172.21.15.92:0/1724949287 learned_addr learned my addr 172.21.15.92:0/1724949287 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:40.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.331+0000 7fd69ad9d700 1 --2- 172.21.15.92:0/1724949287 >> v2:172.21.15.169:3300/0 conn(0x7fd69c0d0d60 0x7fd69c0d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:40.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.331+0000 7fd699d9b700 1 -- 172.21.15.92:0/1724949287 >> v2:172.21.15.169:3300/0 conn(0x7fd69c0d0d60 msgr2=0x7fd69c0d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:40.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.331+0000 7fd699d9b700 1 --2- 172.21.15.92:0/1724949287 >> v2:172.21.15.169:3300/0 conn(0x7fd69c0d0d60 0x7fd69c0d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:40.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.331+0000 7fd699d9b700 1 -- 172.21.15.92:0/1724949287 >> v1:172.21.15.169:6789/0 conn(0x7fd69c0b8510 legacy=0x7fd69c06b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:40.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.331+0000 7fd699d9b700 1 -- 172.21.15.92:0/1724949287 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd69c0cd8f0 con 0x7fd69c0cd460 2023-11-28T11:25:40.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.331+0000 7fd69ad9d700 1 --2- 172.21.15.92:0/1724949287 >> v2:172.21.15.169:3300/0 conn(0x7fd69c0d0d60 0x7fd69c0d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:25:40.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.331+0000 7fd699d9b700 1 --2- 172.21.15.92:0/1724949287 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd69c0cd460 0x7fd69c06c080 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fd69c0ce8c0 tx=0x7fd690009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35e483c0037a1014 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:40.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.332+0000 7fd69959a700 1 -- 172.21.15.92:0/1724949287 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69000b860 con 0x7fd69c0cd460 2023-11-28T11:25:40.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.332+0000 7fd69959a700 1 -- 172.21.15.92:0/1724949287 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd69000b9c0 con 0x7fd69c0cd460 2023-11-28T11:25:40.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.332+0000 7fd69959a700 1 -- 172.21.15.92:0/1724949287 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69000bd50 con 0x7fd69c0cd460 2023-11-28T11:25:40.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.332+0000 7fd6a09ff700 1 -- 172.21.15.92:0/1724949287 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd69c0cd460 msgr2=0x7fd69c06c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:40.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.332+0000 7fd6a09ff700 1 --2- 172.21.15.92:0/1724949287 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd69c0cd460 0x7fd69c06c080 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fd69c0ce8c0 tx=0x7fd690009c00 comp rx=0 tx=0).stop 2023-11-28T11:25:40.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.332+0000 7fd6a09ff700 1 -- 172.21.15.92:0/1724949287 shutdown_connections 2023-11-28T11:25:40.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.332+0000 7fd6a09ff700 1 --2- 172.21.15.92:0/1724949287 >> v2:172.21.15.169:3300/0 conn(0x7fd69c0d0d60 0x7fd69c0d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:40.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.332+0000 7fd6a09ff700 1 --2- 172.21.15.92:0/1724949287 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd69c0cd460 0x7fd69c06c080 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:40.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.332+0000 7fd6a09ff700 1 -- 172.21.15.92:0/1724949287 >> 172.21.15.92:0/1724949287 conn(0x7fd69c0719a0 msgr2=0x7fd69c071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:40.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.332+0000 7fd6a09ff700 1 -- 172.21.15.92:0/1724949287 shutdown_connections 2023-11-28T11:25:40.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.332+0000 7fd6a09ff700 1 -- 172.21.15.92:0/1724949287 wait complete. 2023-11-28T11:25:40.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.335+0000 7fd6a09ff700 1 Processor -- start 2023-11-28T11:25:40.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.335+0000 7fd6a09ff700 1 -- start start 2023-11-28T11:25:40.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.336+0000 7fd6a09ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd69c0b8510 0x7fd69c1e3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:40.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.336+0000 7fd6a09ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd69c0cd460 0x7fd69c06bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:40.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.336+0000 7fd6a09ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd69c0d0d60 0x7fd69c1e1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:40.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.336+0000 7fd6a09ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd69c1e78b0 con 0x7fd69c0b8510 2023-11-28T11:25:40.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.336+0000 7fd6a09ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd69c1e6a00 con 0x7fd69c0cd460 2023-11-28T11:25:40.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.336+0000 7fd6a09ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd69c1e6ca0 con 0x7fd69c0d0d60 2023-11-28T11:25:40.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.336+0000 7fd69a59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd69c0b8510 0x7fd69c1e3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:40.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.336+0000 7fd69a59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd69c0b8510 0x7fd69c1e3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38454/0 (socket says 172.21.15.92:38454) 2023-11-28T11:25:40.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.336+0000 7fd69a59c700 1 -- 172.21.15.92:0/515602040 learned_addr learned my addr 172.21.15.92:0/515602040 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:40.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.337+0000 7fd69a59c700 1 -- 172.21.15.92:0/515602040 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd69c0d0d60 msgr2=0x7fd69c1e1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:40.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.337+0000 7fd69a59c700 1 --2- 172.21.15.92:0/515602040 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd69c0d0d60 0x7fd69c1e1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:40.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.337+0000 7fd69a59c700 1 -- 172.21.15.92:0/515602040 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd69c0cd460 msgr2=0x7fd69c06bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:40.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.337+0000 7fd699d9b700 1 --2- 172.21.15.92:0/515602040 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd69c0cd460 0x7fd69c06bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:40.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.337+0000 7fd69a59c700 1 --2- 172.21.15.92:0/515602040 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd69c0cd460 0x7fd69c06bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:40.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.337+0000 7fd69a59c700 1 -- 172.21.15.92:0/515602040 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd690009550 con 0x7fd69c0b8510 2023-11-28T11:25:40.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.337+0000 7fd699d9b700 1 --2- 172.21.15.92:0/515602040 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd69c0cd460 0x7fd69c06bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:25:40.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.337+0000 7fd69a59c700 1 --2- 172.21.15.92:0/515602040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd69c0b8510 0x7fd69c1e3ce0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fd68c00c4e0 tx=0x7fd68c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:40.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.337+0000 7fd68affd700 1 -- 172.21.15.92:0/515602040 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68c0052d0 con 0x7fd69c0b8510 2023-11-28T11:25:40.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.337+0000 7fd68affd700 1 -- 172.21.15.92:0/515602040 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd68c005430 con 0x7fd69c0b8510 2023-11-28T11:25:40.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.337+0000 7fd68affd700 1 -- 172.21.15.92:0/515602040 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68c0038c0 con 0x7fd69c0b8510 2023-11-28T11:25:40.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.337+0000 7fd6a09ff700 1 -- 172.21.15.92:0/515602040 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd69c0cfff0 con 0x7fd69c0b8510 2023-11-28T11:25:40.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.337+0000 7fd6a09ff700 1 -- 172.21.15.92:0/515602040 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd69c0cfd10 con 0x7fd69c0b8510 2023-11-28T11:25:40.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.338+0000 7fd6a09ff700 1 -- 172.21.15.92:0/515602040 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd69c0a9970 con 0x7fd69c0b8510 2023-11-28T11:25:40.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.339+0000 7fd68affd700 1 -- 172.21.15.92:0/515602040 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd68c019030 con 0x7fd69c0b8510 2023-11-28T11:25:40.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.340+0000 7fd68affd700 1 --2- 172.21.15.92:0/515602040 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd67c0682f0 0x7fd67c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:40.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.340+0000 7fd68affd700 1 -- 172.21.15.92:0/515602040 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fd68c084f10 con 0x7fd69c0b8510 2023-11-28T11:25:40.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.340+0000 7fd699d9b700 1 --2- 172.21.15.92:0/515602040 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd67c0682f0 0x7fd67c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:40.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.341+0000 7fd699d9b700 1 --2- 172.21.15.92:0/515602040 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd67c0682f0 0x7fd67c06a7a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd69c0b1de0 tx=0x7fd690008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:40.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.344+0000 7fd68affd700 1 -- 172.21.15.92:0/515602040 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd68c051860 con 0x7fd69c0b8510 2023-11-28T11:25:40.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.540+0000 7fd6a09ff700 1 -- 172.21.15.92:0/515602040 --> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fd69c067880 con 0x7fd67c0682f0 2023-11-28T11:25:40.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.541+0000 7fd68affd700 1 -- 172.21.15.92:0/515602040 <== mgr.4103 v2:172.21.15.92:6832/115482 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd69c067880 con 0x7fd67c0682f0 2023-11-28T11:25:40.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.543+0000 7fd6a09ff700 1 -- 172.21.15.92:0/515602040 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd67c0682f0 msgr2=0x7fd67c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:40.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.543+0000 7fd6a09ff700 1 --2- 172.21.15.92:0/515602040 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd67c0682f0 0x7fd67c06a7a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd69c0b1de0 tx=0x7fd690008030 comp rx=0 tx=0).stop 2023-11-28T11:25:40.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.543+0000 7fd6a09ff700 1 -- 172.21.15.92:0/515602040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd69c0b8510 msgr2=0x7fd69c1e3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:40.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.543+0000 7fd6a09ff700 1 --2- 172.21.15.92:0/515602040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd69c0b8510 0x7fd69c1e3ce0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fd68c00c4e0 tx=0x7fd68c00c8a0 comp rx=0 tx=0).stop 2023-11-28T11:25:40.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.543+0000 7fd6a09ff700 1 -- 172.21.15.92:0/515602040 shutdown_connections 2023-11-28T11:25:40.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.543+0000 7fd6a09ff700 1 --2- 172.21.15.92:0/515602040 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fd69c0d0d60 0x7fd69c1e1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:40.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.543+0000 7fd6a09ff700 1 --2- 172.21.15.92:0/515602040 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fd67c0682f0 0x7fd67c06a7a0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:40.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.543+0000 7fd6a09ff700 1 --2- 172.21.15.92:0/515602040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd69c0b8510 0x7fd69c1e3ce0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:40.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.543+0000 7fd6a09ff700 1 --2- 172.21.15.92:0/515602040 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fd69c0cd460 0x7fd69c06bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:40.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.543+0000 7fd6a09ff700 1 -- 172.21.15.92:0/515602040 >> 172.21.15.92:0/515602040 conn(0x7fd69c0719a0 msgr2=0x7fd69c0c9710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:40.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.543+0000 7fd6a09ff700 1 -- 172.21.15.92:0/515602040 shutdown_connections 2023-11-28T11:25:40.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.543+0000 7fd6a09ff700 1 -- 172.21.15.92:0/515602040 wait complete. 2023-11-28T11:25:40.546 INFO:teuthology.orchestra.run.smithi092.stderr:instructed osd(s) 3 to deep-scrub 2023-11-28T11:25:40.563 INFO:tasks.ceph:Scrubbing osd.4 2023-11-28T11:25:40.563 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-11-28T11:25:40.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.763+0000 7f1831bff700 1 Processor -- start 2023-11-28T11:25:40.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.763+0000 7f1831bff700 1 -- start start 2023-11-28T11:25:40.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.764+0000 7f1831bff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f182c0b8790 0x7f182c0cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:40.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.764+0000 7f1831bff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f182c06c050 0x7f182c0d0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:40.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.764+0000 7f1831bff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f182c0d00b0 con 0x7f182c0cd460 2023-11-28T11:25:40.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.764+0000 7f1831bff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f182c0b5240 con 0x7f182c0b8790 2023-11-28T11:25:40.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.764+0000 7f1831bff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f182c0b5520 con 0x7f182c06c050 2023-11-28T11:25:40.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.765+0000 7f182bfff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f182c06c050 0x7f182c0d0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:40.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.765+0000 7f182b7fe700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f182c0b8790 0x7f182c0cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:40.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.765+0000 7f182bfff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f182c06c050 0x7f182c0d0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.92:50956/0 (socket says 172.21.15.92:50956) 2023-11-28T11:25:40.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.765+0000 7f182bfff700 1 -- 172.21.15.92:0/1903808655 learned_addr learned my addr 172.21.15.92:0/1903808655 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:40.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.765+0000 7f182bfff700 1 -- 172.21.15.92:0/1903808655 >> v2:172.21.15.169:3300/0 conn(0x7f182c0b8790 msgr2=0x7f182c0cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:40.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.765+0000 7f182bfff700 1 --2- 172.21.15.92:0/1903808655 >> v2:172.21.15.169:3300/0 conn(0x7f182c0b8790 0x7f182c0cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:40.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.765+0000 7f182a7fc700 1 -- 172.21.15.92:0/1903808655 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1985959941 0 0) 0x7f182c0d00b0 con 0x7f182c0cd460 2023-11-28T11:25:40.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.765+0000 7f182bfff700 1 -- 172.21.15.92:0/1903808655 >> v1:172.21.15.169:6789/0 conn(0x7f182c0cd460 legacy=0x7f182c06b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:40.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.765+0000 7f182bfff700 1 -- 172.21.15.92:0/1903808655 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f182c0d0a80 con 0x7f182c06c050 2023-11-28T11:25:40.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.765+0000 7f182b7fe700 1 --2- 172.21.15.92:0/1903808655 >> v2:172.21.15.169:3300/0 conn(0x7f182c0b8790 0x7f182c0cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:25:40.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.765+0000 7f182bfff700 1 --2- 172.21.15.92:0/1903808655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f182c06c050 0x7f182c0d0fa0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1824009e80 tx=0x7f182400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49c504d6246e3a20 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:40.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.766+0000 7f182a7fc700 1 -- 172.21.15.92:0/1903808655 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182400e070 con 0x7f182c06c050 2023-11-28T11:25:40.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.766+0000 7f182a7fc700 1 -- 172.21.15.92:0/1903808655 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f182400b8d0 con 0x7f182c06c050 2023-11-28T11:25:40.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.766+0000 7f182a7fc700 1 -- 172.21.15.92:0/1903808655 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182400bc80 con 0x7f182c06c050 2023-11-28T11:25:40.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.766+0000 7f1831bff700 1 -- 172.21.15.92:0/1903808655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f182c06c050 msgr2=0x7f182c0d0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:40.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.766+0000 7f1831bff700 1 --2- 172.21.15.92:0/1903808655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f182c06c050 0x7f182c0d0fa0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1824009e80 tx=0x7f182400b330 comp rx=0 tx=0).stop 2023-11-28T11:25:40.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.766+0000 7f1831bff700 1 -- 172.21.15.92:0/1903808655 shutdown_connections 2023-11-28T11:25:40.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.766+0000 7f1831bff700 1 --2- 172.21.15.92:0/1903808655 >> v2:172.21.15.169:3300/0 conn(0x7f182c0b8790 0x7f182c0cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:40.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.766+0000 7f1831bff700 1 --2- 172.21.15.92:0/1903808655 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f182c06c050 0x7f182c0d0fa0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:40.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.766+0000 7f1831bff700 1 -- 172.21.15.92:0/1903808655 >> 172.21.15.92:0/1903808655 conn(0x7f182c0719a0 msgr2=0x7f182c071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:40.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.766+0000 7f1831bff700 1 -- 172.21.15.92:0/1903808655 shutdown_connections 2023-11-28T11:25:40.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.766+0000 7f1831bff700 1 -- 172.21.15.92:0/1903808655 wait complete. 2023-11-28T11:25:40.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.768+0000 7f1831bff700 1 Processor -- start 2023-11-28T11:25:40.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.768+0000 7f1831bff700 1 -- start start 2023-11-28T11:25:40.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.769+0000 7f1831bff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f182c06c050 0x7f182c158550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:40.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.769+0000 7f1831bff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f182c0b8790 0x7f182c0cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:40.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.769+0000 7f1831bff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f182c0cd460 0x7f182c0cc390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:40.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.769+0000 7f1831bff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182c156870 con 0x7f182c06c050 2023-11-28T11:25:40.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.769+0000 7f1831bff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182c0cb2a0 con 0x7f182c0b8790 2023-11-28T11:25:40.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.769+0000 7f1831bff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f182c155c30 con 0x7f182c0cd460 2023-11-28T11:25:40.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.769+0000 7f182b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f182c06c050 0x7f182c158550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:40.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.769+0000 7f182b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f182c06c050 0x7f182c158550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38468/0 (socket says 172.21.15.92:38468) 2023-11-28T11:25:40.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.769+0000 7f182b7fe700 1 -- 172.21.15.92:0/769915758 learned_addr learned my addr 172.21.15.92:0/769915758 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:40.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.770+0000 7f182bfff700 1 --2- 172.21.15.92:0/769915758 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f182c0cd460 0x7f182c0cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:40.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.770+0000 7f182affd700 1 --2- 172.21.15.92:0/769915758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f182c0b8790 0x7f182c0cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:40.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.770+0000 7f182b7fe700 1 -- 172.21.15.92:0/769915758 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f182c0cd460 msgr2=0x7f182c0cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:40.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.770+0000 7f182b7fe700 1 --2- 172.21.15.92:0/769915758 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f182c0cd460 0x7f182c0cc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:40.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.770+0000 7f182b7fe700 1 -- 172.21.15.92:0/769915758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f182c0b8790 msgr2=0x7f182c0cbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:40.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.770+0000 7f182b7fe700 1 --2- 172.21.15.92:0/769915758 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f182c0b8790 0x7f182c0cbe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:40.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.770+0000 7f182b7fe700 1 -- 172.21.15.92:0/769915758 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f181c0094a0 con 0x7f182c06c050 2023-11-28T11:25:40.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.770+0000 7f182b7fe700 1 --2- 172.21.15.92:0/769915758 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f182c06c050 0x7f182c158550 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f181c00ebe0 tx=0x7f181c00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:40.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.770+0000 7f1813fff700 1 -- 172.21.15.92:0/769915758 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c00c7f0 con 0x7f182c06c050 2023-11-28T11:25:40.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.770+0000 7f1813fff700 1 -- 172.21.15.92:0/769915758 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f181c00c950 con 0x7f182c06c050 2023-11-28T11:25:40.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.770+0000 7f1813fff700 1 -- 172.21.15.92:0/769915758 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c00cce0 con 0x7f182c06c050 2023-11-28T11:25:40.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.770+0000 7f1831bff700 1 -- 172.21.15.92:0/769915758 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1824009b30 con 0x7f182c06c050 2023-11-28T11:25:40.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.770+0000 7f1831bff700 1 -- 172.21.15.92:0/769915758 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f182c0d14e0 con 0x7f182c06c050 2023-11-28T11:25:40.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.771+0000 7f1831bff700 1 -- 172.21.15.92:0/769915758 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f182c0b5800 con 0x7f182c06c050 2023-11-28T11:25:40.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.772+0000 7f1813fff700 1 -- 172.21.15.92:0/769915758 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f181c019030 con 0x7f182c06c050 2023-11-28T11:25:40.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.773+0000 7f1813fff700 1 --2- 172.21.15.92:0/769915758 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f18180682f0 0x7f181806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:40.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.773+0000 7f1813fff700 1 -- 172.21.15.92:0/769915758 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f181c085ae0 con 0x7f182c06c050 2023-11-28T11:25:40.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.773+0000 7f182affd700 1 --2- 172.21.15.92:0/769915758 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f18180682f0 0x7f181806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:40.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.773+0000 7f1813fff700 1 --2- 172.21.15.92:0/769915758 >> [v2:172.21.15.169:6816/116035,v1:172.21.15.169:6817/116035] conn(0x7f181806eb90 0x7f1818070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:40.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.773+0000 7f1813fff700 1 -- 172.21.15.92:0/769915758 --> [v2:172.21.15.169:6816/116035,v1:172.21.15.169:6817/116035] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f18180716b0 con 0x7f181806eb90 2023-11-28T11:25:40.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.773+0000 7f1813fff700 1 -- 172.21.15.92:0/769915758 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f181c08a2a0 con 0x7f182c06c050 2023-11-28T11:25:40.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.774+0000 7f182affd700 1 --2- 172.21.15.92:0/769915758 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f18180682f0 0x7f181806a7a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f182c0d2d00 tx=0x7f1820009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:40.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.774+0000 7f182bfff700 1 --2- 172.21.15.92:0/769915758 >> [v2:172.21.15.169:6816/116035,v1:172.21.15.169:6817/116035] conn(0x7f181806eb90 0x7f1818070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:40.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.775+0000 7f182bfff700 1 --2- 172.21.15.92:0/769915758 >> [v2:172.21.15.169:6816/116035,v1:172.21.15.169:6817/116035] conn(0x7f181806eb90 0x7f1818070fe0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:40.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.775+0000 7f1813fff700 1 -- 172.21.15.92:0/769915758 <== osd.4 v2:172.21.15.169:6816/116035 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f18180716b0 con 0x7f181806eb90 2023-11-28T11:25:40.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.790+0000 7f1831bff700 1 -- 172.21.15.92:0/769915758 --> [v2:172.21.15.169:6816/116035,v1:172.21.15.169:6817/116035] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f182c0d1670 con 0x7f181806eb90 2023-11-28T11:25:40.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.791+0000 7f1813fff700 1 -- 172.21.15.92:0/769915758 <== osd.4 v2:172.21.15.169:6816/116035 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f182c0d1670 con 0x7f181806eb90 2023-11-28T11:25:40.793 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-28T11:25:40.794 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T11:25:40.794 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-28T11:25:40.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.791+0000 7f1831bff700 1 -- 172.21.15.92:0/769915758 >> [v2:172.21.15.169:6816/116035,v1:172.21.15.169:6817/116035] conn(0x7f181806eb90 msgr2=0x7f1818070fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:40.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.791+0000 7f1831bff700 1 --2- 172.21.15.92:0/769915758 >> [v2:172.21.15.169:6816/116035,v1:172.21.15.169:6817/116035] conn(0x7f181806eb90 0x7f1818070fe0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:40.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.791+0000 7f1831bff700 1 -- 172.21.15.92:0/769915758 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f18180682f0 msgr2=0x7f181806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:40.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.791+0000 7f1831bff700 1 --2- 172.21.15.92:0/769915758 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f18180682f0 0x7f181806a7a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f182c0d2d00 tx=0x7f1820009040 comp rx=0 tx=0).stop 2023-11-28T11:25:40.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.791+0000 7f1831bff700 1 -- 172.21.15.92:0/769915758 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f182c06c050 msgr2=0x7f182c158550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:40.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.791+0000 7f1831bff700 1 --2- 172.21.15.92:0/769915758 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f182c06c050 0x7f182c158550 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f181c00ebe0 tx=0x7f181c00eef0 comp rx=0 tx=0).stop 2023-11-28T11:25:40.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.791+0000 7f182bfff700 1 -- 172.21.15.92:0/769915758 reap_dead start 2023-11-28T11:25:40.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.791+0000 7f1831bff700 1 -- 172.21.15.92:0/769915758 shutdown_connections 2023-11-28T11:25:40.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.791+0000 7f1831bff700 1 -- 172.21.15.92:0/769915758 >> 172.21.15.92:0/769915758 conn(0x7f182c0719a0 msgr2=0x7f182c0b1da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:40.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.791+0000 7f1831bff700 1 -- 172.21.15.92:0/769915758 shutdown_connections 2023-11-28T11:25:40.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:40.791+0000 7f1831bff700 1 -- 172.21.15.92:0/769915758 wait complete. 2023-11-28T11:25:40.811 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-11-28T11:25:41.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.008+0000 7f6d797ff700 1 Processor -- start 2023-11-28T11:25:41.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.009+0000 7f6d797ff700 1 -- start start 2023-11-28T11:25:41.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.010+0000 7f6d797ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6d740cd460 0x7f6d7406c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:41.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.010+0000 7f6d797ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d740d0d60 0x7f6d740d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:41.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.010+0000 7f6d797ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d740b88e0 con 0x7f6d740b8510 2023-11-28T11:25:41.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.010+0000 7f6d797ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d740cdbd0 con 0x7f6d740cd460 2023-11-28T11:25:41.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.010+0000 7f6d797ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d740d02d0 con 0x7f6d740d0d60 2023-11-28T11:25:41.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.010+0000 7f6d6bfff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6d740cd460 0x7f6d7406c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:41.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.010+0000 7f6d72ffd700 1 --1- >> v1:172.21.15.169:6789/0 conn(0x7f6d740b8510 0x7f6d7406b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.169:6789/0 says I am v1:172.21.15.92:42696/0 (socket says 172.21.15.92:42696) 2023-11-28T11:25:41.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.010+0000 7f6d6bfff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f6d740cd460 0x7f6d7406c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.92:50742/0 (socket says 172.21.15.92:50742) 2023-11-28T11:25:41.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.010+0000 7f6d737fe700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d740d0d60 0x7f6d740d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:41.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.010+0000 7f6d72ffd700 1 -- 172.21.15.92:0/2033008870 learned_addr learned my addr 172.21.15.92:0/2033008870 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:25:41.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.011+0000 7f6d727fc700 1 -- 172.21.15.92:0/2033008870 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1206340405 0 0) 0x7f6d740b88e0 con 0x7f6d740b8510 2023-11-28T11:25:41.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.011+0000 7f6d737fe700 1 -- 172.21.15.92:0/2033008870 >> v2:172.21.15.169:3300/0 conn(0x7f6d740cd460 msgr2=0x7f6d7406c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:41.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.011+0000 7f6d737fe700 1 --2- 172.21.15.92:0/2033008870 >> v2:172.21.15.169:3300/0 conn(0x7f6d740cd460 0x7f6d7406c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:41.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.011+0000 7f6d737fe700 1 -- 172.21.15.92:0/2033008870 >> v1:172.21.15.169:6789/0 conn(0x7f6d740b8510 legacy=0x7f6d7406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:41.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.011+0000 7f6d737fe700 1 -- 172.21.15.92:0/2033008870 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d740cd8f0 con 0x7f6d740d0d60 2023-11-28T11:25:41.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.011+0000 7f6d737fe700 1 --2- 172.21.15.92:0/2033008870 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d740d0d60 0x7f6d740d1200 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c009d90 tx=0x7f6d6c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad3a3835dd9aa07d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:41.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.011+0000 7f6d727fc700 1 -- 172.21.15.92:0/2033008870 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c00e070 con 0x7f6d740d0d60 2023-11-28T11:25:41.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.011+0000 7f6d727fc700 1 -- 172.21.15.92:0/2033008870 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d6c00bb20 con 0x7f6d740d0d60 2023-11-28T11:25:41.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.011+0000 7f6d727fc700 1 -- 172.21.15.92:0/2033008870 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c012430 con 0x7f6d740d0d60 2023-11-28T11:25:41.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.012+0000 7f6d797ff700 1 -- 172.21.15.92:0/2033008870 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d740d0d60 msgr2=0x7f6d740d1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:41.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.012+0000 7f6d797ff700 1 --2- 172.21.15.92:0/2033008870 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d740d0d60 0x7f6d740d1200 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c009d90 tx=0x7f6d6c00b330 comp rx=0 tx=0).stop 2023-11-28T11:25:41.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.012+0000 7f6d797ff700 1 -- 172.21.15.92:0/2033008870 shutdown_connections 2023-11-28T11:25:41.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.012+0000 7f6d797ff700 1 --2- 172.21.15.92:0/2033008870 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d740d0d60 0x7f6d740d1200 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:41.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.012+0000 7f6d797ff700 1 --2- 172.21.15.92:0/2033008870 >> v2:172.21.15.169:3300/0 conn(0x7f6d740cd460 0x7f6d7406c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:41.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.012+0000 7f6d797ff700 1 -- 172.21.15.92:0/2033008870 >> 172.21.15.92:0/2033008870 conn(0x7f6d740719a0 msgr2=0x7f6d74071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:41.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.012+0000 7f6d797ff700 1 -- 172.21.15.92:0/2033008870 shutdown_connections 2023-11-28T11:25:41.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.012+0000 7f6d797ff700 1 -- 172.21.15.92:0/2033008870 wait complete. 2023-11-28T11:25:41.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.014+0000 7f6d797ff700 1 Processor -- start 2023-11-28T11:25:41.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.014+0000 7f6d797ff700 1 -- start start 2023-11-28T11:25:41.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.015+0000 7f6d797ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d740b8510 0x7f6d741e67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:41.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.015+0000 7f6d797ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d740cd460 0x7f6d741f2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:41.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.015+0000 7f6d797ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d740d0d60 0x7f6d740cb500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:41.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.015+0000 7f6d797ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d740ca460 con 0x7f6d740b8510 2023-11-28T11:25:41.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.015+0000 7f6d797ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d740ca740 con 0x7f6d740cd460 2023-11-28T11:25:41.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.015+0000 7f6d797ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d740caa20 con 0x7f6d740d0d60 2023-11-28T11:25:41.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.015+0000 7f6d72ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d740b8510 0x7f6d741e67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:41.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.015+0000 7f6d72ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d740b8510 0x7f6d741e67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38482/0 (socket says 172.21.15.92:38482) 2023-11-28T11:25:41.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.015+0000 7f6d72ffd700 1 -- 172.21.15.92:0/512822104 learned_addr learned my addr 172.21.15.92:0/512822104 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:41.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.015+0000 7f6d6bfff700 1 --2- 172.21.15.92:0/512822104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d740cd460 0x7f6d741f2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:41.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.015+0000 7f6d737fe700 1 --2- 172.21.15.92:0/512822104 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d740d0d60 0x7f6d740cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:41.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.015+0000 7f6d72ffd700 1 -- 172.21.15.92:0/512822104 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d740d0d60 msgr2=0x7f6d740cb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:41.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.015+0000 7f6d72ffd700 1 --2- 172.21.15.92:0/512822104 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d740d0d60 0x7f6d740cb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:41.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.015+0000 7f6d72ffd700 1 -- 172.21.15.92:0/512822104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d740cd460 msgr2=0x7f6d741f2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:41.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.015+0000 7f6d72ffd700 1 --2- 172.21.15.92:0/512822104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d740cd460 0x7f6d741f2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:41.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.015+0000 7f6d72ffd700 1 -- 172.21.15.92:0/512822104 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d6c009a70 con 0x7f6d740b8510 2023-11-28T11:25:41.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.016+0000 7f6d72ffd700 1 --2- 172.21.15.92:0/512822104 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d740b8510 0x7f6d741e67e0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f6d640011e0 tx=0x7f6d6400ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:41.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.016+0000 7f6d6b7fe700 1 -- 172.21.15.92:0/512822104 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d64004c80 con 0x7f6d740b8510 2023-11-28T11:25:41.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.016+0000 7f6d6b7fe700 1 -- 172.21.15.92:0/512822104 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d64004de0 con 0x7f6d740b8510 2023-11-28T11:25:41.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.016+0000 7f6d797ff700 1 -- 172.21.15.92:0/512822104 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d740cfff0 con 0x7f6d740b8510 2023-11-28T11:25:41.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.016+0000 7f6d6b7fe700 1 -- 172.21.15.92:0/512822104 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d64007310 con 0x7f6d740b8510 2023-11-28T11:25:41.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.016+0000 7f6d797ff700 1 -- 172.21.15.92:0/512822104 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d740cfd10 con 0x7f6d740b8510 2023-11-28T11:25:41.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.017+0000 7f6d797ff700 1 -- 172.21.15.92:0/512822104 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d74067880 con 0x7f6d740b8510 2023-11-28T11:25:41.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.018+0000 7f6d6b7fe700 1 -- 172.21.15.92:0/512822104 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6d6401a020 con 0x7f6d740b8510 2023-11-28T11:25:41.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.019+0000 7f6d6b7fe700 1 --2- 172.21.15.92:0/512822104 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6d540682f0 0x7f6d5406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:41.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.019+0000 7f6d6b7fe700 1 -- 172.21.15.92:0/512822104 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f6d64085760 con 0x7f6d740b8510 2023-11-28T11:25:41.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.019+0000 7f6d6bfff700 1 --2- 172.21.15.92:0/512822104 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6d540682f0 0x7f6d5406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:41.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.020+0000 7f6d6bfff700 1 --2- 172.21.15.92:0/512822104 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6d540682f0 0x7f6d5406a7a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f6d7415aed0 tx=0x7f6d5c009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:41.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.023+0000 7f6d6b7fe700 1 -- 172.21.15.92:0/512822104 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6d640520b0 con 0x7f6d740b8510 2023-11-28T11:25:41.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.213+0000 7f6d797ff700 1 -- 172.21.15.92:0/512822104 --> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f6d740d2940 con 0x7f6d540682f0 2023-11-28T11:25:41.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.213+0000 7f6d6b7fe700 1 -- 172.21.15.92:0/512822104 <== mgr.4103 v2:172.21.15.92:6832/115482 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f6d740d2940 con 0x7f6d540682f0 2023-11-28T11:25:41.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.215+0000 7f6d797ff700 1 -- 172.21.15.92:0/512822104 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6d540682f0 msgr2=0x7f6d5406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:41.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.215+0000 7f6d797ff700 1 --2- 172.21.15.92:0/512822104 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6d540682f0 0x7f6d5406a7a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f6d7415aed0 tx=0x7f6d5c009210 comp rx=0 tx=0).stop 2023-11-28T11:25:41.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.215+0000 7f6d797ff700 1 -- 172.21.15.92:0/512822104 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d740b8510 msgr2=0x7f6d741e67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:41.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.215+0000 7f6d797ff700 1 --2- 172.21.15.92:0/512822104 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d740b8510 0x7f6d741e67e0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f6d640011e0 tx=0x7f6d6400ca30 comp rx=0 tx=0).stop 2023-11-28T11:25:41.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.215+0000 7f6d797ff700 1 -- 172.21.15.92:0/512822104 shutdown_connections 2023-11-28T11:25:41.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.215+0000 7f6d797ff700 1 --2- 172.21.15.92:0/512822104 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f6d740d0d60 0x7f6d740cb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:41.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.215+0000 7f6d797ff700 1 --2- 172.21.15.92:0/512822104 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f6d540682f0 0x7f6d5406a7a0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:41.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.215+0000 7f6d797ff700 1 --2- 172.21.15.92:0/512822104 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d740b8510 0x7f6d741e67e0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:41.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.215+0000 7f6d797ff700 1 --2- 172.21.15.92:0/512822104 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f6d740cd460 0x7f6d741f2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:41.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.215+0000 7f6d797ff700 1 -- 172.21.15.92:0/512822104 >> 172.21.15.92:0/512822104 conn(0x7f6d740719a0 msgr2=0x7f6d7406b970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:41.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.216+0000 7f6d797ff700 1 -- 172.21.15.92:0/512822104 shutdown_connections 2023-11-28T11:25:41.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.216+0000 7f6d797ff700 1 -- 172.21.15.92:0/512822104 wait complete. 2023-11-28T11:25:41.221 INFO:teuthology.orchestra.run.smithi092.stderr:instructed osd(s) 4 to deep-scrub 2023-11-28T11:25:41.234 INFO:tasks.ceph:Scrubbing osd.5 2023-11-28T11:25:41.235 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-11-28T11:25:41.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.432+0000 7f409b7ff700 1 Processor -- start 2023-11-28T11:25:41.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.432+0000 7f409b7ff700 1 -- start start 2023-11-28T11:25:41.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.433+0000 7f409b7ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f40940b8790 0x7f40940cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:41.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.433+0000 7f409b7ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40940cd460 0x7f409406b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:41.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.433+0000 7f409b7ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40940d00b0 con 0x7f409406be80 2023-11-28T11:25:41.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.433+0000 7f409b7ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f409406c300 con 0x7f40940b8790 2023-11-28T11:25:41.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.433+0000 7f409b7ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40940b5240 con 0x7f40940cd460 2023-11-28T11:25:41.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.433+0000 7f4099d9c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f409406be80 0x7f40941602b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56600/0 (socket says 172.21.15.92:56600) 2023-11-28T11:25:41.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.434+0000 7f4099d9c700 1 -- 172.21.15.92:0/3089899785 learned_addr learned my addr 172.21.15.92:0/3089899785 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:25:41.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.434+0000 7f4098d9a700 1 --2- 172.21.15.92:0/3089899785 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40940cd460 0x7f409406b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:41.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.434+0000 7f409959b700 1 --2- 172.21.15.92:0/3089899785 >> v2:172.21.15.169:3300/0 conn(0x7f40940b8790 0x7f40940cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:41.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.434+0000 7f408bfff700 1 -- 172.21.15.92:0/3089899785 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3813301935 0 0) 0x7f40940d00b0 con 0x7f409406be80 2023-11-28T11:25:41.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.434+0000 7f408bfff700 1 -- 172.21.15.92:0/3089899785 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40800035b0 con 0x7f409406be80 2023-11-28T11:25:41.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.434+0000 7f408bfff700 1 -- 172.21.15.92:0/3089899785 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 178332551 0 0) 0x7f40800035b0 con 0x7f409406be80 2023-11-28T11:25:41.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.434+0000 7f4098d9a700 1 -- 172.21.15.92:0/3089899785 >> v2:172.21.15.169:3300/0 conn(0x7f40940b8790 msgr2=0x7f40940cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:41.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.434+0000 7f4098d9a700 1 --2- 172.21.15.92:0/3089899785 >> v2:172.21.15.169:3300/0 conn(0x7f40940b8790 0x7f40940cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:41.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.434+0000 7f4098d9a700 1 -- 172.21.15.92:0/3089899785 >> v1:172.21.15.92:6789/0 conn(0x7f409406be80 legacy=0x7f40941602b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:41.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.434+0000 7f4098d9a700 1 -- 172.21.15.92:0/3089899785 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40940b5520 con 0x7f40940cd460 2023-11-28T11:25:41.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.434+0000 7f409959b700 1 --2- 172.21.15.92:0/3089899785 >> v2:172.21.15.169:3300/0 conn(0x7f40940b8790 0x7f40940cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:25:41.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.435+0000 7f4098d9a700 1 --2- 172.21.15.92:0/3089899785 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40940cd460 0x7f409406b940 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4084009840 tx=0x7f4084009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bed735b24ab2ae0b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:41.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.435+0000 7f408bfff700 1 -- 172.21.15.92:0/3089899785 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f408c0019f0 con 0x7f40940cd460 2023-11-28T11:25:41.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.435+0000 7f408bfff700 1 -- 172.21.15.92:0/3089899785 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4084004030 con 0x7f40940cd460 2023-11-28T11:25:41.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.435+0000 7f408bfff700 1 -- 172.21.15.92:0/3089899785 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40840043c0 con 0x7f40940cd460 2023-11-28T11:25:41.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.435+0000 7f409b7ff700 1 -- 172.21.15.92:0/3089899785 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40940cd460 msgr2=0x7f409406b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:41.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.435+0000 7f409b7ff700 1 --2- 172.21.15.92:0/3089899785 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40940cd460 0x7f409406b940 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4084009840 tx=0x7f4084009b50 comp rx=0 tx=0).stop 2023-11-28T11:25:41.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.436+0000 7f409b7ff700 1 -- 172.21.15.92:0/3089899785 shutdown_connections 2023-11-28T11:25:41.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.436+0000 7f409b7ff700 1 --2- 172.21.15.92:0/3089899785 >> v2:172.21.15.169:3300/0 conn(0x7f40940b8790 0x7f40940cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:41.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.436+0000 7f409b7ff700 1 --2- 172.21.15.92:0/3089899785 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40940cd460 0x7f409406b940 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:41.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.436+0000 7f409b7ff700 1 -- 172.21.15.92:0/3089899785 >> 172.21.15.92:0/3089899785 conn(0x7f40940719a0 msgr2=0x7f4094071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:41.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.436+0000 7f409b7ff700 1 -- 172.21.15.92:0/3089899785 shutdown_connections 2023-11-28T11:25:41.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.436+0000 7f409b7ff700 1 -- 172.21.15.92:0/3089899785 wait complete. 2023-11-28T11:25:41.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.438+0000 7f409b7ff700 1 Processor -- start 2023-11-28T11:25:41.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.438+0000 7f409b7ff700 1 -- start start 2023-11-28T11:25:41.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.439+0000 7f409b7ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f409406be80 0x7f40940c7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:41.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.439+0000 7f409b7ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40940b8790 0x7f40940cafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:41.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.439+0000 7f409b7ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40940cd460 0x7f40940cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:41.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.439+0000 7f409b7ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40940c6f20 con 0x7f40940cd460 2023-11-28T11:25:41.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.439+0000 7f409b7ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40940c7200 con 0x7f409406be80 2023-11-28T11:25:41.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.439+0000 7f409b7ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40940c74e0 con 0x7f40940b8790 2023-11-28T11:25:41.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.440+0000 7f4099d9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40940cd460 0x7f40940cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:41.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.440+0000 7f4099d9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40940cd460 0x7f40940cbf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38494/0 (socket says 172.21.15.92:38494) 2023-11-28T11:25:41.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.440+0000 7f4099d9c700 1 -- 172.21.15.92:0/705390247 learned_addr learned my addr 172.21.15.92:0/705390247 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:41.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.440+0000 7f409959b700 1 --2- 172.21.15.92:0/705390247 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f409406be80 0x7f40940c7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:41.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.440+0000 7f4098d9a700 1 --2- 172.21.15.92:0/705390247 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40940b8790 0x7f40940cafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:41.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.440+0000 7f4099d9c700 1 -- 172.21.15.92:0/705390247 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40940b8790 msgr2=0x7f40940cafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:41.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.440+0000 7f4099d9c700 1 --2- 172.21.15.92:0/705390247 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f40940b8790 0x7f40940cafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:41.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.440+0000 7f4099d9c700 1 -- 172.21.15.92:0/705390247 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f409406be80 msgr2=0x7f40940c7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:41.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.440+0000 7f4099d9c700 1 --2- 172.21.15.92:0/705390247 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f409406be80 0x7f40940c7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:41.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.440+0000 7f4099d9c700 1 -- 172.21.15.92:0/705390247 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40900094a0 con 0x7f40940cd460 2023-11-28T11:25:41.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.440+0000 7f4099d9c700 1 --2- 172.21.15.92:0/705390247 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40940cd460 0x7f40940cbf20 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f408c00cca0 tx=0x7f408c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:41.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.441+0000 7f4089ffb700 1 -- 172.21.15.92:0/705390247 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f408c00ab70 con 0x7f40940cd460 2023-11-28T11:25:41.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.441+0000 7f4089ffb700 1 -- 172.21.15.92:0/705390247 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f408c00acf0 con 0x7f40940cd460 2023-11-28T11:25:41.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.441+0000 7f4089ffb700 1 -- 172.21.15.92:0/705390247 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f408c007310 con 0x7f40940cd460 2023-11-28T11:25:41.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.441+0000 7f409b7ff700 1 -- 172.21.15.92:0/705390247 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40840094a0 con 0x7f40940cd460 2023-11-28T11:25:41.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.441+0000 7f409b7ff700 1 -- 172.21.15.92:0/705390247 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40940b5800 con 0x7f40940cd460 2023-11-28T11:25:41.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.441+0000 7f409b7ff700 1 -- 172.21.15.92:0/705390247 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f40940d0a80 con 0x7f40940cd460 2023-11-28T11:25:41.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.443+0000 7f4089ffb700 1 -- 172.21.15.92:0/705390247 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f408c010070 con 0x7f40940cd460 2023-11-28T11:25:41.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.443+0000 7f4089ffb700 1 --2- 172.21.15.92:0/705390247 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f40780682f0 0x7f407806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:41.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.443+0000 7f4089ffb700 1 -- 172.21.15.92:0/705390247 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f408c085510 con 0x7f40940cd460 2023-11-28T11:25:41.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.444+0000 7f409959b700 1 --2- 172.21.15.92:0/705390247 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f40780682f0 0x7f407806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:41.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.444+0000 7f4089ffb700 1 --2- 172.21.15.92:0/705390247 >> [v2:172.21.15.169:6800/116033,v1:172.21.15.169:6801/116033] conn(0x7f407806eb90 0x7f4078070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:41.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.444+0000 7f4089ffb700 1 -- 172.21.15.92:0/705390247 --> [v2:172.21.15.169:6800/116033,v1:172.21.15.169:6801/116033] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f40780716b0 con 0x7f407806eb90 2023-11-28T11:25:41.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.444+0000 7f4089ffb700 1 -- 172.21.15.92:0/705390247 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f408c087af0 con 0x7f40940cd460 2023-11-28T11:25:41.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.444+0000 7f409959b700 1 --2- 172.21.15.92:0/705390247 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f40780682f0 0x7f407806a7a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f4090009d70 tx=0x7f4090009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:41.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.444+0000 7f4098d9a700 1 --2- 172.21.15.92:0/705390247 >> [v2:172.21.15.169:6800/116033,v1:172.21.15.169:6801/116033] conn(0x7f407806eb90 0x7f4078070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:41.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.445+0000 7f4098d9a700 1 --2- 172.21.15.92:0/705390247 >> [v2:172.21.15.169:6800/116033,v1:172.21.15.169:6801/116033] conn(0x7f407806eb90 0x7f4078070fe0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:41.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.446+0000 7f4089ffb700 1 -- 172.21.15.92:0/705390247 <== osd.5 v2:172.21.15.169:6800/116033 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f40780716b0 con 0x7f407806eb90 2023-11-28T11:25:41.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.460+0000 7f409b7ff700 1 -- 172.21.15.92:0/705390247 --> [v2:172.21.15.169:6800/116033,v1:172.21.15.169:6801/116033] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f40940d1fb0 con 0x7f407806eb90 2023-11-28T11:25:41.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.461+0000 7f4089ffb700 1 -- 172.21.15.92:0/705390247 <== osd.5 v2:172.21.15.169:6800/116033 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f40940d1fb0 con 0x7f407806eb90 2023-11-28T11:25:41.462 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-28T11:25:41.462 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T11:25:41.463 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-28T11:25:41.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.462+0000 7f409b7ff700 1 -- 172.21.15.92:0/705390247 >> [v2:172.21.15.169:6800/116033,v1:172.21.15.169:6801/116033] conn(0x7f407806eb90 msgr2=0x7f4078070fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:41.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.462+0000 7f409b7ff700 1 --2- 172.21.15.92:0/705390247 >> [v2:172.21.15.169:6800/116033,v1:172.21.15.169:6801/116033] conn(0x7f407806eb90 0x7f4078070fe0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:41.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.462+0000 7f409b7ff700 1 -- 172.21.15.92:0/705390247 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f40780682f0 msgr2=0x7f407806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:41.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.462+0000 7f409b7ff700 1 --2- 172.21.15.92:0/705390247 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f40780682f0 0x7f407806a7a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f4090009d70 tx=0x7f4090009210 comp rx=0 tx=0).stop 2023-11-28T11:25:41.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.462+0000 7f409b7ff700 1 -- 172.21.15.92:0/705390247 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40940cd460 msgr2=0x7f40940cbf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:41.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.462+0000 7f409b7ff700 1 --2- 172.21.15.92:0/705390247 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40940cd460 0x7f40940cbf20 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f408c00cca0 tx=0x7f408c00a060 comp rx=0 tx=0).stop 2023-11-28T11:25:41.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.462+0000 7f4099d9c700 1 -- 172.21.15.92:0/705390247 reap_dead start 2023-11-28T11:25:41.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.462+0000 7f409b7ff700 1 -- 172.21.15.92:0/705390247 shutdown_connections 2023-11-28T11:25:41.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.462+0000 7f409b7ff700 1 -- 172.21.15.92:0/705390247 >> 172.21.15.92:0/705390247 conn(0x7f40940719a0 msgr2=0x7f4094071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:41.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.462+0000 7f409b7ff700 1 -- 172.21.15.92:0/705390247 shutdown_connections 2023-11-28T11:25:41.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.462+0000 7f409b7ff700 1 -- 172.21.15.92:0/705390247 wait complete. 2023-11-28T11:25:41.480 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-11-28T11:25:41.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.677+0000 7fa8c57ff700 1 Processor -- start 2023-11-28T11:25:41.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.677+0000 7fa8c57ff700 1 -- start start 2023-11-28T11:25:41.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.678+0000 7fa8c57ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8c00cd460 0x7fa8c006c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:41.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.678+0000 7fa8c57ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8c00b88e0 con 0x7fa8c00d0d60 2023-11-28T11:25:41.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.678+0000 7fa8c57ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8c00cd8f0 con 0x7fa8c00b8510 2023-11-28T11:25:41.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.678+0000 7fa8c57ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c00cdbd0 con 0x7fa8c00cd460 2023-11-28T11:25:41.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.679+0000 7fa8bf7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fa8c00d0d60 0x7fa8c00d11e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56610/0 (socket says 172.21.15.92:56610) 2023-11-28T11:25:41.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.679+0000 7fa8bf7fe700 1 -- 172.21.15.92:0/2705952900 learned_addr learned my addr 172.21.15.92:0/2705952900 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:25:41.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.679+0000 7fa8be7fc700 1 --2- 172.21.15.92:0/2705952900 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8c00cd460 0x7fa8c006c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:41.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.679+0000 7fa8bdffb700 1 -- 172.21.15.92:0/2705952900 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1115530238 0 0) 0x7fa8c00b88e0 con 0x7fa8c00d0d60 2023-11-28T11:25:41.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.679+0000 7fa8bdffb700 1 -- 172.21.15.92:0/2705952900 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8a80035b0 con 0x7fa8c00d0d60 2023-11-28T11:25:41.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.679+0000 7fa8bdffb700 1 -- 172.21.15.92:0/2705952900 <== mon.1 v1:172.21.15.169:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2856404736 0 0) 0x7fa8c00cd8f0 con 0x7fa8c00b8510 2023-11-28T11:25:41.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.679+0000 7fa8bdffb700 1 -- 172.21.15.92:0/2705952900 --> v1:172.21.15.169:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8c00b88e0 con 0x7fa8c00b8510 2023-11-28T11:25:41.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.679+0000 7fa8be7fc700 1 -- 172.21.15.92:0/2705952900 >> v1:172.21.15.169:6789/0 conn(0x7fa8c00b8510 legacy=0x7fa8c006b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:41.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.679+0000 7fa8be7fc700 1 -- 172.21.15.92:0/2705952900 >> v1:172.21.15.92:6789/0 conn(0x7fa8c00d0d60 legacy=0x7fa8c00d11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:41.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.679+0000 7fa8bdffb700 1 -- 172.21.15.92:0/2705952900 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4157855165 0 0) 0x7fa8a80035b0 con 0x7fa8c00d0d60 2023-11-28T11:25:41.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.679+0000 7fa8be7fc700 1 -- 172.21.15.92:0/2705952900 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c00cfd10 con 0x7fa8c00cd460 2023-11-28T11:25:41.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.680+0000 7fa8be7fc700 1 --2- 172.21.15.92:0/2705952900 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8c00cd460 0x7fa8c006c080 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa8b4009870 tx=0x7fa8b4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4995b620617a98f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:41.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.680+0000 7fa8bdffb700 1 -- 172.21.15.92:0/2705952900 <== mon.2 v2:172.21.15.169:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b80019f0 con 0x7fa8c00cd460 2023-11-28T11:25:41.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.680+0000 7fa8bdffb700 1 -- 172.21.15.92:0/2705952900 <== mon.2 v2:172.21.15.169:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8b4004030 con 0x7fa8c00cd460 2023-11-28T11:25:41.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.680+0000 7fa8bdffb700 1 -- 172.21.15.92:0/2705952900 <== mon.2 v2:172.21.15.169:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b40043c0 con 0x7fa8c00cd460 2023-11-28T11:25:41.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.681+0000 7fa8c57ff700 1 -- 172.21.15.92:0/2705952900 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8c00cd460 msgr2=0x7fa8c006c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:41.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.681+0000 7fa8c57ff700 1 --2- 172.21.15.92:0/2705952900 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8c00cd460 0x7fa8c006c080 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa8b4009870 tx=0x7fa8b4009b80 comp rx=0 tx=0).stop 2023-11-28T11:25:41.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.681+0000 7fa8c57ff700 1 -- 172.21.15.92:0/2705952900 shutdown_connections 2023-11-28T11:25:41.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.681+0000 7fa8c57ff700 1 --2- 172.21.15.92:0/2705952900 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8c00cd460 0x7fa8c006c080 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:41.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.681+0000 7fa8c57ff700 1 -- 172.21.15.92:0/2705952900 >> 172.21.15.92:0/2705952900 conn(0x7fa8c00719a0 msgr2=0x7fa8c0071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:41.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.681+0000 7fa8c57ff700 1 -- 172.21.15.92:0/2705952900 shutdown_connections 2023-11-28T11:25:41.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.681+0000 7fa8c57ff700 1 -- 172.21.15.92:0/2705952900 wait complete. 2023-11-28T11:25:41.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.682+0000 7fa8c57ff700 1 Processor -- start 2023-11-28T11:25:41.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.683+0000 7fa8c57ff700 1 -- start start 2023-11-28T11:25:41.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.683+0000 7fa8c57ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8c00b8510 0x7fa8c01554f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:41.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.684+0000 7fa8c57ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8c00cd460 0x7fa8c00ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:41.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.684+0000 7fa8c57ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8c00d0d60 0x7fa8c01e4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:41.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.684+0000 7fa8c57ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c00c7910 con 0x7fa8c00d0d60 2023-11-28T11:25:41.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.684+0000 7fa8c57ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c0155a30 con 0x7fa8c00cd460 2023-11-28T11:25:41.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.684+0000 7fa8c57ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c00cb2d0 con 0x7fa8c00b8510 2023-11-28T11:25:41.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.684+0000 7fa8bf7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8c00d0d60 0x7fa8c01e4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:41.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.684+0000 7fa8beffd700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8c00b8510 0x7fa8c01554f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:41.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.684+0000 7fa8be7fc700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8c00cd460 0x7fa8c00ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:41.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.684+0000 7fa8beffd700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8c00b8510 0x7fa8c01554f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3301/0 says I am v2:172.21.15.92:51004/0 (socket says 172.21.15.92:51004) 2023-11-28T11:25:41.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.684+0000 7fa8beffd700 1 -- 172.21.15.92:0/677429291 learned_addr learned my addr 172.21.15.92:0/677429291 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:41.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.684+0000 7fa8bf7fe700 1 -- 172.21.15.92:0/677429291 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8c00b8510 msgr2=0x7fa8c01554f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:41.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.684+0000 7fa8bf7fe700 1 --2- 172.21.15.92:0/677429291 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8c00b8510 0x7fa8c01554f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:41.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.684+0000 7fa8bf7fe700 1 -- 172.21.15.92:0/677429291 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8c00cd460 msgr2=0x7fa8c00ca8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:41.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.684+0000 7fa8bf7fe700 1 --2- 172.21.15.92:0/677429291 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8c00cd460 0x7fa8c00ca8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:41.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.684+0000 7fa8bf7fe700 1 -- 172.21.15.92:0/677429291 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8b40094a0 con 0x7fa8c00d0d60 2023-11-28T11:25:41.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.685+0000 7fa8bf7fe700 1 --2- 172.21.15.92:0/677429291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8c00d0d60 0x7fa8c01e4620 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8001700 tx=0x7fa8b800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:41.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.685+0000 7fa8a77fe700 1 -- 172.21.15.92:0/677429291 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b800a9b0 con 0x7fa8c00d0d60 2023-11-28T11:25:41.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.685+0000 7fa8a77fe700 1 -- 172.21.15.92:0/677429291 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8b800ab10 con 0x7fa8c00d0d60 2023-11-28T11:25:41.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.685+0000 7fa8a77fe700 1 -- 172.21.15.92:0/677429291 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b80070d0 con 0x7fa8c00d0d60 2023-11-28T11:25:41.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.685+0000 7fa8c57ff700 1 -- 172.21.15.92:0/677429291 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8c00d29c0 con 0x7fa8c00d0d60 2023-11-28T11:25:41.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.685+0000 7fa8c57ff700 1 -- 172.21.15.92:0/677429291 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8c00cfff0 con 0x7fa8c00d0d60 2023-11-28T11:25:41.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.686+0000 7fa8c57ff700 1 -- 172.21.15.92:0/677429291 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8c00d2b50 con 0x7fa8c00d0d60 2023-11-28T11:25:41.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.687+0000 7fa8a77fe700 1 -- 172.21.15.92:0/677429291 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa8b8010070 con 0x7fa8c00d0d60 2023-11-28T11:25:41.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.688+0000 7fa8a77fe700 1 --2- 172.21.15.92:0/677429291 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa8ac0682f0 0x7fa8ac06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:41.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.688+0000 7fa8a77fe700 1 -- 172.21.15.92:0/677429291 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fa8b8085330 con 0x7fa8c00d0d60 2023-11-28T11:25:41.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.688+0000 7fa8beffd700 1 --2- 172.21.15.92:0/677429291 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa8ac0682f0 0x7fa8ac06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:41.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.689+0000 7fa8beffd700 1 --2- 172.21.15.92:0/677429291 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa8ac0682f0 0x7fa8ac06a7a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa8b00011e0 tx=0x7fa8b0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:41.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.692+0000 7fa8a77fe700 1 -- 172.21.15.92:0/677429291 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa8b8051cc0 con 0x7fa8c00d0d60 2023-11-28T11:25:41.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.886+0000 7fa8c57ff700 1 -- 172.21.15.92:0/677429291 --> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fa8c0067880 con 0x7fa8ac0682f0 2023-11-28T11:25:41.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.887+0000 7fa8a77fe700 1 -- 172.21.15.92:0/677429291 <== mgr.4103 v2:172.21.15.92:6832/115482 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa8c0067880 con 0x7fa8ac0682f0 2023-11-28T11:25:41.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.889+0000 7fa8c57ff700 1 -- 172.21.15.92:0/677429291 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa8ac0682f0 msgr2=0x7fa8ac06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:41.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.889+0000 7fa8c57ff700 1 --2- 172.21.15.92:0/677429291 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa8ac0682f0 0x7fa8ac06a7a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa8b00011e0 tx=0x7fa8b0009040 comp rx=0 tx=0).stop 2023-11-28T11:25:41.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.889+0000 7fa8c57ff700 1 -- 172.21.15.92:0/677429291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8c00d0d60 msgr2=0x7fa8c01e4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:41.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.889+0000 7fa8c57ff700 1 --2- 172.21.15.92:0/677429291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8c00d0d60 0x7fa8c01e4620 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8001700 tx=0x7fa8b800ce30 comp rx=0 tx=0).stop 2023-11-28T11:25:41.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.889+0000 7fa8c57ff700 1 -- 172.21.15.92:0/677429291 shutdown_connections 2023-11-28T11:25:41.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.889+0000 7fa8c57ff700 1 --2- 172.21.15.92:0/677429291 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fa8ac0682f0 0x7fa8ac06a7a0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:41.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.889+0000 7fa8c57ff700 1 --2- 172.21.15.92:0/677429291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8c00d0d60 0x7fa8c01e4620 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:41.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.889+0000 7fa8c57ff700 1 --2- 172.21.15.92:0/677429291 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fa8c00b8510 0x7fa8c01554f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:41.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.889+0000 7fa8c57ff700 1 --2- 172.21.15.92:0/677429291 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fa8c00cd460 0x7fa8c00ca8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:41.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.889+0000 7fa8c57ff700 1 -- 172.21.15.92:0/677429291 >> 172.21.15.92:0/677429291 conn(0x7fa8c00719a0 msgr2=0x7fa8c00b1ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:41.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.889+0000 7fa8c57ff700 1 -- 172.21.15.92:0/677429291 shutdown_connections 2023-11-28T11:25:41.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:41.889+0000 7fa8c57ff700 1 -- 172.21.15.92:0/677429291 wait complete. 2023-11-28T11:25:41.892 INFO:teuthology.orchestra.run.smithi092.stderr:instructed osd(s) 5 to deep-scrub 2023-11-28T11:25:41.908 INFO:tasks.ceph:Scrubbing osd.6 2023-11-28T11:25:41.908 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-11-28T11:25:42.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.108+0000 7f94393ff700 1 Processor -- start 2023-11-28T11:25:42.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.108+0000 7f94393ff700 1 -- start start 2023-11-28T11:25:42.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.110+0000 7f94393ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94340b8790 0x7f94340cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:42.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.110+0000 7f94393ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f943406c050 0x7f94340d0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:42.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.110+0000 7f94393ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94340d00b0 con 0x7f94340cd460 2023-11-28T11:25:42.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.110+0000 7f94393ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94340b5240 con 0x7f943406c050 2023-11-28T11:25:42.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.110+0000 7f94393ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94340b5520 con 0x7f94340b8790 2023-11-28T11:25:42.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.110+0000 7f94337fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f943406c050 0x7f94340d0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:42.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.110+0000 7f942bfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f94340cd460 0x7f943406b940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56620/0 (socket says 172.21.15.92:56620) 2023-11-28T11:25:42.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.110+0000 7f942bfff700 1 -- 172.21.15.92:0/4063745225 learned_addr learned my addr 172.21.15.92:0/4063745225 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:25:42.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.110+0000 7f9432ffd700 1 --2- 172.21.15.92:0/4063745225 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94340b8790 0x7f94340cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:42.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.110+0000 7f94327fc700 1 -- 172.21.15.92:0/4063745225 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2827135514 0 0) 0x7f94340d00b0 con 0x7f94340cd460 2023-11-28T11:25:42.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.110+0000 7f94327fc700 1 -- 172.21.15.92:0/4063745225 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94200035b0 con 0x7f94340cd460 2023-11-28T11:25:42.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.110+0000 7f94337fe700 1 -- 172.21.15.92:0/4063745225 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94340b8790 msgr2=0x7f94340cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:42.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.110+0000 7f94337fe700 1 --2- 172.21.15.92:0/4063745225 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94340b8790 0x7f94340cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:42.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.110+0000 7f94337fe700 1 -- 172.21.15.92:0/4063745225 >> v1:172.21.15.92:6789/0 conn(0x7f94340cd460 legacy=0x7f943406b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:42.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.110+0000 7f94337fe700 1 -- 172.21.15.92:0/4063745225 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94340d0a80 con 0x7f943406c050 2023-11-28T11:25:42.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.111+0000 7f94337fe700 1 --2- 172.21.15.92:0/4063745225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f943406c050 0x7f94340d0fa0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f942c009d90 tx=0x7f942c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16498c1313f08f01 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:42.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.111+0000 7f94327fc700 1 -- 172.21.15.92:0/4063745225 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942c00e070 con 0x7f943406c050 2023-11-28T11:25:42.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.111+0000 7f94327fc700 1 -- 172.21.15.92:0/4063745225 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f942c00bb20 con 0x7f943406c050 2023-11-28T11:25:42.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.111+0000 7f94327fc700 1 -- 172.21.15.92:0/4063745225 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942c012430 con 0x7f943406c050 2023-11-28T11:25:42.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.111+0000 7f94393ff700 1 -- 172.21.15.92:0/4063745225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f943406c050 msgr2=0x7f94340d0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:42.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.111+0000 7f94393ff700 1 --2- 172.21.15.92:0/4063745225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f943406c050 0x7f94340d0fa0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f942c009d90 tx=0x7f942c00b330 comp rx=0 tx=0).stop 2023-11-28T11:25:42.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.112+0000 7f94393ff700 1 -- 172.21.15.92:0/4063745225 shutdown_connections 2023-11-28T11:25:42.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.112+0000 7f94393ff700 1 --2- 172.21.15.92:0/4063745225 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94340b8790 0x7f94340cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:42.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.112+0000 7f94393ff700 1 --2- 172.21.15.92:0/4063745225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f943406c050 0x7f94340d0fa0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:42.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.112+0000 7f94393ff700 1 -- 172.21.15.92:0/4063745225 >> 172.21.15.92:0/4063745225 conn(0x7f94340719a0 msgr2=0x7f9434071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:42.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.112+0000 7f94393ff700 1 -- 172.21.15.92:0/4063745225 shutdown_connections 2023-11-28T11:25:42.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.112+0000 7f94393ff700 1 -- 172.21.15.92:0/4063745225 wait complete. 2023-11-28T11:25:42.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.113+0000 7f94393ff700 1 Processor -- start 2023-11-28T11:25:42.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.113+0000 7f94393ff700 1 -- start start 2023-11-28T11:25:42.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.114+0000 7f94393ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f943406c050 0x7f94340cb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:42.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.114+0000 7f94393ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94340b8790 0x7f94341f5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:42.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.114+0000 7f94393ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94340cd460 0x7f9434154120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:42.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.114+0000 7f94393ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94340cb0d0 con 0x7f94340b8790 2023-11-28T11:25:42.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.114+0000 7f94393ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94341608b0 con 0x7f943406c050 2023-11-28T11:25:42.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.114+0000 7f94393ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94340cba30 con 0x7f94340cd460 2023-11-28T11:25:42.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.115+0000 7f942bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94340b8790 0x7f94341f5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:42.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.115+0000 7f942bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94340b8790 0x7f94341f5d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38518/0 (socket says 172.21.15.92:38518) 2023-11-28T11:25:42.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.115+0000 7f942bfff700 1 -- 172.21.15.92:0/1547418669 learned_addr learned my addr 172.21.15.92:0/1547418669 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:42.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.115+0000 7f9432ffd700 1 --2- 172.21.15.92:0/1547418669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f943406c050 0x7f94340cb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:42.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.115+0000 7f94337fe700 1 --2- 172.21.15.92:0/1547418669 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94340cd460 0x7f9434154120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:42.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.115+0000 7f942bfff700 1 -- 172.21.15.92:0/1547418669 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94340cd460 msgr2=0x7f9434154120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:42.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.115+0000 7f942bfff700 1 --2- 172.21.15.92:0/1547418669 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f94340cd460 0x7f9434154120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:42.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.115+0000 7f942bfff700 1 -- 172.21.15.92:0/1547418669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f943406c050 msgr2=0x7f94340cb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:42.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.115+0000 7f942bfff700 1 --2- 172.21.15.92:0/1547418669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f943406c050 0x7f94340cb4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:42.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.115+0000 7f942bfff700 1 -- 172.21.15.92:0/1547418669 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f942c009a70 con 0x7f94340b8790 2023-11-28T11:25:42.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.115+0000 7f9432ffd700 1 --2- 172.21.15.92:0/1547418669 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f943406c050 0x7f94340cb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:25:42.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.115+0000 7f942bfff700 1 --2- 172.21.15.92:0/1547418669 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94340b8790 0x7f94341f5d10 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f941c00c4d0 tx=0x7f941c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:42.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.115+0000 7f942b7fe700 1 -- 172.21.15.92:0/1547418669 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941c005d70 con 0x7f94340b8790 2023-11-28T11:25:42.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.115+0000 7f942b7fe700 1 -- 172.21.15.92:0/1547418669 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f941c0070d0 con 0x7f94340b8790 2023-11-28T11:25:42.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.115+0000 7f942b7fe700 1 -- 172.21.15.92:0/1547418669 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941c007460 con 0x7f94340b8790 2023-11-28T11:25:42.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.116+0000 7f94393ff700 1 -- 172.21.15.92:0/1547418669 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94340d14e0 con 0x7f94340b8790 2023-11-28T11:25:42.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.116+0000 7f94393ff700 1 -- 172.21.15.92:0/1547418669 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94340b5800 con 0x7f94340b8790 2023-11-28T11:25:42.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.116+0000 7f94393ff700 1 -- 172.21.15.92:0/1547418669 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f94340d1670 con 0x7f94340b8790 2023-11-28T11:25:42.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.117+0000 7f942b7fe700 1 -- 172.21.15.92:0/1547418669 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f941c019030 con 0x7f94340b8790 2023-11-28T11:25:42.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.118+0000 7f942b7fe700 1 --2- 172.21.15.92:0/1547418669 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f94140682f0 0x7f941406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:42.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.118+0000 7f942b7fe700 1 -- 172.21.15.92:0/1547418669 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f941c084f90 con 0x7f94340b8790 2023-11-28T11:25:42.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.119+0000 7f9432ffd700 1 --2- 172.21.15.92:0/1547418669 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f94140682f0 0x7f941406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:42.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.119+0000 7f942b7fe700 1 --2- 172.21.15.92:0/1547418669 >> [v2:172.21.15.169:6808/116034,v1:172.21.15.169:6809/116034] conn(0x7f941406eb90 0x7f9414070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:42.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.119+0000 7f942b7fe700 1 -- 172.21.15.92:0/1547418669 --> [v2:172.21.15.169:6808/116034,v1:172.21.15.169:6809/116034] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f94140716b0 con 0x7f941406eb90 2023-11-28T11:25:42.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.119+0000 7f942b7fe700 1 -- 172.21.15.92:0/1547418669 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f941c08a5a0 con 0x7f94340b8790 2023-11-28T11:25:42.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.119+0000 7f94337fe700 1 --2- 172.21.15.92:0/1547418669 >> [v2:172.21.15.169:6808/116034,v1:172.21.15.169:6809/116034] conn(0x7f941406eb90 0x7f9414070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:42.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.119+0000 7f9432ffd700 1 --2- 172.21.15.92:0/1547418669 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f94140682f0 0x7f941406a7a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f9424009c30 tx=0x7f9424008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:42.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.120+0000 7f94337fe700 1 --2- 172.21.15.92:0/1547418669 >> [v2:172.21.15.169:6808/116034,v1:172.21.15.169:6809/116034] conn(0x7f941406eb90 0x7f9414070fe0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:42.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.121+0000 7f942b7fe700 1 -- 172.21.15.92:0/1547418669 <== osd.6 v2:172.21.15.169:6808/116034 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f94140716b0 con 0x7f941406eb90 2023-11-28T11:25:42.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.135+0000 7f94393ff700 1 -- 172.21.15.92:0/1547418669 --> [v2:172.21.15.169:6808/116034,v1:172.21.15.169:6809/116034] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f94340ca4b0 con 0x7f941406eb90 2023-11-28T11:25:42.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.136+0000 7f942b7fe700 1 -- 172.21.15.92:0/1547418669 <== osd.6 v2:172.21.15.169:6808/116034 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f94340ca4b0 con 0x7f941406eb90 2023-11-28T11:25:42.137 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-28T11:25:42.138 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T11:25:42.138 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-28T11:25:42.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.137+0000 7f94393ff700 1 -- 172.21.15.92:0/1547418669 >> [v2:172.21.15.169:6808/116034,v1:172.21.15.169:6809/116034] conn(0x7f941406eb90 msgr2=0x7f9414070fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:42.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.137+0000 7f94393ff700 1 --2- 172.21.15.92:0/1547418669 >> [v2:172.21.15.169:6808/116034,v1:172.21.15.169:6809/116034] conn(0x7f941406eb90 0x7f9414070fe0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:42.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.137+0000 7f94393ff700 1 -- 172.21.15.92:0/1547418669 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f94140682f0 msgr2=0x7f941406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:42.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.137+0000 7f94393ff700 1 --2- 172.21.15.92:0/1547418669 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f94140682f0 0x7f941406a7a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f9424009c30 tx=0x7f9424008040 comp rx=0 tx=0).stop 2023-11-28T11:25:42.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.137+0000 7f94393ff700 1 -- 172.21.15.92:0/1547418669 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94340b8790 msgr2=0x7f94341f5d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:42.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.137+0000 7f94393ff700 1 --2- 172.21.15.92:0/1547418669 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94340b8790 0x7f94341f5d10 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f941c00c4d0 tx=0x7f941c00c890 comp rx=0 tx=0).stop 2023-11-28T11:25:42.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.137+0000 7f94337fe700 1 -- 172.21.15.92:0/1547418669 reap_dead start 2023-11-28T11:25:42.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.137+0000 7f94393ff700 1 -- 172.21.15.92:0/1547418669 shutdown_connections 2023-11-28T11:25:42.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.137+0000 7f94393ff700 1 -- 172.21.15.92:0/1547418669 >> 172.21.15.92:0/1547418669 conn(0x7f94340719a0 msgr2=0x7f943415aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:42.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.137+0000 7f94393ff700 1 -- 172.21.15.92:0/1547418669 shutdown_connections 2023-11-28T11:25:42.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.137+0000 7f94393ff700 1 -- 172.21.15.92:0/1547418669 wait complete. 2023-11-28T11:25:42.155 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-11-28T11:25:42.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.352+0000 7fc2ce7ff700 1 Processor -- start 2023-11-28T11:25:42.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.352+0000 7fc2ce7ff700 1 -- start start 2023-11-28T11:25:42.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.354+0000 7fc2ce7ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2c80b8510 0x7fc2c806b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:42.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.354+0000 7fc2ce7ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2c80b88e0 con 0x7fc2c80cd460 2023-11-28T11:25:42.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.354+0000 7fc2ce7ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2c80cd8f0 con 0x7fc2c80d0d60 2023-11-28T11:25:42.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.354+0000 7fc2ce7ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c80cdbd0 con 0x7fc2c80b8510 2023-11-28T11:25:42.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.354+0000 7fc2c77fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc2c80cd460 0x7fc2c806beb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56626/0 (socket says 172.21.15.92:56626) 2023-11-28T11:25:42.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.354+0000 7fc2c77fe700 1 -- 172.21.15.92:0/861029447 learned_addr learned my addr 172.21.15.92:0/861029447 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:25:42.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.354+0000 7fc2c7fff700 1 --2- 172.21.15.92:0/861029447 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2c80b8510 0x7fc2c806b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:42.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.354+0000 7fc2c6ffd700 1 -- 172.21.15.92:0/861029447 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 848315510 0 0) 0x7fc2c80b88e0 con 0x7fc2c80cd460 2023-11-28T11:25:42.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.355+0000 7fc2c6ffd700 1 -- 172.21.15.92:0/861029447 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2b00035b0 con 0x7fc2c80cd460 2023-11-28T11:25:42.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.355+0000 7fc2c6ffd700 1 -- 172.21.15.92:0/861029447 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4270555369 0 0) 0x7fc2b00035b0 con 0x7fc2c80cd460 2023-11-28T11:25:42.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.355+0000 7fc2c6ffd700 1 -- 172.21.15.92:0/861029447 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2c80b8510 msgr2=0x7fc2c806b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:42.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.355+0000 7fc2c6ffd700 1 --2- 172.21.15.92:0/861029447 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2c80b8510 0x7fc2c806b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:42.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.355+0000 7fc2c6ffd700 1 -- 172.21.15.92:0/861029447 >> v1:172.21.15.169:6789/0 conn(0x7fc2c80d0d60 legacy=0x7fc2c80d11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:42.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.355+0000 7fc2c6ffd700 1 -- 172.21.15.92:0/861029447 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c80cfd10 con 0x7fc2c80cd460 2023-11-28T11:25:42.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.355+0000 7fc2c6ffd700 1 -- 172.21.15.92:0/861029447 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532161587 0 0) 0x7fc2bc001670 con 0x7fc2c80cd460 2023-11-28T11:25:42.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.355+0000 7fc2c6ffd700 1 -- 172.21.15.92:0/861029447 >> v1:172.21.15.92:6789/0 conn(0x7fc2c80cd460 legacy=0x7fc2c806beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:42.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.355+0000 7fc2c6ffd700 1 --2- 172.21.15.92:0/861029447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2b0003a30 0x7fc2b0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:42.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.355+0000 7fc2c6ffd700 1 --2- 172.21.15.92:0/861029447 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2c80b8510 0x7fc2b0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:42.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.355+0000 7fc2c6ffd700 1 --2- 172.21.15.92:0/861029447 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2b0004d90 0x7fc2b0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:42.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.355+0000 7fc2c6ffd700 1 -- 172.21.15.92:0/861029447 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c80cdbd0 con 0x7fc2b0003a30 2023-11-28T11:25:42.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.355+0000 7fc2c6ffd700 1 -- 172.21.15.92:0/861029447 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b000bac0 con 0x7fc2b0004d90 2023-11-28T11:25:42.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.355+0000 7fc2c6ffd700 1 -- 172.21.15.92:0/861029447 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b000bc00 con 0x7fc2c80b8510 2023-11-28T11:25:42.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.355+0000 7fc2c7fff700 1 --2- 172.21.15.92:0/861029447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2b0003a30 0x7fc2b0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:42.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.355+0000 7fc2ccd9c700 1 --2- 172.21.15.92:0/861029447 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2b0004d90 0x7fc2b0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:42.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.356+0000 7fc2c77fe700 1 --2- 172.21.15.92:0/861029447 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2c80b8510 0x7fc2b0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:42.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.356+0000 7fc2c7fff700 1 -- 172.21.15.92:0/861029447 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2c80b8510 msgr2=0x7fc2b0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:42.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.356+0000 7fc2c7fff700 1 --2- 172.21.15.92:0/861029447 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2c80b8510 0x7fc2b0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:42.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.356+0000 7fc2c7fff700 1 -- 172.21.15.92:0/861029447 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2b0004d90 msgr2=0x7fc2b0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:42.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.356+0000 7fc2c7fff700 1 --2- 172.21.15.92:0/861029447 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2b0004d90 0x7fc2b0005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:42.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.356+0000 7fc2c7fff700 1 -- 172.21.15.92:0/861029447 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b00035b0 con 0x7fc2b0003a30 2023-11-28T11:25:42.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.356+0000 7fc2c7fff700 1 --2- 172.21.15.92:0/861029447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2b0003a30 0x7fc2b0003e00 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8009b60 tx=0x7fc2b0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bf367f6428f9f7d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:42.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.356+0000 7fc2c6ffd700 1 -- 172.21.15.92:0/861029447 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b800e6c0 con 0x7fc2b0003a30 2023-11-28T11:25:42.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.356+0000 7fc2c6ffd700 1 -- 172.21.15.92:0/861029447 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2b800e820 con 0x7fc2b0003a30 2023-11-28T11:25:42.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.356+0000 7fc2c6ffd700 1 -- 172.21.15.92:0/861029447 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b800eb90 con 0x7fc2b0003a30 2023-11-28T11:25:42.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.357+0000 7fc2ce7ff700 1 -- 172.21.15.92:0/861029447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2b0003a30 msgr2=0x7fc2b0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:42.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.357+0000 7fc2ce7ff700 1 --2- 172.21.15.92:0/861029447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2b0003a30 0x7fc2b0003e00 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8009b60 tx=0x7fc2b0005880 comp rx=0 tx=0).stop 2023-11-28T11:25:42.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.357+0000 7fc2ccd9c700 1 -- 172.21.15.92:0/861029447 reap_dead start 2023-11-28T11:25:42.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.357+0000 7fc2ce7ff700 1 -- 172.21.15.92:0/861029447 shutdown_connections 2023-11-28T11:25:42.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.357+0000 7fc2ce7ff700 1 -- 172.21.15.92:0/861029447 >> 172.21.15.92:0/861029447 conn(0x7fc2c80719a0 msgr2=0x7fc2c8071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:42.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.357+0000 7fc2ce7ff700 1 -- 172.21.15.92:0/861029447 shutdown_connections 2023-11-28T11:25:42.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.357+0000 7fc2ce7ff700 1 -- 172.21.15.92:0/861029447 wait complete. 2023-11-28T11:25:42.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.358+0000 7fc2ce7ff700 1 Processor -- start 2023-11-28T11:25:42.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.359+0000 7fc2ce7ff700 1 -- start start 2023-11-28T11:25:42.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.359+0000 7fc2ce7ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2b0003a30 0x7fc2c815b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:42.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.359+0000 7fc2ce7ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2c81f1d60 0x7fc2c81f21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:42.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.359+0000 7fc2ce7ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2c80cc680 0x7fc2c80ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:42.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.360+0000 7fc2ce7ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c81f2d70 con 0x7fc2c80cc680 2023-11-28T11:25:42.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.360+0000 7fc2ce7ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c8156b40 con 0x7fc2c81f1d60 2023-11-28T11:25:42.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.360+0000 7fc2ce7ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c81f27b0 con 0x7fc2b0003a30 2023-11-28T11:25:42.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.360+0000 7fc2ccd9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2c80cc680 0x7fc2c80ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:42.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.360+0000 7fc2ccd9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2c80cc680 0x7fc2c80ccb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38544/0 (socket says 172.21.15.92:38544) 2023-11-28T11:25:42.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.360+0000 7fc2ccd9c700 1 -- 172.21.15.92:0/448041641 learned_addr learned my addr 172.21.15.92:0/448041641 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:42.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.360+0000 7fc2c7fff700 1 --2- 172.21.15.92:0/448041641 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2b0003a30 0x7fc2c815b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:42.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.360+0000 7fc2c77fe700 1 --2- 172.21.15.92:0/448041641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2c81f1d60 0x7fc2c81f21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:42.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.360+0000 7fc2ccd9c700 1 -- 172.21.15.92:0/448041641 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2b0003a30 msgr2=0x7fc2c815b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:42.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.360+0000 7fc2ccd9c700 1 --2- 172.21.15.92:0/448041641 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2b0003a30 0x7fc2c815b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:42.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.360+0000 7fc2ccd9c700 1 -- 172.21.15.92:0/448041641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2c81f1d60 msgr2=0x7fc2c81f21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:42.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.360+0000 7fc2ccd9c700 1 --2- 172.21.15.92:0/448041641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2c81f1d60 0x7fc2c81f21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:42.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.360+0000 7fc2ccd9c700 1 -- 172.21.15.92:0/448041641 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c80d29c0 con 0x7fc2c80cc680 2023-11-28T11:25:42.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.360+0000 7fc2c7fff700 1 --2- 172.21.15.92:0/448041641 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2b0003a30 0x7fc2c815b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:25:42.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.360+0000 7fc2c77fe700 1 --2- 172.21.15.92:0/448041641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2c81f1d60 0x7fc2c81f21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:25:42.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.360+0000 7fc2ccd9c700 1 --2- 172.21.15.92:0/448041641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2c80cc680 0x7fc2c80ccb00 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fc2c80d1130 tx=0x7fc2c000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:42.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.360+0000 7fc2c4ff9700 1 -- 172.21.15.92:0/448041641 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0003c10 con 0x7fc2c80cc680 2023-11-28T11:25:42.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.361+0000 7fc2c4ff9700 1 -- 172.21.15.92:0/448041641 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2c0003d90 con 0x7fc2c80cc680 2023-11-28T11:25:42.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.361+0000 7fc2ce7ff700 1 -- 172.21.15.92:0/448041641 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2c80cfff0 con 0x7fc2c80cc680 2023-11-28T11:25:42.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.361+0000 7fc2c4ff9700 1 -- 172.21.15.92:0/448041641 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0005310 con 0x7fc2c80cc680 2023-11-28T11:25:42.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.361+0000 7fc2ce7ff700 1 -- 172.21.15.92:0/448041641 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2c80d2b50 con 0x7fc2c80cc680 2023-11-28T11:25:42.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.362+0000 7fc2ce7ff700 1 -- 172.21.15.92:0/448041641 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2c8158050 con 0x7fc2c80cc680 2023-11-28T11:25:42.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.363+0000 7fc2c4ff9700 1 -- 172.21.15.92:0/448041641 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc2c0005470 con 0x7fc2c80cc680 2023-11-28T11:25:42.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.363+0000 7fc2c4ff9700 1 --2- 172.21.15.92:0/448041641 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc2b40682f0 0x7fc2b406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:42.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.364+0000 7fc2c4ff9700 1 -- 172.21.15.92:0/448041641 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fc2c0085dd0 con 0x7fc2c80cc680 2023-11-28T11:25:42.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.364+0000 7fc2c7fff700 1 --2- 172.21.15.92:0/448041641 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc2b40682f0 0x7fc2b406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:42.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.364+0000 7fc2c7fff700 1 --2- 172.21.15.92:0/448041641 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc2b40682f0 0x7fc2b406a7a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8158660 tx=0x7fc2b8008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:42.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.367+0000 7fc2c4ff9700 1 -- 172.21.15.92:0/448041641 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc2c0052720 con 0x7fc2c80cc680 2023-11-28T11:25:42.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.558+0000 7fc2ce7ff700 1 -- 172.21.15.92:0/448041641 --> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fc2c8067880 con 0x7fc2b40682f0 2023-11-28T11:25:42.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.558+0000 7fc2c4ff9700 1 -- 172.21.15.92:0/448041641 <== mgr.4103 v2:172.21.15.92:6832/115482 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc2c8067880 con 0x7fc2b40682f0 2023-11-28T11:25:42.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.560+0000 7fc2ce7ff700 1 -- 172.21.15.92:0/448041641 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc2b40682f0 msgr2=0x7fc2b406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:42.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.560+0000 7fc2ce7ff700 1 --2- 172.21.15.92:0/448041641 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc2b40682f0 0x7fc2b406a7a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8158660 tx=0x7fc2b8008030 comp rx=0 tx=0).stop 2023-11-28T11:25:42.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.560+0000 7fc2ce7ff700 1 -- 172.21.15.92:0/448041641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2c80cc680 msgr2=0x7fc2c80ccb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:42.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.560+0000 7fc2ce7ff700 1 --2- 172.21.15.92:0/448041641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2c80cc680 0x7fc2c80ccb00 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fc2c80d1130 tx=0x7fc2c000ef20 comp rx=0 tx=0).stop 2023-11-28T11:25:42.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.561+0000 7fc2ce7ff700 1 -- 172.21.15.92:0/448041641 shutdown_connections 2023-11-28T11:25:42.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.561+0000 7fc2ce7ff700 1 --2- 172.21.15.92:0/448041641 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc2b40682f0 0x7fc2b406a7a0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:42.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.561+0000 7fc2ce7ff700 1 --2- 172.21.15.92:0/448041641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2c80cc680 0x7fc2c80ccb00 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:42.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.561+0000 7fc2ce7ff700 1 --2- 172.21.15.92:0/448041641 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc2b0003a30 0x7fc2c815b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:42.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.561+0000 7fc2ce7ff700 1 --2- 172.21.15.92:0/448041641 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc2c81f1d60 0x7fc2c81f21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:42.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.561+0000 7fc2ce7ff700 1 -- 172.21.15.92:0/448041641 >> 172.21.15.92:0/448041641 conn(0x7fc2c80719a0 msgr2=0x7fc2c815c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:42.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.561+0000 7fc2ce7ff700 1 -- 172.21.15.92:0/448041641 shutdown_connections 2023-11-28T11:25:42.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.561+0000 7fc2ce7ff700 1 -- 172.21.15.92:0/448041641 wait complete. 2023-11-28T11:25:42.564 INFO:teuthology.orchestra.run.smithi092.stderr:instructed osd(s) 6 to deep-scrub 2023-11-28T11:25:42.580 INFO:tasks.ceph:Scrubbing osd.7 2023-11-28T11:25:42.580 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-11-28T11:25:42.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.779+0000 7f5faefff700 1 Processor -- start 2023-11-28T11:25:42.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.780+0000 7f5faefff700 1 -- start start 2023-11-28T11:25:42.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.781+0000 7f5faefff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fa80cd460 0x7f5fa806b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:42.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.781+0000 7f5faefff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f5fa806be80 0x7f5fa80d0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:42.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.781+0000 7f5faefff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fa80b5240 con 0x7f5fa80b8790 2023-11-28T11:25:42.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.781+0000 7f5faefff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fa806c300 con 0x7f5fa806be80 2023-11-28T11:25:42.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.781+0000 7f5faefff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa80b5520 con 0x7f5fa80cd460 2023-11-28T11:25:42.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.781+0000 7f5facd9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f5fa80b8790 0x7f5fa80cfb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56634/0 (socket says 172.21.15.92:56634) 2023-11-28T11:25:42.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.781+0000 7f5facd9b700 1 -- 172.21.15.92:0/3815817813 learned_addr learned my addr 172.21.15.92:0/3815817813 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:25:42.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.781+0000 7f5fad59c700 1 --2- 172.21.15.92:0/3815817813 >> v2:172.21.15.169:3300/0 conn(0x7f5fa806be80 0x7f5fa80d0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:42.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.781+0000 7f5fa7fff700 1 --2- 172.21.15.92:0/3815817813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fa80cd460 0x7f5fa806b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:42.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.781+0000 7f5fa77fe700 1 -- 172.21.15.92:0/3815817813 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2191094049 0 0) 0x7f5fa80b5240 con 0x7f5fa80b8790 2023-11-28T11:25:42.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.781+0000 7f5fa77fe700 1 -- 172.21.15.92:0/3815817813 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f980035b0 con 0x7f5fa80b8790 2023-11-28T11:25:42.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.782+0000 7f5fa77fe700 1 -- 172.21.15.92:0/3815817813 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 992430110 0 0) 0x7f5f980035b0 con 0x7f5fa80b8790 2023-11-28T11:25:42.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.782+0000 7f5fa77fe700 1 -- 172.21.15.92:0/3815817813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fa80cd460 msgr2=0x7f5fa806b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:42.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.782+0000 7f5fa77fe700 1 --2- 172.21.15.92:0/3815817813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fa80cd460 0x7f5fa806b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:42.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.782+0000 7f5fa77fe700 1 -- 172.21.15.92:0/3815817813 >> v2:172.21.15.169:3300/0 conn(0x7f5fa806be80 msgr2=0x7f5fa80d0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:42.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.782+0000 7f5fa77fe700 1 --2- 172.21.15.92:0/3815817813 >> v2:172.21.15.169:3300/0 conn(0x7f5fa806be80 0x7f5fa80d0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:42.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.782+0000 7f5fa77fe700 1 -- 172.21.15.92:0/3815817813 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa80d0a80 con 0x7f5fa80b8790 2023-11-28T11:25:42.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.782+0000 7f5fa77fe700 1 -- 172.21.15.92:0/3815817813 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3532161587 0 0) 0x7f5f9c001670 con 0x7f5fa80b8790 2023-11-28T11:25:42.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.782+0000 7f5fa77fe700 1 -- 172.21.15.92:0/3815817813 >> v1:172.21.15.92:6789/0 conn(0x7f5fa80b8790 legacy=0x7f5fa80cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:42.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.782+0000 7f5fa77fe700 1 --2- 172.21.15.92:0/3815817813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f98003a00 0x7f5f98003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:42.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.782+0000 7f5fa77fe700 1 --2- 172.21.15.92:0/3815817813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fa80cd460 0x7f5f98004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:42.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.782+0000 7f5fa77fe700 1 --2- 172.21.15.92:0/3815817813 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f98004d90 0x7f5f98005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:42.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.782+0000 7f5fa77fe700 1 -- 172.21.15.92:0/3815817813 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa806c300 con 0x7f5f98004d90 2023-11-28T11:25:42.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.782+0000 7f5fa77fe700 1 -- 172.21.15.92:0/3815817813 --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa80b5520 con 0x7f5f98003a00 2023-11-28T11:25:42.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.782+0000 7f5fa77fe700 1 -- 172.21.15.92:0/3815817813 --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9800ba10 con 0x7f5fa80cd460 2023-11-28T11:25:42.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.782+0000 7f5facd9b700 1 --2- 172.21.15.92:0/3815817813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f98003a00 0x7f5f98003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:42.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.782+0000 7f5fad59c700 1 --2- 172.21.15.92:0/3815817813 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f98004d90 0x7f5f98005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:42.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.782+0000 7f5fa7fff700 1 --2- 172.21.15.92:0/3815817813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fa80cd460 0x7f5f98004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:42.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.782+0000 7f5fad59c700 1 -- 172.21.15.92:0/3815817813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fa80cd460 msgr2=0x7f5f98004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:42.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.782+0000 7f5fad59c700 1 --2- 172.21.15.92:0/3815817813 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fa80cd460 0x7f5f98004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:42.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.782+0000 7f5fad59c700 1 -- 172.21.15.92:0/3815817813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f98003a00 msgr2=0x7f5f98003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:42.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.782+0000 7f5fad59c700 1 --2- 172.21.15.92:0/3815817813 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f98003a00 0x7f5f98003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:42.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.782+0000 7f5fad59c700 1 -- 172.21.15.92:0/3815817813 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f980035b0 con 0x7f5f98004d90 2023-11-28T11:25:42.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.783+0000 7f5fad59c700 1 --2- 172.21.15.92:0/3815817813 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f98004d90 0x7f5f98005200 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0009fc0 tx=0x7f5f98005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6211b3936836532 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:42.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.783+0000 7f5fa77fe700 1 -- 172.21.15.92:0/3815817813 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa000ec30 con 0x7f5f98004d90 2023-11-28T11:25:42.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.783+0000 7f5fa77fe700 1 -- 172.21.15.92:0/3815817813 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fa000f040 con 0x7f5f98004d90 2023-11-28T11:25:42.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.783+0000 7f5fa77fe700 1 -- 172.21.15.92:0/3815817813 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa0014480 con 0x7f5f98004d90 2023-11-28T11:25:42.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.783+0000 7f5faefff700 1 -- 172.21.15.92:0/3815817813 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f98004d90 msgr2=0x7f5f98005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:42.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.783+0000 7f5faefff700 1 --2- 172.21.15.92:0/3815817813 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f98004d90 0x7f5f98005200 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0009fc0 tx=0x7f5f98005880 comp rx=0 tx=0).stop 2023-11-28T11:25:42.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.783+0000 7f5fad59c700 1 -- 172.21.15.92:0/3815817813 reap_dead start 2023-11-28T11:25:42.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.784+0000 7f5faefff700 1 -- 172.21.15.92:0/3815817813 shutdown_connections 2023-11-28T11:25:42.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.784+0000 7f5faefff700 1 -- 172.21.15.92:0/3815817813 >> 172.21.15.92:0/3815817813 conn(0x7f5fa80719a0 msgr2=0x7f5fa8071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:42.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.784+0000 7f5faefff700 1 -- 172.21.15.92:0/3815817813 shutdown_connections 2023-11-28T11:25:42.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.784+0000 7f5faefff700 1 -- 172.21.15.92:0/3815817813 wait complete. 2023-11-28T11:25:42.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.785+0000 7f5faefff700 1 Processor -- start 2023-11-28T11:25:42.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.786+0000 7f5faefff700 1 -- start start 2023-11-28T11:25:42.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.786+0000 7f5faefff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f98004d90 0x7f5fa81f6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:42.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.786+0000 7f5faefff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fa80caf30 0x7f5fa80b09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:42.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.786+0000 7f5faefff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fa80cc570 0x7f5fa80b0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:42.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.786+0000 7f5faefff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa81f65a0 con 0x7f5fa80cc570 2023-11-28T11:25:42.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.786+0000 7f5faefff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa80cb4d0 con 0x7f5f98004d90 2023-11-28T11:25:42.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.786+0000 7f5faefff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa80cb7b0 con 0x7f5fa80caf30 2023-11-28T11:25:42.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.787+0000 7f5fad59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fa80cc570 0x7f5fa80b0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:42.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.787+0000 7f5fad59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fa80cc570 0x7f5fa80b0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38560/0 (socket says 172.21.15.92:38560) 2023-11-28T11:25:42.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.787+0000 7f5fad59c700 1 -- 172.21.15.92:0/2162496192 learned_addr learned my addr 172.21.15.92:0/2162496192 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:42.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.787+0000 7f5facd9b700 1 --2- 172.21.15.92:0/2162496192 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f98004d90 0x7f5fa81f6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:42.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.787+0000 7f5fa7fff700 1 --2- 172.21.15.92:0/2162496192 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fa80caf30 0x7f5fa80b09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:42.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.787+0000 7f5fad59c700 1 -- 172.21.15.92:0/2162496192 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fa80caf30 msgr2=0x7f5fa80b09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:42.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.787+0000 7f5fad59c700 1 --2- 172.21.15.92:0/2162496192 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fa80caf30 0x7f5fa80b09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:42.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.787+0000 7f5fad59c700 1 -- 172.21.15.92:0/2162496192 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f98004d90 msgr2=0x7f5fa81f6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:42.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.787+0000 7f5fad59c700 1 --2- 172.21.15.92:0/2162496192 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f5f98004d90 0x7f5fa81f6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:42.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.787+0000 7f5fad59c700 1 -- 172.21.15.92:0/2162496192 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa806b940 con 0x7f5fa80cc570 2023-11-28T11:25:42.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.787+0000 7f5fa7fff700 1 --2- 172.21.15.92:0/2162496192 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f5fa80caf30 0x7f5fa80b09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:25:42.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.787+0000 7f5fad59c700 1 --2- 172.21.15.92:0/2162496192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fa80cc570 0x7f5fa80b0ee0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f5fa000b010 tx=0x7f5fa0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:42.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.788+0000 7f5fa57fa700 1 -- 172.21.15.92:0/2162496192 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa0007000 con 0x7f5fa80cc570 2023-11-28T11:25:42.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.788+0000 7f5fa57fa700 1 -- 172.21.15.92:0/2162496192 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fa0014c90 con 0x7f5fa80cc570 2023-11-28T11:25:42.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.788+0000 7f5faefff700 1 -- 172.21.15.92:0/2162496192 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fa80b5800 con 0x7f5fa80cc570 2023-11-28T11:25:42.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.788+0000 7f5fa57fa700 1 -- 172.21.15.92:0/2162496192 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa0027470 con 0x7f5fa80cc570 2023-11-28T11:25:42.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.788+0000 7f5faefff700 1 -- 172.21.15.92:0/2162496192 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fa806bad0 con 0x7f5fa80cc570 2023-11-28T11:25:42.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.788+0000 7f5faefff700 1 -- 172.21.15.92:0/2162496192 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5fa8067880 con 0x7f5fa80cc570 2023-11-28T11:25:42.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.789+0000 7f5fa57fa700 1 -- 172.21.15.92:0/2162496192 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5fa001e430 con 0x7f5fa80cc570 2023-11-28T11:25:42.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.791+0000 7f5fa57fa700 1 --2- 172.21.15.92:0/2162496192 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5f8c0682f0 0x7f5f8c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:42.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.791+0000 7f5fa57fa700 1 -- 172.21.15.92:0/2162496192 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f5fa0087d00 con 0x7f5fa80cc570 2023-11-28T11:25:42.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.791+0000 7f5facd9b700 1 --2- 172.21.15.92:0/2162496192 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5f8c0682f0 0x7f5f8c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:42.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.791+0000 7f5fa57fa700 1 --2- 172.21.15.92:0/2162496192 >> [v2:172.21.15.169:6821/116044,v1:172.21.15.169:6823/116044] conn(0x7f5f8c06eb90 0x7f5f8c070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:42.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.791+0000 7f5fa57fa700 1 -- 172.21.15.92:0/2162496192 --> [v2:172.21.15.169:6821/116044,v1:172.21.15.169:6823/116044] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5f8c0716b0 con 0x7f5f8c06eb90 2023-11-28T11:25:42.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.791+0000 7f5fa57fa700 1 -- 172.21.15.92:0/2162496192 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5fa008a2e0 con 0x7f5fa80cc570 2023-11-28T11:25:42.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.791+0000 7f5facd9b700 1 --2- 172.21.15.92:0/2162496192 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5f8c0682f0 0x7f5f8c06a7a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c000c00 tx=0x7f5f9c007980 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:42.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.792+0000 7f5fa7fff700 1 --2- 172.21.15.92:0/2162496192 >> [v2:172.21.15.169:6821/116044,v1:172.21.15.169:6823/116044] conn(0x7f5f8c06eb90 0x7f5f8c070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:42.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.792+0000 7f5fa7fff700 1 --2- 172.21.15.92:0/2162496192 >> [v2:172.21.15.169:6821/116044,v1:172.21.15.169:6823/116044] conn(0x7f5f8c06eb90 0x7f5f8c070fe0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:42.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.793+0000 7f5fa57fa700 1 -- 172.21.15.92:0/2162496192 <== osd.7 v2:172.21.15.169:6821/116044 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f5f8c0716b0 con 0x7f5f8c06eb90 2023-11-28T11:25:42.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.808+0000 7f5faefff700 1 -- 172.21.15.92:0/2162496192 --> [v2:172.21.15.169:6821/116044,v1:172.21.15.169:6823/116044] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f5fa815dde0 con 0x7f5f8c06eb90 2023-11-28T11:25:42.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.808+0000 7f5fa57fa700 1 -- 172.21.15.92:0/2162496192 <== osd.7 v2:172.21.15.169:6821/116044 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f5fa815dde0 con 0x7f5f8c06eb90 2023-11-28T11:25:42.810 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-28T11:25:42.810 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T11:25:42.811 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-28T11:25:42.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.809+0000 7f5faefff700 1 -- 172.21.15.92:0/2162496192 >> [v2:172.21.15.169:6821/116044,v1:172.21.15.169:6823/116044] conn(0x7f5f8c06eb90 msgr2=0x7f5f8c070fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:42.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.809+0000 7f5faefff700 1 --2- 172.21.15.92:0/2162496192 >> [v2:172.21.15.169:6821/116044,v1:172.21.15.169:6823/116044] conn(0x7f5f8c06eb90 0x7f5f8c070fe0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:42.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.809+0000 7f5faefff700 1 -- 172.21.15.92:0/2162496192 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5f8c0682f0 msgr2=0x7f5f8c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:42.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.809+0000 7f5faefff700 1 --2- 172.21.15.92:0/2162496192 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f5f8c0682f0 0x7f5f8c06a7a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c000c00 tx=0x7f5f9c007980 comp rx=0 tx=0).stop 2023-11-28T11:25:42.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.809+0000 7f5faefff700 1 -- 172.21.15.92:0/2162496192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fa80cc570 msgr2=0x7f5fa80b0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:42.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.809+0000 7f5faefff700 1 --2- 172.21.15.92:0/2162496192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fa80cc570 0x7f5fa80b0ee0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f5fa000b010 tx=0x7f5fa0009750 comp rx=0 tx=0).stop 2023-11-28T11:25:42.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.809+0000 7f5fad59c700 1 -- 172.21.15.92:0/2162496192 reap_dead start 2023-11-28T11:25:42.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.809+0000 7f5faefff700 1 -- 172.21.15.92:0/2162496192 shutdown_connections 2023-11-28T11:25:42.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.809+0000 7f5faefff700 1 -- 172.21.15.92:0/2162496192 >> 172.21.15.92:0/2162496192 conn(0x7f5fa80719a0 msgr2=0x7f5fa80d2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:42.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.809+0000 7f5faefff700 1 -- 172.21.15.92:0/2162496192 shutdown_connections 2023-11-28T11:25:42.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:42.809+0000 7f5faefff700 1 -- 172.21.15.92:0/2162496192 wait complete. 2023-11-28T11:25:42.827 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-11-28T11:25:43.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.024+0000 7f43d85ff700 1 Processor -- start 2023-11-28T11:25:43.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.024+0000 7f43d85ff700 1 -- start start 2023-11-28T11:25:43.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.025+0000 7f43d85ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f43d00b8510 0x7f43d006b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:43.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.025+0000 7f43d85ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f43d00cd460 0x7f43d006beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:43.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.026+0000 7f43d85ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43d00b88e0 con 0x7f43d006c3f0 2023-11-28T11:25:43.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.026+0000 7f43d85ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43d00cdbd0 con 0x7f43d00cd460 2023-11-28T11:25:43.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.026+0000 7f43d85ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43d00d02d0 con 0x7f43d00b8510 2023-11-28T11:25:43.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.026+0000 7f43d6b9c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f43d006c3f0 0x7f43d015cc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56640/0 (socket says 172.21.15.92:56640) 2023-11-28T11:25:43.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.026+0000 7f43d6b9c700 1 -- 172.21.15.92:0/1980022983 learned_addr learned my addr 172.21.15.92:0/1980022983 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-11-28T11:25:43.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.026+0000 7f43d5b9a700 1 --2- 172.21.15.92:0/1980022983 >> v2:172.21.15.92:3300/0 conn(0x7f43d00cd460 0x7f43d006beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:43.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.026+0000 7f43d639b700 1 --2- 172.21.15.92:0/1980022983 >> v2:172.21.15.169:3300/0 conn(0x7f43d00b8510 0x7f43d006b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:43.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.026+0000 7f43d5399700 1 -- 172.21.15.92:0/1980022983 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1522292599 0 0) 0x7f43d00b88e0 con 0x7f43d006c3f0 2023-11-28T11:25:43.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.026+0000 7f43d5399700 1 -- 172.21.15.92:0/1980022983 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43bc0035b0 con 0x7f43d006c3f0 2023-11-28T11:25:43.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.026+0000 7f43d5b9a700 1 -- 172.21.15.92:0/1980022983 >> v2:172.21.15.169:3300/0 conn(0x7f43d00b8510 msgr2=0x7f43d006b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:43.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.026+0000 7f43d5b9a700 1 --2- 172.21.15.92:0/1980022983 >> v2:172.21.15.169:3300/0 conn(0x7f43d00b8510 0x7f43d006b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:43.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.026+0000 7f43d5b9a700 1 -- 172.21.15.92:0/1980022983 >> v1:172.21.15.92:6789/0 conn(0x7f43d006c3f0 legacy=0x7f43d015cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:43.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.026+0000 7f43d5b9a700 1 -- 172.21.15.92:0/1980022983 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d00cd8f0 con 0x7f43d00cd460 2023-11-28T11:25:43.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.027+0000 7f43d5b9a700 1 --2- 172.21.15.92:0/1980022983 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43d00cd460 0x7f43d006beb0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f43c0009870 tx=0x7f43c0009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11f97c9fdce8132 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:43.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.027+0000 7f43d5399700 1 -- 172.21.15.92:0/1980022983 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c0011830 con 0x7f43d00cd460 2023-11-28T11:25:43.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.027+0000 7f43d5399700 1 -- 172.21.15.92:0/1980022983 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c0011990 con 0x7f43d00cd460 2023-11-28T11:25:43.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.027+0000 7f43d5399700 1 -- 172.21.15.92:0/1980022983 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c0011d20 con 0x7f43d00cd460 2023-11-28T11:25:43.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.027+0000 7f43d85ff700 1 -- 172.21.15.92:0/1980022983 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43d00cd460 msgr2=0x7f43d006beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:43.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.027+0000 7f43d85ff700 1 --2- 172.21.15.92:0/1980022983 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43d00cd460 0x7f43d006beb0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f43c0009870 tx=0x7f43c0009b80 comp rx=0 tx=0).stop 2023-11-28T11:25:43.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.027+0000 7f43d85ff700 1 -- 172.21.15.92:0/1980022983 shutdown_connections 2023-11-28T11:25:43.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.027+0000 7f43d85ff700 1 --2- 172.21.15.92:0/1980022983 >> v2:172.21.15.169:3300/0 conn(0x7f43d00b8510 0x7f43d006b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:43.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.028+0000 7f43d85ff700 1 --2- 172.21.15.92:0/1980022983 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43d00cd460 0x7f43d006beb0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:43.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.028+0000 7f43d85ff700 1 -- 172.21.15.92:0/1980022983 >> 172.21.15.92:0/1980022983 conn(0x7f43d00719a0 msgr2=0x7f43d0071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:43.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.028+0000 7f43d85ff700 1 -- 172.21.15.92:0/1980022983 shutdown_connections 2023-11-28T11:25:43.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.028+0000 7f43d85ff700 1 -- 172.21.15.92:0/1980022983 wait complete. 2023-11-28T11:25:43.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.029+0000 7f43d85ff700 1 Processor -- start 2023-11-28T11:25:43.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.029+0000 7f43d85ff700 1 -- start start 2023-11-28T11:25:43.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.030+0000 7f43d85ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43d006c3f0 0x7f43d00d0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:43.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.030+0000 7f43d85ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43d00b8510 0x7f43d01e5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:43.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.030+0000 7f43d85ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43d00cd460 0x7f43d015b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:43.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.030+0000 7f43d85ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d01e70f0 con 0x7f43d006c3f0 2023-11-28T11:25:43.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.030+0000 7f43d85ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d01e6050 con 0x7f43d00b8510 2023-11-28T11:25:43.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.030+0000 7f43d85ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43d01e6330 con 0x7f43d00cd460 2023-11-28T11:25:43.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.031+0000 7f43d639b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43d006c3f0 0x7f43d00d0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:43.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.031+0000 7f43d639b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43d006c3f0 0x7f43d00d0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38572/0 (socket says 172.21.15.92:38572) 2023-11-28T11:25:43.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.031+0000 7f43d639b700 1 -- 172.21.15.92:0/1480058553 learned_addr learned my addr 172.21.15.92:0/1480058553 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:43.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.031+0000 7f43d639b700 1 -- 172.21.15.92:0/1480058553 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43d00cd460 msgr2=0x7f43d015b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:43.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.031+0000 7f43d639b700 1 --2- 172.21.15.92:0/1480058553 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43d00cd460 0x7f43d015b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:43.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.031+0000 7f43d639b700 1 -- 172.21.15.92:0/1480058553 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43d00b8510 msgr2=0x7f43d01e5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:43.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.031+0000 7f43d639b700 1 --2- 172.21.15.92:0/1480058553 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43d00b8510 0x7f43d01e5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:43.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.031+0000 7f43d639b700 1 -- 172.21.15.92:0/1480058553 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c00094a0 con 0x7f43d006c3f0 2023-11-28T11:25:43.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.031+0000 7f43d639b700 1 --2- 172.21.15.92:0/1480058553 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43d006c3f0 0x7f43d00d0e80 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f43cc00e9b0 tx=0x7f43cc00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:43.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.031+0000 7f43c6ffd700 1 -- 172.21.15.92:0/1480058553 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43cc009710 con 0x7f43d006c3f0 2023-11-28T11:25:43.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.031+0000 7f43c6ffd700 1 -- 172.21.15.92:0/1480058553 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43cc00c850 con 0x7f43d006c3f0 2023-11-28T11:25:43.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.031+0000 7f43d85ff700 1 -- 172.21.15.92:0/1480058553 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43d00cfff0 con 0x7f43d006c3f0 2023-11-28T11:25:43.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.031+0000 7f43c6ffd700 1 -- 172.21.15.92:0/1480058553 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43cc00cbc0 con 0x7f43d006c3f0 2023-11-28T11:25:43.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.032+0000 7f43d85ff700 1 -- 172.21.15.92:0/1480058553 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43d00cfd10 con 0x7f43d006c3f0 2023-11-28T11:25:43.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.033+0000 7f43d85ff700 1 -- 172.21.15.92:0/1480058553 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43d01e6f40 con 0x7f43d006c3f0 2023-11-28T11:25:43.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.033+0000 7f43c6ffd700 1 -- 172.21.15.92:0/1480058553 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f43cc019030 con 0x7f43d006c3f0 2023-11-28T11:25:43.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.034+0000 7f43c6ffd700 1 --2- 172.21.15.92:0/1480058553 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f43b40682f0 0x7f43b406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:43.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.034+0000 7f43c6ffd700 1 -- 172.21.15.92:0/1480058553 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f43cc085ae0 con 0x7f43d006c3f0 2023-11-28T11:25:43.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.034+0000 7f43d5b9a700 1 --2- 172.21.15.92:0/1480058553 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f43b40682f0 0x7f43b406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:43.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.035+0000 7f43d5b9a700 1 --2- 172.21.15.92:0/1480058553 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f43b40682f0 0x7f43b406a7a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f43d00b1de0 tx=0x7f43c0010000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:43.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.038+0000 7f43c6ffd700 1 -- 172.21.15.92:0/1480058553 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f43cc052500 con 0x7f43d006c3f0 2023-11-28T11:25:43.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.226+0000 7f43d85ff700 1 -- 172.21.15.92:0/1480058553 --> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f43d0067880 con 0x7f43b40682f0 2023-11-28T11:25:43.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.227+0000 7f43c6ffd700 1 -- 172.21.15.92:0/1480058553 <== mgr.4103 v2:172.21.15.92:6832/115482 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f43d0067880 con 0x7f43b40682f0 2023-11-28T11:25:43.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.229+0000 7f43d85ff700 1 -- 172.21.15.92:0/1480058553 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f43b40682f0 msgr2=0x7f43b406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:43.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.229+0000 7f43d85ff700 1 --2- 172.21.15.92:0/1480058553 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f43b40682f0 0x7f43b406a7a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f43d00b1de0 tx=0x7f43c0010000 comp rx=0 tx=0).stop 2023-11-28T11:25:43.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.229+0000 7f43d85ff700 1 -- 172.21.15.92:0/1480058553 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43d006c3f0 msgr2=0x7f43d00d0e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:43.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.229+0000 7f43d85ff700 1 --2- 172.21.15.92:0/1480058553 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43d006c3f0 0x7f43d00d0e80 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f43cc00e9b0 tx=0x7f43cc00ed70 comp rx=0 tx=0).stop 2023-11-28T11:25:43.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.229+0000 7f43d85ff700 1 -- 172.21.15.92:0/1480058553 shutdown_connections 2023-11-28T11:25:43.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.229+0000 7f43d85ff700 1 --2- 172.21.15.92:0/1480058553 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f43d00cd460 0x7f43d015b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:43.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.229+0000 7f43d85ff700 1 --2- 172.21.15.92:0/1480058553 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f43b40682f0 0x7f43b406a7a0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:43.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.229+0000 7f43d85ff700 1 --2- 172.21.15.92:0/1480058553 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43d006c3f0 0x7f43d00d0e80 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:43.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.229+0000 7f43d85ff700 1 --2- 172.21.15.92:0/1480058553 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f43d00b8510 0x7f43d01e5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:43.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.229+0000 7f43d85ff700 1 -- 172.21.15.92:0/1480058553 >> 172.21.15.92:0/1480058553 conn(0x7f43d00719a0 msgr2=0x7f43d00b78f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:43.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.229+0000 7f43d85ff700 1 -- 172.21.15.92:0/1480058553 shutdown_connections 2023-11-28T11:25:43.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.229+0000 7f43d85ff700 1 -- 172.21.15.92:0/1480058553 wait complete. 2023-11-28T11:25:43.232 INFO:teuthology.orchestra.run.smithi092.stderr:instructed osd(s) 7 to deep-scrub 2023-11-28T11:25:43.248 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:25:43.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.443+0000 7fc7379ff700 1 Processor -- start 2023-11-28T11:25:43.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.443+0000 7fc7379ff700 1 -- start start 2023-11-28T11:25:43.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.444+0000 7fc7379ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc730158cb0 0x7fc7300ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:43.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.444+0000 7fc7379ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fc730159080 0x7fc73006dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:43.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.444+0000 7fc7379ff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc73006d870 con 0x7fc7300c9300 2023-11-28T11:25:43.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.444+0000 7fc7379ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc73006cc60 con 0x7fc730158cb0 2023-11-28T11:25:43.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.444+0000 7fc7379ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc73006cf40 con 0x7fc730159080 2023-11-28T11:25:43.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.444+0000 7fc73579b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc730158cb0 0x7fc7300ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:43.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.445+0000 7fc73579b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc730158cb0 0x7fc7300ca1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38580/0 (socket says 172.21.15.92:38580) 2023-11-28T11:25:43.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.445+0000 7fc73579b700 1 -- 172.21.15.92:0/2965858079 learned_addr learned my addr 172.21.15.92:0/2965858079 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:43.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.445+0000 7fc734f9a700 1 --2- 172.21.15.92:0/2965858079 >> v2:172.21.15.169:3300/0 conn(0x7fc730159080 0x7fc73006dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:43.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.445+0000 7fc73579b700 1 -- 172.21.15.92:0/2965858079 >> v2:172.21.15.169:3300/0 conn(0x7fc730159080 msgr2=0x7fc73006dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:25:43.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.445+0000 7fc73579b700 1 --2- 172.21.15.92:0/2965858079 >> v2:172.21.15.169:3300/0 conn(0x7fc730159080 0x7fc73006dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:43.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.445+0000 7fc73579b700 1 -- 172.21.15.92:0/2965858079 >> v1:172.21.15.169:6789/0 conn(0x7fc7300c9300 legacy=0x7fc730070910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:43.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.445+0000 7fc73579b700 1 -- 172.21.15.92:0/2965858079 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc73006d220 con 0x7fc730158cb0 2023-11-28T11:25:43.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.445+0000 7fc734f9a700 1 --2- 172.21.15.92:0/2965858079 >> v2:172.21.15.169:3300/0 conn(0x7fc730159080 0x7fc73006dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:25:43.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.445+0000 7fc73579b700 1 --2- 172.21.15.92:0/2965858079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc730158cb0 0x7fc7300ca1f0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fc73015c8c0 tx=0x7fc72c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3664d058cfc27fd2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:43.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.445+0000 7fc727fff700 1 -- 172.21.15.92:0/2965858079 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72c00b870 con 0x7fc730158cb0 2023-11-28T11:25:43.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.446+0000 7fc727fff700 1 -- 172.21.15.92:0/2965858079 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc72c00b9d0 con 0x7fc730158cb0 2023-11-28T11:25:43.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.446+0000 7fc727fff700 1 -- 172.21.15.92:0/2965858079 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72c00bd60 con 0x7fc730158cb0 2023-11-28T11:25:43.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.446+0000 7fc7379ff700 1 -- 172.21.15.92:0/2965858079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc730158cb0 msgr2=0x7fc7300ca1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:43.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.446+0000 7fc7379ff700 1 --2- 172.21.15.92:0/2965858079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc730158cb0 0x7fc7300ca1f0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fc73015c8c0 tx=0x7fc72c009bd0 comp rx=0 tx=0).stop 2023-11-28T11:25:43.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.446+0000 7fc7379ff700 1 -- 172.21.15.92:0/2965858079 shutdown_connections 2023-11-28T11:25:43.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.446+0000 7fc7379ff700 1 --2- 172.21.15.92:0/2965858079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc730158cb0 0x7fc7300ca1f0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:43.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.446+0000 7fc7379ff700 1 --2- 172.21.15.92:0/2965858079 >> v2:172.21.15.169:3300/0 conn(0x7fc730159080 0x7fc73006dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:43.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.446+0000 7fc7379ff700 1 -- 172.21.15.92:0/2965858079 >> 172.21.15.92:0/2965858079 conn(0x7fc7300cb490 msgr2=0x7fc7300cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:43.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.446+0000 7fc7379ff700 1 -- 172.21.15.92:0/2965858079 shutdown_connections 2023-11-28T11:25:43.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.446+0000 7fc7379ff700 1 -- 172.21.15.92:0/2965858079 wait complete. 2023-11-28T11:25:43.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.448+0000 7fc7379ff700 1 Processor -- start 2023-11-28T11:25:43.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.448+0000 7fc7379ff700 1 -- start start 2023-11-28T11:25:43.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.449+0000 7fc7379ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7300c9300 0x7fc7300c3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:43.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.449+0000 7fc7379ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc730158cb0 0x7fc73015b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:43.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.449+0000 7fc7379ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc73015b590 0x7fc730145760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:43.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.449+0000 7fc7379ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc73015c470 con 0x7fc7300c9300 2023-11-28T11:25:43.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.449+0000 7fc7379ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc73014ac70 con 0x7fc73015b590 2023-11-28T11:25:43.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.449+0000 7fc7379ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7300c3220 con 0x7fc730158cb0 2023-11-28T11:25:43.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.449+0000 7fc73579b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7300c9300 0x7fc7300c3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:43.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.449+0000 7fc73579b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7300c9300 0x7fc7300c3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38586/0 (socket says 172.21.15.92:38586) 2023-11-28T11:25:43.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.449+0000 7fc73579b700 1 -- 172.21.15.92:0/596682965 learned_addr learned my addr 172.21.15.92:0/596682965 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:25:43.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.449+0000 7fc734f9a700 1 --2- 172.21.15.92:0/596682965 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc730158cb0 0x7fc73015b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:43.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.449+0000 7fc735f9c700 1 --2- 172.21.15.92:0/596682965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc73015b590 0x7fc730145760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:43.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.450+0000 7fc73579b700 1 -- 172.21.15.92:0/596682965 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc730158cb0 msgr2=0x7fc73015b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:43.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.450+0000 7fc73579b700 1 --2- 172.21.15.92:0/596682965 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc730158cb0 0x7fc73015b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:43.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.450+0000 7fc73579b700 1 -- 172.21.15.92:0/596682965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc73015b590 msgr2=0x7fc730145760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:43.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.450+0000 7fc73579b700 1 --2- 172.21.15.92:0/596682965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc73015b590 0x7fc730145760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:43.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.450+0000 7fc73579b700 1 -- 172.21.15.92:0/596682965 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc72c009550 con 0x7fc7300c9300 2023-11-28T11:25:43.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.450+0000 7fc73579b700 1 --2- 172.21.15.92:0/596682965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7300c9300 0x7fc7300c3b30 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fc73006ec00 tx=0x7fc72c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:43.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.450+0000 7fc725ffb700 1 -- 172.21.15.92:0/596682965 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72c004540 con 0x7fc7300c9300 2023-11-28T11:25:43.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.450+0000 7fc725ffb700 1 -- 172.21.15.92:0/596682965 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc72c0036a0 con 0x7fc7300c9300 2023-11-28T11:25:43.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.450+0000 7fc725ffb700 1 -- 172.21.15.92:0/596682965 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72c003a30 con 0x7fc7300c9300 2023-11-28T11:25:43.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.450+0000 7fc7379ff700 1 -- 172.21.15.92:0/596682965 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc730071b20 con 0x7fc7300c9300 2023-11-28T11:25:43.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.450+0000 7fc7379ff700 1 -- 172.21.15.92:0/596682965 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc73006d500 con 0x7fc7300c9300 2023-11-28T11:25:43.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.451+0000 7fc7379ff700 1 -- 172.21.15.92:0/596682965 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc730071020 con 0x7fc7300c9300 2023-11-28T11:25:43.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.452+0000 7fc725ffb700 1 -- 172.21.15.92:0/596682965 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc72c00f050 con 0x7fc7300c9300 2023-11-28T11:25:43.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.453+0000 7fc725ffb700 1 --2- 172.21.15.92:0/596682965 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc7140682f0 0x7fc71406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:25:43.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.453+0000 7fc725ffb700 1 -- 172.21.15.92:0/596682965 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fc72c087450 con 0x7fc7300c9300 2023-11-28T11:25:43.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.453+0000 7fc734f9a700 1 --2- 172.21.15.92:0/596682965 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc7140682f0 0x7fc71406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:25:43.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.454+0000 7fc734f9a700 1 --2- 172.21.15.92:0/596682965 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc7140682f0 0x7fc71406a7a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc73015c5d0 tx=0x7fc720008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:25:43.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.457+0000 7fc725ffb700 1 -- 172.21.15.92:0/596682965 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc72c053da0 con 0x7fc7300c9300 2023-11-28T11:25:43.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.643+0000 7fc7379ff700 1 -- 172.21.15.92:0/596682965 --> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc7300c3360 con 0x7fc7140682f0 2023-11-28T11:25:43.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.646+0000 7fc725ffb700 1 -- 172.21.15.92:0/596682965 <== mgr.4103 v2:172.21.15.92:6832/115482 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72598 (secure 0 0 0) 0x7fc72000cda0 con 0x7fc7140682f0 2023-11-28T11:25:43.648 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:25:43.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.648+0000 7fc7379ff700 1 -- 172.21.15.92:0/596682965 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc7140682f0 msgr2=0x7fc71406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:43.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.648+0000 7fc7379ff700 1 --2- 172.21.15.92:0/596682965 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc7140682f0 0x7fc71406a7a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc73015c5d0 tx=0x7fc720008040 comp rx=0 tx=0).stop 2023-11-28T11:25:43.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.648+0000 7fc7379ff700 1 -- 172.21.15.92:0/596682965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7300c9300 msgr2=0x7fc7300c3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:25:43.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.648+0000 7fc7379ff700 1 --2- 172.21.15.92:0/596682965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7300c9300 0x7fc7300c3b30 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fc73006ec00 tx=0x7fc72c004750 comp rx=0 tx=0).stop 2023-11-28T11:25:43.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.648+0000 7fc7379ff700 1 -- 172.21.15.92:0/596682965 shutdown_connections 2023-11-28T11:25:43.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.648+0000 7fc7379ff700 1 --2- 172.21.15.92:0/596682965 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fc73015b590 0x7fc730145760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:43.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.648+0000 7fc7379ff700 1 --2- 172.21.15.92:0/596682965 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fc7140682f0 0x7fc71406a7a0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:43.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.648+0000 7fc7379ff700 1 --2- 172.21.15.92:0/596682965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7300c9300 0x7fc7300c3b30 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:43.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.648+0000 7fc7379ff700 1 --2- 172.21.15.92:0/596682965 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fc730158cb0 0x7fc73015b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:25:43.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.648+0000 7fc7379ff700 1 -- 172.21.15.92:0/596682965 >> 172.21.15.92:0/596682965 conn(0x7fc7300cb490 msgr2=0x7fc730151280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:25:43.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.648+0000 7fc7379ff700 1 -- 172.21.15.92:0/596682965 shutdown_connections 2023-11-28T11:25:43.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:25:43.649+0000 7fc7379ff700 1 -- 172.21.15.92:0/596682965 wait complete. 2023-11-28T11:25:43.651 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-11-28T11:25:43.668 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":216,"stamp":"2023-11-28T11:25:41.923165+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":91012,"kb_used_data":8132,"kb_used_omap":0,"kb_used_meta":82816,"kb_avail":754883708,"statfs":{"total":773094113280,"available":773000916992,"internally_reserved":0,"allocated":8327168,"data_stored":4539874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":84803584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":-1,"num_object_clones":0,"num_object_copies":-2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1,"num_whiteouts":0,"num_read":-2,"num_read_kb":-2,"num_write":-3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001674"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":60,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-28T11:25:31.801900+0000","last_change":"2023-11-28T11:20:51.020016+0000","last_active":"2023-11-28T11:25:31.801900+0000","last_peered":"2023-11-28T11:25:31.801900+0000","last_clean":"2023-11-28T11:25:31.801900+0000","last_became_active":"2023-11-28T11:20:51.019492+0000","last_became_peered":"2023-11-28T11:20:51.019492+0000","last_unstale":"2023-11-28T11:25:31.801900+0000","last_undegraded":"2023-11-28T11:25:31.801900+0000","last_fullsized":"2023-11-28T11:25:31.801900+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:27.884538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":64,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-28T11:25:31.796639+0000","last_change":"2023-11-28T11:20:50.521908+0000","last_active":"2023-11-28T11:25:31.796639+0000","last_peered":"2023-11-28T11:25:31.796639+0000","last_clean":"2023-11-28T11:25:31.796639+0000","last_became_active":"2023-11-28T11:20:50.521059+0000","last_became_peered":"2023-11-28T11:20:50.521059+0000","last_unstale":"2023-11-28T11:25:31.796639+0000","last_undegraded":"2023-11-28T11:25:31.796639+0000","last_fullsized":"2023-11-28T11:25:31.796639+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:05.302498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":57,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-28T11:25:32.658737+0000","last_change":"2023-11-28T11:20:50.619004+0000","last_active":"2023-11-28T11:25:32.658737+0000","last_peered":"2023-11-28T11:25:32.658737+0000","last_clean":"2023-11-28T11:25:32.658737+0000","last_became_active":"2023-11-28T11:20:50.618315+0000","last_became_peered":"2023-11-28T11:20:50.618315+0000","last_unstale":"2023-11-28T11:25:32.658737+0000","last_undegraded":"2023-11-28T11:25:32.658737+0000","last_fullsized":"2023-11-28T11:25:32.658737+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:27.884538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":64,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-28T11:25:31.796895+0000","last_change":"2023-11-28T11:20:50.522582+0000","last_active":"2023-11-28T11:25:31.796895+0000","last_peered":"2023-11-28T11:25:31.796895+0000","last_clean":"2023-11-28T11:25:31.796895+0000","last_became_active":"2023-11-28T11:20:50.522157+0000","last_became_peered":"2023-11-28T11:20:50.522157+0000","last_unstale":"2023-11-28T11:25:31.796895+0000","last_undegraded":"2023-11-28T11:25:31.796895+0000","last_fullsized":"2023-11-28T11:25:31.796895+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:27.884538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":210,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-28T11:25:28.746914+0000","last_change":"2023-11-28T11:20:43.294845+0000","last_active":"2023-11-28T11:25:28.746914+0000","last_peered":"2023-11-28T11:25:28.746914+0000","last_clean":"2023-11-28T11:25:28.746914+0000","last_became_active":"2023-11-28T11:20:43.294483+0000","last_became_peered":"2023-11-28T11:20:43.294483+0000","last_unstale":"2023-11-28T11:25:28.746914+0000","last_undegraded":"2023-11-28T11:25:28.746914+0000","last_fullsized":"2023-11-28T11:25:28.746914+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:42.270512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:42.270512+0000","last_clean_scrub_stamp":"2023-11-28T11:20:42.270512+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:17:03.376680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":57,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-28T11:25:28.746968+0000","last_change":"2023-11-28T11:20:50.526865+0000","last_active":"2023-11-28T11:25:28.746968+0000","last_peered":"2023-11-28T11:25:28.746968+0000","last_clean":"2023-11-28T11:25:28.746968+0000","last_became_active":"2023-11-28T11:20:50.526576+0000","last_became_peered":"2023-11-28T11:20:50.526576+0000","last_unstale":"2023-11-28T11:25:28.746968+0000","last_undegraded":"2023-11-28T11:25:28.746968+0000","last_fullsized":"2023-11-28T11:25:28.746968+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:09:18.460112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":57,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-28T11:25:28.747677+0000","last_change":"2023-11-28T11:20:51.020043+0000","last_active":"2023-11-28T11:25:28.747677+0000","last_peered":"2023-11-28T11:25:28.747677+0000","last_clean":"2023-11-28T11:25:28.747677+0000","last_became_active":"2023-11-28T11:20:51.019702+0000","last_became_peered":"2023-11-28T11:20:51.019702+0000","last_unstale":"2023-11-28T11:25:28.747677+0000","last_undegraded":"2023-11-28T11:25:28.747677+0000","last_fullsized":"2023-11-28T11:25:28.747677+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:26:20.246873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":60,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-28T11:25:31.802396+0000","last_change":"2023-11-28T11:20:50.524251+0000","last_active":"2023-11-28T11:25:31.802396+0000","last_peered":"2023-11-28T11:25:31.802396+0000","last_clean":"2023-11-28T11:25:31.802396+0000","last_became_active":"2023-11-28T11:20:50.524012+0000","last_became_peered":"2023-11-28T11:20:50.524012+0000","last_unstale":"2023-11-28T11:25:31.802396+0000","last_undegraded":"2023-11-28T11:25:31.802396+0000","last_fullsized":"2023-11-28T11:25:31.802396+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:05.302498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":56,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-28T11:25:28.748430+0000","last_change":"2023-11-28T11:20:50.524838+0000","last_active":"2023-11-28T11:25:28.748430+0000","last_peered":"2023-11-28T11:25:28.748430+0000","last_clean":"2023-11-28T11:25:28.748430+0000","last_became_active":"2023-11-28T11:20:50.524402+0000","last_became_peered":"2023-11-28T11:20:50.524402+0000","last_unstale":"2023-11-28T11:25:28.748430+0000","last_undegraded":"2023-11-28T11:25:28.748430+0000","last_fullsized":"2023-11-28T11:25:28.748430+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:20:49.497198+0000","last_clean_scrub_stamp":"2023-11-28T11:20:49.497198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:12:22.699447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574911,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":15280,"kb_used_data":1448,"kb_used_omap":0,"kb_used_meta":13824,"kb_avail":94356560,"statfs":{"total":96636764160,"available":96621117440,"internally_reserved":0,"allocated":1482752,"data_stored":1009716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14155776},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.623,"15min":0.623},"min":{"1min":0.46899999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.89600000000000002,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.65000000000000002,"15min":0.64900000000000002},"min":{"1min":0.47399999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.89800000000000002,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.65000000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65100000000000002,"15min":0.64500000000000002},"min":{"1min":0.44800000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.90100000000000002,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.66700000000000004,"15min":0.67500000000000004},"min":{"1min":0.497,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.80000000000000004,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.89400000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.72099999999999997,"15min":0.72299999999999998},"min":{"1min":0.57199999999999995,"5min":0.495,"15min":0.495},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.86199999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.65500000000000003,"15min":0.66300000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.90900000000000003,"5min":1.004,"15min":1.004},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69999999999999996,"15min":0.70499999999999996},"min":{"1min":0.52300000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.83999999999999997,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.57899999999999996}]},{"osd":4,"last update":"Tue Nov 28 11:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.61699999999999999,"15min":0.65700000000000003},"min":{"1min":0.36399999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.873,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.69099999999999995,"15min":0.69999999999999996},"min":{"1min":0.42899999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.97099999999999997,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.81399999999999995}]},{"osd":5,"last update":"Tue Nov 28 11:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.66900000000000004,"15min":0.68100000000000005},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.90100000000000002,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.68899999999999995,"15min":0.70999999999999996},"min":{"1min":0.40999999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.92500000000000004,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.93600000000000005}]},{"osd":6,"last update":"Tue Nov 28 11:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.74199999999999999,"15min":0.77200000000000002},"min":{"1min":0.45700000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.92300000000000004,"5min":1.633,"15min":1.633},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.70599999999999996,"15min":0.745},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.92200000000000004,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.628}]}]},{"osd":6,"up_from":13,"seq":55834574911,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9968,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94361872,"statfs":{"total":96636764160,"available":96626556928,"internally_reserved":0,"allocated":892928,"data_stored":419970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.63700000000000001,"15min":0.64100000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.167,"5min":1.409,"15min":1.409},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.63800000000000001,"15min":0.63900000000000001},"min":{"1min":0.47099999999999997,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89700000000000002,"5min":1.244,"15min":1.244},"last":0.59299999999999997}]},{"osd":1,"last update":"Tue Nov 28 11:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.64200000000000002,"15min":0.63200000000000001},"min":{"1min":0.437,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.67100000000000004,"15min":0.68300000000000005},"min":{"1min":0.49199999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.02,"5min":1.038,"15min":1.038},"last":0.51200000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.63,"15min":0.629},"min":{"1min":0.45400000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.73099999999999998,"15min":0.77300000000000002},"min":{"1min":0.51600000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.198,"5min":1.996,"15min":1.996},"last":0.60299999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.69999999999999996,"15min":0.70099999999999996},"min":{"1min":0.58299999999999996,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.66300000000000003,"15min":0.68100000000000005},"min":{"1min":0.501,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.92300000000000004,"5min":1.008,"15min":1.008},"last":0.54300000000000004}]},{"osd":4,"last update":"Tue Nov 28 11:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.68899999999999995,"15min":0.72599999999999998},"min":{"1min":0.39800000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.1439999999999999,"5min":2.048,"15min":2.048},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.68999999999999995,"15min":0.71899999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.46600000000000003}]},{"osd":5,"last update":"Tue Nov 28 11:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.65200000000000002,"15min":0.69099999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.86399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.754,"15min":0.80800000000000005},"min":{"1min":0.46000000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.99099999999999999,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":0.66300000000000003}]},{"osd":7,"last update":"Tue Nov 28 11:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.75,"15min":0.77400000000000002},"min":{"1min":0.56200000000000006,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.0009999999999999,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.749,"5min":0.76700000000000002,"15min":0.79400000000000004},"min":{"1min":0.58299999999999996,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.006,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.628}]}]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10096,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":9216,"kb_avail":94361744,"statfs":{"total":96636764160,"available":96626425856,"internally_reserved":0,"allocated":892928,"data_stored":419970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9437184},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.42499999999999999,"15min":0.45600000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.67900000000000005,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.45000000000000001,"15min":0.45900000000000002},"min":{"1min":0.33100000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.59799999999999998,"5min":0.999,"15min":0.999},"last":0.38500000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.48499999999999999,"15min":0.51000000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.64600000000000002,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.47699999999999998,"15min":0.501},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.57299999999999995,"5min":1.002,"15min":1.002},"last":0.32100000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.52800000000000002,"15min":0.55000000000000004},"min":{"1min":0.39100000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.627,"5min":1.198,"15min":1.198},"last":0.443},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.51200000000000001,"15min":0.52100000000000002},"min":{"1min":0.38600000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.626,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.42899999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.73499999999999999,"15min":0.753},"min":{"1min":0.48799999999999999,"5min":0.438,"15min":0.438},"max":{"1min":0.91300000000000003,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.73399999999999999,"15min":0.753},"min":{"1min":0.498,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0780000000000001,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.65800000000000003}]},{"osd":5,"last update":"Tue Nov 28 11:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.746,"15min":0.76600000000000001},"min":{"1min":0.53500000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.98299999999999998,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.746,"15min":0.76800000000000002},"min":{"1min":0.50800000000000001,"5min":0.5,"15min":0.5},"max":{"1min":0.86299999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.54700000000000004}]},{"osd":6,"last update":"Tue Nov 28 11:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.76300000000000001,"15min":0.78100000000000003},"min":{"1min":0.55300000000000005,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.99399999999999999,"5min":1.177,"15min":1.177},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.77900000000000003,"15min":0.80300000000000005},"min":{"1min":0.53000000000000003,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.97799999999999998,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.68799999999999994}]},{"osd":7,"last update":"Tue Nov 28 11:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.76100000000000001},"min":{"1min":0.56200000000000006,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.92000000000000004,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.77900000000000003,"15min":0.79800000000000004},"min":{"1min":0.50800000000000001,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0089999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.64600000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574911,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":15348,"kb_used_data":1452,"kb_used_omap":0,"kb_used_meta":13888,"kb_avail":94356492,"statfs":{"total":96636764160,"available":96621047808,"internally_reserved":0,"allocated":1486848,"data_stored":1010338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14221312},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.42299999999999999,"15min":0.44},"min":{"1min":0.27600000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.59199999999999997,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.53400000000000003,"15min":0.62},"min":{"1min":0.29799999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.875,"5min":3.8420000000000001,"15min":3.8420000000000001},"last":0.60299999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.48099999999999998,"15min":0.501},"min":{"1min":0.32200000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70899999999999996,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.57199999999999995,"15min":0.66000000000000003},"min":{"1min":0.33500000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.60899999999999999,"5min":3.8940000000000001,"15min":3.8940000000000001},"last":0.56699999999999995}]},{"osd":3,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.54300000000000004,"15min":0.622},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.72699999999999998,"5min":3.681,"15min":3.681},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55600000000000005,"15min":0.58599999999999997},"min":{"1min":0.42499999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.67800000000000005,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.58999999999999997}]},{"osd":4,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.80300000000000005,"15min":0.90300000000000002},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.96299999999999997,"5min":3.9790000000000001,"15min":3.9790000000000001},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.81599999999999995,"15min":0.92400000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.97899999999999998,"5min":3.5779999999999998,"15min":3.5779999999999998},"last":0.90400000000000003}]},{"osd":5,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.77300000000000002,"15min":0.81399999999999995},"min":{"1min":0.56000000000000005,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.0069999999999999,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.82499999999999996,"15min":0.93999999999999995},"min":{"1min":0.50600000000000001,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1,"5min":3.9409999999999998,"15min":3.9409999999999998},"last":0.70599999999999996}]},{"osd":6,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.81999999999999995,"15min":0.92600000000000005},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0469999999999999,"5min":3.6160000000000001,"15min":3.6160000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.78400000000000003,"15min":0.83299999999999996},"min":{"1min":0.55200000000000005,"5min":0.501,"15min":0.501},"max":{"1min":0.94799999999999995,"5min":1.363,"15min":1.363},"last":0.79700000000000004}]},{"osd":7,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.88,"15min":1.0109999999999999},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.149,"5min":4.0099999999999998,"15min":4.0099999999999998},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.85699999999999998,"15min":0.97199999999999998},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.075,"5min":3.653,"15min":3.653},"last":0.85899999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574909,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9968,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94361872,"statfs":{"total":96636764160,"available":96626556928,"internally_reserved":0,"allocated":892928,"data_stored":419970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.46899999999999997,"15min":0.54400000000000004},"min":{"1min":0.255,"5min":0.219,"15min":0.219},"max":{"1min":0.56499999999999995,"5min":4.1109999999999998,"15min":4.1109999999999998},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.442,"5min":0.51700000000000002,"15min":0.58299999999999996},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.75800000000000001,"5min":3.6230000000000002,"15min":3.6230000000000002},"last":0.47399999999999998}]},{"osd":1,"last update":"Tue Nov 28 11:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.61099999999999999,"15min":0.66600000000000004},"min":{"1min":0.40400000000000003,"5min":0.371,"15min":0.371},"max":{"1min":0.77700000000000002,"5min":3.6640000000000001,"15min":3.6640000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.502,"5min":0.58399999999999996,"15min":0.67500000000000004},"min":{"1min":0.35299999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.73199999999999998,"5min":4.0830000000000002,"15min":4.0830000000000002},"last":0.58899999999999997}]},{"osd":3,"last update":"Tue Nov 28 11:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.55200000000000005,"15min":0.63600000000000001},"min":{"1min":0.35399999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.78200000000000003,"5min":4.1529999999999996,"15min":4.1529999999999996},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.55300000000000005,"15min":0.57999999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.68799999999999994,"5min":1.304,"15min":1.304},"last":0.42399999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.77500000000000002,"15min":0.80500000000000005},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0569999999999999,"5min":1.649,"15min":1.649},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.80300000000000005,"15min":0.89900000000000002},"min":{"1min":0.53300000000000003,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.98099999999999998,"5min":4.0060000000000002,"15min":4.0060000000000002},"last":0.626}]},{"osd":5,"last update":"Tue Nov 28 11:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.81599999999999995,"15min":0.91000000000000003},"min":{"1min":0.504,"5min":0.5,"15min":0.5},"max":{"1min":0.94399999999999995,"5min":4.0529999999999999,"15min":4.0529999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.78400000000000003,"15min":0.80800000000000005},"min":{"1min":0.56200000000000006,"5min":0.497,"15min":0.497},"max":{"1min":1.125,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.66800000000000004}]},{"osd":6,"last update":"Tue Nov 28 11:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.79900000000000004,"15min":0.83599999999999997},"min":{"1min":0.51600000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.083,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.82899999999999996,"15min":0.85499999999999998},"min":{"1min":0.57399999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.155,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.746}]},{"osd":7,"last update":"Tue Nov 28 11:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.77100000000000002,"15min":0.80200000000000005},"min":{"1min":0.56999999999999995,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0109999999999999,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.80100000000000005,"15min":0.82699999999999996},"min":{"1min":0.52900000000000003,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1060000000000001,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.67700000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574910,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10096,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":9216,"kb_avail":94361744,"statfs":{"total":96636764160,"available":96626425856,"internally_reserved":0,"allocated":892928,"data_stored":419970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9437184},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.39900000000000002,"15min":0.40400000000000003},"min":{"1min":0.28599999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.55000000000000004,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.45800000000000002,"15min":0.46100000000000002},"min":{"1min":0.30399999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.67800000000000005,"5min":1.157,"15min":1.157},"last":0.57699999999999996}]},{"osd":1,"last update":"Tue Nov 28 11:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.50700000000000001,"15min":0.57099999999999995},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.71099999999999997,"5min":4.8019999999999996,"15min":4.8019999999999996},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.52700000000000002,"15min":0.58499999999999996},"min":{"1min":0.33400000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.625,"5min":4.7539999999999996,"15min":4.7539999999999996},"last":0.51800000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.57699999999999996,"15min":0.64900000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.66900000000000004,"5min":4.8419999999999996,"15min":4.8419999999999996},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.502,"5min":0.57799999999999996,"15min":0.64600000000000002},"min":{"1min":0.34999999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.71199999999999997,"5min":4.7919999999999998,"15min":4.7919999999999998},"last":0.60299999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.76400000000000001,"15min":0.82899999999999996},"min":{"1min":0.49199999999999999,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.93500000000000005,"5min":4.8869999999999996,"15min":4.8869999999999996},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.80800000000000005,"15min":0.89000000000000001},"min":{"1min":0.54200000000000004,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.89900000000000002,"5min":5.2460000000000004,"15min":5.2460000000000004},"last":0.88400000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.81699999999999995,"15min":0.90000000000000002},"min":{"1min":0.54300000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.96799999999999997,"5min":5.0759999999999996,"15min":5.0759999999999996},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.79400000000000004,"15min":0.86299999999999999},"min":{"1min":0.51300000000000001,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.88700000000000001,"5min":5.3010000000000002,"15min":5.3010000000000002},"last":0.90300000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.80900000000000005,"15min":0.876},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.91400000000000003,"5min":5.3399999999999999,"15min":5.3399999999999999},"last":1.004},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.80100000000000005,"15min":0.86099999999999999},"min":{"1min":0.53800000000000003,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.91600000000000004,"5min":4.8330000000000002,"15min":4.8330000000000002},"last":0.96199999999999997}]},{"osd":7,"last update":"Tue Nov 28 11:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.80600000000000005,"15min":0.86899999999999999},"min":{"1min":0.52800000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.90500000000000003,"5min":4.8600000000000003,"15min":4.8600000000000003},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.83499999999999996,"15min":0.92300000000000004},"min":{"1min":0.56399999999999995,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.90100000000000002,"5min":5.1909999999999998,"15min":5.1909999999999998},"last":0.97899999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574910,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":10288,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":9408,"kb_avail":94361552,"statfs":{"total":96636764160,"available":96626229248,"internally_reserved":0,"allocated":892928,"data_stored":419970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9633792},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.66900000000000004,"15min":0.68200000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.93999999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.67100000000000004,"15min":0.66400000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.97199999999999998,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.98799999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.67400000000000004,"15min":0.65200000000000002},"min":{"1min":0.40000000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.70599999999999996,"15min":0.70899999999999996},"min":{"1min":0.496,"5min":0.379,"15min":0.379},"max":{"1min":1.0700000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":1.004}]},{"osd":2,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.74099999999999999,"15min":0.745},"min":{"1min":0.47899999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.754,"5min":0.71399999999999997,"15min":0.70299999999999996},"min":{"1min":0.46600000000000003,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.92400000000000004}]},{"osd":3,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.69099999999999995,"15min":0.68500000000000005},"min":{"1min":0.498,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.141,"5min":1.179,"15min":1.179},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.77300000000000002,"15min":0.78100000000000003},"min":{"1min":0.55000000000000004,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.298,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.89600000000000002}]},{"osd":5,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.67500000000000004,"15min":0.68600000000000005},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.1140000000000001,"5min":1.161,"15min":1.161},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.66400000000000003,"15min":0.66600000000000004},"min":{"1min":0.437,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.27,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.95099999999999996}]},{"osd":6,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.71299999999999997,"15min":0.73099999999999998},"min":{"1min":0.438,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.359,"5min":1.514,"15min":1.514},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.72399999999999998,"15min":0.73599999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.873}]},{"osd":7,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.70199999999999996,"15min":0.70299999999999996},"min":{"1min":0.47999999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.80100000000000005,"15min":0.83199999999999996},"min":{"1min":0.56200000000000006,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.3999999999999999,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":0.85299999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574910,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9968,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94361872,"statfs":{"total":96636764160,"available":96626556928,"internally_reserved":0,"allocated":892928,"data_stored":419970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.64700000000000002,"15min":0.63100000000000001},"min":{"1min":0.42299999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.85799999999999998,"5min":1.032,"15min":1.032},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.68799999999999994,"15min":0.69599999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.91300000000000003,"5min":1.02,"15min":1.02},"last":0.51400000000000001}]},{"osd":1,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.67700000000000005,"15min":0.66200000000000003},"min":{"1min":0.48699999999999999,"5min":0.442,"15min":0.442},"max":{"1min":0.93600000000000005,"5min":1.355,"15min":1.355},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.68700000000000006,"15min":0.69399999999999995},"min":{"1min":0.48599999999999999,"5min":0.442,"15min":0.442},"max":{"1min":0.83399999999999996,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.73199999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69399999999999995,"15min":0.69799999999999995},"min":{"1min":0.52200000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.93999999999999995,"5min":1.198,"15min":1.198},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.72199999999999998,"15min":0.73199999999999998},"min":{"1min":0.47799999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.77000000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.73399999999999999,"15min":0.751},"min":{"1min":0.50800000000000001,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.88700000000000001,"5min":1.141,"15min":1.141},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.747,"5min":0.72399999999999998,"15min":0.72599999999999998},"min":{"1min":0.55700000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.81399999999999995}]},{"osd":4,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.67800000000000005,"15min":0.71399999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.84099999999999997,"5min":2.5139999999999998,"15min":2.5139999999999998},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.70599999999999996,"15min":0.72999999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.98699999999999999,"5min":1.198,"15min":1.198},"last":0.61599999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.71899999999999997,"15min":0.73099999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.71899999999999997,"15min":0.72699999999999998},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.88900000000000001,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":0.82499999999999996}]},{"osd":7,"last update":"Tue Nov 28 11:24:51 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.77300000000000002,"15min":0.81299999999999994},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.96499999999999997,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.79100000000000004,"15min":0.81999999999999995},"min":{"1min":0.51600000000000001,"5min":0.496,"15min":0.496},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.71099999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:25:43.673 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-11-28T11:20:49.497198+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=20, tm_sec=49, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=25, tm_sec=36, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T11:25:43.673 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-11-28T11:20:49.497198+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=20, tm_sec=49, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=25, tm_sec=36, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T11:25:43.673 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-11-28T11:20:49.497198+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=20, tm_sec=49, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=25, tm_sec=36, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T11:25:43.674 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-11-28T11:20:49.497198+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=20, tm_sec=49, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=25, tm_sec=36, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T11:25:43.674 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-11-28T11:20:42.270512+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=20, tm_sec=42, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=25, tm_sec=36, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T11:25:43.674 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-11-28T11:20:49.497198+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=20, tm_sec=49, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=25, tm_sec=36, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T11:25:43.674 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-11-28T11:20:49.497198+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=20, tm_sec=49, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=25, tm_sec=36, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T11:25:43.674 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-11-28T11:20:49.497198+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=20, tm_sec=49, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=25, tm_sec=36, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T11:25:43.674 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-11-28T11:20:49.497198+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=20, tm_sec=49, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=25, tm_sec=36, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T11:25:43.675 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-11-28T11:25:49.042 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:25:49.050 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:26:03.676 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:26:03.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:03.878+0000 7f57f3fff700 1 Processor -- start 2023-11-28T11:26:03.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:03.878+0000 7f57f3fff700 1 -- start start 2023-11-28T11:26:03.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:03.879+0000 7f57f3fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f57ec0cd460 0x7f57ec06c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:26:03.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:03.880+0000 7f57f3fff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f57ec0d0d60 0x7f57ec0d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:26:03.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:03.880+0000 7f57f3fff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57ec0b88e0 con 0x7f57ec0b8510 2023-11-28T11:26:03.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:03.880+0000 7f57f3fff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57ec0cdbd0 con 0x7f57ec0cd460 2023-11-28T11:26:03.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:03.880+0000 7f57f3fff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57ec0d02d0 con 0x7f57ec0d0d60 2023-11-28T11:26:03.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:03.880+0000 7f57f159a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f57ec0cd460 0x7f57ec06c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:26:03.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:03.880+0000 7f57f159a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f57ec0cd460 0x7f57ec06c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40564/0 (socket says 172.21.15.92:40564) 2023-11-28T11:26:03.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:03.880+0000 7f57f159a700 1 -- 172.21.15.92:0/1817220238 learned_addr learned my addr 172.21.15.92:0/1817220238 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:26:03.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:03.880+0000 7f57f259c700 1 --2- 172.21.15.92:0/1817220238 >> v2:172.21.15.169:3300/0 conn(0x7f57ec0d0d60 0x7f57ec0d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:26:03.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:03.880+0000 7f57f159a700 1 -- 172.21.15.92:0/1817220238 >> v2:172.21.15.169:3300/0 conn(0x7f57ec0d0d60 msgr2=0x7f57ec0d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:26:03.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:03.880+0000 7f57f159a700 1 --2- 172.21.15.92:0/1817220238 >> v2:172.21.15.169:3300/0 conn(0x7f57ec0d0d60 0x7f57ec0d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:26:03.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:03.880+0000 7f57f159a700 1 -- 172.21.15.92:0/1817220238 >> v1:172.21.15.169:6789/0 conn(0x7f57ec0b8510 legacy=0x7f57ec06b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:26:03.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:03.880+0000 7f57f159a700 1 -- 172.21.15.92:0/1817220238 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57ec0cd8f0 con 0x7f57ec0cd460 2023-11-28T11:26:03.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:03.881+0000 7f57f159a700 1 --2- 172.21.15.92:0/1817220238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57ec0cd460 0x7f57ec06c080 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f57ec0ce8c0 tx=0x7f57dc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67dc12f823541677 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:26:03.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:03.881+0000 7f57f0d99700 1 -- 172.21.15.92:0/1817220238 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57dc011830 con 0x7f57ec0cd460 2023-11-28T11:26:03.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:03.881+0000 7f57f0d99700 1 -- 172.21.15.92:0/1817220238 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57dc011990 con 0x7f57ec0cd460 2023-11-28T11:26:03.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:03.881+0000 7f57f0d99700 1 -- 172.21.15.92:0/1817220238 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57dc011d20 con 0x7f57ec0cd460 2023-11-28T11:26:03.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:03.881+0000 7f57f3fff700 1 -- 172.21.15.92:0/1817220238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57ec0cd460 msgr2=0x7f57ec06c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:26:03.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:03.882+0000 7f57f3fff700 1 --2- 172.21.15.92:0/1817220238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57ec0cd460 0x7f57ec06c080 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f57ec0ce8c0 tx=0x7f57dc009c00 comp rx=0 tx=0).stop 2023-11-28T11:26:03.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:03.882+0000 7f57f3fff700 1 -- 172.21.15.92:0/1817220238 shutdown_connections 2023-11-28T11:26:03.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:03.882+0000 7f57f3fff700 1 --2- 172.21.15.92:0/1817220238 >> v2:172.21.15.169:3300/0 conn(0x7f57ec0d0d60 0x7f57ec0d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:26:03.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:03.882+0000 7f57f3fff700 1 --2- 172.21.15.92:0/1817220238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57ec0cd460 0x7f57ec06c080 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:26:03.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:03.882+0000 7f57f3fff700 1 -- 172.21.15.92:0/1817220238 >> 172.21.15.92:0/1817220238 conn(0x7f57ec0719a0 msgr2=0x7f57ec071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:26:03.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:03.882+0000 7f57f3fff700 1 -- 172.21.15.92:0/1817220238 shutdown_connections 2023-11-28T11:26:03.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:03.882+0000 7f57f3fff700 1 -- 172.21.15.92:0/1817220238 wait complete. 2023-11-28T11:26:03.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:03.883+0000 7f57f3fff700 1 Processor -- start 2023-11-28T11:26:03.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:03.884+0000 7f57f3fff700 1 -- start start 2023-11-28T11:26:03.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:03.884+0000 7f57f3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57ec0b8510 0x7f57ec1e1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:26:03.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:03.884+0000 7f57f3fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57ec0cd460 0x7f57ec1e4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:26:03.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:03.885+0000 7f57f3fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f57ec0d0d60 0x7f57ec14de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:26:03.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:03.885+0000 7f57f3fff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57ec1e5120 con 0x7f57ec0b8510 2023-11-28T11:26:03.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:03.885+0000 7f57f3fff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57ec1e2320 con 0x7f57ec0cd460 2023-11-28T11:26:03.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:03.885+0000 7f57f3fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57ec1e2600 con 0x7f57ec0d0d60 2023-11-28T11:26:03.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:03.885+0000 7f57f1d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57ec0b8510 0x7f57ec1e1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:26:03.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:03.885+0000 7f57f1d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57ec0b8510 0x7f57ec1e1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40572/0 (socket says 172.21.15.92:40572) 2023-11-28T11:26:03.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:03.885+0000 7f57f1d9b700 1 -- 172.21.15.92:0/795520167 learned_addr learned my addr 172.21.15.92:0/795520167 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:26:03.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:03.885+0000 7f57f259c700 1 --2- 172.21.15.92:0/795520167 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f57ec0d0d60 0x7f57ec14de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:26:03.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:03.885+0000 7f57f159a700 1 --2- 172.21.15.92:0/795520167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57ec0cd460 0x7f57ec1e4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:26:03.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:03.885+0000 7f57f1d9b700 1 -- 172.21.15.92:0/795520167 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f57ec0d0d60 msgr2=0x7f57ec14de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:26:03.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:03.885+0000 7f57f1d9b700 1 --2- 172.21.15.92:0/795520167 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f57ec0d0d60 0x7f57ec14de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:26:03.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:03.885+0000 7f57f1d9b700 1 -- 172.21.15.92:0/795520167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57ec0cd460 msgr2=0x7f57ec1e4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:26:03.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:03.885+0000 7f57f1d9b700 1 --2- 172.21.15.92:0/795520167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57ec0cd460 0x7f57ec1e4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:26:03.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:03.885+0000 7f57f1d9b700 1 -- 172.21.15.92:0/795520167 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57dc009550 con 0x7f57ec0b8510 2023-11-28T11:26:03.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:03.885+0000 7f57f1d9b700 1 --2- 172.21.15.92:0/795520167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57ec0b8510 0x7f57ec1e1ad0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f57e800c4e0 tx=0x7f57e800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:26:03.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:03.886+0000 7f57e27fc700 1 -- 172.21.15.92:0/795520167 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e80052d0 con 0x7f57ec0b8510 2023-11-28T11:26:03.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:03.886+0000 7f57e27fc700 1 -- 172.21.15.92:0/795520167 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57e8005430 con 0x7f57ec0b8510 2023-11-28T11:26:03.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:03.886+0000 7f57f3fff700 1 -- 172.21.15.92:0/795520167 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57ec0cfff0 con 0x7f57ec0b8510 2023-11-28T11:26:03.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:03.886+0000 7f57e27fc700 1 -- 172.21.15.92:0/795520167 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e80038c0 con 0x7f57ec0b8510 2023-11-28T11:26:03.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:03.886+0000 7f57f3fff700 1 -- 172.21.15.92:0/795520167 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57ec0cfd10 con 0x7f57ec0b8510 2023-11-28T11:26:03.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:03.887+0000 7f57f3fff700 1 -- 172.21.15.92:0/795520167 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57ec067880 con 0x7f57ec0b8510 2023-11-28T11:26:03.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:03.888+0000 7f57e27fc700 1 -- 172.21.15.92:0/795520167 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f57e8019030 con 0x7f57ec0b8510 2023-11-28T11:26:03.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:03.889+0000 7f57e27fc700 1 --2- 172.21.15.92:0/795520167 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f57d00682f0 0x7f57d006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:26:03.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:03.889+0000 7f57e27fc700 1 -- 172.21.15.92:0/795520167 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f57e8084e00 con 0x7f57ec0b8510 2023-11-28T11:26:03.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:03.889+0000 7f57f159a700 1 --2- 172.21.15.92:0/795520167 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f57d00682f0 0x7f57d006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:26:03.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:03.889+0000 7f57f159a700 1 --2- 172.21.15.92:0/795520167 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f57d00682f0 0x7f57d006a7a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f57ec0b1de0 tx=0x7f57dc008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:26:03.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:03.893+0000 7f57e27fc700 1 -- 172.21.15.92:0/795520167 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f57e8051750 con 0x7f57ec0b8510 2023-11-28T11:26:04.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.082+0000 7f57f3fff700 1 -- 172.21.15.92:0/795520167 --> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f57ec0d2940 con 0x7f57d00682f0 2023-11-28T11:26:04.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.084+0000 7f57e27fc700 1 -- 172.21.15.92:0/795520167 <== mgr.4103 v2:172.21.15.92:6832/115482 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73156 (secure 0 0 0) 0x7f57ec0d2940 con 0x7f57d00682f0 2023-11-28T11:26:04.086 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:26:04.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.087+0000 7f57f3fff700 1 -- 172.21.15.92:0/795520167 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f57d00682f0 msgr2=0x7f57d006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:26:04.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.087+0000 7f57f3fff700 1 --2- 172.21.15.92:0/795520167 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f57d00682f0 0x7f57d006a7a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f57ec0b1de0 tx=0x7f57dc008030 comp rx=0 tx=0).stop 2023-11-28T11:26:04.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.087+0000 7f57f3fff700 1 -- 172.21.15.92:0/795520167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57ec0b8510 msgr2=0x7f57ec1e1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:26:04.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.087+0000 7f57f3fff700 1 --2- 172.21.15.92:0/795520167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57ec0b8510 0x7f57ec1e1ad0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f57e800c4e0 tx=0x7f57e800c7f0 comp rx=0 tx=0).stop 2023-11-28T11:26:04.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.087+0000 7f57f3fff700 1 -- 172.21.15.92:0/795520167 shutdown_connections 2023-11-28T11:26:04.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.087+0000 7f57f3fff700 1 --2- 172.21.15.92:0/795520167 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f57ec0d0d60 0x7f57ec14de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:26:04.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.087+0000 7f57f3fff700 1 --2- 172.21.15.92:0/795520167 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f57d00682f0 0x7f57d006a7a0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:26:04.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.087+0000 7f57f3fff700 1 --2- 172.21.15.92:0/795520167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57ec0b8510 0x7f57ec1e1ad0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:26:04.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.087+0000 7f57f3fff700 1 --2- 172.21.15.92:0/795520167 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f57ec0cd460 0x7f57ec1e4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:26:04.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.087+0000 7f57f3fff700 1 -- 172.21.15.92:0/795520167 >> 172.21.15.92:0/795520167 conn(0x7f57ec0719a0 msgr2=0x7f57ec0b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:26:04.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.087+0000 7f57f3fff700 1 -- 172.21.15.92:0/795520167 shutdown_connections 2023-11-28T11:26:04.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.087+0000 7f57f3fff700 1 -- 172.21.15.92:0/795520167 wait complete. 2023-11-28T11:26:04.092 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-11-28T11:26:04.106 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":227,"stamp":"2023-11-28T11:26:03.927790+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":91012,"kb_used_data":8132,"kb_used_omap":0,"kb_used_meta":82816,"kb_avail":754883708,"statfs":{"total":773094113280,"available":773000916992,"internally_reserved":0,"allocated":8327168,"data_stored":4539874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":84803584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002551"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":75,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-28T11:25:40.179862+0000","last_change":"2023-11-28T11:25:40.179839+0000","last_active":"2023-11-28T11:25:40.179862+0000","last_peered":"2023-11-28T11:25:40.179862+0000","last_clean":"2023-11-28T11:25:40.179862+0000","last_became_active":"2023-11-28T11:20:51.019492+0000","last_became_peered":"2023-11-28T11:20:51.019492+0000","last_unstale":"2023-11-28T11:25:40.179862+0000","last_undegraded":"2023-11-28T11:25:40.179862+0000","last_fullsized":"2023-11-28T11:25:40.179862+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:25:40.179762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:25:40.179762+0000","last_clean_scrub_stamp":"2023-11-28T11:25:40.179762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:07:56.612041+0000","scrub_duration":0.003498808,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":79,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-28T11:25:45.733539+0000","last_change":"2023-11-28T11:25:45.733506+0000","last_active":"2023-11-28T11:25:45.733539+0000","last_peered":"2023-11-28T11:25:45.733539+0000","last_clean":"2023-11-28T11:25:45.733539+0000","last_became_active":"2023-11-28T11:20:50.521059+0000","last_became_peered":"2023-11-28T11:20:50.521059+0000","last_unstale":"2023-11-28T11:25:45.733539+0000","last_undegraded":"2023-11-28T11:25:45.733539+0000","last_fullsized":"2023-11-28T11:25:45.733539+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:25:45.733333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:25:45.733333+0000","last_clean_scrub_stamp":"2023-11-28T11:25:45.733333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:13:05.008166+0000","scrub_duration":0.0041334079999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":72,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-28T11:25:41.718658+0000","last_change":"2023-11-28T11:25:41.718634+0000","last_active":"2023-11-28T11:25:41.718658+0000","last_peered":"2023-11-28T11:25:41.718658+0000","last_clean":"2023-11-28T11:25:41.718658+0000","last_became_active":"2023-11-28T11:20:50.618315+0000","last_became_peered":"2023-11-28T11:20:50.618315+0000","last_unstale":"2023-11-28T11:25:41.718658+0000","last_undegraded":"2023-11-28T11:25:41.718658+0000","last_fullsized":"2023-11-28T11:25:41.718658+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:25:41.718560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:25:41.718560+0000","last_clean_scrub_stamp":"2023-11-28T11:25:41.718560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:58:40.120394+0000","scrub_duration":0.0034678920000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":79,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-28T11:25:46.774279+0000","last_change":"2023-11-28T11:25:46.774249+0000","last_active":"2023-11-28T11:25:46.774279+0000","last_peered":"2023-11-28T11:25:46.774279+0000","last_clean":"2023-11-28T11:25:46.774279+0000","last_became_active":"2023-11-28T11:20:50.522157+0000","last_became_peered":"2023-11-28T11:20:50.522157+0000","last_unstale":"2023-11-28T11:25:46.774279+0000","last_undegraded":"2023-11-28T11:25:46.774279+0000","last_fullsized":"2023-11-28T11:25:46.774279+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:25:46.774146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:25:46.774146+0000","last_clean_scrub_stamp":"2023-11-28T11:25:46.774146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:35.404912+0000","scrub_duration":0.0042056339999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"73'119","reported_seq":227,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-28T11:25:45.872050+0000","last_change":"2023-11-28T11:25:45.871971+0000","last_active":"2023-11-28T11:25:45.872050+0000","last_peered":"2023-11-28T11:25:45.872050+0000","last_clean":"2023-11-28T11:25:45.872050+0000","last_became_active":"2023-11-28T11:20:43.294483+0000","last_became_peered":"2023-11-28T11:20:43.294483+0000","last_unstale":"2023-11-28T11:25:45.872050+0000","last_undegraded":"2023-11-28T11:25:45.872050+0000","last_fullsized":"2023-11-28T11:25:45.872050+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"73'119","last_scrub_stamp":"2023-11-28T11:25:45.871805+0000","last_deep_scrub":"73'119","last_deep_scrub_stamp":"2023-11-28T11:25:45.871805+0000","last_clean_scrub_stamp":"2023-11-28T11:25:45.871805+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:08:09.097474+0000","scrub_duration":0.013198,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":72,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-28T11:25:47.868075+0000","last_change":"2023-11-28T11:25:47.868015+0000","last_active":"2023-11-28T11:25:47.868075+0000","last_peered":"2023-11-28T11:25:47.868075+0000","last_clean":"2023-11-28T11:25:47.868075+0000","last_became_active":"2023-11-28T11:20:50.526576+0000","last_became_peered":"2023-11-28T11:20:50.526576+0000","last_unstale":"2023-11-28T11:25:47.868075+0000","last_undegraded":"2023-11-28T11:25:47.868075+0000","last_fullsized":"2023-11-28T11:25:47.868075+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:25:47.867851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:25:47.867851+0000","last_clean_scrub_stamp":"2023-11-28T11:25:47.867851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:21:36.498617+0000","scrub_duration":0.0043310800000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":72,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-28T11:25:48.834617+0000","last_change":"2023-11-28T11:25:48.834540+0000","last_active":"2023-11-28T11:25:48.834617+0000","last_peered":"2023-11-28T11:25:48.834617+0000","last_clean":"2023-11-28T11:25:48.834617+0000","last_became_active":"2023-11-28T11:20:51.019702+0000","last_became_peered":"2023-11-28T11:20:51.019702+0000","last_unstale":"2023-11-28T11:25:48.834617+0000","last_undegraded":"2023-11-28T11:25:48.834617+0000","last_fullsized":"2023-11-28T11:25:48.834617+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:25:48.834447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:25:48.834447+0000","last_clean_scrub_stamp":"2023-11-28T11:25:48.834447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:54:39.674865+0000","scrub_duration":0.0044537480000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":75,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-28T11:25:42.173756+0000","last_change":"2023-11-28T11:25:42.173717+0000","last_active":"2023-11-28T11:25:42.173756+0000","last_peered":"2023-11-28T11:25:42.173756+0000","last_clean":"2023-11-28T11:25:42.173756+0000","last_became_active":"2023-11-28T11:20:50.524012+0000","last_became_peered":"2023-11-28T11:20:50.524012+0000","last_unstale":"2023-11-28T11:25:42.173756+0000","last_undegraded":"2023-11-28T11:25:42.173756+0000","last_fullsized":"2023-11-28T11:25:42.173756+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:25:42.173578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:25:42.173578+0000","last_clean_scrub_stamp":"2023-11-28T11:25:42.173578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:47:57.803268+0000","scrub_duration":0.0041365550000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":71,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-28T11:25:43.877978+0000","last_change":"2023-11-28T11:25:43.877948+0000","last_active":"2023-11-28T11:25:43.877978+0000","last_peered":"2023-11-28T11:25:43.877978+0000","last_clean":"2023-11-28T11:25:43.877978+0000","last_became_active":"2023-11-28T11:20:50.524402+0000","last_became_peered":"2023-11-28T11:20:50.524402+0000","last_unstale":"2023-11-28T11:25:43.877978+0000","last_undegraded":"2023-11-28T11:25:43.877978+0000","last_fullsized":"2023-11-28T11:25:43.877978+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:25:43.877856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:25:43.877856+0000","last_clean_scrub_stamp":"2023-11-28T11:25:43.877856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:21:40.719587+0000","scrub_duration":0.003557209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574916,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":15280,"kb_used_data":1448,"kb_used_omap":0,"kb_used_meta":13824,"kb_avail":94356560,"statfs":{"total":96636764160,"available":96621117440,"internally_reserved":0,"allocated":1482752,"data_stored":1009716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14155776},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:25:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.63300000000000001,"15min":0.627},"min":{"1min":0.47199999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.91600000000000004,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.65800000000000003,"15min":0.65200000000000002},"min":{"1min":0.40999999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.88,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.74399999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:25:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.66400000000000003,"15min":0.65000000000000002},"min":{"1min":0.434,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.88300000000000001,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.67600000000000005,"15min":0.67800000000000005},"min":{"1min":0.53200000000000003,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.89400000000000002,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.78500000000000003}]},{"osd":2,"last update":"Tue Nov 28 11:25:48 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.66800000000000004,"15min":0.66000000000000003},"min":{"1min":0.56100000000000005,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.86799999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.751,"5min":0.72699999999999998,"15min":0.72499999999999998},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.91800000000000004,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.80000000000000004}]},{"osd":3,"last update":"Tue Nov 28 11:25:48 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.66000000000000003,"15min":0.66400000000000003},"min":{"1min":0.51000000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.85399999999999998,"5min":1.004,"15min":1.004},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.70099999999999996,"15min":0.70499999999999996},"min":{"1min":0.53100000000000003,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.83999999999999997,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.76000000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:25:48 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60499999999999998,"15min":0.65300000000000002},"min":{"1min":0.44,"5min":0.316,"15min":0.316},"max":{"1min":0.83499999999999996,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.68899999999999995,"15min":0.69899999999999995},"min":{"1min":0.48599999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.90500000000000003,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.40200000000000002}]},{"osd":5,"last update":"Tue Nov 28 11:25:48 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.65600000000000003,"15min":0.67700000000000005},"min":{"1min":0.45100000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.90000000000000002,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.67500000000000004,"15min":0.70499999999999996},"min":{"1min":0.45300000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.93600000000000005,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.54600000000000004}]},{"osd":6,"last update":"Tue Nov 28 11:25:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.72599999999999998,"15min":0.76700000000000002},"min":{"1min":0.49199999999999999,"5min":0.435,"15min":0.435},"max":{"1min":0.90000000000000002,"5min":1.633,"15min":1.633},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.68500000000000005,"15min":0.73799999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.91800000000000004,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.45800000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574916,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9968,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94361872,"statfs":{"total":96636764160,"available":96626556928,"internally_reserved":0,"allocated":892928,"data_stored":419970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.64600000000000002,"15min":0.64400000000000002},"min":{"1min":0.40000000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.92900000000000005,"5min":1.409,"15min":1.409},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63700000000000001,"15min":0.63900000000000001},"min":{"1min":0.45100000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.83199999999999996,"5min":1.244,"15min":1.244},"last":0.70099999999999996}]},{"osd":1,"last update":"Tue Nov 28 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.65200000000000002,"15min":0.63500000000000001},"min":{"1min":0.503,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.92300000000000004,"5min":1.089,"15min":1.089},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.66900000000000004,"15min":0.68200000000000005},"min":{"1min":0.50900000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89800000000000002,"5min":1.038,"15min":1.038},"last":0.79500000000000004}]},{"osd":2,"last update":"Tue Nov 28 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.64300000000000002,"15min":0.63300000000000001},"min":{"1min":0.48799999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.90900000000000003,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.71699999999999997,"15min":0.76800000000000002},"min":{"1min":0.57699999999999996,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.90800000000000003,"5min":1.996,"15min":1.996},"last":0.92500000000000004}]},{"osd":3,"last update":"Tue Nov 28 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.55500000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.88300000000000001,"5min":1.179,"15min":1.179},"last":0.748},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.66500000000000004,"15min":0.68100000000000005},"min":{"1min":0.52200000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.89500000000000002,"5min":1.008,"15min":1.008},"last":0.84699999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.66300000000000003,"15min":0.71699999999999997},"min":{"1min":0.49099999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.88400000000000001,"5min":2.048,"15min":2.048},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.67500000000000004,"15min":0.71399999999999997},"min":{"1min":0.46600000000000003,"5min":0.433,"15min":0.433},"max":{"1min":0.83299999999999996,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.63500000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63600000000000001,"15min":0.68600000000000005},"min":{"1min":0.46999999999999997,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79100000000000004,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.73199999999999998,"15min":0.80000000000000004},"min":{"1min":0.58899999999999997,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.86199999999999999,"5min":2.1000000000000001,"15min":2.1000000000000001},"last":0.56200000000000006}]},{"osd":7,"last update":"Tue Nov 28 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.73399999999999999,"15min":0.76800000000000002},"min":{"1min":0.61299999999999999,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.84599999999999997,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.752,"15min":0.78900000000000003},"min":{"1min":0.59999999999999998,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.871,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.66800000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10096,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":9216,"kb_avail":94361744,"statfs":{"total":96636764160,"available":96626425856,"internally_reserved":0,"allocated":892928,"data_stored":419970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9437184},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:25:48 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.41399999999999998,"15min":0.45200000000000001},"min":{"1min":0.28899999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.64100000000000001,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.45000000000000001,"15min":0.45900000000000002},"min":{"1min":0.309,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.70199999999999996,"5min":0.999,"15min":0.999},"last":0.54400000000000004}]},{"osd":2,"last update":"Tue Nov 28 11:25:48 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.48299999999999998,"15min":0.50900000000000001},"min":{"1min":0.34899999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69999999999999996,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.47499999999999998,"15min":0.501},"min":{"1min":0.32100000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.70299999999999996,"5min":1.002,"15min":1.002},"last":0.47799999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:25:48 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.52200000000000002,"15min":0.54800000000000004},"min":{"1min":0.40699999999999997,"5min":0.378,"15min":0.378},"max":{"1min":0.72599999999999998,"5min":1.198,"15min":1.198},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50900000000000001,"15min":0.52000000000000002},"min":{"1min":0.34899999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.66500000000000004,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.59399999999999997}]},{"osd":4,"last update":"Tue Nov 28 11:25:48 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.72899999999999998,"15min":0.751},"min":{"1min":0.503,"5min":0.438,"15min":0.438},"max":{"1min":0.89200000000000002,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":1.286},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.73299999999999998,"15min":0.753},"min":{"1min":0.46100000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.96499999999999997,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.94499999999999995}]},{"osd":5,"last update":"Tue Nov 28 11:25:48 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.73599999999999999,"15min":0.76300000000000001},"min":{"1min":0.51600000000000001,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.90700000000000003,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.755,"5min":0.74099999999999999,"15min":0.76600000000000001},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.89100000000000001,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":1.0309999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:25:48 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.75900000000000001,"15min":0.77900000000000003},"min":{"1min":0.55100000000000005,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.97799999999999998,"5min":1.177,"15min":1.177},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.77500000000000002,"15min":0.80100000000000005},"min":{"1min":0.54100000000000004,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.995,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":1.246}]},{"osd":7,"last update":"Tue Nov 28 11:25:48 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.754,"15min":0.76200000000000001},"min":{"1min":0.61699999999999999,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.97399999999999998,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.77400000000000002,"15min":0.79600000000000004},"min":{"1min":0.58499999999999996,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.94799999999999995,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":1.149}]}]},{"osd":0,"up_from":13,"seq":55834574916,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":15348,"kb_used_data":1452,"kb_used_omap":0,"kb_used_meta":13888,"kb_avail":94356492,"statfs":{"total":96636764160,"available":96621047808,"internally_reserved":0,"allocated":1486848,"data_stored":1010338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14221312},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.41399999999999998,"15min":0.437},"min":{"1min":0.26100000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.66500000000000004,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.48999999999999999,"15min":0.60599999999999998},"min":{"1min":0.33400000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.629,"5min":3.8420000000000001,"15min":3.8420000000000001},"last":0.46700000000000003}]},{"osd":2,"last update":"Tue Nov 28 11:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47299999999999998,"15min":0.499},"min":{"1min":0.35299999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.64600000000000002,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.52700000000000002,"15min":0.64500000000000002},"min":{"1min":0.379,"5min":0.311,"15min":0.311},"max":{"1min":0.70099999999999996,"5min":3.8940000000000001,"15min":3.8940000000000001},"last":0.51400000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.504,"15min":0.60899999999999999},"min":{"1min":0.35199999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.58699999999999997,"5min":3.681,"15min":3.681},"last":0.439},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.54200000000000004,"15min":0.58099999999999996},"min":{"1min":0.41799999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.72799999999999998,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.48999999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.76200000000000001,"15min":0.88900000000000001},"min":{"1min":0.51500000000000001,"5min":0.439,"15min":0.439},"max":{"1min":1.006,"5min":3.9790000000000001,"15min":3.9790000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.76400000000000001,"15min":0.90600000000000003},"min":{"1min":0.52000000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.0249999999999999,"5min":3.5779999999999998,"15min":3.5779999999999998},"last":0.83399999999999996}]},{"osd":5,"last update":"Tue Nov 28 11:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.75900000000000001,"15min":0.80900000000000005},"min":{"1min":0.53900000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.98399999999999999,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.747,"5min":0.77500000000000002,"15min":0.92300000000000004},"min":{"1min":0.54800000000000004,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.92600000000000005,"5min":3.9409999999999998,"15min":3.9409999999999998},"last":0.85299999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77900000000000003,"15min":0.91200000000000003},"min":{"1min":0.59699999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.98999999999999999,"5min":3.6160000000000001,"15min":3.6160000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.76700000000000002,"15min":0.82699999999999996},"min":{"1min":0.60499999999999998,"5min":0.501,"15min":0.501},"max":{"1min":0.95999999999999996,"5min":1.363,"15min":1.363},"last":0.73899999999999999}]},{"osd":7,"last update":"Tue Nov 28 11:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.81399999999999995,"15min":0.98899999999999999},"min":{"1min":0.57099999999999995,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0529999999999999,"5min":4.0099999999999998,"15min":4.0099999999999998},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.80600000000000005,"15min":0.95499999999999996},"min":{"1min":0.63200000000000001,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.91600000000000004,"5min":3.653,"15min":3.653},"last":0.76100000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574914,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9968,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94361872,"statfs":{"total":96636764160,"available":96626556928,"internally_reserved":0,"allocated":892928,"data_stored":419970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:25:57 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.42999999999999999,"15min":0.53100000000000003},"min":{"1min":0.28499999999999998,"5min":0.219,"15min":0.219},"max":{"1min":0.50800000000000001,"5min":4.1109999999999998,"15min":4.1109999999999998},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.48299999999999998,"15min":0.57099999999999995},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.68999999999999995,"5min":3.6230000000000002,"15min":3.6230000000000002},"last":0.50600000000000001}]},{"osd":1,"last update":"Tue Nov 28 11:25:57 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.57499999999999996,"15min":0.65400000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.79600000000000004,"5min":3.6640000000000001,"15min":3.6640000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.53900000000000003,"15min":0.66000000000000003},"min":{"1min":0.32400000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.71299999999999997,"5min":4.0830000000000002,"15min":4.0830000000000002},"last":0.63500000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:25:57 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.51100000000000001,"15min":0.623},"min":{"1min":0.34499999999999997,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.64800000000000002,"5min":4.1529999999999996,"15min":4.1529999999999996},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.54500000000000004,"15min":0.57699999999999996},"min":{"1min":0.35899999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.84899999999999998,"5min":1.304,"15min":1.304},"last":0.67600000000000005}]},{"osd":4,"last update":"Tue Nov 28 11:25:57 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76300000000000001,"15min":0.80100000000000005},"min":{"1min":0.59599999999999997,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0880000000000001,"5min":1.649,"15min":1.649},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.76000000000000001,"15min":0.88400000000000001},"min":{"1min":0.54200000000000004,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.054,"5min":4.0060000000000002,"15min":4.0060000000000002},"last":0.78200000000000003}]},{"osd":5,"last update":"Tue Nov 28 11:25:57 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.77800000000000002,"15min":0.89700000000000002},"min":{"1min":0.56599999999999995,"5min":0.5,"15min":0.5},"max":{"1min":0.91600000000000004,"5min":4.0529999999999999,"15min":4.0529999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.76900000000000002,"15min":0.80300000000000005},"min":{"1min":0.58999999999999997,"5min":0.497,"15min":0.497},"max":{"1min":0.97999999999999998,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.91200000000000003}]},{"osd":6,"last update":"Tue Nov 28 11:25:57 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.78600000000000003,"15min":0.83199999999999996},"min":{"1min":0.60999999999999999,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0249999999999999,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.873},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.81399999999999995,"15min":0.84999999999999998},"min":{"1min":0.63,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.0329999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.89800000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:25:57 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.75800000000000001,"15min":0.79800000000000004},"min":{"1min":0.53700000000000003,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0049999999999999,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.79100000000000004,"15min":0.82399999999999995},"min":{"1min":0.60999999999999999,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.95599999999999996,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.93200000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574915,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10096,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":9216,"kb_avail":94361744,"statfs":{"total":96636764160,"available":96626425856,"internally_reserved":0,"allocated":892928,"data_stored":419970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9437184},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.40000000000000002,"15min":0.40400000000000003},"min":{"1min":0.28899999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.45900000000000002,"15min":0.46100000000000002},"min":{"1min":0.32300000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.60099999999999998,"5min":1.157,"15min":1.157},"last":0.51300000000000001}]},{"osd":1,"last update":"Tue Nov 28 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.47399999999999998,"15min":0.56000000000000005},"min":{"1min":0.30299999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.65700000000000003,"5min":4.8019999999999996,"15min":4.8019999999999996},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.499,"15min":0.57499999999999996},"min":{"1min":0.36299999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.60799999999999998,"5min":4.7539999999999996,"15min":4.7539999999999996},"last":0.438}]},{"osd":2,"last update":"Tue Nov 28 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.54700000000000004,"15min":0.63900000000000001},"min":{"1min":0.38400000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.746,"5min":4.8419999999999996,"15min":4.8419999999999996},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.54800000000000004,"15min":0.63600000000000001},"min":{"1min":0.42199999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.623,"5min":4.7919999999999998,"15min":4.7919999999999998},"last":0.55500000000000005}]},{"osd":4,"last update":"Tue Nov 28 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.73599999999999999,"15min":0.81999999999999995},"min":{"1min":0.47199999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.91700000000000004,"5min":4.8869999999999996,"15min":4.8869999999999996},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.77400000000000002,"15min":0.879},"min":{"1min":0.53100000000000003,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.92000000000000004,"5min":5.2460000000000004,"15min":5.2460000000000004},"last":0.71999999999999997}]},{"osd":5,"last update":"Tue Nov 28 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.78100000000000003,"15min":0.88800000000000001},"min":{"1min":0.55100000000000005,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.024,"5min":5.0759999999999996,"15min":5.0759999999999996},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.75800000000000001,"15min":0.85099999999999998},"min":{"1min":0.60299999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.90300000000000002,"5min":5.3010000000000002,"15min":5.3010000000000002},"last":0.77800000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.77600000000000002,"15min":0.86499999999999999},"min":{"1min":0.57299999999999995,"5min":0.495,"15min":0.495},"max":{"1min":1.004,"5min":5.3399999999999999,"15min":5.3399999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.77400000000000002,"15min":0.85199999999999998},"min":{"1min":0.54800000000000004,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.96199999999999997,"5min":4.8330000000000002,"15min":4.8330000000000002},"last":0.68300000000000005}]},{"osd":7,"last update":"Tue Nov 28 11:25:49 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.77800000000000002,"15min":0.85999999999999999},"min":{"1min":0.58399999999999996,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.85399999999999998,"5min":4.8600000000000003,"15min":4.8600000000000003},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.79800000000000004,"15min":0.91100000000000003},"min":{"1min":0.56299999999999994,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.998,"5min":5.1909999999999998,"15min":5.1909999999999998},"last":0.76500000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":10288,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":9408,"kb_avail":94361552,"statfs":{"total":96636764160,"available":96626229248,"internally_reserved":0,"allocated":892928,"data_stored":419970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9633792},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.67400000000000004,"15min":0.68400000000000005},"min":{"1min":0.46300000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.67300000000000004,"15min":0.66500000000000004},"min":{"1min":0.48399999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.98799999999999999,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.69899999999999995}]},{"osd":1,"last update":"Tue Nov 28 11:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.68799999999999994,"15min":0.65700000000000003},"min":{"1min":0.47899999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.93899999999999995,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70499999999999996,"15min":0.70899999999999996},"min":{"1min":0.499,"5min":0.379,"15min":0.379},"max":{"1min":1.004,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.83499999999999996}]},{"osd":2,"last update":"Tue Nov 28 11:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.73799999999999999,"15min":0.74399999999999999},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.96599999999999997,"5min":1.242,"15min":1.242},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71699999999999997,"15min":0.70399999999999996},"min":{"1min":0.59599999999999997,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.92400000000000004,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.80400000000000005}]},{"osd":3,"last update":"Tue Nov 28 11:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.69499999999999995,"15min":0.68700000000000006},"min":{"1min":0.54000000000000004,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.90800000000000003,"5min":1.179,"15min":1.179},"last":0.745},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.76200000000000001,"15min":0.77800000000000002},"min":{"1min":0.53800000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.89600000000000002,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.86599999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.67000000000000004,"15min":0.68400000000000005},"min":{"1min":0.45900000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.85699999999999998,"5min":1.161,"15min":1.161},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.65800000000000003,"15min":0.66400000000000003},"min":{"1min":0.42299999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.95099999999999996,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.53800000000000003}]},{"osd":6,"last update":"Tue Nov 28 11:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.70099999999999996,"15min":0.72699999999999998},"min":{"1min":0.44400000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.84299999999999997,"5min":1.514,"15min":1.514},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71899999999999997,"15min":0.73399999999999999},"min":{"1min":0.45900000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.92700000000000005,"5min":1.157,"15min":1.157},"last":0.61699999999999999}]},{"osd":7,"last update":"Tue Nov 28 11:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.69699999999999995,"15min":0.70099999999999996},"min":{"1min":0.498,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.83399999999999996,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.755,"5min":0.78200000000000003,"15min":0.82599999999999996},"min":{"1min":0.51800000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.97999999999999998,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":0.71699999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574915,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9968,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94361872,"statfs":{"total":96636764160,"available":96626556928,"internally_reserved":0,"allocated":892928,"data_stored":419970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:25:52 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.66000000000000003,"15min":0.63500000000000001},"min":{"1min":0.57599999999999996,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79000000000000004,"5min":1.032,"15min":1.032},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.68999999999999995,"15min":0.69599999999999995},"min":{"1min":0.48799999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.94799999999999995,"5min":1.02,"15min":1.02},"last":0.89200000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:25:52 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.67700000000000005,"15min":0.66200000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.81799999999999995,"5min":1.355,"15min":1.355},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.68799999999999994,"15min":0.69399999999999995},"min":{"1min":0.50600000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.84899999999999998,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.752}]},{"osd":2,"last update":"Tue Nov 28 11:25:52 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.68500000000000005,"15min":0.69499999999999995},"min":{"1min":0.44500000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.88,"5min":1.198,"15min":1.198},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.71599999999999997,"15min":0.72999999999999998},"min":{"1min":0.59699999999999998,"5min":0.435,"15min":0.435},"max":{"1min":0.86199999999999999,"5min":1.419,"15min":1.419},"last":0.66800000000000004}]},{"osd":3,"last update":"Tue Nov 28 11:25:52 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.72299999999999998,"15min":0.747},"min":{"1min":0.53900000000000003,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.91700000000000004,"5min":1.141,"15min":1.141},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.73099999999999998,"15min":0.72799999999999998},"min":{"1min":0.60299999999999998,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.90000000000000002,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.85199999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:25:52 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.65000000000000002,"15min":0.70499999999999996},"min":{"1min":0.47199999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.70999999999999996,"5min":2.5139999999999998,"15min":2.5139999999999998},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.69199999999999995,"15min":0.72599999999999998},"min":{"1min":0.48799999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.85899999999999999,"5min":1.198,"15min":1.198},"last":0.91600000000000004}]},{"osd":6,"last update":"Tue Nov 28 11:25:52 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.69899999999999995,"15min":0.72499999999999998},"min":{"1min":0.48999999999999999,"5min":0.439,"15min":0.439},"max":{"1min":0.80600000000000005,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.70799999999999996,"15min":0.72299999999999998},"min":{"1min":0.55200000000000005,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.82499999999999996,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":0.81399999999999995}]},{"osd":7,"last update":"Tue Nov 28 11:25:52 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.73999999999999999,"15min":0.80200000000000005},"min":{"1min":0.53900000000000003,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.79200000000000004,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.76700000000000002,"15min":0.81200000000000006},"min":{"1min":0.53900000000000003,"5min":0.496,"15min":0.496},"max":{"1min":0.88,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.82799999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:26:04.118 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-11-28T11:26:04.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.308+0000 7fe4e5fff700 1 Processor -- start 2023-11-28T11:26:04.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.308+0000 7fe4e5fff700 1 -- start start 2023-11-28T11:26:04.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.309+0000 7fe4e5fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe4e00b8510 0x7fe4e006b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:26:04.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.309+0000 7fe4e5fff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe4e00cd460 0x7fe4e006beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:26:04.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.309+0000 7fe4e5fff700 1 -- --> v1:172.21.15.169:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4e00b88e0 con 0x7fe4e006c3f0 2023-11-28T11:26:04.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.309+0000 7fe4e5fff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4e00cdbd0 con 0x7fe4e00cd460 2023-11-28T11:26:04.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.309+0000 7fe4e5fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e006c830 con 0x7fe4e00b8510 2023-11-28T11:26:04.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.310+0000 7fe4deffd700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe4e00cd460 0x7fe4e006beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:26:04.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.310+0000 7fe4df7fe700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe4e00b8510 0x7fe4e006b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:26:04.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.310+0000 7fe4deffd700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7fe4e00cd460 0x7fe4e006beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.169:3300/0 says I am v2:172.21.15.92:43788/0 (socket says 172.21.15.92:43788) 2023-11-28T11:26:04.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.310+0000 7fe4deffd700 1 -- 172.21.15.92:0/2435513124 learned_addr learned my addr 172.21.15.92:0/2435513124 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:26:04.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.310+0000 7fe4deffd700 1 -- 172.21.15.92:0/2435513124 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe4e00b8510 msgr2=0x7fe4e006b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:26:04.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.310+0000 7fe4deffd700 1 --2- 172.21.15.92:0/2435513124 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe4e00b8510 0x7fe4e006b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:26:04.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.310+0000 7fe4deffd700 1 -- 172.21.15.92:0/2435513124 >> v1:172.21.15.169:6789/0 conn(0x7fe4e006c3f0 legacy=0x7fe4e015cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:26:04.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.310+0000 7fe4deffd700 1 -- 172.21.15.92:0/2435513124 --> v2:172.21.15.169:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4e00cd8f0 con 0x7fe4e00cd460 2023-11-28T11:26:04.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.311+0000 7fe4deffd700 1 --2- 172.21.15.92:0/2435513124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4e00cd460 0x7fe4e006beb0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4009920 tx=0x7fe4d4009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d02e846bfb38f140 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:26:04.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.311+0000 7fe4de7fc700 1 -- 172.21.15.92:0/2435513124 <== mon.1 v2:172.21.15.169:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d4011830 con 0x7fe4e00cd460 2023-11-28T11:26:04.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.311+0000 7fe4de7fc700 1 -- 172.21.15.92:0/2435513124 <== mon.1 v2:172.21.15.169:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4d4011990 con 0x7fe4e00cd460 2023-11-28T11:26:04.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.311+0000 7fe4de7fc700 1 -- 172.21.15.92:0/2435513124 <== mon.1 v2:172.21.15.169:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d4011d20 con 0x7fe4e00cd460 2023-11-28T11:26:04.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.311+0000 7fe4e5fff700 1 -- 172.21.15.92:0/2435513124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4e00cd460 msgr2=0x7fe4e006beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:26:04.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.311+0000 7fe4e5fff700 1 --2- 172.21.15.92:0/2435513124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4e00cd460 0x7fe4e006beb0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4009920 tx=0x7fe4d4009c30 comp rx=0 tx=0).stop 2023-11-28T11:26:04.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.311+0000 7fe4e5fff700 1 -- 172.21.15.92:0/2435513124 shutdown_connections 2023-11-28T11:26:04.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.311+0000 7fe4e5fff700 1 --2- 172.21.15.92:0/2435513124 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe4e00b8510 0x7fe4e006b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:26:04.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.311+0000 7fe4e5fff700 1 --2- 172.21.15.92:0/2435513124 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4e00cd460 0x7fe4e006beb0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:26:04.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.311+0000 7fe4e5fff700 1 -- 172.21.15.92:0/2435513124 >> 172.21.15.92:0/2435513124 conn(0x7fe4e00719a0 msgr2=0x7fe4e0071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:26:04.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.312+0000 7fe4e5fff700 1 -- 172.21.15.92:0/2435513124 shutdown_connections 2023-11-28T11:26:04.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.312+0000 7fe4e5fff700 1 -- 172.21.15.92:0/2435513124 wait complete. 2023-11-28T11:26:04.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.313+0000 7fe4e5fff700 1 Processor -- start 2023-11-28T11:26:04.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.313+0000 7fe4e5fff700 1 -- start start 2023-11-28T11:26:04.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.314+0000 7fe4e5fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4e006c3f0 0x7fe4e015b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:26:04.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.314+0000 7fe4e5fff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4e00b8510 0x7fe4e01e39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:26:04.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.314+0000 7fe4e5fff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe4e00cd460 0x7fe4e01e3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:26:04.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.314+0000 7fe4e5fff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e01f2040 con 0x7fe4e006c3f0 2023-11-28T11:26:04.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.314+0000 7fe4e5fff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e00b74a0 con 0x7fe4e00b8510 2023-11-28T11:26:04.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.314+0000 7fe4e5fff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e01e7ff0 con 0x7fe4e00cd460 2023-11-28T11:26:04.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.314+0000 7fe4df7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4e006c3f0 0x7fe4e015b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:26:04.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.315+0000 7fe4df7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4e006c3f0 0x7fe4e015b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40576/0 (socket says 172.21.15.92:40576) 2023-11-28T11:26:04.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.315+0000 7fe4df7fe700 1 -- 172.21.15.92:0/3753437689 learned_addr learned my addr 172.21.15.92:0/3753437689 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:26:04.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.315+0000 7fe4deffd700 1 --2- 172.21.15.92:0/3753437689 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4e00b8510 0x7fe4e01e39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:26:04.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.315+0000 7fe4dffff700 1 --2- 172.21.15.92:0/3753437689 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe4e00cd460 0x7fe4e01e3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:26:04.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.315+0000 7fe4df7fe700 1 -- 172.21.15.92:0/3753437689 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe4e00cd460 msgr2=0x7fe4e01e3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:26:04.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.315+0000 7fe4df7fe700 1 --2- 172.21.15.92:0/3753437689 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe4e00cd460 0x7fe4e01e3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:26:04.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.315+0000 7fe4df7fe700 1 -- 172.21.15.92:0/3753437689 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4e00b8510 msgr2=0x7fe4e01e39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:26:04.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.315+0000 7fe4df7fe700 1 --2- 172.21.15.92:0/3753437689 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4e00b8510 0x7fe4e01e39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:26:04.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.315+0000 7fe4df7fe700 1 -- 172.21.15.92:0/3753437689 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d4009550 con 0x7fe4e006c3f0 2023-11-28T11:26:04.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.315+0000 7fe4df7fe700 1 --2- 172.21.15.92:0/3753437689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4e006c3f0 0x7fe4e015b8a0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fe4d000e9b0 tx=0x7fe4d000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:26:04.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.315+0000 7fe4c7fff700 1 -- 172.21.15.92:0/3753437689 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d0009710 con 0x7fe4e006c3f0 2023-11-28T11:26:04.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.315+0000 7fe4c7fff700 1 -- 172.21.15.92:0/3753437689 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4d000ee50 con 0x7fe4e006c3f0 2023-11-28T11:26:04.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.316+0000 7fe4e5fff700 1 -- 172.21.15.92:0/3753437689 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4e00cfff0 con 0x7fe4e006c3f0 2023-11-28T11:26:04.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.316+0000 7fe4c7fff700 1 -- 172.21.15.92:0/3753437689 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d000ca50 con 0x7fe4e006c3f0 2023-11-28T11:26:04.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.316+0000 7fe4e5fff700 1 -- 172.21.15.92:0/3753437689 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4e00cfd10 con 0x7fe4e006c3f0 2023-11-28T11:26:04.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.317+0000 7fe4e5fff700 1 -- 172.21.15.92:0/3753437689 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4e01e8510 con 0x7fe4e006c3f0 2023-11-28T11:26:04.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.318+0000 7fe4c7fff700 1 -- 172.21.15.92:0/3753437689 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe4d0019030 con 0x7fe4e006c3f0 2023-11-28T11:26:04.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.318+0000 7fe4c7fff700 1 --2- 172.21.15.92:0/3753437689 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe4cc0682f0 0x7fe4cc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:26:04.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.319+0000 7fe4c7fff700 1 -- 172.21.15.92:0/3753437689 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fe4d0085aa0 con 0x7fe4e006c3f0 2023-11-28T11:26:04.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.319+0000 7fe4deffd700 1 --2- 172.21.15.92:0/3753437689 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe4cc0682f0 0x7fe4cc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:26:04.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.319+0000 7fe4deffd700 1 --2- 172.21.15.92:0/3753437689 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe4cc0682f0 0x7fe4cc06a7a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4000c00 tx=0x7fe4d4008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:26:04.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.322+0000 7fe4c7fff700 1 -- 172.21.15.92:0/3753437689 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe4e01e8510 con 0x7fe4e006c3f0 2023-11-28T11:26:04.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.512+0000 7fe4e5fff700 1 -- 172.21.15.92:0/3753437689 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7fe4e01e8510 con 0x7fe4e006c3f0 2023-11-28T11:26:04.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.516+0000 7fe4c7fff700 1 -- 172.21.15.92:0/3753437689 <== mon.0 v2:172.21.15.92:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fe4d0052390 con 0x7fe4e006c3f0 2023-11-28T11:26:04.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.516+0000 7fe4c7fff700 1 -- 172.21.15.92:0/3753437689 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v16)=0 v16) v1 ==== 123+0+0 (secure 0 0 0) 0x7fe4e01e8510 con 0x7fe4e006c3f0 2023-11-28T11:26:04.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.519+0000 7fe4e5fff700 1 -- 172.21.15.92:0/3753437689 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe4cc0682f0 msgr2=0x7fe4cc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:26:04.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.519+0000 7fe4e5fff700 1 --2- 172.21.15.92:0/3753437689 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe4cc0682f0 0x7fe4cc06a7a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4000c00 tx=0x7fe4d4008030 comp rx=0 tx=0).stop 2023-11-28T11:26:04.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.519+0000 7fe4e5fff700 1 -- 172.21.15.92:0/3753437689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4e006c3f0 msgr2=0x7fe4e015b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:26:04.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.519+0000 7fe4e5fff700 1 --2- 172.21.15.92:0/3753437689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4e006c3f0 0x7fe4e015b8a0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fe4d000e9b0 tx=0x7fe4d000ecc0 comp rx=0 tx=0).stop 2023-11-28T11:26:04.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.519+0000 7fe4e5fff700 1 -- 172.21.15.92:0/3753437689 shutdown_connections 2023-11-28T11:26:04.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.519+0000 7fe4e5fff700 1 --2- 172.21.15.92:0/3753437689 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7fe4e00cd460 0x7fe4e01e3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:26:04.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.519+0000 7fe4e5fff700 1 --2- 172.21.15.92:0/3753437689 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7fe4cc0682f0 0x7fe4cc06a7a0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:26:04.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.519+0000 7fe4e5fff700 1 --2- 172.21.15.92:0/3753437689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4e006c3f0 0x7fe4e015b8a0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:26:04.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.519+0000 7fe4e5fff700 1 --2- 172.21.15.92:0/3753437689 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7fe4e00b8510 0x7fe4e01e39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:26:04.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.519+0000 7fe4e5fff700 1 -- 172.21.15.92:0/3753437689 >> 172.21.15.92:0/3753437689 conn(0x7fe4e00719a0 msgr2=0x7fe4e00b5dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:26:04.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.519+0000 7fe4e5fff700 1 -- 172.21.15.92:0/3753437689 shutdown_connections 2023-11-28T11:26:04.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.519+0000 7fe4e5fff700 1 -- 172.21.15.92:0/3753437689 wait complete. 2023-11-28T11:26:04.538 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T11:26:04.539 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:26:04.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.734+0000 7f0c305ff700 1 Processor -- start 2023-11-28T11:26:04.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.734+0000 7f0c305ff700 1 -- start start 2023-11-28T11:26:04.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.735+0000 7f0c305ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0c280b8510 0x7f0c2806b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:26:04.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.735+0000 7f0c305ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c280cd460 0x7f0c2806beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:26:04.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.735+0000 7f0c305ff700 1 --2- >> v2:172.21.15.169:3300/0 conn(0x7f0c2806c3f0 0x7f0c2815cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:26:04.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.735+0000 7f0c305ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c280cdbd0 con 0x7f0c280b8510 2023-11-28T11:26:04.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.735+0000 7f0c305ff700 1 -- --> v2:172.21.15.169:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c280d02d0 con 0x7f0c2806c3f0 2023-11-28T11:26:04.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.735+0000 7f0c305ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c280b88e0 con 0x7f0c280cd460 2023-11-28T11:26:04.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.736+0000 7f0c2e39b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0c280b8510 0x7f0c2806b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:26:04.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.736+0000 7f0c2e39b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0c280b8510 0x7f0c2806b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40590/0 (socket says 172.21.15.92:40590) 2023-11-28T11:26:04.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.736+0000 7f0c2e39b700 1 -- 172.21.15.92:0/1372573463 learned_addr learned my addr 172.21.15.92:0/1372573463 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:26:04.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.736+0000 7f0c2eb9c700 1 --2- 172.21.15.92:0/1372573463 >> v2:172.21.15.169:3300/0 conn(0x7f0c2806c3f0 0x7f0c2815cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:26:04.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.736+0000 7f0c2db9a700 1 --2- 172.21.15.92:0/1372573463 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c280cd460 0x7f0c2806beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:26:04.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.736+0000 7f0c2e39b700 1 -- 172.21.15.92:0/1372573463 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c280cd460 msgr2=0x7f0c2806beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:26:04.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.736+0000 7f0c2e39b700 1 --2- 172.21.15.92:0/1372573463 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c280cd460 0x7f0c2806beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:26:04.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.736+0000 7f0c2e39b700 1 -- 172.21.15.92:0/1372573463 >> v2:172.21.15.169:3300/0 conn(0x7f0c2806c3f0 msgr2=0x7f0c2815cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:26:04.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.736+0000 7f0c2e39b700 1 --2- 172.21.15.92:0/1372573463 >> v2:172.21.15.169:3300/0 conn(0x7f0c2806c3f0 0x7f0c2815cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:26:04.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.736+0000 7f0c2e39b700 1 -- 172.21.15.92:0/1372573463 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c280cd8f0 con 0x7f0c280b8510 2023-11-28T11:26:04.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.736+0000 7f0c2e39b700 1 --2- 172.21.15.92:0/1372573463 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c280b8510 0x7f0c2806b970 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f0c280b4c80 tx=0x7f0c24009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=743d23f9e31713be server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:26:04.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.737+0000 7f0c2d399700 1 -- 172.21.15.92:0/1372573463 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2400b860 con 0x7f0c280b8510 2023-11-28T11:26:04.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.737+0000 7f0c2d399700 1 -- 172.21.15.92:0/1372573463 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f0c2400b9c0 con 0x7f0c280b8510 2023-11-28T11:26:04.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.737+0000 7f0c2d399700 1 -- 172.21.15.92:0/1372573463 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2400bd70 con 0x7f0c280b8510 2023-11-28T11:26:04.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.737+0000 7f0c305ff700 1 -- 172.21.15.92:0/1372573463 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c280b8510 msgr2=0x7f0c2806b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:26:04.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.737+0000 7f0c305ff700 1 --2- 172.21.15.92:0/1372573463 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c280b8510 0x7f0c2806b970 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f0c280b4c80 tx=0x7f0c24009c00 comp rx=0 tx=0).stop 2023-11-28T11:26:04.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.737+0000 7f0c305ff700 1 -- 172.21.15.92:0/1372573463 shutdown_connections 2023-11-28T11:26:04.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.737+0000 7f0c305ff700 1 --2- 172.21.15.92:0/1372573463 >> v2:172.21.15.169:3300/0 conn(0x7f0c2806c3f0 0x7f0c2815cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:26:04.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.737+0000 7f0c305ff700 1 --2- 172.21.15.92:0/1372573463 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c280b8510 0x7f0c2806b970 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:26:04.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.737+0000 7f0c305ff700 1 --2- 172.21.15.92:0/1372573463 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c280cd460 0x7f0c2806beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:26:04.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.737+0000 7f0c305ff700 1 -- 172.21.15.92:0/1372573463 >> 172.21.15.92:0/1372573463 conn(0x7f0c280719a0 msgr2=0x7f0c28071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:26:04.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.737+0000 7f0c305ff700 1 -- 172.21.15.92:0/1372573463 shutdown_connections 2023-11-28T11:26:04.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.737+0000 7f0c305ff700 1 -- 172.21.15.92:0/1372573463 wait complete. 2023-11-28T11:26:04.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.739+0000 7f0c305ff700 1 Processor -- start 2023-11-28T11:26:04.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.739+0000 7f0c305ff700 1 -- start start 2023-11-28T11:26:04.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.740+0000 7f0c305ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c2806c3f0 0x7f0c281f2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:26:04.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.740+0000 7f0c305ff700 1 --2- >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c280b8510 0x7f0c280cc650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:26:04.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.740+0000 7f0c305ff700 1 --2- >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c280cd460 0x7f0c280b6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:26:04.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.740+0000 7f0c305ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c281f5450 con 0x7f0c2806c3f0 2023-11-28T11:26:04.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.740+0000 7f0c305ff700 1 -- --> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c281e2210 con 0x7f0c280b8510 2023-11-28T11:26:04.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.740+0000 7f0c305ff700 1 -- --> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c281f2550 con 0x7f0c280cd460 2023-11-28T11:26:04.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.740+0000 7f0c2e39b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c2806c3f0 0x7f0c281f2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:26:04.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.740+0000 7f0c2e39b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c2806c3f0 0x7f0c281f2010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40596/0 (socket says 172.21.15.92:40596) 2023-11-28T11:26:04.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.740+0000 7f0c2e39b700 1 -- 172.21.15.92:0/1983412315 learned_addr learned my addr 172.21.15.92:0/1983412315 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-11-28T11:26:04.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.741+0000 7f0c2e39b700 1 -- 172.21.15.92:0/1983412315 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c280cd460 msgr2=0x7f0c280b6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:26:04.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.741+0000 7f0c2e39b700 1 --2- 172.21.15.92:0/1983412315 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c280cd460 0x7f0c280b6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:26:04.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.741+0000 7f0c2e39b700 1 -- 172.21.15.92:0/1983412315 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c280b8510 msgr2=0x7f0c280cc650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:26:04.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.741+0000 7f0c2e39b700 1 --2- 172.21.15.92:0/1983412315 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c280b8510 0x7f0c280cc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:26:04.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.741+0000 7f0c2e39b700 1 -- 172.21.15.92:0/1983412315 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c24009550 con 0x7f0c2806c3f0 2023-11-28T11:26:04.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.741+0000 7f0c2e39b700 1 --2- 172.21.15.92:0/1983412315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c2806c3f0 0x7f0c281f2010 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f0c24000c00 tx=0x7f0c24004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:26:04.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.741+0000 7f0c1effd700 1 -- 172.21.15.92:0/1983412315 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c24004380 con 0x7f0c2806c3f0 2023-11-28T11:26:04.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.741+0000 7f0c1effd700 1 -- 172.21.15.92:0/1983412315 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f0c2401f070 con 0x7f0c2806c3f0 2023-11-28T11:26:04.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.741+0000 7f0c305ff700 1 -- 172.21.15.92:0/1983412315 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c280cfff0 con 0x7f0c2806c3f0 2023-11-28T11:26:04.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.741+0000 7f0c1effd700 1 -- 172.21.15.92:0/1983412315 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c240036a0 con 0x7f0c2806c3f0 2023-11-28T11:26:04.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.741+0000 7f0c305ff700 1 -- 172.21.15.92:0/1983412315 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c280cfd10 con 0x7f0c2806c3f0 2023-11-28T11:26:04.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.743+0000 7f0c305ff700 1 -- 172.21.15.92:0/1983412315 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c28067880 con 0x7f0c2806c3f0 2023-11-28T11:26:04.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.743+0000 7f0c1effd700 1 -- 172.21.15.92:0/1983412315 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0c2402a020 con 0x7f0c2806c3f0 2023-11-28T11:26:04.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.744+0000 7f0c1effd700 1 --2- 172.21.15.92:0/1983412315 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f0c0c0682f0 0x7f0c0c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:26:04.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.744+0000 7f0c1effd700 1 -- 172.21.15.92:0/1983412315 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f0c24086ff0 con 0x7f0c2806c3f0 2023-11-28T11:26:04.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.744+0000 7f0c2db9a700 1 --2- 172.21.15.92:0/1983412315 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f0c0c0682f0 0x7f0c0c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:26:04.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.745+0000 7f0c2db9a700 1 --2- 172.21.15.92:0/1983412315 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f0c0c0682f0 0x7f0c0c06a7a0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0c280d10b0 tx=0x7f0c18008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:26:04.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.749+0000 7f0c1effd700 1 -- 172.21.15.92:0/1983412315 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0c24053a10 con 0x7f0c2806c3f0 2023-11-28T11:26:04.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.980+0000 7f0c305ff700 1 -- 172.21.15.92:0/1983412315 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0c280d2940 con 0x7f0c2806c3f0 2023-11-28T11:26:04.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.981+0000 7f0c1effd700 1 -- 172.21.15.92:0/1983412315 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f0c24015090 con 0x7f0c2806c3f0 2023-11-28T11:26:04.982 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-28T11:26:04.983 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":35,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5525,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6835/2067956596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6833","nonce":2067956596},{"type":"v1","addr":"172.21.15.169:6835","nonce":2067956596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5971,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.169:6834/2367049160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.169:6832","nonce":2367049160},{"type":"v1","addr":"172.21.15.169:6834","nonce":2367049160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5977,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1424858585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":1424858585},{"type":"v1","addr":"172.21.15.92:6837","nonce":1424858585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6624,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/632425094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":632425094},{"type":"v1","addr":"172.21.15.92:6836","nonce":632425094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[]} 2023-11-28T11:26:04.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.983+0000 7f0c305ff700 1 -- 172.21.15.92:0/1983412315 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f0c0c0682f0 msgr2=0x7f0c0c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:26:04.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.983+0000 7f0c305ff700 1 --2- 172.21.15.92:0/1983412315 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f0c0c0682f0 0x7f0c0c06a7a0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0c280d10b0 tx=0x7f0c18008040 comp rx=0 tx=0).stop 2023-11-28T11:26:04.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.983+0000 7f0c305ff700 1 -- 172.21.15.92:0/1983412315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c2806c3f0 msgr2=0x7f0c281f2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:26:04.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.983+0000 7f0c305ff700 1 --2- 172.21.15.92:0/1983412315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c2806c3f0 0x7f0c281f2010 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f0c24000c00 tx=0x7f0c24004750 comp rx=0 tx=0).stop 2023-11-28T11:26:04.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.983+0000 7f0c305ff700 1 -- 172.21.15.92:0/1983412315 shutdown_connections 2023-11-28T11:26:04.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.983+0000 7f0c305ff700 1 --2- 172.21.15.92:0/1983412315 >> [v2:172.21.15.169:3301/0,v1:172.21.15.169:6790/0] conn(0x7f0c280cd460 0x7f0c280b6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:26:04.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.983+0000 7f0c305ff700 1 --2- 172.21.15.92:0/1983412315 >> [v2:172.21.15.92:6832/115482,v1:172.21.15.92:6833/115482] conn(0x7f0c0c0682f0 0x7f0c0c06a7a0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:26:04.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.983+0000 7f0c305ff700 1 --2- 172.21.15.92:0/1983412315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c2806c3f0 0x7f0c281f2010 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:26:04.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.983+0000 7f0c305ff700 1 --2- 172.21.15.92:0/1983412315 >> [v2:172.21.15.169:3300/0,v1:172.21.15.169:6789/0] conn(0x7f0c280b8510 0x7f0c280cc650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:26:04.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.983+0000 7f0c305ff700 1 -- 172.21.15.92:0/1983412315 >> 172.21.15.92:0/1983412315 conn(0x7f0c280719a0 msgr2=0x7f0c280b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:26:04.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.983+0000 7f0c305ff700 1 -- 172.21.15.92:0/1983412315 shutdown_connections 2023-11-28T11:26:04.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-28T11:26:04.983+0000 7f0c305ff700 1 -- 172.21.15.92:0/1983412315 wait complete. 2023-11-28T11:26:04.989 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 35 2023-11-28T11:26:05.002 DEBUG:tasks.cephfs.filesystem:already dead... 2023-11-28T11:26:05.003 INFO:teuthology.misc:Shutting down mds daemons... 2023-11-28T11:26:05.003 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T11:26:05.003 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:26:05.017 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T11:26:05.018 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T11:26:05.018 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:26:05.072 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T11:26:05.073 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T11:26:05.073 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:26:05.089 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T11:26:05.089 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T11:26:05.089 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:26:05.146 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T11:26:05.146 INFO:teuthology.misc:Shutting down osd daemons... 2023-11-28T11:26:05.147 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-11-28T11:26:05.147 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:26:05.183 INFO:tasks.ceph.osd.0:Stopped 2023-11-28T11:26:05.184 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-11-28T11:26:05.184 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:26:05.260 INFO:tasks.ceph.osd.1:Stopped 2023-11-28T11:26:05.260 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-11-28T11:26:05.261 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:26:05.342 INFO:tasks.ceph.osd.2:Stopped 2023-11-28T11:26:05.343 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-11-28T11:26:05.343 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:26:05.387 INFO:tasks.ceph.osd.3:Stopped 2023-11-28T11:26:05.387 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-11-28T11:26:05.387 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:26:05.432 INFO:tasks.ceph.osd.4:Stopped 2023-11-28T11:26:05.432 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-11-28T11:26:05.432 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:26:05.489 INFO:tasks.ceph.osd.5:Stopped 2023-11-28T11:26:05.490 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-11-28T11:26:05.490 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:26:05.538 INFO:tasks.ceph.osd.6:Stopped 2023-11-28T11:26:05.538 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-11-28T11:26:05.539 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:26:05.589 INFO:tasks.ceph.osd.7:Stopped 2023-11-28T11:26:05.589 INFO:teuthology.misc:Shutting down mgr daemons... 2023-11-28T11:26:05.589 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-11-28T11:26:05.589 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:26:05.653 INFO:tasks.ceph.mgr.y:Stopped 2023-11-28T11:26:05.654 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-11-28T11:26:05.654 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:26:05.713 INFO:tasks.ceph.mgr.x:Stopped 2023-11-28T11:26:05.713 INFO:teuthology.misc:Shutting down mon daemons... 2023-11-28T11:26:05.714 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-11-28T11:26:05.714 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:26:05.734 INFO:tasks.ceph.mon.a:Stopped 2023-11-28T11:26:05.734 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-11-28T11:26:05.734 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:26:05.753 INFO:tasks.ceph.mon.b:Stopped 2023-11-28T11:26:05.753 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-11-28T11:26:05.754 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:26:05.812 INFO:tasks.ceph.mon.c:Stopped 2023-11-28T11:26:05.813 INFO:tasks.ceph:Checking cluster log for badness... 2023-11-28T11:26:05.813 DEBUG:teuthology.orchestra.run.smithi092:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2023-11-28T11:26:05.899 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi092.front.sepia.ceph.com 2023-11-28T11:26:05.900 DEBUG:teuthology.orchestra.run.smithi092:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-11-28T11:26:06.129 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi092.front.sepia.ceph.com 2023-11-28T11:26:06.129 DEBUG:teuthology.orchestra.run.smithi092:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-11-28T11:26:06.250 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi092.front.sepia.ceph.com 2023-11-28T11:26:06.251 DEBUG:teuthology.orchestra.run.smithi092:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-11-28T11:26:06.366 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi092.front.sepia.ceph.com 2023-11-28T11:26:06.366 DEBUG:teuthology.orchestra.run.smithi092:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-11-28T11:26:06.510 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi169.front.sepia.ceph.com 2023-11-28T11:26:06.510 DEBUG:teuthology.orchestra.run.smithi169:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-11-28T11:26:06.688 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi169.front.sepia.ceph.com 2023-11-28T11:26:06.688 DEBUG:teuthology.orchestra.run.smithi169:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-11-28T11:26:06.806 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi169.front.sepia.ceph.com 2023-11-28T11:26:06.806 DEBUG:teuthology.orchestra.run.smithi169:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-11-28T11:26:06.930 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi169.front.sepia.ceph.com 2023-11-28T11:26:06.931 DEBUG:teuthology.orchestra.run.smithi169:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-11-28T11:26:07.070 INFO:tasks.ceph:Archiving mon data... 2023-11-28T11:26:07.072 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469955/data/mon.a.tgz 2023-11-28T11:26:07.072 DEBUG:teuthology.orchestra.run.smithi092:> mktemp 2023-11-28T11:26:07.120 INFO:teuthology.orchestra.run.smithi092.stdout:/tmp/tmp.3Z2HfeC4mJ 2023-11-28T11:26:07.121 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.3Z2HfeC4mJ 2023-11-28T11:26:07.289 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0666 /tmp/tmp.3Z2HfeC4mJ 2023-11-28T11:26:07.407 DEBUG:teuthology.orchestra.remote:smithi092:/tmp/tmp.3Z2HfeC4mJ is 411KB 2023-11-28T11:26:07.509 DEBUG:teuthology.orchestra.run.smithi092:> rm -fr /tmp/tmp.3Z2HfeC4mJ 2023-11-28T11:26:07.555 DEBUG:teuthology.misc:Transferring archived files from smithi169:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469955/data/mon.b.tgz 2023-11-28T11:26:07.556 DEBUG:teuthology.orchestra.run.smithi169:> mktemp 2023-11-28T11:26:07.605 INFO:teuthology.orchestra.run.smithi169.stdout:/tmp/tmp.l7CuqH5c2w 2023-11-28T11:26:07.606 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.l7CuqH5c2w 2023-11-28T11:26:07.805 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 0666 /tmp/tmp.l7CuqH5c2w 2023-11-28T11:26:07.927 DEBUG:teuthology.orchestra.remote:smithi169:/tmp/tmp.l7CuqH5c2w is 537KB 2023-11-28T11:26:08.033 DEBUG:teuthology.orchestra.run.smithi169:> rm -fr /tmp/tmp.l7CuqH5c2w 2023-11-28T11:26:08.079 DEBUG:teuthology.misc:Transferring archived files from smithi169:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469955/data/mon.c.tgz 2023-11-28T11:26:08.080 DEBUG:teuthology.orchestra.run.smithi169:> mktemp 2023-11-28T11:26:08.166 INFO:teuthology.orchestra.run.smithi169.stdout:/tmp/tmp.uPP9tg70xw 2023-11-28T11:26:08.166 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.uPP9tg70xw 2023-11-28T11:26:08.392 DEBUG:teuthology.orchestra.run.smithi169:> sudo chmod 0666 /tmp/tmp.uPP9tg70xw 2023-11-28T11:26:08.513 DEBUG:teuthology.orchestra.remote:smithi169:/tmp/tmp.uPP9tg70xw is 537KB 2023-11-28T11:26:08.620 DEBUG:teuthology.orchestra.run.smithi169:> rm -fr /tmp/tmp.uPP9tg70xw 2023-11-28T11:26:08.668 INFO:tasks.ceph:Cleaning ceph cluster... 2023-11-28T11:26:08.668 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-28T11:26:08.673 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-28T11:26:08.777 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-11-28T11:26:08.778 DEBUG:teuthology.orchestra.run.smithi092:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-28T11:26:08.783 DEBUG:teuthology.orchestra.run.smithi169:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-28T11:26:08.862 INFO:teuthology.orchestra.run.smithi169.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-28T11:26:08.874 INFO:teuthology.orchestra.run.smithi092.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-28T11:26:08.892 INFO:tasks.ceph:Archiving crash dumps... 2023-11-28T11:26:08.894 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469955/remote/smithi092/crash 2023-11-28T11:26:08.895 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-28T11:26:08.974 DEBUG:teuthology.misc:Transferring archived files from smithi169:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469955/remote/smithi169/crash 2023-11-28T11:26:08.975 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-28T11:26:09.045 INFO:tasks.ceph:Shutting down logrotate 2023-11-28T11:26:09.045 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:26:09.050 DEBUG:teuthology.orchestra.run.smithi169:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:26:09.151 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-28T11:26:09.221 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-28T11:26:09.292 INFO:tasks.ceph:Compressing logs... 2023-11-28T11:26:09.292 DEBUG:teuthology.orchestra.run.smithi092:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-28T11:26:09.297 DEBUG:teuthology.orchestra.run.smithi169:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-28T11:26:14.100 INFO:tasks.ceph:Archiving logs... 2023-11-28T11:26:14.100 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/log/ceph to /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469955/remote/smithi092/log 2023-11-28T11:26:14.102 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-28T11:26:15.066 DEBUG:teuthology.misc:Transferring archived files from smithi169:/var/log/ceph to /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469955/remote/smithi169/log 2023-11-28T11:26:15.067 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-28T11:26:15.686 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-11-28T11:26:15.696 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-11-28T11:26:15.696 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-28T11:26:15.702 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-28T11:26:15.831 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-11-28T11:26:15.840 INFO:teuthology.task.clock:Checking final clock skew... 2023-11-28T11:26:15.841 DEBUG:teuthology.orchestra.run.smithi092:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-28T11:26:15.846 DEBUG:teuthology.orchestra.run.smithi169:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-28T11:26:15.887 INFO:teuthology.orchestra.run.smithi092.stderr:bash: ntpq: command not found 2023-11-28T11:26:15.890 INFO:teuthology.orchestra.run.smithi169.stderr:bash: ntpq: command not found 2023-11-28T11:26:15.895 INFO:teuthology.orchestra.run.smithi092.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-28T11:26:15.895 INFO:teuthology.orchestra.run.smithi092.stdout:=============================================================================== 2023-11-28T11:26:15.895 INFO:teuthology.orchestra.run.smithi092.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 47 +721us[ +745us] +/- 74ms 2023-11-28T11:26:15.895 INFO:teuthology.orchestra.run.smithi092.stdout:^* hv02.front.sepia.ceph.com 3 6 377 46 -667us[ -643us] +/- 66ms 2023-11-28T11:26:15.895 INFO:teuthology.orchestra.run.smithi092.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 46 +111us[ +135us] +/- 84ms 2023-11-28T11:26:15.895 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:26:15.898 INFO:teuthology.orchestra.run.smithi169.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-28T11:26:15.898 INFO:teuthology.orchestra.run.smithi169.stdout:=============================================================================== 2023-11-28T11:26:15.898 INFO:teuthology.orchestra.run.smithi169.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 45 +787us[ +787us] +/- 74ms 2023-11-28T11:26:15.898 INFO:teuthology.orchestra.run.smithi169.stdout:^* hv02.front.sepia.ceph.com 3 6 377 47 -591us[ -585us] +/- 66ms 2023-11-28T11:26:15.898 INFO:teuthology.orchestra.run.smithi169.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 46 +188us[ +188us] +/- 84ms 2023-11-28T11:26:15.898 INFO:teuthology.orchestra.run.smithi169.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:26:15.899 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-11-28T11:26:15.908 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-11-28T11:26:15.908 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-11-28T11:26:15.918 DEBUG:teuthology.orchestra.run.smithi092:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-28T11:26:16.109 DEBUG:teuthology.orchestra.run.smithi169:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-28T11:26:16.272 DEBUG:teuthology.orchestra.run.smithi092:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-28T11:26:16.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:26:16.349 DEBUG:teuthology.orchestra.run.smithi169:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-28T11:26:16.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:26:16.427 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-11-28T11:26:16.437 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-11-28T11:26:16.446 INFO:teuthology.task.internal:Duration was 1510.093925 seconds 2023-11-28T11:26:16.448 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-11-28T11:26:16.460 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-11-28T11:26:16.460 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-28T11:26:16.465 DEBUG:teuthology.orchestra.run.smithi169:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-28T11:26:16.579 INFO:teuthology.orchestra.run.smithi092.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-28T11:26:16.584 INFO:teuthology.orchestra.run.smithi169.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-28T11:26:16.960 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-11-28T11:26:16.960 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi092.front.sepia.ceph.com 2023-11-28T11:26:16.961 DEBUG:teuthology.orchestra.run.smithi092:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-28T11:26:17.023 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi169.front.sepia.ceph.com 2023-11-28T11:26:17.023 DEBUG:teuthology.orchestra.run.smithi169:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-28T11:26:17.089 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-11-28T11:26:17.090 DEBUG:teuthology.orchestra.run.smithi092:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-28T11:26:17.095 DEBUG:teuthology.orchestra.run.smithi169:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-28T11:26:17.180 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-11-28T11:26:17.180 DEBUG:teuthology.orchestra.run.smithi092:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-28T11:26:17.185 DEBUG:teuthology.orchestra.run.smithi169:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-28T11:26:17.306 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-11-28T11:26:17.315 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-11-28T11:26:17.316 DEBUG:teuthology.orchestra.run.smithi092:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-28T11:26:17.387 DEBUG:teuthology.orchestra.run.smithi169:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-28T11:26:17.460 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-11-28T11:26:17.471 DEBUG:teuthology.orchestra.run.smithi092:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-28T11:26:17.476 DEBUG:teuthology.orchestra.run.smithi169:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-28T11:26:17.540 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = core 2023-11-28T11:26:17.546 INFO:teuthology.orchestra.run.smithi169.stdout:kernel.core_pattern = core 2023-11-28T11:26:17.580 DEBUG:teuthology.orchestra.run.smithi092:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-28T11:26:17.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:26:17.626 DEBUG:teuthology.orchestra.run.smithi169:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-28T11:26:17.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:26:17.671 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-11-28T11:26:17.684 INFO:teuthology.task.internal:Transferring archived files... 2023-11-28T11:26:17.685 DEBUG:teuthology.misc:Transferring archived files from smithi092:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469955/remote/smithi092 2023-11-28T11:26:17.685 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-28T11:26:17.770 DEBUG:teuthology.misc:Transferring archived files from smithi169:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469955/remote/smithi169 2023-11-28T11:26:17.771 DEBUG:teuthology.orchestra.run.smithi169:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-28T11:26:17.864 INFO:teuthology.task.internal:Removing archive directory... 2023-11-28T11:26:17.864 DEBUG:teuthology.orchestra.run.smithi092:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-28T11:26:17.871 DEBUG:teuthology.orchestra.run.smithi169:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-28T11:26:17.918 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-11-28T11:26:17.928 INFO:teuthology.task.internal:Not uploading archives. 2023-11-28T11:26:17.928 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-11-28T11:26:17.938 INFO:teuthology.task.internal:Tidying up after the test... 2023-11-28T11:26:17.938 DEBUG:teuthology.orchestra.run.smithi092:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-28T11:26:17.955 DEBUG:teuthology.orchestra.run.smithi169:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-28T11:26:17.997 INFO:teuthology.orchestra.run.smithi092.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 28 11:26 /home/ubuntu/cephtest 2023-11-28T11:26:18.002 INFO:teuthology.orchestra.run.smithi169.stdout: 524293 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 28 11:26 /home/ubuntu/cephtest 2023-11-28T11:26:18.005 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-11-28T11:26:18.019 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-11-28T11:26:18.046 INFO:teuthology.nuke:Checking targets against current locks 2023-11-28T11:26:18.074 DEBUG:teuthology.nuke:shortname: smithi092 2023-11-28T11:26:18.074 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-28T11:26:18.099 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi092.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469955', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-28 10:49:56.154011', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLBWKDssAcujaTJGYoh0m+6KRf79YlBnyP6+4+KzFgJNnb+zlPv8t0bboGt8tbKRVck/k776TBljTkSk6kIOe1M='} 2023-11-28T11:26:18.105 DEBUG:teuthology.nuke:shortname: smithi169 2023-11-28T11:26:18.106 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-28T11:26:18.127 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi169.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469955', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-28 10:49:56.152407', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBspMRSneSLchVXtu1lFxgIfmFRbU9QHxtOh73ymQiDEyhUjA07U4CTxv98VSN9qWSUXOJ7F0WXBIc5AsA4w4ss='} 2023-11-28T11:26:18.163 INFO:teuthology.orchestra.console.smithi092:Power off 2023-11-28T11:26:18.164 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-28T11:26:18.192 DEBUG:teuthology.orchestra.console.smithi092:power off output: Chassis Power Control: Down/Off 2023-11-28T11:26:18.192 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T11:26:18.218 INFO:teuthology.orchestra.console.smithi169:Power off 2023-11-28T11:26:18.220 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-28T11:26:18.233 DEBUG:teuthology.orchestra.console.smithi092:check power output: Chassis Power is on 2023-11-28T11:26:18.244 DEBUG:teuthology.orchestra.console.smithi169:power off output: Chassis Power Control: Down/Off 2023-11-28T11:26:18.244 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T11:26:18.271 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is on 2023-11-28T11:26:22.235 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T11:26:22.272 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T11:26:22.349 DEBUG:teuthology.orchestra.console.smithi092:check power output: Chassis Power is on 2023-11-28T11:26:22.383 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is on 2023-11-28T11:26:26.354 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T11:26:26.384 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T11:26:26.469 DEBUG:teuthology.orchestra.console.smithi092:check power output: Chassis Power is off 2023-11-28T11:26:26.496 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is off 2023-11-28T11:26:26.571 INFO:teuthology.orchestra.console.smithi092:Power off completed 2023-11-28T11:26:26.597 INFO:teuthology.orchestra.console.smithi169:Power off completed 2023-11-28T11:26:26.704 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 client.1-kernel-sha1: 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub} duration: 1510.0939247608185 failure_reason: 'Test failure: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub)' flavor: default owner: scheduled_neesingh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=861142adecb64dd698fcaea137ec43ab status: fail success: false 2023-11-28T11:26:26.704 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-28T11:26:26.784 INFO:teuthology.run:FAIL